The Great Conflict – How to Rank Research Ideas?

A job that begins with horror, but in the best of cases ends with feelings of happiness and vitality. That is how professor Anu Koivunen, a Skolar Award preliminary jury member, describes the tough task of ranking, reviewing and choosing research ideas. But how is it even possible to compare proposals coming from different branches of science?

Anu Koivunen

For professor Anu Koivunen, a good research proposal is defined by one crucial factor. It has to make sense.

“For me, sense refers to the relevance of the idea. What is it about and where is it going? Is it understandable?” Koivunen explains.

“In the ideal case, the sense is so clear that it makes you feel a rush of meaning. A bright idea arouses curiosity and communicates with previous research.”

Koivunen is one of the 12 members of the Skolar Award preliminary jury. She works as a media researcher and professor of cinema studies at Stockholm University, but is currently on leave of absence to work as professor of gender studies at the University of Tampere. She is also a well-known media commentator in Finland. She is who reporters call when they need an analysis on the #metoo phenomenon in the film industry or a comment on the state of journalism. She is a regular guest on television and radio shows, and very active on Twitter.

For her, being part of the Skolar Award preliminary jury is what she calls “voluntary academic work”, which is part of the job of a professor.

“I joined the preliminary jury because of my curiosity, a quality of mine that has led me to many good places before”, she says.

Conflicting interests are part of the ranking process

Koivunen is more than familiar with reviewing applications, as she has previously been a review panel member at the Swedish Research Council and the Academy of Finland.

“Ranking good proposals is horrible. But I’m sure that the result of it all will be the same as always when reading research proposals: I’ll end up feeling wonderful, inspired, alive and like a vital part of the scientific community.”

Skolar Award is a competition that strives to receive proposals from all fields of research. Diversity is an admirable and important goal, but it is easier said than done. How is it even possible to compare ideas hailing from different branches of science?

To make the competition as fair as possible, the 12 preliminary jury members come from various scientific fields. However, it is impossible to come up with a completely uniform rating system for different research ideas. Koivunen believes one must accept that this battle is part of the job – and science in general.

“There are conflicting interests in the research world. While scoring the applications, one judge may value innovation, the second may prefer basic research and the third can be attracted by proposals that ask questions no one else has asked. Battling between different interests is an inevitable characteristic of the whole scientific community.”

Skolar Award’s pitch format may be the most restrictive, yet simultaneously the most liberating factor for research ideas. Koivunen thinks that as a pitching competition, Skolar Award favours proposals that are performative and can be communicated clearly in this particular format.

“Pitching measures the brightness of the idea, and thus favours some ideas over others. But that is what all formats do, no format is neutral”, she says.

Research has to increase understanding

The winner of Skolar Award gets a research grant of 100,000 euros. The competition and prize are funded by seven Finnish foundations. In Koivunen’s view, foundations play a key role in funding research that does not fit the currently dominant solution-oriented approach  in research funding.

“Foundations can give money to research projects that don’t concern solving the five most pressing problems in the world. If research is always required to provide solutions, the important part of research that tries to understand and explain different phenomena is excluded. I’m not depreciating research that aims to solve wicked problems, but if we focus solely on benefits and applications, we risk neglecting the value of understanding”, she says.

Koivunen thinks that now is exactly the time for research to increase our understanding of various issues. That will be one of her main guidelines when ranking Skolar Award applications.

“Research can help us better understand our complex world. Having a better understanding means engaging in better, deeper and more nuanced thinking. Research should be driven by the will to wonder.”

The first review of Skolar Award applications is done by a preliminary jury, which is composed of twelve experts from various scientific fields. After the first round of reviews, the eight finalists will be selected by a final jury of six scientific experts.