Current incentives make quantity the enemy of quality, says Professor Philipp Koellinger, chief scientific officer and co-founder of open access science platform DeSci Labs.

Science is at the behest of a numbers game. The current publish or perish culture means researchers are forced to publishing frequently. These incentives have led to problematic research practices, a proliferation of low quality or marginal research and a widespread replication crisis that undermines scientific progress.

Incentives must change to accelerate scientific progress, and rethink how research is published, validated & evaluated.

The right incentives

“Show me the incentives and I will show you the outcome.” This quote from Charlie Munger very much applies to the scientific enterprise.

The current publish or perish culture in academia is problematic because quantity is often the enemy of quality. Constantly getting published in prestigious journals improves the chances of taking that next career step or winning a research grant. This encourages some to engage in questionable research practices such as p-hacking or salami slicing of results, overclaiming results, and even downright illegitimate behaviours such as plagiarism or forging data, images, and results.

 These incentives are one of the root causes of the replication crisis that affects many fields of science, eroding trust in science and slowing down progress.

High impact journals are currently seen as a proxy for quality, but their replication rates are often no better than those that aren’t ranked as highly. The current publishing system also doesn’t incentivise researchers to actively engage in independent replication studies.

Instead of rewarding publications per-se, both rigour and novelty must be explicitly incentivised. And I really mean both at the same time – one without the other is highly problematic.

With rigour and no novelty, we would have scientific hygiene, but we would not be able to generate new knowledge. Novelty without rigour, on the other hand, brings about click bait that does not stand up to scientific scrutiny and results in a replication crisis.

To address this, we need better and more objective ways to measure novelty and rigour. For example, novelty can be objectively measured by looking at how the unique the combination of topics and concepts that a manuscript talks about are. Having an objective novelty score can give authors more bargaining power when submitting their work for consideration by their chosen journal. Scoring novelty can also take away some of the subjective nature of the peer review process where work is often judged by editors and a handful of reviewers, who often do not agree in their evaluations.

Objectively measuring rigour means the peer review process must be broken down into clearly defined, objective claims about the research can be verified by independent parties (e.g. data available, code available, results reproducible). Reviewers should also be incentivised to report truthfully, diligently and quickly. We will soon release details about how we are planning to facilitate this through DeSci Publish.

Building the necessary infrastructure

The current publishing infrastructure does not cater for this, as it is built entirely around the distribution of manuscripts and PDF files. Though valuable, a manuscript is often only one out of many valuable outputs of the research process. Without access to data and code, reviewers and readers are often simply forced to believe the author’s claims and interpretations, without being able to verify their validity.

There is a clear need to build an infrastructure that supports data, code, the final manuscript and all other artefacts as equal citizens. This is a necessary step towards solving the replication crisis. In addition, as well as making research more reproducible, robust and trustworthy, it can aid the acceleration of scientific progress, removing the need for people to keep reinventing the wheel.

All of these artefacts should be free from data silos so anyone can participate in the storage of scientific data in networks, allowing individuals to host whatever data they choose on their servers.

No paywalls around science

This can be made possible via an open-source peer-to-peer network, where anybody can participate as a storage provider. All data posted on the network will have a unique identifier which enables the files to be stored by different providers – enabling multiple copies.

This kind of technology makes it impossible to erect paywalls around the content as anyone is free to copy it and host it on their own server. Making data, content, code and manuscripts freely available to enable reproducibility, while also moving towards a more open and objective way to evaluate science that rewards quality instead of quantity would allow us to move away from the current publish or perish model and help accelerate scientific progress.