Who stands to win from double-blind peer review?

The launch of Advances in Regenerative Biology deserves to be celebrated not only for being a promising new journal in an exciting field of the life sciences but also for adopting a notable approach of manuscript evaluation: double-blind peer review. This approach is primarily notable for its exceptional rarity in natural science journals, despite overwhelming support for it from the research community. Here, I summarise what in my view are the key advantages of double-blind peer review over the alternatives and discuss how research and researchers can benefit from its wider implementation.

Three main systems of pre-publication peer review have been proposed, which differ with respect to whether the reviewers know the identity of the authors and vice versa: open, single-blind and double-blind. In open peer review both sides know who they are, in the single-blind system only the reviewers know the identities of the authors, whereas in the double-blind approach the identities of both sides are not revealed during the review process. The masking of author and reviewer identities is intended to minimise bias. For example, reviewers may be biased in evaluating the authors’ work based on who they are, where and with whom they work; similarly, authors disgruntled by critical reviews may be biased in their future interactions with the reviewers. There are some who would dispute that ‘serious’ scientists can have any bias at all, be it in their role as reviewers or as authors, but this can hardly be taken as anything more than wishful thinking. Bias is at the core of human nature. Indeed much of basic research methodology is designed to neutralise our inevitable biases in interpreting our data. Moreover, pervasive – even if subtle and unconscious – bias among researchers with respect to gender, race, country of origin or affiliation has been consistently documented (1–6). As long as research is carried out by humans, bias will be here to stay, and instead of denying it, we should try to reduce it. In doing so, there is no need to reinvent the wheel, as a gold standard approach for minimising bias in situations involving human interactions is well established: double blinding. For example, in a clinical trial setting, neither the doctors nor the patients know who belongs to the placebo control group and who gets the drug, impeding biased data reporting, collection or analysis. One might thus reasonably expect that a similar double-blind approach would be standard fare for the peer-review process. Indeed, in some branches of academia, such as the social sciences, double-blind peer review has become an accepted system – perhaps because social scientists are better aware of the ubiquity and pitfalls of human bias. In the natural sciences, however, by far the dominant form of research evaluation is single-blind review, whereas the double-blind practice is exceedingly uncommon. So why is there such a profound rift between the way in which the ‘hard sciences’ are conducted and the way in which they are evaluated?

It is certainly not due to insufficient popularity of double-blind review. Researcher surveys regularly reveal double-blind review as the method of research assessment that is most highly favoured by scientists across disciplines (7–11). Yet, journal editors, the group most supportive of the reigning single-blind system, have largely resisted the introduction of double-blind review (12, 13). Different justifications have been put forward for this policy – for example, the burden of concealing the authors’ identities or the fact that knowing who the authors are helps to check the novelty of their work and to identify conflicts of interest (12–15). These arguments, however, do not stand up to scrutiny. Electronic submission systems can be designed to automatically exclude the author information in the manuscript received by reviewers; the novelty of the work should be judged against all previous publications, not only those by the authors themselves; conflicts of interest arise from the subject matter of the manuscript, not from its authors – and in cases when the conflicts are of a personal nature, double blinding can only reduce their occurrence. The most frequent objection against the double-blind model is that masking the authors’ identities will not always be successful (16, 17). This is undoubtedly true – for instance, the reviewer may have seen the data in the manuscript at a meeting. But this cannot argue against double-blind peer review any more than the inability to fully blind some treatment and control groups (e.g. due to characteristic side effects of the drug) can argue against the use of double-blind clinical trials. Importantly, research has shown that in most cases reviewers cannot divine the authors’ identities (16–18). Whenever they manage to do it, double-blind review is reduced to the single-blind form, but this still leaves a majority of cases in which the authors will have a better chance of impartial evaluation. The logic and data behind this thinking should seem clear, but the vast majority of life science and biomedical journals remain unmoved by such arguments and persist in their refusal to implement double-blind review. I therefore wonder if there may be some other, less palatable and thus less widely advertised reasons for this attitude. The ‘never-change-a-winning-team’ type of mindset may be one such reason. But I suspect that there is another, more important one. Some hints about it are provided in a large survey by the Publishing Research Consortium, which showed that apart from editors, the other group with lowest support for double-blind peer review are researchers aged over 56 (11). That latter group is certainly enriched in established scientific leaders, who may be unwilling to forfeit the advantages conveyed by their prior reputations in a single-blind review system and would exert pressure on editors to maintain it (19). Editors, for their part, may not be averse to boosting their journals’ readership and impact factors by featuring more frequently work by scientific celebrities, who are more likely to make it through the review process if their work is not anonymised. Certainly, this is not intended to imply any conspiracy – it is simply that the interests of famous scientists and journal editors may align on this issue. But in a world where academic survival often depends on publication within the limited space of leading journals, such subtle biases could turn into career breakers for young and talented researchers, who only just fail to make the grade in the single-blind system.

Some of those who recognise the shortcomings of the single-blind method argue that open review is the best remedy (10). Revealing the reviewers’ names, it is claimed, is likely to make them more responsible in their assessment and averse to unjustified criticism. This is probably correct, but it is just as likely to make them averse to justified criticism, which lies at the heart of real, constructive peer review. As intuitively appealing as the concept of openness may appear, it is telling that open peer review is by far the least favoured system among scientists, and a substantial fraction of researchers state that they would decline to review a paper if their identities were revealed (8, 10) (11). Therefore, while we should certainly strive to instil in reviewers a culture of accountability and respect for the work of all authors, I believe that double-blind peer review is the best available method for levelling the playing field, minimising bias, and giving everyone the best chance to have their work judged based on its true scientific merit. It is to be expected that such changes will not be to everyone's liking. Indeed, the staunchest opposition against double-blind review is likely to come from the most preeminent and well-connected members of the scientific establishment. In the face of such hurdles, the transition to a widely accepted double-blind peer review system will neither be quick nor easy. But given the extensive popular support for this approach, the effort is worth it and the chances of success are real.

In this context, it is very encouraging to see Advances in Regenerative Biology embracing the double-blind model as the only method of peer review. Conservation Biology has recently announced a similar move (20), but to my knowledge, Advances in Regenerative Biology is the first life science journal to implement this system from its inception. Obligatory double-blind peer review, in my view, is the way to go. Several journals, including Nature Geoscience, Nature Climate Change, and Nature Nanotechnology, have recently adopted double-blind review as an option available in parallel to their standard single-blind review (21–23). Such a half-hearted approach, however, may bring more confusion than clarity about the benefits of author anonymity. A likely scenario is that senior authors will opt against double-blind review and those who opt for it will often be assumed, rightly or not, to be junior or from outside the privileged circle of the rich, scientifically prominent nations and institutes. In the end, bias may be exacerbated rather than mitigated. Unsurprisingly, few authors opt for double-blind review at these journals (23). I would opt against it too. It is my conviction that for double-blind review to work, it must be compulsory.

If double-blind peer review became a more widely adopted model, what changes could one expect? I do not think any seismic shifts are likely to happen with respect to who publishes what and where. Renowned scientists have not won their reputations on the lottery, they have earned them through talent and hard work; junior researchers, for their part, do have to struggle to truly achieve the level of their more senior peers. So good scientists will continue to publish good papers in good journals under a double-blind peer review system. But every now and again a lesser known research team could make it on the strength of their work against a more famous one, when they would have failed in the single-blind setting – and this would be reason enough for me to endorse the shift to double-blind review. Inevitably, authors will try to exploit loopholes in the new system, for example, by trying to unblind potential reviewers. Some of the more obvious tactics for that (24), such as explicit self-reference (e.g. ‘we have recently shown that’), can be weeded out easily by instructing authors to refrain from them (‘it has been recently shown that’) and enforcing compliance through automated text analysis tools. Manuscript sections that can reveal author-related information without contributing to the scientific content, such as Acknowledgements, Funding, or Author Contributions, should also be excluded from the manuscript version sent to reviewers. Other unblinding approaches, including more frequently presenting work in progress at meetings, may in fact be a welcome development in the current climate, which has rendered scientists increasingly reluctant to reveal results before manuscript submission. At the same time, one should be careful not to impose unreasonably harsh conditions to maintain anonymity at any cost, such as limiting essential, non-explicit self-citation, or the description of equipment and tools that are unique to a research team. Overall, though, double-blind review would require little extra effort or expense, will not have any appreciable negative effects, but may well contribute substantially to enhancing the objectivity of the refereeing process. Previous reports have suggested that this enhancement may not be as significant as we would wish (17, 25) (26). However, real world routine application of the approach may be better suited to revealing its true benefits than limited controlled studies, in which reviewers know that their objectivity is being put to test. Perhaps, the most important change brought about by double-blind peer review could be a shift in research culture, leading us to genuinely care about the quality of the data and not about who produced them, turning research into less of an old boys’ club and more of an equal opportunity undertaking that rewards good ideas and good work.

There is one more pertinent point, without which the discussion would be incomplete. Editors reject many, sometimes most, of the papers submitted to a journal before a reviewer can see them. While motives will differ between editors and reviewers, there is no reason to assume that the former will be less susceptible to bias than the latter. Therefore, if the concept of objectivity were to be consistently carried through, a blinding of editors to the identity of the authors would also seem logical. That would presumably require some additional tweaks to current manuscript tracking systems and will prevent personal communication between authors and editors – although I suspect that from an editor's point of view this will not always be a bad thing. That said, the system of double-blind review remains largely unexplored territory in the life sciences, so treading cautiously, one step at a time, seems like the most rational strategy. In any case, the editorial team of Advances in Regenerative Biology are trailblazers on the double-blind peer review front and should be warmly congratulated for that! It is my hope that the bold example of this journal will be followed by others, so that true double-blind peer review can finally gain a solid foothold in the natural sciences. It is worth re-emphasising that this system of review is common in the social sciences and, tellingly, social scientists are most satisfied with it and most supportive of its continued use (11). There is no reason to think that the same could not happen in the natural sciences. If the endeavour of establishing double-blind peer review as a standard for academic publication succeeds, I believe that science and scientists will be the winners.

Boyan K. Garvalov Institute of Neuropathology, Justus Liebig University 35392 Giessen, Germany [email protected]