The need to publish, especially in top journals, has come to dominate academic life. No measure of performance is more important for academics. Promotion and research funding are at stake. Failure to publish enough can bring demotion or even dismissal. Hardly surprising, then, that academics can take desperate measures in their desire to publish. Sotaro Shibayama and Yasunori Baba, from the University of Tokyo, focus on just one example of the lengths to which desperate academics will go.
In ‘Dishonest conformity in peer review’, newly published in Prometheus, Shibayama and Baba examine the alterations referees require to papers submitted to academic journals. How do the authors of these papers react to the referee’s demands? Do authors obey even when they know the referee to be wrong? Are authors really willing to alter the meaning of a paper and distort its results in order to increase its chances of publication? Of course they are, according to Shibayama and Baba. When publication is at stake, academic authors are quite ready to call black white and white black. To be published in top journals, they would sell their mothers.
Shibayama and Baba consider the behaviour of authors willing to do whatever referees want to be dishonest. Some will cavil at this. Even one of their own referees protested that heeding the advice of peer reviewers could not possibly be dishonest. A referee, he insisted, represented the author’s peers, his own opinions merely a condensation of those of the academy as a whole. When the referee spoke, it was less ex cathedra than as the voice of disciplinary consensus.
As general editor of Prometheus, I did not cavil. I think that the reluctance of authors to stand up to referees has serious implications for research. The price of publication may not always – or even often – be the distortion of knowledge, but suspicion rots confidence in academic research. However, gutless authors should not shoulder all the blame for this situation; they are just one part of an academic publishing system that has become less concerned with generating knowledge than with producing profitable performance measures. The academic publishing system itself is dishonest.
My own research examines how journals compete with other journals, especially through the Impact Factor. This crucial measure of journal quality is calculated from the frequency with which a journal’s papers are cited in the two years following their publication. The more cited its papers, the higher the journal’s Impact Factor and the greater the journal’s value. The journal’s other stakeholders (universities, funding bodies, editors, editorial boards, referees – and, of course, publishers) have as much interest in journal value as authors.
Citability is key to value. Journal papers, I argue, have become academic currency, published less to be read than to be cited as many times as possible. Papers with lots of authors, each of whom busily self-cites, are in demand, as are those with prestigious ghost authors, and papers which cite the journal’s own publications. Reaching the journal’s required level of internal citation has become a customary condition of acceptance in the top journals of the social sciences.
Papers that contribute most to the Impact Factor (and are consequently most attractive to journals) are those that can be cited almost anywhere, papers that confirm rather than challenge established wisdom. Reference lists grow ever longer, stacked with evidence of established thinking, the same old papers cited more and more with the passing of the years. The system weeds out papers reporting novel, niche or negative research, screening papers to ensure a neat fit into an existing body of knowledge. No matter how important, how brilliant, such papers will be little cited over the period in which the Impact Factor is calculated.
When a single reservation from a single referee is quite enough to bring about a paper’s automatic rejection, academic authors see peer review as mainly an obstacle to publication. Workshops on how to get papers past referees are ubiquitous. I am expected to join other editors in addressing these depressing gatherings of aspiring authors, who regard satisfying referees as a matter of technique. Many referees have become reluctant to waste their expertise on such game playing and many editors now struggle to find referees. Increasingly, they resort to electronic lists of academics’ research interests maintained by publishers and simply match keywords. Occasionally, publishers take total charge of the refereeing process. Invitations to referee are issued by the publisher’s manuscript tracking system, and the referee’s report is sometimes no more than a tick-box form.
I recently accepted an email invitation from a manuscript tracking system to referee my own paper. My report was wickedly glowing: “the best paper ever written”, “Macdonald deserves an instant Nobel prize.” As no one remarked on this nonsense, I eventually confessed direct to the editor, who declared that I had brought peer review into disrepute and instantly rejected my paper. And peer review really is at risk of being brought into disrepute. Referees are supposed to detect dishonesty – as when they spot plagiarism or fiddled research findings – not encourage it. The very system that is supposed to keep authors honest is encouraging them to be dishonest.
It is difficult for referees and authors to work for the public good within an academic publishing system that offers huge private benefits to so many interest groups. Whatever its remaining academic merits, peer review has become a marketing tool, exploited to promote the publishing product. This is fast becoming a standard product. The dishonest conformity that Shibayama and Baba find is not restricted to authors desperate to publish. The rest of the academic publishing system is party to this dishonesty.
Additional information
Notes on contributors
Stuart Macdonald
Stuart Macdonald was, until recently, Professor of Management at Aalto University in Helsinki. He is now an Honorary Professor at Leicester University. He is also general editor of Prometheus, a Taylor & Francis journal concerned with innovation, and taking a critical approach to the subject.