The anxiety of the lone editor: fraud, paper mills and the protection of the scientific record

Although it might be biomedical disciplines that first spring to mind when we think about scientific deception within scholarly publishing, no discipline is immune to fraudulent conduct and all journals need to be vigilant. The Journal of Mental Health (JMH) is no exception. Editors and publishers have taken a stand against scientific misconduct and fraud, but the 24-hour attention required to prevent untruths and downright lies getting into the publication record does take a toll. Editors are the first line of defence against fraudsters, and more than a watchful eye is required. Their vigilance requires a knowledge of the cues to wrongdoing. Keeping ahead of those trends is time-consuming, leaving editors exhausted and stressed in case they let something through that taints the scientific record.

The pandemic taught us that misinformation is powerful and can affect our health and our mental health, and when it is published in a credible journal it is often taken as a “fact”. Remember the Wakefield scandal. This was one paper linking the MMR vaccine with autism that was published in a high-profile journal and for which the editor later apologised. That one paper produced a reduction in vaccinations to the detriment of children’s health and even increased deaths. Apart from breaking ethical codes, the author was found to have manipulated data and eventually The Lancet retracted the paper and the editor said the journal “had been deceived” (The Editors of The Lancet, 2010; Boseley, 2010). We do not want that to happen to our journal. Peer review is one way of checking the worth of the scientific method and likely veracity of the results, but reviewers cannot go into the laboratory or check individual data and anyway the systems for producing fake data are getting more sophisticated so more difficult to spot. Our entire system depends upon trust, but trust may not be enough.

What is the problem?

Publishing misconduct covers a variety of unethical behaviours, often deliberate but not always so. Technological advances have changed the way that research is produced, submitted, and published. This has provided opportunities for deliberate misconduct on an industrial scale, and for financial gain. We are talking here about paper mills, which are described by the Committee on Publication Ethics (COPE, 2020) as “profit oriented, unofficial and potentially illegal organisations that produce and sell fraudulent manuscripts that seem to resemble genuine research”. They subvert journal systems by submitting manufactured manuscripts for a fee on behalf of researchers to provide them with an easy publication. They also offer authorship for sale so that an institution or a person can benefit. Paper mills pose a serious threat to the integrity of the scholarly record, and they are resulting in the retraction – sometimes mass retraction – of ever-increasing numbers of journal articles. Jay Flynn, Executive Vice-President at Wiley, has written about the problems that the publisher has recently faced with the retraction of papers and delisting of journals from Web of Science (owned by Clarivate) as a consequence of paper mill activity (Flynn, 2023; King, 2023). However, this is only one example of many and publishers across the board have been hit by paper mills, including Taylor and Francis. A joint study by COPE and STM (2022) found that up to 2% of papers submitted to journals were identified as potentially fake. In the case of the recent de-listings from Web of Science, Clarivate has not gone into detail about why the journals were delisted but we must assume that concerns about the mechanisms of control over peer review and fake papers may have been a key consideration. We have noted that one mega-journal published 9500 papers in 2020 and 17,000 in 2022 which is a load no editor would relish.

Paper mill activity usually combines several unethical practices in one paper: dual submission, plagiarism, data fabrication, image manipulation, and manipulation of the author list. We also see attempted manipulation of the peer review process, for example by authors suggesting “fake” reviewers. The estimated global revenue of paper mills is one billion Euros and probably more with brokers advertising $800 for a paper in a middle tier domestic journal (Olcott E. & Smith, 2023). So, this is big business. Of course, the above are all examples of misconduct that can also happen on an individual non-papermill article, so even journals that are not the recipients of paper mill-produced manuscripts are almost inevitably faced with unethical conduct to one degree or another.

Why has this become an increasing problem? Many universities place a high value on peer reviewed publications in their assessment of initial employment and promotion, and for some it determines financial reward. Universities, even prestigious ones, tend to value quantity over quality despite the examples set by Nobel Laureates. Peter Higgs had only published a few papers (h-index of 9–11 on some estimates) when he received a Nobel Prize, and John F. Clauser’s h-index in Scopus is only 22 (and he has only 38 publications) and that includes citations since being awarded his Nobel Prize. Publishing has always been a pressure for academics and with the increasing educational workload in many universities, it is sometimes hard to find time to develop a scientific question, get the required funding, carry out the study and write the publication. This leads some to delve into these unscrupulous procedures.

Retraction Watch now has more than 40,000 papers logged as retracted with at least 321 on Covid alone. We have seen instances such as the case of P.V. Arun in 2014 (Arun, 2014) who submitted the same paper to three journals which were all published – including by our own publisher – then retracted (Geocarto International, 2018). This paper may be entirely legitimate research, but its triplicate publication will skew the scientific record. Retracted papers are a global issue. Countries such as the USA and China unsurprisingly see high overall volumes of retracted articles given their high research output, but an analysis of PubMed retractions by Bhatt (2020) found that papers from Iran, Tunisia, Pakistan, Bangladesh, and India had the highest rates of retraction, and Cabanac et al. (2023) found that Japan and Egypt were also major sources. The recent COPE & STM report (2022) notes instances of papermills operating in China, Russia, Latvia and Iran, and research by Albalkina (2022) and Sabel et al. (2023) into paper mills and the production of fraudulent content also found that certain countries were affected by these operations more than others.

How do we combat this?

Combatting publishing misconduct, whether it results from paper mill activity or the behaviour of individuals, is a significant challenge and requires the involvement of institutions, funders, publishers, and editors.

Taylor and Francis, the publisher of JMH, in common with other publishers, has policies, practices and processes in place to ensure the integrity of the content that we publish (Taylor and Francis, 2023a). These are constantly under review and evolve and adapt to meet the needs of the changing publishing landscape. For instance, they have updated the authorship policy to make it clear that artificial intelligence-based tools for content generation (such as Chat GPT) do not meet the criteria for authorship (Taylor and Francis, 2023b).

There is a risk in going into any detail about our methods of protection or the checks that we carry out or could carry out on JMH submitted papers. This information is valuable to bad actors trying to circumvent these safeguards. However, we can say that technology is critical in the fight against unethical behaviour and the publication of fraudulent content. The use of software such as CrossRef’s Similarity Check is now routine, and increases our capacity to detect author misconduct, and to do so quickly. Despite this, some types of unethical behaviour are easier to detect than others. While technology may be important, editors (and reviewers) play a vital role too. Editors can spot where language is odd enough to warrant further investigation before sending the submission for peer review. Peer reviewers can assess the scientific method, the reliability of the results and importantly whether the findings change the dial on science. The editors of this journal even suggested the kinds of papers that would be accepted into the journal on Covid-19, mainly to answer questions above and beyond that Covid-19 was stressful to all (Wykes et al., 2021). In addition, we have stringent requirements to prevent potentially unethical content from entering the submission pipeline in the first place, for example, controls around the way that submissions to special collections are handled, and education also plays an important role in helping authors avoid inadvertent misconduct such as self-plagiarism or affiliation misrepresentation.

These challenges are happening against a background of high submission numbers. Almost five million papers were published in total, worldwide in 2022 according to data from Dimensions (Dimensions, 2023). That is not the number of papers submitted, which will be significantly higher. According to Retraction Watch, almost 5,000 papers were retracted in 2022 (Retraction Watch, 2022) and this number will increase unless we have the technology, the beady eye of an experienced editor and the good will of our peer reviewers.

The mental health of a mental health journal editor

Our journal, like others, has seen an increase in submissions. Last year we published over 100 articles, and our rejection rate was 85%. Many of these were rejected by the editors prior to peer review after an assessment of the fit of the paper, the design, and methods adopted. As we are dealing with large numbers of articles, and despite everyone’s best efforts, it is likely that problematic papers will inadvertently be published. We hope that our rigorous editorial assessment means we capture the most obvious ones, but this is a hope not based in any empirical research.

It is not surprising that this is leading to editor anxiety. Much has been written about the mechanics of unethical publishing behaviour – what is driving it, the implications for the scholarly record, the ways in which we can combat it – but little about its effect on those who are dealing with these papers, and the worry it can cause editors who are ultimately responsible for making that “accept” or “reject” decision on a paper.

Identifying potentially fraudulent activity means dealing with uncertainties, and many different individual uncertainties are present at each stage of the decision pipeline. This all takes time. Each of these micro decisions increases the everyday workload, and when they are multiplied tenfold, then dealing with each decision is even harder. If not dealt with strategically the stress caused can lead to poor mental health outcomes, especially as the scientific community expects editors and peer reviewers to take a leading role in preventing fake papers entering the scientific record.

Anxiety is one of those poor mental health outcomes and it also affects behaviour. These behaviours are known as safety behaviours that reduce the immediate anxiety but tend not to resolve the problem. They usually consist of avoidance, distraction, preparing, and checking. For editors, it can lead to shortcuts where decisions are made fast on less information (avoidance). Fast decisions are not always good decisions and may only reduce anxiety in the short term. This fast decision making is going to become standard given the increased number of submissions in just about every journal, so careful consideration needs to be given to how editors can best be supported in making these decisions. This is something that cross-departmental teams at Taylor & Francis are working closely on.

Fast decisions may also mean introducing biases. For example, only papers from high profile institutions being sent for peer review in the misconception that they are immune to paper mills and, in extreme cases, the immediate rejection of papers from countries known to suffer the most from these unethical methods of achieving publications. Clearly, this is discriminatory behaviour and if editors head down this path it is extremely problematic and indeed unethical, and steps would need to be taken to address it. This bias is not something that we have seen in JMH and we publish papers from a wide variety of authors, institutions, and countries. This variety can be seen by any reader browsing the journal but the following articles give a snapshot of the breadth of authorship: Aluh et al., 2023; da Silva et al., 2022; Hazan & Chan, 2022; Husain et al., 2023; Lin et al., 2022; Motilal et al., 2022; Phuspa et al., 2022; Şen Doğan & Deveci Şirin, 2022; Wang et al., 2023; Zhang et al., 2023. We nonetheless need to be watchful and ensure that all papers are scrutinised appropriately, and as submissions increase and paper mills become more sophisticated this will increase the workload further. It is also another safety behaviour – checking.

Verifying the identity of the submitting individual means being suspicious of the information provided at submission, and not taking anything at face value. But it is not remotely possible for an editor to check all authors on all submissions and whether they are present at the university where they say they are employed and so we avoid that, another safety behaviour. This is another area where technology and publisher intervention can help by taking on some of this checking and alleviating the editorial burden, therefore reducing the need for safety behaviours.

We do prepare for some of these eventualities and scrutinise the reviewers’ comments carefully and, unlike many journals, JMH has had double anonymous reviewing for many years which may alleviate some publication biases. But this has not made us immune to retractions. We retracted a published paper recently after we became aware of a conflict of interest that was not disclosed at submission. We believed that the conflict could have compromised the reliability of the reviews and the paper’s findings. The paper was subsequently removed entirely due to a breach of our editorial policies (Taylor and Francis, 2022). This is evidence that we can and do respond quickly to reader’s suspicions.

We also check our submission record and have discovered papers in our rejection list that had been submitted elsewhere at the same time. The editorial decision to reject prevented a recent paper being reviewed and considered for publication, but that was not useful in calming anxiety, it just confirmed the problem.

What now?

The problem is here and now, not just in life sciences journals. Publishers can develop technology to identify the worst offenders. Editors still need to be careful about wasting reviewers’ time by a stringent decision-making process that considers scientific merit and whether the paper fits the journal. Fast decisions also provide authors with the opportunity to submit to a more suitable journal. Editors need to consider their likely decision bias as they resort to faster decisions and this potential bias needs checking regularly to ensure that the scholarly record still contains global publications. If our peer reviewers and readers spot problems, then we will react.

We are also grateful to Retraction Watch and the relentless activity of just a few players in this space to help ensure that our science is reliable and reproducible. To all editors, we share your concerns and the burden of being in a position of trust. Some papers are likely to slip through your net, probably an inevitability, but you should work with your publisher to build the technology and awareness of methods of reducing your workload of checking. Your anxieties are shared but let’s make the effects as small as possible.

Acknowledgements

With thanks to Laura Wilson, Head of Research Integrity and Ethics at Taylor and Francis, for her input.

Disclosure statement

Anna Parkinson is a portfolio manager at Taylor and Francis Group.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.