Reshaping the publication process: Addiction Research and Theory joins Peer Community In Registered Reports

There is growing awareness of reproducibility concerns within the scientific literature; whilst journals continue to be filled with seemingly beautiful results, teams of researchers are lifting the lid on the veracity of scientific claims and the replicability of findings in a growing number of disciplines. Highlighting the global scale of this issue, the Open Science Collaboration (2015) performed direct replications of 100 psychological research studies and found that whilst 97% of the original studies reported significant results, this declined to 36% for replications of these. Moreover, the average effect size of the replications was half the magnitude of the average across original studies. Several human decision-making factors have been proposed to underlie irreproducibility, such as selective reporting and questionable research practices (QRPs; Simmons et al. 2011; John et al. 2012) fueled by pressures to achieve novel, surprising results and incentive structures that reward quantity over quality (Giner-Sorolla 2012; Frith 2020; Munafò et al. 2020). As can be seen in Figure 1, there are many threats that can arise at each stage of the hypothetico-deductive scientific process, which can undermine the robustness of published research and constrain knowledge accumulation. In their ‘manifesto for reproducible science’, Munafò et al. (2017) propose five measures that target these threats with the goal of improving research: (1) methods, (2) reporting and dissemination, (3) reproducibility, (4) evaluation, and (5) incentives. Adoption of open science practices will allow many of these measures to be achieved and calls for their implementation in addiction science are becoming louder (Heirene 2021; Louderback et al. 2021; Pennington et al. 2021). However, to be successful, academic journals also need to adapt and recentre their focus on scientific rigor rather than the metamorphosis of study results. In this Editorial, we introduce two types of Registered Reports (RRs) that satisfy all these measures and highlight our commitment by offering these publishing tracks to authors through ART. We first outline the ‘traditional’ RR format (which ART has offered since 2019) and its associated advantages, followed by the introduction of a new initiative launched in April 2021 – Peer Community In Registered Reports (PCI RR).

Figure 1. Illustration of the reproducibility crisis: the blue circle represents the hypothetico-deductive process of the scientific method, and at each step, threats to this model are indicated in underneath. Figure reproduced from Munafò et al. (2017), without adaptations, under a CC-BY 4.0 license.

Traditional RRs

RRs represent a publishing model whereby initial peer review is performed before a study is conducted or analyses of secondary data are initiated (Chambers 2013; Nosek & Lakens 2014). RRs differ from study preregistration in that the study protocol is submitted to a journal rather than solely to a registry. Whilst study preregistration has many advantages, such as clearly distinguishing hypothesis testing ('prediction') from hypothesis generation (‘postdiction’; Nosek et al. 2018), recent research suggests that it requires careful oversight to protect against selective reporting and undisclosed discrepancies (see Vassar et al. 2019; Bakker et al. 2020). RRs help prevent these issues from occurring in the first place by splitting the peer-review process into two stages: pre-study (Stage 1) and post-study review (Stage 2). At Stage 1, the authors submit their introduction, methods and analysis plan to a journal and expert peer-reviewers then assess the validity of the research questions, the robustness of the proposed methodology, and the rigor of the planned analyses. Following detailed review and revision, proposals that are favorably assessed are issued an ‘In Principle Acceptance’ (IPA). This means that the journal commits to publishing the final article regardless of whether the hypotheses are supported, so long as the authors adhere to their approved protocol and the journal’s formatting requirements. Upon receiving IPA, authors then typically register their protocol in a repository (e.g. via the Open Science Framework). After finalizing their study, the authors then submit a Stage 2 manuscript that includes their original Stage 1 protocol with the results and discussion now affixed (Chambers and Tzavella 2020; see Figure 2). Importantly, at this stage confirmatory and exploratory analyses are distinguished and reviewers cannot reject the article on the basis of the study’s results. Through more rigorous bias control, RRs help to create more valid findings in the general literature to maximize trust in scientific research.

Figure 2. Traditional RR workflow. Figure reproduced from the Center for Open Science (www.cos.io/rr. ), without adaptations, under a CC-BY 4.0 license.

RRs therefore improve (1) methods by allowing peer-reviewers to provide valuable feedback on the study protocol during the crucial planning stages, (2) reporting and dissemination by ensuring that the accepted protocol is followed and results are interpreted appropriately, (3) reproducibility by reducing questionable research practices and requiring open data and/or code, (4) evaluation by mitigating the outmoded emphasis on study results to dissipate publication bias, and (5) incentives by restructuring the publication process to give authors control and rewarding open science practices. RRs also champion replication attempts, which are beginning to become more mainstream in psychological and other sciences but are largely absent within the addiction literature (Heirene 2021). Emerging findings attest to the value of RRs: null findings are approximately five times more likely in RRs relative to regular articles, suggesting that they reduce publication bias and/or QRPs (Allen and Mehler 2019; Scheel et al. 2021). Furthermore, RRs enhance study quality (Soderberg et al. 2020), enact higher levels of open data and computational reproducibility (Obels et al. 2020), and attract higher altmetric scores and similar-to-slightly greater numbers of citations compared to the general (non-RR) literature (Hummer et al. 2017). Despite their substantial benefits to both researchers and journals, ART represents one of only four addiction journals offering this publishing format to date (see Gorman 2019; Pennington et al. 2021). As well as our journal guidelines, there are several practical guides for authors of RRs (Kiyonaga and Scimeca 2019; Stewart et al. 2020; Center for Open Science 2021).

Peer community in registered reports

A new type of RR initiative is publisher-independent and known as Peer Community In Registered Reports (PCI RR). Launched in April 2021, PCI RR is a community-driven initiative dedicated to reviewing and recommending RRs across the full spectrum of STEM, medicine, social sciences, and the humanities. Through this route the Stage 1 protocol and Stage 2 manuscript are submitted for consideration by PCI RR and reviewed independently of journals by expert peer-reviewers. The review process is overseen by accredited ‘Recommenders’, who must pass an entrance test to demonstrate an in-depth understanding of RRs and ensure high quality review and decision-making. Following a favorable recommendation at each stage of the process, the manuscript is then posted at a preprint server and the peer reviews and recommendation of the preprint are made openly available via the PCI RR website. Figure 3 outlines the full process of the PCI RR track. Following completion of the peer review process, authors of recommended RRs have the option to publish in a growing list of ‘PCI RR-friendly’ journals that commit to accepting the recommendation without the need for additional peer review. Authors are informed at the point of Stage 1 in-principle acceptance (IPA) which PCI RR-friendly journals are eligible outlets. All eligible journals then automatically offer Stage 1 IPA without requiring authors to submit their Stage 1 manuscript. Authors who intend to publish their RR in a PCI RR-friendly journal need not submit their manuscript to a journal until after the final positive Stage 2 recommendation. The novelty of this initiative is that it puts authors in control: they choose the destination journal.

Figure 3. Process of PCI RR track. Figure reproduced from https://rr.peercommunityin.org/about/about, without adaptations, under a CC-BY 4.0 license.

In support of this initiative, ART has signed up as a ‘PCI RR friendly’ journal, which means that we accept Stage 2 manuscripts that have received a positive final recommendation through PCI RR. To guarantee acceptance, authors need to ensure that their RR meets the scope of the journal and the journal requirements concerning bias control and formatting. It is our view that both traditional and PCI RRs will help reshape the publication process and contribute to a more representative literature, and we look forward to seeing their success.

Conclusions

Academic publishing is changing, and ART is changing with it. In addition to publishing non-RR articles, we now offer two distinct RR tracks: authors can submit articles for pre-study peer-review directly to ART through the traditional RR route, or take advantage of guaranteed acceptance in ART without additional peer review once their study has been recommended through the PCI RR initiative. We hope that these developments will contribute to addiction science more actively adopting open science practices and help mitigate reproducibility concerns within the published literature. If you are interested in submitting through the RR track, please consult our instructions for authors.

Disclosure statement

Charlotte Pennington is a Handling Editor at Addiction Research & Theory with a particular brief to support ART’s commitment to open science by overseeing Registered Reports. She is also a Recommender of PCI RRs and is the Network Lead for the UK Reproducibility Network (UKRN) for Aston University. Derek Heim is the Editor in Chief of ART and has initiated both traditional RRs and registered ART as a ‘PCI RR friendly’ journal.