The Clinical Toxicology Recommendations Collaborative: purpose, organization, and methodology

Abstract

Introduction

Scientific societies aim to provide a collective voice and unified stance on important issues. The Clinical Toxicology Recommendations Collaborative was formed in 2016 to develop evidence- and consensus-based recommendations for the management of patients exposed to common and/or serious poisonings for which the management is unclear or controversial.

Organization

The Clinical Toxicology Recommendations Collaborative is led jointly by the American Academy of Clinical Toxicology, the Asia Pacific Association of Medical Toxicology, and the European Association of Poison Centres and Clinical Toxicologists. The Governance Committee is chaired by a Past-President of one of these Societies and comprised of the six Presidents and Immediate Past-Presidents of the three Societies. A Steering Committee oversees the process of each project workgroup.

Methodology

The overall process is guided by standards set forth by the Institute of Medicine for developing trustworthy guidelines and the Appraisal of Guidelines for Research and Evaluation Instrument. Systematic reviews are produced using the framework set in the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) methodology. Workgroup members jointly review the evidence and prepare statements on which they vote anonymously using a 9-point Likert scale. A two-round modified Delphi method is used to reach a consensus on clinical recommendations using the RAND/UCLA Appropriateness Method. Final recommendations are approved by unanimous consent of the workgroup and are expressed as both levels of evidence and strength of recommendations.

Limitations

The major limitations of the Clinical Toxicology Recommendations Collaborative process centre around the amount and quality of evidence, the assessment of that evidence, and the voting of the panel.

Conclusions

By using a transparent evidence- and consensus-based approach to produce systematic reviews and clinical recommendations, the Clinical Toxicology Recommendations Collaborative aims to create an international framework for clinical toxicology education and decision-making and foster positive change for the benefit of poisoned patients.

Keywords:

• Clinical Toxicology Recommendations Collaborative
• Delphi method
• Likert scale
• Systematic reviews
• Clinical recommendations

Introduction

One of the functions of scientific societies is to provide a collective voice and unified stance on important issues. Documents such as position statements, guidelines or recommendations serve as guiding principles, reflecting the knowledge, values and priorities of an organization or a community. These documents create a framework for education, decision-making, and hopefully influence positive change. By publicly expressing an opinion, societies foster open dialogue, engage stakeholders, and mobilize action towards addressing unique challenges. Ultimately, society-based clinical guidance documents foster cohesion, clarity, and impactful progress in shaping the future of a given specialty.

The Clinical Toxicology Recommendations Collaborative was formed in 2016 with the aim to develop evidence- and consensus-based recommendations for the management of patients exposed to either common or serious poisoning for which the management may require clarification.

Organization

The Clinical Toxicology Recommendations Collaborative is led jointly by the American Academy of Clinical Toxicology (AACT), the Asia Pacific Association of Medical Toxicology (APAMT), and the European Association of Poison Centres and Clinical Toxicologists (EAPCCT). Collaborating organizations contributing to individual projects include America’s Poison Centers (APC), the American College of Medical Toxicology (ACMT), and the Canadian Association of Poison Control Centres (CAPCC). Furthermore, in order to ensure global appropriateness and usefulness of resulting recommendations, invitations to other organizations not listed for participation above are actively encouraged according to the topic chosen. Funding is provided by the sponsoring organizations.

The Collaborative is overseen by a Governance Committee led by a chair who is a Past-President of one of the three lead organizations and is chosen by consensus of the committee members who are the current Presidents and Immediate Past-Presidents of each of the three lead Societies. The chair serves for a two-year term, and the position rotates among the three lead Societies. The Governance Committee functions to help select topics for clinical recommendations, oversee the work of the Recommendation Steering Committee, resolve disagreements, and communicates with and obtains support from the Boards of the lead Societies.

Individual projects of the Collaborative are initiated and overseen by the Recommendation Steering Committee, whose chair and associate chair are appointed by the Governance Committee for two-year terms. Upon completion of their mandate, the chair will remain on the Steering Committee for a further two years, whereas the associate chair will take over as chair. Other members include the chairs of the Scientific Committees of the lead Societies or their designees. The major function of the Steering Committee is to propose appropriate clinical toxicology management topics based on the medical community’s need for guidance, provide support and guidance for each project, and liaise with the Governance Committee, in which the chair and associate chair are ex-officio advisory members.

Individual projects are organized in workgroups, which are divided into three committees that work in parallel to produce a systematic review and final recommendations: a Systematic Review Committee, a Voting Statements Development Committee, and a Recommendations Development Committee. Chairs are selected by the Recommendation Steering Committee based on expertise, interest, and availability and serve for the duration of the project. Committee members are recruited by each Society and selected for interest and expertise in the project. Societies use their own criteria for selecting members. Society nominees are vetted by the chair and serve for the duration of the project. Although members may belong to multiple Societies, each member is designated as a representative of a single Society. Additional members include a medical librarian and a methodologist. The full terms of reference can be found on the EAPCCT website at https://eapcct.org/index.php?page=joint and the AACT website at https://www.clintox.org/resources/position-statements.

All members of the collaborative must disclose any potential conflicts of interest in writing. These disclosures are reviewed by the chair of the Steering Committee and discussed with the Governance Committee if necessary. The Governance Committee is empowered to request recusal if a significant conflict exists.

Methodology

Because of its immediate applicability to poisoning, the selected methodology builds on that of the Extracorporeal Treatments in Poisoning (EXTRIP) Workgroup [1] by creating a mechanism for Society oversight (the Governance Committee) and shared participation from three Societies. The overall process is guided by the standards set forth by the Institute of Medicine (IOM) for developing trustworthy guidelines [2] and follows the Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument [3]. The concept of the Clinical Toxicology Recommendations Collaborative was piloted with the AACT Lipid Emulsion Therapy Workgroup [4] that ultimately produced three systematic reviews [5–7], a scoping review [8], and evidence-based recommendations [9].

Systematic reviews

In line with the IOM framework [2], systematic reviews are the first step in the process used by the Collaborative. Standard evidence-based questions are formulated by the workgroup in consultation with the Recommendation Steering Committee using the population, intervention, control/comparator, outcomes (PICO) format [10,11]. A medical librarian is recruited to perform a database search from Medline (Ovid) and a minimum of two other databases (such as Embase, Biosis, Google Scholar, and Cochrane). All databases are searched from their inception and without any language restrictions. The search output is uploaded into reference manager software such as Covidence (Melbourne, Australia, https://www.covidence.org/) and title and abstract screening are performed with a minimum of two workgroup members reviewing each citation. A third member adjudicates differences. Another member performs a secondary screen of all excluded articles to ensure the integrity of the process. Full-text copies of selected articles are obtained along with translations if necessary, and another round of screening and adjudication is performed to select the final articles for inclusion. Workgroup members cross-reference full-text articles to identify articles missed in the searches.

Data from included articles are extracted by members on a predetermined standardized spreadsheet and checked by at least one additional member. Two members use the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) methodology [12] to independently assess the quality and risk of bias of each included study. All workgroup members are provided with access to the spreadsheet and full-text citations. The systematic review is authored and reviewed by all members, and unanimous agreement for the final manuscript is obtained. If there is a substantial delay between the systematic review and the formulation of the recommendations, the search strategy will be updated to ensure that no new information is omitted.

Voting statements

A generic format of voting statements is developed by the Voting Statements Development Committee and revised during several in-person and virtual meetings to ensure clarity of language and generalizability. Specific voting statements to answer clinically relevant issues are developed in response to PICO questions [10,11] and guidance from the workgroup and are again reviewed for clarity.

A two-round modified Delphi method is used in an attempt to reach a consensus on clinical recommendations. Workgroup members cast their anonymous votes on a 9-point Likert scale for each statement and provide comments in free text (Table 1). An open discussion follows to review the results of the first vote, clarify issues, and consider minority opinions. Only two rounds of voting occur to avoid artificially forcing consensus that might occur with further iterations, as votes would be expected to coalesce around the central tendency (mean/median). Panelists are made aware that the failure to reach a consensus is an expected and acceptable outcome so that there is no need to change opinions on the second vote unless panelists choose to do so based on their re-evaluation. The RAND/UCLA Appropriateness Method [1,13] is used to quantify disagreement between the votes cast by the panel. The median values, the lower/upper quartiles, and the disagreement indices are calculated for each of the two rounds of votes. The disagreement index describes the dispersion of ratings with a disagreement index value less than or equal to 1 indicating agreement. An example of this calculation for a hypothetical series of votes is provided in Table 2.

Table 1. 9-point Likert scale for voting statements.

Vote	Meaning
1	The risks of the action greatly surpass or outweighs the potential benefits, and it should never be given in the circumstances described in the sentence
2	The risks of the action greatly surpass or outweighs the potential benefits, and it should almost never be given in the circumstance described in the sentence
3	The risks of the action likely surpasses or outweighs the potential benefits and it should generally not be given in the circumstance described in the sentence
4	The risks of the action probably surpasses or outweighs the potential benefits and it should generally not be given in the circumstance described in the sentence
5	The benefits of the action is equivalent to its risks (neutral)
6	The benefits of the action probably surpasses or outweighs the risks and it should probably be given in the circumstance described in the sentence
7	The benefits of the action surpass or outweighs the risks and it should generally be given in the circumstance described in the sentence
8	The benefits of the action greatly surpass or outweighs the risks and it should almost always be given in the circumstance described in the sentence
9	The benefits of the action greatly surpass or outweighs the risks and it should always be given in the circumstance described in the sentence
N/A	Not applicable: you don’t feel you can vote due to a lack of information to provide an evidence-base answer

Table 2. Sample disagreement index and vote calculations.

Panelists	Vote 1	Vote 2	Vote 3	Vote 4
1	9	9	9	9
2	8	8	8	9
3	7	7	7	9
4	6	6	6	6
5	9	9	4	6
6	9	9	2	2
7	8	8	4	6
8	7	7	7	9
9	6	6	6	6
10	9	9	9	1
11	8	8	8	8
12	8	8	8	2
13	6	6	6	6
14	7	6	7	1
15	9	9	9	3
16	9	9	5	5
17	5	5	5	1
18	6	6	6	1
19	7	7	5	1
20	4	1	4	4
21	9	9	9	9
Median	8	8	6	6
75th percentile	9	9	8	8
Lower limit of interpercentile range	7	6	5	2
Upper limit of interpercentile range	9	9	8	6
Interpercentile range	2	3	3	4
Interpercentile range center point	8	7.5	6.5	4
Asymmetry index	3	2.5	1.5	1
Interpercentile range adjusted for symmetry	6.85	6.1	4.6	3.85
Disagreement index (interpercentile range/ interpercentile range adjusted for symmetry)	0.29	0.49	0.65	1.04
Interpretation	Consensus around a strong recommendation	Consensus around a weak conditional recommendation	Consensus around a neutral recommendation	No consensus

In a hypothetical panel with 21 members (first column), the panellists vote on four different statements (second through fifth columns). The range of possible votes is shown on a Likert scale of 1 (strong disagreement with the statement) through 9 (strong agreement with the statement (Table 1). The midpoint of the Likert scale is 5. The authors first identify the median, 75th percentile, 25th percentile, and the interpercentile range center point (midway between the 75th and 25th percentiles). The asymmetry index is the absolute value of the distance between the interpercentile range center point and the scale center point of 5. The interpercentile range adjusted for asymmetry is calculated as 2.35 + (1.5* the asymmetry index). The disagreement index is then calculated as shown. Figure 1 shows how the median and the upper or lower quartile are used to provide the recommendation once the disagreement index is < 1.

Recommendations

The level of evidence (Table 3), the voting results and the disagreement index (Figure 1) are combined to inform on the strength of recommendations. The workgroup then formally discusses the results of these votes and uses the evidence to confirm a decision on the final recommendations by consensus [14]. Each recommendation will contain both a level of evidence and a measure of consensus. A limited number of good practice statements [15] are permitted to provide recommendations in the absence of data.

Figure 1. Use of the disagreement index, the median of votes, and upper or lower quartile of the votes to arrive at the strength of recommendations made by the workgroup.

Table 3. Level of evidence to inform on the strength of recommendations.

Level of evidence (GRADE^a system)

Grade A: High level of evidence (The true effect lies close to our estimate of the effect).

Grade B: Moderate level of evidence (The true effect is likely to be close to our estimate of the effect, but there is a possibility that it is substantially different).

Grade C: Low level of evidence (The true effect may be substantially different from our estimate of the effect).

Grade D: Very low level of evidence (Our estimate of the effect is just a guess, and it is very likely that the true effect is substantially different from our estimate of the effect).

Strength of recommendation (consensus-based)

Level 1: Strong recommendation (The course of action is considered appropriate by the large majority of panel members with no major dissension. The panel is confident that the desirable effects of adherence to the recommendation outweigh the undesirable effects.)

Level 2: Weak/conditional recommendation (The course of action is considered appropriate by the majority of panel members but some degree of dissension exists among the panel. The desirable effects of adherence to the recommendation probably outweigh the undesirable effects.)

Neutral position: The course of action is neither preferred nor rejected by the majority of panel members; either due to a balance in the desirable and undesirable effects of adherence to the recommendation or due to major uncertainties to its evaluation.)

No recommendation: The panel members reached no agreement

^aGRADE: Grading of Recommendations, Assessment, Development, and Evaluations.

Before finalizing the recommendations, the workgroup presents its findings at annual meetings of AACT, APAMT and EAPCCT to inform members of work progress and seek input to clarify clinical relevance to the membership. The workgroup then considers these comments in their deliberations to form the final recommendations.

Updates of previous society guidelines or position statements

The same methods as described above will be applied to updates of any existing works. Since the standards and methodology required to develop evidence-based framework for any clinical guidance documents have evolved significantly in the last twenty years, the three lead Societies jointly agreed when forming the Clinical Toxicology Recommendations Collaborative that it would be best to appraise the entirety of the evidence using a singular methodology rather than layer a new method onto an update of a previous work. While this procedure is clearly more time-consuming, it ensures that all data will be assessed using a single methodology and standard.

Society endorsement and acknowledgement of participation

Systematic reviews will be published from the Clinical Toxicology Recommendations Collaborative without additional input from the sponsoring Societies. The Boards of the Lead and Collaborating Organizations are required to vote whether to endorse the final recommendations but will not be obligated to endorse any publication. Organizations endorsing the recommendation will be listed as endorsing the guidelines within the publication. Any organization(s) not endorsing the recommendations may decide whether to be listed as “contributing”. Authorship will be decided within the Clinical Toxicology Recommendations Collaborative, but the final author of recommendations will be listed as “for the Clinical Toxicology Recommendations Collaborative” and all participants regardless of whether they qualify for authorship will be listed in a table as a representative of one of the societies.

Publication in the Societies' Journal, Clinical Toxicology

The Clinical Toxicology Recommendations Collaborative is expected to submit their manuscript to Clinical Toxicology which is sponsored by the three Societies. Open-access will be provided by the sponsoring Societies to enhance knowledge translation and visibility of the Societies’ work.

Limitations

The major limitations of the Clinical Toxicology Recommendations Collaborative process center around the amount and quality of evidence, the assessment of that evidence, and the voting of the panel. Each of these will be discussed, along with plans to mitigate those limitations.

Amount of evidence

For many important clinical questions, an extensive evidence base does not exist. When evidence is limited, recommendations are one important tool to help clinicians make the best-informed decisions for their patients. In order to reduce the risks associated with a limited amount of evidence the Clinical Toxicology Recommendations Collaborative plans to search multiple databases from their inception without any language requirements. Full translations of citations accepted for the systematic review will be obtained and made available for data extraction. The references of included articles will be reviewed to ensure completeness. Panelists have the option of adding citations if they are aware of items that are missed. If no relevant evidence is found, then the Clinical Toxicology Recommendations Collaborative will not make any form of recommendation other than the rare “good practice statements” mentioned in the methodology above.

Quality of evidence

As with the amount of evidence, the quality of evidence will vary with each clinical question. The quality of evidence will be assessed by a standard evidence-based medicine rubric such as GRADE. That assessment will be performed independently by two trained members of the Clinical Toxicology Recommendations Collaborative, who will adjudicate according to a standard which down-grades quality when in doubt. Many modern prominent guidelines [16–18] commonly include and rely on low-quality evidence, including expert opinion when higher quality data are lacking. As shown in Table 3, the quality of evidence will drive the certainty of the recommendation.

Consensus panel

The use of a consensus panel has obvious limitations. It is possible that a different group might reach different conclusions when evaluating the existing and often limited data. The Clinical Toxicology Recommendations Collaborative is composed of a large panel of members who represent Societies with membership from diverse geographic regions and include representatives who specifically focus on adults and children. Both rounds of votes will be anonymous to prevent members from influencing the voting of others. While open discussion occurs between rounds, there will always be a request for dissenting opinions. Only two rounds of voting will occur to prevent movement toward the Collaborative workgroup central tendency, as might be expected with repeated voting. Panelists are aware that failure to reach consensus on proposed recommendations is an acceptable option. After final recommendations are voted on, those that meet the requirements for consensus will require unanimous approval before adoption. This approach is designed to minimize the influence of a single or small group of vocal members and to not force consensus on items for which no consensus exists.

Conclusions

The Clinical Toxicology Recommendations Collaborative is led by three international toxicology Societies with the mandate of developing recommendations for the management of poisoned patients. By using a transparent evidence- and consensus-based approach to produce systematic reviews, voting statements and recommendations, the Clinical Toxicology Recommendations Collaborative aims to create an international framework for clinical toxicology education and decision-making and foster positive change for the benefit of poisoned patients.

Acknowledgment

The authors would like to thank Sophie Gosselin, methodologist, Angela Chiew and Christopher Yates, chair and past-chair of the Recommendation Steering Committee, respectively, for their valuable input when preparing this manuscript.

Disclosure statement

The authors are members of the Governance Committee of the Clinical Toxicology Recommendations Collaborative and currently hold the following positions in its lead organizations:

American Academy of Clinical Toxicology: President (CH) and Immediate Past President (KLC); Asia Pacific Association of Medical Toxicology: President (HHM) and Immediate Past President and current Chair of the Governance Committee (MLT); European Association of Poison Centres and Clinical Toxicologists: President (AFD) and Immediate Past President (HT), Past President and Chair of the Governance Committee until October 2023 (MFW).

Additional information

Funding

The authors reported there is no funding associated with the work featured in this article.