Validity of Routine Health Data To Identify Safety Outcomes of Interest For Covid-19 Vaccines and Therapeutics in the Context of the Emerging Pandemic: A Comprehensive Literature Review

Figures & data

Figure 1 Study flowchart.

Notes: Individual outcomes count sums to more than the total as five studies (Sundbøll et al,⁸ Ammann et al,⁹ Arana et al,¹⁰ Dalsgaard et al¹¹ and Psaty et al¹²) provide information for multiple outcomes.

Table 1 Characteristics of Selected Studies for Cardiovascular Outcomes

OUTCOME	AUTHOR (YEAR)	DATABASE (COUNTRY)	SETTING (HEALTHCARE SETTING & TYPE OF DATA SOURCE)	CODING DICTIONARY	ALGORITHM DEFINITION	GOLD STANDARD	PPV
Myocardial infarction	Ammann et al (2018)^Citation9	Sentinel Distributed Database (USA)	Claims (secondary care)	ICD-9-CM	Codes in any diagnostic field	Medical record review	MI: 75%
Myocardial infarction	Arana et al (2020)^Citation10	Clinical Practice Research Datalink (UK)	EHR data (primary care)	Read codes	Read codes for myocardial infarction	GP questionnaire	MI: 98%
Atrial fibrillation	Ashburner et al (2017)^Citation14	Primary Care Practice Based Research Network at Massachusetts General Hospital (USA)	EHR (primary care)	ICD-9 & ICD-10	At least one code plus at least one problem list entry or two codes within the previous 3 years.	Medical record review	MI: 96%
Myocardial infarction	Brouwer et al (2015)^Citation15	North Carolina Medicaid (USA)	Claims (secondary care)	ICD-9-CM	Codes in any diagnostic field	Registry data	MI: 44%
Myocardial infarction	Bush et al (2018)^Citation16	Medicare and Medicaid Services (USA)	EHR data (primary care)	ICD-9	Codes in any diagnostic field	Registry data	MI: 67%
Myocardial infarction	Colantonio et al (2019)^Citation17	Medicare (USA)	Claims (secondary care)	ICD-9	Codes in a primary diagnosis field; Codes in any diagnosis field	Registry data	MI (primary position):90%; MI (any field):84%
Myocardial infarction	Cozzolino et al (2019)^Citation18	Centralised administrative database of the Umbria Region (Italy)	EHR data (secondary care)	ICD-9	Codes in a primary diagnosis field	Medical record data	MI: 95%
Myocardial infarction	Dalsgaard et al (2019)^Citation11	Danish National Patient Register (Denmark)	EHR (secondary care)	ICD-10	Codes in a primary diagnosis field	Medical record review	MI: 75%
Myocardial infarction	Di Chiara et al (2019)^Citation19	Clinical-administrative databases of the Health Information System of Friuli Venezia Giulia (Italy)	EHR (secondary care)	ICD-9	Codes in a primary diagnosis field	Troponin measure	MI:96%
Atrial fibrillation	Ding et al (2019)^Citation20	MIMIC-III database (USA)	EHR (secondary care)	ICD-9	Codes in any diagnostic field	Rhythm assessment	AF: 79%
Myocardial infarction	Floyd et al (2016)^Citation21	Veterans’ Health Administration (USA)	Claims (secondary care)	ICD-9	Codes in any diagnostic field	Medical record review	MI:80%
Myocardial infarction	Govatsmark et al (2020)^Citation22	Norwegian Myocardial Infarction Register (Norway); Norwegian Patient Register	EHR (secondary care)	ICD-10	Codes in a primary diagnosis field	Medical record review	MI (MI register):98%; MI (NPR) data):95%
Myocardial infarction	Psaty et al (2016)^Citation12	Centre for Medicare and Medicaid Services (USA)	Claims (secondary care)	ICD-9	Codes in a primary diagnosis field; Codes in any diagnosis field	Registry data	MI (primary position):91%; MI (any field):70%
Myocardial infarction; Atrial fibrillation; Bradycardia; Ventricular tachycardia; Myocarditis	Sundbøll et al (2016)^Citation8	Danish National Patient Register (Denmark)	EHR (secondary care)	ICD-8 & ICD-10	Codes in a primary diagnosis field	Medical record review	MI: 97%; AF: 95%; Bradycardia: 87%; Ventricular tachycardia: 80%; Myocarditis: 64%
Atrial fibrillation	Tu et al (2016)^Citation23	Health administrative data holdings for the province of Ontario held at ICES (Canada)	EHR (secondary care)	ICD-9 & ICD-10	Codes in any diagnostic field	Medical record review	AF:71%
Atrial fibrillation	Wei et al (2016)^Citation24	Vanderbilt University Medical Centre (USA)	EHR (secondary care)	ICD-9	At least one code; ≥2 codes	Medical record review	AF (≥2 codes): 88%; AF (1 code): 72%
Myocardial infarction	Youngson et al (2016)^Citation25	The Discharge Abstract Database (Canada)	EHR (secondary care)	ICD-10	Codes in a primary diagnosis field	Registry data	Not reported

Table 2 Characteristics of Selected Studies for Pulmonary Conditions

OUTCOME	AUTHOR (YEAR)	DATABASE (COUNTRY)	SETTING (HEALTHCARE SETTING & TYPE OF DATA SOURCE)	CODING DICTIONARY	ALGORITHM DEFINITION	GOLD STANDARD	PPV
RDS	Kerchberger et al (2020)^Citation26	Validating biomarkers in Acute Lung Injury for Diagnosis (VALID) study (USA)	Electronic medical records (EMR) data (secondary care)	Unknown	Diagnostic billing and procedural codes. Structured data elements indicating presence of an ARDS risk factor, acute hypoxemic respiratory failure with mechanical ventilation, and oxygenation and expert interpretation of chest radiograph	Physician review	RDS:46%
Pneumonitis	Juurlink et al (2006)^Citation7	Discharge abstract database (Canada)	EMR data (secondary care)	ICD-10	Codes in the primary diagnosis position/ Codes in any diagnostic field	Medical chart review	Pneumonitis: 72% (primary position); 67% (any diagnostic field)

Table 3 QUADAS Quality Assessment

Study	Q1	Q2	Q3	Q4	Q5	Q6	Q7	Q8	Q9	Q10	Q11	Q12	Q13	Q14	SCORE
Al-Ani et al (2015)^Citation27	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	13
Alotaibi et al (2015)^Citation28	No	Unclear	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	Yes	11
Ammann et al (2018)^Citation29	No	Yes	Yes	Yes	No	No	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	10
Arana et al (2020)^Citation10	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	12
Ashburner et al (2017)^Citation14	No	Yes	Unclear	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	11
Brouwer et al (2015)^Citation15	No	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Unclear	Unclear	Yes	Yes	Yes	10
Bush et al (2018)^Citation16	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	Unclear	Yes	Yes	Yes	12
Colantonio et al (2019)^Citation17	No	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Unclear	Unclear	Yes	Yes	Yes	10
Cozzolino et al (2019)^Citation18	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	No	Yes	Yes	Yes	12
Dalsgaard et al (2019)^Citation11	No	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Unclear	Unclear	Yes	Yes	Yes	10
Di Chiara et al (2019)^Citation19	Yes	Yes	Yes	Yes	No	Unclear	Yes	Yes	Yes	Unclear	No	Yes	Yes	Yes	10
Ding et al (2019)^Citation20	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	13
Fang et al (2017)^Citation30	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	12
Floyd et al (2016)^Citation21	No	Unclear	Yes	Yes	Yes	No	Yes	Yes	Yes	Unclear	Unclear	Yes	Yes	Yes	9
Govatsmark et al (2020)^Citation22	Yes	Yes	Yes	Yes	No	No	Yes	No	Yes	Unclear	Unclear	Yes	Yes	Yes	9
Hall et al (2016)^Citation31	No	Yes	Unclear	Yes	Yes	Yes	Yes	Yes	No	Unclear	Unclear	Yes	Yes	Yes	9
Juurlink et al (2006)^Citation7	Yes	No	No	Yes	No	Yes	Yes	No	Yes	Unclear	Unclear	Yes	Yes	Yes	8
Kerchberger et al (2020)^Citation26	No	No	Yes	Yes	Yes	Yes	Yes	No	No	Unclear	Unclear	Yes	Yes	Yes	8
Kivimaki et al (2017)^Citation32	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Unclear	Yes	Yes	Yes	12
Klil-Drori et al (2019)^Citation33	No	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	12
Koola et al (2018)^Citation34	No	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	Unclear	No	Yes	Yes	Yes	10
Logan et al (2019)^Citation35	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Unclear	Yes	Yes	Yes	11
Lowenstern et al (2019)^Citation36	No	Yes	Yes	Yes	Unclear	Unclear	Yes	No	Yes	Yes	Unclear	Yes	Yes	Yes	8
Luhdorf et al (2017)^Citation37	Yes	No	Yes	Yes	Yes	No	Yes	Yes	Unclear	Yes	Unclear	Yes	Yes	Yes	10
Ohman et al (2018)^Citation38	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	13
Prat et al (2018)^Citation39	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	12
Psaty et al (2016)^Citation12	Yes	Yes	Yes	Yes	Yes	Unclear	Unclear	Yes	No	Unclear	Yes	Yes	Yes	Yes	10
Rebholz et al (2016)^Citation40	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	13
Ruigómez et al (2020)^Citation41	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	12
Sanfilippo et al (2015)^Citation42	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	13
Strom et al (2019)^Citation43	Yes	Yes	Unclear	Yes	Yes	Yes	Yes	Yes	No	No	No	Yes	Yes	Yes	10
Sundbøll et al (2016)^Citation8	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	13
Tu et al (2016)^Citation23	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	12
Van Walraven et al (2018)^Citation44	No	No	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	11
Varmdal et al (2015)^Citation45	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Unclear	Yes	Yes	Yes	12
Wei et al (2016)^Citation24	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Unclear	Yes	Yes	Yes	11
Xie et al (2018)^Citation46	Yes	Yes	Unclear	Yes	Yes	Yes	Yes	Yes	Yes	Unclear	Unclear	Yes	Yes	Yes	11
Youngson et al (2016)^Citation25	No	Yes	Unclear	Yes	Yes	Yes	Yes	Yes	No	Unclear	Unclear	Yes	Yes	Yes	9

Table 4 Characteristics of Selected Studies for Coagulopathy Outcomes

OUTCOME	AUTHOR (YEAR)	DATABASE (COUNTRY)	SETTING (HEALTHCARE SETTING & TYPE OF DATA SOURCE)	CODING DICTIONARY	ALGORITHM DEFINITION	GOLD STANDARD	PPV
DVT; PE	Al-Ani et al (2015)^Citation27	Canadian Institute for Health Information National Ambulatory Care Reporting System (Canada)	EHR data (secondary care)	ICD-10	Codes in any diagnostic field	Medical record review	DVT: 42%PE: 56%
DVT; PE; VT	Alotaibi et al (2015)^Citation28	University of Alberta Hospital radiology database (Canada)	EHR data (secondary care)	ICD-9-CM & ICD-10	Codes in any diagnostic field	Medical record review	DVT: 78%PE: 71%VT: 73%
PE	Amman et al (2018)^Citation29	Sentinel Distributed Database (USA)	Claims (secondary care)	ICD-9-CM	Codes in any diagnostic field	Medical record review	PE: 61%
PE; VT	Fang et al (2017)^Citation30	Cardiovascular Research Network Venous Thromboembolism study (USA)	EHR data (secondary care)	ICD-9	Codes in the primary diagnosis position in the inpatient setting; Code in any diagnostic field in the outpatient setting	Medical record review	PE: 89% (primary diagnosis); 28% (outpatient)VT: 79% (primary diagnosis); 31% (outpatient)
PE	Klil-Drori et al (2019)^Citation33	Ambulatory Care Database of the Alberta Health Services Calgary Zone (Canada)	EHR data (secondary care)	ICD-10	Codes in any diagnostic field	Medical record review	PE: 97%
DVT; PE; VT	Ohman et al (2018)^Citation38	Swedish National Patient Register (Sweden)	EHR data (secondary care)	ICD-9 & ICD-10	Codes in primary or secondary diagnosis position	Medical record review	DVT: 54%PE: 86%VT: 71%
PE	Prat et al (2019)^Citation39	Programme de Medicalisation des d’Information (France)	EHR data (secondary care)	ICD-10	Codes in the primary diagnosis position	Medical record review	PE: 99%
VT	Sanfilippo et al (2015)^Citation42	Veterans Affairs Central Cancer Registry (USA)	EHR data (secondary care)	ICD-9	Codes in any diagnostic field; Codes in any diagnostic field plus procedure codes for treatment or death; Codes in any diagnostic field plus procedure codes for treatment or death plus evidence of a diagnostic study	Medical record review	VT: 72% (only ICD codes); VT: 91% (ICD codes + treatment/death); VT: 92% (ICD codes + treatment/death + diagnostic study).
DVT; PE; VT	Sundbøll et al (2016)^Citation8	Danish National Patient Register (Denmark)	EHR data (secondary care)	ICD-8 & ICD-10	Codes in the primary or secondary diagnostic position	Medical record review	DVT: 86%PE: 90%VT: 88%
VT	Ruigómez et al (2020)^Citation41	The Health improvement network (UK)	EHR data (primary care)	Read codes	Coded entry for VT	Medical record review	VT: 40%

Table 5 Characteristics of Selected Studies for Cerebrovascular Outcomes

OUTCOME	AUTHOR (YEAR)	DATABASE (COUNTRY)	SETTING (HEALTHCARE SETTING & TYPE OF DATA SOURCE)	CODING DICTIONARY	ALGORITHM DEFINITION	GOLD STANDARD	PPV
Ischaemic stroke	Arana et al (2020)^Citation10	Clinical Practice Research Datalink (UK)	EHR data (primary care)	Read codes	Codes for ischaemic stroke	GP questionnaire	Ischaemic stroke: 95%
Stroke	Dalsgaard et al (2019)^Citation11	Danish National Patient Register (Denmark)	EHR data (secondary care)	ICD-10	Codes in a primary diagnosis position	Medical record review	Stroke: 70%
Stroke	Hall et al (2016)^Citation31	National Ambulatory Care Reporting System (Canada)	EHR data (secondary care)	ICD-10-CA	Codes in any diagnostic field	Registry data	Stroke: 69%
Stroke	Kivimaki et al (2017)^Citation32	Hospital Episode Statistics (UK)	EHR data (secondary care)	ICD-10	Codes in a primary or secondary diagnosis position	Registry data	Stroke: 72%
Stroke	Luhdorf et al (2017)^Citation37	Danish National Patient Register (Denmark)	EHR data (secondary care)	ICD-8 & ICD-10	Codes in a primary or secondary diagnosis position	Medical record review	Stoke: 70%
Stroke	Psaty et al (2016)^Citation12	Centre for Medicare and Medicaid Services (USA)	Claims (secondary care)	ICD-9 CM	Codes in the primary diagnosis position/ Codes in any diagnostic field	Registry data	Stroke (primary): 80%; Stroke (any): 44%
Ischaemic stroke; transient ischaemic attack	Strom et al (2019)^Citation43	Medicare Provider Analysis and Review (USA)	Claims (secondary care)	ICD-9 & ICD-10	Codes in any diagnostic field	Clinical trial data	Ischaemic stroke:99% (ICD-10), 87% (ICD-9); Transient ischaemic attack: 75% (ICD-10), 40% (ICD-9)
Stroke	Varmdal et al (2015)^Citation45	National stroke Register & Norwegian National Patient Register (Norway)	EHR data (secondary care)	ICD-10	Codes in any diagnosis position in the Stroke register; Codes in any diagnosis position in the Norwegian national patient register; Codes in the primary diagnosis fields in the Norwegian national patient register	Medical record review	Stroke (stroke registry): 99%; Stroke (any diagnosis): 94%; Stroke (main diagnosis): 80%
Stroke	Xie et al (2018)^Citation46	Medicare (USA)	Claims (secondary care)	ICD-9	Codes in any diagnostic field	Registry data	Stroke: 77%

Table 6 Characteristics of Selected Studies for Renal Conditions

OUTCOME	AUTHOR	DATABASE (COUNTRY)	SETTING (HEALTHCARE SETTING & TYPE OF DATA SOURCE)	CODING DICTIONARY	ALGORITHM DEFINITION	GOLD STANDARD	PPV
HRS	Koola et al (2018)^Citation34	Veterans Information Systems and Technology Architecture/Computerized Patient Record System (ViSTa/CPRS) (USA)	EHR data (All settings)	ICD-9	Code at discharge; Code at any time in the inpatient stage	Medical record review	HRS: 79% (at discharge); HRS: 76% (at any time)
AKI	Logan et al (2019)^Citation35	NHS Tayside (UK)	EHR data (All settings)	ICD-10	Code in any diagnosis position	Creatinine values	Not reported
Renal failure	Lowenstern et al (2019)^Citation36	Medicare (USA)	Claims (secondary care)	ICD-9	Code in any diagnosis position	GP criteria	Renal failure: 13%
Renal failure	Rebholz et al (2016)^Citation40	Atherosclerosis Risk in Communities (ARIC) Study. (USA)	EHR data (secondary care)	ICD-9	Code in any diagnosis position	Medical record review	Renal failure: 70%
Severe renal failure	van Walraven et al (2018)^Citation44	Not reported (USA)	EHR (not reported)	ICD-10	Code in any diagnosis position	Creatinine values	Renal failure: 60%

Sundbøll J, Adelborg K, Munch T, et al. Positive predictive value of cardiovascular diagnoses in the Danish National Patient Registry: a validation study. BMJ Open. 2016;6(11):e012832. doi:10.1136/bmjopen-2016-012832

 PubMed Web of Science ®Google Scholar

Ammann EM, Schweizer ML, Robinson JG, et al. Chart validation of inpatient ICD-9-CM administrative diagnosis codes for acute myocardial infarction (AMI) among intravenous immune globulin (IGIV) users in the Sentinel Distributed Database. Pharmacoepidemiol Drug Saf. 2018;27(4):398–404. doi:10.1002/pds.4398

 PubMed Web of Science ®Google Scholar

Arana A, Margulis AV, Varas-Lorenzo C, et al. Validation of cardiovascular outcomes and risk factors in the Clinical Practice Research Datalink in the United Kingdom. Pharmacoepidemiol Drug Saf. 2020;30(2):237–247. doi:10.1002/pds.5150

 PubMed Web of Science ®Google Scholar

Dalsgaard E-M, Witte DR, Charles M, Jørgensen ME, Lauritzen T, Sandbæk A. Validity of Danish register diagnoses of myocardial infarction and stroke against experts in people with screen-detected diabetes. BMC Public Health. 2019;19(1):228. doi:10.1186/s12889-019-6549-z

 PubMedGoogle Scholar

Psaty BM, Delaney JA, Arnold AM, et al. Study of Cardiovascular Health Outcomes in the Era of Claims Data. Circulation. 2016;133(2):156–164. doi:10.1161/circulationaha.115.018610

 PubMed Web of Science ®Google Scholar

Ashburner JM, Singer DE, Lubitz SA, Borowsky LH, Atlas SJ. Changes in use of anticoagulation in patients with atrial fibrillation within a primary care network associated with the introduction of direct oral anticoagulants. Am J Cardiol. 2017;120(5):786–791. doi:10.1016/j.amjcard.2017.05.055

 PubMed Web of Science ®Google Scholar

Brouwer ES, Napravnik S, Eron JJ, et al. Validation of medicaid claims-based diagnosis of myocardial infarction using an HIV clinical cohort. Med Care. 2015;53(6):e41–8. doi:10.1097/MLR.0b013e318287d6fd

 PubMed Web of Science ®Google Scholar

Bush M, Sturmer T, Stearns SC, et al. Position matters: validation of medicare hospital claims for myocardial infarction against medical record review in the atherosclerosis risk in communities study. Pharmacoepidemiol Drug Saf. 2018;27(10):1085–1091. doi:10.1002/pds.4396

 PubMed Web of Science ®Google Scholar

Colantonio LD, Levitan EB, Yun H, et al. Use of medicare claims data for the identification of myocardial infarction: the reasons for geographic and racial differences in stroke study. Med Care. 2018;56(12):1051–1059. doi:10.1097/MLR.0000000000001004

 PubMed Web of Science ®Google Scholar

Cozzolino F, Montedori A, Abraha I, et al. A diagnostic accuracy study validating cardiovascular ICD-9-CM codes in healthcare administrative databases. The Umbria Data-Value Project. PLoS One. 2019;14(7):e0218919. doi:10.1371/journal.pone.0218919

 PubMed Web of Science ®Google Scholar

Di Chiara A, Clagnan E, Valent F. Epidemiology and mortality in an Italian region after the adoption of the universal definition of myocardial infarction. Comparative study validation study. J Cardiovasc Med. 2020;21(1):34–39. doi:10.2459/JCM.0000000000000893

 Google Scholar

Ding EY, Albuquerque D, Winter M, et al. Novel method of atrial fibrillation case identification and burden estimation using the MIMIC-III electronic health data set. J Intensive Care Med. 2019;34(10):851–857. doi:10.1177/0885066619866172

 PubMed Web of Science ®Google Scholar

Floyd JS, Blondon M, Moore KP, Boyko EJ, Smith NL. Validation of methods for assessing cardiovascular disease using electronic health data in a cohort of Veterans with diabetes. Pharmacoepidemiol Drug Saf. 2016;25(4):467–471. doi:10.1002/pds.3921

 PubMed Web of Science ®Google Scholar

Govatsmark RE, Janszky I, Slordahl SA, et al. Completeness and correctness of myocardial infarction diagnoses in a medical quality register and an administrative health register. Eur Heart J. 2017;38(Supplement 1):1026. doi:10.1093/eurheartj/ehx502.P4894

 Google Scholar

Tu K, Nieuwlaat R, Cheng SY, et al. Identifying Patients With Atrial Fibrillation in Administrative Data. Can J Cardiol. 2016;32(12):1561–1565. doi:10.1016/j.cjca.2016.06.006

 PubMed Web of Science ®Google Scholar

Wei W-Q, Teixeira PL, Mo H, Cronin RM, Warner JL, Denny JC. Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance. J Am Med Inform Assoc. 2016;23(e1):e20–e27. doi:10.1093/jamia/ocv130

 PubMed Web of Science ®Google Scholar

Youngson E, Welsh RC, Kaul P, McAlister F, Quan H, Bakal J. Defining and validating comorbidities and procedures in ICD-10 health data in ST-elevation myocardial infarction patients. Medicine. 2016;95(32):e4554. doi:10.1097/md.0000000000004554

 PubMed Web of Science ®Google Scholar

Kerchberger VE, Bastarache JA, Shaver CM, et al. Chest radiograph interpretation is critical for identifying acute respiratory distress syndrome patients from electronic health record data. Conference Abstract. American Journal of Respiratory and Critical Care Medicine Conference: American Thoracic Society International Conference, ATS; 2020;201(1).

 Google Scholar

Juurlink D, Preyra C, Croxford R, et al. Canadian Institute for Health InformationDischarge Abstract Database: a Validation Study; 2006. Available from: https://www.ices.on.ca/publications/research-reports/canadian-institute-for-health-information-discharge-abstract-database-a-validation-study/. Accessed 14 December 2023.

 Google Scholar

Al-Ani F, Shariff S, Siqueira L, Seyam A, Lazo-Langner A. Identifying venous thromboembolism and major bleeding in emergency room discharges using administrative data. Thromb Res. 2015;136(6):1195–1198. doi:10.1016/j.thromres.2015.10.035

 PubMed Web of Science ®Google Scholar

Alotaibi GS, Wu C, Senthilselvan A, McMurtry MS. The validity of ICD codes coupled with imaging procedure codes for identifying acute venous thromboembolism using administrative data. Vasc Med. 2015;20(4):364–368. doi:10.1177/1358863X15573839

 PubMed Web of Science ®Google Scholar

Ammann EM, Cuker A, Carnahan RM, et al. Chart validation of inpatient International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) administrative diagnosis codes for venous thromboembolism (VTE) among intravenous immune globulin (IGIV) users in the Sentinel Distributed Database. Medicine. 2018;97(8):e9960. doi:10.1097/MD.0000000000009960

 PubMed Web of Science ®Google Scholar

Fang MC, Fan D, Sung SH, et al. Validity of using inpatient and outpatient administrative codes to identify acute venous thromboembolism: the CVRN VTE study. Med Care. 2017;55(12):e137–e143. doi:10.1097/MLR.0000000000000524

 PubMed Web of Science ®Google Scholar

Hall R, Mondor L, Porter J, Fang J, Kapral MK. Accuracy of administrative data for the coding of acute stroke and TIAs. Can J Neurol Sci. 2016;43(6):765–773. doi:10.1017/cjn.2016.278

 PubMed Web of Science ®Google Scholar

Kivimaki M, Batty GD, Singh-Manoux A, Britton A, Brunner EJ, Shipley MJ. Validity of cardiovascular disease event ascertainment using linkage to UK hospital records. Epidemiology. 2017;28(5):735–739. doi:10.1097/EDE.0000000000000688

 PubMed Web of Science ®Google Scholar

Klil-Drori AJ, Prajapati D, Liang Z, et al. External validation of ASPECT (Algorithm for Suspected Pulmonary Embolism Confirmation and Treatment). Med Care. 2019;57(8):e47–e52. doi:10.1097/MLR.0000000000001055

 PubMed Web of Science ®Google Scholar

Koola JD, Davis SE, Al-Nimri O, et al. Development of an automated phenotyping algorithm for hepatorenal syndrome. J Biomed Inform. 2018;80:87–95. doi:10.1016/j.jbi.2018.03.001

 PubMed Web of Science ®Google Scholar

Logan R, Davey P, De Souza N, Baird D, Guthrie B, Bell S. Assessing the accuracy of ICD-10 coding for measuring rates of and mortality from acute kidney injury and the impact of electronic alerts: an observational cohort study. Clin Kidney J. 2020;13(6):1083–1090. doi:10.1093/ckj/sfz117

 PubMed Web of Science ®Google Scholar

Lowenstern A, Lippmann SJ, Brennan JM, et al. Use of medicare claims to identify adverse clinical outcomes after mitral valve repair. Circ Cardiovasc Interv. 2019;12(5):e007451. doi:10.1161/CIRCINTERVENTIONS.118.007451

 PubMed Web of Science ®Google Scholar

Luhdorf P, Overvad K, Schmidt EB, Johnsen SP, Bach FW. Predictive value of stroke discharge diagnoses in the Danish National Patient Register. Scand J Public Health. 2017;45(6):630–636. doi:10.1177/1403494817716582

 PubMed Web of Science ®Google Scholar

Ohman L, Johansson M, Jansson JH, Lind M, Johansson L. Positive predictive value and misclassification of diagnosis of pulmonary embolism and deep vein thrombosis in Swedish patient registries. Clin Epidemiol. 2018;10:1215–1221. doi:10.2147/CLEP.S177058

 PubMed Web of Science ®Google Scholar

Prat M, Derumeaux H, Sailler L, Lapeyre-Mestre M, Moulis G. Positive predictive values of peripheral arterial and venous thrombosis codes in French hospital database. Fundament Clinic Pharmacol. 2018;32(1):108–113. doi:10.1111/fcp.12326

 PubMed Web of Science ®Google Scholar

Rebholz CM, Coresh J, Ballew SH, et al. Kidney Failure and ESRD in the Atherosclerosis Risk in Communities (ARIC) Study: comparing ascertainment of treated and untreated kidney failure in a cohort study. Am J Kidney Dis. 2015;66(2):231–239. doi:10.1053/j.ajkd.2015.01.016

 PubMed Web of Science ®Google Scholar

Ruigómez A, Brobert G, Vora P, García Rodríguez LA. Validation of venous thromboembolism diagnoses in patients receiving rivaroxaban or warfarin in the health improvement network. Pharmacoepidemiol Drug Saf. 2020. doi:10.1002/pds.5146

 Web of Science ®Google Scholar

Sanfilippo KM, Wang T-F, Gage BF, Liu W, Carson KR. Improving accuracy of international classification of diseases codes for venous thromboembolism in administrative data. Thromb Res. 2015;135(4):616–620. doi:10.1016/j.thromres.2015.01.012

 PubMed Web of Science ®Google Scholar

Strom JB, Zhao Y, Faridi KF, et al. Comparison of clinical trials and administrative claims to identify stroke among patients undergoing aortic valve replacement. Circ Cardiovasc Interv. 2019;12(11):e008231. doi:10.1161/circinterventions.119.008231

 PubMed Web of Science ®Google Scholar

van Walraven C Improved correction of misclassification bias with bootstrap imputation. Med Care. 2018;56(7):e39–e45.

 PubMed Web of Science ®Google Scholar

Varmdal T, Bakken IJ, Janszky I, et al. Comparison of the validity of stroke diagnoses in a medical quality register and an administrative health register. Scand J Public Health. 2016;44(2):143–149. doi:10.1177/1403494815621641

 PubMed Web of Science ®Google Scholar

Xie F, Colantonio LD, Curtis JR, et al. Development of algorithms for identifying fatal cardiovascular disease in Medicare claims. Pharmacoepidemiol Drug Saf. 2018;27(7):740–750. doi:10.1002/pds.4421

 PubMed Web of Science ®Google Scholar