Evaluation of magnetic resonance imaging to detect cholesteatoma in clinical healthcare

Abstract

Background

Non-Echo-Planar Diffusion Weighed Magnetic Resonance Imaging (non-EPI DW MRI) plays a role in the planning of cholesteatoma surgery.

Objectives

To outline the usage of Non-EPI DW MRI in cholesteatoma diagnostics and to determine its accuracy, using otosurgery diagnostics as reference standard.

Material and Methods

A retrospective descriptive study. All subjects operated, with suspicion of cholesteatoma as indication, that had a preceding examination with non-EPI DW MRI, between October 2010 and March 2019. Calculating sensitivity, specificity, predicative values, and likelihood ratios, using non-EPI DW MRI as index test and diagnosis from otosurgery as reference standard.

Results

Fifty-two subject episodes were included. Non-EPI DW MRI had a sensitivity of 0.50, specificity of 0.75, positive and negative predictive values of 0.74 and 0.52 and, positive and negative likelihood ratios of 2.0 and 0.67 respectively.

Conclusions and Significance

There is a clear annual trend with increased numbers of executed examinations during the study period. The diagnostic accuracy of non-EPI DW MRI does not reach acceptable levels in the existing everyday routine practice conditions. The accuracy of the examination increases when interpreted by an experienced radiologist and when using the definition of cholesteatoma recommended by EAONO/JOS.

Chinese Abstract

背景：非回波平面扩散加权磁共振成像（non-EPI DW MRI）在胆脂瘤手术计划中发挥作用。

目的：概述非-EPI DW MRI 在胆脂瘤诊断中的应用并确定其准确性, 使用耳外科诊断作为参考标准。

材料和方法：这是一项回顾性描述性研究。 所有接受手术的受试者, 疑患胆脂瘤作为指征, 在2010 年10 月和 2019 年 3 月期间进行过非-EPI DW MRI 的前期检查。计算敏感性、特异性、预测值和似然比, 将non-EPI DW MRI作为指标测试, 以耳外科诊断为参考标准。

结果：包含 52 个受试者测试事件。 非-EPI DW MRI 的敏感性为 0.50, 特异性0.75, 阳性和阴性预测值分别为 0.74 和 0.52, 阳性和阴性似然比分别为2.0和0.67。

结论与意义：在研究期间, 每年进行的检查数量呈明显上升趋势。 非EPI DW MRI的诊断准确率达不到现有日常行医条件下可接受的水平。当由经验丰富的放射科医生作解释时, 以及根据EAONO/JOS 的建议使用胆脂瘤的定义时, 检查的准确性会增加。

Keywords:

• Cholesteatoma
• non-epi diffusion weighted imaging
• magnetic resonance imaging
• data accuracy
• sensitivity and specificity
• ear
• middle
• temporal bone

Introduction

Cholesteatoma is a chronic middle ear disease that affects patients’ quality of life [1] as well as the health-care system [2]. In most cases otomicroscopy is diagnostic and otosurgery the treatment [3]. Otosurgery has however, particularly regarding residual or recurrent disease, sometimes been the only way to confirm a diagnosis. A second look surgery has been common to exclude residual/recurrent cholesteatoma [3], but has also been used to reconstruct the ossicular chain. Lately it has been more common in Sweden to do the whole reconstruction, inclusive the auditory part during the primary surgery. This procedure has increased the need of diagnostic tools to exclude residual and recurrent disease without further operations. The possibility to use non-echo-planar diffusion weighted magnetic resonance image (non-EPI DW MRI) in cholesteatoma diagnostics was first reported in 2006 [4]. An increasing ability to detect both primary and recurrent cholesteatoma with non-EPI DW MRI have been reported [5,6]. The use, and accuracy in a clinical healthcare situation for non-EPI DW MRI in Sweden has not previously been evaluated. The aim of the present study was to outline the usage of non-EPI DW MRI for cholesteatoma diagnostics in northern Sweden, and to evaluate its accuracy in routine practice conditions.The main hypothesis was that non-EPI DW MRI, has a high statistical methodological reliability.

Materials and methods

A retrospective descriptive and diagnostic study conducted in accordance with the STARD guidelines [7] where non-EPI DW MRI was used as index test and diagnosis from otosurgery as reference standard. All subjects operated, with suspicion of cholesteatoma as indication, that had a preceding examination with non-EPI DW MRI, between October 2010 and March 2019, were included (Figure 1). Data collection was performed in 6 secondary to tertiary care hospitals in 2 regions in northern Sweden. A retrospective diagnostic search was performed in the radiological PACS-system with additional data on demographics from medical records (IBS, Cross). Clinical information, before otosurgery, was available to the radiologists. The primary radiological interpretation was performed by 44 radiologists with experience ranging from resident physician to consultant. In case of incomplete radiological dictums or disparity between radiological statements and surgical reference diagnosis, the examinations were reviewed by a blinded radiologist experienced in MRI and head and neck radiology. A specialist in otorhinolaryngology reviewed the medical records and followed The European Academy of Otology and Neurotology and The Japanese Otological Society (EAONO/JOS) definition on cholesteatoma and retraction pockets [8]. A contingency table was constructed (Table 1) and data calculations of sensitivity, specificity, predictive values and, likelihood ratios with 95% confidence intervals were performed in SPSS version 26 (IBM Corp. in Armonk, NY). All other data were analyzed with Microsoft Excel 2019 (Microsoft Corporation, Redmond, WA). The study was approved by the Regional Ethical Review Board, Umeå, Sweden (Approval number: 2018-392-31 M).

Figure 1. Flow chart of all subject episodes grouped according to the index test and reference standard. Group names are marked in bold.

Table 1. Contingency table with otosurgery as the reference standard and non-EPI DW MRI as the index test.

	Otosurgery
Non-EPI DW MRI	Cholesteatoma	Not Cholesteatoma	Total
Cholesteatoma	14	5	19
Not cholesteatoma	14	15	29
Equivocal	3	1	4
Total	31	21	52

Data presented are number of cases. Non-EPI DW MRI: Non-Echo Planar Imagining Diffusion Weighed Magnetic Resonance Imaging.

Results

Forty-two subjects were included whereof some had performed bilateral and/or repeated non-EPI DW-MRI and operations, thus resulting in 52 included subject episodes.

In 33 out of 52 of these subject episodes, a cholesteatoma was diagnosed intraoperatively.

There was a clear annual increase of executed non-EPI DW MRI between 2010 and 2019 (Figure 2). In 19 subject episodes the non-EPI DW MRI showed cholesteatoma, with oto-surgical confirmation of diagnose in 14 cases (true positive). Two out of these 14 subject episodes were referred to as retraction pockets after review according to the EAONO/JOS definition on cholesteatoma. In 5 subject episodes, otosurgery did not confirm cholesteatoma (false positive) (Figure 1). In 29 subject episodes the non-EPI DW MRI showed no cholesteatoma and in 15 of these otosurgery was also negative (true negative). In the remaining 14 subject episodes cholesteatomas were confirmed by otosurgery (false negative), whereas 5 subject episodes were referred as retraction pockets (Figure 1, Tables 1 and 2). Four subject episodes were regarded as equivocal according to the non-EPI DW MRI and 3 of these were diagnosed as cholesteatomas by otosurgery (Figure 1, Tables 1 and 2). The mean time between non-EPI DW MRI and otosurgery was 3.3 months (SD 5.2) for the true positive group, and 22.7 months (SD 32.2) for the false positive group. Corresponding mean times for the true negative, and false negative groups were 9.2 (SD 18.1) and 12.9 (SD 16.0) months, respectively (Figure 3). When a blinded, experienced ENT radiologist reinterpreted the non-EPI DW MRI examinations in the false positive, false negative, and equivocal groups eight out of 33 subject episodes obtained a new classification, rendering an increased accuracy (Table 3). Statistical measure of accuracy for the primary radiological interpretation, equivocal dictum included as cholesteatoma, recurrent/residual cholesteatomas separated, re-examined non-EPI DW MRI, and re-diagnosing according to EAONO/JOS staging system respectively are presented in Table 3.

Figure 2. Total number of executed non-EPI DW MRI per year separated into each radiological diagnosis. non- EPI DW MRI: Non-Echo Planar Imagining Diffusion Weighed Magnetic Resonance Imaging.

Figure 3. Time to otosurgery from the date of non-EPI DW MRI for all operated groups. non- EPI DW MRI: Non-Echo Planar Imagining Diffusion Weighed Magnetic Resonance Imaging.

Table 2. Sex, type of disease and follow-up for all operated groups.

	Sex	Cholesteatoma		Follow-up
Group	Male/female (n)	Primary (n)	Residual/Recurrent (n)	Still clinically monitored.(n)	Discharged from follow up. (n)	Not conducted as planned. (n)	Unknown (n)
True positive n = 14	12/2	6	8	9	3	2	0
False positive n = 5	3/2	–	–	2	3	0	0
True negative n = 15	9/6	–	–	8	5	1	1
False negative n = 14	10/4	7	7	12	1	1	0
Equivocal- cholesteatoma n = 3	1/2	2	1	3	0	0	0
Equivocal-Not cholesteatoma n = 1	1/0	–	–	1	0	0	0

Table 3. Sensitivity, specificity, predictive values, and likelihood ratios for non-EPI DW MRI to detect cholesteatoma.

	Sensitivity	Specificity	PPV	NPV	LR+	LR-
Primary radiological interpretation	0.50 (0.32, 0.68)	0.75 (0.54, 0.90)	0.74 (0.52, 0.90)	0.52 (0.34, 0.69)	2.00 (0.86, 4.65)	0.67 (0.43, 1.04)
Primary	0.46 (0.22, 0.72)	0.75 (0.54, 0.90)	0.55 (0.27, 0.81)	0.68 (0.48, 0.85)	1.85 (0.71, 4.82)	0.72 (0.41, 1.26)
Recurrent/ Residual	0.53 (0.29, 0.77)	0.75 (0.54, 0.90)	0.62 (0.35, 0.84)	0.68 (0.48, 0.85)	2.13 (0.87, 5.22)	0.62 (0.34, 1.13)
Re-examined	0.66 (0.47, 0.81)	0.80 (0.60, 0.93)	0.83 (0.64, 0.94)	0.62 (0.42, 0.79)	3.28 (1.31, 8.18)	0.43 (0.25, 0.75)
EAONO/JOS	0.57 (0.36, 0.77)	0.74 (0.56, 0.88)	0.63 (0.41, 0.82)	0.69 (0.51, 0.84)	2.20 (1.05, 4.61)	0.58 (0.34, 1.00)
Re-examined + EAONO/JOS	0.74 (0.54, 0.89)	0.77 (0.59, 0.90)	0.74 (0.54, 0.89)	0.77 (0.59, 0.90)	3.20 (1.52, 6.73)	0.34 (0.17, 0.70)

non-EPI DW MRI: Diffusion Weighed Magnetic Resonance Imaging; PPV: positive predictive value; NPV: negative predictive value; LR+: Positive likelihood ratio; LR-: Negative likelihood ratio. 95% confidence intervals are in parentheses.

Discussion

In the present study, we show that the accuracy for non-EPI DW MRI to detect cholesteatoma is not sufficient, but can substantially be improved when using the definition of cholesteatoma recommended by EAONO/JOS [8], and when interpreted by an experienced radiologist. The number of annually executed non-EPI DW MRI has increased significantly over the study period, indicating a need of diagnostic tools for cholesteatoma. When retraction pockets are considered as not cholesteatomas and false results re-examined, sensitivity was improved from 0.50 to 0.74, specificity from 0.75 to 0.77, LR + from 2.0 to 3.2 and LR- from 0.67 to 0.34 compared to the original numbers. A meta-analysis by Lingam et al. with 1152 patient episodes showed diagnostic performance of non-EPI DW MRI in detecting primary and recurrent/residual cholesteatoma with a pooled sensitivity of 0.91 (95% CI: 0.87–0.95) and specificity of 0.92 (95% CI: 0.86–0.96) [9]. Most published studies show results in line with the meta-analysis [9], but there are exceptions. Three previous prospective studies comparing non-EPI DW MRI and oto-surgery in detecting cholesteatomas, reported a sensitivity ranging from 0.43 to 0.62 [10–12]. Although these are prospective studies with experienced radiologists, Kasbekar et al. found a sensitivity lower than our original values. The authors discussed that the use of propeller sequences, with 3 mm slices, might give poorer results [10]. Other explanations for divergent outcome might be large proportion of small cholesteatomas [12] and DW MRI executed shortly after the primary surgery [11]. To the best of our knowledge, there is no consensus on what sensitivity is needed for non-EPI DW MRI to be considered safe to detect cholesteatoma in routine clinical healthcare. A high sensitivity is clinically more desirable than a high specificity since this will reduce the likelihood of cholesteatomas being missed and development of late complications. We suggest that a sensitivity above 0.9 is desirable for non-EPI DW MRI to detect cholesteatoma in clinical care. The positive likelihood ratio is an estimate of how much a non-EPI DW MRI result will change the odds of having a cholesteatoma. Good diagnostic tests have a LR + above 10 and LR- under 0.1 [13], our results do not reach this level.

In 2017, EAONO/JOS made a joint consensus statement on the definition, classification and staging of middle ear cholesteatoma. The document includes the statement that a retraction pocket is not a cholesteatoma [8]. The subject of retraction pocket has been discussed among otosurgeons over a long period, and our study shows that not all otosurgeons have adopted the definition of cholesteatoma from EAONO/JOS. In previous studies, retraction pockets have been described to cause false negative subject episodes [14,15]. When following the EAONO/JOS consensus statement and not considering retractions pocket as cholesteatoma [8], the accuracy in our study improved.

Non-EPI DW MRI has begun to replace second look surgery [16–18], and there are reports suggesting further expansion of the indications regarding early detection of residual cholesteatoma [19], as well as to complete the follow-up with annual non-EPI DW MRI at least five years after surgery to detect recurrent cholesteatoma [20]. Since non-EPI DW MRI is increasingly used it is crucial that the accuracy of the examination to diagnose cholesteatoma in routine practice conditions is high.

To the authors’ knowledge, there are no other studies that have looked at the use of non-EPI DW MRI for cholesteatoma in Sweden. In our study, the non-EPI DW MRI has been used preoperatively to confirm primary cholesteatoma in some cases, which may be assumed to be due to difficulties in establishing a diagnosis solely via otomicroscopy. In a meta-analysis, it was concluded that non-EPI DWI MRI may have a role in the diagnosis of primary cholesteatoma where the diagnosis may be difficult to make based on clinical examination alone [9]. Computed tomography (CT) of the temporal bone is often used preoperatively in cholesteatoma cases for preoperative planning, but are poor at differentiating cholesteatoma tissue [21]. For the future, fusion imaging between CT and MRI may further assist in operative planning [22,23].

We show that diagnostic accuracy of non-EPI DW MRI does not reach acceptable levels to diagnose cholesteatoma. However, when interpreted by an experienced radiologist, and an internationally diagnostic consensus is used, accuracy improves substantially.

Methodological considerations

Our study assesses diagnostic accuracy under routine practice conditions, identifying two enhancing factors: experienced radiologists and the application of diagnostic consensus. The retrospective nature of the study implicates limitations with missing data, though otosurgery exclusively can be performed based on clinical indications.

In contrast to many studies, our focused on primary radiological interpretation. Results indicate that involving a blinded, experienced radiologist in re-evaluating false subject episodes significantly improves diagnostic accuracy. Unlike previous studies where radiologists were selectively chosen for long-term experience [24–26], our approach increases external validity by reflecting real-world variations in routine practice conditions. While observer bias is a potential risk in retrospective studies, efforts to mitigate this included blinded radiologist image reviews and data extraction by individuals unconnected to operations and patients. Our retrospective study, with its inherent reflection of reality, offer higher external validity compared to earlier investigations.

Acknowledgments

We would like to gratefully acknowledge contributions from Lennart Flygare, with review of radiological images.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available from the corresponding author, [EK], upon reasonable request.

Additional information

Funding

EK is currently receiving funding [LVNFOU92905] from Region Västernorrland/Kempe Foundation, MW have received funding [FS 2.1.6-2408-18; KV00-1.3-20-151] from Umeå University. Sponsorships: None.