Validation of a Diagnostic Model to Differentiate Multiple Myeloma from Bone Metastasis

Abstract

Purpose

A diagnostic model to differentiate multiple myeloma (MM) from bone metastasis (BM) in patients with destructive bone lesions (MM-BM DDx) was developed to promote timely and appropriate referral of patients with MM to hematologists. External validation has never been conducted. This study aims to externally validate the performance of the MM-BM DDx model.

Patients and Methods

This multi-center external validation study was conducted using retrospective data of patients over 45 years old diagnosed with MM or BM at six university-affiliated hospitals in Thailand from 2016 to 2022. The MM-BM DDx development dataset, including patients from 2012 to 2015, was utilized during external validation. Diagnostic indicators for MM included in the MM-BM DDx model are serum creatinine, serum globulin, and serum alkaline phosphatase (ALP). MM and BM diagnosis was based on the documented International Classification of Diseases 10th Revision codes. Model performance was evaluated in terms of discrimination, calibration, and accuracy.

Results

A total of 3018 patients were included in the validation dataset (586 with MM and 2432 with BM). Clinical characteristics were similar between the validation and development datasets. The MM-BM DDx model’s predictions showed an AUC of 0.89 (95% CI, 0.87, 0.90). The predicted probabilities of MM from the model increased concordantly with the observed proportion of MM within the validation dataset. The estimated sensitivity, specificity, and LR for each odds class in the validation dataset were similar to those of the development dataset.

Conclusion

The discriminative ability and calibration of the MM-BM DDx model were found to be preserved during external validation. These findings provide support for the practical use of the MM-BM DDx model to assist clinicians in identifying patients with destructive bone lesions who are likely to have MM and enable them to arrange timely referrals for further evaluation by hematologists.

Keywords:

• diagnosis
• primary care
• clinical prediction model
• referral
• bone metastases
• multiple myeloma

Data Sharing Statement

Supplementary information accompanies this paper can be found in Supplementary Materials.

Ethical Approval

The ethical approval was exempted by the Ethical Committee from all contributing centers including six university-affiliated hospitals and medical schools participated in this study owing to the retrospective nature of deidentified data collection: Faculty of Medicine, Chiang Mai University, Khon Kaen University, Siriraj Hospital at Mahidol University, Queen Savang Vadhana Memorial Hospital, Khon Kaen Hospital, and Chulabhorn Hospital. All collected data were kept confidential and accessible to the investigators.

Acknowledgments

This research was partially supported by Chiang Mai University and Faculty of Medicine, Chiang Mai University.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

Additional information

Funding

This research was supported by Chiang Mai University, and Faculty of Medicine, Chiang Mai University, grant no 114-2564.