https://orcid.org/0000-0002-9901-8141
![Sample our Bioscience journals, sign in here to start your access, Latest two full volumes FREE to you for 14 days]({"type":"image" , "src" : "/sda/5925/TOC-Subject-Sample-2021-Bioscience.jpg"})
Abstract
Aim: Evaluation of the performance of ChatGPT-4.0 in providing prediagnosis and treatment plans for cardiac clinical cases by expert cardiologists. Methods: 20 cardiology clinical cases developed by experienced cardiologists were divided into two groups according to preparation methods. Cases were reviewed and analyzed by the ChatGPT-4.0 program, and analyses of ChatGPT were then sent to cardiologists. Eighteen expert cardiologists evaluated the quality of ChatGPT-4.0 responses using Likert and Global quality scales. Results: Physicians rated case difficulty (median 2.00), revealing high ChatGPT-4.0 agreement to differential diagnoses (median 5.00). Management plans received a median score of 4, indicating good quality. Regardless of the difficulty of the cases, ChatGPT-4.0 showed similar performance in differential diagnosis (p: 0.256) and treatment plans (p: 0.951). Conclusion: ChatGPT-4.0 excels at delivering accurate management and demonstrates its potential as a valuable clinical decision support tool in cardiology.
Plain language summary
Have you ever wondered if an artificial intelligence (AI) program could help doctors figure out what the problem is when someone has heart complaints? Our research examined this by testing an AI program called ChatGPT-4.0 on clinical cases. We wanted to see if it could help doctors by giving good advice on what might be wrong with patients who have heart issues and what should be done to help them. To test this, we used ChatGPT-4.0 to look at 20 different stories about patients with heart problems. These stories were made to cover a variety of common heart conditions faced by heart doctors. Then, we asked 18 heart doctors to check if the advice from ChatGPT-4.0 was good and made sense. What we found was quite interesting! Most of the time, the doctors agreed that the computer gave good advice on what might be wrong with the patients and how to help them. This means that this smart computer program could be a helpful tool for doctors, especially when they are trying to figure out tricky heart problems. But, it's important to say that computers like ChatGPT-4.0 are not ready to replace doctors. They are tools that can offer suggestions. Doctors still need to use their knowledge and experience to make the final call on what's best for their patients. In simple terms, our study shows that with more development and testing, AI like ChatGPT-4.0 could be a helpful assistant to doctors in treating heart disease, making sure patients get the best care possible.
TWEETABLE ABSTRACT
ChatGPT-4.0 impresses expert cardiologists with accurate diagnoses and management plans in cardiology cases. Consistent quality across difficulty levels highlights its potential as a valuable clinical decision support tool. #AIinCardiology
The aim of this study is to evaluate the performance of ChatGPT-4.0, an artificial intelligence-supported decision-support tool, by expert cardiologists in providing preliminary diagnosis and management plans for cardiac clinical cases.
ChatGPT-4.0 was used to provide preliminary diagnosis and management plans for a series of 20 cardiology clinical case scenarios developed by experienced cardiologists.
Clinical cases were divided into two groups according to the way they were prepared. Eighteen expert cardiologists evaluated the fit and quality of ChatGPT-4.0's responses.
By physicians: For 20 clinical cases, difficulty was rated on an overall easy-to-moderate scale (median 2.00, IQR 1).
ChatGPT-4.0 had a high physician adherence rate to its differential diagnoses (median 5.00, IQR 1). ChatGPT-4.0's management plan received a median score of 4 (IQR 1), indicating a good quality of response as perceived by the physicians.
The difficulty of cases as assessed by cardiology experts averaged 2.23 ± 1.04 for Group 1 and 2.63 ± 1.1 for Group 2 (p < 0.001). The performance of ChatGPT-4.0 in providing differential diagnosis and management plans was similar between the two groups (p :0.256; p :0.951, respectively).
ChatGPT-4.0 successfully showed accurate preliminary diagnoses in cardiology, according to the evaluations of expert cardiologists, indicating its potential as a clinical decision support tool.
ChatGPT-4.0 successfully navigated complex cardiac cases, providing evidence-based treatment options that align with current guidelines, demonstrating its capability for real-time educational support in medical training.
This study highlights the integration of AI with healthcare, suggesting significant advancements in AI-driven clinical practice and decision-making in cardiology.
Supplemental material
Supplemental data for this article can be accessed at https://doi.org/10.1080/14796678.2024.2348898
Author contributions
Author Y Emre Yavuz was responsible for study conception and design; authors Y Emre Yavuz and F Kahraman. were responsible for acquisition of data, data analysis and drafting and revision of the manuscript. All authors read and approved the final manuscript.
Financial disclosure
The authors have no financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.
Competing interests disclosure
The authors have no competing interests or relevant affiliations with any organization or entity with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.
Writing disclosure
No writing assistance was utilized in the production of this manuscript.
Ethical conduct of research
This is an observational study that falls under the category of cross-sectional research. The study did not involve patient participants or use data from patients or hospitals. The study used responses from ChatGPT, which are publicly available. Hence, there was no need to get approval from an institutional review board (IRB), which is typically required for research involving human subjects or private data. The research was conducted in accordance with the principles of the Declaration of Helsinki.
Data sharing statement
The authors acknowledge that this manuscript presents original data from clinical trials. Raw data may be made available upon reasonable request via email [email protected] at any time required.