Ankita Jawanpuria
Aruna Rani Behera
Chinmaya Dash
Mohammad Hifz Ur Rahman
ChatGPT in hospital infection prevention and control – assessing knowledge of an AI model based on a validated questionnaire
Ankita Jawanpuria
Aruna Rani Behera
Chinmaya Dash
Mohammad Hifz Ur Rahman
Department of Microbiology, Manipal Tata Medical College, Manipal Academy of Higher Education, Manipal, India
Department of Microbiology, Manipal Tata Medical College, Manipal Academy of Higher Education, Manipal, India
Department of Microbiology, Manipal Tata Medical College, Manipal Academy of Higher Education, Manipal, India
Department of Community Medicine, Manipal Tata Medical College, Manipal Academy of Higher Education, Manipal, India
Received: 13.12.2023 / Revised: 1.02.2024 / Accepted: 5.02.2024 / Published: 30.06.2024
Abstract
Introduction and aim. An AI model like ChatGPT is a good source of knowledge. We can explore the potential of AI models to complement the expertise of healthcare professionals by providing real-time, evidence-based information in infection prevention and control (IPC).
Material and methods. This study involved 110 queries related to IPC, validated by subject experts in IPC. The responses from ChatGPT were evaluated using Bloom’s taxonomy by experienced microbiologists. The scores were divided as 4 as a good response. Statistical analysis was done by correlation coefficient and Cohen’s Kappa.
Results. The overall score was 4.33 (95% CI, q1 3.65–q3 4.64) indicating ChatGPT’s substantial IPC knowledge. A good response (i.e.>4 score) was found in 70 (63.6%) questions, while in 10 (9%) questions, it showed a poor response. The poor response was seen in needle stick injury and personal protective equipment (PPE) doffing-related questions. The overall correlations were found to be significant. Cohen’s Kappa confirmed moderate to substantial agreement between evaluators.
Conclusion. ChatGPT demonstrated a commendable understanding of IPC principles in various domains and the study identifies specific instances where the model may require further refinement especially in critical scenarios such as needlestick injuries and PPE doffing.
Cite
Jawanpuria A, Behera AR, Dash C, Rahman MHU. ChatGPT in hospital infection prevention and control – assessing knowledge of an AI model based on a validated questionnaire. Eur J Clin Exp Med. 2024;22(2):347–352. doi: 10.15584/ejcem.2024.2.19.
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited