TY - JOUR
T1 - Exploring the role of artificial intelligence, large language models
T2 - Comparing patient-focused information and clinical decision support capabilities to the gynecologic oncology guidelines
AU - Reicher, Lee
AU - Lutsker, Guy
AU - Michaan, Nadav
AU - Grisaru, Dan
AU - Laskov, Ido
N1 - Publisher Copyright:
© 2024 The Author(s). International Journal of Gynecology & Obstetrics published by John Wiley & Sons Ltd on behalf of International Federation of Gynecology and Obstetrics.
PY - 2025/2
Y1 - 2025/2
N2 - Gynecologic cancer requires personalized care to improve outcomes. Large language models (LLMs) hold the potential to provide intelligent question-answering with reliable information about medical queries in clear and plain English, which can be understood by both healthcare providers and patients. We aimed to evaluate two freely available LLMs (ChatGPT and Google's Bard) in answering questions regarding the management of gynecologic cancer. The LLMs' performances were evaluated by developing a set questions that addressed common gynecologic oncologic findings from a patient's perspective and more complex questions to elicit recommendations from a clinician's perspective. Each question was presented to the LLM interface, and the responses generated by the artificial intelligence (AI) model were recorded. The responses were assessed based on the adherence to the National Comprehensive Cancer Network and European Society of Gynecological Oncology guidelines. This evaluation aimed to determine the accuracy and appropriateness of the information provided by LLMs. We showed that the models provided largely appropriate responses to questions regarding common cervical cancer screening tests and BRCA-related questions. Less useful answers were received to complex and controversial gynecologic oncology cases, as assessed by reviewing the common guidelines. ChatGPT and Bard lacked knowledge of regional guideline variations, However, it provided practical and multifaceted advice to patients and caregivers regarding the next steps of management and follow up. We conclude that LLMs may have a role as an adjunct informational tool to improve outcomes.
AB - Gynecologic cancer requires personalized care to improve outcomes. Large language models (LLMs) hold the potential to provide intelligent question-answering with reliable information about medical queries in clear and plain English, which can be understood by both healthcare providers and patients. We aimed to evaluate two freely available LLMs (ChatGPT and Google's Bard) in answering questions regarding the management of gynecologic cancer. The LLMs' performances were evaluated by developing a set questions that addressed common gynecologic oncologic findings from a patient's perspective and more complex questions to elicit recommendations from a clinician's perspective. Each question was presented to the LLM interface, and the responses generated by the artificial intelligence (AI) model were recorded. The responses were assessed based on the adherence to the National Comprehensive Cancer Network and European Society of Gynecological Oncology guidelines. This evaluation aimed to determine the accuracy and appropriateness of the information provided by LLMs. We showed that the models provided largely appropriate responses to questions regarding common cervical cancer screening tests and BRCA-related questions. Less useful answers were received to complex and controversial gynecologic oncology cases, as assessed by reviewing the common guidelines. ChatGPT and Bard lacked knowledge of regional guideline variations, However, it provided practical and multifaceted advice to patients and caregivers regarding the next steps of management and follow up. We conclude that LLMs may have a role as an adjunct informational tool to improve outcomes.
KW - ChatGPT
KW - artificial intelligence
KW - gynecologic cancer management
KW - health communication
KW - large language model
KW - patient education
UR - http://www.scopus.com/inward/record.url?scp=85201566936&partnerID=8YFLogxK
U2 - 10.1002/ijgo.15869
DO - 10.1002/ijgo.15869
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.systematicreview???
C2 - 39161265
AN - SCOPUS:85201566936
SN - 0020-7292
VL - 168
SP - 419
EP - 427
JO - International Journal of Gynecology and Obstetrics
JF - International Journal of Gynecology and Obstetrics
IS - 2
ER -