文献类型：期刊文献

英文题名：Assisting Recommendations Formulation of Practice Guideline with Large Language Models: A Feasibility Study

作者：Ye, Ziying[1,2]; Lai, Honghai[1,2]; Sun, Mingyao[3]; Huang, Jiajie[4]; Liu, Jiayi[1,2]; Xia, Danni[1,2]; Zhao, Weilong[1,2]; Liu, Jianing[4]; Ge, Long[1,2,5]

第一作者：Ye, Ziying

机构：[1] Department of Health Policy and Management, School of Public Health, Lanzhou University, Lanzhou, China; [2] Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China; [3] Evidence-Based Nursing Center, School of Nursing, Lanzhou University, Lanzhou, China; [4] College of Nursing, Gansu University of Chinese Medicine, Lanzhou, China; [5] Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou, China

第一机构：Department of Health Policy and Management, School of Public Health, Lanzhou University, Lanzhou, China

年份：2024

外文期刊名：SSRN

收录：EI(收录号：20240116814)

语种：英文

外文关键词：Artificial intelligence - Computational linguistics - Risk assessment

摘要：Background: Formulating recommendations in developing practice guidelines involves complex and rigorous processes. Therefore, more resources and expertise are needed. Artificial intelligence (AI) is promising in accelerating the guideline development process. This study aims to assess the feasibility of three large language models (ChatGPT-3.5, Claude, and Bard) in generating guideline recommendations, evaluate their concordance among the generated recommendations, and further explore the feasibility of AI-assisted evidence-based decision-making. Methods: The general and specific prompts of the three large language models were drafted and validated. We searched Embase, Web of Science, PubMed, and guidelines websites to include evidence-based guidelines related to health and lifestyle. We randomly selected recommendations from the included guidelines as the sample and extracted the evidence base supporting the selected recommendations. The prompts and evidence were fed into three AI systems to generate structured recommendations. Results: ChatGPT-3.5 exhibited the highest proficiency in comprehensively extracting and amalgamating evidence information to formulate novel insights. Bard consistently adhered to guidelines, aligning its algorithm closely with existing guidelines' intrinsic principles. Conversely, Claude tended to generate fewer topics, focusing on evidence analysis and mitigating the risk of extraneous and irrelevant information. Among the six recommendations generated in this study, the average consistency ranges from 50% to 100%. Conclusions: The findings suggest that AI can expedite the formulation of recommendations in developing practice guidelines. While AI can enhance efficiency, its role should be complementary rather than substitutive, working in tandem with medical professionals in guideline development. ? 2024, The Authors. All rights reserved.