Citation screening using large language models for creating clinical practice guidelines: A protocol for a prospective study

Abstract

Background:
The development of clinical practice guidelines requires a meticulous literature search and screening process. This study aims to explore the potential of large language models in the development of the Japanese Clinical Practice Guidelines for Management of Sepsis and Septic Shock (J-SSCG), focusing on enhancing literature search quality and reducing the citation screening workload.
Methods:
A prospective study will be conducted to compare the efficiency and accuracy of literature citation screening between the conventional method and a novel approach using large language models. We will use the large language model, namely GPT-4, to conduct literature searches for predefined clinical questions. We will objectively measure the time required for citation screening and compare it to the time taken using the conventional method. Following the screening, we will calculate and compare the sensitivity and specificity of the results obtained from the conventional method and the large language models-assisted process. The total time spent using both approaches will also be compared to assess workload reduction.

Competing Interest Statement

The authors have declared no competing interest.

Clinical Protocols

https://www.umin.ac.jp/ctr/index-j.htm

Funding Statement

This study did not receive any funding.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

Since this research does not include the conditions as defined in “Life Science and Medical Research Involving Human Subjects”, we consider this research to be outside the scope of research for which an application for ethical approval is required.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes