Uncategorized

Personalized Impression Generation for PET Reports Using Large Language Models





doi: 10.1007/s10278-024-00985-3.


Online ahead of print.

Affiliations

Item in Clipboard

Xin Tie et al.


J Imaging Inform Med.


.

Abstract

Large language models (LLMs) have shown promise in accelerating radiology reporting by summarizing clinical findings into impressions. However, automatic impression generation for whole-body PET reports presents unique challenges and has received little attention. Our study aimed to evaluate whether LLMs can create clinically useful impressions for PET reporting. To this end, we fine-tuned twelve open-source language models on a corpus of 37,370 retrospective PET reports collected from our institution. All models were trained using the teacher-forcing algorithm, with the report findings and patient information as input and the original clinical impressions as reference. An extra input token encoded the reading physician’s identity, allowing models to learn physician-specific reporting styles. To compare the performances of different models, we computed various automatic evaluation metrics and benchmarked them against physician preferences, ultimately selecting PEGASUS as the top LLM. To evaluate its clinical utility, three nuclear medicine physicians assessed the PEGASUS-generated impressions and original clinical impressions across 6 quality dimensions (3-point scales) and an overall utility score (5-point scale). Each physician reviewed 12 of their own reports and 12 reports from other physicians. When physicians assessed LLM impressions generated in their own style, 89% were considered clinically acceptable, with a mean utility score of 4.08/5. On average, physicians rated these personalized impressions as comparable in overall utility to the impressions dictated by other physicians (4.03, P = 0.41). In summary, our study demonstrated that personalized impressions generated by PEGASUS were clinically useful in most cases, highlighting its potential to expedite PET reporting by automatically drafting impressions.


Keywords:

Informatics; Large Language Models; Natural Language Processing; Nuclear Medicine; Positron Emission Tomography; Radiology Report Summarization.

PubMed Disclaimer

References

    1. R. D. Niederkohr et al., “Reporting Guidance for Oncologic 18 F-FDG PET/CT Imaging,” J Nucl Med, vol. 54, no. 5, pp. 756–761, May 2013. https://doi.org/10.2967/jnumed.112.112177 .



      DOI



      PubMed

    1. M. P. Hartung, I. C. Bickle, F. Gaillard, and J. P. Kanne, “How to Create a Great Radiology Report,” RadioGraphics, vol. 40, no. 6, pp. 1658–1670, Oct. 2020. https://doi.org/10.1148/rg.2020200020 .



      DOI



      PubMed

    1. Y. Zhang, D. Y. Ding, T. Qian, C. D. Manning, and C. P. Langlotz, “Learning to Summarize Radiology Findings,” in Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis, Brussels, Belgium: Association for Computational Linguistics, 2018, pp. 204–213. https://doi.org/10.18653/v1/W18-5623 .

    1. J. Hu, Z. Li, Z. Chen, Z. Li, X. Wan, and T.-H. Chang, “Graph Enhanced Contrastive Learning for Radiology Findings Summarization.” arXiv, Jun. 08, 2022. Accessed: Mar. 02, 2023. [Online]. Available: http://arxiv.org/abs/2204.00203

    1. J.-B. Delbrouck, M. Varma, and C. P. Langlotz, “Toward expanding the scope of radiology report summarization to multiple anatomies and modalities.” arXiv, Nov. 18, 2022. Accessed: Mar. 02, 2023. [Online]. Available: http://arxiv.org/abs/2211.08584



Source link

Leave a Reply

Your email address will not be published. Required fields are marked *