doi: 10.1016/j.metrad.2023.100045.
Epub 2023 Nov 24.
Affiliations
Item in Clipboard
Meta Radiol.
2023 Nov.
Abstract
The emergence of artificial general intelligence (AGI) is transforming radiation oncology. As prominent vanguards of AGI, large language models (LLMs) such as GPT-4 and PaLM 2 can process extensive texts and large vision models (LVMs) such as the Segment Anything Model (SAM) can process extensive imaging data to enhance the efficiency and precision of radiation therapy. This paper explores full-spectrum applications of AGI across radiation oncology including initial consultation, simulation, treatment planning, treatment delivery, treatment verification, and patient follow-up. The fusion of vision data with LLMs also creates powerful multimodal models that elucidate nuanced clinical patterns. Together, AGI promises to catalyze a shift towards data-driven, personalized radiation therapy. However, these models should complement human expertise and care. This paper provides an overview of how AGI can transform radiation oncology to elevate the standard of patient care in radiation oncology, with the key insight being AGI’s ability to exploit multimodal clinical data at scale.
Keywords:
AGI; Large foundation model; Medical imaging; Radiation oncology; SAM.
Conflict of interest statement
Declaration of interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The Author Tianming Liu is the Editor in Chief of the journal, the authors Wei Li, Xiang Li, Dajiang Zhu and Dinggang Shen are the Editorial Board Members of the journal, but were not involved in the peer review procedure. This paper was handled by another Editor Board member.
Figures
Fig. 1.
Large Foundational Models for Radiation Oncology. The left half of the revised figure represents the existing challenge at hand, while the right half portrays the proposed solution provided by the large foundational models in radiation oncology.
Fig. 2.
The accuracy of GPT-4 in re-labeling structure names according to the TG-263 report.
Fig. 3.
Large Multimodal Foundation Models for Radiation Oncology. Here we use a brain tumor case as an example and incorporate a visual representation of all data sources.
Fig. 4.
AI based treatment planning workflow. a) KBP method b) Dose Prediction method. The dotted box means an improved enhancement for the workflow.
Fig. 5.
Synthetic CT generation using deep learning.
Fig. 6.
AGI for Image registration.
References
-
-
Subir Nag, David Beyer, Jay Friedland, Peter Grimm, Ravinder Nath. American brachyther- apy society (abs) recommendations for transperineal permanent brachytherapy of prostate cancer. Int J Radiat Oncol Biol Phys. 1999;44(4):789–799.
–
PubMed
-
-
-
Lott JS, Smith Ivan H. Cobalt-60 beam therapy in carcinoma of the esophagus. Radiology. 1958;71(3):321–326.
–
PubMed
-
-
-
Siebers JV, Gardner JK, Gordon JJ, Wang S, Ververs JD. Quantification of exit fluence variations and implications for exit fluence-based dose reconstruction based. Int J Radiat Oncol Biol Phys. 2008;72(1):S552.
-
