Meeting Banner
Abstract #3383

Automatic Generation of Impressions from Brain MRI Report Findings using Large Language Models: A Multi-centers Retrospective Analysis

Chao Chai1, Zhiyang Liu2, Minghao Zhang2, Can Liu2, Yongquan Yu3, Huiying Wang1, Wen Shen1, and Shuang Xia1
1Department of Radiology, Tianjin First Central Hospital, Tianjin Medical Imaging Institute, School of Medicine, NankaiUniversity, Tianjin, China, Tianjin, China, 2Tianjin Key Laboratory of Optoelectronic Sensor and Sensing Network Technology, College ofElectronic Information and Optical Engineering, Nankai University, Tianjin, China, Tianjin, China, 3Department of Radiology, Weihai Central Hospital, Weihai, Shandong, China., Weihai, China

Synopsis

Keywords: Language Models, Language Models

Motivation: The performance of privacy-preserving LLMs for the generation of impressions in the radiology reports in multi-centers has not yet been investigated.

Goal(s): To develop privacy-preserving LLMs that generates the Impressions from Findings, and compare the performance with a public LLM (GPT4-turbo) on data from two centers.

Approach: Four privacy-preserving LLMs, including ChatGLM-6B, LLaMA2-Chinese-7B, Qwen1.5-7B and Baichuan2-7B, were finetuned. GPT4-turbo’s output was also optimized by prompt engineering. An automatic method for evaluating the similarities between impression items was proposed.

Results: Privacy-preserving LLMs offer enhanced accuracy in generating impressions, but performance varies across centers, highlighting their potential as a quality improvement tool under expert review.

Impact: we find that while LLMs can correct some diagnostic errors, they also introduce inaccuracies, underscoring the critical role of radiologist oversight. We believe these findings demonstrate the potential of LLMs as a valuable quality improvement tool in radiology.

How to access this content:

For one year after publication, abstracts and videos are only open to registrants of this annual meeting. Registrants should use their existing login information. Non-registrant access can be purchased via the ISMRM E-Library.

After one year, current ISMRM & ISMRT members get free access to both the abstracts and videos. Non-members and non-registrants must purchase access via the ISMRM E-Library.

After two years, the meeting proceedings (abstracts) are opened to the public and require no login information. Videos remain behind password for access by members, registrants and E-Library customers.

Click here for more information on becoming a member.

Keywords