Meeting Banner
Abstract #0618

Comparison of Radiologists and Multimodal Large Language Models Responses to Radiology ImageQuest

Qingxia Wu1, Qingxia Wu2, Jon Xue3, Dinggang Shen3,4, and Meiyun Wang5
1Henan Provincial People's Hospital, Zhengzhou, China, 2Beijing United Imaging Research Institute of Intelligent Imaging, Beijing, China, 3Shanghai United Imaging Intelligence Co.,Ltd., Shanghai, China, 4School of Biomedical Engineering, ShanghaiTech University, Shanghai, China, 5Henan Provincial People’s Hospital, Zhengzhou, China

Synopsis

Keywords: Language Models, Language Models

Motivation: What are the capabilities of multimodal large language models (LLMs) in addressing radiology-related questions, and can they enhance the performance of junior radiologists?

Goal(s): To compare the performance of multimodal LLMs against radiologists of varying expertise levels and assess the impact of LLM assistance on junior radiologists' skills.

Approach: This study evaluated the performance of multimodal LLMs against radiologists using the Radiology ImageQuest dataset, comprising 1,251 cases from six reputable sources.

Results: Advanced LLMs like GPT-4o and Claude-3.5-sonnet demonstrated performance comparable to senior radiologists. Junior radiologists, with GPT-4o's assistance, nearly doubled their accuracy and achieved mid-level performance after a three-month period.

Impact: Multimodal LLMs show promise in radiology education and practice, while further research is needed to validate their impact on real clinical applications

How to access this content:

For one year after publication, abstracts and videos are only open to registrants of this annual meeting. Registrants should use their existing login information. Non-registrant access can be purchased via the ISMRM E-Library.

After one year, current ISMRM & ISMRT members get free access to both the abstracts and videos. Non-members and non-registrants must purchase access via the ISMRM E-Library.

After two years, the meeting proceedings (abstracts) are opened to the public and require no login information. Videos remain behind password for access by members, registrants and E-Library customers.

Click here for more information on becoming a member.

Keywords