Keywords: Tumors (Pre-Treatment), Tumors, Glioma; Vision-Language Model; Large Language Model
Motivation: Leveraging a pre-trained large vision-language model may show robust performance for radiology of adult-type diffuse gliomas.
Goal(s): To establish a robust vision-language model for molecular subtyping, radiology report generation, and visual question answering (VQA) in adult-type diffuse gliomas.
Approach: MRI and paired radiology reports from 1,001 adult-type diffuse gliomas patients were included as the institutional training set. A vision-language model, Glio-LLaMA-Vision, was developed from LLaMA 3.1 pre-trained on 2.79 million biomedical text-image pairs and was optimized via fine-tuning from the training set. The performance was validated on external test sets.
Results: Glio-LLaMA-Vision showed robust performance on molecular subtyping, radiology report generation, and VQA.
Impact: Glio-LLaMA-Vision shows promising performance in molecular subtype prediction, radiology report generation, and VQA in adult-type diffuse gliomas. Notably, our current study provides a practical paradigm of adapting general domain LLMs to applications in a specific medical domain.
How to access this content:
For one year after publication, abstracts and videos are only open to registrants of this annual meeting. Registrants should use their existing login information. Non-registrant access can be purchased via the ISMRM E-Library.
After one year, current ISMRM & ISMRT members get free access to both the abstracts and videos. Non-members and non-registrants must purchase access via the ISMRM E-Library.
After two years, the meeting proceedings (abstracts) are opened to the public and require no login information. Videos remain behind password for access by members, registrants and E-Library customers.
Keywords