1,153 Downloads Updated 2 weeks ago
ollama run MedAIBase/MedGemma1.5:4b
MedGemma 1.5 model card
Note: This card describes MedGemma 1.5, which is only available as a 4B multimodal instruction-tuned variant. For information on MedGemma 1 variants, refer to the MedGemma 1 model card.
Model documentation: MedGemma
Resources:
MedGemma’s training may make it more sensitive to the specific prompt used than Gemma 3.
When adapting MedGemma developer should consider the following:
Author: Google
Model information
This section describes the specifications and recommended use of the MedGemma model.
Description
MedGemma is a collection of Gemma 3 variants that are trained for performance on medical text and image comprehension. Developers can use MedGemma to accelerate building healthcare-based AI applications.
MedGemma 1.5 4B is an updated version of the MedGemma 1 4B model.
MedGemma 1.5 4B expands support for several new medical imaging and data processing applications, including:
In addition to these new features, MedGemma 1.5 4B delivers improved accuracy on medical text reasoning and modest improvement on standard 2D image interpretation compared to MedGemma 1 4B.
MedGemma utilizes a SigLIP image encoder that has been specifically pre-trained on a variety of de-identified medical data, including chest X-rays, dermatology images, ophthalmology images, and histopathology slides. The LLM component is trained on a diverse set of medical data, including medical text, medical question-answer pairs, FHIR-based electronic health record data, 2D and 3D radiology images, histopathology images, ophthalmology images, dermatology images, and lab reports for document understanding.
MedGemma 1.5 4B has been evaluated on a range of clinically relevant benchmarks to illustrate its baseline performance. These evaluations are based on both open benchmark datasets and internally curated datasets. Developers are expected to fine-tune MedGemma for improved performance on their use case. Consult the Intended use section for more details.
MedGemma is optimized for medical applications that involve a text generation component. For medical image-based applications that do not involve text generation, such as data-efficient classification, zero-shot classification, or content-based or semantic image retrieval, the MedSigLIP image encoder is recommended. MedSigLIP is based on the same image encoder that powers MedGemma 1 and MedGemma 1.5.
How to use
The following are some example code snippets to help you quickly get started running the model locally on GPU.
Note: If you need to use the model at scale, we recommend creating a production version using Model Garden. Model Garden provides various deployment options and tutorial notebooks, including specialized server-side image processing options for efficiently handling large medical images: Whole Slide Digital Pathology (WSI) or volumetric scans (CT/MRI) stored in Cloud DICOM Store or Google Cloud Storage (GCS).
First, install the Transformers library. Gemma 3 is supported starting from transformers 4.50.0.
$ pip install -U transformers
Next, use either the pipeline wrapper or the transformer API directly to send a chest X-ray image and a question to the model.
Note that CT, MRI and whole-slide histopathology images require some pre-processing; see the CT and WSI notebook for examples.
Run model with the pipeline API
from modelscope import pipeline
from PIL import Image
import requests
import torch
pipe = pipeline(
“image-text-to-text”,
model=“google/medgemma-1.5-4b-it”,
torch_dtype=torch.bfloat16,
device=“cuda”,
)
# Image attribution: Stillwaterising, CC0, via Wikimedia Commons
image_url = “https://upload.wikimedia.org/wikipedia/commons/c/c8/Chest_Xray_PA_3-8-2010.png”
image = Image.open(requests.get(image_url, headers={“User-Agent”: “example”}, stream=True).raw)
messages = [
{
“role”: “user”,
“content”: [
{“type”: “image”, “image”: image},
{“type”: “text”, “text”: “Describe this X-ray”}
]
}
]
output = pipe(text=messages, max_new_tokens=2000)
print(output[0][“generated_text”][-1][“content”])
Run the model directly
# Make sure to install the accelerate library first via pip install accelerate
from modelscope import AutoProcessor, AutoModelForImageTextToText
from PIL import Image
import requests
import torch
model_id = “google/medgemma-1.5-4b-it”
model = AutoModelForImageTextToText.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map=“auto”,
)
processor = AutoProcessor.from_pretrained(model_id)
# Image attribution: Stillwaterising, CC0, via Wikimedia Commons
image_url = “https://upload.wikimedia.org/wikipedia/commons/c/c8/Chest_Xray_PA_3-8-2010.png”
image = Image.open(requests.get(image_url, headers={“User-Agent”: “example”}, stream=True).raw)
messages = [
{
“role”: “user”,
“content”: [
{“type”: “image”, “image”: image},
{“type”: “text”, “text”: “Describe this X-ray”}
]
}
]
inputs = processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors=“pt”
).to(model.device, dtype=torch.bfloat16)
input_len = inputs[“input_ids”].shape[-1]
with torch.inference_mode():
generation = model.generate(**inputs, max_new_tokens=2000, do_sample=False)
generation = generation[0][input_len:]
decoded = processor.decode(generation, skip_special_tokens=True)
print(decoded)
Examples
Refer to the growing collection of tutorial notebooks to see how to use or fine-tune MedGemma.
Model architecture overview
The MedGemma model is built based on Gemma 3 and uses the same decoder-only transformer architecture as Gemma 3. To read more about the architecture, consult the Gemma 3 model card.
Technical specifications
Citation
When using this model, please cite: Sellergren et al. “MedGemma Technical Report.“ *arXiv preprint arXiv:2507.05201* (2025).
@article{sellergren2025medgemma,
title={MedGemma Technical Report},
author={Sellergren, Andrew and Kazemzadeh, Sahar and Jaroensri, Tiam and Kiraly, Atilla and Traverse, Madeleine and Kohlberger, Timo and Xu, Shawn and Jamil, Fayaz and Hughes, Cían and Lau, Charles and others},
journal={arXiv preprint arXiv:2507.05201},
year={2025}
}
Inputs and outputs
Input:
Output:
Performance and evaluations
MedGemma was evaluated across a range of different multimodal classification, report generation, visual question answering, and text-based tasks.
Key performance metrics
Imaging evaluations
The multimodal performance of MedGemma 1.5 4B was evaluated across a range of benchmarks, focusing on radiology (2D, longitudinal 2D, and 3D), dermatology, histopathology, ophthalmology, document understanding, and multimodal clinical reasoning. See Data card for details of individual datasets.
We also list the previous results for MedGemma 1 4B and 27B (multimodal models only), as well as for Gemma 3 4B for comparison.
| Task / Dataset | Metric | Gemma 3 4B | MedGemma 1 4B | MedGemma 1.5 4B | MedGemma 1 27B |
|---|---|---|---|---|---|
| 3D radiology image classification | |||||
| CT Dataset 1*(7 conditions/abnormalities) | Macro accuracy | 54.5 | 58.2 | 61.1 | 57.8 |
| CT-RATE (validation, 18 conditions/abnormalities ) | Macro F1 | 23.5 | 27.0 | ||
| Macro precision | 34.5 | 34.2 | |||
| Macro recall | 34.1 | 42.0 | |||
| MRI Dataset 1*(10 conditions/abnormalities) | Macro accuracy | 51.1 | 51.3 | 64.7 | 57.4 |
| 2D image classification | |||||
| MIMIC CXR** | Macro F1 (top 5 conditions) | 81.2 | 88.9 | 89.5 | 90.0 |
| CheXpert CXR | Macro F1 (top 5 conditions) | 32.6 | 48.1 | 48.2 | 49.9 |
| CXR14 | Macro F1 (3 conditions) | 32.0 | 50.1 | 48.4 | 45.3 |
| PathMCQA* (histopathology) | Accuracy | 37.1 | 69.8 | 70.0 | 71.6 |
| WSI-Path* (whole-slide histopathology) | ROUGE | 2.3 | 2.2 | 49.4 | 4.1 |
| US-DermMCQA* | Accuracy | 52.5 | 71.8 | 73.5 | 71.7 |
| EyePACS* (fundus) | Accuracy | 14.4 | 64.9 | 76.8 | 75.3 |
| Disease Progression Classification (Longitudinal) | |||||
| MS-CXR-T | Macro Accuracy | 59.0 | 61.11 | 65.7 | 50.1 |
| Visual question answering | |||||
| SLAKE (radiology) | Tokenized F1 | 40.2 | 72.3 | 59.7**** | 70.3 |
| Accuracy (on closed subset) | 62.0 | 87.6 | 82.8 | 85.9 | |
| VQA-RAD*** (radiology) | Tokenized F1 | 33.6 | 49.9 | 48.1 | 46.7 |
| Accuracy (on closed subset) | 42.1 | 69.1 | 70.2 | 67.1 | |
| Region of interest detection | |||||
| Chest ImaGenome: Anatomy bounding box detection | Intersection over union | 5.7 | 3.1 | 38.0 | 16.0 |
| Multimodal medical knowledge and reasoning | |||||
| MedXpertQA (text + multimodal questions) | Accuracy | 16.4 | 18.8 | 20.9 | 26.8 |
* Internal datasets. CT Dataset 1 and MRI Dataset 1 are described below – for evaluation, perfectly balanced samples were drawn per condition. US-DermMCQA is described in Liu et al. (2020, Nature medicine), presented as a 4-way MCQ per example for skin condition classification. PathMCQA is based on multiple datasets, presented as 3-9 way MCQ per example for identification, grading, and subtype for breast, cervical, and prostate cancer. WSI-Path is a dataset of deidentified H&E WSIs and associated final diagnosis text from original pathology reports, comprising single WSI examples and previously described in Ahmed et al. (2024, arXiv). EyePACS is a dataset of fundus images with classification labels based on 5-level diabetic retinopathy severity (None, Mild, Moderate, Severe, Proliferative). A subset of these datasets are described in more detail in the MedGemma Technical Report.
** Based on radiologist adjudicated labels, described in Yang (2024, arXiv) Section A.1.1.
*** Based on “balanced split,” described in Yang (2024, arXiv).
**** While MedGemma 1.5 4B exhibits strong radiology interpretation capabilities, it was less optimized for the SLAKE Q&A format compared to MedGemma 1 4B. Fine-tuning on SLAKE may improve results.
Chest X-ray report generation
MedGemma chest X-ray (CXR) report generation performance was evaluated on MIMIC-CXR using the RadGraph F1 metric. We compare MedGemma 1.5 4B against a fine-tuned version of MedGemma 1 4B, and the MedGemma 1 27B base model.
| Task / Dataset | Metric | MedGemma 1 4B (tuned for CXR) | MedGemma 1.5 4B | MedGemma 1 27B |
|---|---|---|---|---|
| Chest X-ray report generation | ||||
| MIMIC CXR - RadGraph F1 | 30.3 | 27.2 | 27.0 | |
Text evaluations
MedGemma 1.5 4B was evaluated across a range of text-only benchmarks for medical knowledge and reasoning. Existing results for MedGemma 1 variants and Gemma 3 are shown for comparison.
| Dataset | Gemma 3 4B | MedGemma 1 4B | MedGemma 1.5 4B | MedGemma 1 27B |
|---|---|---|---|---|
| MedQA (4-op) | 50.7 | 64.4 | 69.1 | 85.3 |
| MedMCQA | 45.4 | 55.7 | 59.8 | 70.2 |
| PubMedQA | 68.4 | 73.4 | 68.2 | 77.2 |
| MMLU Med | 67.2 | 70.0 | 69.6 | 86.2 |
| MedXpertQA (text only) | 11.6 | 14.2 | 16.4 | 23.7 |
| AfriMed-QA (25 question test set) | 48.0 | 52.0 | 56.0 | 72.0 |
Medical record evaluations
EHR understanding and interpretation was evaluated for synthetic longitudinal text-based EHR data and real-world de-identified discharge summaries via question-answering benchmark datasets for MedGemma 1.5 4B, MedGemma 1 variants, and Gemma 3 4B.
| Dataset | Metric | Gemma 3 4B | MedGemma 1 4B | MedGemma 1.5 4B | MedGemma 1 27B |
|---|---|---|---|---|---|
| EHRQA* | Accuracy | 70.9 | 67.6 | 89.6 | 90.5 |
| EHRNoteQA | Accuracy | 78.0 | 79.4 | 80.4 | 90.7 |
* Internal dataset
Document understanding evaluations
Evaluation of converting unstructured medical lab reports documents (PDFs/images) into structured JSON data.
| Task / Dataset | Metric | Gemma 3 4B | MedGemma 1 4B | MedGemma 1.5 4B | MedGemma 1 27B |
|---|---|---|---|---|---|
| PDF-to-JSON Lab Test Data Conversion | |||||
| EHR Dataset 2* (raw PDF to JSON) | Macro F1 (average over per document F1 scores) | 84.0 | 78.0 | 91.0 | 76.0 |
| Micro F1 (F1 across all extracted data fields) | 81.0 | 75.0 | 88.0 | 70.0 | |
| EHR Dataset 3* (raw PDF to JSON) | Macro F1 | 61.0 | 50.0 | 71.0 | 66.0 |
| Micro F1 | 61.0 | 51.0 | 70.0 | 69.0 | |
| Mendeley Clinical Laboratory Test Reports (PNG image to JSON) | Macro F1 | 83.0 | 85.0 | 85.0 | 69.0 |
| Micro F1 | 78.0 | 81.0 | 83.0 | 68.0 | |
| EHR Dataset 4* | Macro F1 | 41.0 | 25.0 | 64.0 | |
| Micro F1 | 41.0 | 33.0 | 67.0 | ||
* Internal datasets.
Ethics and safety evaluation
Evaluation approach
Our evaluation methods include structured evaluations and internal red-teaming testing of relevant content policies. Red-teaming was conducted by a number of different teams, each with different goals and human evaluation metrics. These models were evaluated against a number of different categories relevant to ethics and safety, including:
In addition to development level evaluations, we conduct “assurance evaluations” which are our “arms-length” internal evaluations for responsibility governance decision making. They are conducted separately from the model development team and inform decision making about release. High-level findings are fed back to the model team but prompt sets are held out to prevent overfitting and preserve the results’ ability to inform decision making. Notable assurance evaluation results are reported to our Responsibility & Safety Council as part of release review.
Evaluation results
For all areas of safety testing, we saw safe levels of performance across the categories of child safety, content safety, and representational harms compared to previous Gemma models. All testing was conducted without safety filters to evaluate the model capabilities and behaviors. For both text-to-text and image-to-text the model produced minimal policy violations. A limitation of our evaluations was that they included primarily English language prompts.
Data card
Dataset overview
Training
The base Gemma models are pre-trained on a large corpus of text and code data. MedGemma multimodal variants utilize a SigLIP image encoder that has been specifically pre-trained on a variety of de-identified medical data, including radiology images, histopathology images, ophthalmology images, and dermatology images. Their LLM component is trained on a diverse set of medical data, including medical text, medical question-answer pairs, FHIR-based electronic health record data (27B multimodal only), radiology images, histopathology patches, ophthalmology images, and dermatology images.
Evaluation
MedGemma models have been evaluated on a comprehensive set of clinically relevant benchmarks across multiple datasets, tasks and modalities. These benchmarks include both open and internal datasets.
Source
MedGemma utilizes a combination of public and private datasets.
This model was trained on diverse public datasets including MIMIC-CXR (chest X-rays and reports), ChestImaGenome: Set of bounding boxes linking image findings with anatomical regions for MIMIC-CXR SLAKE (multimodal medical images and questions), PAD-UFES-20 (skin lesion images and data), SCIN (dermatology images), TCGA (cancer genomics data), CAMELYON (lymph node histopathology images), PMC-OA (biomedical literature with images), and Mendeley Digital Knee X-Ray (knee X-rays).
Additionally, multiple diverse proprietary datasets were licensed and incorporated (described next).
Data ownership and documentation
In addition to the public datasets listed above, MedGemma was also trained on de-identified, licensed datasets or datasets collected internally at Google from consented participants.
Data citation
De-identification/anonymization:
Google and its partners utilize datasets that have been rigorously anonymized or de-identified to ensure the protection of individual research participants and patient privacy.
Implementation information
Details about the model internals.
Software
Training was done using JAX.
JAX allows researchers to take advantage of the latest generation of hardware, including TPUs, for faster and more efficient training of large models.
Use and limitations
Intended use
MedGemma is an open multimodal generative AI model intended to be used as a starting point that enables more efficient development of downstream healthcare applications involving medical text and images. MedGemma is intended for developers in the life sciences and healthcare space. Developers are responsible for training, adapting, and making meaningful changes to MedGemma to accomplish their specific intended use. MedGemma models can be fine-tuned by developers using their own proprietary data for their specific tasks or solutions.
MedGemma is based on Gemma 3 and has been further trained on medical images and text. MedGemma enables further development in medical contexts (image and textual); however, the model has been trained using chest x-ray, histopathology, dermatology, fundus images, CT, MR, medical text/documents and electronic health records (EHR) data. Examples of tasks within MedGemma’s training include visual question answering pertaining to medical images, such as radiographs, document understanding, or providing answers to textual medical questions.
Benefits
Limitations
MedGemma is not intended to be used without appropriate validation, adaptation, and/or making meaningful modification by developers for their specific use case. The outputs generated by MedGemma are not intended to directly inform clinical diagnosis, patient management decisions, treatment recommendations, or any other direct clinical practice applications. All outputs from MedGemma should be considered preliminary and require independent verification, clinical correlation, and further investigation through established research and development methodologies.
MedGemma’s multimodal capabilities have been primarily evaluated on single-image tasks. MedGemma has not been evaluated in use cases that involve comprehension of multiple images.
MedGemma has not been evaluated or optimized for multi-turn applications.
MedGemma’s training may make it more sensitive to the specific prompt used than Gemma 3.
When adapting MedGemma developer should consider the following:
Release notes
MedGemma 4B IT
