MadriMed-VL-2B

A 2B-parameter multimodal medical vision-language model, trained for medical image understanding, radiology report assistance, clinical visual question answering, and medical text reasoning.

📊 Benchmarks

Vision-Language (Medical Image Dataset evaluation)

Text-Only (Zero-Shot)

⚠️ Important: These scores reflect zero-shot generalization from visual fine-tuning alone, yet it still emerges as one of the best-performing 2B medical models on text-only dataser.

🚀 Quick Start

Installation

pip install transformers torch Pillow

Run the model directly

import torch
import re
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

MODEL_PATH = "madrisight/MadriMed-VL-2B"
print(f"🚀 Loading {MODEL_PATH} for MedQA inference...")

# -----------------------
# Load model + processor
# -----------------------
processor = AutoProcessor.from_pretrained(MODEL_PATH, trust_remote_code=True)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    MODEL_PATH,
    device_map="mps",
    trust_remote_code=True
)
model.eval()

# -----------------------
# Output cleaner
# -----------------------
def clean_output(text: str) -> str:
    text = re.sub(r"</?think>", "", text)
    text = re.sub(r"<\|?image\|?>", "", text)
    return text.strip()

prompt = (
    "You are a medical reasoning classification system.\n\n"
    "Your task is to classify each question into the SINGLE most appropriate underlying immune mechanism.\n\n"
    "RULES:\n"
    "- Output ONLY one standardized category label\n"
    "- Choose the most specific textbook mechanism\n"
    "- If multiple are possible, choose the most primary cause\n\n"
    "ALLOWED OUTPUT CATEGORIES (use exact wording):\n"
    "- CD4+ T-cell deficiency\n"
    "- CD8+ T-cell dysfunction\n"
    "- Neutropenia\n"
    "- Defective phagocytic oxidative burst\n"
    "- Humoral immunity deficiency (B-cell dysfunction)\n"
    "- Complement deficiency\n"
    "- Bone marrow suppression\n"
    "- Cytokine signaling defect\n"
    "- Hypersensitivity reaction Type I\n"
    "- Hypersensitivity reaction Type II\n"
    "- Hypersensitivity reaction Type III\n"
    "- Hypersensitivity reaction Type IV\n"
    "- General immunosuppression (only if no specific mechanism applies)\n\n"
    "OUTPUT FORMAT:\n"
    "Reasoning: <brief reasoning>\n"
    "Final Answer: <category>\n\n"
    "Question:\n"
    "A patient with HIV develops progressive loss of helper T-cell function leading to opportunistic infections."
)

messages = [
    {
        "role": "system",
        "content": "You are an expert medical AI. You must deeply analyze the question and must provide your final answer."
    },
    {
        "role": "user",
        "content": [{"type": "text", "text": prompt}]
    }
]

# -----------------------
# Inference
# -----------------------
with torch.inference_mode():

    text = processor.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )

    inputs = processor(
        text=text,
        # images=image,
        return_tensors="pt",
        padding=True,
        truncation=True,
    ).to("mps")

    generated_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
        repetition_penalty=1.05
    )

    output_text = processor.batch_decode(
        generated_ids[:, inputs.input_ids.shape[1]:],
        skip_special_tokens=True
    )[0]

output_text = clean_output(output_text)
print(output_text)

Reasoning: The loss of helper T-cell function in HIV leads to impaired immune response, particularly affecting CD4+ T-cells which are crucial for orchestrating immune responses. This aligns with CD4+ T-cell deficiency as it directly impacts the ability of these cells to support other immune cells.
Final Answer: CD4+ T-cell deficiency

GGUF Version

Available via mradermacher: https://huggingface.co/mradermacher/MadriMed-VL-2B-GGUF Run locally with LM Studio, llama.cpp, or Ollama.

⚠️ Limitations & Safety

🎯 What This Model Does

MadriMed-VL-2B is optimized on medical image like Xrays, CTscan, MRI, etc.:

• 🔬 Radiology VQA — answer questions about medical images
• 🫁 Modality/Plane Detection — identify imaging type and orientation
• 🏥 Organ/Position Recognition — localize anatomical structures
• ⚕️ Binary Screening — yes/no presence/absence questions

What It Does NOT Do

• ❌ Diagnosis — do not use for patient diagnosis
• ❌ Treatment Recommendations — not trained on clinical guidelines
• ❌ Pathology Detection — abnormality recognition is limited

Known Issues

Safety Guidelines

• NOT for clinical diagnosis
• NOT for treatment decisions
• Use as educational aid only
• Always verify binary answers
• Check for corruption

🔬 Technical Details

Training Configuration

🙏 Acknowledgments

• Training data: mint-medmax/medmax_data

📄 Citation

If you use this model in research, please cite:

@software{madrimedvl2b,
  title = {MadriMed-VL-2B: A Compact Multimodal Medical Vision-Language Model},
  author = {Madrisight},
  year = {2026},
  url = {https://huggingface.co/madrisight/MadriMed-VL-2B}
}

Disclaimer: This model is provided for research and educational purposes only. It is not FDA-approved, not clinically validated, and must not be used for patient care without expert human oversight. The authors assume no liability for clinical use.