parrotlet-v-2.5-pro

A fine-tune of google/gemma-4-E2B-it (Gemma 4, E2B — ~2B effective parameters) for structured, schema-driven extraction from Indian medical records: prescriptions, invoices, lab reports, and discharge summaries, plus PII detection.

Live demo · Launch blog

The language-model backbone was fully fine-tuned; the vision and audio towers and the multimodal projector were kept frozen. Weights are released in uniform bf16, matching the model's declared dtype and training precision.

How to use

The model is prompted with a document image plus one of the tags below as the user turn. No other instruction text is required — the tag alone selects the extraction schema.

pip install torch torchvision "transformers>=5.14" accelerate pillow

Keep the transformers>=5.14 floor: versions below 5.6 either lack the gemma4 architecture entirely or (5.5.0) load it with silently re-initialized layers. Expect a ~11 GB download and ~10.7 GiB peak VRAM in bf16.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image

model_id = "ekacare/parrotlet-v-2.5-pro"
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained(model_id)

image = Image.open("prescription.jpg")

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": "<prescription>"},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True,
    tokenize=True, return_dict=True, return_tensors="pt",
).to(model.device)

output = model.generate(**inputs, max_new_tokens=2048, do_sample=False)
result = processor.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(result)

Supported tags

Validate the tag string before sending: a mistyped tag does not error — the model returns valid-looking but empty JSON (e.g. <lab_repot> yields {"lab_report_elements": {"tests": []}}).

Evaluation

Internal evaluation on a held-out set of 1,147 pages across five document types, judged by an LLM rubric judge. Overall rubric score = mean(coverage, grounding), pooled from per-page judge scores. Baselines shown are this model's own un-fine-tuned base and four frontier models evaluated on the same pages with the same rubric.

Blended score, all five tasks

Fine-tuning improves the blended score by ~40 points over the base model and clears every frontier model tested by 1.7–4.4 points. Gemini 3.6 Flash is the closest. The lead is entirely in grounding — this model's coverage (82.35) is third-best, behind Gemini and Claude Sonnet 5, while its grounding of 89.09 is 4 points above the best frontier model. It reproduces slightly less of the reference, but far less of what it writes is unsupported.

Per document type (overall rubric score)

Wins 4 of 5 document types. Invoice is the exception and the model's weakest task — Gemini 3.6 Flash leads there, just ahead of Claude Sonnet 5. PII is the widest margin, ~12.5–16 points clear of every frontier model.

Lab reports: complex vs. simple

Complex = report-style/non-tabular modalities (echo, ECG, manometry, histopathology); simple = standard tabular reports.

The complex cohort is where the gap closes: Gemini 3.6 Flash comes within 0.12 points of this model there, while still trailing by 3.7 on simple pages. Gemini also has the smallest complex→simple drop of any model tested (8.0 points, against this model's 11.6), so its advantage is relative robustness to non-tabular layouts rather than better extraction overall.

Limitations

• Weakest on invoices relative to frontier models; this is the one document type where a fine-tune does not lead. Handwritten bills are the hardest invoices for every model tested — all of them score ~30 points lower on handwritten item rows than on printed ones — and this model's characteristic failure there is giving up entirely rather than extracting partially: it returns an empty line-item list on 18 of 200 invoices, 16 of which have hand-written items. Recovering those pages would lift the invoice score from 74.41 to ~76.2, closing about a fifth of the gap; the remainder is spread across ordinary printed invoices.
• Complex/non-tabular lab report layouts (echo, ECG, manometry, histopathology) show the largest remaining headroom, and are the one cohort where a frontier model (Gemini 3.6 Flash) draws level.
• Evaluated only on an internal held-out set; not benchmarked on out-of-distribution document layouts, languages, or non-Indian medical record formats.
• This is a document-parsing model, not a diagnostic or clinical decision-making tool. Outputs (including PII detection) should be reviewed by a human before use in any compliance-sensitive workflow.

License

Apache-2.0. The base model, google/gemma-4-E2B-it, is released by Google under Apache 2.0 (unlike Gemma 1–3's Gemma Terms of Use), so this fine-tune is distributed under Apache-2.0 as well. Google publishes a Prohibited Use Policy and Intended Use statement for the Gemma 4 family that users should review.

Citation

If you use this model, please cite:

@software{parrotlet_v_2_5_pro,
  author = {{Eka Care}},
  title  = {Parrotlet-v 2.5 Pro: schema-driven extraction from Indian medical documents},
  year   = {2026},
  url    = {https://huggingface.co/ekacare/parrotlet-v-2.5-pro}
}