🩺 Docto Decision ModernBERT-fr-base (v0.1)

✨ Summary

Docto Decision is a family of French medical decision models. You give a text (a patient message, a clinical note, a case report), a question and the possible answers, and the model returns one probability per answer.

This model is the encoder of the family: a 150M-parameter ModernBERT fine-tuned from DoctoModernBERT-fr-base. It scores every option in one forward pass, which makes it the smallest and fastest model of the family, several times faster than the LLM versions on the same hardware.

The task is defined at inference time by the instruction, the question and the answers. Tasks close to the training ones work best, and new tasks can be tried zero-shot.

🎯 Use cases

🚀 Usage

import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer

repo = "bofenghuang/docto-decision-modernbert-fr-base-v0.1"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForMaskedLM.from_pretrained(repo, dtype=torch.bfloat16).eval().to("cuda")
ONE, ZERO = tok.encode("1", add_special_tokens=False)[-1], tok.encode("0", add_special_tokens=False)[-1]


def decide(task, question, options, context=None, profile=None, item=None, kind="choice"):
    """options: [(key, definition)]. kind: "choice", "score" (levels in order) or "noul" (one option: the proposition)."""
    parts = [f"[TASK] {task}"]
    if profile:
        parts.append(f"[PROFILE] {profile}")
    if context:
        parts.append(f"[CONTEXT]\n{context}")
    if item:
        parts.append(f"[ITEM] {item}")
    parts += [f"[QUESTION] {question}", "[OPTIONS]"]
    parts += [f"({i}) {d} -> [unused0] {tok.mask_token}" for i, (_, d) in enumerate(options, 1)]
    enc = tok("\n".join(parts), return_tensors="pt").to(model.device)
    with torch.no_grad():
        logits = model(**enc).logits[0]
    s = logits[enc.input_ids[0] == tok.mask_token_id][:, [ONE, ZERO]].float()
    s = s[:, 0] - s[:, 1]  # one score per option: logit("1") - logit("0") at its mask
    if kind == "noul":
        return torch.sigmoid(s[0]).item()  # P(true)
    return dict(zip([k for k, _ in options], torch.softmax(s, -1).tolist()))


decide(task="Déterminer la conduite à tenir la plus appropriée face à ce message.",
       profile="Patient : homme, 58 ans. Antécédents : hypertension.",
       context="Patient : Bonjour, j'ai une douleur dans la poitrine qui serre depuis 30 minutes et qui descend dans le bras gauche.",
       question="Quel niveau d'urgence correspond à cette situation ?",
       options=[("home_care", "Pas d'urgence : auto-surveillance à domicile chez un adulte sans fragilité, ou demande sans symptôme."),
                ("consult_48h", "Consultation dans les 48 heures : médecin traitant, sans caractère de gravité mais qui dure ou gêne."),
                ("same_day", "Consultation le jour même : signe préoccupant sans détresse vitale."),
                ("emergency_now", "Urgence immédiate : appeler le 15 ou se rendre immédiatement aux urgences.")],
       kind="score")
# -> {'home_care': 0.001, 'consult_48h': 0.000, 'same_day': 0.001, 'emergency_now': 0.998}

decide(task="Dire si le signe indiqué est décrit comme présent chez l'enfant dans la conversation.",
       profile="Enfant : bébé de 4 mois, garçon",
       context="Parent : Bonjour, mon fils a 39 de fièvre depuis ce matin, il ne boit presque plus ses biberons et il est très mou.",
       item="Signe : nourrisson de moins de 6 mois qui boit nettement moins que d'habitude",
       question="Ce signe est-il présent d'après le parent ?",
       options=[("present", "Oui, le parent décrit ce signe comme présent.")],
       kind="noul")
# -> 0.97

• Write each option as a full French sentence: the model reads the definitions. For choice, option order does not matter. For score, keep the levels in order.
• For noul, pass a single option (the proposition) and read P(true).
• The text is French and the tags ([TASK], [CONTEXT]...) are English. Leave out empty fields. The context window is 8,192 tokens: if a text is too long, shorten the context, not the options.

📊 Evaluation

Results on the Docto Decision Bench: 12 decision tasks on real French patient messages and two clinical datasets (DiaMed, MORFITT). The benchmark repository has the data and the evaluation code.

This model on each task:

• All scores are percentages. Labels: silver = Qwen3.8-27B and Qwen3.5-122B-A10B agree on the answer, gold = human labels of the dataset, audit = blind relabelling by another model.
• OpenJev is a fine-tune of Qwen3.8-27B, one of the two teachers behind the silver labels, so on the 9 silver tasks it is partly scored against its own teacher's answers.
• 95% CIs from 1,000 bootstrap resamples of the items in each task. Order consistency: share of answers that stay the same when the options of the 6 choice tasks are shown in 3 different orders.
• Latency: median time for one request, on one H100 in BF16 (OpenJev in FP8), without inference optimisation. Jev and Gemini are timed from the client, network included.

🏋️ Training

Full fine-tune of DoctoModernBERT-fr-base for 2 epochs on docto-decision-data-fr-v0.1 (943,013 decision records, 108 tasks). The target of each record is half the correct answer and half the answer distribution of Qwen3.8-27B.

⚠️ Limitations

• This is a research model, not a medical device, and it has not been validated clinically. Keep a human in the loop.
• It can under-estimate urgency. Do not use it as the only safety net for triage.
• It is weaker on tasks far from the training tasks and on medical knowledge questions.
• Most training labels come from one model family (Qwen), so the model shares its conventions and its errors.
• French only. Check calibration on your own task before choosing a threshold.