ONCA 3.0-12B

ONCA 3.0-12B is an 11.95B-parameter research model derived from google/gemma-4-12B. It handles pathological TNM classification, clinical-trial triage and criterion reasoning, and CIViC molecular-evidence classification through compact JSON outputs.

This repository contains the final full-GRPO model checkpoint. GRPO started from Phase 2 SFT and completed 285 optimizer steps over three epochs

[!CAUTION] ONCA 3.0-12B is a research model, not a medical device or substitute for qualified clinical judgment. Human oncology/pathology validation remains pending. Valid JSON can still be clinically wrong.

Results

Figure 3 is reproduced from the final manuscript draft v11. Results use the frozen 4,599-row development-validation set. Exact match requires the complete expected structured output; schema validity measures format only.

The TrialGPT evidence-sentence score was 83.88%. Nineteen validation inputs were truncated by the declared 4,096-token evaluation limit. No validation or test row was used for SFT or GRPO selection. Gemma 4 and Woollie-13B in the figure use candidate scoring with deterministic assembly, so their results are not controlled native-generation comparisons.

Figure data and provenance

Model details

The same checkpoint supports all six tasks. No separate task model is needed.

Transformers usage

The release was produced with Transformers 5.16.1.

pip install "transformers>=5.10,<6" accelerate safetensors torch

import json
import torch
from transformers import AutoModelForMultimodalLM, AutoTokenizer

MODEL_ID = "Joesh1/onca-3.0-12b"
SYSTEM = (
    "Return only valid JSON matching the requested schema. Use only the "
    "provided input. This is a research classification, not medical advice."
)

report = "...insert a de-identified pathology report..."
request = f"""TASK: TNM_T
Classify the historical TCGA pathological T category represented by the report.
Use only T1, T2, T3, T4.
Return only JSON: {{"task":"tnm_t","label":"<LABEL>"}}

Cancer type: pancreatic adenocarcinoma
Pathology report:
{report}"""

# Exact text-only serialization used for training and evaluation.
prompt = (
    "<bos>"
    f"<|turn>system\n{SYSTEM}<turn|>\n"
    f"<|turn>user\n{request}<turn|>\n"
    "<|turn>model\n<|channel>thought\n<channel|>"
)

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForMultimodalLM.from_pretrained(
    MODEL_ID,
    dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa",
).eval()

inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
with torch.inference_mode():
    generated = model.generate(
        **inputs,
        do_sample=False,
        max_new_tokens=128,
        pad_token_id=tokenizer.eos_token_id,
    )

text = tokenizer.decode(
    generated[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
).strip()
print(json.loads(text))

Keep the task key, allowed labels, and exact JSON contract in each prompt. Validate every response against the selected schema and fail closed on parse errors or unknown fields. Do not request chain-of-thought.

GGUF

Quantized text-only builds are available in the Joesh1/onca-3.0-12b-gguf repository in F16, Q8_0, Q6_K, and Q4_K_M formats.

Training and provenance

Two NF4 QLoRA SFT phases trained on 16,625 unique examples. Phase 2 ended at step 2,184 with validation loss 0.020104. The final GRPO stage used six sampled completions per eligible prompt, learning rate 1e-6, KL coefficient 0.02, temperature 0.8, top-p 0.95, group-relative reward normalization, and three epochs.

The merged release contains 677 FP16 tensors. The merge applied 328 LoRA tensor pairs from the full-GRPO adapter.

Training sources include historical TCGA pathology reports, MatchMiner AI synthetic trial-space examples, TrialGPT criterion annotations, and accepted CIViC evidence statements. Group-aware splitting used TCGA patients, MatchMiner patients, TrialGPT patients, and CIViC PMIDs.

Limitations

• Results are from frozen development-validation data, not the locked final test partitions.
• Prospective clinical validation and independent expert adjudication are incomplete.
• GRPO produced only a marginal aggregate gain over Phase 2 SFT.
• TrialGPT data is small; trial-criterion exact match is the weakest result.
• MatchMiner examples are synthetic and trial-space output is triage, not eligibility.
• The model can hallucinate, misclassify, omit evidence, or return unsupported JSON.
• The release is text-only; do not submit images or identifiable patient information.

License

Released under CC BY-NC 4.0 as a conservative designation because MatchMiner AI training material is CC BY-NC 4.0. Gemma 4 is Apache 2.0; TCGA-derived sources are CC BY 4.0; TrialGPT is marked public domain in the frozen manifest; and CIViC data is CC0 1.0. Review all upstream terms before use.

Citation

@misc{onca30_2026,
  title  = {ONCA 3.0-12B: Oncology Language Model},
  author = {ONCA Research Team},
  year   = {2026},
  url    = {https://huggingface.co/Joesh1/onca-3.0-12b}
}

Related: ONCA 2.0 · Gemma 4 12B