A medical rubric grader for OnePO. Trained on GPT-4.1-annotated samples, HuatuoGPT-3-Grader-8B checks multiple criteria in one generation to provide rewards for open-ended medical responses.
📝 Grading template
Use this template exactly: replace only {conversation} and {rubrics}. Keep the wording, tags, and section order unchanged. Send the formatted prompt as one user message. The included chat template fixes the input format to no-thinking, matching training; no extra template or thinking flag is needed.
| Placeholder | Content |
|---|---|
{conversation} | Conversation as role: content lines, ending with the response to evaluate. |
{rubrics} | Numbered criteria with signed points, e.g. 1. (+3pts) Criterion text. |
GRADING_TEMPLATE = """Score the assistant's response against each rubric item.
## Conversation
<Conversation>
{conversation}
</Conversation>
## Rubrics
<Rubric_items>
{rubrics}
</Rubric_items>
## Output
Return only a JSON list of booleans, one for each rubric in order.
- true: the criterion is met
- false: the criterion is not met
- For a negative criterion (bad behavior with negative points), return true only when the response shows that bad behavior."""
🚀 Usage
Run the template definition above, followed by this example:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "FreedomIntelligence/HuatuoGPT-3-Grader-8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id, dtype="auto", device_map="auto"
).eval()
conversation = """user: Why can an incomplete medical history limit an assessment?
assistant: Missing symptom details and medication history can make it difficult to distinguish possible causes. The assessment should acknowledge this uncertainty and identify what further information is needed."""
rubrics = """1. (+3pts) Acknowledges uncertainty caused by missing information.
2. (+2pts) Identifies examples of missing information that could affect assessment.
3. (-4pts) Makes a definitive diagnosis without patient-specific evidence."""
prompt = GRADING_TEMPLATE.format(conversation=conversation, rubrics=rubrics)
inputs = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
tokenize=True, add_generation_prompt=True,
return_dict=True, return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(
outputs[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True
))
Illustrative output — one boolean per criterion, in the same order:
[true, true, false]
true means the criterion is met. For a negative-point criterion, it means the undesirable behavior is present and its points are deducted.
OnePO divides the sum of matched points by the total positive points, then clips to [0, 1]. The example above scores (3 + 2) / (3 + 2) = 1.0.
⚡ Batch scoring
Download this repository, then run:
python score.py --input example.json
python score.py --input batch.json --batch-size 8
Each record contains prompt, response, and rubrics (criterion, points). For batches, supply a JSON list of records. Each response's rubrics are evaluated together in one generation. Multiple responses share a batch for efficient inference.
The script follows OnePO's training protocol: extract the final answer and keep its first 10,000 characters. It returns criteria_met, raw_score, and the clipped training_reward, before optional training penalties.
📖 Citation
@inproceedings{chen2026onepo,
title={OnePO: Direct One-stage Policy Optimization for SFT-free Domain Adaptation},
author={Chen, Junying and Xie, Xinyuan and Li, Ziniu and Wang, Benyou},
booktitle={Proceedings of the 43rd International Conference on Machine Learning},
year={2026}
}