Qwen3-4B-OT3-tau2-1ep (tool-use / τ²-bench teacher, SFT, epoch 1)
Qwen3-4B-OT3-tau2-1ep is the tool-use (τ²-bench) domain teacher of the MMOPD study: MMOPD/Qwen3-4B-OT3-2ep fine-tuned
(supervised) on the agent turns of inclusionAI/AReaL-tau2-data (airline / retail / telecom customer-service dialogues with
tool calls and thinking). This repository holds the epoch-1 checkpoint (step 130 of a 3-epoch run); the sibling
repositories MMOPD/Qwen3-4B-OT3-tau2-{1ep,2ep,3ep} hold the other epochs of the same run.
Training
| init | MMOPD/Qwen3-4B-OT3-2ep |
| data | inclusionAI/AReaL-tau2-data tau2_sft_train.jsonl: 32,758 of 33,531 single-turn rows (retail 11,395 / airline 12,320 / telecom 9,043); each row = dialogue context → the next assistant turn (thinking + reply or tool calls) |
| filtering | 251 telecom rows whose task id is in the τ²-bench telecom base or test split were removed (evaluation contamination); 522 airline rows longer than 32,768 tokens were dropped (no truncation of system prompts) |
| rendering | native Qwen3 chat template with the official τ²-bench tool schemas (Tool.openai_schema of each domain) in the system turn; the target is the final assistant turn only: <think>…</think> + text, or <think>…</think> + <tool_call> block; earlier assistant turns in the context carry text + tool calls without thinking (as at inference) |
| recipe | SFT, 1 of 3 epochs (this checkpoint), LR 5e-6 cosine to 5e-7 (5% warmup), 32,768-token flatten packing (8,273 bins per epoch, 64 bins per optimizer step on 64 GPUs), bf16 + ZeRO-2, loss on the assistant target only; ~265M target-context tokens per epoch |
Evaluation (τ²-bench telecom)
Official harness sierra-research/tau2-bench v1.0.1, telecom base split (114 tasks), 1 trial, user simulator
gpt-4.1-2025-04-14 (temperature 0), agent served by vLLM 0.12 with --tool-call-parser hermes --reasoning-parser qwen3
(the stock Qwen3 parsers), temperature 1.0 / top-p 1.0 / top-k off, 32,768-token context, --max-retries 3 (no retry was
needed in these runs). "Official prompt" = the harness' own LLMAgent system prompt (policy + technical-support manual);
"AReaL prompt" = the shorter system prompt used in the training data, a NON-official protocol. pass^1 in %; "clean" =
pass^1 on the 94 base tasks whose ids do not occur in the SFT data.
| Model | prompt | pass^1 | clean | terminations: user_stop / too_many_errors / max_steps |
|---|---|---|---|---|
| Qwen3-4B-OT3-tau2-1ep (this) | official | 43.9 | 39.4 | 77 / 31 / 6 |
| Qwen3-4B-OT3-tau2-1ep (this) | AReaL | 24.6 | 25.5 | 45 / 67 / 2 |
| Qwen3-4B-OT3-tau2-2ep | official | 57.9 | 55.3 | 98 / 11 / 5 |
| Qwen3-4B-OT3-tau2-2ep | AReaL | 63.2 | 60.6 | 103 / 11 / 0 |
| Qwen3-4B-OT3-tau2-3ep | official | 64.9 | 62.8 | 109 / 5 / 0 |
| Qwen3-4B-OT3-tau2-3ep | AReaL | 63.2 | 58.5 | 103 / 11 / 0 |
An earlier measurement of the 1-epoch and 2-epoch checkpoints used a stricter reasoning parser that rejected agent turns
emitting <think> followed directly by a tool call without </think>; every such turn raised an exception and the harness
re-ran the task from scratch (--max-retries 3), so those numbers include a best-of-up-to-4 effect on format slips and are
NOT comparable with the table above (official prompt / AReaL prompt):
| Model | official (strict parser + retries) | AReaL (strict parser + retries) |
|---|---|---|
| Qwen3-4B-OT3-tau2-1ep | 42.1 | 16.7 |
| Qwen3-4B-OT3-tau2-2ep | 71.9 | 60.5 |
Notes
- Weights are stored in bfloat16 (the SFT checkpoint as saved). Apache-2.0.
- The model keeps the Qwen3 thinking format. At temperature 1.0 the 1-epoch checkpoint often emits
<think>immediately followed by a<tool_call>without closing the think block (the training data contains 848 turns with empty thinking); the 2- and 3-epoch checkpoints rarely do. vLLM's stockqwen3reasoning parser passes such turns through (the tool call is executed, the stray<think>stays in the message content); a parser that requires a closed think block rejects them. - Telecom
base-split scores are reported both on all 114 tasks and on the 94 tasks whose ids do not occur in the SFT data (the τ²-bench telecom task ids are deterministic subtask compositions and 20 of the 114 base tasks appear inAReaL-tau2-data; those rows were excluded from training, but the scenario texts of the full split also underlie the AReaL RL data, so the whole telecom split should be considered exposed). - Part of the MMOPD model family together with
MMOPD/Qwen3-4B-OT3-{1ep,2ep},MMOPD/Qwen3-1.7B-OT3-{1ep,2ep}and the other domain teachersMMOPD/Qwen3-4B-OT3-{medical,law,finance,if}.
How to use
Serve with vLLM and the Qwen3 tool-call / reasoning parsers, then run the official τ²-bench harness against it:
vllm serve MMOPD/Qwen3-4B-OT3-tau2-1ep --served-model-name student --max-model-len 32768 \
--enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser qwen3 --generation-config vllm
tau2 run --domain telecom --task-split-name base --agent llm_agent --user user_simulator \
--agent-llm openai/student --agent-llm-args '{"api_base":"http://127.0.0.1:8000/v1","api_key":"EMPTY","temperature":1.0,"top_p":1.0}' \
--user-llm gpt-4.1-2025-04-14 --user-llm-args '{"temperature":0.0}' --num-trials 1 --save-to my_run
Plain chat with tools (transformers):
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "MMOPD/Qwen3-4B-OT3-tau2-1ep"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")
tools = [{"type": "function", "function": {"name": "get_customer_by_phone", "description": "Look up a customer by phone number.",
"parameters": {"type": "object", "properties": {"phone_number": {"type": "string"}}, "required": ["phone_number"]}}}]
messages = [{"role": "system", "content": "You are a customer service agent for a telecom company. Follow the policy."},
{"role": "user", "content": "Hi, I can't send MMS messages since this morning. My number is 555-123-2002."}]
text = tok.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True, enable_thinking=True)
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=4096, do_sample=True, temperature=1.0, top_p=1.0)
print(tok.decode(out[0][tok(text, return_tensors="pt").input_ids.shape[1]:], skip_special_tokens=False))