Qwen3-8B TMax SFT v3 — iteration 353
Supervised fine-tuning (SFT) checkpoint of Qwen/Qwen3-8B,
taken at training iteration 353 of the tmax_sft_v3 run.
This checkpoint is the initialization (policy + reference model) for the
tmax_aenv_v39b reinforcement-learning run, whose checkpoints are published
separately.
| Base model | Qwen/Qwen3-8B |
| Stage | SFT |
| Iteration | 353 |
| Architecture | Qwen3, 36 layers, hidden 4096, 32 heads / 8 KV, vocab 151936 |
| Precision | bfloat16 |
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "willamazon1/qwen3-8b-tmax-sft-v3-iter353"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")
msgs = [{"role": "user", "content": "What is 12*8?"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
ids = tok(text, return_tensors="pt").input_ids.to(model.device)
out = model.generate(ids, max_new_tokens=256)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
Conversion
Converted from a Megatron-LM torch_dist training checkpoint to HuggingFace
safetensors with slime's
tools/convert_torch_dist_to_hf.py. Embedding padding was stripped back to the
tokenizer's vocab_size of 151936, so the tensor shapes match upstream
Qwen/Qwen3-8B exactly. Weights are bfloat16; the optimizer state of the
training checkpoint is not included.
Every shard was checked for NaN/Inf (none found) and the model was loaded and sampled from before upload.