Qwen3-8B MaKTO Public-Data SFT
This is a full-parameter supervised fine-tuning (SFT) checkpoint of
Qwen/Qwen3-8B, trained on the public
Chinese portion of
ReneeYe/werewolf_game_reasoning.
The model is an independent public-data reproduction inspired by the MaKTO paper. It is not a reproduction of the complete paper training recipe: the approximately 12K general-purpose SFT examples described by the paper are not public and were not used, and the Multi-agent KTO stage has not yet been performed.
Training data
The converted training corpus contains 12,886 examples:
- 12,134 game-behavior examples
- 372 advanced-technique examples
- 380 fundamental-term examples
The source dataset revision used for conversion was
7d7d516f780879dbe9f512ef29655f51548f7578.
Training procedure
- Full-parameter BF16 SFT
- 4 GPUs with DeepSpeed ZeRO-3
- 3 epochs
- Learning rate:
1e-6 - Scheduler: cosine
- Warmup ratio:
0.05 - Per-device batch size: 1
- Gradient accumulation: 4
- Effective global batch size: 16
- Maximum sequence length: 6,144 tokens
- Qwen3 non-thinking chat template
- Seed: 42
The run completed 2,418 optimizer steps. Final training loss was
0.7517516736258545. No held-out evaluation set was used during this run.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "XDQAQ/Qwen3-8B-MaKTO-Public-SFT"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto",
)
messages = [{"role": "user", "content": "请分析当前狼人杀局势。"}]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
enable_thinking=False,
return_tensors="pt",
).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
Use transformers>=4.51.0 for Qwen3 support.
Limitations
The model is specialized for language-game and Werewolf-style interactions. It has not been evaluated as a general assistant, and its outputs can be incorrect, inconsistent, or strategically deceptive in keeping with the training domain. Users should evaluate it for their own intended use.
References
License
The base model is distributed under the Apache License 2.0. The source dataset is marked as MIT on its Hugging Face dataset page. Users are responsible for complying with the licenses and terms of the base model and source dataset.