Kiwen-27B
A fine-tune of Qwen/Qwen3.8-27B trained on long chain-of-thought traces from Kimi K3.
This is not a broadly stronger model. The fine-tune primarily teaches Qwen3.8-27B to terminate its reasoning and produce an answer before exhausting its generation budget.
Model Summary
Qwen3.8-27B is a strong reasoning model that can sometimes spend its entire token budget deliberating without producing a final answer.
On GSM8K, this behavioral change produces a +10.31 percentage-point improvement under flexible extraction.
Key results
| Benchmark | Qwen3.8-27B | Kiwen-27B | Delta | t |
|---|---|---|---|---|
| GSM8K exact_match (flexible) | 0.7407 | 0.8438 | +10.31 | +6.58 |
| GSM8K exact_match (strict) | 0.6672 | 0.7278 | +6.07 | +3.40 |
| IFEval prompt-level strict | 0.8226 | 0.8429 | +2.03 | +0.90 |
| IFEval prompt-level loose | 0.8447 | 0.8669 | +2.22 | +1.04 |
| IFEval instruction-level strict | 0.8393 | 0.8645 | +2.52 | — |
| IFEval instruction-level loose | 0.8537 | 0.8801 | +2.64 | — |
| VMLU val (744 Vietnamese MMLU questions) | 83.47 | 86.02 | +2.55 | +1.37 |
Evaluated using lm-evaluation-harness 0.4.12 with both models served on identical hardware:
- Hardware: 1× H200
- Inference: SGLang
- Model dtype: BF16
- KV cache: FP8
- Generation limit:
max_gen_toks=4096 - Prompts: identical
- Decoding settings: identical
The largest improvement is on GSM8K. Other benchmarks show smaller but consistently positive changes, with no measured regression.
The Mechanism: Learning When to Stop
GSM8K is evaluated using two extraction methods:
- Strict match: requires the canonical
#### <number>answer format. - Flexible extraction: extracts the final number from the generated response.
The improvement under flexible extraction is substantially larger:
+10.31 points flexible vs. +6.07 points strict
This asymmetry suggests that much of the improvement comes from successfully producing an answer before the generation budget is exhausted, rather than from a fundamental improvement in mathematical reasoning.
Out-of-Domain Regression
To test whether the fine-tuning damaged unrelated capabilities, both models were evaluated on 100 held-out Vietnamese enterprise documents covering:
- Accounting
- CRM
- Internal RAG workloads
The metric was token-level negative log-likelihood:
| Model | NLL / token |
|---|---|
| Qwen3.8-27B | 3.3390 |
| Kiwen-27B | 3.3543 |
Paired difference:
- Δ: +0.0067
- Standard error: 0.0041
- t: 1.65
Usage
from transformers import AutoModelForImageTextToText, AutoProcessor
model = AutoModelForImageTextToText.from_pretrained(
"beyoru/Kiwen-27B",
dtype="bfloat16",
device_map="auto",
)
proc = AutoProcessor.from_pretrained(
"beyoru/Kiwen-27B"
)
MTP Tensors
MTP tensors are absent from the released model.
transformers declares:
_keys_to_ignore_on_load_unexpected = [r"^mtp.*"]
Therefore, the multi-token-prediction block is dropped by HF-based merging, including this release.
For speculative decoding, use an external draft model such as z-lab/Qwen3.8-27B-DFlash2 rather than NEXTN.
A draft model trained against the original weights may have a lower acceptance rate against the merged Kiwen weights. This does not affect correctness because speculative decoding uses rejection verification.
Benchmark Caveat
VMLU validation is publicly distributed with answer keys and may have some representation in pretraining data.
Its absolute score should therefore be interpreted cautiously.
However, the base-versus-fine-tuned comparison remains informative, since both models are exposed to the same potential contamination.
Citation
@misc{kiwen27bk3,
title = {Kiwen-27B},
author = {beyoru},
year = {2026},
url = {https://huggingface.co/beyoru/Kiwen-27B-K3}
}
License & Attribution
Built on:
- Qwen3.8-27B — Apache-2.0
Kiwen-27B is released under the Apache-2.0 license.