CantoneseLLM-v2.0-30B-A3B-Thinking

This is the flagship release of CantoneseLLM v2. It is a 30.5B-parameter mixture-of-experts model (3.3B activated) based on Qwen3-30B-A3B that reasons and answers in Hong Kong Cantonese.

It is the final checkpoint of a five-stage pipeline: continual pre-training → chat-vector merge → supervised fine-tuning → DPO → two-stage RLVR. The full development history, including the stages that failed and why, is documented in the technical report.

📄 Paper: CantoneseLLM v2: Reasoning in a Low-Resource Language (arXiv:2609.06970)

🧪 Evaluation Benchmark: hon9kon9ize/hkeval2025

Reasoning in Cantonese

Open models prompted in Cantonese answer in Cantonese but reason in another language. Here is the same translated GSM8K probe, under a system prompt that explicitly asks for Hong Kong Cantonese.

Chat-vector merged checkpoint — opens in Cantonese, then slides into Written Chinese and ends up discussing the output language instead of thinking in it:

…現在，用香港廣東話回應。…在回應中，要用口語廣東話。

(…Now, respond in Hong Kong Cantonese. … In the response, colloquial Cantonese should be used.)

This model — reasons in Cantonese end to end, with the Cantonese classifier 隻 zek3 and colloquial forms such as 剩低 zing6 dai1:

首先，我要拆解呢條數學題嘅步驟。題目話 Janet 每日生 16 隻蛋，佢朝早食 3 隻，焗鬆餅用咗 4 隻，剩低嘅就去農夫市場賣。

(First, I have to break down the steps of this arithmetic problem. It says Janet's ducks lay 16 eggs a day, she eats 3 in the morning, 4 go into baking muffins, and what is left is sold at the farmers' market.)

Both reach the correct answer, 18 dollars. What changes is where the reasoning happens.

The mechanism is a multiplicative language and script term in the RLVR reward — task_reward × language_multiplier × format_factor — so a wrong-language trace cannot buy its way back by being correct.

Trace length over eight fixed probes (mean tokens between the reasoning tags):

The official and merged checkpoints are predominantly Written Chinese across those probes. This model's traces are Cantonese — and short, which is a real limitation rather than a design goal (see below).

Benchmark results

HKCanto-Eval (Cheng et al., 2025), reasoning mode on:

Read this table carefully. SFT cost 12.46 points; RLVR recovered 8.99 of them, 88% of what was lost after the merge. The model finishes 1.20 points below the merged checkpoint it started from — while carrying the Cantonese reasoning behaviour that the merged checkpoint never had, plus translation, data-judging and Cantonese lexical capability that multiple-choice questions do not measure.

If you select on benchmark average alone you will pick the merged checkpoint and get a model that reasons in Written Chinese. That trade is the subject of the paper.

Artefacts released with this work

Every post-training stage on this card is reproducible from public components. The CPT corpus itself is not released, but its two largest public constituents are.

The three-way parallel structure of the RL corpus is what makes the cross-language pass-rate gap reported in the paper measurable: for the translated environments, prompt language is the only variable.

Other models in this release

All four checkpoints are in the CantoneseLLM v2.0 collection.

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "hon9kon9ize/CantoneseLLM-v2.0-30B-A3B-Thinking"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")

SYSTEM = "你係CantoneseLLM，一個由Hon9Kon9ize開發嘅語言模型，請使用香港嘅廣東話回答用家問題"

messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "小明有 5 個蘋果，佢俾咗 2 個朋友，每人 1 個，跟住又買多 3 個。佢而家有幾多個蘋果？"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95)
print(tokenizer.decode(out[0][len(inputs.input_ids[0]):], skip_special_tokens=True))

vLLM

The stock command works — no special flags are required:

vllm serve hon9kon9ize/CantoneseLLM-v2.0-30B-A3B-Thinking

Add --tensor-parallel-size N to shard across N GPUs, and --reasoning-parser qwen3 if you want the reasoning block returned separately as reasoning_content rather than inline in content (parser names vary by vLLM version).

OpenAI-compatible API

The served endpoint speaks the OpenAI protocol, so the official client works unchanged:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

SYSTEM = "你係CantoneseLLM，一個由Hon9Kon9ize開發嘅語言模型，請使用香港嘅廣東話回答用家問題"

resp = client.chat.completions.create(
    model="hon9kon9ize/CantoneseLLM-v2.0-30B-A3B-Thinking",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "小明有 5 個蘋果，佢俾咗 2 個朋友，每人 1 個，跟住又買多 3 個。佢而家有幾多個蘋果？"},
    ],
    temperature=0.6,
    top_p=0.95,
    max_tokens=2048,
)

msg = resp.choices[0].message
print(getattr(msg, "reasoning_content", None) or "")  # populated with --reasoning-parser
print(msg.content)

Or with curl:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hon9kon9ize/CantoneseLLM-v2.0-30B-A3B-Thinking",
    "messages": [
      {"role": "system", "content": "你係CantoneseLLM，一個由Hon9Kon9ize開發嘅語言模型，請使用香港嘅廣東話回答用家問題"},
      {"role": "user", "content": "點解香港嘅雨季集中喺五月到九月？"}
    ],
    "temperature": 0.6, "top_p": 0.95, "max_tokens": 2048
  }'

Sampling: temperature 0.6, top-p 0.95 — the settings used for every evaluation reported above and in the paper. Avoid greedy decoding.

System prompt: the Cantonese system prompt above was used throughout training and evaluation. Behaviour with other system prompts, or with none, is not characterised.

This is a thinking-only model. Unlike the 8B, its SFT mixture was entirely chain-of-thought, so it has not been trained to answer without a reasoning block. Do not expect /no_think to behave as it does in the stock Qwen3 hybrid models.

Training pipeline

Stage 2 took four attempts; the first three (≈775 GPU-hours) were abandoned to entropy collapse and importance-sampling problems. Attempt 4 is this checkpoint. The failure analysis is in the paper — it is one of the more useful parts.

Released weights are the final step of stage 2, not a best-validation checkpoint.

Risks & Limitations

• Short reasoning traces. 155 tokens on average. No long original Cantonese reasoning traces existed at the scale SFT needed; human-written or human-verified traces amounted to 131 rows. Most Cantonese CoT in training was machine-translated from English or Simplified Chinese. The model reasons in the right language but does not reason at length.
• No Hong Kong grounding in the reasoning data. The SFT mixture contained no content grounded in Hong Kong entities or current events — that knowledge comes from CPT only.
• Benchmark scores are underestimates of knowledge. DPO's pair-selection weighted longer responses with no reward for instruction compliance, so the model sometimes ignores "answer with the letter only". Multiple-choice parsers that read a bare leading option letter will under-score it. Use a marker-anchored extractor.
• Language steering may over-fire toward Cantonese. At the intermediate stage-1 checkpoint the model produced Cantonese in 87.1% of rollouts on mathematics even when explicitly asked for Written Chinese. Stage 2 trained all three languages with the multiplier, but test this if you need reliable zh-hk output.
• One run per stage. Severe compute constraints meant no hyperparameter sweep in any post-training stage.
• Preference data were model-judged, not human-annotated, and are bounded by the judge's own Cantonese ability.
• Long context is inherited and untested. Training sequence lengths were 8,192 (SFT/DPO) and 16,384 (RLVR stage 2). Behaviour beyond that is whatever the Qwen3 base provides.
• Standard LLM caveats apply: it will hallucinate, and it has not been safety- tuned beyond what the chat vector carried over.

Citation

@misc{cantonesellm_v2,
      title={CantoneseLLM v2: Reasoning in a Low-Resource Language},
      author={Tsz Chung Cheng and Chung Shing Cheng and Chaak Ming Lau and Cheuk Hei Chong},
      year={2026},
      eprint={2609.06970},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2609.06970},
}

Acknowledgements

Continual pre-training (CPT) was carried out on Cloud TPUs (Tensor Processing Units) from Google's TPU Research Cloud with MaxText. Post-training was carried out on computer resources offered under the category of General Projects by Research Institute for Information Technology, Kyushu University. Usage fee and cost of data-curation costs with proprietary APIs were covered by Votee AI

The RLVR stage builds on NVIDIA's NeMo-RL and NeMo-Gym, and on the Nemotron post-training and RL datasets.