Qwen3-0.6B-khm-ft3 — a small Khmer model for on-device RAG
A 0.6B Khmer-focused language model fine-tuned from Qwen3-0.6B, built to run on a phone, offline — it's the answering model behind iAny and runs on a 2019 Galaxy S10.
The key trick: the base Qwen3 vocabulary (~150k tokens) is trimmed to ~32k Khmer-focused tokens, which shrinks the output logits buffer enough to fit weak devices while keeping Khmer coverage. It is then continued-pretrained and instruction-tuned on Khmer.
Lineage
| Stage | What | Data |
|---|---|---|
| Base | Qwen3-0.6B | — |
| Vocab trim | ~32k Khmer-focused vocabulary (smaller logits → fits low-RAM phones) | via alphaedge-ai |
| CPT | Continued pre-training for Khmer fluency | FineWeb-2 (Khmer) + ParaCrawl |
| SFT (ft3) | Instruction / Q&A tuning for correct, fuller answers | sengtha/khmer-qa |
ft3 is the current release — retrained on a richer khmer-qa for fuller
answers than the earlier ft/ft2 checkpoints.
Intended use
Grounded Khmer question-answering / RAG on-device: the app retrieves context from the user's own documents and this model writes the answer in Khmer. Small and Khmer-first — not a general-purpose assistant, and it can be wrong or hallucinate, especially without retrieved context.
Usage (transformers)
from transformers import AutoModelForCausalLM, AutoTokenizer
m = "sengtha/Qwen3-0.6B-khm-ft3"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m)
msgs = [{"role": "user", "content": "តើភ្នំពេញជាអ្វី?"}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(inputs, max_new_tokens=256)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
Uses the standard Qwen3 chat template (ChatML). Append /no_think to a
message to skip the reasoning block for faster, direct answers.
On-device (GGUF)
Quantized GGUF builds (Q4_K_M + Q8_0) for llama.cpp / llama.rn live in
sengtha/Qwen3-0.6B-khm-ft3-Q8_0-GGUF.
Q4 (~0.4 GB) fits weak phones; Q8 is higher quality.
License & attribution
Apache-2.0, inherited from the base Qwen3-0.6B
(© Alibaba, Apache-2.0). 32k-vocab base via alphaedge-ai. Training corpora:
FineWeb-2, ParaCrawl, and sengtha/khmer-qa. Please keep this
attribution in derivatives.