Nawah-Math-Reasoning — نموذج استدلال رياضي عربي

A 51.8M-parameter Arabic math reasoning model. It writes its derivation step by step inside <think>…</think>, then gives the answer. It is small enough to run on a CPU.

بالعربية: نموذج عربي صغير (~52 مليون معامل) لحل المسائل الحسابية: يكتب خطوات تفكيره داخل وسم <think> ثم يعطي الإجابة. صغير بما يكفي ليعمل على المعالج (CPU).

Results

Number agreement, greedy decoding. Every cell is measured on identical held-out rows. The Arabic_Reasoning and GSM8K-ar rows are the eval splits fixed at the start of the project and never re-drawn; the synthetic rows are pinned to the same 1,000 items every earlier version was scored on.

The v3 / v4 / v5 columns are internal development runs, kept here because they are what makes the release number mean something. They are not published — the numbers are, so the ablation is readable without them.

The relational row is what this release adds. On problems whose difficulty is the relation between quantities (ضعف, نصف, أكثر بـ…) rather than the arithmetic, it scores 52.2% where the previous run scores 34.0% — a +18.2 point gain and the largest single-cell move anywhere in the development ladder. It did not cost the other distributions: GSM8K-ar is simultaneously the best of the series at 79.0%, and synthetic math gains +1.3.

The one regression is Arabic_Reasoning at -2.2 against v5 — on 400 rows that is close to sampling noise, but it is the second consecutive mix where this column is the give.

(the synth-math column here is the 400-row mix cell; the 40.4% in the table above is the 1,000-row set used for the cross-model comparison.)

Reproduce any cell with code/eval_reasoning.py — it is the same script for every model and every row, which is the only reason these are comparable.

The final checkpoint ships, and eval loss disagrees

Loss bottoms at 0.4559 (epoch 1.86) and rises to 0.5154 by epoch 5 — yet the epoch-5 weights are the better model. This was measured directly on an earlier run whose corpus contained no repeated rows, which rules out memorisation: the minimum-loss checkpoint scored 30.9% where the final scored 35.6%. It happened on four consecutive runs. train_reasoning.py therefore takes LOAD_BEST=0, and that is not an oversight.

Training mix

275,639 rows, 31.1M tokens/epoch:

Of the synthetic corpus's 120,462 rows, 20,139 are relational problems generated specifically for this release, after a pass@k diagnostic showed the previous model went 0/8 on ضعف-style problems and a corpus audit found the relation appears in only 1.34% of rows. The synthetic eval split was pinned, not re-drawn when those rows were added: re-shuffling would have moved 1,955 of the 2,000 previously held-out items into train, turning that column into a memorisation score.

Full fine-tune from the base (not from the previous version). Loss on the assistant turn only, user prompt masked with -100. Arabic_Reasoning is ~25× smaller than GSM8K, so it is repeated 3×.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "oddadmix/Nawah-Math-Reasoning"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).eval()

messages = [{"role": "user", "content": "اشترى خالد 4 دفاتر بسعر 15 جنيهًا للدفتر، ودفع بورقة 100 جنيه. كم المبلغ المتبقي؟"}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt")

out = model.generate(**ids, max_new_tokens=384, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=False))

Split the parts with re.match(r"\s*<think>(.*?)</think>(.*)", completion, re.S). Decode with skip_special_tokens=False — <think> and </think> are real tokens in this tokenizer, and stripping them destroys the split.

It is single-turn: one user message per call. Chat history is out of distribution.

Answer style is not something you can request. The three corpora disagree — GSM8K rows end in a bare numeral, the other two in an إذن، … sentence — and arithmetic word problems look alike in all of them, so the model picks a style per prompt. Score it on number agreement, not exact string match, and parse the answer by extracting its numbers.

Limitations

At ~52M parameters this is a proof of concept, and the honest headline is the synthetic columns — 40.4% and 52.2% on multi-step problems, well below the 79.0% it scores on GSM8K's narrower phrasing. Arithmetic is the dominant failure mode: the reasoning is usually structurally right, one computation step is wrong, and the model then stays faithful to its own bad number.

Each corpus brings its own defect. The GSM8K half is machine-translated, its 140,969 rows expanding from only 2,814 question patterns, so that score partly reflects narrow phrasing. The synthetic half is verified for arithmetic, not for sense — rows survive where every equation checks out but a step introduces an entity never mentioned, or the answer resolves the reverse of what was asked. The Arabic_Reasoning half excludes open-ended expository rows (they have no final answer to place after </think>), so expository prompts remain out of distribution.

Everything is MSA; the synthetic corpus's region axis sets currency and context, not dialect. The Arabic inherits source artifacts including inconsistent gender agreement. Its reasoning trace is not a faithful account of any internal computation. Do not use it for anything consequential.

Citation

@misc{nawah_math_reasoning_2026,
  title  = {Nawah-Math-Reasoning: a 52M-parameter Arabic chain-of-thought math model},
  author = {Ahmed Wasfy},
  year   = {2026},
  url    = {https://huggingface.co/oddadmix/Nawah-Math-Reasoning}
}