Mizan 1.5 · 4B · میزان

▶ Try it live: Mizan demo Space · by Nimruz

A Persian-first decision model. You give it a record (text or JSON) and a set of typed questions. It returns a calibrated probability for every allowed answer, in one forward pass. There is no text generation, no parsing and no hallucinated labels.

Mizan (میزان) means "scale" or "balance" in Persian: the thing you weigh a decision on.

• 82.0% on real Persian data, up from 78.7% in v1. That's within 1–2 points of Jev and Clef-Flash 9B, and Mizan has the best-calibrated confidence of every model tested.
• Built for Persian business decisions: moderation, support triage, review analysis, fraud SMS, marketplace listings, lead routing, chatbot guardrails, RAG checks and CV screening. It still handles English.
• Jev / SystemOne compatible: the same request and response body as /v1/systemone.
• Runs locally: 4B parameters. The 2.7 GB Q4 GGUF scores 82.1%: Mizan-1.5-4B-GGUF.
• Apache-2.0.

What's new in 1.5

v1's mistakes were found on about 18k real Persian records, taken from public datasets' training splits and never from the benchmark. A blind LLM review diagnosed each one: is the model wrong, the dataset label wrong, or the case ambiguous? Noisy labels were dropped. For every real model mistake, 3 new examples teaching the same lesson were written:

• the same failure in new wording;
• a near-identical text whose answer flips (a contrast pair);
• a fresh question wording.

That added 1,362 new examples plus 454 confirmed hard real rows.

Fixed weak spots:

• Treating political criticism as offensive.
• Answering positive or negative for aspects a review never mentions.
• Calling a claim "supported" when it swaps a detail.
• Confusing alarms and reminders.

Digikala Mag was deliberately kept out of the whole process, as an overfitting check. It went up too.

v1 is still available under the v1 tag of this repo.

Quick look

{
  "state": {"message": "سلام، دو بار از کارتم پول کم شده ولی سفارشم ثبت نشده. لطفا سریع پیگیری کنید",
            "customer": {"orders": 14, "vip": true}},
  "questions": {
    "team":    {"type": "choice", "instructions": "کدام تیم باید این پیام را بررسی کند؟",
                "criteria": {"billing": "مالی و بازگشت وجه", "delivery": "ارسال و پیک",
                             "technical": "خطای اپ یا سایت", "other": "سایر موارد"}},
    "urgency": {"type": "score", "instructions": "این درخواست چقدر فوری است؟", "criteria": ["عادی", "مهم", "خیلی فوری"]},
    "refund":  {"type": "noul",  "instructions": "آیا مشتری درخواست بازگشت وجه دارد؟"}
  }
}

Each answer comes back with probabilities over every option: choice returns choice, confidence and probabilities; score returns score, legend and probabilities; noul returns the probability of "yes". Questions can be in Persian or English, and they can be new questions the model has never seen.

Inference

llama.cpp (recommended, CPU or GPU)

Needs llama.cpp b11443 or newer.

huggingface-cli download xmanii/Mizan-1.5-4B-GGUF mizan-1.5-4b-Q4_K_M.gguf --local-dir .
llama-server -m mizan-1.5-4b-Q4_K_M.gguf -c 8192 -b 8192 -ub 8192 -ngl 99 --port 8080

curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": {"review": "غذا سرد رسید و پیک هم خیلی بداخلاق بود"},
  "questions": {"sentiment": {"type": "choice", "instructions": "حس کلی مشتری چیست؟",
                "criteria": {"positive": "راضی", "negative": "ناراضی"}}}}'

A decision reads the whole input in one batch, so -ub sets the longest input you can send. The GGUF carries the decision head and the calibrated temperatures.

Python (Unsloth)

from unsloth import FastDecisionModel

model, processor = FastDecisionModel.from_pretrained("xmanii/Mizan-1.5-4B", max_seq_length=4096)
FastDecisionModel.for_inference(model)
answers = FastDecisionModel.predict(model, processor, state, questions)
print(answers["team"]["choice"], answers["team"]["probabilities"])

This applies the calibrated temperatures, so it gives the same answers as the GGUF. Tested with unsloth==2026.10.3.

Plain transformers

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("xmanii/Mizan-1.5-4B")
sys.path.insert(0, path)
from joint_schema_model import load_release_model, systemone

model, processor = load_release_model(path, device="cuda")
print(systemone(model, processor, {"model": "mizan", "state": state, "questions": questions}))

This path returns uncalibrated probabilities: the top answer is the same, but confidences are sharper. To calibrate them, divide each question's logits by unsloth_decision_config.json's temperature for its type (order: choice, score, noul).

Evaluation on real Persian data

3,000 real, human-labeled records from 7 public Persian datasets. None was used for training, and every model gets identical rows and questions.

Reading it honestly:

• Mizan 1.5 is within 1–2 points of models 2–7× its size.
• It matches or beats Jev on aspect questions, intent routing and article topics.
• Its biggest remaining gap is strict claim checking (FarsTail).
• The mistake mining used the training splits of the same public datasets, so part of the gain on those six tasks may come from familiarity with their style. The untouched Digikala Mag result suggests most of it is real.
• In the blind review, about half of v1's SnappFood and Digikala "errors" turned out to be wrong dataset labels, so roughly 87% on SnappFood is close to that dataset's real ceiling.
• The datasets are public, so the larger models may have seen some of them in training.

Confidence gating works. Auto-decide the records Mizan is sure about, and send the rest to a human:

Training

Data (17.6k decisions, about 70% English and 30% Persian):

• English: Unsloth's decision mixture (BANKING77, CLINC150, MNLI, SNLI, WANLI, BoolQ, AG News, SST-5, MMLU, CommonsenseQA, ARC, prompt injections) plus typed-decisions, to keep the model general.
• Persian business workflows: 6,660 records written to read like real Iranian users (colloquial text, typos, Finglish, emoji), with realistic context such as order data, seller stats, a bot policy or a job's requirements. Workflows: comment moderation, product reviews, app feedback, support routing, SMS fraud, marketplace listings, sales leads, chatbot guardrails, RAG relevance and CV screening.
  • Labels: each record was labeled blind by a hosted decision model, and disagreements with the intended answer went to a blind LLM review. The main question trains on the reviewed intended answer.
  • Question variety: 6 Persian paraphrases per question and rotated option IDs, plus 1,568 custom, record-specific questions.
• New in 1.5: 1,362 examples written around v1's real mistakes, plus 454 confirmed hard real rows (see above).
• Leak control: each record and its copies are split together, training rows are decontaminated against every eval set, and benchmark rows never enter the mistake search.

Limitations

• Most Persian training data is synthetic. Evaluate on 200–300 of your own labeled examples before automating.
• Weakest area: strict entailment (is a claim fully supported?). Large intent taxonomies are better but still behind Clef-Flash.
• Text only. Image inputs are coming in Mizan v2.

License

Apache-2.0, the same as the Qwen3.5 base model.