Jev-AR

A 322M local decision model for fast Arabic intent routing across MSA, Gulf dialects and code-switched Arabic

83.4% on 60-way Gulf-Arabic routing · ~10 ms per query on one GPU · 322M parameters · runs locally

Supply your routes at request time. Jev-AR scores them jointly in one forward pass and returns the best route, a probability for every route, and an escalation signal for when it isn't confident enough to act.

Highlights

• Close to a closed API, at a fraction of the latency. It is 2.8 points from Jev 1.13 on 60-way routing, at about 10 ms per query on one GPU.
• On par with Jev on 20-route lists. 90.0% vs 91.4%; the difference is not statistically significant.
• The strongest model under 1B parameters we evaluated. It is 6.9 points above a classifier fine-tuned from the same encoder, and 11.0 points above multilingual-E5 with logistic regression.
• Built on Laya multilingual. Jev-AR is fine-tuned from Laya multilingual, an open decision model with the same architecture. Our Arabic training takes it from 42.0% to 83.4% on 60 routes, and from 13 to 38 of 40 on the unseen-domain test.
• Early interest. In its first week (25 Sep–1 Oct 2026), 12 people in 7 countries requested access to the weights, mostly for research, and the benchmark was downloaded 436 times.
• Your routes, no retraining. It routes 38 of 40 decisions correctly on a kind of RAG app never seen in training, and no answer changes when the route list is reversed.
• Built for production. It has calibrated confidence and a 95%-accuracy escalation threshold, and it is about 1% the size of a 27B LLM.

Quick start

jev_ar_route.py is one self-contained file in this repo (also on GitHub). It needs only torch, transformers and huggingface_hub.

pip install torch transformers huggingface_hub
hf auth login          # once your access request is approved (automatic)
python jev_ar_route.py "ابي اجدد الاقامة"

from jev_ar_route import JevAR

router = JevAR()  # downloads the model on first use

tools = {  # an assistant's tools: any routes you like, given at request time
    "Calendar": "Create, move or cancel meetings and reminders.",
    "Email": "Write, reply to or send emails.",
    "Web search": "Look up current or public information: news, prices, facts.",
    "Human agent": "The user asks for a person, or the request is sensitive or legal.",
}
router.route("وش آخر أخبار أسعار الذهب اليوم؟", tools)
# {'route': 'Web search', 'confidence': 0.88, 'escalate': False, 'probabilities': {...}}

Called without a list, Jev-AR routes over 60 built-in routes: 58 Gulf government services (residency, IDs, traffic, labour, business, housing, health, education and more), plus other service and out of scope.

router.route("ابي اجدد الاقامة")
# {'route': 'تجديد الإقامة', 'id': 'renew_residence', 'category': 'residency', 'confidence': 0.78,
#  'escalate': False, 'in_scope': 0.998, 'probabilities': {...}}

Tips. Treat escalate: True (top probability below 0.72) as "hand off to a person or a fallback". Keep the instruction plus the route list under 512 tokens: about 60 short names, or about 15–20 routes with one-line descriptions (fewer in Arabic). The helper raises an error rather than silently cutting routes. On a CPU, call torch.set_num_threads(8) first. The full API is on GitHub.

How it works

The message and the routes form one input sequence, with a marker token before each route. The mmBERT-base encoder reads them together, and a 15M decision head (Laya's design) scores every route at its marker. A softmax with a temperature fitted on development data turns the scores into probabilities. Routes are part of the input rather than fixed output classes, so a new list needs no retraining.

Results

Gulf-Arabic routing (Jev-AR Bench): 4,974 held-out queries in MSA, Emirati, Saudi and code-switched Arabic. Accuracy in %, with 95% confidence intervals.

Batch 1. Local models ran on an RTX 5090 Laptop GPU. † Client-side round trip, network included. ‡ The open model Jev-AR was fine-tuned from, as released. Every system gets the same inputs, so Laya's route budget was raised from its default 256 tokens to Jev-AR's 512.

Unseen-domain routing (Jev-AR Bench): one RAG app of a kind never seen in training, with 6 routes, and each question routed over an English and an Arabic route list (40 decisions).

• Macro-F1: 86.4 (60 routes). Category accuracy: 92.4.
• Order robustness: 5.0% of answers change when the 60 routes are shuffled (Jev 1.13: 4.7%).
• Escalation: at a threshold fitted on development data for 95% accuracy, 85.8% of test queries are answered automatically, at 89.7% accuracy.
• Label-agreement subset (queries where an independent labeller confirms the gold route): 86.0%.

• The test set was frozen before training (SHA-256 published on the dataset card). It was written by a different model family than the training data, and it includes adversarial queries, routes held out from training, and requests outside the route list.
• Every system sees the same route lists. Temperatures and thresholds are fitted on development data only.
• Confidence intervals come from a bootstrap that resamples whole generation batches. Differences are checked with paired tests against each baseline.
• Data, a snippet to evaluate your own router, and per-category results: jev-ar-bench.

Model details

Limitations

• Trained on synthetic Arabic routing data, which has not yet been validated by native speakers.
• Confidence is calibrated on the built-in routes, so treat it as approximate on custom lists.
• It is not an official channel of any government or company. Keep a person in the loop for decisions with legal, medical or financial consequences.

Licence

CC BY-NC 4.0 (LICENSE): free for non-commercial use, testing and evaluation, with attribution. Commercial use requires a licence; contact ayoubatmane23@gmail.com. Third-party credits are in NOTICE.

Citation

@misc{jevar2026,
  title        = {Jev-AR: Arabic Intent Routing for MSA, Gulf Dialects and Code-Switching},
  author       = {atmaneayoub},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/atmaneayoub/jev-ar}}
}