RightNow-Arabic-0.5B-Turbo

The smallest open Arabic-specialized decoder LLM

518M parameters | 398 MB on disk (q4_k_m) | 635 tok/s on H100

Built by RightNow AI

What is this?

RightNow-Arabic-0.5B-Turbo is a 518M-parameter Arabic-specialized language model built on top of Qwen2.5-0.5B via vocabulary injection, continued pretraining, supervised fine-tuning, and direct preference optimization. It is the smallest open Arabic-specialized decoder LLM with publicly available weights.

The model targets edge deployment: phones, laptops, embedded devices, and browsers. Quantized to q4_k_m it fits in 398 MB and generates 635 tokens/s at batch size 1.

Key Features

• 27,032 new Arabic tokens added via mean-subtoken initialization, cutting Arabic tokenizer fertility by 17.3% (2.18 to 1.80 tokens/word)
• 504M Arabic pretraining tokens (Arabic Wikipedia) on 8xH100 SXM5 with FSDP + FlashAttention varlen + Liger fused kernels
• 129,116 Arabic instruction pairs for SFT with response-only loss masking
• 6,750 Arabic preference pairs for DPO
• Weight soup merging (DPO 50%, SFT 25%, Pretrain 25%) for optimal accuracy
• 4 GGUF quantizations for instant edge deployment

Benchmarks

Evaluated with lm-evaluation-harness v0.4.11, apply_chat_template=True, limit=200, acc_norm preferred.

Head-to-head comparison

Among 0.5B models: best on COPA-ar (+4.5 vs Qwen), best on HellaSwag-ar (+3.5 vs Qwen), best mean (+1.8 vs Qwen, +5.2 vs Falcon).

Ties Falcon-H1-1.5B on COPA-ar (both 58.4%) at one-third the parameters.

Recovers 67% of SILMA-9B mean accuracy at 5.8% of the parameters.

Available Formats

Quick Start

With Transformers (bf16)

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "RightNowAI/RightNow-Arabic-0.5B-Turbo"
tokenizer = AutoTokenizer.from_pretrained(model_id, subfolder="bf16")
model = AutoModelForCausalLM.from_pretrained(
    model_id, subfolder="bf16",
    torch_dtype="bfloat16", device_map="auto"
)

messages = [
    {"role": "system", "content": "أنت مساعد ذكي يجيب باللغة العربية الفصحى"},
    {"role": "user", "content": "ما هي عاصمة المملكة العربية السعودية؟"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

With llama.cpp (GGUF)

# Download the q8_0 quantization (best speed)
huggingface-cli download RightNowAI/RightNow-Arabic-0.5B-Turbo \
  gguf/RightNow-Arabic-0.5B-Turbo-q8_0.gguf --local-dir .

# Run inference
./llama-cli -m RightNow-Arabic-0.5B-Turbo-q8_0.gguf \
  -p "ما هي أكبر مدينة في مصر؟" \
  -n 128 --temp 0.7

Training Pipeline

Qwen2.5-0.5B (494M, 151,665 vocab)
        |
        v
Tokenizer Surgery (+27,032 Arabic tokens -> 178,697 vocab)
  - SentencePiece unigram 32k on 12.5 GB Arabic corpus
  - Mean-subtoken embedding initialization
  - Fertility: 2.18 -> 1.80 tok/word (-17.3%)
        |
        v
Continued Pretraining (504M arwiki tokens)
  - 2,500 steps, 8xH100 SXM5
  - FSDP _HYBRID_SHARD_ZERO2 + FlashAttention varlen + Liger
  - Loss: 14.21 -> 1.69 | Wall time: 6h 57m
        |
        v
Supervised Fine-Tuning (129,116 instructions)
  - 5 datasets: evol-instruct-arabic, alpaca-gpt4-arabic,
    sharegpt-arabic, CIDAR, aya_dataset
  - Response-only loss masking (72.1% of tokens carry loss)
  - 5 epochs, 418 steps | Wall time: 12m
        |
        v
Direct Preference Optimization (6,750 pairs)
  - argilla-dpo-mix-7k-arabic
  - 2 epochs, 844 steps | Wall time: 34m
        |
        v
Weight Soup Merging
  - Linear(DPO 0.5, SFT 0.25, Pretrain 0.25)
  - +0.44 points mean accuracy over DPO alone
        |
        v
Export: bf16, int8, GGUF {f16, q8_0, q5_k_m, q4_k_m}

Training Data

Limitations

• Knowledge ceiling: At 518M parameters, ArabicMMLU-style knowledge tasks lag 7B+ models by 12-30 points. This is a parameter-count limit, not a training limit.
• MSA only: Trained on Wikipedia (Modern Standard Arabic). Dialects (Egyptian, Gulf, Levantine) get MSA responses.
• 504M pretraining tokens: Below Chinchilla-optimal ratio. More Arabic data would improve knowledge tasks.
• DPO was weak: 6,750 machine-translated preference pairs provided minimal signal at 0.5B scale. The weight soup merge was more impactful.
• GGUF tile alignment: q4_k_m and q5_k_m fall back to higher-bit quantization for 144/290 tensors due to the expanded vocabulary not aligning with k-quant tile sizes.

Hardware

All training ran on a single Nebius gpu-h100-sxm node:

• 8x NVIDIA H100 80 GB SXM5 HBM3, NVLink4
• 128 vCPUs, 1.5 TiB RAM
• CUDA 13.0, PyTorch 2.11, flash-attn 2.8.3, transformers 5.5.0

Citation

@article{jaber2025rightnow,
  title={RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment},
  author={Jaber, Jaber and Jaber, Osama},
  year={2025},
  url={https://huggingface.co/RightNowAI/RightNow-Arabic-0.5B-Turbo}
}

License

Apache 2.0 (same as the base Qwen2.5-0.5B model).

Built by RightNow AI