Nawah-VL-50M
An Arabic image-captioning VLM, assembled from existing parts rather than trained from scratch.
| Part | Source | Params | Trained |
|---|---|---|---|
| Vision tower | SigLIP2 (NaFlex) from LiquidAI/LFM2.5-VL-450M | 85.8M | frozen |
| Connector | Lfm2VlMultiModalProjector, 3072→2048→512 | 7.1M | yes |
| Language model | oddadmix/50M-2048-Emhotob (Llama, hidden 512, 12 layers, 2048 ctx) | 51.8M | stage 2 |
145M total. Generates Modern Standard Arabic captions for a single image.
Usage
from transformers import AutoProcessor, AutoModelForImageTextToText
from PIL import Image
model = AutoModelForImageTextToText.from_pretrained("oddadmix/Nawah-VL-50M")
proc = AutoProcessor.from_pretrained("oddadmix/Nawah-VL-50M")
inputs = proc(text=["<s><image>"], images=[[Image.open("photo.jpg").convert("RGB")]],
return_tensors="pt", padding=True, padding_side="left")
out = model.generate(**inputs, max_new_tokens=40)
print(proc.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])
There is no chat template — the prompt is literally BOS followed by one <image> token, which the
processor expands into 32–64 placeholders depending on aspect ratio.
Results
Held-out test split of visheratin/laion-coco-nllb:
| metric | value |
|---|---|
| chrF++ | 29.0 |
| BLEU | 9.5 |
| perplexity | 8.6 |
| grounding gap | 2.36 |
| distinct captions | 98.7% |
Grounding gap is eval loss with images rotated by one position minus loss with correct pairings. Near zero would mean the model ignores the image and generates from a language prior; 2.36 says the captions are driven by pixels.
Training
Two stages on 405,526 image/caption pairs (the arb_Arab slice of LAION-COCO-NLLB):
- Align — projector only, 2000 updates, LR 1e-3
- Finetune — projector + language model, 8000 updates, LR 2e-4 / 5e-5
The projector's linear_1 is warm-started from LFM2.5-VL (identical 2048×3072 shape); linear_2 is
randomly initialised and then rescaled so visual tokens enter the residual stream at the same
magnitude as text tokens. Without that calibration the visual tokens were ~36× larger and alignment
stalled.
Limitations
51.8M parameters in the language model (~35M non-embedding) is still well below a standard VLM backbone. Against the 25M variant it fixes most Arabic agreement errors and invents far fewer details, but it still reads text inside images poorly and will occasionally state something confidently wrong. The training captions are themselves doubly machine-generated (BLIP English → NLLB Arabic), which caps how much reference-based metrics mean.
Reproducing
Everything needed is in training/ in this repo. The filtered corpus is published at
oddadmix/laion-coco-nllb-arabic-filtered,
so data_prep.py can be skipped. training/HOWTO.md covers the four non-obvious parts, including
the projector scale calibration without which alignment stalls.
Set ARABIC_VLM_LM=oddadmix/50M-2048-Emhotob and run scripts/run_training_50m.sh.
Attribution
The language model is oddadmix/50M-2048-Emhotob, trained from scratch on Arabic. The vision
tower is SigLIP2 as distributed in LiquidAI/LFM2.5-VL-450M, used unmodified and frozen; those
weights remain LiquidAI's and their licence terms apply to that component.
License
cc-by-nc-4.0, inherited from the training data — non-commercial use only.
Compared with oddadmix/Nawah-VL-25M
| metric | 25M | 50M |
|---|---|---|
| chrF++ | 27.4 | 29.0 |
| BLEU | 8.2 | 9.5 |
| perplexity | 12.2 | 8.6 |
| grounding gap | 2.09 | 2.36 |
Same vision tower, same 405,526 pairs, same schedule -- only the language backbone differs. The grounding gap rising while loss fell is the part worth noting: the extra capacity went into using the image, not into memorising Arabic caption phrasing. A larger model that had simply learned the caption distribution better would show the opposite.
In practice the 50M fixes most of the 25M's Arabic agreement errors and invents far fewer details.
Where the 25M writes زجاجة ماء "سوبر" مع مقبض أحمر for a plain red bottle, this one writes
زجاجة ماء حمراء مع غطاء أبيض, which is correct on both colour and part.