GLM-OCR Farsi
Fine-tuned GLM-OCR for Persian / Farsi OCR (full end-to-end SFT: vision + projector + language model).
Training summary
| Item | Value |
|---|---|
| Base model | zai-org/GLM-OCR (~0.9B) |
| Stage 1 | LM-only full SFT → checkpoint-5500 |
| Stage 2 | Full e2e (vision + projector + LM) from stage-1 best |
| Hardware | 1× RTX 3090 24GB |
| Epochs (e2e) | 2 |
| Steps | 19,984 |
| Effective batch | 16 (batch=1, grad_accum=16) |
| Learning rate | 5e-6 cosine |
| Prompt | <image>Text Recognition: |
| Final eval_loss | 0.0317 |
| Train runtime | ~13h 41m |
Eval loss (e2e stage)
| Epoch | eval_loss |
|---|---|
| 0.10 | 0.268 |
| 0.50 | 0.121 |
| 1.00 | 0.057 |
| 1.50 | 0.037 |
| 2.00 | 0.0317 |
Files
model.safetensors— best weights (checkpoint-19984)- Tokenizer / processor configs for Transformers inference
training_loss.png,training_eval_loss.png— training curves
Usage
Requires transformers ≥ 5.x (GLM-OCR / glm_ocr support).
import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "larshiakzemil/glm-ocr-farsi"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
model_id, trust_remote_code=True, dtype=dtype, device_map=device
)
model.eval()
image = Image.open("page.jpg").convert("RGB")
messages = [{
"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": "Text Recognition:"},
],
}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt")
inputs = {k: v.to(device) if hasattr(v, "to") else v for k, v in inputs.items()}
with torch.inference_mode():
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
decoded = processor.decode(out[0], skip_special_tokens=True)
print(decoded.split("Text Recognition:", 1)[-1].strip())
Notes
- Optimized for handwriting / document text recognition with the official GLM-OCR text prompt.
- For full document layout parsing, use the GLM-OCR SDK with this checkpoint as the recognition model.