Rukopys-OCR-4B

Rukopys-OCR-4B is an open vision-language model for Ukrainian handwritten document OCR. It detects document regions, classifies them, and returns their transcriptions as structured JSON.

The model was created for the Handwritten to Data competition and was used in the 3rd-place final solution. It is a full fine-tune of Qwen3.5-4B.
For training, evaluation, and ensemble details, see the competition writeup.

Output

[
  {
    "bbox": [84, 107, 912, 168],
    "type": "handwritten",
    "text": "Приклад рукописного тексту"
  }
]

bbox is [x1, y1, x2, y2] in normalized 0..1000 coordinates. Valid types are handwritten, printed, formula, table, annotation, image, and graph. Formula text uses LaTeX; table text is pipe-separated; image and graph use empty text.

Inference

Use Transformers 5.8.1 or newer. The exact prompt used for training is included below and should be kept unchanged.

Transformers

from PIL import Image
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

MODEL_ID = "ebinan92/Rukopys-OCR-4B"
PROMPT = (
    "Detect every text region in this Ukrainian handwritten document and "
    "return a JSON array of regions. Each region has bbox (x1 y1 x2 y2 in "
    "0..1000 normalized image coordinates), type (handwritten | printed | "
    "formula | table | annotation | image | graph), and text (transcription; "
    "empty for image/graph; LaTeX for formula; pipe-separated for table)."
)

processor = AutoProcessor.from_pretrained(MODEL_ID)
model = AutoModelForMultimodalLM.from_pretrained(
    MODEL_ID, dtype=torch.bfloat16, device_map="auto"
)
image = Image.open("document.jpg").convert("RGB")
messages = [{
    "role": "user",
    "content": [{"type": "image"}, {"type": "text", "text": PROMPT}],
}]
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

with torch.inference_mode():
    output_ids = model.generate(**inputs, max_new_tokens=8192, do_sample=False)

new_tokens = output_ids[:, inputs["input_ids"].shape[1]:]
print(processor.batch_decode(new_tokens, skip_special_tokens=True)[0])

vLLM

from PIL import Image
from transformers import AutoProcessor
from vllm import LLM, SamplingParams

MODEL_ID = "ebinan92/Rukopys-OCR-4B"
PROMPT = (
    "Detect every text region in this Ukrainian handwritten document and "
    "return a JSON array of regions. Each region has bbox (x1 y1 x2 y2 in "
    "0..1000 normalized image coordinates), type (handwritten | printed | "
    "formula | table | annotation | image | graph), and text (transcription; "
    "empty for image/graph; LaTeX for formula; pipe-separated for table)."
)

processor = AutoProcessor.from_pretrained(MODEL_ID)
image = Image.open("document.jpg").convert("RGB")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": PROMPT},
    ],
}]
prompt = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)

factor = processor.image_processor.patch_size * processor.image_processor.merge_size
llm = LLM(
    model=MODEL_ID,
    dtype="bfloat16",
    max_model_len=16384,
    limit_mm_per_prompt={"image": 1},
    mm_processor_kwargs={
        "min_pixels": 256 * factor * factor,
        "max_pixels": 4096 * factor * factor,
    },
)
params = SamplingParams(max_tokens=8192, temperature=0.0)
outputs = llm.generate(
    [{"prompt": prompt, "multi_modal_data": {"image": image}}],
    sampling_params=params,
)
print(outputs[0].outputs[0].text)

Training data and license

Training used RUKOPYS gold/silver data, external Cyrillic handwriting data, and pseudo-labels, some of which were generated with Gemini (gemini-3-flash-preview).

The model weights are released under the Apache License 2.0. Training datasets are not redistributed here and remain subject to their own licenses.

Citation

@misc{ebinan2026rukopysocr4b,
  title        = {Rukopys-OCR-4B: Ukrainian Handwritten Document OCR},
  author       = {ebinan92},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ebinan92/Rukopys-OCR-4B}}
}