Falcon-OCR v1.5: kernel-ready CPU files
Pre-packed weights of tiiuae/Falcon-OCR v1.5 (revision fe757d59) for the Rust CPU runner at github.com/amazingvince/falcon-ocr-cpu, plus a trained draft head that speeds up decoding.
Each model file holds every tensor in the exact layout the runner's kernels read, so it is memory-mapped and used in place with no conversion: loading takes about 10 ms, against about 2 s to quantize the original FP32 checkpoint at start-up. These are not transformers checkpoints; use them with the runner above.
Files
| File | What | Size |
|---|---|---|
falcon-ocr-v1.5-near-exact.safetensors | near-exact mode (the default): 16-bit body weights with one scale per 64 weights; 16-bit KV cache | 806 MB |
falcon-ocr-v1.5-fast.safetensors | fast mode: 8-bit GPTQ body weights (act-order, groups of 64); 8-bit KV cache | 638 MB |
falcon-ocr-v1.5-draft-head.safetensors | the draft head for speculative decoding (below); used automatically when it is in the model folder | 94 MB |
tokenizer.json, tokenizer_config.json, config.json | from the original model |
Both model files also carry the FP32 embedding, vocabulary head, norms, projector and attention sinks, and an INT8 screen of the vocabulary head with its error bounds, which the runner uses to pick exactly the same greedy token as the full FP32 head. Their safetensors metadata records the source checkpoint hash, the recipe and a digest of every tensor (--verify-model-file checks it).
Usage
git clone https://github.com/amazingvince/falcon-ocr-cpu && cd falcon-ocr-cpu
cargo build --release --locked
huggingface-cli download amazingvince/falcon-ocr-v1.5-cpu --local-dir models/falcon-ocr-cpu
target/release/falcon-ocr --model models/falcon-ocr-cpu run page.png --text # near-exact
target/release/falcon-ocr --model models/falcon-ocr-cpu --mode fast run page.png --text # fast
target/release/falcon-ocr --model models/falcon-ocr-cpu doctor --text # what it will do
CPU targets are x86-64 with AVX2 or AVX-512 and aarch64 (NEON), with a scalar fallback.
Accuracy
Against the FP32 model, teacher-forced along its outputs on 55 calibration pages (24,262 tokens):
| Mode | Mean KL per token | Tokens where the greedy choice differs |
|---|---|---|
| exact (the original FP32 checkpoint, not in this repo) | 0 | 0 |
| near-exact | 9e-8 | 1 |
| fast | 2.7e-4 | 63 |
On 118 fresh English OmniDocBench pages (figures the ground truth leaves out masked on every system's input), near-exact produced exactly FP32's tokens on every page. Fast mode matched FP32 token for token on 93 pages, and its character error rate against the ground truth was 0.11 points lower than FP32's overall, with no repetition loops: no measurable loss on printed English. On handwriting and degraded scans in the runner's 200-page held-out set, fast mode's 8-bit weights tipped some uncertain pages into repetition loops (the runner's repetition stop cuts them short); use near-exact for such pages. Details: docs/MODES.md.
Speed
Ryzen 9 7950X (16 cores, DDR5), Windows 11, default flags, without the draft head:
| Mode | Journal page (6,544 image tokens) | Decode per token |
|---|---|---|
| exact (FP32) | 38 s | 29 ms |
| near-exact | 21.5 s | 15 ms |
| fast | 12.6 s | 8.5 ms |
With the draft head (used automatically when it is in the model folder), 16 English held-out pages, total time with the n-gram drafter → with the head, tokens identical:
| Mode | n-gram drafts only | with the draft head | Decode vs no speculation |
|---|---|---|---|
| exact | 531 s | 393 s (−26%) | 1.67× |
| near-exact | 277 s | 208 s (−25%) | 1.87× |
| fast | 173 s | 136 s (−21%) | 1.50× |
--max-dimension auto (opt-in) picks 768, 1024 or 1536 pixels per page from image statistics and saves about a quarter of the CPU time; it changes the output and is not yet validated on held-out pages (details).
The draft head
An EAGLE-3-style drafter: one decoder block over the features of the target's layers 2, 11 and 19 (text only), a 16,383-token draft vocabulary, trained first on next-token prediction and then on seven-step chains (weights 0.5^j). It proposes up to four tokens per step while its confidence is at least 0.35, and the model verifies them in one pass, so it never changes the output, only the speed. The runner uses it automatically when this file sits next to the model files and speculation is on (--drafter ngram or --speculate 0 turn it off).
Training data: about 45,000 English document pages, rendered as images and transcribed by the official Falcon-OCR vLLM image (greedy), after removing duplicates and every page that shares an image or a 13-gram of text with OmniDocBench. No non-commercial or evaluation-only data set was used. Sources, with the pages of the final training stage:
| Source | Pages | License or terms |
|---|---|---|
| allenai/olmOCR-mix-1025 | 25,841 | ODC-BY-1.0 (intended for research and educational use under Ai2's Responsible Use Guidelines) |
| pixparse/pdfa-eng-wds | 12,000 | Common Crawl terms of use and Digital Corpora terms of use |
| pixparse/idl-wds | 7,000 | UCSF Industry Documents Library terms |
| docling-project/DocLayNet-v1.2 | 1,937 | CDLA-Permissive-1.0 |
| biglam/loc_beyond_words | 500 | CC0-1.0 |
| NoTeS-Bank/ICDAR_2025_Handwritten_Notes_Understanding_Challenge | 392 | Apache-2.0 |
| PDFPages/zenodo-presentations-open (CC-BY-4.0 records only) | 230 | CC-BY-4.0 |
| HumynLabs handwritten notes (math, physics, chemistry, biology, computer science) | 169 | CC-BY-4.0 |
The pipeline, the recipe and every measurement are in research/draft-head.
Rebuilding the model files
python scripts/fetch_reference.py --output artifacts/model
falcon-ocr --mode near-exact pack --output falcon-ocr-v1.5-near-exact.safetensors
bash tools/make_gptq_overlay.sh # the GPTQ overlay fast mode needs (about 20 minutes)
falcon-ocr --mode fast pack --output falcon-ocr-v1.5-fast.safetensors
License and attribution
Derived from tiiuae/Falcon-OCR by the Technology Innovation Institute, released under the Apache License 2.0. The model files are re-encodings and quantizations of those weights, and the draft head was trained on that model's features and outputs; all are redistributed under the same license. They are not an official TII release. Cite the original work: Falcon OCR tech report. The draft head's training data remains under its sources' own terms (table above).