LayoutLMv3 fine-tuned on CORD-v2 for receipt field extraction
Token classification over receipt images, extracting 59 BIO-tagged field types (menu items, quantities, unit prices, subtotals, tax, service charges, payment amounts).
Part of a document-intelligence pipeline: Perlious-Savage/financial-doc-intelligence.
Results
Evaluated on the CORD-v2 test split (100 documents, 1,309 entities).
| method | precision | recall | F1 |
|---|---|---|---|
| keyword + position rules, no model | 0.557 | 0.468 | 0.509 |
| this model | 0.942 | 0.955 | 0.948 |
Scoring is entity-level: a prediction counts only when its type and its full span match the
reference exactly. Partial extractions score zero, because a partially extracted field is not a
usable one. Implementation:
src/metrics.py.
Where the gain comes from
The rule baseline handles fields printed next to their own label and scores exactly zero on fields identified by position alone. Those are the fields this model exists for:
| field | support | rules F1 | this model |
|---|---|---|---|
menu.cnt | 220 | 0.000 | 0.984 |
menu.unitprice | 67 | 0.000 | 0.933 |
menu.sub.nm | 36 | 0.000 | 0.892 |
menu.sub.price | 20 | 0.000 | 0.947 |
menu.sub.cnt | 17 | 0.000 | 0.944 |
Known weaknesses
Every remaining weak field is a rare class: menu.etc (3 test examples, F1 0.000),
total.total_etc (3, 0.000), menu.itemsubtotal (6, 0.000), total.emoneyprice (2, 0.400),
menu.discountprice (10, 0.571). Two are catch-all etc categories, where scarcity compounds
with genuine ambiguity about what belongs in them. This is a data-coverage limitation, not an
architectural one.
Training
| base model | microsoft/layoutlmv3-base |
| epochs | 8 |
| learning rate | 5e-5 |
| batch size | 4 |
| training documents | 800 |
| label classes | 59 |
| hardware | A100 (Colab), 711s |
Validation F1 by epoch: 0.820, 0.923, 0.921, 0.950, 0.946, 0.952, 0.955, 0.955 — plateaus from epoch 6, so the final two epochs added nothing.
Limitations
- Single training run. No repeated seeds, no confidence intervals. With 100 test documents the interval around 0.948 is not tight; a rerun would move the third decimal.
- Receipts, not KYC. CORD is Indonesian retail receipts. This measures document extraction, not regulatory or compliance performance.
- Indonesian number formatting. Amounts use
.as a thousands separator (12.000is twelve thousand). Parsing logic assumes this convention. - Comparable published work (
nielsr/layoutlmv3-finetuned-cord) reports 0.964 F1 under a different evaluation setup — context, not a like-for-like comparison.
Usage
from transformers import AutoProcessor, LayoutLMv3ForTokenClassification
processor = AutoProcessor.from_pretrained("perlious-Savage/layoutlmv3-cord-extraction", apply_ocr=True)
model = LayoutLMv3ForTokenClassification.from_pretrained("perlious-Savage/layoutlmv3-cord-extraction")
License
Inherits CC-BY-NC-4.0 from microsoft/layoutlmv3-base (non-commercial).
Training data: CORD-v2, CC-BY-4.0.