pi-r2 — DETR for document layout detection (loc_beyond_words)
Fine-tuned DETR (ResNet-50) for detecting layout regions in historical newspaper
page scans, trained on biglam/loc_beyond_words
(2,846 train / 712 validation images, 7 classes).
Model / license
- Base model:
facebook/detr-resnet-50— Apache-2.0 (open license, free to share). - The fine-tuned model inherits the Apache-2.0 license.
- Dataset
biglam/loc_beyond_wordsis CC0-1.0 (public domain).
Classes (7)
Photograph, Illustration, Map, Comics/Cartoon, Editorial Cartoon, Headline, Advertisement
Validation results (COCO-style metrics on the held-out validation split, 712 images)
| Metric | Value |
|---|---|
| AP @[IoU .50:.95] | 0.423 |
| AP @IoU .50 | 0.566 |
| AP @IoU .75 | 0.486 |
| AP small | 0.050 |
| AP medium | 0.250 |
| AP large | 0.453 |
| AR max=1 | 0.286 |
| AR max=10 | 0.574 |
| AR max=100 | 0.625 |
Per-class AP
| Class | AP |
|---|---|
| Photograph | n/a |
| Illustration | n/a |
| Map | n/a |
| Comics/Cartoon | n/a |
| Editorial Cartoon | n/a |
| Headline | n/a |
| Advertisement | n/a |
Note:
AP_small/AP_medium/AP_largeare computed on the resized evaluation inputs, not original pixel areas; treat small-object numbers cautiously.
Training details
- Recipe: standard DETR fine-tuning (AdamW, lr=1e-4, backbone lr=1e-5, wd=1e-4, grad clip 0.1, fp16, linear warmup/decay, batch size 2 with grad accumulation 2).
- Input: images resized to shortest edge 480px / longest edge 800px (aspect kept), RGB.
- Epochs: up to 15 with early stopping (patience 3 on AP@.5) and validation every 4 epochs.
- Hardware: 1x NVIDIA T4 (16GB) via Hugging Face Jobs (t4-small).
- Cost: < $5 total GPU compute.
Quickstart
from transformers import DetrImageProcessor, DetrForObjectDetection
from PIL import Image
import torch
processor = DetrImageProcessor.from_pretrained("harness-race/pi-r2")
model = DetrForObjectDetection.from_pretrained("harness-race/pi-r2")
image = Image.open("page.png").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
results = processor.post_process_object_detection(
outputs, threshold=0.5, target_sizes=[(image.height, image.width)])[0]
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
print(model.config.id2label[label.item()], round(score.item(), 3), [round(v, 1) for v in box.tolist()])
Files
model.safetensors,config.json,preprocessor_config.json— fine-tuned model + processorval_metrics.json— full COCO validation metrics (incl. per-class AP)figures/— sample predictions on the validation settrain_detr.py,requirements.txt— training code for reproducibility
Limitations
- Trained on grayscale newspaper scans; color documents may perform worse.
- Classes are heavily imbalanced (Headline/Advertisement dominate; Map/Editorial Cartoon are rare).
- Small text-heavy regions may be missed at 480–800px input resolution.