Bo-Boundary-mmBERT 0.2
A ModernBERT token classifier fine-tuned to detect text / outline boundaries in Tibetan Buddhist volume text. Each token is labelled B (boundary) or O (non-boundary).
This is the 0.2 release of BDRC/Bo-Boundary-mmBert. Weights are checkpoint-46000 from run run_bo_outline_neg06 (epoch 2 of 3). The run later early-stopped at step 52,000; 46k is the checkpoint that scored highest micro-F1 on the frozen 100-document benchmark after a decoder sweep.
Hosted as BDRC/Bo-Boundary-mmbert-0.2.
Model details
| Property | Value |
|---|---|
| Base model | jhu-clsp/mmBERT-base |
| Architecture | ModernBertForTokenClassification |
| Parameters | ~307 M |
| Hidden size | 768 |
| Layers | 22 |
| Attention heads | 12 |
| Max sequence length | 8,192 |
| Vocab size | 256,000 |
| Labels | O (id 0), B (id 1) |
| Checkpoint | checkpoint-46000 (step 46,000 / epoch 2) |
Training data
Prepared from BDRC outline annotations (data/Bo_Outline_neg06). Splits are work-level and stratified by print_format, so volumes of the same BDRC work never appear in more than one split. Mixed-format works keep only majority-format volumes (119 minority-format volumes dropped). There is no test split in the training set — evaluation uses a separately curated 100-document benchmark.
| Item | Value |
|---|---|
| Source volumes scanned | 17,381 |
| Unique BDRC works | 10,578 |
| Volumes kept | 17,262 |
| Annotated breakpoints | 204,514 |
| Tokenizer | jhu-clsp/mmBERT-base |
| Window size | 8,192 tokens |
| Stride | 256 tokens |
| Boundary radius | ±3 characters around the boundary char |
| Prepare-time O-only keep ratio | 0.6 |
| Train-time extra negative sampling | none (ratio = 1.0) |
| Seed | 42 |
Splits
| Split | Works | Volumes | Windows |
|---|---|---|---|
| train | 10,050 | 16,356 | 374,660 |
| validation | 528 | 906 | 32,376 |
| test | 0 | 0 | 0 (hand-crafted benchmark instead) |
Stratification by print_format (work-level)
| print_format | Train works | Val works |
|---|---|---|
| modern_book | 1,146 | 60 |
| pering | 8,904 | 468 |
Train label distribution
The boundary class is extremely rare. All windows that contain a B label are kept; only O-only train windows are downsampled at prepare time.
| Stage | Windows | B labels | O labels | B:O |
|---|---|---|---|---|
| Raw (before downsampling) | 555,837 | 1,233,749 | 4,486,541,054 | 1 : 3,636 |
| Final (after 0.6 O-only keep) | 374,660 | 1,233,749 | 3,021,774,477 | 1 : 2,449 |
Training
Run directory: output/run_bo_outline_neg06. Started 2026-09-11. The full run early-stopped at step 52,000 after 105 h 42 m (patience = 10 evals with no doc-level F2 gain). This Hub checkpoint is step 46,000.
Hyperparameters
| Parameter | Value |
|---|---|
| Epochs planned | 3 |
| Batch size (per GPU) | 4 |
| Gradient accumulation | 4 |
| Effective batch size | 16 |
| Optimiser | AdamW |
| Learning rate | 1 × 10⁻⁵ |
| Schedule | cosine decay with warmup |
| Warmup | 7,024 steps (10 %) |
| Steps / epoch | 23,416 |
| Planned total steps | 70,248 |
| This checkpoint | 46,000 (epoch 2) |
| Weight decay | 0.01 |
| Max grad norm | 1.0 |
| Loss | Focal (γ = 2.0, α = [O: 0.20, B: 0.80]) |
| Mixed precision | bfloat16 |
torch.compile | enabled |
| Gradient checkpointing | enabled |
| Neg-sample ratio (train loader) | 1.0 |
| Eval / save interval | every 2,000 steps |
| In-loop eval threshold | 0.70 |
Selection metric (run best/) | doc-level F2 on 80 frozen val docs |
| Early-stop patience | 10 evals |
| Seed | 42 |
In-loop validation used the production decoder at threshold 0.70. Those numbers are not the benchmark operating point below.
Infrastructure
- GPU: 1 × NVIDIA A100-SXM4-40GB (39.5 GB)
- Time to this checkpoint: ~92 h 55 m (step 46,000)
- Full run: 105 h 42 m (early stop at step 52,000)
Validation at this checkpoint (80 holdout docs, threshold 0.70)
| Metric | Value |
|---|---|
| Precision | 0.788 |
| Recall | 0.817 |
| F1 | 0.802 |
| F2 | 0.811 |
| TP / FP / FN | 887 / 239 / 199 |
| Train loss | 4.04 × 10⁻⁵ |
Key in-loop milestones (same 80-doc, threshold-0.70 decoder):
| Step | Epoch | Train loss | Val P | Val R | Val F1 | Val F2 |
|---|---|---|---|---|---|---|
| 2,000 | 1 | 3.74e-3 | 0.320 | 0.378 | 0.347 | 0.365 |
| 8,000 | 1 | 1.01e-3 | 0.840 | 0.712 | 0.771 | 0.734 |
| 16,000 | 1 | 5.35e-4 | 0.894 | 0.768 | 0.826 | 0.790 |
| 34,000 | 2 | 4.24e-5 | 0.807 | 0.828 | 0.817 | 0.824 |
| 46,000 | 2 | 4.04e-5 | 0.788 | 0.817 | 0.802 | 0.811 |
| 52,000 | 3 | 2.97e-5 | 0.823 | 0.669 | 0.738 | 0.695 |
Step 34,000 was the run’s best in-loop F2 (saved as checkpoints/best). Step 46,000 is released here because it generalises better on the frozen benchmark once the decode threshold is swept.
Evaluation on the benchmark
Frozen 100-document outline-boundary benchmark (78 works, 1,643 gold breakpoints, 23.7 M characters). A predicted boundary matches gold if it falls within ±25 characters. Decoder: centre-weighted sliding-window B probabilities → threshold → merge nearby peaks.
| Documents | 100 (81 with ≥1 gold boundary, 19 with none) |
| Gold breakpoints | 1,643 |
| Sources | bdrc_outline 27, tei 18, qwen_ocr 18, google_books 14, ocrv1 12, google_vision 11 |
| Print format | pering 64, modern_book 36 |
Recommended operating point
threshold = 0.55, merge window = 75 characters (best micro-F1 on this checkpoint).
| Metric | Value |
|---|---|
| Micro precision | 0.836 |
| Micro recall | 0.847 |
| Micro F1 | 0.842 |
| F2 | 0.845 |
| Work-macro F1 | 0.599 |
| has_gt F1 (81 docs) | 0.849 |
| TP / FP / FN | 1,392 / 273 / 251 |
| Predicted / gold | 1,665 / 1,643 |
| FP on zero-GT docs | 27 |
Other useful decode settings
postprocess did not change scores on this sweep.
| Threshold | Merge | P | R | F1 | F2 | FP | FN | Notes |
|---|---|---|---|---|---|---|---|---|
| 0.45 | 25 | 0.749 | 0.890 | 0.813 | 0.858 | 490 | 181 | best F2 / highest recall |
| 0.50 | 50 | 0.798 | 0.870 | 0.833 | 0.855 | 362 | 213 | balanced |
| 0.55 | 75 | 0.836 | 0.847 | 0.842 | 0.845 | 273 | 251 | recommended (best F1) |
| 0.60 | 25 | 0.862 | 0.816 | 0.838 | 0.824 | 215 | 303 | slightly more precise |
| 0.70 | 50 | 0.909 | 0.712 | 0.799 | 0.744 | 117 | 473 | in-training eval setting |
Source breakdown (threshold 0.70, merge 50)
| Source | P | R | F1 |
|---|---|---|---|
| tei | 0.953 | 0.920 | 0.936 |
| bdrc_outline | 0.853 | 0.799 | 0.825 |
| google_vision | 0.908 | 0.688 | 0.783 |
| qwen_ocr | 0.966 | 0.644 | 0.773 |
| ocrv1 | 0.930 | 0.524 | 0.670 |
| google_books | 0.892 | 0.294 | 0.443 |
Print format (threshold 0.70, merge 50)
| Format | P | R | F1 |
|---|---|---|---|
| pering | 0.923 | 0.820 | 0.869 |
| modern_book | 0.884 | 0.569 | 0.692 |
How to use
from transformers import AutoModelForTokenClassification, AutoTokenizer
repo_id = "BDRC/Bo-Boundary-mmbert-0.2"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForTokenClassification.from_pretrained(repo_id)
Production decode (centre-weighted windows, threshold, 50-char merge) is implemented in the training package:
mmbert-boundary predict input.txt --model BDRC/Bo-Boundary-mmbert-0.2 --threshold 0.55
Or via OpenPecha/outline-detection:
from outline_detection import MmBertDetector
det = MmBertDetector(model_name_or_path="BDRC/Bo-Boundary-mmbert-0.2")
boundaries = det.predict(text) # [(offset, confidence, "mmbert"), ...]
Recommended defaults for this checkpoint: --threshold 0.55, merge nearby peaks within 75 characters, match gold within ±25 characters. Raise the threshold (0.60–0.70) if you need fewer false positives; lower it (0.45–0.50) if you need higher recall.
Limitations
- Trained on BDRC outline-annotated Tibetan volumes (
peringandmodern_book). Other layouts, scripts, or OCR engines may degrade. - Sparse and zero-boundary volumes still produce extra detections. Downstream use should keep a confidence threshold.
modern_bookrecall is lower thanperingat the high-precision 0.70 setting (0.57 vs 0.82).- In-loop checkpoint selection used F2 at threshold 0.70 on 80 val docs; published scores use a held-out 100-doc benchmark and a swept decoder.
Credits
Trained for the Buddhist Digital Resource Center (BDRC) etext corpus, with funding from the Khyentse Foundation. Successor to BDRC/Bo-Boundary-mmBert.