ModernBERT fine-tuned on Banking77
Fine-tune of answerdotai/ModernBERT-base for 77-class customer-intent classification.
Usage
from transformers import pipeline
clf = pipeline("text-classification", model="Ahmed167/modernbert-banking77", top_k=5)
clf("My card hasn't arrived yet, what should I do?")
Training
- Base model:
answerdotai/ModernBERT-base(149M params, encoder-only, 8k context) - Dataset: PolyAI/banking77 (~13k train / 3k test, 77 classes)
- Recipe: full fine-tune, lr=5e-5 cosine schedule with 10% warmup, 4 epochs, bf16, weight-decay=0.01, label-smoothing=0.1, batch size 32
- Validation split: 10% of train, stratified by label
- Metric for best model: macro F1
Calibration
Logits are saved as-is. To get calibrated probabilities, divide logits by the
fitted temperature (reported in eval_results.json in the source repo) before
softmax. The fitted T value is typically slightly > 1, meaning the raw model is
mildly overconfident.
Source
GitHub: AhmedMostafa167/Projects/04-Deep-Learning/text-classification-modernbert