ModernBERT Bug Severity Classifier
This model is a fully fine-tuned version of
answerdotai/ModernBERT-base for classifying short bug descriptions
into six severity levels:
- blocker
- critical
- major
- normal
- minor
- trivial
Intended use
This model is an educational demonstration of automated bug-severity classification.
It should not be used as the sole authority for production severity decisions. High-impact predictions should be reviewed by a human.
Training dataset
The model was trained using:
AliArshad/Bugzilla_Eclipse_Bug_Reports_Dataset
Only the Short Description field was used as the model input. The
Severity Label field was used as the target label.
Base model
answerdotai/ModernBERT-base
Training approach
This model was trained using full fine-tuning. It is not a LoRA or adapter-only model.
Training included:
- Removing missing and empty descriptions
- Converting severity names into numeric labels
- Stratified training, validation, and test splits
- Batched tokenization
- Dynamic padding and attention masks
- Hugging Face Trainer
- Macro F1 checkpoint selection
Evaluation
- Test accuracy: 0.871
- Test macro F1: 0.32951261884727445
Performance should also be examined separately for each severity using the classification report and confusion matrix.
Limitations
- Severity cannot always be determined from a short description alone.
- The training dataset may contain noisy or inconsistent labels.
- The severity classes are imbalanced.
- Historical bug reports may not represent current software practices.
- Softmax confidence is not guaranteed to be a calibrated probability.
- Human review is recommended for blocker and critical predictions.
License status
The ModernBERT base model uses the Apache 2.0 license. The training dataset's Hugging Face page does not currently declare a dataset license. Confirm the applicable dataset and source-data terms before making this fine-tuned model public.
Example usage
Install the required library:
pip install transformers torch
Run inference:
from transformers import pipeline
classifier = pipeline(
"text-classification",
model="abhishes/modernbert-bug-severity",
revision="v1.0.0"
)
bug_reports = [
"Application crashes immediately and all data is lost.",
"There is a spelling mistake in the documentation."
]
results = classifier(bug_reports)
print(results)