Bolt CLS PII
Bolt CLS PII is a binary text classifier for detecting personally identifiable information (PII) in raw text. It is one of four Bolt CLS classifiers (PII, Content, Jailbreak, Router) used together in the production request-safety pipeline.
Model Overview
Bolt CLS PII is trained specifically to flag whether a piece of raw text is likely to contain personally identifiable information, acting as the first-pass sieve in front of a heavier downstream reviewer model.
Key design goals:
- Very high recall (catch nearly all PII-bearing prompts) even at the cost of precision
- Fast, lightweight scoring suitable for gating every request in a production pipeline
- A calibrated score that downstream systems can threshold for different use cases
Model Details
- Type: Text classifier (binary, single-label)
- Architecture: ModernBERT (
ModernBertForSequenceClassification) - Base model:
answerdotai/ModernBERT-base— frozen during training (only the classification head was trained). This was internally verified against the training run's own configuration and evaluation records. - Hidden size: 768 · Layers: 22 · Attention heads: 12
- Max input length: 8,192 tokens
- Pooling: mean
Training Data
Training data combines three sources:
- An in-house PII dataset, providing entity-level labeled examples of PII-bearing text.
unsloth/alpaca-cleaned, a public general-instructions dataset, used as clean negatives.nvidia/Nemotron-PII, a public PII-spans dataset, providing additional PII-bearing examples with its own span labels.
The combined data is split 90% train / 10% test (seed 42). Exact label counts and dataset composition are not published in this card. The specific PII types represented follow whatever the in-house dataset and Nemotron-PII's own entity/span labels cover.
Training
Trained with AI Squared's internal classifier training pipeline, shared across all four Bolt CLS classifiers. This is a frozen-base run — only the classification head was trained on top of the frozen ModernBERT-base.
- Objective/loss: weighted focal loss (γ = 2.0 by default) on top of standard cross-entropy, with inverse-class-frequency weighting plus an extra ×2.0 multiplier on the positive (PII) class — biases the model toward catching positives, consistent with the sieve design.
- Threshold selection: after training, the eval-split positive-class scores are swept to find the F1-optimal decision threshold (ties broken by recall, then precision); that threshold and its metrics are saved to
threshold.jsonand shipped with the model — this is the same fileThreshold Guidancebelow is drawn from. - Hardware: Nvidia H200
Threshold Guidance
Two thresholds are relevant to this model:
| Threshold | Value | Where it's used |
|---|---|---|
| F1-optimal | 0.5238 | The threshold that maximizes F1 on the held-out eval set — use this if consuming the raw score standalone. |
| Production sieve threshold | 0.10 | The low threshold the production sieve stage uses to flag prompts for PII. Set deliberately low because a sieve flag is not a hard block — it routes to a downstream reviewer model for the final decision, so false positives are cheap and false negatives are the thing to avoid. |
If you're integrating this model outside of the production sieve, default to the F1-optimal threshold (0.5238) unless you have the same "flag → human/model review" downstream step, in which case the lower 0.10 threshold is appropriate.
Evaluation
Reported production target: ~0% FPR at near-100% accuracy at the operating threshold. Held-out eval metrics at the F1-optimal threshold (0.5238):
| Metric | Value |
|---|---|
| Accuracy | 0.99936 |
| F1 | 0.99952 |
| Precision | 0.99926 |
| Recall | 0.99979 |
Intended Use
- Low-threshold pre-filter (sieve) in the production request pipeline, flagging prompts that may contain PII for downstream review by a larger reviewer model.
- Standalone PII detection where a binary flag on raw text is sufficient.
Out of scope: entity-level PII extraction/redaction (this model classifies, it does not locate or redact specific spans), and any use as a sole/final blocking decision without a secondary reviewer — per the sieve design, it is a first-pass flag, not a final call.
Deployment Notes (Hugging Face Inference Endpoints)
If you're deploying this model on HF Inference Endpoints:
- The two frontmatter lines above are required. Without
pipeline_tagandlibrary_nameset, HF cannot identify this repo's task/architecture, which blocks every specialized engine (vLLM, TEI, TGI, SGLang) from even appearing as selectable options — you'll only see "Default," along with a "no handler.py found" warning. Adding these two lines is what unlocks the full engine list on the deployment screen. - Manually select vLLM as the Inference Engine on the deployment screen — it is not selected by default, and the default recommendation may pick a different engine that doesn't correctly support this architecture's classification head. This has to be set explicitly every time you create a new endpoint; it is not remembered from prior deployments and cannot currently be pinned via any repo file.
- No
handler.pyorrequirements.txtis needed once vLLM is selected — vLLM readsconfig.jsondirectly.
Usage
from transformers import pipeline
classifier = pipeline("text-classification", model="aisquared/bolt-cls-pii")
result = classifier("Your text here")
print(result)
Limitations
- Classifies at the document/prompt level, not span-level — does not identify which text is PII, only whether PII is likely present.
- Evaluated against an internal PII eval set; performance on domains/PII types not represented there is not guaranteed.
- As with any threshold-based classifier, behavior near the threshold boundary should be spot-checked before relying on it in a new context.
License
Bolt CLS PII is released under the AI Squared Community License.