phi-redact — compact English PII/PHI redaction model

phi-redact is an 11M-parameter BERT-mini token classifier for detecting PII and PHI in English text. It covers 29 entity types with BIOES labels and is designed for on-device deployment (ONNX, Core ML, LiteRT) where data must not leave the device.

It was knowledge-distilled from a 66M-parameter DistilBERT teacher and trained on ~157k documents spanning clinical notes, government benefits text, student essays, and Wikipedia-style prose.

Not a certified de-identification solution. This model assists redaction workflows. It is not certified under HIPAA Expert Determination or Safe Harbor methods. Evaluate it on your own data before relying on it in compliance-sensitive contexts. English only.

Model details

Entity types (30)

NAME_PATIENT · NAME_FAMILY · NAME_PROVIDER · DATE · ADDRESS · PHONE · EMAIL · SSN · MRN · HEALTHPLAN · ACCOUNT · IDNUM · LICENSE · EMPLOYER · PROFESSION · AGE · CITY · STATE · COUNTRY · ZIP · URL · IP · DEVICE · ROUTING_NUMBER · IBAN · CREDITCARD · HOSPITAL · FAX · PASSPORT · VIN

Note on TAX_ID: included in the schema design but not in any commercial-safe training source for v7 (ai4privacy excluded). TAX_ID predictions will be absent. Fix queued for v8 (add EIN templates to gen_synth.py).

Benchmark results (v7)

Strict span-level scoring: a prediction counts only if both the character boundaries and the label match exactly.

Standard benchmarks

Composite = (WikiANN + FewNERD + LA essays recall + govsynth + MultiNERD†) / 5.

†MultiNERD (CC-BY-NC-SA 4.0) is included for completeness: v7 recall = 94.3%, within −0.6 pt of Desert Ant's 94.9% on the same dataset.

Nemotron-PII structured benchmark (CC-BY 4.0)

First valid run — 13,134 entities across 25 entity types in 2,000 test documents. MICRO F1 = 0.860, Recall = 0.843.

Selected per-entity F1 scores:

Known weak spot — PROFESSION (F1=0.494, Precision=0.413): the model over-fires on job-title nouns in non-PII context ("She saw a doctor" vs "Occupation: Doctor"). Not a concern for most PHI/PII redaction use cases where PROFESSION is rarely the sensitive entity, but flag it if your data is occupation-heavy.

URL in informal prose: Nemotron URL F1=0.828 (formal text) but LA essays URL recall=0.318 (informal prose). The model handles URLs well in structured documents; it misses them in casual first-person writing.

Comparison to Desert Ant Redact v0.4.0 (English)

phi-redact v7 is essentially tied with Desert Ant's English composite at half the parameters and half the on-device footprint.

Inference latency

Measured 2026-10-03 against the live demo (corytrimm.com/phi-redact): the same 15-template typing-length corpus as evals/latency-bench/bench.py, 15 templates × 10 cycles = 150 redactions, timed from the page's own per-redaction readout — TypeScript rules scan + ONNX detect + merge + placeholders, end to end, onnxruntime-web WASM (Chromium on Linux).

Model load: 0.5 s cached, 4.4 s cold (11.3 MB int8 ONNX + runtime download).

Caveats: the model was cached for this run; browser version and hardware details were not obtainable from the measurement setup (Linux VM, not a phone); readout resolution is whole milliseconds. Not a head-to-head with any vendor's published number — different hardware, corpus, and setup. Methodology travels with the number; see evals/latency-bench/.

Synthetic redaction eval (164 cases)

evals/synthetic-164/ carries a 164-case synthetic redaction test set, the scoring harness, and v7's results: 0.812 full-value recall (term recall 0.615, precision 0.698, clean-case FP rate 0.120). Perfect 1.000 on BANK_ACCOUNT, CREDIT_CARD, DRIVERS_LICENSE, EMAIL, PHONE, ROUTING_NUMBER; weak on NAME (0.347) and ADDRESS (0.333). See the directory README for methodology and the taxonomy note.

Training data

All sources are commercial-safe (CC-BY 4.0 or CC-BY-SA 4.0). No real patient data was used.

Excluded: ai4privacy/pii-masking-400k (non-commercial license — not used in training or evaluation). MultiNERD (CC-BY-NC-SA 4.0 — evaluation only, not in training corpus).

Training procedure

Teacher (distilbert-base-uncased, 66M params):

epochs 10, batch 32, lr 3e-5, cosine LR, label-smoothing 0.05, device A100

Student (this model — BERT-mini, 11.13M params):

epochs 60, batch 64, lr 5e-5, cosine LR, label-smoothing 0.05,
distill-alpha 0.7, device A100 SXM4-40GB (~6h)

Knowledge distillation loss = 0.7 × KL(teacher logits ∥ student logits) + 0.3 × cross-entropy.

How to use

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("CDT5058/phi-redact")
model = AutoModelForTokenClassification.from_pretrained("CDT5058/phi-redact")
model.eval()

text = "Patient Maria Garcia, DOB 1985-03-22, MRN 4471829, called 415-555-0189."

enc = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    logits = model(**enc).logits          # [1, seq, 117]

pred_ids = logits.argmax(-1)[0].tolist()  # [seq]
id2label = model.config.id2label

# Decode BIOES tags to character spans
tokens    = tokenizer.convert_ids_to_tokens(enc["input_ids"][0])
offsets   = enc.encodings[0].offsets      # (char_start, char_end) per token

spans = []
span_start, span_label = None, None
for i, tag_id in enumerate(pred_ids):
    tag = id2label[tag_id]
    prefix, *label_parts = tag.split("-") if "-" in tag else (tag, [])
    label = "-".join(label_parts)
    if prefix == "S":
        spans.append((offsets[i][0], offsets[i][1], label))
    elif prefix == "B":
        span_start, span_label = offsets[i][0], label
    elif prefix == "E" and span_start is not None:
        spans.append((span_start, offsets[i][1], span_label))
        span_start = None
    elif prefix in ("O", "B"):
        span_start = None  # reset on unexpected transition

for start, end, label in spans:
    print(f"[{start}:{end}] {label!r:20s}  {text[start:end]!r}")

For production use, layer the deterministic rules.py (Luhn, ABA routing checksum, SSN, IBAN, regex patterns) on top — it wins on structured identifiers regardless of model confidence.

Limitations

• English only. The bert-base-uncased WordPiece vocabulary has no meaningful cross-lingual representations. French, Spanish, or other languages will produce silent misses — not safe degradation.
• PROFESSION over-fires (F1=0.494 on Nemotron). Treat PROFESSION predictions with lower confidence or disable the entity type if not needed.
• URL in informal prose (recall=0.318 on student essays vs 0.828 on formal text). Check URL recall on your specific document type.
• Trained on synthetic data. Real-world typos, OCR noise, and unusual formatting will degrade performance relative to the benchmarks above.
• Not a certified de-identification system. Does not satisfy HIPAA Expert Determination or Safe Harbor. Use as a detection aid alongside human review.

License

Apache-2.0. Training data licenses: CC-BY 4.0 (Nemotron-PII, LA essays, WNUT-17), CC-BY-SA 4.0 (FewNERD), CC-BY-SA 3.0 (WikiANN). CC-BY-SA ShareAlike requirements apply to the datasets, not the model weights, but consult your legal counsel for commercial deployment.

Card generated 2026-09-25 · Repo: CDT5058/phi-redact