MeridianPII — Hindi / Hinglish PII Detection (v2)
On-device PII detection for Hindi (Devanagari), Hinglish (romanized/code-switched), and Indian-English text. Fine-tuned from microsoft/Multilingual-MiniLM-L12-H384, vocabulary-trimmed to a 55 MB INT8 ONNX that runs in the browser via transformers.js. An India locale pack — route Indian text here; use Rampart (or a future pack) for European languages.
What changed from v1
- Dropped the European anti-forgetting slice (pack positioning — that capacity is now spent on India).
- Added ~6,000 real human-Hindi sentences (Naamapadam, CC0) to reduce reliance on synthetic-only data.
- Net effect: marginally better across the board and cleaner scope.
Evaluation
Span-level, on the same frozen 2,000-row test set (1,000 Devanagari · 625 Hinglish · 375 Indian-English), template-disjoint from training. Recall is the headline metric — a missed entity is a leak.
| Metric | Overall | Hindi | Hinglish | EN-India |
|---|---|---|---|---|
| Private-term recall | 99.8% | 99.9% | 99.4% | 99.5% |
| Public-term retention (keep-set CITY/STATE/ZIP) | 99.6% | 99.7% | 98.0% | 99.7% |
| Span-F1 (relaxed IoU≥0.5) | 0.993 | — | — | — |
vs v1 (99.6% / 99.2% retention) and the English-vocab Rampart baseline (62.6% overall / 50.0% Hindi).
Out-of-distribution (human-written HiNER, person-names): 80.4% recall (v1: 79.7%; Rampart: 0.7%) — the model generalizes to real Hindi names it never saw.
Recommended pipeline
Ship alongside a deterministic recognizer layer (regex + checksums) that owns the structured IDs — Aadhaar (Verhoeff), PAN, IFSC, GSTIN (mod-36), vehicle registration, voter ID, +91 phones, email, URL. These are premasked in training so the model spends capacity on names/addresses/cities/free-form contextual PII. Reference implementation (JS + Python): github.com/plingampally/meridianpii. Keep-set: CITY/STATE/ZIP are detected but retained (not redacted).
Usage
import { pipeline } from '@huggingface/transformers';
const pipe = await pipeline('token-classification', 'plingampally/meridianpii-hi-v2', { dtype: 'q8' });
const out = await pipe('मेरा नाम प्रिया शर्मा है, फ़ोन 9876543210', { aggregation_strategy: 'simple' });
Use aggregation_strategy="simple" — raw BIO subword tags fragment names. ONNX: onnx/model_quantized.onnx (INT8). Recommended confidence floor 0.15 (recall-biased; INT8 flattens scores so Rampart's 0.4 floor is wrong here).
Limitations
- Scope is Indian locales by design (locale pack). Route European text elsewhere.
- CITY/STATE are frequently interchanged (both geographic, both keep-set → no redaction impact).
- Non-Indian geography (countries/rivers) is out of scope.
- Structured IDs are premasked — they need the deterministic layer, not the model.
- Test set is synthetic (OOD-validated on HiNER); evaluate on your own data.
Labels (35 BIO = O + B-/I- × 17)
GIVEN_NAME, SURNAME, EMAIL, PHONE, URL, TAX_ID, BANK_ACCOUNT, ROUTING_NUMBER, GOVERNMENT_ID, PASSPORT, DRIVERS_LICENSE, BUILDING_NUMBER, STREET_NAME, SECONDARY_ADDRESS, CITY, STATE, ZIP_CODE (EMAIL/URL/TAX_ID/ROUTING premasked at serve time).
Training & license
Base microsoft/Multilingual-MiniLM-L12-H384 (Apache 2.0), vocab trimmed 250k→94k pieces (100% output parity). Data: ~40k rows — Hindi/Hinglish/EN-India synthetic + ai4privacy openpii-1.5m mapping + 6k Naamapadam real Hindi. NFC normalization (never NFKD — it strips Devanagari matras). CC BY 4.0. Derives from Rampart (CC BY 4.0) — architecture, schema, premasking methodology.