gliner2-sa-names-lora
A LoRA fine-tune of fastino/gliner2-privacy-filter-PII-multi
(GLiNER2, 205M params) that fixes a measured recall gap on South African names in
cue-free, label-style text — e.g. address/attention lines and email signatures, where
the model previously had little contextual signal to lean on beyond the token itself.
This is the model behind privacy-ext, an
on-device PII redactor for web forms. The deployed variant
(onnx_int8/) is what privacy-ext's Rust inference engine (gliner2-rs) actually runs.
Problem this fixes
The base PII model was trained mostly on cue-rich text ("My name is ...", "Contact: ..."). On South African names appearing in address/attn lines or email signatures — with no explicit naming cue — recall dropped sharply, especially for isiZulu, isiXhosa, Sesotho, Setswana, Tshivenda, and Xitsonga names.
Training
- Base checkpoint:
fastino/gliner2-privacy-filter-PII-multi(the trainable PyTorch checkpoint the deployed ONNX model was originally exported from — not the baregliner2-base-v1, to preserve existing PII tuning). - Method: LoRA,
lora_target_modules=["encoder"]only (r=16, alpha=32, dropout=0.1). 0.86% of parameters trainable (2.65M / 309.75M). - Data: ~2,500 templated examples built from real name/place pools sourced from
nwu-ctext (CC-BY 2.5 SA) and MphayaNER (Apache 2.0), covering 7 label types
(
name,street address,email,phone_num,id_num,url,username) across 7 context styles (address_attn, email_sig, form_field, narrative_no_cue, cue_explicit, billing, third_person), weighted toward the address_attn/email_sig failure modes while keeping the other contexts well-represented. - Schedule: batch_size=16, task_lr=5e-4, up to 15 epochs, early stopping (patience=3). Best checkpoint at epoch 3 (eval_loss=0.238); training halted at epoch 7.
Regression eval
266-case eval spanning 6 South African name groups (isiZulu, isiXhosa, Sesotho/Setswana,
Afrikaans, Tshivenda/Xitsonga, Western control) x 7 contexts, 1,491 total name instances.
Recall = a name span overlapping the expected span was returned by the model.
| Baseline (shipped) | Fine-tuned (fp16) | Fine-tuned + INT8 encoder | |
|---|---|---|---|
| address_attn recall | 48.4% | 100.0% | 98.1% |
| email_sig recall | 58.7% | 99.5% | 99.1% |
| Overall recall | — | 99.7% (1486/1491) | 99.5% (1484/1491) |
| Exact-span match | — | 1479/1491 | 1458/1491 |
| isiZulu recall | 76.4% | 100.0% | 98.2% |
| isiXhosa recall | 80.7% | 100.0% | 100.0% |
| Sesotho/Setswana recall | 85.0% | 100.0% | 99.6% |
| Afrikaans recall | 85.4% | 98.2% | 99.6% |
| Tshivenda/Xitsonga recall | 93.4% | 100.0% | 100.0% |
| Western control recall | 97.1% | 100.0% | 100.0% |
No material regression on the other 6 contexts (form_field, narrative_no_cue, cue_explicit, billing, third_person), which were already at 97-100% on the baseline.
INT8 quantization (dynamic, weight-only, encoder fragment only, reduce_range=True)
trades a small amount of exact-span precision for a 39% smaller encoder
(556 MB → 337 MB fp16 vs int8) and ~2x throughput (1.88 → 3.66 req/s, mean
latency 532ms → 273ms, single-request, CPU, batch size 1) — measured against the
running pii-server daemon on an 8-core dev machine. Overall recall is unchanged
(99.5%, same as without reduce_range).
reduce_range=True matters for correctness, not just accuracy: the first
int8 export (default 8-bit weight range) passed every local test but
reproducibly returned zero entities on GitHub Actions runners lacking
AVX-VNNI — a documented ONNX Runtime caveat where u8s8 quantized matmul can
overflow the accumulator on non-VNNI x86 CPUs. reduce_range=True quantizes
weights to 7 bits instead of 8, avoiding the overflow at negligible cost.
Confirmed stable across Intel Xeon Platinum 8370C, AMD EPYC 9V74, and AMD EPYC
7763 CI runners after the fix. If you re-quantize this model yourself, keep
reduce_range=True unless you can guarantee every deployment target has
AVX-VNNI (most current-gen server/desktop CPUs do; many older or low-power
CPUs don't).
Files
config.json,encoder_config/,model.safetensors,tokenizer*— merged PyTorch checkpoint (LoRA weights already merged into the base). Load with:from gliner2 import GLiNER2 model = GLiNER2.from_pretrained("stefanj0/gliner2-sa-names-lora")onnx_int8/— the 8-fragment ONNX split (encoder,token_gather,span_rep,schema_gather,count_pred_argmax,count_lstm_fixed,scorer,classifier) thatgliner2-rsexpects, exported with the fragment splitter vendored in the gliner2-sa-names-finetune repo. Note: despite the_fp16.onnxfilenames (required bygliner2-rs's autodetection, which keys off this suffix),encoder_fp16.onnxin this folder is dynamically INT8-quantized (weight-only,onnxruntime.quantization.quantize_dynamic) — the other 7 fragments are fp16 as named. This is the exact fragment set currently deployed in privacy-ext.
To run with gliner2-rs / privacy-ext:
PII_MODELS_DIR=<path-to-onnx_int8-dir> PII_TOKEN=<secret> ./pii-server
Limitations
- LoRA was applied to the encoder only; span-boundary/classification behavior is
unchanged from the base checkpoint, so entity types other than
namewere not targeted by this fine-tune (though the eval shows no regression on them). - Training data is templated/synthetic (real name pools, synthetic sentence templates), not naturally-occurring text — may not generalize to name contexts/templates far outside the 7 covered here.
- Evaluated on 266 constructed cases; no held-out real-world corpus eval yet.
License
Apache 2.0, inherited from the base fastino/gliner2-privacy-filter-PII-multi
checkpoint and the Apache/CC-BY-2.5-SA training data sources.