privacy-filter-mlx
On-device PII / secret detection for Apple Silicon (MLX / Metal). A repackaging of
openai/privacy-filter (a gpt-oss-style
128-expert MoE token classifier) for the MLX runtime,
used by the native Swift redactor pf.
It tags every token with one of 33 BIOES labels across 8 categories
(account_number, private_address, private_date, private_email, private_person,
private_phone, private_url, secret) so a downstream redactor can mask spans it has
never seen before — names, emails, phone numbers, API keys, URLs — that exact-match
maskers miss.
Variants
| Path | Format | Size | Labels intact¹ | Use |
|---|---|---|---|---|
/ (root) | bf16, upstream tensor names | ~2.6 GB | 100% (ceiling) | Full fidelity; quantize at runtime to any config |
q4-8emb/ | 4-bit MoE + 8-bit embeddings | ~0.87 GB | 99.4% | Pre-quantized — smaller download, no runtime quant |
¹ Argmax labels matching the fp32 reference, measured on a 200-text / 40k-token synthetic PII eval set.
Quantization frontier (measured)
The MoE experts are ~90% of the weights, so they dominate the size/quality trade-off:
| Config | Size | Labels | Cosine |
|---|---|---|---|
| fp16 | 2799 MB | 99.9% | 0.99982 |
| 8-bit MoE | 1619 MB | 99.8% | — |
4-bit MoE + 8-bit embed (q4-8emb/) | 870 MB | 99.4% | 0.998 |
| 3-bit/128 + 8-bit embed | 670 MB | 98.9% | — |
| 2-bit | 642 MB | 97.9% | — |
q4-8emb is the recommended default: a 69% size cut for ~0.5% label drift. 3-/2-bit are
certified but risky for a redactor (label drift on a fail-closed task).
Usage
Download
# bf16 (root) — full fidelity, runtime-quantizable
hf download beshkenadze/privacy-filter-mlx --local-dir ./privacy-filter
# pre-quantized 870 MB only
hf download beshkenadze/privacy-filter-mlx --include "q4-8emb/*" --local-dir ./privacy-filter
Python (MLX)
The reference forward (pf_mlx.py) loads the bf16 weights and quantizes in-memory via a
runtime knob:
PF_QBITS=4 PF_QEMBED=8 python pf_mlx.py # 870 MB path
PF_QBITS=0 python pf_mlx.py # fp16, full size
Swift CLI (pf)
pf streams stdin → stdout and replaces detected spans with stable <CATEGORY_n>
tokens, fail-closed (no raw value ever reaches stdout unless --fail-open):
cat app.log | pf --model ./privacy-filter
# Contact <PRIVATE_PERSON_0> at <PRIVATE_EMAIL_0>, key <SECRET_0>
Architecture
gpt-oss-style MoE token classifier: 8 layers, hidden=640, 14/2 attention heads × 64,
intermediate=640, 128 experts top-4, attention sinks, bidirectional sliding-window
attention (radius 128), interleaved YaRN RoPE (θ=150000, factor=32, truncate=false),
o200k tokenizer, 33-label BIOES head. See config.json.
Why MLX, not Core ML / ANE
A 128-expert top-4 MoE needs a sparse gather, which the Apple Neural Engine cannot do
— Core ML evicts the MoE to the GPU and runs it dense (32× redundant compute, ~336 ms).
MLX's sparse gather_mm path on Metal is both exact and ~44× faster (~7.6 ms). For MoE on
Apple Silicon, use MLX.
License & attribution
Apache-2.0, inherited from the upstream model. This repo is a format repackaging of
openai/privacy-filter — all model credit
to OpenAI. The bf16 weights at the root are byte-equivalent to the upstream safetensors;
q4-8emb/ is a quantization of those weights.