pplx-pii-masking-GGUF
GGUF packaging of
perplexity-ai/pplx-pii-masking,
a PII masking model for conversational data. The model uses a ~600M-parameter
bidirectional Qwen3 encoder with a
perplexity-ai/pplx-embed-v1-0.6b
backbone and two heads:
- Token classification head (1024 → 37): BIOES tags over 9 PII categories
(
private_person,account_number,private_url,private_date,private_address,private_email,private_phone,other_pii,secret), decoded with a constrained Viterbi decoder. - Sensitivity head (1024 → 1): conversation-level sensitivity classifier on mean-pooled hidden states.
The backbone is provided as an f16 GGUF for llama.cpp. Both heads remain in
fp32 in heads.safetensors. The included Python client applies the heads to
per-token hidden states from llama-server, decodes PII spans, and masks text.
Usage
Install a compatible llama-server build and the Python client dependencies,
then download the repository:
pip install huggingface_hub torch safetensors transformers
hf download perplexity-ai/pplx-pii-masking-GGUF --local-dir pplx-pii-masking-GGUF
cd pplx-pii-masking-GGUF
llama-server -m pplx-pii-masking-backbone-f16.gguf \
--host 127.0.0.1 --port 8080 \
--embeddings --pooling none -c 4096 -b 4096 -ub 4096
Keep the server running. In a second terminal, enter the downloaded model directory and run:
python example_client.py \
"Hi, I'm Daniel Whitfield, you can reach me at daniels@meridiancap.com or 415-555-0123."
The client prints sensitivity, PII spans, and masked text. It includes the
constrained BIOES Viterbi decoder and reads the label list and biases from
pii_head_config.json.
Checkpoint layout and inference
The GGUF stores qwen3.attention.causal = false for bidirectional attention.
Use --pooling none to obtain one 1024-dimensional hidden state per token.
The classification and sensitivity heads are applied separately by the client.
Use llama-server's native /embeddings endpoint for these unnormalized
per-token hidden states:
curl -s http://127.0.0.1:8080/embeddings \
-H 'Content-Type: application/json' \
-d '{"content":"My email is daniels@meridiancap.com"}'
The client computes token_logits = h @ W_cls.T + b_cls, then decodes the
37 BIOES labels with constrained Viterbi. Sequence sensitivity is
sigmoid(mean_pool(h) @ W_sen.T + b_sen).
Tokenization and character offsets use the supplied tokenizer files.
Limitations
- Inputs must contain between 1 and 4096 tokens. Chunk longer documents before calling the example client; it sends the entire supplied text to the server while its local offset tokenizer truncates at 4096 tokens.
- Bidirectional attention requires the whole input in one micro-batch.
Keep
-band-ubat 4096 for the supported input window. /v1/embeddingsdoes not support--pooling none; use/embeddings.- Conversion precision and the inference backend can affect numerical outputs. Evaluate detection quality and sensitivity thresholds on your target data; false positives and missed PII are possible.
Files
| Path | Purpose |
|---|---|
pplx-pii-masking-backbone-f16.gguf | Bidirectional Qwen3 backbone in f16 GGUF format |
heads.safetensors | fp32 token classification and sensitivity heads |
pii_head_config.json | 37 BIOES labels, Viterbi biases, and head configuration |
example_client.py | Per-token embeddings → heads → Viterbi → spans and masked text |
tokenizer.json, tokenizer_config.json | Tokenizer and character offset mapping |
License
MIT, matching the base model. See LICENSE.