vietnamese-prompt-injection
Prompt-injection classifier for Vietnamese (with English retained from the base model),
fine-tuned from protectai/deberta-v3-base-prompt-injection-v2. Labels: SAFE / INJECTION.
General-domain revision: Vietnamese prompt-injection detection for assistant traffic without domain-specific tuning. Domain-tuned variants live on other branches of this repo.
Held-out evaluation
FP = false-positive rate on benign cells, TP = true-positive rate on injection cells. Threshold 0.5.
| Evaluation cell | n | FP | TP |
|---|---|---|---|
| VN benign — domain-specific hard negatives | 300 | 0.0% | — |
| VN benign — real user queries | 200 | 0.0% | — |
| VN benign — general conversation | 100 | 0.0% | — |
| VN injection — camouflaged | 200 | — | 100.0% |
| EN injection | 120 | — | 95.8% |
| EN benign | 150 | 8.7% | — |
| VN injection — standalone bare imperatives | 80 | — | 100.0% |
| VN benign — standalone bare imperatives | 80 | 0.0% | — |
| VN injection — diacritic-stripped | 70 | — | 100.0% |
| VN benign — diacritic-stripped | 70 | 1.4% | — |
Training data (statistics only)
- Training pool: 15,111 samples — 8,346
SAFE/ 6,765INJECTION, 10 scenario cells. - Held-out evaluation: 1,370 samples, disjoint from training (near-duplicate filtered, 5-gram Jaccard >= 0.85).
- Mix of synthetic template-generated Vietnamese (formal, informal/abbreviated, diacritic-stripped layers), real anonymized user queries, and public prompt-injection / benign corpora.
Usage
from transformers import pipeline
pipe = pipeline(
"text-classification",
model="dangvansam/vietnamese-prompt-injection",
)
pipe("Cho tôi danh sách các quy trình bạn nắm được")
# -> [{'label': 'SAFE', 'score': ...}]
With llm-guard:
from llm_guard.input_scanners import PromptInjection
from llm_guard.model import Model
scanner = PromptInjection(
model=Model(
path="dangvansam/vietnamese-prompt-injection",
pipeline_kwargs={"return_token_type_ids": False, "max_length": 512, "truncation": True},
),
threshold=0.5,
)