Llama-Prompt-Guard-2-86M-onnx
This repository provides a ONNX converted and quantized version of meta-llama/Llama-Prompt-Guard-2-86M
🧠 Built With
- Meta LLaMA – Foundation model powering the classifier
- 🤗 Hugging Face Transformers – Model and tokenizer loading
- ONNX – Model export and runtime format
- ONNX Runtime – Efficient inference backend
📥 Evaluation Dataset
We use jackhhao/jailbreak-classification
for the evaluation (train+test)
🧪 Evaluation Results
| Model | Accuracy | Precision | Recall | F1 Score | AUC-ROC |
|---|---|---|---|---|---|
| Llama-Prompt-Guard-2-22M | 0.9564 | 0.9888 | 0.9249 | 0.9558 | 0.9234 |
| Llama-Prompt-Guard-2-22M-q | 0.9579 | 0.9967 | 0.9204 | 0.9449 | 0.9180 |
| Llama-Prompt-Guard-2-86M | 0.9801 | 0.9984 | 0.9625 | 0.9801 | 0.9519 |
| Llama-Prompt-Guard-2-86M-q | 0.8989 | 1.0000 | 0.8018 | 0.89 | 0.7452 |
🤗 Usage
from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSequenceClassification
import numpy as np
# Load model and tokenizer using optimum
model = ORTModelForSequenceClassification.from_pretrained("gravitee-io/Llama-Prompt-Guard-2-86M-onnx", file_name="model.quant.onnx")
tokenizer = AutoTokenizer.from_pretrained("gravitee-io/Llama-Prompt-Guard-2-86M-onnx")
# Tokenize input
text = "Your comment here"
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
# Run inference
outputs = model(**inputs)
logits = outputs.logits
# Optional: convert to probabilities
probs = 1 / (1 + np.exp(-logits))
print(probs)
🐙 GitHub Repository:
You can find the full source code, CLI tools, and evaluation scripts in the official GitHub repository.