Detoxify ONNX 🚀

This project provides an ONNX-exported and quantized version of the Detoxify multilingual model, optimized for runtime inference.
It enables faster and lighter toxicity detection using ONNX Runtime.

🧪 ONNX Evaluation Results

Original Model (using Detoxify lib and ONNX):

Time for 1 threshold evaluation =~ 3 min 30s

Quantized model:

Time for 1 threshold evaluation =~ 2 min 41s

🤗 Usage

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSequenceClassification
import numpy as np

# Load model and tokenizer using optimum
model = ORTModelForSequenceClassification.from_pretrained("gravitee-io/detoxify-onnx", file_name="model.quant.onnx")
tokenizer = AutoTokenizer.from_pretrained("gravitee-io/detoxify-onnx")

# Tokenize input
text = "Your comment here"
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)

# Run inference
outputs = model(**inputs)
logits = outputs.logits

# Optional: convert to probabilities
probs = 1 / (1 + np.exp(-logits))
print(probs)

🐙 GitHub Repository:

You can find the full source code, CLI tools, and evaluation scripts in the official GitHub repository.