Mezzo Content Guard v1.5

Mezzo Content Guard v1.5 is a series of ModernBERT-based, English-focused Content Moderation Models trained on approximately 14M tokens (360k+ rows) of labelled examples, utilizing the same dataset as in v1, but with additional data cleaning.

Mezzo Content Guard comes in 5 different sizes, based on Ettin Encoder 400m, 150m, 68m, 32m and 17m:

Try out the demo at the [Mezzo Content Guard Demo] Space

Improvements Over v1

The Ettin Encoder models from the [Ettin Suite] by jhu-clsp was used instead of RoBERTa due to their support for up to 8192 tokens and optimizations with the ModernBERT architecture

These models were trained using MEAN pooling rather than CLS pooling due to observed performance improvements between the 2

Categories

Mezzo Content Guard covers 5 different categories

• Hate Speech: Content that attacks or uses discriminatory and pejorative language toward a person or group based on inherent characteristics such as race, religion, ethnicity, gender, sexual orientation, or disability.

• Self Harm: Content where individuals express a desire to harm themselves, or encouraging acts that lead to self harm.

• Sexual: Content involving any references, descriptions or intent of sexual acts.

• Toxic: Content that attacks, harasses, or discriminates towards an individual.

• Violence: Content that depicts violence and gore, or content that incites acts of violence.

Benchmarks

All benchmarks were done with a threshold of 0.5, though the threshold can be increased or decreased to trade between precision and recall

Latency tests were done on a 5060ti 16GB

Macro Averages

Latency

Batch Inference

One-by-One Inference (batch_size=1)

Quickstart

Introducing our new custom mezzo-guard library that supports the Mezzo Prompt Guard and Mezzo Content Guard models. It offers automatic chunking, organized policies, and redactions.

Installation:

pip install mezzo-guard

from mezzoguard.content_guard import ContentPolicy, Category, Guard

model = Guard("RyanStudio/Mezzo-Content-Guard-v1.5-Small")
content_policy = ContentPolicy().add_threshold(Category.SEXUAL, 0.5) # Only flag Sexual messages

sexual_query = "I want to fuck you"
benign_query = "I want to have a nice day"
violent_query = "I want to kill you"

result_1 = model.scan(text=sexual_query)
print(content_policy.evaluate(result_1).is_unsafe())
# True

result_2 = model.scan(text=benign_query)
print(content_policy.evaluate(result_2).is_unsafe())
# False

result_3 = model.scan(text=violent_query)
print(content_policy.evaluate(result_3).is_unsafe())
# False

With transformers

from transformers import pipeline

model = pipeline("text-classification", model="RyanStudio/Mezzo-Content-Guard-v1.5-Small")


safe_prompt = "I love mezzo content guard!!!"
print(model(safe_prompt))

hate_speech_prompt = "I hate faggots"
print(model(hate_speech_prompt))

self_harm_prompt = "I want to kill myself"
print(model(self_harm_prompt))

sexual_prompt = "I want to fuck someone"
print(model(sexual_prompt))

toxic_prompt = "You are a cunt"
print(model(toxic_prompt))

violence_prompt = "I want to kill someone"
print(model(violence_prompt))

violence_hate_speech_toxic = "I want to kill you because you're a gay faggot"
print(model(violence_hate_speech_toxic, top_k=None))

Training:

The training data was sourced from various open-sourced datasets, as well as synthetically generated from LLMs such as Deepseek v4 Pro, Claude Sonnet 4.6, and Kimi K2.6.

Due to inconsistent labelling and definitions across various datasets, the data was re-laballed using Qwen3Guard-4B and Qwen3.5-4B to fit the specific categorical definitions.

The dataset was cleaned from the v1 dataset using MinHash to remove similar examples accross the same label, and exact duplicates were removed across splits to reduce cross-contamination and to reduce over-fitting to similar phrasing

The following table shows the data distribution:

Limitations

• Re-labelling: Due to the training data being relabelled by Qwen3Guard and Qwen3.5, any inaccuracies from when these models were trained may be passed on to the model
• Edge Cases: A large majority of the open sourced datasets used were often dated, and may not take into account modern day slang words or more subtle bypasses, we recommend finetuning the model on your own usecase
• English Only: The Ettin Encoder models are primarily english-based models, and will suffer in multilingual contexts
• Over‑blocking of marginalized groups: Due to training datasets containing mentions of marginalized groups, the model may overflag mentions of such groups