PrivaScope Topic Classifier
PrivaScope is an end-to-end pipeline for fine-grained, scope-aware analysis of privacy policies. Instead of treating a policy as one uniform document, it classifies each sentence by the product or service it applies to (its Scope, for example Website, App, or Device), by its general theme (Topic), by its specific content (Content), and by the concrete values it mentions (Attributes). This repository holds the fine-tuned Topic classifier, one of 16 classifiers in the pipeline. Given a single sentence from a privacy policy, it predicts which of 15 Topic categories the sentence discusses. A sentence can carry more than one Topic label at once.
Labels
- Processing
- Purpose
- ThirdParty
- UserRights
- Policy
- Sharing
- Contact
- LegalBasis
- Security/Privacy
- Audience
- Other
- Control
- Retention
- Deletion
- Selling
Model details
- Base model, PrivBERT (
mukund/privbert), a RoBERTa model further pretrained on about one million privacy policies - Task, multi-label sentence classification, 15 classes
Training data
The training set contains 3,078 sentences, 2,237 manually annotated sentences drawn from privacy policies, plus 841 additional sentences synthesized with an LLM to reduce class imbalance in underrepresented categories.
Evaluation results
| Metric | Score |
|---|---|
| F1 Macro | 0.87 |
| F1 Micro | 0.88 |
How to use
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline
model_id = "Wravn/privacy-policy-topic"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
classifier = pipeline("text-classification", model=model, tokenizer=tokenizer, function_to_apply="sigmoid", top_k=None)
classifier("We may retain your data for up to two years after account closure.")
Intended use
- Trained on English-language privacy policies.
- Covers only the Topic layer. Scope, Content, and Attribute extraction are handled by separate models and rules in the full pipeline.
- Part of a research pipeline.
Related resources
- Pipeline code:
- Dataset:
- Paper: