PrivaScope Scope Classifier
PrivaScope is an end-to-end pipeline for fine-grained, scope-aware analysis of privacy policies. Instead of treating a policy as one uniform document, it classifies each sentence by the product or service it applies to (its Scope, for example Website, App, or Device), by its general theme (Topic), by its specific content (Content), and by the concrete values it mentions (Attributes).
This repository holds the fine-tuned Scope classifier, one of 16 classifiers in the pipeline. Given a single sentence from a privacy policy, it predicts which of 11 Scope categories the sentence applies to.
Labels
- Device
- Website
- App
- Store
- Account
- Communication
- BackendService
- Event/Program
- Recruitment
- Services
- Unspecific
Model details
- Base model, PrivBERT (
mukund/privbert), a RoBERTa model further pretrained on about one million privacy policies - Task, single-label sentence classification, 11 classes
Training data
The training set contains 3,078 sentences, 2,237 manually annotated sentences drawn from privacy policies, plus 841 additional sentences synthesized with an LLM to reduce class imbalance in underrepresented categories.
Evaluation results
| Metric | Score |
|---|---|
| F1 Macro | 0.87 |
| F1 Micro | 0.91 |
How to use
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline
model_id = "Wravn/privacy-policy-context]"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
classifier = pipeline("text-classification", model=model, tokenizer=tokenizer, function_to_apply="sigmoid", top_k=None)
classifier("This device collects your location data to provide navigation features.")
Intended use
- Trained on English-language privacy policies.
- Covers only the Scope layer. Topic, Content, and Attribute extraction are handled by separate models and rules in the full pipeline.
- Part of a research pipeline.
Related resources
- Pipeline code:
- Dataset:
- Paper: