ClaimLens-M Verifier
ClaimLens-M Verifier is a fine-tuned XLM-RoBERTa sequence-classification model for evidence-grounded claim verification.
The model classifies an evidence-claim pair into:
entailmentcontradictionneutral
In the full ClaimLens-M pipeline, these NLI labels are mapped to:
SUPPORTEDREFUTEDNOT_ENOUGH_INFO
Training Data
The verifier was fine-tuned on a FEVER-NLI subset:
| Split | Count |
|---|---|
| Train | 20,000 |
| Validation | 2,000 |
| Test | 2,000 |
The project also includes curated Italian demo pairs for multilingual product testing.
Metrics
| Metric | Value |
|---|---|
| Test loss | 0.5157 |
| Test accuracy | 0.8105 |
| Test macro F1 | 0.8057 |
| Test weighted F1 | 0.8094 |
Usage
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch
repo_id = "mokarami/claimlens-m-verifier"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
premise = "QLoRA enables fine-tuning large language models with 4-bit quantization, reducing GPU memory requirements."
hypothesis = "QLoRA reduces GPU memory requirements during fine-tuning."
inputs = tokenizer(premise, hypothesis, truncation=True, max_length=384, return_tensors="pt")
with torch.no_grad():
probs = torch.softmax(model(**inputs).logits[0], dim=-1)
label_id = int(torch.argmax(probs))
print(model.config.id2label[label_id], float(probs[label_id]))
Full Project
The full ClaimLens-M application combines:
- BM25 retrieval
- lexical dense retrieval fallback
- hybrid evidence ranking
- XLM-RoBERTa NLI verification
- calibrated verdict aggregation
- structured JSON outputs
- FastAPI and Gradio interfaces
The Hugging Face Gradio Space package is prepared separately, but creating Gradio Spaces on this account currently requires a Hugging Face PRO subscription.