ro-political-fake-news-v2

Fine-tuned dumitrescustefan/bert-base-romanian-cased-v1 for Romanian fake-news / misinformation scoring (v2).

Unlike v1 (manuelraileanu/ro-political-fake-news, XLM-RoBERTa, output normalized to [0, 1]), this model uses a single regression head trained on the raw 0-100 labeling scale (v2 labeling pipeline, combined ground-truth + gemma2:9b labeled data, all 4 training sources).

Output

A single regression value on the 0-100 scale:

• 0 = factual, verifiable, truthful
• 50 = questionable claims, exaggerations
• 100 = blatant disinformation, conspiracy

Outputs can slightly fall outside [0, 100] (regression head); clamp if needed. For binary classification, threshold at 50. This measures misinformation truthfulness of the claims in the text, independent of whether the text is human- or AI-written.

Training

• Base model: dumitrescustefan/bert-base-romanian-cased-v1
• Dataset 3 (all sources): ClickbaitSciTechRO + SaRoCo (ground-truth binary 0/100 labels)
  • ro-news-offense + ro-political (labeled with gemma2:9b via Ollama)
• Labeled rows: 159266 (train), 17697 (test, 10% split, seed 42)
• fp16, batch_size=16, grad_accum=2, max_length=512, weight_decay=0.01
• learning_rate=2e-05, 7 epochs, MSE loss, early stopping on mae (patience=2)

Dataset sources

Results (test set, 0-100 scale)

• MAE: 9.34
• RMSE: 19.41
• Binary accuracy @ threshold 50: 0.833
• Macro F1 @ threshold 50: 0.833

Cross-dataset evaluation (same model, tested on the other curated datasets):

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model_id = "manuelraileanu/ro-political-fake-news-v2"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()

inputs = tok("Vaccinurile conțin cipuri 5G iar guvernul ascunde adevărul.", return_tensors="pt")
with torch.no_grad():
    score = model(**inputs).logits.reshape(-1)[0].item()  # 0 (truthful) .. 100 (disinformation)