Video Deepfake Detection — video-deepfake-detection-GenD_DINOv3_L_FF
State-of-the-art Video Deepfake Detection model trained on FaceForensics++ (FF++) based on the GenD framework fine-tuned with Sharpness-Aware Minimization (SAM) and Label Smoothing.
This model uses facebook/dinov3-vitl16-pretrain-lvd1689m as visual foundation backbone and fine-tunes only the Layer Normalization parameters (accounting for only ~0.03% of total parameters) while enforcing a hyperspherical feature manifold through L2-normalization and metric learning (Uniformity & Alignment losses).
📊 Benchmark Results
| Metric | Score |
|---|---|
| Video AUROC | 91.84% |
| Video mAP | 89.88% |
| Video Accuracy | 84.91% |
| Video EER | 15.09% |
| Frame AUROC | 86.93% |
| Frame mAP | 83.73% |
| Frame Accuracy | 80.40% |
📌 Model Details
- Training Dataset:
FaceForensics++ (FF++)(FF++) - Visual Backbone:
facebook/dinov3-vitl16-pretrain-lvd1689m - Classification Head:
LinearNorm - Optimizer:
SAM-AdamW(SAM $\rho=0.05$, adaptive=True) - Loss Formulation: Cross-Entropy with Label Smoothing (0.1), Uniformity (0.5), Alignment (0.1)
- Training Epochs:
30 - Batch Size:
96 - Precision:
bf16-mixed - Learning Rate:
0.0003
🚀 Quickstart & Inference
1. Using the Model in Python
import torch
from PIL import Image
from transformers import AutoModel
# Load the model directly from Hugging Face Hub
model = AutoModel.from_pretrained("HoopitAI/video-deepfake-detection-GenD_DINOv3_L_FF", trust_remote_code=True)
model.eval()
# Preprocess image crop (aligned face)
image = Image.open("path/to/face_crop.png").convert("RGB")
tensor = model.feature_extractor.preprocess(image).unsqueeze(0)
# Run inference
with torch.no_grad():
logits = model(tensor)
# Output class 0: Real, Output class 1: Fake
fake_prob = logits.softmax(dim=-1)[0, 1].item()
print(f"Deepfake Probability: {fake_prob:.2%}")
2. Using with src.hf.modeling_gend
from src.hf.modeling_gend import GenD
model = GenD.from_pretrained("HoopitAI/video-deepfake-detection-GenD_DINOv3_L_FF")
model.eval()
🏗️ Architecture & Training Methodology
The GenD method achieves superior cross-dataset generalization by avoiding catastrophic overfitting on manipulation-specific artifacts:
- Training on FF++: Trained on face crops from FaceForensics++ (FF++).
- LayerNorm Tuning: Keeps the visual transformer backbone frozen while updating only normalization scaling and bias terms.
- Normalized Linear Head: Normalizes feature vectors onto a hypersphere before linear projection.
- SAM Optimization: Smooths the loss landscape to find flat minima that resist out-of-distribution domain shifts.