Model Card for Model ID
This model is a fine-tuned version of MCG-NJU/videomae-base for binary violence classification (violent / non-violent).
It builds on Nikeytas/videomae-crime-detector-production-v1, which was itself fine-tuned from videomae-base on a subset of UCF Crime. Starting from that checkpoint, this model was further fine-tuned on the Bus Violence Dataset to close the domain gap to public-transport surveillance footage.
Dataset & Training
Fine-tuned on:
- UCF Crime (
jinmang2/ucf_crime) — inherited from the base checkpoint - Bus Violence Dataset (Zenodo) — real moving-bus footage, binary violent / non-violent labels, used for domain-specific fine-tuning
Performance
Evaluated on a held-out Bus Violence Dataset test split (n = 280).
| Model | n | Accuracy | Error | FPR | FNR | Precision | Recall | TP | TN | FP | FN |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Nikeytas/videomae-crime-detector-production-v1 (zero-shot, no bus-violence fine-tuning) | 280 | 0.48 | 0.52 | 0.64 | 0.40 | 0.48 | 0.60 | 84 | 50 | 90 | 56 |
| This model (fine-tuned on Bus Violence Dataset) | 280 | 0.87 | 0.13 | 0.11 | 0.14 | 0.88 | 0.86 | 120 | 124 | 16 | 20 |
The base UCF-Crime checkpoint transfers poorly to on-bus footage out of the box (near chance-level accuracy, high false-positive rate), which is consistent with the domain-shift findings reported for transport-surveillance data (illumination changes, vehicle/passenger motion, scrolling backgrounds). Fine-tuning on in-domain bus violence footage substantially improves all metrics.
Quick Start
pip install transformers torch torchvision opencv-python pillow
import torch
from transformers import AutoModelForVideoClassification, AutoProcessor
import cv2
import numpy as np
# Load model and processor
model = AutoModelForVideoClassification.from_pretrained("<your-repo-id>")
processor = AutoProcessor.from_pretrained("<your-repo-id>")
def classify_video(video_path, num_frames=16):
cap = cv2.VideoCapture(video_path)
frames = []
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
indices = np.linspace(0, total_frames - 1, num_frames, dtype=int)
for idx in indices:
cap.set(cv2.CAP_PROP_POS_FRAMES, idx)
ret, frame = cap.read()
if ret:
frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
frames.append(frame_rgb)
cap.release()
inputs = processor(frames, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
predicted_class = torch.argmax(predictions, dim=-1).item()
confidence = predictions[0][predicted_class].item()
label = "Violent" if predicted_class == 1 else "Non-Violent"
return label, confidence
video_path = "path/to/your/video.mp4"
prediction, confidence = classify_video(video_path)
print(f"Prediction: {prediction} (Confidence: {confidence:.3f})")
Technical Specifications
- Base Model: MCG-NJU/videomae-base (via Nikeytas/videomae-crime-detector-production-v1)
- Architecture: Vision Transformer (ViT) adapted for video
- Input Resolution: 224x224 pixels per frame
- Temporal Resolution: 16 frames per video clip
- Output Classes: 2 (binary: violent / non-violent)
- Training Framework: HuggingFace Transformers