TEMPURA-Qwen2.5-VL-3B
TEMPURA checkpoint built on Qwen/Qwen2.5-VL-3B-Instruct.
TEMPURA (Temporal Event Masked Prediction and Understanding for Reasoning in Action, COLM 2026) teaches video-language models to reason about causal event structure and to describe untrimmed videos with fine-grained, timestamp-aligned events. This model performs
-
dense video captioning – partition a video into consecutive, non-overlapping events with timestamps and detailed descriptions;
-
masked event prediction – reason step by step about what happens in a hidden segment of the video;
-
video temporal grounding / highlight detection – locate the moments that match a natural-language query.
-
Project page: https://andy-cheng.github.io/TEMPURA/
-
Code, inference and benchmark evaluation: https://github.com/Andy-Cheng/TEMPURA
-
Dataset: andaba/TEMPURA-VER
Input format
The video is given as a sequence of frames sampled at 1 fps, each with its timestamp (seconds, two decimals) drawn on the top-left corner. The helper functions in the GitHub repository take care of this.
Usage
git clone https://github.com/Andy-Cheng/TEMPURA.git && cd TEMPURA
bash scripts/install/install.sh && source .venv/bin/activate
python -m src.inference.dense_video_captioning_demo --model_path andaba/TEMPURA-Qwen2.5-VL-3B --video test_videos_demo/hotdog.mp4 --fps 1
from src.inference.model_utils import load_model, build_messages, generate
from src.inference.video_utils import load_video_for_model
from src.inference import prompts
processor, model, family = load_model("andaba/TEMPURA-Qwen2.5-VL-3B")
frames, timestamps = load_video_for_model("video.mp4", sample_fps=1, add_timestamp=True)
messages = build_messages(frames, prompts.DVC, family, min_pixels=336 * 336, max_pixels=336 * 336)
print(generate(processor, model, family, messages, max_new_tokens=2048))
Benchmark evaluation (Charades-STA temporal grounding, QVHighlights highlight detection):
bash scripts/eval/eval_charades.sh andaba/TEMPURA-Qwen2.5-VL-3B results/charades/TEMPURA-Qwen2.5-VL-3B qwen 0
bash scripts/eval/eval_qvhighlights.sh andaba/TEMPURA-Qwen2.5-VL-3B results/qvhighlights/TEMPURA-Qwen2.5-VL-3B qwen 0
Results obtained with this pipeline are listed in the repository README.
License
These weights are a fine-tune of Qwen2.5-VL-3B-Instruct and are released under the same Qwen Research License (a copy is included as LICENSE); non-commercial research use only, see the license for details.
Citation
@inproceedings{
cheng2026tempura,
title={{TEMPURA}: Temporal Event Masked Prediction and Understanding for Reasoning in Action},
author={Cheng, Jen-Hao and Peng, Yi-Hao and Zhou, Huapeng and Wang, Vivian and Wang, Huayu and Huang, Hsiang-Wei and Chai, Wenhao and Liu, Hou-I and Chen, Kuang-Ming and Yang, Cheng-Yen and Chen, Yi-Ling and Vineet, Vibhav and Cai, Qin and Hwang, Jenq-Neng},
booktitle={Third Conference on Language Modeling},
year={2026}
}