🎯 General Video Embedder (GVE)

One Embedder for All Video Retrieval Scenarios
Queries of text, image, video, or any combination modalities — GVE understands them all for representations, zero-shot, without in-domain training.

GVE is the first video embedding model that generalizes across 9 abilities, including 3 diverse retrieval tasks and 6 domains — from coarse text-to-video to fine-grained spatial/temporal queries, composed (text+image) queries, and long-context retrieval — all evaluated on our new Universal Video Retrieval Benchmark (UVRB).

Built on Qwen2.5-VL and trained only with LoRA with 13M collected and synthesized multimodal data, GVE achieves SOTA zero-shot performance than competitors.

🌟 Why GVE?

📊 Performance on UVRB

• TXT: Textual Video Retrieval
• CMP: Composed Video Retrieval
• VIS: Visual Video Retrieval
• CG: Coarse-grained Video Retrieval
• FG: Fine-grained Video Retrieval
• LC: Long-Context Video Retrieval
• S: Spatial Video Retrieval
• T: Temporal Video Retrieval
• PR: Partially Relevant Video Retrieval

For each column: highest score is bolded, second-highest is underlined.

🚀 Get Started

1. Loading model

model_path = 'Alibaba-NLP/GVE-3B'
model = AutoModel.from_pretrained(model_path, trust_remote_code=True, device_map='auto', low_cpu_mem_usage=True, torch_dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, use_fast=True)
processor.tokenizer.padding_side = 'left'

2. Processing inputs

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {
        "role": "user",
        "content": [
            {
                "type": "video",
                "video": "./asset/video_example.mp4",
                "max_pixels": 200 * 28 * 28,
                "fps": 1.0,
                "max_frames": 8,
            },
            {"type": "text", "text": "Describe this video."},
        ],
    }
]
texts = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs, video_kwargs = process_vision_info(messages, return_video_kwargs=True)
inputs = processor(
    text=[texts],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    truncation=True,
    max_length=1200,
    return_tensors="pt",
    **video_kwargs,
).to("cuda")

3. Embedding

outputs = model(**inputs)
embedding = F.normalize(outputs['last_hidden_state'][:, -1, :], p=2, dim=1)

📚 Citation

@misc{guo2025gve,
  title={Towards Universal Video Retrieval: Generalizing Video Embedding via Synthesized Multimodal Pyramid Curriculum}, 
  author={Zhuoning Guo and Mingxin Li and Yanzhao Zhang and Dingkun Long and Pengjun Xie and Xiaowen Chu},
  year={2025},
  eprint={2510.27571},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2510.27571}, 
}