VideoFace-X-LLM — ViT + LLM-LoRA fine-tune (step 3200 of 3406)

InternVideo2.5-Chat-8B fine-tuned on the ExplainFace corpus (Gemini descriptions of CelebV-HQ videos) for the ACCV 2026 paper "When, Where, and Why: Hierarchical Causal Explainability for Video Face Understanding with MLLMs". This is the "Ours (ViT+LLM LoRA)" row of the results table and the model all H-TCC explainability results were computed on.

Contents

• Root: the merged full checkpoint (LoRA folded into the LLM weights, W += α/r · B·A), loadable with trust_remote_code=True.
• lora_adapter/: the trainable-only checkpoint as saved during training (trainable.safetensors: vision encoder, connector and LoRA weights; trainable_config.json).
• merge_lora_ckpt.py: the script that produced the merged checkpoint from the base model and lora_adapter/.

Training

• Trainable: vision encoder (lr 1e-5) + connector mlp1 (lr 4e-5) + LoRA on the frozen language model (rank 64, α 128, dropout 0.05, targets wqkv, wo, w1, w2, w3, lr 1e-4).
• 1 epoch = 3,406 steps over the 27,304-clip identity-independent training split (Aak579/CelebV-HQ-identity-split20), micro-batch 1, global batch 8, AdamW, weight decay 0.01, 3 % linear warm-up then cosine decay, gradient clip 1.0, 8–32 frames per clip, seed 42, FSDP full sharding, 2 × NVIDIA L40S. Full args: train_run_config.json. Checkpoints were saved every 400 steps; this is step 3,200.

Test results (6,923-clip identity-independent test split)

For the JSON tasks the model was decoded with the training-style prompt prefix and a { prefill, as in the paper's inference scripts.