CFPO-G-Qwen2.5-VL-3B
CFPO-G-Qwen2.5-VL-3B is a multimodal reasoning model trained from Qwen2.5-VL-3B-Instruct with CFPO-G, a GRPO-based variant of CounterFactual Policy Optimization (CFPO).
CFPO is designed to improve causal consistency between visual perception and textual reasoning in Large Vision-Language Models. Instead of only optimizing final-answer correctness, CFPO introduces a counterfactual visual-evidence intervention during policy optimization. The model is encouraged to produce reasoning that is sensitive to relevant visual evidence rather than relying mainly on language priors or shortcut reasoning.
This checkpoint is intended for research on multimodal reasoning, visual grounding, and reinforcement learning for vision-language models.
Model Details
- Model name: CFPO-G-Qwen2.5-VL-3B
- Base model: Qwen/Qwen2.5-VL-3B-Instruct
- Training method: CFPO-G, i.e. CFPO integrated with GRPO
- Training dataset: ViRL39K
- Task type: image-text-to-text multimodal reasoning
- Input: image + text prompt
- Output: textual reasoning and final answer
- Project repository: https://github.com/Raven-July/CFPO
- Paper: https://arxiv.org/abs/2606.23206
Method Summary
CFPO introduces a cross-modal counterfactual intervention into the reinforcement learning loop. Given an image-question pair, the model first produces a factual policy distribution from the original visual-textual representation. CFPO then constructs a counterfactual path by suppressing high-saliency cross-modal visual cues at the representation level.
If removing critical visual evidence does not affect the model prediction, the answer is likely driven by language priors rather than genuine visual reasoning. CFPO therefore regularizes the policy by encouraging divergence between factual and counterfactual policy distributions.
In this checkpoint, CFPO is integrated with GRPO, denoted as CFPO-G.
Training Data
The model is trained on ViRL39K, a verifiable multimodal reasoning dataset containing approximately 38.9K image-question-answer pairs for vision-language reinforcement learning.
Please check the dataset license and original sources before redistribution or commercial use.
Training Details
Key training settings:
| Item | Value |
|---|---|
| Base model | Qwen2.5-VL-3B-Instruct |
| Training method | CFPO-G |
| RL backbone | GRPO |
| Training dataset | ViRL39K |
| Number of QA pairs | 38,870 |
| Training epochs | 2 |
| Learning rate | 1e-6 |
| Weight decay | 1e-2 |
| Responses per prompt | 5 |
| Response format | <think>...</think> + \boxed{} |
| Counterfactual coefficient | without Ent regularization |
| Saliency threshold | μ + 2σ |
| Hardware | 2 × NVIDIA A800 80G |
Citation
If you use this model or the CFPO method, please cite:
@misc{yu2026cfpocounterfactualpolicyoptimization,
title={CFPO: Counterfactual Policy Optimization for Multimodal Reasoning},
author={Zhangyuan Yu and Wanran Sun and Guangjing Yang and Xiaohu Wu and Qicheng Lao},
year={2026},
eprint={2606.23206},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2606.23206}
}
Please also cite Qwen2.5-VL if you use this checkpoint:
@misc{qwen2.5-VL,
title={Qwen2.5-VL},
url={https://qwenlm.github.io/blog/qwen2.5-vl/},
author={Qwen Team},
month={January},
year={2025}
}
License
This repository is released under the Apache-2.0 license. The base model is derived from Qwen/Qwen2.5-VL-3B-Instruct; users should also comply with the license and usage terms of the original Qwen2.5-VL model.
Acknowledgements
This model is based on Qwen2.5-VL-3B-Instruct and trained with the CFPO framework. The CFPO implementation builds upon existing open-source reinforcement learning infrastructure for large vision-language models.