CG-MLLM 4B image-to-3D
CG-MLLM: Captioning and Generating 3D Content via Multi-modal Large Language Models (ICML 2026)
CG-MLLM is a 3D multimodal large language model built on a Qwen3-VL backbone and a Hunyuan3D-2.1 VAE.
This repository is the 4B image-to-3D checkpoint. It is a specialized training of that architecture for image-to-3D on HY3D-Bench, on Qwen3-VL-4B-Instruct. It is not the multi-task checkpoint.
The v0.1 multi-task checkpoint (Qwen3-VL-2B) is at JreamH/CGMLLM.
Links
- Code (how to run): https://github.com/dreaming-huang/CG-MLLM
- Paper (arXiv): https://arxiv.org/abs/2601.21798
- Hugging Face Papers: https://huggingface.co/papers/2601.21798
- Project page: https://cv.jream.top/CG-MLLM-page/
- ICML poster: https://icml.cc/virtual/2026/poster/63909
Model Details
| Item | Value |
|---|---|
| Version | 4B image-to-3D |
| Vision-language backbone | Qwen3-VL-4B-Instruct |
| 3D latent tokenizer | Hunyuan3D-2.1 VAE |
| Training | Specialized image-to-3D training on HY3D-Bench |
| Tasks | Image-to-3D |
| Checkpoint file | ema.safetensors |
| Venue | ICML 2026 |
| License | Apache-2.0 |
| Authors | Junming Huang, Chi Wang, Letian Li, Guangkai Xu, Donglin Huang, Hao Chen, Qiang Dai, Weiwei Xu |
| Affiliations | Zhejiang University; LIGHTSPEED |
Usage is documented in the GitHub repository. This card does not duplicate the inference commands.
Citation
If you find this work useful, please cite:
@article{huang2026cg,
title={CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models},
author={Huang, Junming and Wang, Chi and Li, Letian and Xu, Guangkai and Huang, Donglin and Chen, Hao and Dai, Qiang and Xu, Weiwei},
journal={arXiv preprint arXiv:2601.21798},
year={2026}
}
Acknowledgments
This work builds on Qwen3-VL, BAGEL, and the Hunyuan3D-2.1 shape VAE. Please follow their licenses when using those components.