3D-PLOT-LLM-7B
Weights of 3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models (NeurIPS 2026).
- Code: https://github.com/jintangxue/3D-PLOT-LLM
- Dataset (PartVerse-QA): https://huggingface.co/datasets/jintangx/PartVerse-QA
- Paper: https://arxiv.org/abs/2606.19828
3D-PLOT-LLM extends PointLLM (Vicuna-v1.5-7B with a frozen Point-BERT encoder on 8192-point Objaverse clouds). The encoder's patch tokens are partitioned into K=16 regions; each region is prefixed with a learnable marker and a reserved vocabulary token <part_k>, refined by a lightweight Marker-Space Refinement module. The model can answer which regions a part description refers to, describe a given set of regions, and caption whole objects, with fewer than one million added parameters.
Usage
git clone https://github.com/jintangxue/3D-PLOT-LLM && cd 3D-PLOT-LLM
source env.sh # set POINTLLM_DATA, BCP_PACK_DIR, PARTVERSE_QA_DIR
python pointllm/eval/eval_partverse_caption2slots.py --model_name jintangx/3D-PLOT-LLM-7B \
--anno_path $PARTVERSE_QA_DIR/eval_c2s.json \
--data_path $POINTLLM_DATA/objaverse_data --bcp_pack_dir $BCP_PACK_DIR
The checkpoint is in PointLLM format (PointLLMLlamaForCausalLM, bf16) and loads with the pinned transformers commit used by the code. Each input object needs its partition pack (<object_id>.pack.npz), shipped with the dataset for the evaluation objects and generated by the repository tools otherwise. See the repository README for installation and the environment used for the paper.
Results
| Benchmark | Metric | Score |
|---|---|---|
| PartVerse-QA caption-to-slots (392 queries) | Jaccard / exact match | 0.459 / 13.78% |
| PartVerse-QA slots-to-caption (196 queries) | SBERT | 62.0 |
Caption-to-slots uses greedy decoding and reproduces the paper's predictions exactly on an A100 with the paper's environment. Slots-to-caption is the mean over 5 sampled runs (temperature 1.0, top-p 0.95, top-k 50). See the paper for the full tables.
License
Non-commercial research use: the weights inherit the Llama 2 community license of the Vicuna backbone and the CC-BY-NC-4.0 license of the PointLLM initialization. The code is CC BY-NC-SA 4.0.
Citation
@article{xue20263d,
title={3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models},
author={Xue, Jintang and Wang, Xinyu and Wu, Yixing and Chen, Jingwen and Kuo, C-C Jay},
journal={arXiv preprint arXiv:2606.19828},
year={2026}
}
Accepted at NeurIPS 2026; the entry will be updated when the proceedings version is published.