SpatioLM-Understanding-InternVL3.5
This is the official SpatioLM Understanding checkpoint based on InternVL3.5 (8B). It is intended for spatial reasoning and scene understanding.
SpatioLM adds a plug-and-play spatio-vision module to a frozen vision-language model and learns physically coherent representations from pseudo depth and camera-ray supervision. No additional 3D input is required at inference time.
Resources
Installation
git clone https://github.com/xiaomi-research/spatio-lm.git
cd spatio-lm
pip install -e .
Image inference
import torch
from lmms_eval.models.simple.internvl2 import load_image
from PIL import Image
from transformers import AutoTokenizer
from spatiolm.models import InternVL3RChatModel
checkpoint = "xiaomi-research/SpatioLM-Understanding-InternVL3.5"
image = Image.open("/path/to/image.jpg").convert("RGB")
model = InternVL3RChatModel.from_pretrained(
checkpoint,
dtype=torch.bfloat16,
low_cpu_mem_usage=True,
).eval().cuda()
tokenizer = AutoTokenizer.from_pretrained(
checkpoint,
trust_remote_code=True,
use_fast=False,
)
pixel_values = load_image(image, input_size=448).to(
device="cuda",
dtype=torch.bfloat16,
)
answer = model.chat(
tokenizer,
pixel_values,
"Which object is closer to the camera?",
{"max_new_tokens": 128, "do_sample": False},
)
print(answer)
For video inference, benchmark evaluation, training details, and the Action checkpoint interface, see the SpatioLM repository.
Intended use and limitations
- This checkpoint is intended for research on physical spatial intelligence.
- Outputs can be inaccurate and should not be used as the sole signal in safety-critical or high-impact decisions.
- Performance can vary with image quality, viewpoint, scene domain, prompting, and video sampling strategy.
- The custom SpatioLM model implementation is required; loading with only stock Transformers auto classes is not supported.
Citation
@inproceedings{wu2026spatiolm,
title={SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models},
author={Wu, Jing and Wu, Jianhua and Guan, Jiayi and Chen, Jiahong and Lu, Jinghui and Ye, Hangjun and Gao, Bingzhao and Chen, Long},
booktitle={International Conference on Machine Learning (ICML)},
year={2026},
note={To appear},
eprint={2608.01899},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.01899}
}