SpaceOm

📚 Contents

• 🧠 Model Overview
• 📊 Evaluation & Benchmarks
• 📊 External Benchmark
• 🏃‍♀️ Running SpaceOm
• 🏋️‍♂️ Training Configuration
• 📂 Dataset Info
• ⚠️ Limitations
• 📜 Citation

Model Overview

SpaceOm improves over SpaceThinker by adding:

• the target module o_proj in LoRA fine-tuning
• SpaceOm dataset for longer reasoning traces
• Robo2VLM-Reasoning dataset for more robotics domain and MCVQA examples

The choice to include o_proj among the target modules in LoRA finetuning was inspired by the study here, which argues for the importance of this module in reasoning models.

The reasoning traces in the SpaceThinker dataset average ~200 "thinking" tokens so now we've included longer reasoning traces in the training data to help the model use more tokens in reasoning.

Aiming to improve alignment for robotics applications, we've trained with synthetic reasoning traces, derived from the Robo2VLM-1 dataset.

Running SpaceOm

Ollama

To launch with ollama, run:

ollama run hf.co/remyxai/SpaceOm:latest

or

ollama run remyxai/spaceom

llama.cpp

To run locally with llama.cpp, install and build this branch and download the .gguf weights here

./llama-qwen2vl-cli -m spaceom-F16.gguf
--mmproj spaceom-vision.gguf
--image images/example_1.jpg --threads 24 -ngl 9
-p "Does the man in blue shirt working have a greater \
height compared to the wooden pallet with boxes on floor?" 

Transformers

Run locally using Transformers

import torch
from PIL import Image
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
import requests
from io import BytesIO

# Configuration
model_id = "remyxai/SpaceOm"
image_path = "images/example_1.jpg"  # or local path
prompt = "What can you infer from this image about the environment?"
system_message = (
  "You are VL-Thinking 🤔, a helpful assistant with excellent reasoning ability. "
  "You should first think about the reasoning process and then provide the answer. "
  "Use <think>...</think> and <answer>...</answer> tags."
)

# Load model and processor
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id, device_map="auto", torch_dtype=torch.bfloat16
)
processor = AutoProcessor.from_pretrained(model_id)

# Load and preprocess image
if image_path.startswith("http"):
    image = Image.open(BytesIO(requests.get(image_path).content)).convert("RGB")
else:
    image = Image.open(image_path).convert("RGB")
if image.width > 512:
    ratio = image.height / image.width
    image = image.resize((512, int(512 * ratio)), Image.Resampling.LANCZOS)

# Format input
chat = [
    {"role": "system", "content": [{"type": "text", "text": system_message}]},
    {"role": "user", "content": [{"type": "image", "image": image},
                                {"type": "text", "text": prompt}]}
]
text_input = processor.apply_chat_template(chat, tokenize=False,
                                                  add_generation_prompt=True)

# Tokenize
inputs = processor(text=[text_input], images=[image],
                                      return_tensors="pt").to("cuda")

# Generate response
generated_ids = model.generate(**inputs, max_new_tokens=1024)
output = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

print("Response:
", output)

Dataset Info

The SpaceThinker dataset includes over 12K samples synthesized using VQASynth on a subset of images in the localized narratives split of the cauldron. SpaceThinker is formatted similar to the Llama-Nemotron-Post-Training-Dataset-v1 to toggle reasoning.

The SpaceOm dataset includes ~1K samples synthesized using VQASynth to include longer reasoning traces.

The Robo2VLM-Reasoning datasert is a subset of the original Robo2VLM dataset modified to include reasoning traces.

These datasets were combined to create the final training data for this model.

The model builds upon the ideas from SpatialVLM (Chen et al., 2024), introducing synthetic reasoning traces grounded on a 3D scene reconstruction pipeline using Molmo, VGGT, SAM2.

Training SpaceOm

PEFT Configuration

• Architecture: Qwen2.5-VL-3B
• Base model: UCSC-VLAA/VLAA-Thinker-Qwen2.5VL-3B
• Method: LoRA finetuning (PEFT)
• LoRA Alpha: 256
• LoRA Rank: 128
• Target Modules: q_proj, v_proj, o_proj
• Optimizer: AdamW (lr=2e-5), batch size = 1, epochs = 3
• Max input length: 1024 tokens

Reproduce LoRA SFT training with included script:

python train.py

Model Evaluation

OmniSpatial

Benchmark leaderboard with SpaceOm highlighted.

See full SpaceOm results here for the OmniSpatial benchmark.

SpatialScore

Top scores in each category are bolded in partial table of 3B/4B models.

See all results for evaluating SpaceOm on the SpatialScore benchmark.

Here we add SpaceOm to the 3B/4B table for comparison on the SpatialScore-Hard split:

See all results for SpatialScore-Hard.

SpaCE-10

Top scores in each category are bolded in partial table of 3B/4B models.

Legend:

• EQ: Entity Quantification
• SQ: Scene Quantification
• SA: Size Assessment
• OO: Object-Object spatial relations
• OS: Object-Scene spatial relations
• EP: Entity Presence
• FR: Functional Reasoning
• SP: Spatial Planning

ℹ️ Note: Scores for SpaceQwen, SpaceThinker, SpaceOm are generated via gpt_eval_score on single-choice (*-single) versions of the SpaCE-10 benchmark tasks. Other entries reflect leaderboard accuracy scores from the official SpaCE-10 evaluation table.

Read more about the SpaCE-10 benchmark or see results here

MindCube

Consider the Qwen2.5-VL-3B-Instruct evaluation:

The following table highlights evaluation results where SpaceOm outperformed the Qwen2.5-VL-3B-Instruct base model, averaging 41.48 % accuracy over the tested cog map output settings.

See the results of the MindCube benchmark evaluation from Spatial Mental Modeling from Limited Views.

External Benchmark

This model, SpaceOm, was presented in the paper Spatial Mental Modeling from Limited Views.

Project Page: https://mll-lab-nu.github.io/mind-cube

Code: https://github.com/mll-lab-nu/MindCube

Limitations

• Performance may degrade in cluttered environments or camera perspective.
• This model was fine-tuned using synthetic reasoning over an internet image dataset.
• Multimodal biases inherent to the base model (Qwen2.5-VL) may persist.
• Not intended for use in safety-critical or legal decision-making.

Users are encouraged to evaluate outputs critically and consider fine-tuning for domain-specific safety and performance. Distances estimated using autoregressive transformers may help in higher-order reasoning for planning and behavior but may not be suitable replacements for measurements taken with high-precision sensors, calibrated stereo vision systems, or specialist monocular depth estimation models capable of more accurate, pixel-wise predictions and real-time performance.

Citation

@article{chen2024spatialvlm,
  title = {SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities},
  author = {Chen, Boyuan and Xu, Zhuo and Kirmani, Sean and Ichter, Brian and Driess, Danny and Florence, Pete and Sadigh, Dorsa and Guibas, Leonidas and Xia, Fei},
  journal = {arXiv preprint arXiv:2401.12168},
  year = {2024},
  url = {https://arxiv.org/abs/2401.12168},
}

@misc{qwen2.5-VL,
  title = {Qwen2.5-VL},
  url = {https://qwenlm.github.io/blog/qwen2.5-vl/},
  author = {Qwen Team},
  month = {January},
  year = {2025}
}

@misc{vl-thinking2025,
  title={SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models },
  author={Hardy Chen and Haoqin Tu and Fali Wang and Hui Liu and Xianfeng Tang and Xinya Du and Yuyin Zhou and Cihang Xie},
  year = {2025},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/UCSC-VLAA/VLAA-Thinking}},
}


@article{wu2025spatialscore,
    author    = {Wu, Haoning and Huang, Xiao and Chen, Yaohui and Zhang, Ya and Wang, Yanfeng and Xie, Weidi},
    title     = {SpatialScore: Towards Unified Evaluation for Multimodal Spatial Understanding},
    journal   = {arXiv preprint arXiv:2505.17012},
    year      = {2025},
}


@misc{yin2025spatial,
  title        = {Spatial Mental Modeling from Limited Views},
  author       = {Baiqiao Yin and Qineng Wang and Pingyue Zhang and Jianshu Zhang
                  and Kangrui Wang and Zihan Wang and Jieyu Zhang
                  and Keshigeyan Chandrasegaran and Han Liu and Ranjay Krishna
                  and Saining Xie and Manling Li and Jiajun Wu and Li Fei-Fei},
  year         = {2025},
  archivePrefix= {arXiv},
  eprint       = {2506.21458},
  primaryClass = {cs.AI},
  url          = {https://arxiv.org/abs/2506.21458}
}