SpatialAxiom-35B-A3B

Introduction

We are thrilled to release SpatialAxiom, an open spatial intelligence model for general spatial reasoning that achieves significant advances in 3D relational inference, perspective taking, multi-view correspondence, and embodied video understanding. Built on the Qwen3.5 VLM family without altering the base architecture, SpatialAxiom derives its strength from a spatial data-centric training recipe: a systematic taxonomy of spatial tasks, balanced task distribution, and data synthesis to raise data quality, applied to large-scale spatial supervision spanning indoor scenes, egocentric views, and multi-camera settings. Trained purely with full-parameter SFT, SpatialAxiom serves as a clean starting point for further post-training. We release two open-weight models: SpatialAxiom-9B, a compact dense model, and SpatialAxiom-35B-A3B, a mixture-of-experts model with 3B active parameters.

Highlights

• Leading spatial reasoning: On average, our models surpass proprietary models and larger open-source alternatives, with leading results on VSI-Bench, MMSI-Bench, MindCube, ViewSpatial, and EmbSpatial.

• Spatial data-centric training recipe: A systematic taxonomy of spatial tasks, balanced task distribution, and data synthesis to raise data quality. SpatialAxiom is trained purely with full-parameter SFT and serves as a clean starting point for downstream fine-tuning or RL.

• Qwen3.5 VLM backbone: Inherits the Qwen3.5 vision-language model architecture, preserving a general-purpose multimodal design without task-specific architectural modifications.

• Open-weight release: SpatialAxiom-9B and SpatialAxiom-35B-A3B are publicly released on Hugging Face, compatible with transformers and vLLM out of the box.

Model Overview

• Type: Causal Language Model with Vision Encoder
• Training Stage: Pre-training & Post-training
• Language Model
  • Number of Parameters: 35B in total and 3B activated
  • Hidden Dimension: 2048
  • Token Embedding: 248320 (Padded)
  • Number of Layers: 40
  • Hidden Layout: 10 × (3 × (Gated DeltaNet → MoE) → 1 × (Gated Attention → MoE))
  • Gated DeltaNet:
    • Number of Linear Attention Heads: 32 for V and 16 for QK
    • Head Dimension: 128
  • Gated Attention:
    • Number of Attention Heads: 16 for Q and 2 for KV
    • Head Dimension: 256
    • Rotary Position Embedding Dimension: 64
  • Mixture Of Experts
    • Number of Experts: 256
    • Number of Activated Experts: 8 Routed + 1 Shared
    • Expert Intermediate Dimension: 512
  • LM Output: 248320 (Padded)
  • MTP: trained with multi-steps
• Context Length: 262,144 natively and extensible up to 1,010,000 tokens.

Spatial Benchmark Results

Quickstart

[!Important] SpatialAxiom-35B-A3B is fine-tuned from Qwen3.5-35B-A3B for direct (non-thinking) responses only. It does not produce thinking blocks (no chain-of-thought wrapped in special tags). Do not enable thinking mode at inference time (enable_thinking must remain false). The model does not support /think or /nothink soft switches.

Transformers (image)

The latest transformers with Qwen3.5 VLM support is required. Make sure torchvision and pillow are installed.

from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "<HF_REPO_ID>"
model = AutoModelForMultimodalLM.from_pretrained(
    model_id, dtype="bfloat16", device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "room.jpg"},
            {
                "type": "text",
                "text": "If I stand at the door facing the bed, is the chair to my left or right?",
            },
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    enable_thinking=False,
).to(model.device)

out = model.generate(**inputs, max_new_tokens=32768)
print(
    processor.batch_decode(
        out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True
    )[0]
)

Transformers (video)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "video", "url": "https://example.com/walkthrough.mp4"},
            {
                "type": "text",
                "text": "Estimate the size of the room in square meters and describe the route taken.",
            },
        ],
    }
]
# Same apply_chat_template / generate flow as above.

Serving with vLLM

vLLM is recommended for production serving. A recent vLLM build with Qwen3.5 support is required, which can be installed in a fresh environment with:

uv pip install vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly

vllm serve <HF_REPO_ID> \
  --port 8000 \
  --tensor-parallel-size 4 \
  --max-model-len 65536 \
  --default-chat-template-kwargs '{"enable_thinking": false}' \
  --served-model-name spatialaxiom-35b-a3b

OpenAI-compatible API

Install the OpenAI SDK and point it at your vLLM server:

pip install -U openai

export OPENAI_BASE_URL="http://localhost:8000/v1"
export OPENAI_API_KEY="EMPTY"

Image input

from openai import OpenAI

client = OpenAI()

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://example.com/room.jpg"}},
            {
                "type": "text",
                "text": "If I stand at the door facing the bed, is the chair to my left or right?",
            },
        ],
    }
]

chat_response = client.chat.completions.create(
    model="spatialaxiom-35b-a3b",
    messages=messages,
    max_tokens=32768,
    temperature=0,
    top_p=0.95,
)
print(chat_response.choices[0].message.content)

Video input

from openai import OpenAI

client = OpenAI()

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video_url",
                "video_url": {"url": "https://example.com/walkthrough.mp4"},
            },
            {
                "type": "text",
                "text": "Describe the spatial layout and the route taken through the room.",
            },
        ],
    }
]

# Video frame sampling (vLLM): default fps=2, do_sample_frames=True.
# Customizing fps via extra_body requires launching vLLM with
# --media-io-kwargs '{"video": {"num_frames": -1}}'.
chat_response = client.chat.completions.create(
    model="spatialaxiom-35b-a3b",
    messages=messages,
    max_tokens=32768,
    temperature=0,
    top_p=0.95,
    extra_body={
        "mm_processor_kwargs": {"fps": 2, "do_sample_frames": True},
    },
)
print(chat_response.choices[0].message.content)

If the vLLM server was not launched with --default-chat-template-kwargs '{"enable_thinking": false}', add it per request:

extra_body={
    "chat_template_kwargs": {"enable_thinking": False},
}

Citation

If you find our work helpful, feel free to give us a cite.

@misc{spatialaxiom,
    title  = {SpatialAxiom: An Open Spatial Intelligence Model for General Spatial Reasoning},
    author = {Lou, Yujing and Chen, Pingyi and Cao, Shen and Gu, Jiaqi and Guo, Jinhui and Tong, Jintao and Hao, Yunzhuo and Liu, Yao and Wu, Yue and Fan, Lubin and Ye, Jieping},
    month  = {July},
    year   = {2026},
    url    = {https://d2i-ai.github.io/SpatialAxiom}
}