InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360° Image
Venue: ECCV 2026 · Malmö, Sweden
Introduction
InSpace generates a complete, asset-aware 3D indoor scene from a single 360° (equirectangular) panorama, producing a full-room mesh along with individual, separable, textured furniture meshes. It is built on the TRELLIS.2 O-Voxel representation and extends it with a panorama-native, structure-aware generation pipeline: view-selective cross-attention driven by the camera center, layout-guided structure inversion from monocular depth, a 3D bounding-box estimator, and asset-aware shape and texture generation with global-local hybrid attention.
Concretely, the scene is produced in three cascaded flow-matching stages: (1) estimating Partial Scene Geometry (PSG) as a spatial prior, (2) generating coarse scene structure (CSG) with view-selective cross-attention, and (3) producing detailed layout and asset geometry with textures through a global-local hybrid attention.
Model Details
- Model Type: Three-stage flow-matching framework on O-Voxel structured latents, with a CenterPoint-based 3D bounding-box estimator
- Input: Single 360° equirectangular (ERP) panorama
- Output: Complete 3D indoor scene (structural layout + separable, textured asset meshes with PBR materials)
- Base Model: TRELLIS.2-4B (sparse-structure / shape / texture VAEs and decoders)
Key Features
- Structure-aware scene generation: Recovers a coherent global layout from a single 360° image, not just isolated assets.
- Asset-aware output: The scene is decomposed into a layout (floor and walls) and individual objects, each exported as its own mesh rather than a single fused blob.
- View-selective cross-attention: The panorama is unwrapped into 6 cubemap faces (FOV 120°), and each voxel attends only to the faces visible from its 3D position.
- Layout-Guided Structure Inversion (optional): A monocular-depth (Depth-Anything-2) point cloud, the Partial Scene Geometry, seeds coarse generation via SDEdit-style noise inversion for better room-scale fidelity.
- PBR materials: Base color, roughness, metallic, and opacity, inherited from the TRELLIS.2 texture decoder.
Checkpoints
| Folder | Component | Role | Size |
|---|---|---|---|
erp_ss_flow_img_dit_L_16l8_bf16_spatial/ | Coarse geometry | Coarse scene structure (sparse-structure flow, view-selective cross-attention) | ~4.9 GB |
bbox_centerpoint/ | 3D BBox | Per-asset oriented bounding-box estimator (CenterPoint) | ~48 MB |
erp_slat_flow_img2shape_asset_aware_bf16/ | Asset shape | Asset-aware shape generation | ~4.9 GB |
erp_slat_flow_imgshape2tex_asset_aware_bf16/ | Asset texture | Asset-aware texture generation (PBR) | ~4.9 GB |
Each folder holds the EMA weight under ckpts/. Model configs ship with the code repository
(under configs/), so no config.json is needed here.
Requirements
- System: Tested on Linux.
- Hardware: An NVIDIA GPU with at least 24 GB of memory (verified on NVIDIA A100 and H100).
- Software:
- The CUDA Toolkit (recommended 12.4).
- Conda for managing dependencies.
- Python 3.8 or higher.
Usage
Please refer to the official GitHub repository for
installation. InSpace is run through the repository's scripts (demo/app_inspace_*.py,
eval/pipeline/eval_pipeline.py), which load these checkpoints and chain the multi-stage pipeline
together.
# 1. Get the code and set up the environment (same env as TRELLIS.2)
git clone https://github.com/kookie12/InSpace --recursive && cd InSpace
. ./setup.sh --new-env --basic --flash-attn --nvdiffrast --nvdiffrec --cumesh --o-voxel --flexgemm
# 2. Download the checkpoints into ckpts/ (repo mirrors the local layout)
pip install -U "huggingface_hub[cli]"
hf download GwanHyeong/InSpace --include "ckpts/*" --local-dir .
# 3a. Interactive demo (pick a scene, run the pipeline stage by stage)
python demo/app_inspace_erp_front.py --port 7860
# 3b. Batch inference over the test set
python eval/pipeline/eval_pipeline.py \
--data_dir datasets/ERP_3D_FRONT_test \
--noise_mode sdedit --sdedit_alpha 0.5 --bbox_mode predicted
The inference code loads each checkpoint from ckpts/<folder>/ckpts/*.pt; the matching model
config is read from the code repository's configs/ directory.
Dataset
InSpace is trained on ERP-FRONT-30K, a paired ERP-Image-to-3D indoor scene dataset built on 3D-FRONT, with 26.5K training and 2.5K test ERP-image-mesh pairs (~30K total). Each room is paired with 360° ERP observations rendered from inside the scene and covers a wide range of room sizes.
hf download GwanHyeong/ERP-FRONT-30K --repo-type dataset --local-dir datasets/
Known Limitations
- Domain of training data: InSpace is trained on ERP-FRONT (synthetic 3D-FRONT scenes). Results on real captured panoramas may vary; for real images the pipeline relies on monocular depth to build the Partial Scene Geometry.
- Raw mesh artifacts: As with TRELLIS.2, generated raw meshes may occasionally contain small holes or minor topological discontinuities; mesh post-processing (hole-filling, remeshing) is provided.
Citation
InSpace has been accepted to ECCV 2026. If you find our work useful, please cite:
@article{koo2026inspace,
title = {InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360{\deg} Image},
author = {Koo, Gwanhyeong and Kim, Hyunsu and Kim, Youngji and Lee, Taejae and
Lim, Siwoo and Yoon, Sunjae and Yeon, Suyong and Yoo, Chang D.},
journal = {arXiv preprint arXiv:2607.03990},
year = {2026}
}
License
Released under the MIT License. This work builds on TRELLIS.2 (MIT, Microsoft). Some dependencies (e.g. nvdiffrast, nvdiffrec) carry their own licenses.