Rex-3D
Rex-3D turns a single image into a textured 3D mesh that is ready to drop into a game.
It is built on InstantMesh (Apache-2.0). The multi-view stage (the Zero123++ UNet, which imagines the object from 6 angles) is fine-tuned on a curated set of about 10,000 permissively licensed Objaverse-LVIS objects. The reconstruction stage is InstantMesh-large, unchanged.
What's in this repo
| Path | What it is |
|---|---|
unet/diffusion_pytorch_model.bin | Fine-tuned multi-view UNet (LoRA merged in, fp16). Drop-in replacement for InstantMesh's diffusion_pytorch_model.bin. |
lora/rex3d_lora.safetensors | The LoRA on its own (rank 32 on every attention projection). |
rex3d_infer.py | One command: background removal → Rex-3D → high-res texture → cleanup and decimation. |
samples/ | Held-out objects during training: input, generated 6 views, ground truth. samples/base.png is the untouched InstantMesh UNet, for comparison. |
checkpoints/ | Resumable training state. |
Use it
git clone https://github.com/TencentARC/InstantMesh && cd InstantMesh
# install InstantMesh's requirements (see their README), then:
pip install "rembg[gpu]" pymeshlab omegaconf pillow huggingface_hub
wget https://huggingface.co/Radinkazemian/Rex-3d/resolve/main/rex3d_infer.py
python rex3d_infer.py my_image.png --faces 5000 --tex 2048
The result is written to outputs/my_image_rex3d.obj. --faces 5000 is a good budget for Roblox. Raise it for more detail.
Training
- Data: a curated slice of Objaverse-LVIS. It is balanced across the LVIS categories, keeps only CC-BY, CC-BY-SA and CC0 models, and filters on face count, file size and popularity. Each object is rendered in Blender Cycles as 1 random input view plus the 6 fixed Zero123++ views (azimuth +30/90/150/210/270/330°, elevation 20/-10°, FOV 30°) on a transparent background.
- Method: LoRA (rank 32) on the white-background Zero123++ UNet from InstantMesh, v-prediction loss, fp16, on Kaggle T4 GPUs.
Licenses
InstantMesh code and its reconstruction model are Apache-2.0. The multi-view UNet is derived from sudo-ai/zero123plus-v1.2, whose weights are, to my knowledge, released under CC-BY-NC 4.0 (non-commercial). Check that license before any commercial use of Rex-3D's UNet. The training objects are CC-BY / CC-BY-SA / CC0 from Objaverse (Sketchfab). Attribution is required for CC-BY items.