ShadeNet 28M
A lightweight inverse rendering model that decomposes RGB images into PBR material maps (basecolor, normal, roughness/metallic/depth) and reconstructs them back to RGB.
Examples
Examples
Each result shows: Input (blue) → Basecolor, Normal, Depth, Roughness, Metallic (green) → Recon RGB (orange). Click an image to view full size.
Architecture
The model is a MobileNetUNet (27.9M params) with:
- MobileNetV2 backbone (frozen except last 8 layers) for feature extraction
- Parallel Encoder for additional learned features
- UNet-style decoder with skip connections, channel attention, and spatial attention
- Dual mode forward pass:
- Mode 0: RGB → Inverse Maps (basecolor, normal, roughness/metallic/depth)
- Mode 1: Inverse Maps → RGB reconstruction
Output Maps
| Map | Channels | Description |
|---|---|---|
| Basecolor | 3 | Albedo / diffuse color |
| Normal | 3 | Surface normals (tangent space) |
| Roughness | 1 | R channel of RMD - surface roughness |
| Metallic | 1 | G channel of RMD - metalness |
| Depth | 1 | B channel of RMD - relative depth |
| RGB | 3 | Reconstructed RGB from inverse maps |
Files
shadenet/
├── app.py # Gradio Space app
├── inference.py # Standalone inference script (CLI)
├── inference_utils.py # Inference utilities (tiling, compositing)
├── model.py # Model architecture
├── layers.py # Layer components
├── config.py # Configuration
├── requirements.txt # Python dependencies
├── README.md # This file
├── checkpoints/
│ └── last.ckpt # PyTorch Lightning checkpoint (model weights)
└── onnx/
├── model_mode0.onnx # Mode 0 ONNX (RGB → inverse maps)
├── model_mode0_quantized.onnx # Quantized mode 0
├── model_mode1.onnx # Mode 1 ONNX (inverse maps → RGB)
└── model_mode1_quantized.onnx # Quantized mode 1
Usage
Gradio Space
A HuggingFace Space hosts this model as an interactive web app — upload an image in your browser and see results instantly, no installation needed.
The app.py in this repo is the Space entrypoint. To create one:
- Go to huggingface.co/new-space
- Select Gradio SDK, choose this repo as the source
- Space will auto-launch with the Gradio interface
CLI Inference
pip install -r requirements.txt
python inference.py input.jpg --output_dir ./output
ONNX Inference
The onnx/ folder contains exported ONNX models for deployment without PyTorch:
model_mode0.onnx/model_mode0_quantized.onnx: RGB → basecolor, normal, RMDmodel_mode1.onnx/model_mode1_quantized.onnx: Inverse maps → RGB
Input shape: [1, 3, 512, 512], values in [-1, 1]
Training
Trained on Flickr8k with paired inverse-rendered data. The model learns both forward (RGB→inverse) and reverse (inverse→RGB) mappings simultaneously using a combined L1 + MSE loss per output map.
- Optimizer: AdamW / Prodigy
- Image size: 512×512
- Precision: 16-mixed
- Loss weights: basecolor=1.0, normal=1.5, RMD=1.0, RGB=1.0
Citation
@software{shadenet,
author = {Sachin},
title = {ShadeNet},
year = {2026},
}