ColonCrafter: A Depth Estimation Model for Colonoscopy Videos Using Diffusion Priors
ColonCrafter builds upon DepthCrafter and Stable Video Diffusion to provide temporally consistent depth predictions for colonoscopy video.
Model Details
- Model Type: Video Depth Estimation (Diffusion-based)
- Base Architecture: DepthCrafter UNet with LoRA adaptation
- LoRA Configuration:
- Rank: 16
- Target modules:
to_q,to_k,to_v,to_out.0 - Dropout: 0.1
- Precision: FP16
Installation
Please refer to the installation instructions in our repository.
Usage
import torch
from src.depth.models.model import ColonCrafterInference
# Load the model
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = ColonCrafterInference.from_pretrained(
"romainhardy/coloncrafter",
device=device
)
# Prepare video tensor: (N, C, H, W) in [0, 1] range
# video = ...
# Run inference
pred_depth, pred_disparity = model.predict_depth(
video,
num_inference_steps=1,
window_size=16,
overlap=8,
guidance_scale=1.0,
seed=42
)
Citation
If you use this model in your research, please cite:
@article{hardy2025coloncrafter,
title={ColonCrafter: A Depth Estimation Model for Colonoscopy Videos Using Diffusion Priors},
author={Hardy, Romain and Berzin, Tyler and Rajpurkar, Pranav},
journal={arXiv preprint arXiv:2509.13525},
year={2025}
}
Acknowledgments
This model builds upon DepthCrafter and Stable Video Diffusion.