LSP-DETR: Efficient and Scalable Nuclei Segmentation in Whole Slide Images

Matěj Pekár, Vít Musil, Rudolf Nenutil, Petr Holub, Tomáš Brázdil

[GitHub]

LSP-DETR (Local Star Polygon DEtection TRansformer) is a lightweight, efficient, and end-to-end deep learning model for nuclei instance segmentation in histopathological images. It combines a DETR-based transformer decoder with star-convex polygon shape descriptors to enable accurate and fast segmentation without complex post-processing.

NOTE: This checkpoint was pretrained on the complete PanNuke dataset without class labels. This configuration prioritizes maximum recall, making it ideal for robust nucleus localization.

from transformers import AutoModelForObjectDetection, AutoImageProcessor


processor = AutoImageProcessor.from_pretrained(
    "RationAI/LSP-DETR", trust_remote_code=True
)
model = AutoModelForObjectDetection.from_pretrained(
    "RationAI/LSP-DETR", trust_remote_code=True
)

inputs = processor(img, device=device, return_tensors="pt")
outputs = model(**inputs)
results = processor.post_process(outputs)
results = processor.post_process_instance(
  results, height=img.size[1], width=img.size[0]
)

Citing LSP-DETR

@misc{pekar2026lspdetr,
  title={LSP-DETR: Efficient and Scalable Nuclei Segmentation in Whole Slide Images}, 
  author={Matěj Pekár and Vít Musil and Rudolf Nenutil and Petr Holub and Tomáš Brázdil},
  year={2026},
  eprint={2601.03163},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2601.03163}
}