OmniShotCut v1.5 — MLX, 8-bit

uva-cv-lab/OmniShotCut_v1.5 converted for MLX by scripts/convert.py. Run it with the Swift package OmniShotCutMLX:

import OmniShotCutMLX

let detector = try await OmniShotCut.pretrained("starkdmi/OmniShotCut-v1.5-mlx-8bit")
let segments = try await detector.segments(in: videoURL)

• Every linear but the query embedding is quantized to 8 bits, groups of 64; everything else is fp16. 67 MB instead of the original 211 MB.
• Compute in fp32: fp16 activations moved cuts that 8-bit weights did not. 4-bit weights moved one by 29 frames.
• Key names are the official ones, but backbone.0.body. is dropped, each attention's packed in_proj is split into q_proj/k_proj/v_proj, and convolutions are OHWI.

Against the official PyTorch code (fp32, CPU) on three videos, the Swift port with these weights finds the same 385 segments with the same labels, one cut a frame apart.

License

MIT, as is the original. Copyright (c) 2026 UVA Computer Vision Lab.

@article{wang2026omnishotcut,
  title={OmniShotCut: Holistic Relational Shot Boundary Detection with Shot-Query Transformer},
  author={Wang, Boyang and Xu, Guangyi and Zhang, Jiahui and Tang, Zhipeng and Cheng, Zezhou},
  journal={arXiv preprint arXiv:2604.24762},
  year={2026}
}