QMSum locator: MiniLM L6 cross-encoder, 900-word windows (protocol-exact configuration)

The stage-1 locator of a two-stage locate-then-summarize pipeline for query-focused meeting summarization on QMSum. It scores fixed-width windows of a meeting transcript against a query; the top-scored windows are greedily packed to a 3,000-word budget and passed, in transcript order, to the summarizer adapter ErtasAI/qmsum-summarizer-lfm2.5-1.2b-lora.

This is the locator behind the pipeline's protocol-exact configuration (test ROUGE-1 33.39), the setting declared in the frozen evaluation protocol before any results existed. It is part of the release for the paper Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on QMSum (Edward Xi Yang, Ertas AI; arXiv:2609.25028). The promoted sibling is ErtasAI/qmsum-locator-minilm-l12-w375.

A caveat you should know before using this model

The frozen protocol declares 900-word windows, and this model's architecture reads 512 tokens. A 900-word window tokenises to a median of about 1,150 tokens, so at the protocol setting the scorer reads roughly the first half of most windows (97.3% of windows truncate, mean 47.7% of each window read). The published protocol-exact row includes this behaviour deliberately: the protocol was frozen before the truncation was discovered, and the paper reports it as found rather than quietly repairing it. If you want the stronger configuration, use the promoted sibling, whose windows fit its context.

Model details

• Base model: cross-encoder/ms-marco-MiniLM-L-6-v2 (22.7M parameters, Apache 2.0)
• Training: binary relevance over query-window pairs built from QMSum train-split gold spans (a window is positive when it overlaps a gold span). 2 epochs, batch 32, 100 warmup steps, seeded through the frozen protocol (seed 20260723), sentence-transformers CrossEncoder with num_labels=1
• Training cost: about 1.5 minutes on one NVIDIA RTX 5070 Ti, peak 2.82 GB
• Window size at training and inference: 900 words, per the frozen protocol

The system rows this locator affects

Full official QMSum test split, n=281, greedy decoding, one frozen scorer. The summarizer adapter is identical in both rows.

The gap between the rows is not demonstrably robust to training reseeding (the summarizer's seed range on this benchmark is 1.59 ROUGE-1); the paper carries the full analysis.

How to use

from sentence_transformers import CrossEncoder

locator = CrossEncoder("ErtasAI/qmsum-locator-minilm-l6-w900")
scores = locator.predict([(query, window_text) for window_text in windows])

Chunk the transcript into 900-word windows first, then pack the top-scored windows to a 3,000-word budget and restore transcript order. The release repo's pipeline (github.com/ErtasAI/qmsum-retrieved-span-training) implements the exact windowing and packing the published rows used. Scores are unnormalised relevance logits; use them for ranking, and compare them only within one query.

Limits

• The truncation caveat above: at 900-word windows, scoring reflects mostly the first ~512 tokens of each window.
• Trained and evaluated on English meeting transcripts from QMSum's three domains (academic, product design, parliamentary committee). Other document types are untested.
• Relevance here means overlap with QMSum's annotated gold spans, one annotation of what answers the query. It inherits that annotation's judgment calls.

Provenance

Citation

If you use this locator, please cite the paper, and QMSum alongside it:

@misc{yang2026retrievedspan,
  title         = {Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on {QMSum}},
  author        = {Yang, Edward Xi},
  year          = {2026},
  eprint        = {2609.25028},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  doi           = {10.48550/arXiv.2609.25028},
  url           = {https://arxiv.org/abs/2609.25028},
}

@inproceedings{zhong2021qmsum,
  author    = {Ming Zhong and Da Yin and Tao Yu and Ahmad Zaidi and Mutethia Mutuma and Rahul Jha and Ahmed Hassan Awadallah and Asli Celikyilmaz and Yang Liu and Xipeng Qiu and Dragomir Radev},
  title     = {{QMSum}: A New Benchmark for Query-based Multi-domain Meeting Summarization},
  booktitle = {Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  pages     = {5905-5921},
  year      = {2021},
  note      = {ACL Anthology 2021.naacl-main.472. arXiv:2104.05938},
}

Released by Ertas AI. We build custom small models that run on-device and in your own infrastructure.