QMSum locator: MiniLM L12 cross-encoder, 375-word windows (promoted configuration)

The stage-1 locator of a two-stage locate-then-summarize pipeline for query-focused meeting summarization on QMSum. It scores fixed-width windows of a meeting transcript against a query; the top-scored windows are greedily packed to a 2,000-word budget and passed, in transcript order, to the summarizer adapter ErtasAI/qmsum-summarizer-lfm2.5-1.2b-lora.

This is the locator behind the pipeline's promoted configuration (test ROUGE-1 35.41). It is part of the release for the paper Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on QMSum (Edward Xi Yang, Ertas AI; arXiv:2609.25028). The protocol-exact sibling is ErtasAI/qmsum-locator-minilm-l6-w900.

Why this configuration exists

The frozen evaluation protocol declares 900-word windows. This model's architecture reads 512 tokens, and a 900-word window tokenises to a median of about 1,150 tokens, so under the protocol setting the scorer silently read only about the first half of every window (97.3% of windows were truncated). Re-chunking at 375 words makes windows fit the context, and with a deeper reranker and a 2,000-word budget it lifted the pipeline by +2.01 ROUGE-1 (95% CI [+0.77, +3.26]) on the official test split. The protocol itself stayed frozen; this configuration is reported in the paper as a documented deviation on exactly two locator-side values (window size and span budget).

Model details

• Base model: cross-encoder/ms-marco-MiniLM-L-12-v2 (33M parameters, Apache 2.0)
• Training: binary relevance over query-window pairs built from QMSum train-split gold spans (a window is positive when it overlaps a gold span). 2 epochs, batch 32, 100 warmup steps, seeded through the frozen protocol (seed 20260723), sentence-transformers CrossEncoder with num_labels=1
• Training cost: about 7 minutes on one NVIDIA RTX 5070 Ti, peak 5.27 GB
• Window size at training and inference: 375 words. Score windows of the size the model was trained on; other sizes put the inputs off-distribution

The system rows this locator affects

Full official QMSum test split, n=281, greedy decoding, one frozen scorer. The summarizer adapter is identical in both rows.

The gap between the rows is not demonstrably robust to training reseeding (the summarizer's seed range on this benchmark is 1.59 ROUGE-1); the paper carries the full analysis.

How to use

from sentence_transformers import CrossEncoder

locator = CrossEncoder("ErtasAI/qmsum-locator-minilm-l12-w375")
scores = locator.predict([(query, window_text) for window_text in windows])

Chunk the transcript into 375-word windows first, then pack the top-scored windows to a 2,000-word budget and restore transcript order. The release repo's pipeline (github.com/ErtasAI/qmsum-retrieved-span-training) implements the exact windowing and packing the published rows used. Scores are unnormalised relevance logits; use them for ranking, and compare them only within one query.

Limits

• Trained and evaluated on English meeting transcripts from QMSum's three domains (academic, product design, parliamentary committee). Other document types are untested.
• Relevance here means overlap with QMSum's annotated gold spans, one annotation of what answers the query. It inherits that annotation's judgment calls.

Provenance

Citation

If you use this locator, please cite the paper, and QMSum alongside it:

@misc{yang2026retrievedspan,
  title         = {Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on {QMSum}},
  author        = {Yang, Edward Xi},
  year          = {2026},
  eprint        = {2609.25028},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  doi           = {10.48550/arXiv.2609.25028},
  url           = {https://arxiv.org/abs/2609.25028},
}

@inproceedings{zhong2021qmsum,
  author    = {Ming Zhong and Da Yin and Tao Yu and Ahmad Zaidi and Mutethia Mutuma and Rahul Jha and Ahmed Hassan Awadallah and Asli Celikyilmaz and Yang Liu and Xipeng Qiu and Dragomir Radev},
  title     = {{QMSum}: A New Benchmark for Query-based Multi-domain Meeting Summarization},
  booktitle = {Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  pages     = {5905-5921},
  year      = {2021},
  note      = {ACL Anthology 2021.naacl-main.472. arXiv:2104.05938},
}

Released by Ertas AI. We build custom small models that run on-device and in your own infrastructure.