Local Minutes contextualizer (Qwen3.5-4B + SFT + RL)
A 4B model that writes retrieval context for excerpts of municipal meeting documents: minutes, agendas and agenda packets. Given the pages before an excerpt and the excerpt itself, it returns the meeting name, the current subject, and short notes that resolve what the excerpt refers to but doesn't explain (first names, roles, acronyms, "the motion", "the parcel"), using only the source. The context is added to the excerpt before it's indexed, a technique known as contextual retrieval.
This is the contextualizer running in production at Local Minutes, a free, searchable index of Vermont municipal records. It replaced Gemini 2.5 Flash-Lite in that pipeline. The full write-up: Fine-tuning Qwen3.5-4B for the Local Minutes pipeline.
Intended use
- Adding context to chunks of civic documents (meeting minutes, agendas, agenda packets) before embedding or keyword indexing, for search or RAG.
- English. Trained on Vermont municipal documents. It also performed well in tests on out-of-domain documents, with no guarantees (see Limitations).
It is not a general assistant, and it isn't meant for question answering or summarization. Its output should never be treated as authoritative on its own; the source document is.
How to use
The model expects one excerpt per request, in the layout it was trained on.
System prompt (exactly this):
Write RAG context notes for the EXCERPT using only the SOURCE FOR CONTEXT. Reply with JSON: meetingName, currentSubject, additionalContext.
User message:
# SOURCE FOR CONTEXT:
{source}
# EXCERPT:
{excerpt}
{source} is the document text before the excerpt. If it's longer than 270 lines, send the first 70 lines (where attendance lists and full names usually are), a line reading ...[earlier discussion omitted]..., and the last 200 lines. Production also caps those two parts at 10,000 and 30,000 characters, and the excerpt at 30,000, so a document stored as a few giant lines still fits the 32K-token window.
Output: JSON with three fields.
{
"meetingName": "Town of Maple Hill Select Board Meeting, March 3, 2026",
"currentSubject": "Adjournment",
"additionalContext": [
"Mark is Mark Hadley and Carol is Carol Ashby Pratt, members of the Select Board."
]
}
Settings: temperature 0, repetition_penalty 1.05, max_tokens 1024, and thinking off (enable_thinking: false). The model was trained without thinking.
With vLLM:
vllm serve DuaneMB/local-minutes-contextualizer-4b --served-model-name contextualizer
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
SYSTEM = ("Write RAG context notes for the EXCERPT using only the SOURCE FOR CONTEXT. "
"Reply with JSON: meetingName, currentSubject, additionalContext.")
def context_for(source: str, excerpt: str) -> str:
r = client.chat.completions.create(
model="contextualizer",
temperature=0,
max_tokens=1024,
messages=[{"role": "system", "content": SYSTEM},
{"role": "user", "content": f"# SOURCE FOR CONTEXT:\n{source}\n\n# EXCERPT:\n{excerpt}"}],
extra_body={"repetition_penalty": 1.05, "chat_template_kwargs": {"enable_thinking": False}},
)
return r.choices[0].message.content
Validate the JSON before using it; in production, a context that fails to parse falls back to another model.
Training
-
Supervised fine-tuning (imitation). LoRA, rank 32, on about 24,000 contexts written by Claude Opus 5.5 for excerpts sampled from random recent Vermont municipal documents at random positions. One epoch, learning rate 2e-4, loss on the context tokens only.
-
Reinforcement learning (GRPO). Starting from the SFT model: LoRA rank 16, 32 excerpts per step with 8 sampled contexts each at temperature 1.0, and a KL penalty (β = 0.02) toward the SFT model. Learning rate 1e-5 for the first 25 steps, then 3e-5. The run went 100 steps; this is the step-80 checkpoint, which guessed less than step 100.
The reward comes from Claude Opus 5.5 in two stages. First, once per excerpt, an answer key: what a reader of the excerpt alone would need explained, and whether the source settles it, leaves it ambiguous, or doesn't say. Then, for each sampled context, a classification of how it handled each item. The points are fixed in code:
What the context did Points Resolved a reference the source settles +1 Got one wrong, or committed to an answer the source doesn't give −3 Said the source doesn't settle it, when it doesn't +0.5 Said it's unclear, when the source settles it −0.5 Any other unsupported claim −3 each Wrong meeting or subject −1.5 each An item that only restates the excerpt −0.25 each Invalid JSON −5 -
The LoRA weights were merged into the base model (bf16) for serving.
The training data isn't released: it's drawn from public records, but it names residents and officials.
Evaluation
Claude Opus 5.5 graded all five systems blind, side by side, with positions rotated, on 631 held-out test excerpts (402 drawn at random, plus every remaining meeting-minutes excerpt in the test set). Percentages are the share of contexts with at least one such error.
| System | Usefulness (1–5) | Unsupported claim | Missed reference | Meeting and subject right |
|---|---|---|---|---|
| Qwen3.5-4B, no fine-tuning | 2.78 | 35.8% | 27.4% | 86.1% |
| Gemini 2.5 Flash-Lite (previous production model) | 2.93 | 25.5% | 23.6% | 82.6% |
| Qwen3.5-4B + SFT | 4.08 | 11.9% | 15.2% | 97.3% |
| Qwen3.5-4B + SFT + RL (this model) | 4.19 | 7.9% | 9.5% | 97.9% |
| Claude Opus 5.5 (the teacher) | 4.51 | 1.1% | 5.6% | 99.5% |
A Claude judge may favor Claude's writing, so the Opus row is best read as a ceiling rather than a fair comparison.
On live data, in a blind comparison on 185 excerpts from one night's new production documents, this model scored 4.16 usefulness with 11.4% unsupported and 10.3% missed, against 2.44, 29.2% and 39.5% for Gemini 2.5 Flash-Lite on the same prompts.
Limitations
- Look-alike names. When two people in the source share a first name, it named the wrong person in 5.3% of contexts in a targeted test, against 1.5% for the SFT model. RL made it resolve first names more boldly, and the training excerpts rarely contain two people with the same first name.
- It still makes mistakes. About 8–11% of its contexts contain at least one claim the source doesn't support. Agenda packets were its weakest document type in the live check (a small sample).
- Narrow training. Vermont municipal documents only, English only, and one prompt layout. It was also tested on out-of-domain documents and performed well, but there are no guarantees outside the kind of documents it was trained on, or with other prompt layouts.
Serving notes
It runs in bf16 on a single 24 GB GPU. In production it's served with vLLM on Runpod Serverless (RTX 4090), at about 1.3 seconds per excerpt. On a warm GPU that comes to about $76 per million excerpts; each cold start, while the worker loads the model, adds a fixed cost.
License
Apache 2.0, the same as the base model, Qwen/Qwen3.5-4B. Released as is, without warranty.
Built by Duane Millar Barlow for Local Minutes.