distil-qwen3-0.6b-posthog-narrator
A 0.6B specialist that turns a raw PostHog session (timestamped event stream) into a 3-sentence plain-English story of what the user did. One of three tools in the distil-posthog-traffic-analyser harness; trained on the Distil Labs platform.
Task contract
Input — a rendered session prompt:
Session: <sessionId>
User: <distinctId>
Span: <startedAt> → <endedAt> (<duration>, <n> events)
Events:
- <ISO timestamp> <event_name> { key=value, ... }
...
Write exactly 3 sentences describing what this user did.
Output — exactly 3 sentences of past-tense prose. No preamble, no lists, no markdown. Faithful to the events: concrete page names, button labels, error messages, and search queries; frustration signals (rage clicks, repeated failures, abandonment) called out; nothing invented.
Training
- Base model: Qwen3-0.6B (Apache 2.0)
- Teacher: openai.gpt-oss-120b (Apache 2.0)
- Seed data: 25 hand-authored, schema-validated session/narration pairs,
committed at
examples/seeds/narrator.jsonl(20 train / 5 held-out test), generated from typed source byscripts/build_seeds.ts - Synthetic expansion: 10,033 examples generated and validated by the Distil Labs platform from the seed set
- Method: platform-managed fine-tune (task type: question-answering)
Evaluation
Held-out test set (n=5), scored by the platform's LLM judge:
| Untrained Qwen3-0.6B | Teacher (gpt-oss-120b) | This model | |
|---|---|---|---|
| LLM-as-a-Judge | 0.00% (0/5) | 100.00% (5/5) | 100.00% (5/5) |
| ROUGE | 39.20% | 62.07% | 63.98% |
The tuned 0.6B student matches its 120B teacher's judge score on this task.
Live contract check (exactly 3 sentences, length bounds) on 10 sessions — 5 held-out seed sessions plus the repo's 5 bundled demo sessions: 10/10, reproduced on both the platform's hosted vLLM endpoint and this GGUF running locally via Ollama. In the same live setup the untrained base violated the 3-sentence format on 3–4 of 10 sessions per run and fabricated events (e.g. reporting a successful login on a failed password-reset session).
Usage (Ollama)
ollama create posthog-narrator -f Modelfile # FROM ./<this gguf>
Then in the harness .env:
TOOL_NARRATOR_MODEL=posthog-narrator
See the repo README
for the full three-tool pipeline (bun run demo runs it end-to-end, no API key).