MiniCPM5-1B MTP / Medusa Speculative Decoding Heads + GGUF Stock Analyst
Fast local LLM inference for financial analysis: GGUF-quantized MiniCPM5-1B stock signal model, bundled with trained multi-token-prediction (MTP) heads for llama.cpp-class speculative decoding speedup — no quality loss, self-contained repo.
Everything needed to run a faster MiniCPM5-1B stock-analyst locally lives here: the quantized GGUF weights, the original LoRA adapter, and a set of Medusa-style MTP heads that let the model draft multiple tokens per forward pass instead of one at a time.
Why this exists
Autoregressive decoding generates one token per forward pass — the single biggest latency tax on every local LLM deployment. DeepSeek's answer was MTP heads trained into V3/V4 from scratch; their newer DSpark drafter pushes this further, but both require frontier-lab compute budgets. This repo shows the same core idea works at a fraction of the cost on a 1B model: freeze the base weights entirely, train a handful of lightweight residual heads that predict several tokens ahead, and let the base model verify them in bulk instead of one at a time.
Zero risk to output quality. Every drafted token is checked against a real forward pass through the untouched base model — if a draft is wrong, it's discarded and replaced with the verified real token. In greedy decoding, output is bit-for-bit identical to running the base model alone. This only ever buys you speed, never changes what gets generated.
Benchmark results
Measured as: does the head's guess for token t+k match what the base model's own forward pass actually produces? Each match = one forward pass you didn't have to run.
| Offset | Stock prompts (in-domain) | General text |
|---|---|---|
| t+1 | 87.2% | 63.1% |
| t+2 | 85.8% | 23.9% |
| t+3 | 83.6% | 11.2% |
On the domain it was trained for, this is a strong result — 85%+ match rate three tokens deep means the majority of forward passes on stock-analysis generation become redundant and get skipped. Published speculative decoding work generally considers >60-70% match at shallow offsets the threshold where real wall-clock speedup shows up; this clears that bar with room to spare at t+1 and t+2, and holds up surprisingly well even at t+3.
Outside that domain, use with your eyes open. General text match rate falls off sharply past t+1 (63% → 24% → 11%), because the training mix was split evenly between stock-domain data and general instruction text, and three residual blocks trained in under 30 minutes simply haven't had enough exposure to generalize as deep as the in-domain result. For stock-analysis-style generation, expect a real, meaningful speedup. For open-domain chat, stick to shallow speculation (t+1 only) or expect diminishing — possibly negative — returns from the deeper heads, since every rejected draft costs an extra verification pass.
FAQ
What is MTP (multi-token prediction)? A technique where a model predicts several future tokens per forward pass instead of just the next one, first popularized at scale by DeepSeek's V3/V4 models. It's normally trained into a model from scratch; this repo retrofits the same idea onto an already-trained 1B model after the fact.
Is this the same as DeepSeek's DSpark? No. DSpark is DeepSeek's own newer speculative decoding drafter for DeepSeek-V4, using 5-token draft blocks and a "Markov head," built with frontier-lab compute. This repo applies the older, simpler, more established MTP/Medusa-style approach (small residual heads reusing the base model's own output layer) — chosen specifically because it's mature, well-documented, and cheap enough to train on a single free-tier GPU in under an hour.
Does this change the model's stock predictions? No. In greedy decoding, every output token is verified against the real base model — the heads only skip redundant forward passes when their guess happens to be correct. Output is identical to the base model alone.
Will this speed up general chat, not just stock analysis? Partially, and only shallow (t+1) speculation — see the general-text numbers above. Deeper speculation (t+2, t+3) was trained mostly on stock-domain text and doesn't generalize as well yet.
What quantization formats are included? Q4_K_M and Q8_0 GGUF, plus the original LoRA adapter for further fine-tuning or full-precision use.
What's in this repo
| File | Description |
|---|---|
MiniCPM5-1B.Q4_K_M.gguf | 4-bit quantized base stock-analyst model, fastest inference |
q8/MiniCPM5-1B.Q8_0.gguf | 8-bit quantized, slightly higher fidelity |
adapter/ | Original v1 LoRA adapter (r=32, alpha=64) for full-precision or further fine-tuning |
medusa_heads.pt | Trained MTP/Medusa heads (3 heads: t+1, t+2, t+3) |
heads_config.json | Architecture spec needed to reconstruct the heads |
Architecture
Each head is a small residual block (Linear → SiLU → Linear, added back to the input) applied to the base model's final hidden state, then projected to vocabulary logits through the base model's own frozen lm_head — no separate output matrix per head, which is what keeps the trainable footprint tiny (a few million params total, not hundreds of millions) and the training this cheap.
Training
- Base: MiniCPM5-1B + v1 stock adapter, fully frozen
- 3,000 samples: 1,500 stock (matching deployment distribution) + 1,500 general (
yahma/alpaca-cleaned) - 1 epoch, batch size 8, ~375 steps, single T4, well under an hour end to end
Usage with llama.cpp
./llama-cli -m MiniCPM5-1B.Q4_K_M.gguf \
-p "<|system|>You are a financial analyst. Predict: BUY, SELL, or HOLD.<|end|><|user|>NVDA $148. RSI 72, MACD bullish, PE 30, ROE 115%, 122% growth. Beat 4/4. 90% buy. AI chip launch. BUY/SELL/HOLD?<|end|><|assistant|>" \
-n 32
MTP head integration into llama.cpp's speculative decoding path is not yet wired up in this repo — medusa_heads.pt currently targets the PyTorch/transformers checkpoint. See the training notebook for the reference PyTorch implementation of head-guided generation.
Use it if
You're serving MiniCPM5-1B-stock-analyst and want faster local generation on stock-analysis prompts without touching output quality. Don't expect the same win for unrelated general-purpose use — the numbers above tell you exactly where the line is.
Related
- MiniCPM5-1B stock analyst v1 — base model this repo builds on
- LFM2.5-Stock-Breakout-V13 — separate stock-signal model, different base architecture
License
Apache 2.0 (same as base model)