Hunter 4B
Introducing Hunter 4B by Lalakai Labs.
A 4B-parameter conversational reasoning model, distributed as GGUF weights for fast local inference with llama.cpp, Ollama, LM Studio, and any other GGUF-compatible runtime. GGUF is a safe, mmap-friendly format — no pickle files here.
Hunter reasons out loud: it works through problems step by step inside <thought>...</thought> traces before giving you the final answer — handy for math, code, and multi-step tasks. Despite its small footprint, Hunter-4B is built to rival the performance of models in the 10–15B parameter range on reasoning, coding, and cyber tasks.
Training data
Hunter 4B is a strong fine-tune of the Qwen 3.5 4B, trained on ~180,000 high-density examples. No raw web scrapes — every example is curated, deduplicated (MinHash LSH), and formatted into a ChatML envelope with explicit reasoning traces:
| Category | Share | What it teaches |
|---|---|---|
| Complex reasoning | 40% | Step-by-step problem solving, math, distilled reasoning traces |
| Agentic coding | 25% | Deterministic code synthesis, debugging, syntax discipline |
| Multi-step tool use | 20% | Structured function calls, JSON schema adherence, autonomous API loops |
| World knowledge anchor | 15% | General facts and language grounding, so reasoning gains don't cost common sense |
| Made to hallucinate less with better architectural modifications. |
Specialty: deep cyber capabilities. A core focus of the mix is security-flavored reasoning — the model is at home with vulnerability analysis, secure-coding review, and adversarial thinking alongside everyday coding and math.
Method: parameter-efficient QLoRA fine-tuning (rank 64), merged back to FP16, then quantized to Q4_K_M with an importance matrix so the weights that matter for reasoning survive compression.
Files
| File | Size | Description |
|---|---|---|
Hunter-4B-v1.0-FP16.gguf | ~8.05 GB | Full-precision weights. Best quality; needs ~10 GB of RAM/VRAM. |
Hunter-4B-v1.0-Q4_K_M.gguf | ~2.50 GB | 4-bit quantized weights. Great quality-per-GB; comfortable on machines with ~4–8 GB free. Recommended for most users. |
Hunter-4B-v1.0.imatrix.gguf | ~3.9 MB | Importance matrix used during quantization. Only needed if you re-quantize. |
SHA256SUMS | — | Checksums to verify your downloads. |
Verify your download before running:
sha256sum -c SHA256SUMS
Quick start
llama.cpp (no install beyond the binary):
llama-cli -m Hunter-4B-v1.0-Q4_K_M.gguf -p "Hello! Who are you?"
Serve it over HTTP:
llama-server -m Hunter-4B-v1.0-Q4_K_M.gguf --port 8080
Python (via llama-cpp-python):
from llama_cpp import Llama
llm = Llama(
model_path="Hunter-4B-v1.0-Q4_K_M.gguf",
n_ctx=4096,
verbose=False,
)
print(llm("Hello! Who are you?", max_tokens=256)["choices"][0]["text"])
Ollama — create a Modelfile (reasoning traces on, ChatML stops set):
FROM ./Hunter-4B-v1.0-Q4_K_M.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
{{ .Response }}<|im_end|>"""
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.90
PARAMETER top_k 40
PARAMETER repeat_penalty 1.1
SYSTEM """You are Hunter-4B, an autonomous reasoning intelligence developed by Lalakai. Always write out your structured chain of thought inside <thought> ... </thought> tags before issuing your final answer."""
ollama create hunter-4b -f Modelfile
ollama run hunter-4b
Download with the Hugging Face CLI:
hf download Lalakai/Hunter-4B Hunter-4B-v1.0-Q4_K_M.gguf
Which file should I pick?
- Just want to chat locally? →
Q4_K_M. Small download, fast, sips memory. - Best output quality / evaluating the model? →
FP16, if you have the RAM. - Making your own quants? → Grab the
FP16weights plus the.imatrixfile.
Who is Hunter-4B for?
- Students — essay drafting, explaining tough concepts step by step, math homework walked through with visible reasoning traces.
- Researchers & analysts — web scraping helpers, summarization, and research tasks that need structured, multi-step thinking on a laptop.
- Developers — a powerful 4B model purpose-built for agentic coding and autonomous software engineering workflows: code synthesis, debugging, tool use, and API orchestration.
No GPU cluster required. Hunter-4B is designed for the hardware most people actually own — a regular 8 GB RAM laptop or 8 GB MacBook, the standard for most devices. Download the Q4_K_M file and you're running.
Performance
Engineered for everyday hardware: the Q4_K_M build runs in ~3.5 GB of working memory (weights + 4k context + runtime buffers), sustaining roughly 28–38 tokens/sec on Apple Silicon and 18–24 tokens/sec on modern x86 laptops — with reasoning and coding performance designed to rival 10–15B models. Formal benchmarks will be published here when available.
Limitations
- A 4B model won't match frontier-scale models on the hardest reasoning or niche knowledge. Pick the FP16 weights and a larger context window (
-c) when quality matters most. - Like all LLMs, it can hallucinate, reflect biases in its training data, and generate unsafe content if prompted adversarially. Review outputs for your use case and apply your own safety filtering for production deployments.
License
MIT. Use it commercially, fine-tune it, quantize it, ship it — attribution appreciated.
Built with care by Lalakai Labs.