Buddy-E4B-v2
A small chat model that talks like a friend texting, and says "I don't know" instead of making things up. It is a fine-tune of google/gemma-4-E4B-it: supervised fine-tuning for the voice, then DPO for tone and honesty.
- Size: 7,996,156,490 parameters, identical to the base. "E4B" means about 4B parameters are active per token; the file is about 8B because it includes the per-layer embedding tables and the vision/audio parts. Architecture, tokenizer and chat template are unchanged from the base.
- Format: bf16 safetensors (Gemma overflows in fp16).
- License: Apache-2.0, same as the base. See "Training data" for the attribution the data requires.
What it is good at
- Short, casual, natural replies. No emoji, no "OMG!!", no bullet-point essays by default (the plain base model buries every reply in emoji and hype).
- Saying it doesn't know things that don't exist (invented people, products, papers) instead of inventing details.
- Keeping Gemma's native tool calling.
Recommended system prompt
You are a warm, funny, down-to-earth friend having a casual chat. Reply like a real person texting: natural, genuine and usually short. Match the user's tone and energy, and ask a follow-up question when it fits. Give more detail only when asked. If you don't know something, say so.
The model was trained mostly with this prompt (sometimes with none), so use it or no system prompt.
Usage (vLLM)
vllm serve tomdacat-ai/Buddy-E4B-v2 --dtype bfloat16 --max-model-len 4096
Then call it through the OpenAI-compatible API. A 24 GB GPU (e.g. an L4) is enough for the bf16 weights; this is how the results below were measured.
Smaller GPUs / llama.cpp (GGUF)
Quantized GGUF files are in tomdacat-ai/Buddy-E4B-v2-GGUF. The 4-bit Q4_K_M file used about 3.4 GB of VRAM in my test, so it fits a 5 GB GPU. It needs a recent llama.cpp (Gemma 4 support).
Results
Same harness for all models; vLLM, bf16; probes at temperature 0.2.
| metric | v1 (Llama-3.2-3B Buddy) | Gemma 4 E4B base | Buddy-E4B-v2 |
|---|---|---|---|
| Invented answers on 24 non-existent things (lower is better) | 7/24 | not hand-checked | about 4/24 (the regex count is 4/24; hand-check of this release not repeated) |
| Real facts correct (30) | 30/30 | 30/30 | 30/30 |
| Tool call correct / correctly holds back | 6/6, 0/3 | 6/6, 3/3 | 5/6, 3/3 |
| MMLU 5-shot | 0.595 (full) | 0.600 (5% subset) | 0.636 (5% subset, about +-1.8%) |
| TruthfulQA mc1 / mc2 | 0.323 / 0.480 | 0.283 / 0.486 | 0.282 / 0.488 |
Caveats, stated plainly:
- The honesty probe is small (24 invented items, 30 real ones) and was hand-checked; treat it as a rough signal, not a benchmark.
- MMLU was run on a 5% sample of questions (the same questions for both Gemma models); the difference from the base is within noise.
- TruthfulQA is on par with the base model (0.282 vs 0.283 mc1).
Known weaknesses
- Tool calling: in one of six probes it answered from memory instead of calling a search tool ("who won the most recent Tour de France"). It correctly held back in 3 of 3 cases where no tool was needed.
- It can be over-cautious: asked how to pick the lock of its own shed, it helps with the recommended system prompt but refuses without a system prompt.
- Identity: it calls itself Buddy, "built on an open model", and says it does not know the details. (An earlier build said it was "Open Assistant"; that is fixed in this release.)
- It still invents details sometimes, typically after "I don't know much about X, but...". About 3 in 24 probes.
- On opinion questions it occasionally drops into a formal, essay-like voice (2 of 14 casual probes).
- English only was trained and tested. Not additionally safety-tuned beyond what the base model already has. It is a hobby-scale model; don't use it for medical, legal or financial advice.
Training data
All permissively licensed; no Llama-licensed or other restricted-license data.
- SFT (10,701 examples): about 6,900 prompts from WildChat-1M with replies written by Qwen3-14B-AWQ (Apache-2.0) in a human style using real human replies as examples; about 3,100 human-written replies from OpenAssistant oasst2 (Apache-2.0), after removing replies where the assistant names its creator; 350 abstention examples ("never heard of that"); about 340 hand-written identity examples and 28 hand-written everyday-request examples.
- DPO: about 3,060 tone pairs, 80 identity pairs and 188 honesty pairs (the model's own invented answer as the rejected side, an honest "I haven't heard of that" as the chosen side; TriviaQA questions as controls so it still answers things it knows).
- WildChat prompts are licensed ODC-By; attribution: Zhao et al., "WildChat: 1M ChatGPT Interaction Logs in the Wild" (Allen Institute for AI).
Training
QLoRA with Unsloth (r=32, 2 epochs) for SFT, then a LoRA DPO pass (r=16, beta 0.1, 1 epoch), on one NVIDIA L4. The adapters were merged into the original bf16 base weights.