Baihu-V1 (GGUF)

Baihu-V1 is a LoRA fine-tune of google/gemma-4-E2B-it on a curated bilingual (Chinese + English) corpus of multi-turn transfer-dialogue data: conversations where a rule, analogy or format is introduced once and then carried to new cases across several turns.

The fine-tune changes the model's answering behaviour rather than its knowledge. It replies in short, plain prose, states the result directly, and stops, instead of producing long markdown lecture-style answers. Held-out evaluation shows this costs essentially nothing on general knowledge (MMLU 5-shot 53.4% → 52.6%) while roughly doubling reply quality against the reference style (ROUGE-L 0.236 → 0.432).

Files

Both files are text-only conversions of the merged checkpoint. The vision and audio towers of Gemma 4 E2B are not used for text generation and were not fine-tuned, so they are not included.

Quick start

# llama.cpp CLI
llama-cli -hf ZichenAI/Baihu-V1-GGUF:Q4_K_M \
  -p "<bos><|turn>user\nIn one sentence: why does metal feel colder than wood?<turn|>\n<|turn>model\n" \
  -ngl 99 -st

# llama-server (OpenAI-compatible)
llama-server -hf ZichenAI/Baihu-V1-GGUF:Q4_K_M -c 8192 -ngl 99

The model uses the standard Gemma 4 chat template, so any client that speaks the Gemma 4 format works (llama.cpp, LM Studio, Ollama, transformers).

Benchmarks: base vs Baihu-V1

Every number below comes from the same scripts, the same held-out split and the same hardware (1× RTX 4090 D), so the delta is what matters, not the absolute values. The raw result files are in benchmarks/ and a consolidated copy is in benchmarks/summary.json.

1. Held-out transfer-dialogue benchmark — 321 conversations, 12 topic groups never seen in training

Each conversation is truncated before its final assistant turn; the model must produce that turn. ROUGE-L is character-level LCS F1 against the reference reply. "Plain text" means the reply contains no markdown markup at all. Len ratio is reply length ÷ reference length (1.00 = matching the reference style).

On the key-number recall row: the metric asks what fraction of the reference's numbers appear in the reply, which rewards saying more. The base model emits ~4× more text and therefore gets more chances to hit a number; the fine-tune trades a little of that for conciseness, while ROUGE-L — which also penalises the extra text — nearly doubles. No individual subject or pattern loses more than 6 pp.

2. Same benchmark, measured on the shipped GGUF (Q4_K_M) via llama-server

This row set is the end-to-end check of the published artefacts themselves.

The GGUF reproduces the merged HF checkpoint closely (ROUGE-L 0.419 vs 0.432), which confirms that the merge-and-convert pipeline and the Q4_K_M quantisation introduce no meaningful degradation.

3. General capability — MMLU, 5-shot, option log-likelihood

500 test questions from 10 subjects (cais/mmlu, 50 per subject, 5-shot dev prompts). Scored by the log-likelihood of the option letter, the same protocol llama.cpp uses.

The −0.8 pp shift is inside the sampling noise of a 500-question suite (≈ ±2.2 pp at 95 % confidence), so the fine-tune did not cause measurable forgetting. This is not a general-purpose knowledge upgrade — it is a behaviour and style tune on top of the same base.

4. Held-out perplexity (llama-perplexity, Q4_K_M, 60 × 2048-token chunks)

5. Inference speed (llama-bench, RTX 4090 D, Q4_K_M, -ngl 99)

Fine-tuning a subset of the attention and MLP projections does not change the architecture, so throughput matches the base model within run-to-run noise.

Training details

Data

973 training / 321 validation multi-turn conversations (6–10 messages each), synthesised around six transfer patterns — rule_induction, analogy_transfer, format_transfer, counterfactual, cross_domain, teaching_loop — in both Chinese and English. The validation split holds out whole topic groups, so every validation conversation is about a topic the model never saw during training.

Export pipeline

1. Merge the LoRA adapter into the base weights in bf16 (peft.merge_and_unload).
2. convert_hf_to_gguf.py --outtype bf16 (llama.cpp).
3. llama-quantize … Q4_K_M / Q8_0.
4. Re-measure everything on the resulting GGUF with llama-server and llama-perplexity.

Intended use and limitations

Use it for: concise, direct question answering in Chinese or English; multi-turn explanatory dialogues where a short answer is preferred over a formatted essay; on-device or edge deployment where a ~3.4 GB 2B-class model is the budget.

Do not rely on it for: factual lookups, mathematics beyond short arithmetic, long-form writing, tool calling, or anything needing the vision/audio capabilities of the base Gemma 4 E2B — those towers are not part of this GGUF.

The model was fine-tuned on synthetic dialogue data. It can still hallucinate, it inherits the biases of its base model, and its output should be reviewed before being used in any consequential setting. It ships at Q4_K_M by default — use the Q8_0 file when near-lossless output matters.

License

Custom license — personal use free, commercial use requires a paid licence.

Commercial licensing contact:

novaweb6868@outlook.com Suggested subject: Baihu-V1 Commercial License

See LICENSE for the full text. This model is a derivative of google/gemma-4-E2B-it and remains subject to the Gemma 4 license terms. Where the two conflict, the Gemma 4 terms prevail.

Citation

@misc{baihu-v1,
  title  = {Baihu-V1: a transfer-dialogue LoRA fine-tune of Gemma 4 E2B, exported to GGUF},
  author = {ZichenAI},
  year   = {2026},
  url    = {https://huggingface.co/ZichenAI/Baihu-V1-GGUF}
}