Gemma 4 E2B — Home Assistant
A fine-tune of Google's gemma-4-E2B-it-qat-q4_0-unquantized for controlling a smart home through natural language, in the style of the acon96/home-llm project. Given a system prompt describing the available devices and services, the model replies in natural language and emits the tool call needed to carry out the request.
This is the LoRA adapter merged back into the base weights — a complete,
drop-in replacement for the base checkpoint. Like the upstream model it is the
full multimodal Gemma4ForConditionalGeneration (text + vision + audio towers);
only the text/language-model projections were fine-tuned, so the vision and audio
towers are byte-for-byte the base model's.
Base model
- google/gemma-4-E2B-it-qat-q4_0-unquantized
- The
-qat-q4_0-unquantizedcheckpoint is quantization-aware-trained and calibrated for llama.cpp Q4_0. Fine-tuning was done in 16-bit LoRA (not QLoRA/NF4) specifically to keep that QAT calibration intact, so the merged model still quantizes cleanly to Q4_0 for llama.cpp / Ollama deployment.
Training
| Method | LoRA (r=8, alpha=16, dropout 0), 16-bit, merged |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Dataset | acon96/Home-Assistant-Requests-V2 — English split (~17.5k examples ≤ 4096 tokens) |
| Schedule | 1 epoch, 1089 steps, bf16, adamw_8bit, lr 2e-4, batch 1 × grad-accum 16, seq len 4096 |
| Final loss | train ≈ 0.23 (≈ 0.10 over the last steps) |
| Framework | Unsloth on a single RTX A4500 |
Only the assistant turn (its text and its tool call) is trained; injected tool results are masked out so the model learns to call tools, not to hallucinate their responses.
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "mewse/gemma-4-E2B-home-assistant"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
messages = [
{"role": "system", "content": "<your Home Assistant system prompt: devices, services, tool schemas>"},
{"role": "user", "content": "turn on the living room lamp"},
]
enc = tok.apply_chat_template(messages, add_generation_prompt=True,
return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**enc, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=False))
For the best tool-call fidelity, match the system-prompt format used in training (entity states + tool schemas injected into the system prompt, as in home-llm). For deployment, quantize to a GGUF (Q4_0 is the QAT-native target) and serve via llama.cpp or Ollama.
License
Governed by the Gemma Terms of Use, inherited from the base model. You must accept the Gemma license to use these weights.