goramen-blog-gemma-4-sft
A QLoRA fine-tune of mlx-community/gemma-4-e4b-it-4bit
(itself an MLX conversion of google/gemma-4-E4B-it) trained
on the full text archive of the Go Ramen blog. Given a post title and labels, it writes
a ramen review in the blog author's own voice.
from mlx_lm import load, generate
model, tokenizer = load("johnhalloran/goramen-blog-gemma-4-sft")
prompt = 'Write a Go Ramen blog post titled "Afuri - Portland, OR". Labels: OR-Portland.'
messages = [
{"role": "system", "content": "You are the author of Go Ramen, a ramen review blog. "
"Write in the blog's distinctive, casual, food-obsessed voice."},
{"role": "user", "content": prompt},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_dict=False)
response = generate(model, tokenizer, prompt=prompt, max_tokens=400, verbose=True)
pip install mlx-lm
python -m mlx_lm generate --model johnhalloran/goramen-blog-gemma-4-sft \
--prompt 'Write a Go Ramen blog post titled "Afuri - Portland, OR". Labels: OR-Portland.' \
--max-tokens 400
About this model
Precision: bf16, not quantized. The QLoRA adapter was trained against a 4-bit quantized base, but the
weights published here were dequantized to bf16 before fusing, and the LoRA delta was added in that
dense bf16 space — not fused directly into the still-quantized weights and requantized afterward. That
distinction matters: fusing into quantized weights means dequantize → add delta → requantize the sum,
which throws away precision a second time on top of the base model's original quantization error. Fusing
into the dequantized backbone instead adds the adapter cleanly, once, at full precision. If you need a
smaller, quantized artifact, quantize this bf16 model yourself (mlx_lm.convert -q) rather than starting
from a requantized fuse.
Text-only, despite a multimodal base. This checkpoint descends from a nominally multimodal
(text + vision + audio) base model, but mlx_lm's gemma4 model class only ever instantiates the text
decoder (language_model) — it doesn't load the base checkpoint's vision_tower or audio_tower weights at
all, so a LoRA fine-tune run through mlx_lm.lora never sees the training examples' photos, and fusing the
adapter back in through mlx_lm.fuse only ever had language_model.* weights to merge and save. Every
tensor key in model.safetensors.index.json is prefixed language_model.* — no vision or audio weights are
present. If you want an image-conditioned version of this fine-tune, it needs to go through
mlx_vlm instead, which loads the vision tower for real; see the
training repo for that (currently experimental) path.
Training details
| Setting | Value |
|---|---|
| Base model | mlx-community/gemma-4-e4b-it-4bit (4-bit affine, group size 64, during training) |
| Published precision | bf16, dequantized before fusing (see above) — no quantization artifacts from the fuse step |
| Method | QLoRA (rank 32, alpha 64, scale 2.0, dropout 0.05) on all 42 decoder layers of the frozen language model |
| Trainable params | 77.3M / 7.46B (1.04%) |
| Optimizer | AdamW, weight decay 0.01 |
| LR schedule | cosine decay, peak 1e-4, 30-step linear warmup |
| Batch size / grad accum | 1 / 8 (effective batch size 8) |
| Epochs / iterations | 3 / 4,696 |
| Max sequence length | 1024 tokens |
| Training hardware | Apple Silicon (MLX), ~9.5GB peak unified memory, ~4.5 hours (against the 4-bit base) |
| Framework | mlx-lm mlx_lm.lora + mlx_lm.fuse --dequantize |
Data
Trained on johnhalloran/goramen-blog:
1,841 posts scraped from the Go Ramen blog, split 1,567 train / 100 validation / 100 test (fixed seed, no
overlap). Each example is a chat-format record — a system prompt establishing the blog's voice, a user turn
(Write a Go Ramen blog post titled "..." Labels: ...), and the post body as the assistant completion, which
is the only part trained on (prompt tokens are masked out of the loss).
Evaluation
Two checkpoints from the same run were scored on the held-out test split (100 posts, never seen during training or validation) to check for overfitting — evaluated against the 4-bit training checkpoints, before the dequantized fuse published here:
| Checkpoint | Val loss (during training) | Test loss | Test perplexity |
|---|---|---|---|
| iter 1410 (30% through training) | 2.539 (lowest observed) | 3.175 | 23.9 |
| iter 4696 (final) | 2.654 | 3.434 | 31.0 |
Train loss fell monotonically for the entire run (5.24 → 1.6), but validation loss bottomed out at iter 1410 and drifted upward for the remaining 70% of training — classic overfitting on a fairly small (1,567-example) dataset. This upload is fused from the iter-1410 checkpoint, not the final one, since it generalizes measurably better.
Limitations
- Style transfer, not fact-checking. The model learned the blog's voice and structure, not the truth about any specific ramen shop — don't treat generated posts as reliable information about a restaurant.
- Small fine-tuning set. 1,567 training examples is enough to shift tone and structure, not enough to fully avoid overfitting (see Evaluation above); expect some memorized phrasing from the training posts.
- English only, and specific to one blog's idiosyncratic voice — it will not generalize to other writing styles or topics.
- ~14GB download (bf16, 4 shards) — noticeably larger than a quantized model of the same parameter count. If size matters more than fuse precision for your use case, quantize this model yourself instead of using an earlier requantized-fuse artifact.
- Inherits the base model's general capabilities and limitations otherwise; see
google/gemma-4-E4B-itfor the base model card.
License / provenance
Released under the Gemma license, matching the base model. The training text and photos are the copyrighted work of the Go Ramen blog's author; the training dataset is a personal archival copy, not a license grant for reuse of that underlying content.