gemma-3-270m-foodextract
LoRA (merged) SFT of unsloth/gemma-3-270m-it for structured food/drink extraction from a text description, trained on mrdbourke/FoodExtract-135k.
Input / output
User turn (model chat template):
Extract the food and drink information from this description.
Description:
{description}
Output (dataset column gpt-oss-120b-label-condensed, unchanged):
food_or_drink: 0|1
tags: <comma-separated codes>
foods: <comma-separated items>
drinks: <comma-separated items>
Tag codes: np nutrition panel, il ingredient list, me menu, re recipe, fi food items, di drink items, fa food advertisement, fp food packaging.
Training data
| Source rows (train / test) | 121569 / 13508 |
| Rows kept (train / validation / test) | 115130 / 6059 / 13500 |
| Rows used for training | 20000 (deterministic subset) |
| Tokens per example (median / p99 / max) | 115.0 / 466 / 1760 |
Removed: empty descriptions, exact duplicate (description, target) pairs, examples longer than 1024 tokens (not truncated), train rows sharing a description with test. Test: official test split (held-out evaluation). Validation: 5% of cleaned official train, group-aware, seed=42.
Training procedure
| Hyperparameter | Value |
|---|---|
| Method | lora_fallback |
| Epochs | 1 |
| Steps | 1250 |
| Learning rate / scheduler | 5e-05 / cosine |
| Warmup steps | 63 |
| Weight decay | 0.01 |
| Batch size x grad. accumulation | 4 x 4 = 16 |
| Optimizer | adamw_torch |
| Precision | bf16 mixed precision, FP32 master weights |
| Gradient checkpointing | True |
| Max sequence length | 1024 (no truncation) |
| Loss | assistant tokens only (prompt and padding masked) |
| Hardware | NVIDIA GeForce RTX 5060 Ti (16283 MB) |
| Training time | 0.57 h |
| Peak VRAM allocated / reserved | 13760.7 / 13860.0 MB |
| Final validation loss | 0.3752 |
| Software | torch 2.11.0+cu128, transformers 5.17.0, trl 0.24.0 |
Evaluation
Split: test (500 examples), greedy decoding, max_new_tokens=256.
| Metric | Value |
|---|---|
| Exact match (normalized) | 0.4160 |
| ROUGE-1 / ROUGE-2 / ROUGE-L | 0.9142 / 0.7771 / 0.8641 |
| food_or_drink accuracy | 0.9540 |
| Tags micro-F1 | 0.9118 |
| Foods micro P / R / F1 | 0.7148 / 0.7354 / 0.7249 |
| Drinks micro P / R / F1 | 0.6719 / 0.5658 / 0.6143 |
| Malformed output rate | 0.0160 |
| Missing-field rate | 0.0140 |
Food/drink references only (344 examples): exact match 0.1977, foods F1 0.7282, drinks F1 0.6187, tags F1 0.9227.
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "thealper2/gemma-3-270m-foodextract"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).to("cuda")
description = "A plate of grilled chicken with rice and a glass of orange juice."
messages = [{"role": "user", "content": "Extract the food and drink information from this description.\n\nDescription:\n" + description}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())
Limitations
- Targets were generated by
gpt-oss-120bfrom the text description; the model reproduces these LLM labels, including their errors. - Training data mixes real web captions (COYO-700M, PixMo-Cap, Open-Qwen2VL) with synthetic captions and random strings.
- Item lists are comma-separated; items containing commas cannot be represented unambiguously.
- English descriptions; inputs longer than 1024 tokens were not seen in training.
- Trained on a 20000-example subset of the cleaned training split.
- Use is subject to the Gemma Terms of Use of the base model.