🚀 Gemma 3 270M — Structured JSON & Function Calling Extractor
gemma-3-270m-json-extractor is a fine-tuned version of google/gemma-3-270m-it optimized for strict, raw JSON output and structured function-calling extraction without conversational filler.
Despite its tiny footprint (~270 million parameters), this fine-tuned model delivers a 6x reduction in latency and a ~4x increase in valid JSON formatting accuracy compared to the base model.
📊 Key Benchmark & Evaluation Results
Evaluated on a held-out test split of 100 complex structured JSON extraction prompts:
| Metric | Base gemma-3-270m-it | Fine-Tuned (gemma-3-270m-json-extractor) | Improvement |
|---|---|---|---|
| JSON Validity Rate | 23.0% | 90.0% | +291% |
| Exact Schema Match | 0.0% | 21.0% | +21.0% |
| Key Coverage Rate | 0.0% | 59.0% | +59.0% |
| ROUGE-L Score | 26.58 | 67.79 | +155% |
| Avg Inference Latency | 11.54s | 1.92s | 6x Faster |
Key takeaway: The base model frequently rambled with conversational preamble ("Sure, here is your JSON..."), leading to long generation times and broken JSON syntax. The fine-tuned model immediately triggers JSON generation and cleanly emits the
<eos>token upon completion.
💻 How to Use
Basic Inference with Transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "tarif2108/gemma-3-270m-json-extractor"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
system_prompt = "You are a JSON generator. Reply ONLY with a single valid JSON object and nothing else."
user_prompt = "Extract user info into JSON with keys: name, age, city. Input: 'Hi, I'm Ada Lovelace, 28 years old, living in London.'"
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=256,
do_sample=False,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)
# Expected Output: {"name":"Ada Lovelace","age":28,"city":"London"}
🛡️ Guaranteeing 100% Valid JSON (Constrained Decoding) While this fine-tuned model achieves 90% valid JSON natively, you can achieve 100% guaranteed schema enforcement during inference by combining this model with lm-format-enforcer:
pip install lm-formatenforcer pydantic
from pydantic import BaseModel
from lmformatenforcer import JsonSchemaParser
from lmformatenforcer.integrations.transformers import build_transformers_prefix_allowed_tokens_fn
class UserSchema(BaseModel):
name: str
age: int
city: str
parser = JsonSchemaParser(UserSchema.model_json_schema())
prefix_fn = build_transformers_prefix_allowed_tokens_fn(tokenizer, parser)
# Pass prefix_allowed_tokens_fn to model.generate
outputs = model.generate(
**inputs,
max_new_tokens=256,
prefix_allowed_tokens_fn=prefix_fn
)
⚙️ Training Details
- Base Model:
google/gemma-3-270m-it - Dataset:
NousResearch/hermes-function-calling-v1(~11,500 clean JSON rows across all subsets) - Fine-Tuning Technique: QLoRA (4-bit NF4 quantization)
- LoRA Target Modules:
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj - LoRA Parameters: r=16, alpha=32, dropout=0.05
- Hardware: NVIDIA RTX 3050 (4GB VRAM)
- Optimizer:
paged_adamw_8bit - Precision:
bfloat16compute - Sequence Length: 512