Recipe Lens

A recipe assistant fine-tuned to answer three questions with structured, checkable output:

• What can I make? Given the ingredients you have, a recipe that uses only those (plus pantry staples).
• Rescale this. Serve 3 instead of 8, with every quantity multiplied correctly.
• I've run out of X. Swap it for something valid and adjust the amount.

It replies with a single JSON object, or declines when the ingredients cannot make a real dish.

{"title": "...", "servings": 3,
 "ingredients": [{"item": "flour", "qty": 250, "unit": "gram"}],
 "steps": ["...", "..."]}

Try it in the browser  ·  Trainer, evaluation harness and every metric on GitHub

See it work

Real output from the published model, captured by demo/run_demo.py. Every response is checked live by the same validators that produced the metrics below.

What can I make?

You list what you have. It returns a recipe using only those things, plus pantry staples - and tells you what it did not need.

I have these ingredients: chicken breast, onion, garlic, rice, bell pepper, chocolate chip, banana.

Chicken and Rice

Serves 4 people  ·  4 ingredients  ·  5 steps

You will need

• 1 pound chicken breast
• 1 onion
• 1 clove garlic
• 1 cup rice

Method

1. Preheat oven to 350 degrees F (175 degrees C).
2. Place chicken in a 9x13-inch baking dish. Sprinkle with salt and pepper. Top with onion and garlic.
3. Bake in the preheated oven until chicken is no longer pink in the center and the juices run clear, about 45 minutes. An instant-read thermometer inserted into the center should read at least 165 degrees F (74 degrees C).
4. Meanwhile, bring water to a boil in a saucepan. Add rice and reduce heat to medium-low. Cover and simmer until rice is tender and water has been absorbed, about 20 minutes.
5. Remove chicken from the oven and let rest for 5 minutes. Shred and serve over rice.

Not needed: banana, bell pepper, chocolate chip. Having an ingredient is not a reason to use it.

Valid against the output contract: PASS, Uses only what you listed, plus staples: PASS

Cook it for a different number of people

Every quantity is multiplied out. This is arithmetic, so it is verified exactly rather than eyeballed.

Here is a recipe:

Simple Macaroni and Cheese

Serves 3 people  ·  5 ingredients  ·  4 steps

You will need

• 6 ounces macaroni
• 3 tablespoons butter
• 3 tablespoons all-purpose flour
• 1 1/2 cups milk
• 1 1/2 cups cheddar cheese

Method

1. Cook the macaroni in salted water until tender, then drain.
2. Melt the butter, whisk in the flour, and cook for one minute.
3. Whisk in the milk and simmer until thickened.
4. Stir in the cheese and fold through the macaroni.

Valid against the output contract: PASS, Every quantity scaled correctly: PASS

When it can't, it says so

Two unrelated ingredients do not make a dish. It declines instead of inventing one.

I have these ingredients: tortilla chip, cinnamon.

Not enough to cook with

These ingredients do not combine into a recipe on their own.

Declining is a valid answer here. The model is trained to say so rather than invent a dish.

Valid against the output contract: PASS, Declined instead of inventing a dish: PASS

More examples, with the raw JSON behind each one, are in EXAMPLES.md.

From a photograph

The fine-tuned model is text-only and never sees an image. Photos are handled by a separate, prompted vision model whose output is validated against the frozen vocabulary and then passed on as an ingredient list.

photo -> Qwen3-VL-4B-Instruct (prompted, NOT trained) -> ingredients
      -> validated against the vocabulary
      -> the fine-tune (this project's model)          -> a recipe

The two stages are scored separately, so when the result is wrong you can tell which half was wrong. Every photo below is from the held-out test split, which means the real recipe is known and is used as the ground truth for stage 1.

Read the recall number carefully

Stage 1 is the weak link in this pipeline, and these numbers say so. But they understate it in one direction and overstate it in another, and both are worth knowing:

• The ground truth is the full ingredient list, and a photograph of a finished dish cannot show most of it. Anything dissolved, melted or baked in -- stock, vinegar, oil, sugar, egg -- is invisible by the time the dish is plated. Recall counts every one of those as a miss, so a perfect vision model would still score far below 100% here.
• Where the model declines to name an ingredient it cannot actually see, that is the behaviour the prompt asks for, not a failure -- even though the metric scores it as one.
• Against that, it does miss things that ARE visible, and it names things that are not there. Both are shown below.

The honest summary: photographs of finished dishes are close to unsolvable as an ingredient-recovery task, and this stage should be treated as a hint to the cook, not as input to trust. It is prompted, not trained, and improving it was never part of this project (HANDOFF.md s3).

Generated 2026-08-28.

Photo: Simple Hot and Sour Soup

Stage 1 - what the vision model saw

Qwen/Qwen3-VL-4B-Instruct, prompted, not trained. Its output is validated against the frozen vocabulary before anything is passed on.

Stage 2 - what the fine-tune made of it

The fine-tune receives only the recognised list above. It never sees the photograph.

Not enough to cook with

Too few ingredients to make something worth eating.

Declining is a valid answer here. The model is trained to say so rather than invent a dish.

Computed by the same validators that produced the published metrics. No model judges this output.

Photo: Coconut Chicken with Green Beans

Stage 1 - what the vision model saw

Qwen/Qwen3-VL-4B-Instruct, prompted, not trained. Its output is validated against the frozen vocabulary before anything is passed on.

Stage 2 - what the fine-tune made of it

The fine-tune receives only the recognised list above. It never sees the photograph.

Not enough to cook with

Too few ingredients to make something worth eating.

Declining is a valid answer here. The model is trained to say so rather than invent a dish.

Computed by the same validators that produced the published metrics. No model judges this output.

How it was measured

Every metric below is computed by a rule. There is no LLM judge anywhere in this project. Containment is a set operation, scaling is arithmetic, substitution and diet compliance are table lookups, format validity is JSON parsing.

Scored on 150 held-out tasks, 146 of them on ingredient combinations never seen in training. Whole recipes are held out, and training data is asserted to have zero verbatim overlap with the test set.

Where a baseline beats this model

• No baseline beat it on any headline metric.

Reading the numbers honestly

• A retrieval baseline reproduces training recipes verbatim 100% of the time. That is what retrieval is. This model's memorisation rate is 0.0%, and that comparison is the point of publishing both.
• Containment must be read with utilisation. A system can score perfectly on "used only what you have" by returning a recipe that barely uses anything. Utilisation error measures distance from what the real recipe uses, so padding a dish with ingredients it does not need scores badly too.
• Substitution is scored against the swap table shipped in this repo. This model learned that table from training data, so part of its advantage there is definitional rather than earned.

Usage

The adapter is merged. It loads as an ordinary model, with no PEFT and no adapter step:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sanidhya1910/recipe-lens-4b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

messages = [
    {"role": "system", "content": "Reply with a single JSON recipe object: title, servings, ingredients (item, qty, unit), steps."},
    {"role": "user", "content": "I have these ingredients: chicken breast, onion, garlic, rice.\n\nGive me a recipe that uses only ingredients from that list. You do not have to use all of them. Serve 4."},
]

Use the short system prompt above. The model was trained under it; a longer prompt is not needed and was not measured.

Shipped alongside the weights

• ingredients.json - the canonical vocabulary, with the unit dimensions actually observed for each item
• staples.json - assumed on hand, never a containment violation
• substitutions.json - the swap table, with ratios and units
• diets.json - vegetarian, vegan, gluten-free, nut-free, dairy-free exclusion rules

Anything outside the vocabulary is reported as unrecognised, never guessed at. Nutrition is never generated by the model.

Training

LoRA (r=64, alpha=128) on all attention and MLP projections, then merged. LoRA was the training method, not the artifact.

Data is derived from real recipes: the fridge targets are real recipes, and the rescale and substitute targets are computed by code from them. That is mechanically transformed real data, not generated prose.

Merged weights

LoRA was the training method, not the artifact: the adapter is folded into the base weights, so this loads as an ordinary model with no PEFT step. Whether that fold was lossless is checked at the weight level, not inferred from output text.

The merge is exact. All 3,633,315,840 merged weight elements are bit-identical to an fp32-accumulated merge (bf16(fp32(W) + fp32(B @ A) * scale)) -- 0 mismatched, 0.00 bf16 ULP of error.

Greedy generations still differ from the unmerged adapter on 2 of 12 prompts. That is expected: the merged model computes x @ (W + BA) in one matmul where the adapter computes x @ W + (x @ A) @ B * scale in three. Equal in exact arithmetic, unequal in floating point, and at a near-tie logit the argmax flips.

The control: running the merged model against itself, changing only the batch size, reproduces 83.3% of its own outputs -- against 83.3% versus the adapter. It disagrees with itself as much as it disagrees with the adapter, so the merge is not the cause.

How that was established

This model card previously said the opposite -- that folding W + BA into W in bf16 loses precision. That was a guess, and measuring it disproved it. Three findings, in order:

1. The weights are bit-exact. 3,633,315,840 elements, 0 mismatched. So there is no lost precision to blame, and the stated cause could not have been the cause.

2. The divergent token was a perfect tie. At the first disagreeing position the top-2 logit gap was exactly 0.000000, against a median of 13.6 across the run -- the tightest 0.8% of positions.

3. The merged model diverges from itself. Same weights, same prompt, same greedy decoding, changing only the batch it travels in:

   batch	recipe it returned
   alone	Tofu and Mushroom Stir-Fry
   batched with one other prompt	Tofu and Mushroom Egg Rolls

A criterion a model fails against its own weights is not measuring merge fidelity -- it is measuring matmul reduction order. So the published gate is the weight comparison above, which is exact.

Limitations

• The vocabulary is ~570 items; rarer ingredients are reported as unrecognised.
• Compound quantities ("2 tablespoons plus 2 teaspoons") are not summed.
• Coconut counts as a nut for the nut-free check. The rules are recall-first: an ambiguous ingredient is flagged.
• Diet checks are a table lookup, not medical advice. Do not rely on them for a severe allergy.