PantryMax SmolVLM-500M ONNX

This is an ONNX Runtime Web release of a PantryMax LoRA fine-tune of HuggingFaceTB/SmolVLM-500M-Instruct. It is intended for fully local, in-browser product-label and recipe-page extraction in PantryMax.

Runtime assets

The repository contains only browser runtime assets:

• onnx/*_q4f16.onnx for devices with float16 shader support;
• onnx/*_q4.onnx as the compatible fallback;
• the base processor, tokenizer, and chat-template configuration required by Transformers.js.

decoder_model_merged_* has the PantryMax LoRA merged into the base decoder before four-bit weight-only quantization. The vision encoder and embedding files remain the upstream compatible ONNX exports. There are no source images, annotations, prompts with private content, raw generations, or adapter checkpoints in this release.

Intended behavior and limits

PantryMax supplies the selected task (product or recipe) and constrains decoding against that task's JSON schema in the browser. The model should only extract visibly supported content; it must not be treated as an authority or used for high-stakes decisions. Structured decoding makes output parseable, not necessarily factually correct, so users should review extracted values.

Provenance and license

Base model and unmodified runtime components: HuggingFaceTB/SmolVLM-500M-Instruct, released under Apache-2.0. The PantryMax adapter was trained locally on task-conditioned product-label and recipe-page examples with mixed-source provenance. This release contains only derived runtime weights, but does not assert that every underlying training-source obligation is waived. Reusers are responsible for reviewing the base model and training-data provenance before redistribution or commercial use.

Reproduction checks

The merged fp16 decoder was checked against all 64 LoRA projection targets, then each q4/q4f16 output was required to contain 225 MatMulNBits operations and load successfully in ONNX Runtime before release.