LFM2.5-VL-3B-ONNX

LFM2.5 is a new generation of hybrid models developed by Liquid AI, specifically designed for edge AI and on-device deployment. It sets a new standard in terms of quality, speed, and memory efficiency.

Find more details in the original model card: https://huggingface.co/LiquidAI/LFM2.5-VL-3B

🏃 How to run LFM2.5-VL-3B

You can run the ONNX model in the browser with Transformers.js v4.0 or newer:

npm install @huggingface/transformers

import {
  AutoModelForImageTextToText,
  AutoProcessor,
  load_image,
} from "@huggingface/transformers";

const modelId = "LiquidAI/LFM2.5-VL-3B-ONNX";
const processor = await AutoProcessor.from_pretrained(modelId);
const model = await AutoModelForImageTextToText.from_pretrained(modelId, {
  device: "webgpu",
  dtype: {
    embed_tokens: "fp16",
    decoder_model_merged: "q4",
    vision_encoder: "fp16",
  },
});

const messages = [
  {
    role: "user",
    content: [
      { type: "image" },
      { type: "text", text: "Describe this image." },
    ],
  },
];
const prompt = processor.apply_chat_template(messages, {
  add_generation_prompt: true,
});

const image = await load_image("https://placecats.com/300/200");
const inputs = await processor(image, prompt, { add_special_tokens: false });
const outputs = await model.generate({
  ...inputs,
  do_sample: true,
  temperature: 0.2,
  top_k: 50,
  repetition_penalty: 1.0,
  max_new_tokens: 256,
});

const generated = outputs.slice(null, [inputs.input_ids.dims.at(-1), null]);
console.log(processor.batch_decode(generated, { skip_special_tokens: true })[0]);