LiquidAI LFM2.5-VL-450M-SFT-GGUF
This repository contains fine-tuned and merged weights for LiquidAI/LFM2.5-VL-450M, fine-tuned with multimodal LoRA and converted to GGUF format for local, edge-native inference with llama.cpp.
📊 Comprehensive Benchmark Suite (Evaluated on RTX 4050 GPU)
| Benchmark | Focus | Official Base (Paper) | Original Base (Local) | This Model (LoRA SFT) | Delta |
|---|
| RealWorldQA | Spatial & Real-World Reasoning | 58.43% | 58.04% | 51.37% | -6.67% |
| OCRBench | Text, Chart & Diagram Reading | 684 / 1000 | 612 / 1000 | 451 / 1000 | -161 pts |
| POPE | Object Hallucination Resistance | 86.93% | 79.40% | 74.00% | -5.40% |
Model Architecture & Training
- Backbone: Liquid Foundation Model (LFM2 350M), hybrid 1D short convolutions + grouped-query attention
- Vision Encoder: SigLIP2 NaFlex encoder (768 hidden dimension, 12 layers)
- Multimodal Projector: 2-layer GELU spatial projection (downsample factor 2)
- Fine-Tuning: LoRA (rank 16, alpha 16, dropout 0.05) on 3,500 multimodal samples across VQA, chart reasoning, diagram understanding, and localized narratives. Vision encoder kept frozen.
- Context Length: 128,000 tokens
📁 Repository Files
1. GGUF Files (for llama.cpp / edge inference)
| File | Quantization | Size | Description |
|---|
LFM2.5-VL-450M-Q4_K_M.gguf | Q4_K_M | 218.69 MB | 4-bit quantized text backbone (recommended for edge deployment) |
LFM2.5-VL-450M-F16.gguf | F16 | 678.53 MB | Full precision 16-bit text backbone |
mmproj-LFM2.5-VL-450M-F16.gguf | F16 | 180.36 MB | Full precision SigLIP2 multimodal vision projector |
mmproj-LFM2.5-VL-450M-Q8_0.gguf | Q8_0 | 98.05 MB | 8-bit quantized multimodal vision projector |
2. Standalone Hugging Face Checkpoint
model.safetensors (855.91 MB)config.json, processor_config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja, generation_config.json
💻 How to Run
Using llama.cpp (llama-mtmd-cli)
# Run with Q4_K_M text model + F16 vision projector:
llama-mtmd-cli \
-m LFM2.5-VL-450M-Q4_K_M.gguf \
--mmproj mmproj-LFM2.5-VL-450M-F16.gguf \
--image /path/to/image.jpg \
-p "Describe what you see in this image." \
--temp 0.1 \
--repeat-penalty 1.15
Using Python transformers
import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText
model_id = "Varun12387/LFM2.5-VL-450M-SFT-GGUF"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
image = Image.open("example.jpg")
messages = [
{
"role": "user",
"content": [
{"type": "image"},
{"type": "text", "text": "Describe this image in detail."}
]
}
]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=prompt, images=image, return_tensors="pt").to(model.device)
with torch.no_grad():
output = model.generate(**inputs, max_new_tokens=128)
print(processor.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))