preparing.ai - DesignForge Qwen2.5-VL 3B (screenshot + brief -> HTML)
Fine-tune of Qwen/Qwen2.5-VL-3B-Instruct that turns a webpage screenshot plus a short design
brief into a complete self-contained HTML/CSS/JS document.
- Full merged model (fp16):
ivan123-123/preparing.ai - LoRA adapter only:
ivan123-123/preparing.ai-lora
Usage
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
"ivan123-123/preparing.ai", torch_dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained("ivan123-123/preparing.ai")
SYSTEM = ("You are DesignForge, an expert front-end engineer and UI designer. Given a webpage "
"screenshot and a design brief, you write one complete, polished, responsive, "
"self-contained HTML document (HTML, CSS and JavaScript as needed). Output only the HTML.")
msgs = [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": [
{"type": "image", "image": image},
{"type": "text", "text": "Design brief:\n" + brief + "\n\nGenerate the complete webpage."},
]},
]
prompt = processor.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[prompt], images=[image], return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=2500, do_sample=True,
temperature=0.6, top_p=0.9, top_k=50)
html = processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0]
open("page.html", "w", encoding="utf-8").write(html)
Inference settings
- Use sampling (
temperature=0.6, top_p=0.9, top_k=50). Greedy decoding on fp16 can degenerate to repeated characters on some inputs. - The processor ships with the training pixel budget:
min_pixels=100352, max_pixels=301056(≈1008x784). Keep these so images are framed as they were during training. - Expected output length: ~1.7k-2.6k tokens (one HTML document, no markdown fences).
Training / evaluation
- 200 optimizer steps, grad-accum 4, LR 2e-4 cosine, LoRA r=16 on a P100, bf16/4-bit NF4.
- 504 hand-written brief + screenshot pairs (WebSight-style), 500 train / 4 holdout.
- Held-out: 3/3 structurally valid and rendered HTML, mean SSIM 0.67 vs the reference screenshot.
- Artifact test (4-bit + LoRA, sampling): PASS.