Qwen2-VL-7B finetuned on crochetBench Task C
This model is a fine-tuned version of Qwen/Qwen2-VL-7B-Instruct, trained on the crochetBench Task C benchmark.
- Paper: CrochetBench: Can Vision-Language Models Move from Describing to Doing in Crochet Domain?
- Repository: GitHub
Task Description
Task C — Image-to-Pattern Generation: Given an image of a finished crochet product, the model generates a complete set of crochet instructions in standard published-pattern format.
The output follows these conventions:
- Standard abbreviations:
sc(single crochet),hdc(half double crochet),dc(double crochet),tr(treble),ch(chain),sl st(slip stitch),rep(repeat) - Organized row-by-row or round-by-round (e.g.,
Rnd 1: ...,Row 2: ...) - Concise and precise, targeting experienced crocheters
- Pattern instructions only — no commentary or extra text
Model Details
| Property | Value |
|---|---|
| Base model | Qwen/Qwen2-VL-7B-Instruct |
| Fine-tuning method | Full fine-tuning |
| Optimizer | AdamW |
| Learning rate | 1e-5 |
| Epochs | 3 |
| Effective batch size | 8 |
| Best checkpoint | Step 1500 |
| Training date | December 13, 2025 |
Usage
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from PIL import Image
import torch
model_id = "peiyugeorgia/qwen2vl-crochetbench-task-c-finetuned"
model = Qwen2VLForConditionalGeneration.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_id)
image = Image.open("your_crochet_image.jpg").convert("RGB")
messages = [
{
"role": "system",
"content": (
"You are a professional crochet pattern writer. "
"Examine the image of the finished crochet product carefully. "
"Write a complete set of crochet instructions in the standard style used in published patterns. "
"Use standard abbreviations: sc (single crochet), hdc (half double crochet), dc (double crochet), "
"tr (treble), ch (chain), sl st (slip stitch), rep (repeat). "
"Organize the instructions row by row or round by round (e.g., 'Rnd 1: ...', 'Row 2: ...'). "
"Keep the instructions concise and precise, as if for experienced crocheters. "
"Output only the crochet pattern. Do not add any explanations, commentary, or extra text."
)
},
{
"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": "Generate step-by-step crochet instructions for this image."}
]
}
]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
with torch.no_grad():
output_ids = model.generate(**inputs, max_new_tokens=1024)
generated = processor.batch_decode(output_ids, skip_special_tokens=True)[0]
print(generated)
Citation
If you use this model, please cite the crochetBench benchmark.
@inproceedings{peiyuli2026,
title = {CrochetBench: Can Vision-Language Models Move from Describing to
Doing in Crochet Domain?},
author = {Li, Peiyu and Huang, Xiaobao and Hua, Ting and Chawla, Nitesh V.},
booktitle = {Proceedings of the 64th Annual Meeting of the Association for
Computational Linguistics (Volume 1: Long Papers)},
year = {2026}
}