Qwen2.5-VL-7B — Per-Dataset Fine-Tuning Derivatives (LoRA + Full FT)
10 fine-tuned derivatives of Qwen/Qwen2.5-VL-7B-Instruct, one LoRA and one full fine-tune for each of 5 vision-language datasets. Produced as clean single-dataset transfer targets for a visual-fingerprint transfer study.
⚠️ Research artifacts. Each model is trained on a 3,000-example slice of one dataset for 3 epochs — these are experimental derivatives, not production models.
Shared training recipe (all 10)
- Base:
Qwen/Qwen2.5-VL-7B-Instruct - Frozen: vision tower + multimodal projector (
freeze_vision_tower: true,freeze_multi_modal_projector: true) — only the language-model backbone is trained. - Data:
max_samples = 3000per dataset, 3 epochs,cutoff_len = 2048. - Framework: LLaMA-Factory 0.9.3, transformers 4.52.4.
- LoRA variant: target
q_proj, v_proj; rank 16, alpha 32; lr 2e-4; trainable ≈ 0.06 % of params. - Full variant: DeepSpeed ZeRO-3; lr 1e-5;
image_max_pixels = 401408; trainable = 100 % of the LM (vision/projector still frozen).
Contents
| Path | Type | Trainable | train_loss (3 ep) |
|---|---|---|---|
lora/mathv360k | LoRA (q/v r16) | ~0.06 % | 0.151 |
lora/paintingform | LoRA (q/v r16) | ~0.06 % | 0.732 |
lora/stvqa | LoRA (q/v r16) | ~0.06 % | 0.351 |
lora/textvqa | LoRA (q/v r16) | ~0.06 % | 0.177 |
lora/visual7w | LoRA (q/v r16) | ~0.06 % | 0.527 |
full/mathv360k | Full FT (ZeRO-3) | 100 % LM | 0.148 |
full/paintingform | Full FT (ZeRO-3) | 100 % LM | 0.543 |
full/stvqa | Full FT (ZeRO-3) | 100 % LM | 0.262 |
full/textvqa | Full FT (ZeRO-3) | 100 % LM | 0.211 |
full/visual7w | Full FT (ZeRO-3) | 100 % LM | 0.398 |
Datasets: MathV360K, PaintingForm, ST-VQA, TextVQA, Visual7W (single-dataset VQA-style instruction tuning).
Usage
LoRA (needs the base model + PEFT):
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
base = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "DSA1/qwen25vl-7b-perdataset-derivatives", subfolder="lora/textvqa")
proc = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
Full FT (self-contained weights):
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
model = Qwen2_5_VLForConditionalGeneration.from_pretrained("DSA1/qwen25vl-7b-perdataset-derivatives", subfolder="full/textvqa", torch_dtype="auto", device_map="auto")
proc = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct") # processor/chat-template from base
Notes
train_lossis the mean training loss over all optimizer steps (the LoRA runs ~300 steps for the 4 smaller datasets and ~1014 for textvqa; the full runs are 129 steps at global batch 64).full/textvqawas taken from the run'scheckpoint-129(the final training step): its top-level save was interrupted by a cluster preemption, butcheckpoint-129holds the complete 4-shard, 16.6 GB, 729-tensor model. It is the fully-trained end-of-run model.- Full-FT models ship the LM weights only; load the processor / chat template from the base
Qwen/Qwen2.5-VL-7B-Instruct.
License
Inherits the base model's license (Qwen/Qwen2.5-VL-7B-Instruct). Datasets retain their respective licenses.