Image Context Sorter — DINOv2 partial fine-tune
Restoration-work photo classifier for 하우스캐치 (water-damage restoration). Classifies a single photo into one of 8 work-type / stage categories.
Private / internal — trained on client work photos; not for redistribution.
Results (locked test set, 8 classes)
| Model | macro-F1 | top-2 |
|---|---|---|
| 518px frozen linear probe (baseline) | 0.601 | 0.830 |
| 224px frozen linear probe (control) | 0.617 | 0.800 |
| 224px partial fine-tune (this model) | 0.737 | 0.895 |
Every class improved over the probe; the previously weakest class (곰팡이제거) rose 0.44 → 0.67 and its 곰팡이제거↔누수탐지 confusion was eliminated.
Model
- Base:
vit_base_patch14_dinov2.lvd142m(timm / Meta DINOv2) - Fine-tune: partial — the last 4 transformer blocks + final norm + a new 8-class head are unfrozen; the rest of the backbone is frozen. Input 224px (position-embedding interpolation). Discriminative LR (backbone 1e-5 / head 1e-3), AdamW + cosine schedule, float16 autocast (Apple MPS), class-weighted cross-entropy, early stopping on val macro-F1. No vertical-flip augmentation (ceiling/floor structure is a label cue).
- Calibration: temperature scaling, T ≈ 2.44 (fit on the val split). Apply it to logits before softmax for honest confidence; it does not change argmax/accuracy.
Classes (index order)
0 철거중 · 1 비포1 · 2 애프터 · 3 보양작업 · 4 석고보드설치 · 5 누수탐지 · 6 도배작업 · 7 곰팡이제거
How to load
import timm, torch
from torchvision import transforms as T
model = timm.create_model(
"vit_base_patch14_dinov2.lvd142m", pretrained=False, num_classes=8, img_size=224
)
model.load_state_dict(torch.load("finetune.pt", map_location="cpu"))
model.eval()
tf = T.Compose([
T.Resize(224), T.CenterCrop(224), T.ToTensor(),
T.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
# logits = model(tf(img).unsqueeze(0))
# probs = torch.softmax(logits / 2.44, dim=1) # temperature-calibrated
License
This is a derivative of Meta's DINOv2 (vit_base_patch14_dinov2.lvd142m).
Use is governed by the DINOv2 model license — see
https://github.com/facebookresearch/dinov2. Confirm redistribution / commercial
terms before any external distribution.
Notes
- Weights are a raw PyTorch
state_dict(finetune.pt). Consider converting tosafetensorsfor safer, code-free loading. - See
config.jsonfor the exact class list, temperature, and preprocessing.