CyberAI Cup 2026 — Task 1: Packaging Material Difference Mining
Complete working state of the solution: trained weights, all source code, documentation, and the raw evidence behind every reported number. Private repository — this is a working handover, not a release.
Current result
Fold-0 out-of-fold global F1 0.9049 (precision 0.9225, recall 0.8881, score threshold 0.31), measured on 40 held-out validation pairs with the competition's own metric: TP/FP/FN accumulated across all images before computing precision and recall, boxes matched at IoU ≥ 0.5.
For scale, the classical image-difference baseline scores 0.029 on the same split.
Folds 1–4 are untrained and no submission has been generated. HANDOVER.md section 5 holds the
prioritized queue for whoever continues.
What is in here
| Path | Contents |
|---|---|
checkpoints/stage1_fold0_convnext_tiny/best.pt | The model — 128 MB, epoch 19, the 0.9049 checkpoint |
checkpoints/stage1_fold0_convnext_tiny/last.pt | 384 MB, epoch 35, with optimizer and scheduler state for resuming |
checkpoints/oof/ | Out-of-fold candidate boxes and the threshold sweep |
checkpoints/stage2_ab/ | Verifier from the A/B test — reference only, its measured delta was +0.0009 |
src/pmdm/ | 14 modules, ~1500 lines: preprocessing, model, losses, decoding, training, inference, metric |
modal_app.py | Every Modal function and entrypoint used to produce these results |
scripts/ | CPU smoke test, local training without Modal, dataset analysis, error analysis |
runs/ | Raw training logs, per-epoch history, and analysis output from every run |
Task1/ | The competition dataset: 200 training pairs with 1407 annotated boxes, 100 test pairs. Uploading separately — 2.5 GB of page-scale PNGs takes hours on a home uplink, so it lands after the code and weights (scripts/push_dataset_to_hf.py) |
HANDOVER.md | Start here — architecture, results, error breakdown, mistakes, next steps |
RUNBOOK.md | Every command with measured timings |
IMPLEMENTATION_PLAN.md | Original design rationale |
Architecture
A siamese CenterNet. Both images pass through a shared ConvNeXt-tiny encoder taking 4 channels
(BGR plus a local-background-normalized "ink map" that makes the features invariant to the photo's
shadows). Features fuse per scale as conv(concat(a, b, |a − b|)), decode through a U-Net upsampling
path, and emit four heads at output stride 2 — a Gaussian focal heatmap for difference centers,
width/height, a sub-pixel offset, and an auxiliary change-mask segmentation.
Stride 2 is the load-bearing choice: 83 of the ground-truth boxes are 8×8 pixels, which vanish entirely at the stride 4 a standard CenterNet uses.
Three things about the data drove the rest of the design:
- Pairs are already sub-pixel aligned. Of 300 pairs, 284 need no registration at all and 16 need a pure translation. No warping is applied by default.
- Blur, not geometry, is the dominant false-positive source. The photo is softer than the template, so the template is blur-matched to it with a σ grid search minimizing the Laplacian-variance gap before any comparison.
- Nothing is ever deleted. Of 1404 analysed boxes, 842 are additions and 558 are modifications of existing content — and exactly 0 are deletions. That licenses a polarity filter discarding any candidate that is inked in the template and blank in the photo.
Inference tiles each page at 768 px with 25% overlap, then merges candidates with Weighted Boxes Fusion rather than NMS — with boxes this small, averaging the coordinates beats picking one.
Loading the model
import torch
from pmdm.model import SiamCenterNet
model = SiamCenterNet(backbone="convnext_tiny")
ck = torch.load("checkpoints/stage1_fold0_convnext_tiny/best.pt", map_location="cpu")
model.load_state_dict(ck["model"]) # all keys match
model.eval()
ck["metric"] carries the evaluation that selected this checkpoint, including the 0.31 threshold.
Full tiled inference over a page is pmdm.infer; scoring is pmdm.metric.global_f1.
Reproducing
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python -e . torch torchvision timm \
opencv-python-headless "numpy<2" pandas
# CPU pre-flight over the whole pipeline, ~2 min
PMDM_DATA=Task1/PackagingMaterialDifferenceMiningDataset \
PMDM_WORK=/tmp/pmdm_work PMDM_CKPT=/tmp/pmdm_ckpt \
.venv/bin/python scripts/local_smoke.py
# every dataset claim above
PMDM_DATA=Task1/PackagingMaterialDifferenceMiningDataset \
.venv/bin/python scripts/analyze_dataset.py
# the error breakdown, from the archived out-of-fold predictions
.venv/bin/python scripts/error_analysis.py checkpoints/oof/fold0_convnext_tiny.npz
Training runs either on Modal (modal run --detach modal_app.py::train --fold 0 --epochs 40,
~45 min on an A100-40GB) or on any GPU box with no Modal account
(python scripts/train_local.py --fold 0 --epochs 40). Same loop, same checkpoints.
Where the remaining errors are
Of 268 ground-truth boxes across the validation fold: 252 were proposed and matched, 5 were proposed but localized too loosely to clear IoU 0.5, and 11 were never proposed at all — a recall ceiling of 0.959 for the current proposal stage.
By box size, the picture is lopsided: boxes above 24 px match 97.5% of the time, 12–24 px match
97.9%, and boxes under 12 px match 82.8%. Ten of the eleven complete misses are sub-12px marks.
Everything else is solved. That is why the next step is synthetic data weighted toward tiny marks
(src/pmdm/synth.py, written and smoke-tested but never run at scale) rather than more box
refinement or a stronger verifier.
License
The dataset in Task1/ belongs to the CyberAI Cup 2026 organizers and is included here only so this
private repository is self-contained for the team. Do not redistribute it.