inblick YOLOv8n: quantization-aware trained int8 detectors

Six YOLOv8n networks (plus two larger YOLOv8s, below) trained with quantization-aware training (QAT) for a custom int8 FPGA CNN accelerator (the inblick project, AMD AC701 / Artix-7). Two run at 320x320; four run at the non-square 16:9 sizes 640x384 and 480x288 (no letterboxing into a square, so a 16:9 camera wastes about 6 % of the input instead of 44 %). The QAT simulates the accelerator's exact int8 arithmetic:

• per-tensor asymmetric int8 activations with half-to-even rounding;
• per-channel symmetric int8 weights;
• int32 bias;
• SiLU as Q(x), Q(sigmoid), Q(x*sigmoid) through a lookup table.

The exported int8 graphs run bit-exact on the hardware.

Licence: AGPL-3.0 (derived from Ultralytics YOLOv8). The training and export source code is in source/ (AGPL-3.0). The CrowdHuman model is additionally bound by the CrowdHuman dataset terms: non-commercial research use only. See LICENSE-NOTICE.md.

Results (320x320, int8)

COCO val2017 (5000 images), mAP50-95 / mAP50

QAT recovers the whole int8 loss and ends slightly above fp32.

CrowdHuman val (4370 images), AP50-95 / AP50, int8: crowdhuman_qat_320 scores 38.15 / 68.66.

Multi-object tracking, held-out MOT17 (05/10/11/13) and DanceTrack (12 sequences), at 46 fps, with the ByteTrack/OC-SORT-style tracker of the project. Values are HOTA / IDF1:

Rows 2 and 3 use the presets in each folder's tracker_presets.json, which keep the board's on-board post-processing at full frame rate. With 4 and 12 held-out sequences, differences under about 1 HOTA are noise.

Non-square 16:9 models (640x384 and 480x288)

Accuracy (int8, onnxruntime; COCO val2017 mAP50-95 / mAP50 and CrowdHuman val AP50-95 / AP50), accelerator speed on the AC701 board (measured cycles per frame at 150 MHz) and held-out tracking at that frame rate with each folder's tuned preset (HOTA / IDF1; MOT17 and DanceTrack):

The COCO val images are 4:3, so their AP at 640x384 understates what a 16:9 camera gets. Each folder's README compares three ways of getting from the 320 network to the new size: (a) the 320 QAT weights and scales on the new graph, (b) a fresh PTQ export of them, (c) a short QAT re-train (for CrowdHuman, also an fp32 fine-tune at the new size first). All of them compile for the same accelerator bitstream; 640 is the widest input it can take (in_width * ceil(Cin / 16) of the stride-16 layer reaches the 960-word row buffer).

YOLOv8s models (KV260)

Two larger YOLOv8s detectors, 1-class CrowdHuman, same QAT arithmetic and export as above. They need the KV260 accelerator target (cnn_accel_v1_kv260, partial-sum ISA 3.2); the board speed is for the 4-core 250 MHz build. CrowdHuman val (4370 images), int8, onnxruntime:

Recipe: yolov8s.pt, 1-class head re-initialised, 60 fp32 epochs, 15 QAT epochs (lr 1e-4, batch 32 at 480x288 and 16 at 640x384), no 320 stage. No tracking benchmark was run for them; their tracker_presets.json are the YOLOv8n presets of the same size.

Tracker presets and gmc

tracker_presets.json of the 640x384 and 480x288 CrowdHuman folders (YOLOv8n and YOLOv8s) now sets "gmc": 2 in the _board_dt and _board_cmc presets: image camera-motion compensation, a similarity estimated by block matching on the detector's input image (bit-exact on the board; _board_cmc keeps cmc 3 as the fallback). Held-out MOT17 (YOLOv8n, board model): 480x288 41.8 / 51.3 HOTA / IDF1 with gmc 2 against 39.4 / 48.9 without; 640x384 43.9 / 55.1 against 40.8 / 50.2. The plain _board presets are unchanged.

Files (per folder)

Using the weights outside inblick

The .pt files are ordinary Ultralytics YOLOv8n checkpoints:

from ultralytics import YOLO
model = YOLO("crowdhuman_qat_320/crowdhuman_qat_320.pt")
results = model("image.jpg", imgsz=320)

They were fine-tuned to suit int8 rounding, so in fp32 they are slightly below the official weights. Their benefit shows when they are quantized the same way: per-tensor asymmetric activations and per-channel symmetric weights at 320x320. qdq.onnx runs in onnxruntime and outputs the six raw YOLOv8 heads (3 box-distribution and 3 class-score maps); decode them as YOLOv8 does (DFL with reg_max 16, then NMS).

Training

• Hardware: one RTX 5090. About 35 min for the COCO QAT, and about 27 min for the CrowdHuman model (fp32 fine-tune plus QAT).
• QAT optimiser: AdamW, lr 1e-4 for the weights and 2e-4 for the log activation scales, cosine schedule, EMA 0.999, batch 64, seed 0. Ultralytics 8.4.164, torch 2.14.0.
• COCO: 10 QAT epochs on train2017 from the official yolov8n.pt; epoch 6 kept.
• CrowdHuman:
  • start: the official yolov8n.pt with a re-initialised 1-class head;
  • fp32 fine-tune: 60 epochs on full-body (fbox) boxes, with mask and ignore regions dropped;
  • then 15 QAT epochs.
  • The CrowdHuman data came from the Hugging Face mirror sshao0516/CrowdHuman of the official release.

Full details are in each folder's README.md.

Source

source/ holds the QAT training and export code that produced these weights (AGPL-3.0): the fake-quantised QDQ network, the trainer, the YOLOv8n int8 export and the COCO evaluation, with its environment in requirements-train.txt. See source/README.md.