/ Модель

Характеристики

Лицензия
CC BY 4.0
Библиотека
diffusers
Опубликована
24 авг 2026

canolaxray-conditional-DDPM

Генерация изображенийТекст → изображение

Пиксельная DDPM-модель создаёт рентгенограммы семян рапса по карте силуэта и повреждений для дополнения и стресс-тестирования наборов научных изображений.

0скачиваний за 30 дней0лайковвеса обновлены24 авг 2026

Бенчмарки

Как модель справилась с задачами пользователей ailist и что о ней публикуют открытые лидерборды

Публичные оценки

Из открытых лидербордов. Сравнивайте результаты одного теста при одинаковых условиях — отсутствие оценки не значит, что модель хуже.

В подключённых источниках результатов для этой модели пока нет.

Что умеет модель

Описание составлено по карточке модели и метаданным Hugging Face

CanolaXray-DDPM — модель условной генерации рентгенограмм семян рапса (Brassica napus), разработанная для исследования синтеза научных изображений. Она принимает рентгенограмму семени и построенную по ней двухканальную карту силуэта и внутренних повреждений, а также оценку повреждения ND, LD, MD или HD; на выходе создаёт синтетическую рентгенограмму с пространственно заданным рисунком повреждения. Модель построена с нуля на базе пиксельной диффузионной модели DDPM: используется time-conditional U-Net из Diffusers с шириной 64 канала, множителями каналов (1, 2, 2, 4), двумя residual-блоками на уровне и 19,1 млн обучаемых параметров. Предобученные компоненты не используются. Модель обучалась на 4180 рентгенограммах семян четырёх градаций повреждения и генерирует изображения размером 256 × 256 пикселей.

Основные сценарии применения — дополнение размеченных наборов рентгенограмм, генерация образцов с заданными силуэтом и степенью повреждения для стресс-тестирования классификаторов, а также исследовательский базовый вариант условной генерации в узкой области научной визуализации. Это не генератор изображений по тексту и не средство диагностики семян. Результаты согласованности оцениваются тем же детектором, который задаёт карту повреждения, поэтому они не подтверждают соответствие человеческой оценке. Модель опубликована под лицензией CC BY 4.0.

Сценарии применения

  • генерация рентгенограмм семян рапса по заданной карте повреждений
  • дополнение размеченных корпусов изображений семян для обучения классификаторов
  • создание образцов с заданной степенью повреждения для стресс-тестирования классификатора
  • исследовательская условная генерация изображений в фенотипировании семян

Ограничения

Модель требует карты условий, построенной тем же детектором, и предназначена для реконструкции по конкретному образцу, а не для открытой генерации. Качество и полезность для последующих задач оценены ограниченно; синтетические изображения не следует использовать как диагностическое свидетельство о реальных партиях семян.

Карточка модели

README от автора на Hugging Face, как он его опубликовал

Открыть на Hugging Face

CanolaXray-DDPM — structure-conditioned diffusion for soft X-ray seed radiographs

A pixel-space conditional DDPM that synthesises soft X-ray radiographs of individual canola (Brassica napus) seeds from a measured damage map rather than from a text prompt. Trained from random initialisation on the target corpus alone: no pretrained backbone, no text encoder, no autoencoder.

This is Approach 1 of a two-approach study. The counterpart is canolaxray-qwen-lora, a LoRA adaptation of a 20 B pretrained flow-matching transformer conditioned on captions derived from the same measurements.


What it does

For every radiograph, a two-channel condition image is computed automatically:

channelcontentshow it is built
0seed silhouettebackground threshold → largest connected component → binary closing → hole filling → light blur
1dark-structure responsegrey-scale black top-hat, D = φ_B(a) − a, at two structuring-element scales combined by pointwise maximum, size-filtered, kept continuous

The map is concatenated to the noisy sample along the channel axis before the first convolution, and the ordinal grade (ND/LD/MD/HD) enters separately as a learned class embedding added to the timestep embedding. Because the condition lives on the output pixel grid, a fissure is requested at a coordinate, and adherence can be scored afterwards by re-running the same detector on the generated image.

The damage grade is deliberately not encoded in the map and the map is not normalised per class — either would leak the label into the spatial channel and make the ablations uninterpretable.

Model details

Architecturetime-conditional U-Net (diffusers.UNet2DModel), base width 64, channel multipliers (1, 2, 2, 4), 2 residual blocks per level
Parameters19.1 M (all trainable)
Input channels1 + C_cond
Attentionself-attention at 32×32 and 16×16 only
NormalisationGroupNorm, modulated by [γ, β] = MLP(e_t + e_y)
Resolution256 × 256, pixel space
Formulationdiscrete DDPM, cosine schedule, T = 1000
Prediction targetvelocity v, Min-SNR loss weighting with γ = 5
Guidancedecomposed classifier-free guidance over two independently dropped signals (map 0.10, label 0.10)
SamplerDDIM, 250 steps, η = 0, guidance scale 1.5, joint mode
Weights releasedEMA weights (decay min(0.9995, (1+k)/(10+k)))
Precisionfp32 / AMP-trained

Guidance uses the decomposed estimator

ε̃ = ε(∅,∅) + s_c · [ε(c,∅) − ε(∅,∅)] + s_y · [ε(c,y) − ε(c,∅)]

which is only well defined because the map and the label are dropped independently during training, so all four combinations are observed.

Training data

4180 canola seed radiographs in four ordinal internal-damage grades, split by a stratified procedure into 3553 training / 627 held-out images. The split is fixed by a random seed and is identical to the one used for the LoRA model, so any difference in outcome reflects the method rather than the data.

gradetotaltrainheld out
ND (no damage)15651330235
LD (low)990842148
MD (medium)1092928164
HD (high)53345380

Augmentation is restricted to the dihedral group of the square (8 exact pixel permutations), applied jointly to the image and the condition map. No interpolating transform is used, because part of the evaluation asks whether high-frequency detector grain survives, and interpolation blur in the training data would bias that measurement.

Training configuration

settingvalue
OptimiserAdamW, lr 1e-4, cosine decay, 500 warmup steps
Batch16 × 2 gradient accumulation (effective 32)
Dropout0.1 inside residual blocks
Memorygradient checkpointing (batch 32 without it exhausts a 40 GB A100 on activations, not parameters)
Duration300 epochs configured; released checkpoint is epoch 200

⚠️ The released checkpoint is the most recent one on disk at the time of evaluation, not the validation-loss minimum, and the configured run was not completed. Reported numbers may understate what this configuration can reach.

Results (held out, 64 generated images per grade, 256 × 256)

Distributional. FID 0.01 overall against a real-versus-real floor of 0.02 at this sample size. Background luminance ≤ 0.0002 against machine zero for real images. Absolute FID here sits far below the natural-image scale because these radiographs are simple images — one compact bright object on a uniform black field — and is not comparable to FID reported elsewhere or to the 1024 px LoRA model.

Feature-space separability. A per-grade permutation test on the ten-nearest-neighbour statistic finds no significant difference from the null of indistinguishability in any grade: HD 0.539 vs 0.560 expected (p = 0.60), LD 0.581 vs 0.614 (p = 0.36), MD 0.617 vs 0.610 (p = 0.88), ND 0.636 vs 0.612 (p = 0.45). Every per-grade control is correctly calibrated.

Paired fidelity and generalisation.

gradeSSIM (own held-out target)PSNR (dB)SSIM (nearest train image)
ND0.982337.490.9682
LD0.985939.440.9661
MD0.983037.500.9570
HD0.981936.750.9503

Mean 0.9833 to the true target against 0.9604 to the nearest training image; 79.7 % of generated images score higher against their own target than against their nearest training neighbour (Mann–Whitney U = 51,754, p = 4.0 × 10⁻³⁰). That is the signature of composition from the condition, not retrieval of a memorised image.

Condition adherence.

gradesilhouette IoUdamage IoUdamage IoU (chance floor)
ND0.9980.9650.353
LD0.9980.9560.381
MD0.9980.9590.367
HD0.9980.9510.344

Requested vs rendered feature counts correlate at ρ = 0.858 (p = 1.7 × 10⁻⁷⁵, n = 256). The chance floor pairs each request with the damage rendered for a different generated image.

Severity control. Scaling the damage channel by s ∈ [0, 1.5] with the silhouette and sampling seed fixed gives ρ = 0.911 (p = 1.7 × 10⁻¹¹) between s and rendered damage area, monotone at every one of six steps, approaching the real-specimen reference near s = 1. This is a continuous handle that a categorical label cannot express. With the damage channel zeroed, the grade label alone has no reliable effect (ρ = −0.410, p = 0.19) — as designed: the spatial channel carries the severity signal.

Compositionality. Transplanting an HD damage map onto an ND silhouette yields the ND outline carrying the transplanted fissures, so geometry and damage are represented and combined rather than retrieved jointly.

Guidance trade-off. Raising guidance 1.0 → 5.0 drives FID 0.02 → 0.28 while damage IoU moves only 0.956 → 0.960–0.969 and background luminance drifts 0.0002 → 0.0044. Adherence is already near its ceiling at low guidance because the condition map fixes what guidance would otherwise have to enforce. Scale 1.5 sits early on that curve.

Texture statistics. Six pixel descriptors (area fraction, high-frequency noise, Laplacian variance, spectral slope, interior intensity variation, edge density) show rank-biserial effect sizes of −0.02 to 0.21; only spectral slope survives correction (q = 5.2 × 10⁻⁵) and at a small effect size.

Intended use

  • Augmenting annotated radiograph corpora for downstream damage-grading classifiers.
  • Generating spatially specified, severity-controlled specimens for stress-testing a grader — e.g. sweeping severity continuously on a fixed silhouette.
  • A from-scratch baseline for conditional generation in narrow scientific-imaging domains.

Limitations and out-of-scope use

  • This is not open-ended synthesis. Each condition map fixes the silhouette and damage pattern of one real specimen, so the task is closer to conditional reconstruction than to free generation. The favourable separability result should be read in that light; it is not evidence that this model beats a caption-conditioned one at the same task.
  • Not evaluated on coverage or downstream utility. PRDC coverage, the two-sample classifier test and the train-on-synthetic/test-on-real experiment were run for the LoRA model only.
  • One detector both defines and grades adherence. The adherence numbers measure internal consistency with the detector's notion of damage, not agreement with a human grader.
  • Checkpoint not selected by validation loss, and training incomplete (see above).
  • Pooled feature-space test disagrees with the per-grade tests. Running the neighbour test across all four grades together makes even the real-versus-real control significant (p < 0.001), plausibly because pooling without stratifying lets class-conditional clustering masquerade as a distributional difference. The per-grade result, where the control is well behaved, is the one relied on; the discrepancy is unresolved.
  • Resolution. 256 × 256 against the LoRA model's 1024 × 1024. FID is resolution-sensitive and the frequency descriptors live on the pixel grid, so any numerical comparison between the two models must be made at matched resolution.
  • Not a diagnostic tool. Synthetic radiographs are not evidence about any real seed lot. If these images augment a classifier, held-out validation and test partitions must consist exclusively of real acquisitions.

Usage sketch

import torch
from diffusers import UNet2DModel

unet = UNet2DModel.from_pretrained("MarwenBellili/canolaxray-conditional-DDPM", subfolder="unet").eval().cuda()

# cond: [B, 2, 256, 256] float in [0,1] — channel 0 silhouette, channel 1 damage response
# y   : [B] long in {0,1,2,3} for ND/LD/MD/HD
x = torch.randn(cond.shape[0], 1, 256, 256, device="cuda")
# DDIM, 250 steps, eta=0, decomposed CFG at scale 1.5 — see the notebook, §9

The condition map must be built with the detector from §4a of the training notebook. A map built by a different detector is out of distribution and adherence numbers will not transfer.

Related artefacts

Citation

@misc{bellili2026canolaxray,
  title  = {Generative Synthesis of Canola Seed X-Ray Radiographs: A Comparison of
            LoRA Adaptation of a Flow-Matching Transformer and a Conditional
            Denoising Diffusion Model},
  author = {Bellili, Marwen},
  note   = {Supervised by Mohammad Nadimi},
  year   = {2026}
}

Как запустить

На своей GPU в ailist. Цена — по текущим предложениям провайдеров

GPU в ailist

Универсальный рантайм · diffusers

Нельзя запустить

Безусловная генерация изображений пока не поддерживается.

Ревизия весов
6314c4a

0 моделей