Handbag sketch + ten attributes → native 1024px photo
Full pretrained Stable Diffusion XL ControlNet pipeline, with a ByteDance four-step SDXL-Lightning UNet. Native output resolution: 1024×1024 RGB, replacing the old 64px scratch generator. All actual pretrained weights are included in safetensors format, together with inference code, provenance, a runnable notebook, and generated examples.
No fine-tuning on kwongnon's handbag dataset was performed. This is a composition of existing pretrained components and custom sketch preprocessing/attribute prompts. There is no newly trained handbag LoRA. Dataset real photos are used only for visual comparisons, never as generation inputs. Attributes are provided by the user; they are not inferred automatically.
Why these models
Model popularity checked through the Hugging Face API when building this replacement: SDXL base 4,091,502 downloads in the preceding month; Xinsir scribble ControlNet 20,995; SDXL-Lightning 91,190. SDXL was selected as a widely downloaded, high-resolution base with compatible sketch conditioning. Downloads do not prove task performance. Counts change over time. The recorded comparisons to other candidates are in model_selection.json.
Sources: SDXL, Xinsir sketch ControlNet, SDXL-Lightning.
SDXL text encoders, tokenizers and VAE are from base revision 462165984030d82259a11f4367a4eed129e94a7b. The UNet is the pretrained sdxl_lightning_4step_unet.safetensors, revision c9a24f48e1c025556787b0c58dd67a091ece2e44. The pretrained ControlNet revision is 0bed4973b80c2329f76e0abf4196c51dfbf9108c. The original SDXL base UNet is replaced by the distilled UNet.
Inputs and generation
Required attributes: angle, bag_type, strap_type, closure, hardware, material, color, structure, shape, size. All keys must be present; None means unspecified. Two CLIP prompt streams keep the full set within each encoder's 77-token limit. Long custom descriptions raise a clear error instead of silently dropping the final attributes.
The sketch is white-padded to 1024px and converted to white strokes on black with threshold 245. Only the sketch and attribute text guide the image. No filenames, bag IDs, URLs, source photographs or reference-photo embeddings are model inputs.
EulerDiscreteScheduler with trailing timesteps, exactly four denoising steps, guidance_scale=0 (as recommended for Lightning), controlnet scale .8. Recorded examples use seed 24679. Attention slicing, model CPU offload and VAE tiling reduce memory pressure. CUDA is recommended; Apple Silicon MPS was used in the recorded run. CPU is much slower.
Validation and honest performance limits
A native 1024px validation example was generated, then the full pipeline was exported and reloaded successfully. Reloaded weights generated two fixed test examples: bag002_front_sketch, bag018_front_sketch. The examples are the first original from each of the first two test bag groups, chosen before viewing outputs. Generated PNG dimensions are verified at 1024×1024. Source sketches/real photos and generated images are in examples/; comparison.png also shows the previous 64px generator for the same inputs.
Visual samples are sharper and more photograph-like than the scratch model. They do not establish that all requested attributes are correctly followed. No dataset-wide PSNR, SSIM, FID or attribute accuracy is claimed. Exact target-photo reconstruction is not guaranteed. Realism and pixel similarity to the original photo are different objectives. Fine details, closure, hardware, structure, size, material and background color can deviate from the prompt. The model may invent branding or text. Absolute size cannot be established without a reference scale.
Dataset revision: 98d85555d5f2e6d782599b8c73f0d27ff4c67456, kwongnon/handbag-sketch-dataset-100x. Existing bag-level train/validation/test groups remain separated; no dataset rows were used for fitting this pipeline. This test set was also used in earlier project experiments. Unknown overlaps with the upstream pretraining data cannot be excluded by local split checks.
Run inference
Install requirements.txt in a separate Python environment and authenticate for this private repository. Download the repository and run from its directory:
python generate.py --weights . --sketch /path/to/sketch.png --attributes example_attributes.json --output handbag-1024.png
Python:
from PIL import Image
import json
from pipeline import load_pipeline, generate
pipe=load_pipeline(weights='.',steps=4)
attributes=json.load(open('example_attributes.json'))
image=generate(pipe,Image.open('my-sketch.png'),attributes,seed=24679)
image.save('handbag-1024.png')
Use the exported four-step weights with four steps. Eight-step generation requires loading the matching upstream eight-step UNet and is not the exported checkpoint. See USAGE.md. The notebook includes full loading/generation, dataset checks, visual comparisons, export, model card and private upload; it does not perform fine-tuning. Model outputs may vary across devices/library versions even with a fixed seed.
Licenses and access
SDXL and Lightning use OpenRAIL++; Xinsir ControlNet uses Apache-2.0. Follow all upstream component licenses and restrictions; this repository does not grant different rights. Dataset comparisons follow the source dataset's access conditions. The repository remains private. Intended for handbag design/prototyping; review generated details before treating the output as an accurate depiction of a product.