ColorfulXL-Lightning
Model Details
v1.6. Based on ColorfulXL, with lightning ability.
- Fine-tuned on aesthetics
- Unet trained from 512 to 1280 with 64 steps
- Good prompt follow, trained text encoder
- Custom VAE
- Merged with 2,4,8, steps lightning unets from bytedance (supermario)
- Merged back with base model (not SGMUniform Euler works)
- Use with zero/low cfg
- Custom VAE
- Ability to generate pure white/black - colorful model with true colors
High range of resolutions supported (576 - 1280), 576*832 example:
Colorful:
There are problems with hands and faces (But who cares? Just say it's art!):
Detailed review (version 1.0), thx to Stevie2k8!
Usage
from diffusers import DiffusionPipeline
from diffusers import EulerDiscreteScheduler
import torch
pipeline = DiffusionPipeline.from_pretrained("recoilme/ColorfulXL-Lightning", torch_dtype=torch.float16,variant="fp16", use_safetensors=True).to("cuda")
pipeline.scheduler = EulerDiscreteScheduler.from_config(pipeline.scheduler.config, timestep_spacing="trailing")
prompt = "girl sitting on a small hill looking at night sky, fflix_dmatter, back view, distant exploding moon, nights darkness, intricate circuits and sensors, photographic realism style, detailed textures, peacefulness, mysterious."
height = 1024
width = 1024
steps = 3
scale = 0
seed = 2139965163
generator = torch.Generator(device="cpu").manual_seed(seed)
image = pipeline(
prompt = prompt,
height=height,
width=width,
guidance_scale=scale,
num_inference_steps=steps,
generator=generator,
).images[0]
image.show()
image.save("girl.png")
Local inference
https://github.com/recoilme/100lineSDXL
Space (upgrade on GPU)
https://huggingface.co/spaces/recoilme/ColorfulXL-Lightning
Model Details
- Developed by: AiArtLab
- Model type: Diffusion-based text-to-image generative model
- Model Description: This model is a fine-tuned model based on colorfulxl.
- License: This model is not permitted to be used behind API services. Please contact vadim-kulibaba@yandex.ru for business inquires, commercial licensing, custom models, and consultation.
Uses
Direct Use
Research: possible research areas/tasks include:
- Generation of artworks and use in design and other artistic processes.
- Applications in educational or creative tools.
- Research on generative models.
- Safe deployment of models which have the potential to generate harmful content.
- Probing and understanding the limitations and biases of generative models.
Excluded uses are described below.
Out-of-Scope Use
The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
Limitations and Bias
Limitations
- The model does not achieve perfect photorealism
- The model cannot render legible text
- The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”
- Faces and people in general may not be generated properly.
- The autoencoding part of the model is lossy.
Bias
While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.
Contact
- For questions and comments about the model, please join https://aiartlab.org/.
- For future announcements / information about AiArtLab AI models, research, and events, please follow Discord.
- For business and partnership inquiries, please contact https://t.me/recoilme