The model is created using the following steps:
- Find the desired model (checkpoint or lora) on Civitai
- You can see some conversion scripts in diffusesrs. This time, only the scripts for converting checkpoit and lora are used. It depends on the model type of Civitai. If it is a lora model, you need to specify a basic model
- Using __load_lora function from https://towardsdatascience.com/improving-diffusers-package-for-high-quality-image-generation-a50fff04bdd4
from diffusers import DiffusionPipeline
import torch
pipeline = DiffusionPipeline.from_pretrained(
"Andyrasika/lora_diffusion"
,custom_pipeline = "lpw_stable_diffusion"
,torch_dtype=torch.float16
)
lora = ("/content/lora_model.safetensors",0.8)
pipeline = __load_lora(pipeline=pipeline,lora_path=lora[0],lora_weight=lora[1])
pipeline.to("cuda")
# pipeline.enable_xformers_memory_efficient_attention()
#https://huggingface.co/docs/diffusers/optimization/fp16
pipeline.enable_vae_tiling()
prompt = """
shukezouma,negative space,shuimobysim
a branch of flower, traditional chinese ink painting
"""
image = pipeline(prompt).images[0]
image
Since this is only the first official release, I believe there are still many, many imperfections. Please provide feedback in time, and I will continuously make corrections, thank you!