Skip to content

Artiprocher/FastSDXL

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

FastSDXL

This is an efficient implementation of Stable-Diffusion-XL. I make the following improvements:

  • Reconstruct the architecture of UNet. This UNet implementation is faster than others (Diffusers, Fooocus, etc.). If you are interested in this implementation, please see FastSDXL/BlockUNet.py. The source code of this UNet is short and easy to understand. You can also use this component in your own projects.
  • Use a trainable scheduler named OLSS. The implementation of OLSS is based on this project. I find this scheduler can improve the quality of generated images with given steps but we need to train it first (the training process requires a few minutes). To synthesize images with a specific style, OLSS is a good choice.

Usage

The code is headless. I developed this project based on diffusers==0.21.3. If you find it cannot run with another version of diffusers, please open an issue and tell me.

pip install diffusers safetensors torch gradio

To launch a webui without OLSS scheduler, please run the following command.

python launch_without_olss.py

To train an OLSS scheduler, please see run_olss.py for more details.

I trained an OLSS scheduler with the style of "Slightly Cinematic". You can use it by running the following command.

python launch_with_olss.py

Efficiency of the reconstructed UNet

I tested my code using NVidia 3060 laptop (6G, 85W). The resolution is 1024*1024, and the model is converted to float16 format.

  • Diffusers: CUDA out of memory
  • Fooocus: 1.78s/it
  • FastSDXL (ours): 1.17s/it

Performance of OLSS scheduler

Here are some examples of OLSS.

Prompt template: cinematic still {prompt}. emotional, harmonious, vignette, highly detailed, high budget, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy Negative prompt: anime, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured

  • a young girl, black hair, white clothes, in a garden, the background is red and white flowers

In the image generated by OLSS, the details of flowers on the right are more realistic.

DDIM DPM OLSS
ddim dpmsolver olss
  • a forest in spring, birds

More birds.

DDIM DPM OLSS
ddim dpmsolver olss
  • an orange cat and a pink ball on a white sofa

In this example, the image generated by OLSS is completely different with others.

DDIM DPM OLSS
ddim dpmsolver olss
  • a robot, with blue lightsaber, in a city

Sometimes OLSS can modify the composition and fix the composition error.

DDIM DPM OLSS
ddim dpmsolver olss

About

An efficient implementation of Stable-Diffusion-XL

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages