HN user
Model allows image generation in 4-8 steps. 8 steps on M1 Max take 8 seconds, on RTX 3090Ti just 3 seconds. Requires latest chrome canary. Model paper here https://latent-consistency-models.github.io
Hold on, to run your demo does one have to click the "Load Model" button before doing anything?
Yes. I thought it won't be good if it would download 3.5gb once you open the page.
Now I'm wondering whether the top message goes away once the flags are enabled?
No, I haven't added any checks for that (and I'm not sure how the first one can be properly checked), so it's just an info bar. Which is, eventually, misleading.
What error message do you get?
UNET takes about a 1:10 on WebGPU and around a minute on CPU in one thread. VAE is 2 minutes on CPU and about 10 seconds on GPU. It should be because most GPU ops for VAE are already implemented but for UNET are not. So in the latter case browser is just tossing data from GPU to CPU and back on each step
That's really impressive and much more performant. I was following a different approach: to run any ONNX model without prior modifications.
No additional libraries required on Windows (even CUDA). For linux you'll need to install CUDA and onnxruntime gpu lib.