Stable Diffusion WebGPU demo

https://islamov.ai/stable-diffusion-webgpu/

by aislamov • 3 years ago

473 116 3 years ago

ccooffee2• 3 years ago

You need latest Chrome with "Experimental WebAssembly" and "Experimental WebAssembly JavaScript Promise Integration (JSPI)" flags enabled!

Darn, guess I'll have to wait for stuff to land in Firefox.

flangola7• 3 years ago

What does the second flag do

bhaney• 3 years ago

Turns on an implementation of this proposal

https://github.com/WebAssembly/js-promise-integration/blob/m...

bsimpson• 3 years ago

I tried turning them on in Canary, but I still couldn't get it to work.

veb2• 3 years ago

I tried on latest (normal) Chrome, beta and canary/nightly and enabled both options one by one and let it relaunch but still wouldn't work at all. ¯\_(ツ)_/¯

How are others getting it working?!

From someone else's comment, this one works fine: https://websd.mlc.ai/#text-to-image-generation-demo

1letterunixname• 3 years ago

It works on canary on M1 mac and Windows w/ an NVIDIA RTX GPU. I believe there are custom command line options that have to be passed to make it work. The MLC site has the deets that work.

cortesoft• 3 years ago

You refuse to even try something on chrome?

ccooffee2• 3 years ago

Nah, I don't use Chrome so I don't have it installed. I'm not a web developer, so testing across different platforms isn't useful to me. I've used StableDiffusion before, so hacking around to make this demo work in my browser isn't particularly interesting either.

ChuckMcM• 3 years ago

Yes. You don't?

INGSOCIALITE2• 3 years ago

I agree with the poster 100%. Im convinced any Google applications immediately suck every iota of data they possibly can at install time / first launch. It’s not worth it to me either.

tpowell• 3 years ago

I enabled the requested chrome:flags in Brave, but it still doesn't work. I haven't downloaded Chrome on any of my M1 Macs, and don't plan to start now.

radiKal07• 3 years ago

same here, I use Chromium but it's the latest version and I enabled those flags... website still does not work

moffkalast2• 3 years ago

At least Firefox now allows for CSS scrollbar styling, so you have that going for you which is nice.

brucethemoose22• 3 years ago

Also already done here: https://websd.mlc.ai/#text-to-image-generation-demo

MLC uses Apache TVM to generate and autotune the webGPU code, and its respectably performant.

simonw• 3 years ago

The MLC team got that working back in March: https://github.com/mlc-ai/web-stable-diffusion

Even more impressively, they followed up with support for several Large Language Models: https://webllm.mlc.ai/

aislamov4• 3 years ago

That's really impressive and much more performant. I was following a different approach: to run any ONNX model without prior modifications.

FL33TW00D5• 3 years ago

ONNX is bloated! I got some LLMs working on my own Rust + WebGPU framework a few months ago: https://summize.fleetwood.dev/

I've since moved away from ONNX and to a more GGML style.

bb88• 3 years ago

Anyone have metrics on this on performance in webgpu vs running this natively?

aislamov4• 3 years ago

UNET takes about a 1:10 on WebGPU and around a minute on CPU in one thread. VAE is 2 minutes on CPU and about 10 seconds on GPU. It should be because most GPU ops for VAE are already implemented but for UNET are not. So in the latter case browser is just tossing data from GPU to CPU and back on each step

seeknotfind• 3 years ago

If this is fast enough, then you could use it to render images locally for personal use. Websites could deliver prompts only, perhaps rendering different images for different users. At that point, what does it mean for copyrights? Is the model itself copyrighted or does the system break down?

schoen• 3 years ago

If this is fast enough, then you could use it to render images locally for personal use. Websites could deliver prompts only, perhaps rendering different images for different users.

That's a fascinating possibility, but we're very far from that world right now: elsewhere in the thread it's mentioned that this actively uses 8 GB of RAM. And I doubt many web designers would accept the risk that a model misinterprets a prompt, produces distorted output (like the wrong number of fingers on someone's hands), or accidentally produces sexual or violent content in a context where it's not intended.

For many generative image models today, people often pick the best of a dozen or more images, and the others that they throw away may actually be quite bad.

The quality and predictability of the models would need to be significantly higher than it is now in order to routinely dynamically illustrate web sites.

But I don't want to say that we'll never get there. All of the recent models are doing things now that would have been considered inconceivable just a few years ago. (Compare https://xkcd.com/1425/ where it may even be a challenge to explain the issue behind the joke to some younger readers!)

yboris• 3 years ago

Link to repo from the FAQ at the bottom of the page:

https://github.com/dakenf/stable-diffusion-webgpu-minimal

DustinBrett• 3 years ago

I integrated MLC-AI's Stable Diffusion into my website as a custom background generator. It grabs the model data from Hugging Face.

https://dustinbrett.com/

taf2• 3 years ago

That is really impressive loaded on my iPhone 12 mini and my phone was not over heating

mNovak• 3 years ago

How smooth and fast this is just makes me sad that every app on my phone requires 200MB to show a couple views and send API calls.

stavros• 3 years ago

Exactly. We have supercomputers in our pockets, yet showing a simple todo list takes seconds. Where does all the power go?!

franzb• 3 years ago

Impressive website, everything just works super nicely and smoothly! Ended up playing Doom more than I should have :)

aussieguy1234• 3 years ago

Next up....WebGPU chatbot? ChatGPT style chat with your own GPU in the browser?

brucethemoose22• 3 years ago

Also already done... With some limitations.

LLMs are especially tricky for WebGPU because the good models are so RAM/VRAM heavy.

moffkalast2• 3 years ago

https://webllm.mlc.ai/

Already done.

baddash• 3 years ago

Out of curiosity, what are use cases/applications of this?

So what I know is that this generates images via browser rather than server. The only thing I can think of is not having to refresh the page in order to change an image or generate a new image. Which... hmm, well, that could mean websites whose visual design changes in real-time? And maybe changes in a way that would be functionally relevant/useful? That does seem pretty cool, although I'm not sure how useful Stable Diffusion is for generating UI components/visual aspects of a site.

8n4vidtmkvmk• 3 years ago

It's useful because it's expensive to generate these images en masse. It moves the compute cost to the client.

ShamelessC• 3 years ago

What? The use case is to easily run it on your own computer. Without needing to be a programmer or install an application.

asynchronous• 3 years ago

Just like what most web apps allow- quick applications in a sandboxed environment on almost every type of hardware

Eduard• 3 years ago

Privacy is a big plus for (purely) locally running models.

Sensitive prompts will not leak to some remote party.

3cats-in-a-coat5• 3 years ago

"This will load 3.5GB and use 8GB of your RAM".

Interesting what browsers have become. The web ate the operating systems.

pjmlp4• 3 years ago

That is why there is a company selling laptops where the OS is the browser, and another one doing the same for smart TVs.

solardev2• 3 years ago

Wish Windows would take a cue from that other operating system and stop shipping with so much cruft.

Can't remember the last time I used Windows for anything more than launching Chrome and Steam...

3cats-in-a-coat5• 3 years ago

There's however the important detail that this company has been doggedly working for achieving that end by first co-opting the browser of a competitor (Safari WebKit), then forking it, then taking over the web standards process, and putting in every API possible on the web, including access to USB devices and so on, so they can make an OS around it.

Because if it's the web, Google sees it. And if everything is the web, then Google sees everything.

esperent2• 3 years ago

I don't quite understand your criticism here. It's running stable diffusion on your computer via your browser. How would it do this without downloading it and then loading it into RAM?

It'll be the same size download and use about the same RAM if you download it and run it directly without using a browser.

3cats-in-a-coat5• 3 years ago

It's not a criticism. I'm just pointing it out. For good or bad, it is what it is. There are two sides to it. For anyone familiar with system theory, this was the inevitable end game of the web.

However, the web also has a terrible bloat/legacy issue it refuses to deal with. So sooner or later, a new minimal platform will grow from within it, the way the web started in the 90s as the then humble browser. And the web will be replaced.

bawolff2• 3 years ago

Its not a criticism that this is neccesary, its a criticism that it is possible to even do this in a browser.

ale42• 3 years ago

I guess data is compressed? Otherwise, this makes no sense (even if it might happen in practice).

josefx• 3 years ago

AI models can get quite memory intensive while running. I have seen a primitive image improvement AI eat up over 80 GB of RAM on high res images. The data of the model itself "only" used up 4 GB.

quickthrower23• 3 years ago

The web ate Java's original premise.

colechristensen• 3 years ago

Yup, java just didn't actually do it very well.

Or java was too heavy for the computers at the time to get people to use "applets" for everyday things (i.e. go to a new website a do a thing on it)

Flash et al also failed to catch for long.

The web browser's success might have something to do with neverending feature creep as opposed to "this can do everything but as such it's broken and vulnerable".

DrScientist4• 3 years ago

Yes on the client, no on the server.

Java's real success was ( and still is ) on the server - powering a whole generation of internet applications, and creating a cross vendor ecosystem that stopped MS leveraging it's client dominance to take over the server space as well.

I don't believe Unix/Linux would have survived the Windows server onslaught without Java on the backend and the web on the front.

massifist• 3 years ago

I don't know, something tells me a (actual) operating system will still be required (at some level).

Though, they might eat the desktop environment or UIs (in general).

jimmySixDOF• 3 years ago

Yes, and I am coming round to the idea that WebGPU is useful like this for cases other than realtime interactive WebXR pages with streaming multiplayer live state and loaded up with draw calls etc. There is a simplicity to curating the experience through the browser like this and there isn't any easier way to get SD up and running so I hope these kinds of projects keep getting support. Thanks for building this OP!

Waterluvian2• 3 years ago

As bandwidth increases and the web sandbox matures, it’s fascinating to watch the evolution towards apps you just use rather than download and install and maintain. This will bother some but for the masses it opens a lot of doors.

Timon32• 3 years ago

I love web apps because they mean I have to trust the developer a lot less than with native apps. Of course there are still things you'd have to monitor (e.g. network requests) to fully trust any web app. A good solution could be something like OpenBSD's pledge, to allow me to prove to the user nothing malicious is possible (e.g. by disabling fetch and any new requests, even from src attributes, altogether).

amelius3• 3 years ago

As long as they cache it so I only have to load it once ...

_ea1k• 3 years ago

That's what Netscape wanted, back when they still existed. The vision survived, though the company didn't.

MrBra• 3 years ago

Or better: the web ate the OS resources. Or better: an app ate the OS resources.

Which is exactly what resources are for, when eating is performed correctly :)

Kiro• 3 years ago

Having DevTools open will slow everything down to about 2x.

What causes this?

capableweb• 3 years ago

Browser dev tools (and most of them really) hook into the application flow to do additional work compared to running it without any developer tools. Depending on what the application does, it can mean that it has to do a lot of extra work and need a lot of extra memory, just to be able to process and store all the extra information that it needs.

I don't know the specifics of why the slowdown is so extreme in this case, usually it has a negligible impact. But I'm guessing it's related to what I wrote above.

chris37879• 3 years ago

Another one that slows a lot of things down is if the application uses the console. Before you open the inspector, those methods are no-ops and just get skipped, basically, but once open, all of those strings have to get copied and it can slow things down quite a bit.

This isn't unique to the web, either, adding the verbose flag to most linux file utilities and then operating on a large set of files will be slower than without the verbose flag, too, just because printing to stdout takes time.

kurishutofu• 3 years ago

Consider using a service worker for the CPU bound process so the main thread doesn't hang.

jb1991• 3 years ago

So I assume this is using WebGPU "compute" kernels, is that right?

amelius3• 3 years ago

It might as well mine some bitcoins while the GPU is not being used.

Perhaps we could have an ad-free internet after all.

k__• 3 years ago

Ads create more overhead to pay all the executives.

A miner would probably just generate enough work to pay the engineers and infrastructure.

amelius3• 3 years ago

When will WebGPU come to Firefox?

(Not going to install Chrome for a random shiny thing)

Stable Diffusion WebGPU demo

Related Stories