Show HN: Remove-bg – open-source remove background using WebGPU

by anduc • 2 years ago

286 121 2 years ago

Yesterday,I saw a post in X asking for a self-hostable background remover service. I was thinking, can we make it work by using WebGPU? So it will run in the browser and doesn't require any server/queue to run

After a couple of hours, I created this and published the source code on https://github.com/ducan-ne/remove-bg

It's still new so welcome any ideas and contributions

ericol• 2 years ago

The very first image I uploaded (A model lighthouse with a very obvious background) gives just "Error".

dindresto• 2 years ago

Does your browser support WebGPU? https://webgpu.github.io/webgpu-samples/

anduc19• 2 years ago

The error is ambiguous right now and I'll try to make it clearer (welcome for contributions) The idea is it fallbacks to not use webgpu if your browser is not supported, but it was made in 2 hours, bug is acceptable :)

fragmede• 2 years ago

I'm impressed by it working, since the olden days this was a painstaking job in photoshop, and I haven't been keeping up with SOTA, but I'm also impressed at how little code there actually is in src/ai.ts to make it happen.

Good job!

seanthemon• 2 years ago

A lot of tools exist for this, even on my macbook i can just right click, quick action and remove background.

wongarsu• 2 years ago

It's pretty much one click in photoshop now too. With better results and more control than what this tool is offering.

Not that I'm blaming this tool for being worse than a $200+/yr product. If anything it's impressive how close it gets with so little code. And if you just want rough results on a large number of files it even looks superior

anduc19• 2 years ago

Same here, this is the first time working with this library to me, it's really making me believe even more in the future of Transformer.js/WebGPU, it just beginning

ramonverse• 2 years ago

Opened, uploaded image, looked good (it worked!). Then my browser (Arc) started freezing, unfroze after closing your website :/

anduc19• 2 years ago

Sorry for the inconvenience, I've tested with Arc and can confirm this error, I didn't know why this happens. I will check further to determine the reason and report it to the upstream library (I think it's because the engine or Arc itself)

vednig• 2 years ago

Since, Arc is built over chromium and does not have active inbuilt chrome popups and error dialogues.

Most probably it because of cache utilization and network overload, can simply be solved with clearing cache and creating a service worker for managing download of model and invalidating memory.

xnx• 2 years ago

Great to have local tools. Here's another one that uses the exact same combination of technologies: https://huggingface.co/spaces/Xenova/remove-background-web (Feb 2024)

anduc19• 2 years ago

Exactly this, as mentioned in the post I've used the same technology with this playground (copied lots of code from here) What I do is mostly make the UX better

PS: WebGPU is the future

stavros• 2 years ago

Nice! This is the model it uses, for anyone curious (it's also mentioned in the description):

https://huggingface.co/briaai/RMBG-1.4

maven29• 2 years ago

How does this compare to "segment anything" from Meta

m00x• 2 years ago

Much smaller and better at background removal, but doesn't segment everything.

simjnd• 2 years ago

Very cool, but man pulling 900+ dependencies to build and run this thing feels awful. The NPM culture is out of control.

anduc19• 2 years ago

Haha, I didn't notice this! Yeah, only Vite will reduce the dependencies a lot, but I'm not sure which one is the biggest here to talk about

Will try to the deps simpler later, after all this is about a little time of work

thelastparadise• 2 years ago

Will try to the deps simpler later, after all this is about a little time of work

Word of tip, you may not want to jump on every last suggestion from the peanut gallery instantly

ashishb• 2 years ago

JS standard library is very limited. So, pulling in third party dependencies to even left pad a string is normal.

throwaway63820• 2 years ago

you mean the native `str.padStart(targetLength, padString)`

beeboobaa33• 2 years ago

Status: Error

Firefox on Linux: Error: Unsupported device: "webgpu". Should be one of: wasm.

Chromium on Linux: Error: no available backend found. ERR: [webgpu] Error: Failed to get GPU adapter. You may need to enable flag "--enable-unsafe-webgpu" if you are using Chrome.

Passing the --enable-unsafe-webgpu flag results in the same error.

cwillu• 2 years ago

“chrome://gpu/” may give more clues as to what went wrong.

cWave• 2 years ago

for Chromium, on Linux you also need to run it with --enable-features=Vulkan https://github.com/gpuweb/gpuweb/wiki/Implementation-Status#...

beeboobaa33• 2 years ago

This worked, thanks!

arendtio• 2 years ago

This!

anduc19• 2 years ago

Maybe it's because of how I detect the GPU and switch to another backend (to support devices that don't support WebGPU).

Can you try to go https://pmndrs.github.io/detect-gpu/ and pass the result here

beeboobaa33• 2 years ago

{ "fps": 60, "gpu": "amd radeon r9 200", "isMobile": false, "tier": 3, "type": "BENCHMARK" }

It's a AMD Radeon RX 6600.

It worked on Chromium after passing --enable-unsafe-webgpu --enable-features=Vulkan

slimsag2• 2 years ago

Firefox doesn't support WebGPU FYI

nadermx3• 2 years ago

This is cool. I made a background remover for image and video a few years back https://github.com/nadermx/backgroundremover/

Always happy to see other people exploring this niche

anduc19• 2 years ago

It's nice to see it here, imagine it works like your repo but runs entirely in the user's browser, very cool, doesn't need any complicated setup

nadermx3• 2 years ago

Yeah, that is awesome

afuqya• 2 years ago

@nadermx Does it work in real time for webcam? I ask that because you have an animated gif in your link.

nadermx3• 2 years ago

I haven't messed much with it, but it would be possible to probably have ffmpeg pull the stream and run it at 30 fps blocks. Can do a pull request if you want

akpa13• 2 years ago

Very cool idea, but for me it completely doesn't work. I'll open an image, the entire computer (running Linux) will briefly freeze (including stopping Spotify in its tracks) and when it comes back, the only thing that's left is a message from Firefox telling the tab has crashed.

anduc19• 2 years ago

I tried it on my Chrome-Mac device, but not sure how it works on other devices (assuming it works). I'll test it out later. Sorry for the inconvenience

akpa13• 2 years ago

Please don't apologise for any inconvenience! It's a cool project and I'd love to see it working, I'm just sorry I can't give any positive feedback

lxgr3• 2 years ago

A website being able to do this is arguably a browser bug.

If a browser's sandbox can't even protect against accidental resource exhaustion, I'd be very concerned about that as an intentional attack vector.

ipaddr• 2 years ago

What browser protects against resource exhaustion? Chrome and all variants do not. Running out of memory or cpu or even hard drive space can happen and does in all browsers.

codedokode3• 2 years ago

Browsers do not protect from tab using all available memory and swapping out other applications including desktop shell.

jml7c5• 2 years ago

Sounds a bit like running out of memory, followed by the OOM killer going after the process... though I'm not sure the OOM killer would act that quickly. Do you get any interesting messages in `dmesg`?

akpa13• 2 years ago

You're spot on - dmesg shows it's OOMing, and pretty fast too. Turns out that it happens even if I just open the web page, yet alone start trying to remove the background of an image, so I'd guess it's something to do with loading whatever model is being used.

cmgriffing• 2 years ago

Looks like WEBGPU is only on nightly for FF, for now.

codethief• 2 years ago

Ahh, for a second I thought https://remove.bg had open-sourced their product (which I've been quite happy with on the few occasions I've used it).

Very cool, though!

Flop7331• 2 years ago

Non-descript error after 180 seconds. Vivaldi on Android.

Please turn off the rolling animation for the duration timer. It looks really wrong when the numbers wind back (which they wouldn't do on a rotor) and when the trailing zeroes vanish.

tech-no-logical• 2 years ago

firefox / linux. I only get fully transparent output : https://i.imgur.com/kcu2LSR.png

nothing interesting on the console.

nabla9• 2 years ago

Same here. Plus error with Chromium.

Working with just one setup is midpoint of web development.

slimsag2• 2 years ago

Firefox doesn't support WebGPU today.

cultureulterior• 2 years ago

Nice, works perfectly on win/chrome

cutiepatootiee2• 2 years ago

Looking forward to testing this out and sharing w network if it works well

anduc19• 2 years ago

Thank you, looking forward to this, if you have any feedback please let me know via x.com/duc__an

hokkos• 2 years ago

why do you use a canary version of an old react version ?

anduc19• 2 years ago

Hi there, it's because I copied the project template from my previous project, will check and try to upgrade to the React 19 soon if it'll work with the libraries

lelandfe• 2 years ago

Thanks for sharing; it looks like this has trouble with certain kinds of images. Here's likely the most representative example:

Original: https://imgur.com/a/NrEXfua

BG removed: https://imgur.com/a/JWKHVGE

Much of the background was untouched, and almost all of the actual data (the axis and bars) were removed instead.

anduc19• 2 years ago

Sorry for my late I missed the comment

Yeah I think it's because the quanlity of the model, hopefully we will have better quanlity in near future. I will see if anything I can do with the settings

andrewstuart• 2 years ago

"Remove Background" is at Tools/Remove Background in MacOS Preview.

stephenr2• 2 years ago

Or just press and hold (iOS/iPadOS) / right click (macOS) on the subject and choose "copy subject".

nextaccountic• 2 years ago

This one is open source.

stephenr2• 2 years ago

What are you basing that assertion on? There’s no licence information provided in the repo or on the site, that I can see.

sva_2• 2 years ago

It would be cool if it could ask before loading the model, or at least indicate to me how large the download will be, as I'm on a metered connection right now.

But maybe that's just a me-problem.

zamadatix2• 2 years ago

Looks like ~4 MB, I think that's a fair size to not throw up warnings about (unless I'm missing something in the Network view of dev tools w/o cache). That said I wonder what people consider the "Click to enlarge (may take a while to load)" courtesy size to be in 2024.

lynguist• 2 years ago

I would probably consider 50 MB that size, or in the special case of metered connections 20 MB (for example downloading maps or so).

sva_2• 2 years ago

Looks like ~4 MB

You got me!

The model was 176 MB. Total pageload transferred 182 MB.

https://imgur.com/a/6xx3Lgu

It doesn't seem like "Disable cache" in the DevTools empties the Cache Storage.

Loughla• 2 years ago

After living with satellite Internet as the only option for about 15 years, now that I have fiber, I still catch myself declining downloads that are too big and opening the scheduler.

Old habits die hard.

And the modern Internet implicitly assumes the end user is not on a metered connection. Websites are fucking massive these days.

Birch-san• 2 years ago

feels like it could be nice to abide by the license terms https://bria.ai/bria-huggingface-model-license-agreement/

1.1 License. BRIA grants Customer a time-limited, non-exclusive, non-sublicensable, personal and non-transferable right and license to install, deploy and use the Foundation Model for the sole purpose of evaluating and examining the Foundation Model. The functionality of the Foundation Model is limited. Accordingly, Customer are not permitted to utilize the Foundation Model for purposes other than the testing and evaluation thereof.
1.2.Restrictions. Customer may not: 1.2.2. sell, rent, lease, sublicense, distribute or lend the Foundation Model to others, in whole or in part, or host the Foundation Model for access or use by others.
The Foundation Model made available through Hugging Face is intended for internal evaluation purposes and/or demonstration to potential customers only.

Laaas• 2 years ago

The repo doesn’t include the model.

NewJazz4• 2 years ago

Does the site not distribute it?

jfoster5• 2 years ago

A lot of these AI licenses are a lot more restrictive than old school open source licenses were.

My company runs a bunch of similar web-based services and plan to do a background remover at some stage, but as far as I know there's no current models with a sufficiently permissive license that can also feasibly download & run in browsers.

sangnoir• 2 years ago

Meta's second Segment Anything Model (SAM2) has an Apache license. It only does segmenting, and needs additional elbow grease to distill it for browsers, so it's not turnkey, but it's freely licensed.

EMIRELADERO2• 2 years ago

AI model weights are probably not even copyrightable.

thenickdude• 2 years ago

Surely they would at least be protected by Database Rights in the EU (not the US):

The TRIPS Agreement requires that copyright protection extends to databases and other compilations if they constitute intellectual creation by virtue of the selection or arrangement of their contents, even if some or all of the contents do not themselves constitute materials protected by copyright

https://en.wikipedia.org/wiki/Database_right

Mathnerd314• 2 years ago

It is a 2024 model, for comparison https://github.com/danielgatis/rembg/ uses U2-Net which is open source from 2022. There is also https://github.com/ZhengPeng7/BiRefNet (another 2024 model, also open source), it's not too late to switch.

littlestymaar• 2 years ago

Keep in mind that whether or not a model can be copyrighted at all is still an open question.

Everyone publishing AI model is actually acting as if they owned copyright over it and as such are sharing it with a license, but there's no legal basis for such claim at this point, it's all about pretending and hoping the law will be changed later on to make their claim valid.

NewJazz4• 2 years ago

Train on copyrighted material

Claim fair use

Release model

Claim copyright

Infinite copyright!

xdennis• 2 years ago

It's kind of silly to complain about not abiding by the model license when these models are trained on content not explicitly licensed for AI training.

You might say that the models were legally trained since no law mandates consent for AI training. But no law says that models are copyrightable either.

giancarlostoro• 2 years ago

At some point the worlds going to need a Richard Stallman of AI who builds up a foundation that is usable and not in the total control of major corporations. With reasonable licensing. OpenAI was supposed to fit that mold.

amelius• 2 years ago

Why can't I just "apt-get install" tools like this?

I used to have all the powerful tools at my fingertips but now I feel like my Linux distro is slowly disappearing into a vortex of irrelevance.

kinduff• 2 years ago

I wouldn't blame apt-get, there are a lot of command-line tools nowadays. I don't usually install these if its not going to be a frequent task.

Anyways, if you're looking for a cmd tool, rembg [0] is a pretty good one.

[0] https://github.com/danielgatis/rembg

akdev1l• 2 years ago

Because someone just created it so no one has packaged it yet

solardev• 2 years ago

For every Linux power user, there are probably a million regular web users who don't know (or care to know) what a command line is. Web tools are for them.

Why not fork the repo and repackage it as a CLI app if you really want it that way?

fulafel• 2 years ago

People are trying to write software for GPUs more commonly now + because of the troubled GPU programming landscape, it's hard to make programs that work more than one vendor's proprietary flavour of GPU. WebGPU is an attempt to address some of that in browser apps, but it's not ready yet.

zurtri• 2 years ago

I tried it on an image of me and a horse in front of a paddock and it worked perfectly. Really impressed.

Other background removal tools would find me - but erase half the horse or erase his tack or his ears. This tool worked perfectly.

Very well done!

mvdtnz• 2 years ago

Not sure if you realise you've given it the same name as another product that does this exact thing. (Disclosure I work for Canva, the company which owns remove.bg).

https://www.remove.bg/

anduc19• 2 years ago

Ah, I didn't realize that there is any product with a similar name, and in case I already know, I think I'll still pick this name since it's only an experimental open source project with WebGPU. Remove-bg is quite common IMO, but definitely will affect SEO if I want to do more for the project, but I have no plan yet

jacobn• 2 years ago

Nice work! Making AI accessible directly in the browser (even if you have to download quite the payload for the model parameters) is a real accessibility game changer.

Shameless plug: if you prefer API-based background removal we offer a super low cost, high quality option on https://pixian.ai

cutiepatootiee2• 2 years ago

It's not working for me

anduc19• 2 years ago

Can you provide more additional information for me? If you're using Linux and Chrome you probably need to pass GPU flag `--enable-unsafe-webgpu --enable-features=Vulkan`

kamjin2• 2 years ago

Can you take a look at this open source background removal project?

https://www.reddit.com/r/StableDiffusion/comments/1dwkwrx/i_...

It's based on python and neural network modeling, and I was wondering if it's possible to run it via webgpu? Or run it based on wasm? No offense, just because it looks awesome.

anduc19• 2 years ago

First time I see this, by the first impression I think the output quality of this one is better than mine and my code is only based on one model

In my understanding, It'll possible if the model's author build to https://onnxruntime.ai ONNX Runtime. And maybe the downside is user will need to download ton of data to their device, currently it's ~100-200mb

kamjin2• 2 years ago

I tried your site using the image: https://bannerify.co/tools/remove-bg I crashed the page after 10 or so seconds and Chrome prompted me: Crashpad_NotConnectedToHandler, it doesn't work!

anduc19• 2 years ago

can you raise this issue on Github? I will take a look into this later since it's not common error. I think it's because some settings on the browser

I've heard someone reported that Arc had a similar error

ivolimmen• 2 years ago

Well gave it 2 tries before giving up. Does not do anything in FireFox and in Chrome it also failed after loading some data. It told me to restart Chrome with some GPU unsafe flag.

anduc19• 2 years ago

Hey there, sorry for the issue, does it show status "Error"? if so can you try to enable WebGPU using this flag "--enable-unsafe-webgpu --enable-features=Vulkan", if not it doesn't seems like a common error and I will spend more time on testing later

Feel free to raise an issue on Github to keep track progress, it helps open source a lot, or you can DM on x.com/duc__an anytime

anduc19• 2 years ago

Someone mentioned > Looks like WEBGPU is only on nightly for FF, for now

Akash_deshmukh• 2 years ago

thanks ! finds useful .

codedokode3• 2 years ago

You might want to warn about memory usage. I had Chromium consume several GBs of RAM and had to terminate it.

But it is great that there now is more offline tools. It would be great if there was a browser API that allowed the page to voluntarily go offline to guarantee that no data will be leaked.

anduc19• 2 years ago

Several GBs of RAM definitely too much, I'm not aware that this consumes that much memory. I will spend more time trying to figure out how much memory is minimum necessary for this and will warn in the page.

anduc19• 2 years ago

Hi guys, thanks for your interest in my little project, I'm glad you all like it.

I just spend my free time to make it much better based on your feedback

- Mobile support, tested on my iPhone

- GitHub README added

- Added medium zoom for easier viewing on Mobile

- The error banner makes it more friendly and easy troubleshooting

- Troubleshooting section in README

If you have any idea to make the UX better, please let me know, I'll appreciate it

makifoxgirl• 2 years ago

I've tried many different background removal algorithms and found that InSPyReNet has been the most successful one. I use `transparent-background` on pip a whole bunch and it rarely fails on me compared to remove-bg and whatnot

https://github.com/plemeri/transparent-background

hauschildt• 2 years ago

Hi,

nice work. I recently also published a WebGPU version of our browser-only background removal library. It's using the onnx-runtime under the hood. Weights are from isnet. It could also run birefnet – if there is some interest – however BirefNet weights are almost 1GB in size, which is a bit much to download I guess.

There is a blog post about it and also a CPU only version available.

https://img.ly/blog/browser-background-removal-using-onnx-ru...

Source is available at: https://github.com/imgly/background-removal-js

and on npm: @imgly/background-removal

Feel free to check it out!

san_dimitri• 2 years ago

Thanks for sharing this repo. While I don't have time to actively contribute to the code, I have been testing on images to share my feedback for future devs.

1. Background removal is working good on a lot of different types of images. This includes images with background, plain or white background, men, women, children, hair, and pets.

2. After background removal, the new image is warped in some areas. For example, I have a picture of a child eating ice-cream. The background was removed perfectly but left a lot of artifacts on the child. I can share those images for testing.

Please let me know if there are other areas I can test.

ccbikai• 2 years ago

I have used this model before, running it locally with Web GPU is quite fun https://news.ycombinator.com/item?id=40715181

Show HN: Remove-bg – open-source remove background using WebGPU

Related Stories