Show HN: Blur Webcam Background on Linux

https://github.com/jashandeep-sohi/webcam-filters

by dumdumdumdum • 5 years ago

174 48 5 years ago

Needed to blur the clutter behind me for meetings. Zoom and Google Meets don't support that on Linux. Couldn't find any existing solution that would give me a decent frame-rate for HD resolution without lagging. Wrote this instead.

Relies heavily on GStreamer under the hood for the video processing bits. And mediapipe for the selfie segmentation bit. Obviously need v4l2loopback to emulate a webcam. It's functional, but feedback and contributions are welcome :)

tvst• 5 years ago

Chrome on Linux here. Google Meet blurs my background just fine.

dumdumdumdum5• 5 years ago

Not on Firefox though for some reason. Is there some special API only Chrome implements?

nitinreddy88• 5 years ago

Thank you! I am going to try this today asap and test with Zoom. I heavily use Zoom for work and this is something lacking in Linux client

everfrustrated• 5 years ago

Zoom now supports this on Linux natively as of a new release a couple of weeks back.

captn3m04• 5 years ago

Zoom desktop on Linux added support for this recently (20th August). I tested this a bit, and it seems to work okay. Has blur support.

See https://support.zoom.us/hc/en-us/articles/205759689-Release-... and https://support.zoom.us/hc/en-us/articles/360043484511

dumdumdumdum5• 5 years ago

That's good news, but I'm kinda stuck using the web interface because the desktop client doesn't allow sharing single Wayland windows in presentation mode. Unless they've changed that too recently?

sofixa3• 5 years ago

It doesn't support blur:

The blurred background option is only available for the Windows and macOS desktop clients, as well as the Android and iOS mobile apps. Desktop clients must meet the "Image only without a physical green screen" requirements.

edoceo• 5 years ago

To add: OBS can output to v4l2loop so can feed as virtual camera.

And OBS has lots of neat features - one of which is making my fans loud. Like a crowd cheering it on!

chrismorgan• 5 years ago

Perhaps you can use OBS instead of holding the space bar: https://xkcd.com/1172/

dumdumdumdum5• 5 years ago

Yeah, this script will also heat your home :)

Especially, if you use the Zoom desktop client. Zoom on web (Firefox) seems to be easier on the CPU for some reason though.

MayeulC• 5 years ago

Any chance this could be integrated with OBS? To select filters, show previews, etc.

Going forward, I wonder if OBS will offer pipewire or gstreamer streams (if those are a thing) for consumption by other apps?

BossingAround2• 5 years ago

The last time I tried, v4l2loopback wasn't a part of my kernel, at which point I decided it wasn't worth it to tinker with as I didn't know the stability of the mod.

franga20002• 5 years ago

One thing to note about OBS, as much as I love it and use it all the time: if you're using it to record/stream just one source, you're likely wasting a lot of resources. I see many people using it to record/v4loopback their screen or camera, which OBS is not particularly efficient at. Not something you're likely to notice on a modern workstation, but on an older machine or laptop the overhead is definitely noticeable.

Dedicated recording apps for cameras and screens will give you better performance and occasionally a marginally better image since there are fewer steps in the pipeline.

For v4loopback, a short gstreamer pipeline (for those willing to use the CLI) will perform better and give you noticably lower latency - even if some basic cropping is required, like if you're using a capture card with a camera that you can't hide the UI on.

mschuster91• 5 years ago

And OBS has lots of neat features - one of which is making my fans loud.

You're likely using software encoding. My 2015 MBP has a GT750M, but "thanks" to Apple OBS can't use hardware acceleration (https://obsproject.com/forum/threads/question-about-hardware...). Situation on Linux is likely similar.

dheera• 5 years ago

If you happen to have a RealSense camera, I wrote a virtual camera for Linux that blurs your background using the actual measured depth map.

https://github.com/dheera/bokeh-camera

s0rce• 5 years ago

Cool, thanks, I look forward to trying this out. Been annoyed ever since my company moved to teams and I can't do all the cool background effects that my windows and mac coworkers have access to.

qiqitori• 5 years ago

I see it uses SelfieSegmentation and then OpenCV's BoxFilter. Do these automatically use CUDA/OpenCL when available, or does that need special setup?

Would it be possible to avoid the conversion to RGB? (This forum thread says it's CPU-only: https://forums.developer.nvidia.com/t/videoconverts-performa...)

dumdumdumdum5• 5 years ago

Nope, no GPU yet :(

Right now SelfieSegmentation is just a thin wrapper around the selfie segmentation solution provided in https://google.github.io/mediapipe. It operates on RGB frames, so that's why I need the conversion. The model inference is also done on the CPU. Interestingly, there is a GPU mediapipe graph available, but I haven't looked into what's needed to use that yet.

And yes, Boxfilter is just a wrapper around opencv's boxfilter. This is probably the lowest hanging fruit that could be moved to use GPU.

abdullahkhalids2• 5 years ago

I want to prank my students. During class, I want to play a video as my video-background, in which a pre-recorded version of me is doing things behind me.

Any ideas how I can do this on linux?

Anunayj• 5 years ago

well the easiest way I can think of is just using the OBS virtual camera thing (Unsure if they released it officially on Linux, but there is a plugin for that [1]).

Use chroma key to remove the background and add a video there, possibly on a loop.

[1]. https://github.com/CatxFish/obs-v4l2sink

rombert• 5 years ago

It is present for Linux in recent releases.

jabl• 5 years ago

I've been using https://github.com/allo-/virtual_webcam_background

There are many others similar to the above. They all basically work by using the v4l2loopback driver to create a loopback video device. Then you have the program consume video from the real webcam video device, apply a deep learning model that subtracts the background (leaving your face and torso) and replaces it with an image or video of your choice, and outputs the video to the loopback device. Then you configure your video conferencing software (zoom, teams, webex, whatever) to use the loopback device instead of the real one.

orbital-decay• 5 years ago

It's "easily" done with OBS and a green screen, however having sufficient quality for the prank to work is trickier than it sounds (matching the lighting, getting the good keying quality, avoiding the green cast, calibrating and matching the color etc).

abdullahkhalids2• 5 years ago

I am not looking for perfection. Just want to get a laugh out of them.

I also just realized that if I place my laptop camera in the exact same spot, while recording the background video and during class, all I need to do is sit on the left of the camera pane during class, and only stay on the right for the background video, then I just need to stitch the two videos side-by-side in OBS and it should work well enough.

I think I achieved my purpose here. Ask a question from a bunch of smart people. Either they give a great answer, or their response forces you to rethink and figure out a simple solution. Thank you HN for being my AI enabled rubber duck.

mgv11• 5 years ago

Thank you. Will definitely give this a shot!

noisy_boy• 5 years ago

Unable to install - raised issue: https://github.com/jashandeep-sohi/webcam-filters/issues/2

BossingAround2• 5 years ago

I wonder how is the CPU usage as opposed to GMeet with Chrome. Seems like Google Meet on Chrome doesn't rely on the v4l2loopback kernel mod.

rcarmo• 5 years ago

I quite like this as it is one of the few comprehensive GStreamer examples that is readable. Kudos.

dividedbyzero• 5 years ago

I believe the Raspberry 4B can act as a USB client device (e.g. a webcam using the Pi camera as video source); could this run on one of those? That would give you an external webcam with built-in background removal, independent of your local setup and video call app and OS.

tom_usher• 5 years ago

This looks very useful and far less hacky than my solution using OBS' Browser source and Bodypix/Tensorflow.js (https://usher.dev/posts/make-your-webcam-look-slightly-more-...).

Can't get it to work at the moment - but hopefully with a bit more experimenting, this will replace my current options.

Hadn't realised until reading this submission that Zoom has added blur support on Linux recently - on trying it out it does seem quite poor though - a very sharp edge and quite slow to update.

Brakenshire• 5 years ago

Along the same lines, an issue I was interested in a while back, has anyone written a script in Linux to get the webcam to dynamically crop to faces/people?

jensv• 5 years ago

Is there anything like this for Windows or Mac? Many third party videoconferencing solution don't offer blur if you're interviewing with an employer, using Jitsi Meet, etc.

Show HN: Blur Webcam Background on Linux

Related Stories