Chrome, Firefox extension that blocks NSFW images using TensorFlow JS

https://github.com/nsfw-filter/nsfw-filter

by bishalb • 6 years ago

150 157 6 years ago

nextaccountic• 6 years ago

Does the Tensorflow model run in JS? How fast is that?

Would it be feasible to run the model in wasm or in the GPU?

rlayton22• 6 years ago

I can't speak for this specific model/extension, but "most" neural networks aren't that big at the end of the day, and you are just a few matrix dot products away from getting your classification (of course, I'm drastically simplifying).

Its the training that takes forever, due to the fact all those numbers need tweaking. However one you have the model, classifying is pretty fast.

imvetri2• 6 years ago

Are there tools to visualise trained model,?

langitbiru• 6 years ago

Yes, tensorflow model can run on JS. I did that when developing a Deep Learning product, PredictSalary (https://predictsalary.com). The problem is not the model, but the TensorflowJS library (using Node) is big (8MB) after webpack-ing it (even after using compressing flag).

So I decided to move the inference to the server.

It's fast but I didn't toy with GPU setting when running Tensorflow model on JS.

rtz122• 6 years ago

I just installed this, and my whole browser grinds to a halt when I load a page...

navendu_p4• 6 years ago

That should not happen! Could you tell us about the issue? https://github.com/nsfw-filter/nsfw-filter/issues

navendu_p4• 6 years ago

So we use MobileNet V2 model. It is relatively light weight and can run fast on the browser itself

raister• 6 years ago

My kid would just turn off the extension and keep his business going... :'(

Darmody4• 6 years ago

This is not a parental control software. It's made for you to be able to browse the web without worrying about NSFW images.

pjmlp3• 6 years ago

Luckily in Europe, that is hardly an issue as most countries are quite alright with their bodies and sexualities.

speedgoose2• 6 years ago

You can also simply not worry about NSFW images. If you don't like them, scroll more or close the webpage.

fastball• 6 years ago

When I was a kid I probably would have trained this but in reverse so that I only see NSFW images.

EForEndeavour• 6 years ago

Sigh... forks repo

jankiehodgpodge• 6 years ago

You can easily enforce installations of extensions via Chrome's admin templates.

Santosh83• 6 years ago

Anyone know of a tool which can process videos (movies) and black/blur out nude scenes and make the film "family friendly"? The processing doesn't have to be realtime...

Just to expand: there are many excellent films which are not "family friendly" only because of 1 or 2 nude scenes which aren't even germane to the overall plot in many cases. I often wished there was a tool or SaaS that could detect such scenes and cut or blur them out. Would save a lot of manual processing.

simion314• 6 years ago

Wouldn't make sense to remove the entire scene? Doing the Japanese style pixalation/censorship won't help IMO when you watch some movie with your kid and people start moaning. If the movie includes sex and nudity it probably includes some similar adult language so a tool that works in real time that only is trained on porn will be probably terrible and a waste of time though some "entrepreneur" would probably try to sell you such a tool (I mean it is just gluing shit togheter like most tools this days)

jasonlfunk2• 6 years ago

There is a company that was trying to work in this space called VidAngel. https://www.vidangel.com/ I used to use them quite a lot and then they got sued by everyone and now I’m not sure what they are up too. It’s unclear if a company has the right to offer others content in a modified form - ie filtering out unacceptable content.

My favorite thing is that they had a JarJar filter for the Star Wars prequels.

sircastor• 6 years ago

They do on the fly skipping in streaming services. Rather than reselling modified content, they’re skipping time codes.

umvi5• 6 years ago

Yeah, VidAngel is pretty much non-usable now because of the lawsuit(s)

ThinkBeat• 6 years ago

There is or used to be a start that addressed this market.

They had an ever expanding catalog of films they had processed to avoid any nudity or bad language. I think they cut those parts out entirety.

Here I found it.

They got sued out of existence apparently by the movies studios but was recreated and they now somehow do it with streaming content from Netflix or HBO.

I dont know how they do that but I figure they will be sued again.

I am not endorsing the company. I have never used it and never will but it is out there.

https://www.vidangel.com/

sneak• 6 years ago

Chopping out scenes with swearing and sex and violence won’t make an adult story suitable for children.

There’s a whole genre of family friendly films, those will serve you better than trying to hack up stories for full adults.

louisgeo• 6 years ago

SaaS tool that does that: https://sightengine.com

Will flag sections of the video with explicit / suggestive / partial nudity

speedgoose2• 6 years ago

Is there a need? I feel like if you are old enough to watch a movie for adults, you are more than old enough to know about sex. But anyway, when was the last time you saw an erect penis in a movie you wanted to watch in a family setting?

mkoubaa• 6 years ago

If people want to use it then there is a need. You can't just tell people to stop having sensibilities

petespeed• 6 years ago

For non-stream media (iso,dvds), skip files can work. These files allow you to define frames or seconds to skip or mute. For example mplayer has edl (edit decision list). http://www.mplayerhq.hu/DOCS/HTML/en/edl.html

Crowdsourcing such metadata would be a good start, although it can be subjective.

If you find out a better way for streaming media, please share.

tyingq• 6 years ago

CleanFlix[1] did something like that, but was shut down by movie studios.

[1] https://en.m.wikipedia.org/wiki/CleanFlicks

Vid Angel took a different approach, allowing you to fast forward over parts, and appears to still be operating: https://www.vidangel.com/

BagPiper5000• 6 years ago

I'm going to be a little hypocritical here since I hate when I ask for help on the internet and people tell me instead to not want to do that thing... but:

Surely a much easier thing to do would be to simply stop being offended by the human body? Just watch the movies anyway. Your kids won't explode if they see a boob.

This reminds me of when I went thru all different low calorie sweeteners to put in coffee until I realized this is a problem I created myself and coffee is actually better if I just get used to having it plain.

umvi5• 6 years ago

You could use ffmpeg to delete the scenes containing violence/sex, but you'll have to manually find the timestamps yourself.

monadic22• 6 years ago

Or even better, ads! Just show a nice landscape instead.

capableweb5• 6 years ago

If you want to make videos family friendly, it's probably better to aim for removing violence than nudity, as one is clearly worse for the psych than the other.

Expansion: how are nude scenes not family friendly? Not trying to start any flame wars here, just trying to understand how one or two scenes with nudity suddenly means the video is not family friendly? We all go through life seeing nudity, so should be fine. Violence however is something we both strive and should avoid as much as possible.

skrause• 6 years ago

Expansion: how are nude scenes not family friendly?

The answer is "American Prudery".

pledg3• 6 years ago

See also American TV where you can show pretty extreme violence but not swear.

chance_state• 6 years ago

No, not really. Most people don't want to watch graphic sex scenes with their parents. It's not a "muh America" issue.

pmachinery4• 6 years ago

Surely it's not that hard to understand that for religious, cultural or other reasons some people and/or members of their family may feel uncomfortable seeing nudity.

izacus• 6 years ago

It doesn't make that viewpoint any less bizarre when our media is filled with people murdering each other on pretty much hourly basis. And it doesn't make a viewpoint that allows killing over depicting our bodies any less worth challenging.

capableweb5• 6 years ago

Yeah, as far as I know, no religions or cultures forbid people to be naked or see other naked people. What other reasons are you referring to here?

It is indeed hard for me to understand how someone can feel uncomfortable seeing nudity, as it's everywhere in life and always has been.

terhechte• 6 years ago

What Kind of Religion would find the depiction of violence and death fine, and the creation of life / nudity “uncomfortable“?

EugeneOZ• 6 years ago

These people need to follow rest of civilization and educate themselves.

shubb• 6 years ago

That's starting from the assumption you are doing this to engineer your children ands psychology.

Actually I think a lot of people find it awkward to watch graphic sex scenes and extended nudity with their family. As a grown up, the I feel awkward watching parts of game of thrones with my parents.

That's a shame because I would like to watch this show with them. Lately a lot of major TV shows seem to aim to include one nude scene per episode - I guess the money people think that's what sells...

I can understand why some people might find violence equally difficult, but where it features it tends to be much more tied to the plot.

schrijver• 6 years ago

Right, but doesn’t this awkwardness come from you growing up in a culture (I assume U.S.) where depictions of nudity are taboo?

If this taboo wouldn’t have existed for you as a kid I’m not so sure the awkwardness would exist for you as an adult. As a sample size of 1, coming from a country where there is much less of a taboo I feel very differently on this topic.

null_deref2• 6 years ago

Why not both? You could also argue that repeatedly seeing the most beautiful actors in the business nude, can hurt a teenager self image pretty badly, or that learning about sex from movies is probably not the best idea.

sjwright• 6 years ago

IMHO it’s not sex that’s the real problem for teenagers, but rather extreme sexualisation. And that doesn’t require nudity.

goda90• 6 years ago

Lots of nudity in film is of a sexual nature. Uncontrolled exposure to fictional depictions of sex could create unhealthy understanding of sex in children that parents want to avoid. Hence not family friendly. I'm not saying most in film violence is either, but that's the reason.

One film I think of in this regard is Schindler's List. I've seen both the original and the TV edit that Spielberg made to broadcast for teenagers to see. The primary edits I noticed were sex scenes and post-sex nudity. Those scenes weren't gratuitous(they showed something about the characters) or anything, but were edited(either cut or blacked out part of the frame). BUT there was still lots of nudity in the edited film. This was nudity of a different nature. People forced to strip down and paraded in the open for inspection.

Sometimes parents just want to take the easy path and consider all nudity as out of bounds for their kids. I think that's fine.

heavenlyblue• 6 years ago

Sometimes parents just want to take the easy path and consider all nudity as out of bounds for their kids.

Sometimes parents consider vaccinations to be out of bounds for their kids, too.

dirtnugget• 6 years ago

I would generally put Schindlers List onto the "not-so-family-friendly"-list. It’s educational I guess but I wouldn’t want to show it to anyone below 16. And after that the nudity should be fine.

umvi5• 6 years ago

Because it's awkward getting sexually aroused in front of family members?

dang• 6 years ago

We detached this subthread from https://news.ycombinator.com/item?id=24251297.

pledg3• 6 years ago

This would be too late in my workplace, as they'd be logged that I downloaded the images. That I didn't see them in my browser would be irrelevant.

capableweb5• 6 years ago

Yeah, fixing that would require breaking "The extension runs completely on your browser. i.e No user data is being sent to a server for processing." as image requests would have to be hashed, checked on a server and only allowed depending on the response.

So seems this extension is not really for your use case but for others.

gvjddbnvdrbv2• 6 years ago

I'm curious how workplaces do this. Does they have a root cert installed so they can MITM SSL traffic?

pledg3• 6 years ago

Basically yes.

amelius3• 6 years ago

You could still run your own layer of encryption through that pipe though.

dec0dedab0de3• 6 years ago

Every place I ever worked did exactly this. They use a protocol called WCCP which is essentially source routing, so if you're going to the internet on certain ports it routes you to a proxy server instead of whichever router it normally would.

Most companies big enough to do this already have their own internal CA installed on all the machines, for internal sites, so they use that same CA to sign the mitm cert. With so many sites using HSTS it can be annoying if you access a site while off the network.

As far as them knowing the content of a particular image they would need to have some kind of machine learning like this extension.

jwalton• 6 years ago

Yes exactly. And, ironically, many of these solutions can make you less safe; at one of my former employers we had something like this, but the problem is that since you're getting a cert from the MITM server, you're not able to inspect the cert from the real server, and at least in the case of the Cisco product we were using, the MITM server wouldn't bother to inspect it either; expired certs, certs with the wrong CN, self signed cert, didn't matter - the MITM server would ignore the problem and happily replace the cert with a valid one signed by the company CA.

treis• 6 years ago

I don't think it's irrelevant. Having a NSFW pop up in the middle of a meeting and your access logged is much worse than just your access being logged.

jijji• 6 years ago

true, the client still downloads the image so you'd still violate a work related policy... This plugin would have to operate as a proxy and filter the actual images that are NSFW so your client never downloads the actual image.

rtz122• 6 years ago

It would be a nice feature for this extention to collect the URLs of detected porn images and report their hashes back, so they could be integrated into the extention itself.

ndm0002• 6 years ago

Shameless plug - I've been working on something similiar, but with a different aim.

https://github.com/nmurray1984/porn-blocker-chrome-extension

I found that the hardest problem to solve was the prevanlence of false positives. Even if you have a low false positive rate, it's still very likely to have an image blocked regularly just due to volume.

For that reason, the focus of my plugin is - for users that are OK with it - contributing URLs that are not NSFW but have been blocked. The extension includes a right-click menu option to do so.

antpls• 6 years ago

Assuming you meant some SFW images are mistakenly blocked, it is an easier situation than with true negative.

Once an image tested positive with your first model, you could run a second more CPU intensive model (but also more accurate) on it.

wongarsu• 6 years ago

Assuming you have a notably more accurate algorithm. It's a difficult problem even with infinite computing power.

ndm0002• 6 years ago

Great idea. MobileNetv2 is the model used currently due to it's small footprint (I used transfer learning on a dataset of NSFW and SFW images). A more robust model ran on all positives could improve performance overal.

kmfrk• 6 years ago

This is great, except it'd be really, really great with some sort of placeholder image for the blocked images if possible.

navendu_p4• 6 years ago

We had that in the first version but we kind of removed it. We will try to add one option for that.

aasasd• 6 years ago

I happen to not be afraid of boobs, so ever since my days on Reddit I wish for blocking of random gore instead. This is weirdly pertinent sometimes on DDG's image search.

Edit: to clarify, I'm not afraid of some killing either, thanks to the pop culture of the past seventy years or so. Now, why eye-hurting images of bodily damage pop up on rather innocent searches—that's a haunting mystery. On Reddit, the ‘NSFW’ label is used equally for a vaguely sexually suggestive shape or a close-up more suitable for a surgical journal. As if I didn't get plenty of suggestiveness just from music videos anyway! So my long-standing wish was for an ‘NSFL’ filter instead.

iso947• 6 years ago

For some reason in America half a butt cheek is worthy of a 15+ rating in films, but “general release” films are fine with burnt bodies hanging from trees

liability2• 6 years ago

America doesn't have a '15+' rating, lots of 'PG' movies have butts in them, and what 'G' rated movie has burnt corpses hanging from trees??

azalemeth• 6 years ago

I have never understood how acts of love (at best) or reproduction (at the least) are considered in some societies to be dangerous, shocking, and in need of the strictest censorship -- or at the very least, not on prime-time TV. Yet acts of war, killing, maiming, and violence in general are often the mainstay of entertainment.

Sex is part of human nature – and, I would argue, a much bigger and better part of it than violence and aggression. Why do we hide from it?

Flimm• 6 years ago

If porn needs to be filtered then it needs to be filtered regardless of attitudes towards of depictions of violence, and if it doesn't, it doesn't. We shouldn't choose as a baseline depictions of violence.

Many people say they are negatively affected by porn addiction (whether consumers, or partners of consumers, or parents, or people who feel objectified, even some people in the industry), and they feel strongly about it. Just because sex is a good part of human nature doesn't invalidate that in their eyes. In fact, it is precisely because some consider sex to be good and precious that they want to be much more careful about viewing it or depicting it, and I agree with that.

The number of societies that don't want censored sexuality is near zero. It is not just "some societies". And there are plenty of societies that desire that more than western ones.

jpxw• 6 years ago

People want to avoid the porn industry because it’s sleazy, predatory, and reduces sex to a commodity.

umvi5• 6 years ago

Sex is complicated. Sex is not always a beautiful expression of love between committed partners. Sometimes it's a casual pursuit of pleasure. Sometimes it's a whimsical act of exploration.

In many cases sex is the direct cause of much sorrow, misery, and emotional damage. Maybe your partner cheated on you. Maybe you got pregnant and your partner left. Maybe alcohol was involved. Maybe you are lonely and got addicted to pornography. Maybe you didn't use contraceptives and are now seeking (or went and got) an abortion. If you look around in poor communities, you'll find a big source of poverty is... unprotected sex. It turns out, you can pretty much ruin your life as a teenager and guarantee you'll stay in poverty your whole life if you get pregnant and have a baby while you are in middle school/high school.

Unlike war and murder, sex is something most people will participate in (in some form or another). So I would argue that it's more important than ever to consider the types of sexual messages we send in media. Do we want to promote sexual relationships between committed partners or do we want to promote wanton promiscuity? I would argue that whichever you choose will have an impact on some non-negligible % of the future choices of the viewers. To be clear, I think we also need to be careful of depictions of extreme gore/violence as well, but for slightly different reasons.

In general, I'm of the opinion that implicit > explicit for both sex and violence in media, but perhaps there is a time/place for being explicit (i.e. so people understand what really happened during the Holocaust, etc).

zmix• 6 years ago

I'd rather have every news about the Trumpeteer blocked.

liability2• 6 years ago

All the images remain hidden until they are found to be NSFW or not

The demo screenshot/page seems to only show NSFW images, so the demo doesn't convey how quickly this classifier can operate. The demo page should have some non-NSFW images to show off how quickly SFW images are revealed, since the default is to block all images.

navendu_p4• 6 years ago

We will change that. Seems like a good idea

m3kw9• 6 years ago

From experience 99.9% of the time unless I visit a porn site purposely or some warez site by accident, or questionable reddit channels, I see no NSFW. However I do see some benefits if this was for kids.

yonixw• 6 years ago

Sound like you have ad-block or avoid sketchy (pirate) websites.

esperent• 6 years ago

Adblock is a far simpler and probably more effective solution than this, with a host of other benefits such as speeding up websites, whereass presumably this extension will slow down the browsing experience quite a bit. Given that, the only reason I can see for using this is if adblock was not doing a good enough job alone.

Wowfunhappy• 6 years ago

Preventing the last 0.1% could be meaningful though, depending on your environment.

codetrotter• 6 years ago

A few years ago I was at work and was looking up something and the page I landed on had ads that featured a barely dressed lady in them. The funny thing is that I didn’t even notice the ad until one of my coworkers, who was passing by my desk stopped, chuckled and asked me if I was looking at ladies on the computer during working hours lol.

I guess I had developed a kind of builtin ad filter into my brain, from ignoring so many ads over the years.

Thankfully, my place of work had reasonable people working there. But I think if I worked somewhere more strict even that one ad might have prompted a conversation with HR. Nonetheless, I promptly installed an ad-blocker on the browser in case any ads would be shown in the future that would have been more explicit than the one that had been showing this time.

refresher• 6 years ago

I’ve jokingly / seriously been waiting for things like this. I’m the type to go overboard with filtering and mute lists on twitter, but images are harder to deal with. I did have a 60,000 line pastebin that was used in conjunction with a Chrome extension to block Wojack and Pepe memes on 4chan using md5 hashes, but something not rely on specific hashes is obviously superior.

One day someone will release the ‘detect and block anything resembling kpop’ extension for Twitter and I’ll be happy.

solarkraft2• 6 years ago

block Wojack and Pepe memes on 4chan

What are you there for, then?

nabaraz• 6 years ago

to collect dataset?

justanotherhn• 6 years ago

I expressed some interest at some point of hiding profiles which contained a certain phrase in their bio and got downvoted. Does anyone know of any extentions that can do this?

justnotworthit• 6 years ago

I'd love a youtube comment filter that instantly removes anyone with a meme avatar or famous world leader avatar. I can skip the opinions of Hitler, pepe and JC Denton.

sergiotapia2• 6 years ago

I would use this for sure. Certain phrases are great flags for me to know "I won't be interested in anything this person has to say."

danielscrubs• 6 years ago

I want the same for ‘emotional hooks by PR’. It works so well on the internet that we get flooded with it.

Fiveplus• 6 years ago

One day someone will release the ‘detect and block anything resembling kpop’ extension for Twitter and I’ll be happy.

Not only kpop, but entire "fandoms" as a whole. When that software finally arrives, I want to be able to refer back to this comment and say I read it here first.

chownie• 6 years ago

I am very interested in this wojak/pepe filtering thing, I think it's finally started to decrease but there was a solid 5 year gap where you really couldn't escape those dumb pictures

I started doing the same with md5 hash filtering but it was very slow going, an AI that can detect low quality posts and filter them out is the holy grail I guess

umvi5• 6 years ago

Now just make a version that whites-out NSFW text in e-books and market it to Utahns as a way to read PG-13 versions of Game of Thrones books and you'll be rich.

However, you'll probably also be sued into oblivion by angry authors/publishers who don't like people modifying what they read in any way.

KMnO4• 6 years ago

I know that’s tongue in cheek, but I’d really appreciate having a NSFW text filter.

Some of my CX team is getting harassed by trolls who send profane messages and racial slurs through the Zendesk Chat interface. Zendesk does not have a way to filter out profanity (for Chat at least; there are options for emails or tickets).

I’ve found simple client side extensions that can censor words, but a better approach would use NLP to grasp the context.

“I hope you die” contains the same words as “My <product> got wet, I hope it didn’t die”, but with vastly different intention.

dexen• 6 years ago

Google Jigsaw might have what you are looking for, under the "Harassment" headline [1]. Note however both Google (Alphabet, really) and the tool specifically, were accused of various forms of bias [2] by various sides of discourse.

--

[1] https://jigsaw.google.com/issues/

[2] for example https://boingboing.net/2019/08/14/white-aligned-english.html

gftsantana• 6 years ago

I've been wanting for something like this ever since we by accident looked at our 9 year old daughter's screen. She likes to draw characters from her favorite cartoon and searches for images on the web. Apparently there's a porn actress with the same name as one of the characters... DDG was set to "strict", so no full nudity or any explicit act, but still very NSFW.

moooo992• 6 years ago

DDG is kind of fascinating. Ever since I switched to DDG as my main search engine, I feel like the internet is 90% porn. Although that's probably not far off, I'd prefer seeing relevant search results instead.

detaro2• 6 years ago

Far from that bad IMHO, but it has a weird tendency to surface NSFW results, yes.

dwighttk• 6 years ago

yeah, I think that is because Bing provides (some/a lot/all?) of their image results and Bing seems to have tried to maximize their porn results.

Both of those ideas are heresay/rumor.

solarkraft2• 6 years ago

Why would I care about not seeing NSFW images?

What I really do care about is not seeing worthless clickbait. That makes me endlessly angry. Could you build a filter like that on top of this?

cheeze• 6 years ago

Uh, because I'm at work and wouldn't mind an additional layer of protection.

Because I live stream and people try to trick me into viewing NSFW content on stream.

Is this really a question?

yetanothermonk• 6 years ago

Hey, Long here, I meant to keep this private for a while, but I’m building this thing. getpuritan dot com for now. My prototype is a chrome extension that blocks clickbait using a ML model that placed top 3 in a clickbait prediction competition, but it’s really rough. My email is longintuition at protonmail if you’d like to help me test—can also send a video so you can see how it works

goto11• 6 years ago

Why would I care about not seeing NSFW images?

The issue with NSFW images is not you seeing it on your own computer, it is other people seeing it on your computer (e.g. while walking by or for some other reason).

zxcvbn4038• 6 years ago

Missed opportunity - where is the browser extension that makes all of the images NSFW?

You leave your laptop unlocked and suddenly it looks like the nude bomb went off (https://m.imdb.com/title/tt0081249/).

andrekorol• 6 years ago

This extension reminded me of that Black Mirror episode where a mother filtered what her daughter could see and hear.

Great work with TensorFlow by the way, can't wait to see this technology maturing over the years.

bullman• 6 years ago

disappointed this is not called "hotdog / not hotdog"

amelius3• 6 years ago

I want this but for ads.

ihuman• 6 years ago

Adblock extensions already exist, like uBlock origin

amelius3• 6 years ago

Yes, but do they treat the content as an image? Or do they merely hope that the content is identifiable as an ad through other means?

e12e• 6 years ago

I was qurious about the classifier, and as expected it appears to be based on crawling to get the unsafe images - I doubt it's quite in compliance with neither copyright or gdpr..

The classifier lib:

https://github.com/alex000kim/nsfw_data_scraper

Announcement points to this data source (while the readme hints at a "premium" classifier):

https://github.com/infinitered/nsfwjs

olliej• 6 years ago

Has anyone confirmed it isnt tagging non-pornographic images? Breastfeeding, breast exam tutorials, sand dunes, etc

Has anyone tested to see if it correctly handles PoC? That demo site it points to almost exclusively light skinned folk.

These are the basic things I’d want confirmation of.