Fun with Snapchat's Gender Swapping Filter

https://blog.evjang.com/2019/05/fun-with-snapchats-gender-swapping.html

by homarp • 7 years ago

290 82 7 years ago

blog.evjang.com

joshypants• 7 years ago

You can follow artists like Mario Klingemann to see people experimenting on the cutting edge of this stuff:

https://twitter.com/quasimondo/status/1127906469108756480

Raphmedia10• 7 years ago

I have a friend who does drag. It's a lot of work! I'm excited for technology like this, because it will make it easier for makeup & drag artists to experiment with new ideas and identities cheaply and quickly.

Similarly, I'm transgender and use those applications to come up with new ideas on how to make little changes to my appearance to feminize my features. Sometimes coming up with makeup ideas is hard since I lack in experience and equipment. I don't want to purchase a bunch products I'll end up never using. Technologies like this allows me to quickly test makeup ideas. For example, I have green eyes so they automatically add purple eye-shadow to make my eyes look brighter. I've since incorporated this color in my daily makeup. I've noticed that they overdraw the lipstick to make the lips look fuller, so I also do that when I want a more polished look.

However, this is a double edged sword.

Those filters raise expectation on what you might look like after a gender transition.

I've seen a lot of angry and disappointed people because of that. Those applications change a lot about your face, sometimes in subtle ways you won't notice. The curve of the jaw is slightly different, the eyes are a bit farther apart, your skull is more round, etc. Changes that are pretty much impossible even with facial feminization surgeries.

People strive to reach that "perfection" and end up being stuck in limbo, unable to reach it so unable to free themselves from their gender dysphoria. It's easy to spiral down from there.

Hendrikto• 7 years ago

I've noticed that they overdraw the lipstick to make the lips look fuller, so I also do that when I want a more polished look.

Does that not look obvious from close up?

Raphmedia10• 7 years ago

Not if done correctly. This is simply one of the methods of applying lipstick. See: https://imgur.com/a/f8GE3iE

This also serve as an example of what I was saying above. I would have never thought to learn that skill if not for those apps that applied it to me. It's like having your own personal AI makeup assistant.

daenz3• 7 years ago

Similarly, I'm transgender and use those applications to come up with new ideas on how to make little changes to my appearance to feminize my features.

Do you ever feel like these applications are re-enforcing physical gender stereotypes? Like "you selected female, here is what females should look like." If gender is a social construct, tech like this is clearly maintaining that construct, yet nobody seems to have a problem with it.

simonh2• 7 years ago

Gender could be a benign or even beneficial social construct, if that’s what it is, it doesn’t necessarily mean it’s a bad thing.

Raphmedia10• 7 years ago

Do you ever feel like these applications are re-enforcing physical gender stereotypes?

They do. Even people that are not transgender might get stuck trying to reach a goal that is impossible. What I have is "Gender Dysphoria" because it is strongly linked to my gender identity. However, nobody is safe from "Body Dysmorphic Disorder". There has been a increase in people who go see plastic chirurgeons with such filters as reference images[1].

In my case, that's a wanted effect. By sticking to society's gender stereotypes I increase my ability to pass unnoticed. It is fine for women to not follow those stereotypes. There are strong women, women who like to dress in male clothing, women who wear punk haircuts, sit with their legs spread out, etc. The thing is, those habits often cause people to take notice. In their case, people would look and think "Oh, this woman is punk. I [love/hate] how it looks!". In my case, that's often "Oh god, there is a man in the girl's bathroom. Let's call security!".

I am aware of the stereotypes and hide behind them. That's not to say I never break them. In my daily life, I try to blend in. If I go to a bar and want to stand out? There's nothing stopping me from bending those rules. For example, my large shoulders that are a pain in my day to day life look amazing in studded leather coat.

[1] "Please Don’t Ask a Plastic Surgeon to Make You Look Like a Snapchat Filter" - http://fortune.com/2018/08/03/please-dont-ask-a-plastic-surg...

derefr2• 7 years ago

There's nothing inherently wrong with maintaining the construct of gender, if it has been defanged to do no harm to anybody.

If "gender" just means that you put on a dress and makeup and now you're "a girl", or you put on a suit and cut your hair and now you're "a dude", then "gender" isn't really anything to be afraid of, because anyone can put on or take off a gender any time they like, and it's no longer an inherent property of a person. It's just a word describing voluntary roles that anyone can assume at any time—archetypal character profiles that one can choose to "wear" to indicate that people should identify them with that archetype (like wearing a mask in an opera; or "disneybounding.")

We're not there yet, mind you, but that's the direction the LGBTQ movement has been trying to push things for decades now: making it less about "am I female?" and more about e.g. "am I a dainty princess?" where anyone who wants to play the role of "a dainty princess" can do so; and anyone who wants to opt out of that role can do that, too, and nobody in particular is expected to take that role upon themselves.

But if-and-when we have all that, then there would be nothing wrong with wanting to conform to the stereotype of "a dainty princess", or to any other gender-based stereotype; and there wouldn't even be anything wrong with gatekeeping such roles, insisting that "a dainty princess" has a particular look and behavior, being picky about who does or doesn't fit the mold of "a dainty princess", etc. If nobody is already slotted into any gender by default, then "gender" becomes a matter of finding the gender that perfectly suits you; rather than something where you're stuck with a gender whether it suits you or not, and so have to fight for the mind-share required to deform your gender's definition to ensure it includes you.

zamalek• 7 years ago

Isn't there also good to be had here, because it could normalize the concept of transgenderism, drag, etc.?

Raphmedia10• 7 years ago

I've met quite a few people who told me that they questioned their gender identity after playing with such a filter.

That's a good thing, it allows people with an otherwise strong sense of their own gender identity to have a small glimpse into what a gender identity disorder could be like. For a cis person, this happens when an AI morph their features into the opposite gender. For a person with a gender identity disorder, this happen with regular mirrors. It's two very different experience, but the sour taste in your mouth is similar.

It is easier to have empathy for something you have experienced the tip of. Similar to how someone with a broken heart will be able to empathize with the struggles of someone with clinical depression.

peteretep• 7 years ago

I'm transgender and use those applications to come up with new ideas on how to make little changes to my appearance to feminize my features

I've been recommending FaceApp to people who want to buy new glasses or sunglasses for a while. It seems pretty stable.

sirsuki• 7 years ago

Are there other online Machine Learning apps out there for this? Or is Snap-Chat the only one?

I remember back in the VRML/SecondLife heyday where you could pick an avatar that was another gender. I still do that. But the emphasis for voice chat in modern games and MMORPGs make it far more difficult. A low gruff voice does not bode well coming from a feminine avatar. Catches too many off guard.

Raphmedia10• 7 years ago

Are there other online Machine Learning apps out there for this?

Faceapp comes to mind[1]. There's also Artistry Virtual Beauty[2] but it is a bit less advanced. I believe that Facebook Messenger also have some similar features.

A low gruff voice does not bode well coming from a feminine avatar. Catches too many off guard.

A lot of sound cards those days come with tools that allow you to modify your audio input. Often, they'll have a female settings where it changes the pitch to make your voice sound more like a woman's.

[1] https://play.google.com/store/apps/details?id=io.faceapp&hl=...

[2] https://play.google.com/store/apps/details?id=com.perfectcor...

pavel_lishin• 7 years ago

There's software out there to dynamically change your voice on the fly.

mcintyre1994• 7 years ago

Snapchat have filters that do that too.

whuffman2• 7 years ago

My company, Modulate, is building "voice skins" for exactly this purpose: customizing your voice in chat for online videogames!

It's a really interesting technical challenge. On the one hand, changing your timbre to another human voice is much more complicated than basic pitch shifting, so we ended up using deep neural networks for a kind of simultaneous speech-recognition and speech-synthesis approach (though training this system to preserve e.g. the emotional complexity of your input speech while still changing out the timbre convincingly is difficult - we use adversarial training on the raw audio waveform, which is powerful but also pretty much unknown territory compared to images).

On the other hand, it's important to run with very low latency on your device while you're playing, which means that we can't simply "throw the biggest network we can" at the problem. So we have a tradeoff between model latency, which is easy to characterize, and audio quality / voice skin plausibility, which is pretty ambiguous and subjective.

Finally, as this kind of tech improves the potential for misuse becomes an important problem, so we need to build in protections (like watermarking the audio) that can help prevent fraud while not compromising the speed of the algorithm or the quality of the output audio.

Wowfunhappy• 7 years ago

Finally, as this kind of tech improves the potential for misuse becomes an important problem, so we need to build in protections (like watermarking the audio) that can help prevent fraud while not compromising the speed of the algorithm or the quality of the output audio.

Legitamate question: (absolutely not rhetorical, I don't know the answer!)

What the potential misuse of a gender voice changer? If someone wants to change their perceived gender in a video game (presumably to avoid harassment), should that be detectable? If both genders should ideally receive identical treatment anyway, is there any harm in swapping?

hooloovoo_zoo• 7 years ago

Interesting approach; is your system low enough latency to avoid the delayed audio feedback problem?

ve55• 7 years ago

I appreciate seeing posts like this as someone who doesn't use any of these apps themselves. It's intriguing what we're already able to accomplish in this area with sufficient funding and data, it makes the future seem very interesting.

giggles_giggles• 7 years ago

Physical gender stereotypes? The differences in physical appearance, like the shape of the bones in your face and the fact some people have penises and other people have vaginas, caused by activation of the SRY gene (the sex determinism gene), are not social constructs.

daenz3• 7 years ago

Physical gender stereotypes are things like hair length/style, whether makeup is applied, eyebrow grooming etc--all things that these apps are doing. These are not determined by sex. They are physical gender stereotypes, and are social constructs. Apps like these are upholding those stereotypes.

I'll put it another way. If apps like these were framed with a right-wing narrative of "this is what men should look like" and "this is what women should look like," people would be furious. But that's exactly what these apps are implying right now yet I don't see anyone speaking out against them.

tobr• 7 years ago

The makeup and haircut applied by the filter are social constructs, though.

dang• 7 years ago

"Eschew flamebait. Don't introduce flamewar topics unless you have something genuinely new to say. Avoid unrelated controversies and generic tangents."

https://news.ycombinator.com/newsguidelines.html

We detached this subthread from https://news.ycombinator.com/item?id=19900606 and marked it off-topic.

the_watcher2• 7 years ago

I don't usually pay attention to things like this, but I have a younger sister who uses Snapchat. She used this filter and the output was essentially a perfect photo of our brother, but with longer hair. I couldn't help but be impressed.

ljcn• 7 years ago

What about the brother -> sister mapping?

davemp• 7 years ago

Judging from my family's group chat that also works really well.

the_watcher2• 7 years ago

Wasn't anywhere close.

wtdata• 7 years ago

It's fantastic to see how far this tech came in such a short time.

It's also scary, because soon it will be very dificult to trust anything you see in video. What now demands a big CGI budget, will be possible to do with a simple App.

kleer001• 7 years ago

There's still tells. 80% realism is still far enough away to be unrealistic. There's lots of places it breaks. I think soon is still at least 20 years away. And 20 years away in technology is basically never. Reality is very very detailed and we've been optimized in our evolution to get to that fine grit quickly.

It doesn't entirely apply, but here's an article (one of many) on telling deep fakes: https://www.howtogeek.com/341469/how-to-spot-a-deep-fake-fac...

Also this short one on playing with a robo-call: http://www.abovetopsecret.com/forum/thread988484/pg1

chipperyman5733• 7 years ago

I would argue it's a good thing. This was already possible for people with a lot of money, but nobody knew about it because it was so rarely done. When anyone can do it, these fake videos will become common and people will realize you can't trust it just because it's in a video. The only drawback is that there will be an awkward period where smart people can do it but most people can't, so nobody realizes what's going on.

Insanity• 7 years ago

That is just crazy impressive to me. I'm not a fan of those kind of apps myself but it's pretty cool what they're engineers get to spend time on.

throw03172019• 7 years ago

Did the post get deleted? Getting a 404.

redfern314• 7 years ago

Looks like it.

https://web.archive.org/web/20190513170015/https://blog.evja...

xrd• 7 years ago

I wonder if Snap offered him a job to take it down.

dr_zoidberg• 7 years ago

It's a shame it got deleted. From the web archive, it looks like an interesting, in-depth exploration of the filter.

mromanuk• 7 years ago

it's down for me.

ericjang• 7 years ago

My bad, I accidentally unpublished it with my butter fingers.

codedokode• 7 years ago

What an awesome thing. This is probably a rare example of using technologies like machine learning and face recognition for good.

Also, it means that we cannot trust photos anymore.

politician• 7 years ago

What an awesome thing. This is probably a rare example of using technologies like machine learning and face recognition for good. Also, it means that we cannot trust photos anymore.

You can't trust any social media anymore.

Between Lyrebird, GPT-2, Deep Fakes, Sybil attacks on social media sites, and abusing Facebook/Google targeted advertising to leak targeting information on anyone, any sufficiently motivated individual, group or nation state has enough power to locate audiences and reorient them through tailor-made content to the attacker's objectives.

Today's cat-picture-viewing joke-seeking social media user faces adversaries who can deploy coordinated attacks from multiple points-of-origin (via Sybil clones) using multiple mediums (via audio, video, text generation) against specific targets without collateral damage to non-targeted audiences (target discovery via surveillance platforms like Facebook, Google). The information weapons deployed can change minds, destroy relationships with family members, undermine friendships, cause confusion, addict, and waste time that could be spent organizing against the attacker's policies.

In your profile, you state that you're a Russian developer. Consider that it's known that the Russian government interfered with American elections using these divide-and-conquer techniques. Consider that we have to question whether or not your profile is a Sybil account designed to slightly shift the perspective of HN readers towards the idea that Russians are not undermining elections by presenting the idea that educated Russians are naive to the threat. Likewise, consider that this response might be a cointel operation. One can _hope_ that you're genuine, but that's about it.

Sadly, the untrustworthiness goes far beyond photos.

https://www.usatoday.com/story/tech/2019/05/13/deepfakes-why...

stedaniels• 7 years ago

Mirror: https://web.archive.org/web/20190513170015/https://blog.evja...

alpb• 7 years ago

I really wonder why he ended up deleting the post.

Edit: It's back up again.

ahofmann• 7 years ago

Strange, I can still call this up at the original address at the moment. Just in case: http://archive.is/lTgGj

aiCeivi9• 7 years ago

I looks like what you could see in "A Scanner Darkly" https://www.imdb.com/title/tt0405296/

Bobbleoxs• 7 years ago

You read my mind! I was just going to post this -

So here's a real application than Snap and kiddie jokes (not saying they are not fun!) - a scramble suit to protect witness/whistle-blower identity

korethr• 7 years ago

For shits and giggles, I installed Snapchat just to play with this, after seeing some friends having fun with it.

For background, I am guy, but with long hair and a beard. I wouldn't look out of place in a mosh pit at a heavy metal show. I've also been told I look like Jesus.

To my amusement, my beard and hair seem to break the female face filter. I take picture with it, and nothing changes. Conversely, if I use the male face filter there's quite a difference. My hair gets trimmed, my beard is darkened and my jawline made more square than it already was. The male filter still gets confused by my hair and fails to remove all of it much below my jawline; I can still see my natural hair transitioning back into existence about my neck and shoulders.

This makes me think that the filters are mainly working on hair length, or perhaps me already having hair short-circuits the filter and it forgets to address my cheekbones, jawline, lips, nose and brow.

Scratch all of that. It just worked properly. Compared to the male filter, it did soften my jawline, blur away my beard, and kept my natural lighter hair color (though it took away my natural curls). It could have stood to reduced the size of my nose. Still in the uncanny valley for me, but much closer to crossing that threshold in than some other filters I've seen in the past.

ebg13• 7 years ago

I'm guessing this filter is only available on certain phones? I installed Snapchat to play with it, but apparently I don't get access.

chipperyman5733• 7 years ago

Did you find the filters? You have to hold down on your face when the selfie camera is activated. It's on my LG G7, which basically nobody uses, so I doubt they only targeted certain phones (unless they block generic criteria like less than 2 GB ram or something).

jedberg• 7 years ago

Did you find the filters? You have to hold down on your face when the selfie camera is activated.

This right here is why I hate Snapchat. I've never seen a more unintuitive UI.

StavrosK• 7 years ago

I've found the filters, but I can't find one that does this.

striking• 7 years ago

You may have to have someone send you a snap from their phone with the filter installed and hit the Unlock button in that snap. It's weird.

ehsankia• 7 years ago

My pixel phone had it but the filter ran at around 0.5fps, so it was very hard to use.

wjn0• 7 years ago

From a machine learning standpoint, I would guess that this is achieved with a partially supervised approach. The idea would be to embed an entire face dataset from a number of different poses to a latent space by some method (could be as simple as PCA, or as complex as an arbitrary neural network). An arbitrary classification method (using a partial labelling of faces by gender) yields a hyperplane which optimally separates the labelled points. Then, something like a simple reflection across this hyperplane yields another point in the latent space, and the generator yields a corresponding face of the opposite gender.

Anyone know if this has been implemented for a task like this?

Fun with Snapchat's Gender Swapping Filter

Related Stories