A new way to detect ‘deepfake’ picture editing

https://www.lightbluetouchpaper.org/2021/06/07/a-new-way-to-detect-deepfake-picture-editing/

by etiam • 5 years ago

143 88 5 years ago

lightbluetouchpaper.org

quotemstr• 5 years ago

Isn't literally every deepfake detection technique just an oracle that you can use to train a better deepfake creator?

drclau• 5 years ago

At some point in the future, AI will make better humans than naturally occurring humans.

XorNot• 5 years ago

The built in "touch up my appearance" feature in my phone, various video chat software etc. means that future is already here [1].

[1] https://mothership.sg/2021/03/japanese-biker-actually-man/

deepsun• 5 years ago

Friedrich Nietzsche called that "Superhuman", or "Übermensch".

jimmySixDOF• 5 years ago

I know. It's like we are going to need reverse Zero Days after a while. As soon as they publish it becomes the old new way to detect Deepfakes.

water8• 5 years ago

-1 days

jtsiskin• 5 years ago

Especially if it’s differentiable

mattchamb• 5 years ago

I would interested in a way to automatically edit real photos of people in such a way that it looks the same but now tests positive for being a deepfake.

dmurray• 5 years ago

Yes, but if the "fake detection" technique is computationally much faster than the generation technique, having access to a better oracle is not so helpful to the deepfake creators, who are bottlenecked by the other phase.

lmilcin3• 5 years ago

An image owner can modify their image in subtle ways which are not themselves very visible, but will sabotage any attempt to inpaint it by adding visible information determined in advance by the markpainter.

Hmm... so if I ask it to add an apple to the scene it just looks for an apple and says "yep, found an apple".

That is not an algorithm to detect fakes, it is an algorithm to detect predetermined edits.

Steganography/watermarking has been not only known but used for practical purposes for a long time.

axiosgunnar• 5 years ago

no i think that sentence means you can still use photoshop manually to laboriously edit stuff… but the super convenient AI editing will fail, meaning 99,9% of the population won‘t be able to create deepfakes

lmilcin3• 5 years ago

That's exactly what I meant.

Watermarking can be used to find portions of the photo that have been tampered with.

It is actually pretty simple. And using AI for this is pretty stupid in my opinion.

You just disperse a little bit of additional signal in the photo, maybe divide it into lots of blocks (they don't need to be rectangles and they can overlap). Think of it as every small piece of the photo having its own signature embedded.

If you don't know how the signature was embedded or what key was used for it, you are going to disturb the signature and your editing attempt will be foiled. Not only that, but you will be able to tell which parts of the picture were touched and how so you can tell whether the picture was just cropped, brightness changed, or something else happened to it.

As you see, no need for AI...

_Algernon_• 5 years ago

The remaining 0.1 % happen to be the ones that matter. It helps no-one to eliminate all but state actors from creating deep fakes.

sam0x17• 5 years ago

Sounds like something a simple gaussian blur would destroy

mattowen_uk• 5 years ago

...a photo agency that makes stock photos available on its website with copyright watermarks can markpaint them in such a way that anyone using common editing software to remove a watermark will fail; the copyright mark will be markpainted right back. So watermarks can be made a lot more robust.

I fail to see how this can actually combat against someone with the patience to manually paste out copyright marks using something like a clone brush, or even a normal brush ?

If someone wants to steal your work, they will. There's no sure-fire way to stop them.

VSerge• 5 years ago

It won't defeat manual edit, nor do the authors claim it. They do however make a solid point in saying it will defeat cheap automated manipulation / deepfakes at scale that would otherwise be easy with inpainting features.

Anything that can help us differentiate authentic images from manipulated ones, especially when it comes to news, is imo a welcome addition in the arsenal of societies attached to the concept of verifiable facts.

terryf• 5 years ago

Authentic images do not exist. They always include a viewpoint. Example here: https://2.bp.blogspot.com/-rHeyzklapsE/W5-yUBVnbxI/AAAAAAAAA...

d--b• 5 years ago

Adversial techniques are specific to the model used, no?

You'd need to know all the models that one might use to tamper with a picture and modify the image in such a way that it screws them all.

And the first thing that will happen is that people are going to train their models with these new 'tamper-protected' images...

thih9• 5 years ago

I'm having trouble following. Can someone explain this in easier terms? What scenario does this address and how does this work exactly?

rplnt• 5 years ago

The OP's title is incorrect. This doesn't detect deepfakes, it serves for people to watermark their images in a way that are hard to remove by conventional ML approaches.

wokwokwok• 5 years ago

scenario: you want to resell 50000 istock phots, but they are watermarked, making the free high resolution versions valueless.

solution: you use a denoising filter to reconstruct the watermark pixels to plausible original values. Profit!

this: instead of just simple obvious watermarks, you can instead encode visually indistinct fake-noise that deliberately confuses denoising neural networks.

They claim “We find that we can target multiple different models simultaneously with our technique.”, ie. it is reasonably generic.

how? Eh, that’s complicated, look up “adversarial neural networks”, there’s a fairly high level overview here: https://towardsdatascience.com/how-to-systematically-fool-an...

dmos62• 5 years ago

The post is pretty hard to understand. Here's the paper and an excerpt from the abstract:

https://arxiv.org/abs/2106.00660

First, we show how an image owner with access to an inpainting model can augment their image in such a way that any attempt to edit it using that model will add arbitrary visible information. We find that we can target multiple different models simultaneously with our technique. This can be designed to reconstitute a watermark if the editor had been trying to remove it. Second, we show that our markpainting technique is transferable to models that have different architectures or were trained on different datasets, so watermarks created using it are difficult for adversaries to remove. Markpainting is novel and can be used as a manipulation alarm that becomes visible in the event of inpainting.

mzs• 5 years ago

Thanks, this is the code with examples in the readme:

https://github.com/iliaishacked/markpainting

Etheryte• 5 years ago

It's hard to see this as much anything but snake oil. While the technique allegedly allows you to target multiple models, it still requires you to know what adversarial model you're targeting before the fact. If the adversary sees there's visible artifacts they'll just change a few parameters or use a different model and be good to go none the less. In short, while this might be academically interesting, I can't help but feel that this is a futile field to work in. A good way to see why is to look at it this way: a skilled human can manually manipulate an image so that it isn't possible to tell that the image has been manipulated. It is then a matter of teaching the model to do the same, which might take time or be computationally prohibitive for now, but eventually it will be done.

danuker• 5 years ago

I can't help but feel that this is a futile field to work in.

Considering that training a model costs thousands of dollars, and that people reuse models instead of training from scratch, I somewhat disagree with this in general, for now.

But this specific technique implies the attacker sees the result, which means they can try different models, until one does not produce artifacts.

hwers3• 5 years ago

Models come in a broad spectrum and while it's true that gpt-3 costs a ton, many do not. Any mobile ready snapchat esque real time deep fake filter will typically be super fast and cheap to train (since the model has to be lightweight to run on a phone). There's also transfer learning from an expensive model which could easily be modified to remove whatever 'architecture flagging' is built into the structure.

sigmoid105• 5 years ago

A few weeks ago, there was a post here about that guy who deepfaked female profile pictures. He showed that the visible artifacts go away by merely scaling up the number of parameters in the model. Give it a few years and these fake images will be impossible to detect unless we agree on some way of cryptographically signing images from real cameras.

sideshowb• 5 years ago

I'm not hopeful that the latter will do much. Just do your deepfake, point a camera at your screen and press the shutter button to get it cryptographically signed..?

Ok, so that's tongue in cheek and we'd see some artifacts there but the general principle works. You could intercept the signal from the CCD, or just extract the signing key from the camera's ROM, etc etc.

foobar333333• 5 years ago

Cryptographic keys embedded in hardware worked really well for the DRM industry. Eventually someone will work out how to extract the key, then create malware to spread around android devices to send back millions of valid keys. What do you do then? Blacklist millions of real peoples cameras and prevent them from using the internet?

spywaregorilla2• 5 years ago

At best that would allow you to tell if a picture was raw footage. Which is pointless because people edit things all the time.

arbitrage• 5 years ago

define 'real camera', please.

smaryjerry• 5 years ago

Imagine a cryptocurrency like log of data verifying images. Essentially, all cameras will be internet connected with a process to force creation of an NFT at the moment users press the photo button which includes the entire data of the image. Any image without that tech will be considered unreliable. Of course that would have implications that no photo could ever be deleted, I’m guessing this system would be government based and they would be just fine with that.

kkielhofner4• 5 years ago

My new startup Tovera[0] is working on a verification approach that (with some tweaking and user feedback) should be usable by anyone without some of the pesky and potentially privacy concerning aspects of a key/cert based approach. More in my comment here:

https://news.ycombinator.com/item?id=28401256#28407184

[0] https://tovera.com

Closi• 5 years ago

This is fine, we will just train another model to detect which model is being targeted.

Then if there is an adversarial model built for that, we will just create a model to detect images that have been changed to hide the model they were built with.

Then we will build a model for the adversarial AI on top of that and so on.

mdre• 5 years ago

OK cool, but the commonly used Photoshop's content aware fill, which they supposedly target, isn't even using any ML. They don't know what they're doing or what? https://en.wikipedia.org/wiki/PatchMatch

shtps3• 5 years ago

While all this stuff around deep fakes and its detection is interesting I'm still left wondering why we don't use cryptography to prove if an image is authentic or not.

vletal• 5 years ago

Because most people can not even use cryptography to sign more important stuff such as email.

throw2500• 5 years ago

Because the world is impure and the link between the real world and the mathematical domain will never be ironclad.

For instance, say that Joe makes a deepfake and then signs it with his key. Sure, it's beyond doubt (assuming keys weren't leaked, etc) that Joe either took or created the picture, but that doesn't in itself tell you whether Joe made a deepfake or not.

It's the same as in supposed blockchain logistics operations. If Fred the farmer says he harvested x bags worth of grain but some were stolen before he could ship them, there's no way to mathematically verify whether the theft actually took place or the harvest just came up short.

In both cases you're going to need some kind of monitoring, and that's the purpose deepfake detection algorithms would serve.

rocqua2• 5 years ago

Will signing come from the camera? At that point you run into DRM.

Besides, what will the camera sign? The produced JPG? The raw file? What about rescaling? Do we want ZKPs that a JPG was achieved by nothing more than re-scaling and tone-mapping another JPG or RAW file? Those ZKPs are going to be massively big and slow to verify.

shtps3• 5 years ago

My main concern is the faking of official statements by public figures where having those signed addresses the issue of deep fakes. Of course there has to be infrastructure in place for this and people must know that you can verify them.

If we get to a point in society where unsigned images, or images signed from a questionable source are looked at skeptically, then that alone is a step in the right direction. I don't know the final solution and I don't claim to know it, but cryptography seems like a step in the right direction.

kkielhofner4• 5 years ago

I had the same realization several months ago. In the arms race between deepfake detection and generation it's widely understood that generation will win. That's why I started working on Tovera[0].

Main comment here:

https://news.ycombinator.com/item?id=28401256#28407184

[0] https://tovera.com

shtps3• 5 years ago

Interesting approach! I will read into it.

Rucadi• 5 years ago

The better you can detect deepfakes, the better deepfakes will be using that method as discriminator.

hwers3• 5 years ago

I guess that means they better start working on non differentiable ways of detecting deep fakes.

wizzwizz4• 5 years ago

Most functions can be approximated by differentiable ones.

hwers3• 5 years ago

This is all interesting but a huge part of me suspects that this is an arms war that will be impossible to win in the long run.

literallyaduck• 5 years ago

Wouldn't you just build a training set where ML made good edits vs visible ones and train it to avoid the bad version?

mojomark2• 5 years ago

Couldn't reputable photo/video authors just publish content as an NFT? If the NFT is not traced back to a reputable source, then the content is deemed unreliable.

Extigy• 5 years ago

Why not just sign the content with a digital signature?

barbazoo• 5 years ago

You mean devices could sign the images they create and then wherever someone publishes the picture you'd be able to validate the signature using that device's (published somewhere) public key? That would be cool indeed.

Except it wouldn't change anything. Facebook doesn't care about fakes and people wouldn't care whether the picture someone shared hasn't been tampered. Also whoever manipulated the picture could just sign it again probably.

this_was_posted• 5 years ago

Because then you would lose in the buzzword bingo game.

macksd• 5 years ago

The whole point of GANs is to train a model until it can defeat detection by another model. Virtually any technique for detection that can be reproduced could just be incorporated into the training loop and rendered almost useless.

ford_o• 5 years ago

However, the detection process could be so slow, that it wouldn't be practical.

d0100• 5 years ago

Hear me out: blockchain cameras? \s

But maybe we'll just have to limit cameras outputting NFT videos, or signed frames as the only source of trusted media

kkielhofner4• 5 years ago

Working on it! Main comment here:

https://news.ycombinator.com/item?id=28401256#28407184

GenerocUsername• 5 years ago

The github is better than the blog: https://github.com/iliaishacked/markpainting

refactor_master• 5 years ago

Doesn’t take many rounds to jpeg the living pixel out of anything, and still spread it as fake news, with a baity text.

enduku• 5 years ago

Seems like compression is not such a bad deterrent for adversarial deepfakes.

A new way to detect ‘deepfake’ picture editing

Related Stories