AI-generated fake content could unleash a virtual arms race

https://venturebeat.com/2019/11/11/ai-generated-fake-content-could-unleash-a-virtual-arms-race/

by kristintynski • 7 years ago

106 72 7 years ago

echelon• 7 years ago

These deep fake articles are becoming a meme. They mostly seem alarmist, and yet they're not authored by people actually in the industry.

Deep fakes automate what deep pockets and state actors could already do with Photoshop and other professional tools. The world isn't going to become a scary place because the barrier to entry got lower and the technology has been democratized. People are smart. Fakes will be detectable through entropy measures, corroboration, common sense, etc.

FWIW, I've been working on real time voice to voice style transfer.

https://drive.google.com/open?id=1zRvJEGJjTpKvvzel-J0agh3fKB...

There are already a few other (non-real time) players in this field.

I'm hoping to spin this up as a small social app or filter and sell it so I can fund my capital-intensive film making startup.

I think this tech should be widely available. Not only will it make people think and question more, but it'll be fun too.

It's also amusing (and terrifying) to see all the anti-1st Amendment legislation aimed at combating deep fakes. The truth is that there is nothing to fear except our freedoms being taken away.

ipython• 7 years ago

I don’t think you should dismiss these concerns so quickly. It sounds like you have experience in this field. Perhaps that would make it easier for you to spot potential fakes? What about your grandma? How would she fare?

And besides, the end game isn’t to fool everyone into believing a fake. No, the more insidious goal is to flood the zone with enough dis- and misinformation to overload our ability to filter it. It’s like gaslighting at scale- at some point you just stop being able to process information because it’s so voluminous and of dubious quality that you stop believing any of it.

qball• 7 years ago

No, the more insidious goal is to flood the zone with enough dis- and misinformation to overload our ability to filter it. It’s like gaslighting at scale- at some point you just stop being able to process information because it’s so voluminous and of dubious quality that you stop believing any of it.

This isn't new by any stretch of the imagination. It might as well outright be the job of the Fourth Estate ever since the Fourth Estate has existed (funded, as always, by ads).

And when you look at the people complaining about this, which is for the most part people part of said political group (Facebook/Twitter/non-tech journalists), it makes sense that anything that could diminish their ability to control the narrative should be presented to the public as a new and novel threat.

So this isn't going to result in widespread social chaos, because it's already happening. If anything, this technology will democratize what the old media is currently doing; which will likely result in driving political power towards crowdsourced content aggregation instead of the individual content creators themselves.

EGreg2• 7 years ago

I would go further. Think of it like an ELO rating where most grandmas are 1200, and you pride yourself on being 2000. But eventually, the fakes become so good that the best humans can’t distinguish them, and half the time neither can ANY system, except with some probability.

What then? An unlimited amount of glut that is indistinguishable from human writing, but can be produced with any kind of bias, by any number of fake identities and there is no reasoning with hem.

nitwit0053• 7 years ago

There have been photographic fakes for a long time now, and they were quite good even in the pre-digital era.

Have you seen it creating major social issues? I haven't observed much of anything.

For the lies most people want to tell, there isn't any need to make a fake. Want to paint a protest in a bad light? Just use a real photo of a riot and falsely label it as the current protest.

bransonf• 7 years ago

What about your grandma? How would she fare?

Grandma’s still falling for the phone and mail scams. No amount of legislation is going to fix the reality of technological illiteracy among the oldest adults.

Deepfakes might fool some of today’s adults who don’t quite understand, but we are raising a generation that has turned into a meme: “Everything you read on the internet is true” -Abraham Lincoln

I think the real silver lining here is that the internet is an alternate reality. Many of us refuse to believe that, but social media has created manufactured people. The only solution is to bring people back to the real world. The people are real here. Their opinion, no matter how controversial, comes from a real mouth, and the face you see is the one they were born with.

If anyone forms their worldview based entirely on things they read on the internet, they probably would be just as susceptible to our real world forms of propaganda/gaslighting/ whatever you want to call it.

blunte3• 7 years ago

I strongly disagree that this lower barrier of entry won't make things significantly worse. I also disagree that "people are smart". Of course that depends on your definition of smart, but evidence of general intelligence is evaporating rapidly.

If through software a non-technical person can generate hundreds of websites of "content" that either pollutes a topic with noise or misinformation, you can bet that will be used at great scale for everything. Pick an element of human life, and I can describe a way someone will have financial or political (which often boils down to financial) motivation to do it.

cm11• 7 years ago

This reminds me of Ze Frank's Ugly MySpace pages. He used to tease creators of "ugly" MySpace pages during his show. When someone called him out on it, he had this to say about it:

"For a very long time, taste and artistic training have been things that only a small number of people have been able to develop. Only a few people could afford to participate in the production of many types of media. Raw materials like pigments were expensive; same with tools like printing presses; even as late as 1963 it cost Charles Peignot over $600,000 to create and cut a single font family.

The small number of people who had access to these tools and resources created rules about what was good taste or bad taste. These designers started giving each other awards and the rules they followed became even more specific. All sorts of stuff about grids and sizes and color combinations — lots of stuff that the consumers of this media never consciously noticed. Over the last 20 years, however, the cost of tools related to the authorship of media has plummeted. For very little money, anyone can create and distribute things like newsletters, or videos, or bad-ass tunes about "ugly."

Suddenly consumers are learning the language of these authorship tools. The fact that tons of people know names of fonts like Helvetica is weird! And when people start learning something new, they perceive the world around them differently. If you start learning how to play the guitar, suddenly the guitar stands out in all the music you listen to. For example, throughout most of the history of movies, the audience didn't really understand what a craft editing was. Now, as more and more people have access to things like iMovie, they begin to understand the manipulative power of editing. Watching reality TV almost becomes like a game as you try to second-guess how the editor is trying to manipulate you."

http://www.zefrank.com/thewiki/the_show:_07-14-06

tootahe45• 7 years ago

'Deep fakes' are right up there with 'AI is going to turn against us', or 'software engineers are going to be replaced by AI' on my bs list.

The real problem we face with elections is much less exciting to talk about. It's a few people going around making reasonable and factually correct comments, and manipulating upvote/downvote counts with bots to drive-home the belief that one view is popular, and another is extremely unpopular.

luc4sdreyer2• 7 years ago

AI is going to turn against us

Would you mind sharing your argument about why this notion is a waste of time? Machine learning experts are concerned about it [1], but maybe you know something they don't?

[1] https://www.technologyreview.com/s/602776/yes-we-are-worried...

TaylorAlexander• 7 years ago

It’s interesting that you say this. I suppose I have a different experience. I’ve been following the news around the supposed military coup in Bolivia. And it turns out that thousands of pro-coup bots have appeared on twitter basically overnight: https://mobile.twitter.com/BenjaminNorton/status/11937155449...

It’s making it hard for me to engage with others online. I want to talk to others about my political beliefs but I’ve seen infiltrators like the ones on twitter destroy conversations and spread FUD. I experienced this with Venezuela before.

In my opinion it is a real problem. Yes we will eventually adapt but I think actors taking advantage of this tech early will succeed in influencing the public.

unityByFreedom2• 7 years ago

bots and shills are different. The idea that someone can make a bot to effectively spread propaganda without being detected is far-fetched. The idea that you can convince someone to write propaganda is real.

PeterStuer• 7 years ago

The deeper problem isn't the perfection of fakes. It is the further onslaught of social destruction of perceived/accepted 'truth' as everything always could be fake by the mere acceptance of the potential existence of the technology.

It is the bad actors getting of the hook more than the bad actors framing the innocent.

b0rsuk• 7 years ago

People are smart. Fakes will be detectable through entropy measures, corroboration, common sense, etc.

Yes, in the same way that people can detect and filter email spam. Either by a time-consuming manual effort, or complicated systems very few people understand. And we have corporations to swoop in and save us by offering a centralized service! Search wasn't hard enough to enter. It will be fun in the same way that searching for something useful in a landfill is fun.

Even if 5% people click on mail spam, it's still profitable and there will be more of it. If 5% is deceived by a fake video, it can have a big impact on election results. Fake news and outright lies are already damaging enough if you do get a court order to print an apology. By that time it's already damage control, you can't revert it because people move on and believe what they want to believe. On social media people upvote what gut feeling tells them is right.

WA• 7 years ago

I just read an article from the linked blog (http://thismarketingblogdoesnotexist.com/). It takes me way too long to decipher this as gibberish. Far longer than most spam emails.

LoSboccacc• 7 years ago

I see this as a net positive. as you say people with resources and an agenda have been manipulating the media since, idk, Caesar's deeply exaggerated Commentaries or even before and has been going trough the ages, including Stalin's photoshops and Kerry Fonda revisionism or at a simpler level the billion 'anonymous sources say' articles that litter modern journalism.

but now that the tools are getting into the hand of everyone, everyone is starting to questioning everything they see, and it was about time that people start to realize the chain of trust required in reporting and accepting news.

here's hoping that in half generation we'll be back at thinking critically about every piece of information we see.

Braggadocious• 7 years ago

One key thing here is that the sheer quantity of deepfake content will proliferate exponentially quicker than anything user-generated, and continue to be generated faster, and faster, faster, and faster... on all levels, articles, media, AI will be taught to program and create its own websites and social media and apps etc. So while users may be able to tell what fake is from real (although I doubt it since a decent chunk of the population can't even tell what's fake and real now) the amount of BS everyone will have to sift through online will be insurmountable. You'll have AI sifting through the trash for you, which means your feeds and search rankings will be more, and more contrived and curated with less and less feedback from you or other users. THere was a time when exploration of the internet and discovery of websites was a natural experience. There were no ads giving everyone the same five websites for everything, the same solution for every problem. If people had a question, or needed a solution for something, they had to find it themselves and so the function of social media, message boards, google, etc was that everyone on the internet was in the process of finding their own way, their own solutions. So you'd find crazy inventive solutions and websites for things. That's been disappearing slowly. Once 99.9999999% of the content on the internet is AI created, that experience will never come back. It begs the question: what is the innate value of the internet and what would have to be lost for everyone to stop using it?

Maybe someone will create an amish version of the internet whereby every user must be verified as human by another human who was verified by another human etc. before they can log on, and everything on it was user generated, and user programmed, etc.

buboard• 7 years ago

Many things can change. On the one hand, deep fake text/video competes with (legacy) spammers, which means the profit from spam will soon drop to zero. OTOH, our trust in search algorithms will drop dramatically. People will need to start signing their public media and announcements, probably using one of the existing and robust publick blockchains. It's quite possible that google will become a lot more useless, and some curation will be needed for search results.

Even without the advent of deepfakes it was bound to happen as the internet keeps growing in content. Fake news is just the tip of a big iceberg

blunte3• 7 years ago

This pretty much describes the end of the internet as we know it. Even before AI generated "content", the internet has become lower signal-to-noise as time has moved forward.

It is already the case that for many everyday searches I do, I am forced to be very creative in my search phrase in hopes of filtering out the garbage sites that manage to dominate the first results page.

Watching less tech-saavy people use computers (such as elder family) is enlightening and frightening. They either cannot tell real content from fake content, or worse they are satisfied with what they get from obviously suspicious sites.

Maybe my concerns of polluted websites are less relevant considering the general population is getting more of their "information" from within Facebook rather than even going to search engines (of which they use the default for their browser!).

seibelj• 7 years ago

New companies and technologies will be invented to solve this problem. Every problem has a solution. You are falling into the same trap that has caught humans since the dawn of man. The printing press, the car, the internet, and now “deep fakes” will cause hand wringing but will not destroy us. Just give it time.

guelo• 7 years ago

You don't know if your extrapolation will be true. Maybe this tech is different. There's no doubt that it will change society. It is a revolution. It's impossible to predict the outcome of revolutions.

2sk212• 7 years ago

Only if there is money to be made in filtering out fakes. It has been repeatedly demonstrated that most people will not pay for news in general.

glenstein• 7 years ago

The printing press, the car, the internet

These all came with real tradeoffs and we've just accepted them. The printing press and the internet, in their own ways, sped up the world and shortened attention spans. Cars changed cities. The benefits have been there, but we've engaged with or ignored the harms posed by changes in different ways, and the same unconscious trade is going to happen again.

kevingadd4• 7 years ago

Did we solve car accidents yet? I mean, people are working on it with self-driving cars, but we've been paying a death tax for the proliferation of automobiles for something like a hundred years now. Big pile of dead bodies waiting for that one new technological solution... Instead we've been chipping away at the massive set of problems cars created one at a time, because we allowed the problems to appear and we got used to cars. Meanwhile, new problems appear that we can't easily fix - people still look at their phones while driving and that wasn't an issue ~20 years ago.

Iv3• 7 years ago

We quickly need government-backed ID systems and reputation economics.

greesil• 7 years ago

Or sequence your DNA every time you put content on the web to prove you are a human :)

blunte3• 7 years ago

While it may appear from my comments that I trust no government, that is not the case. However, I don't believe that most governments are immune to corruption or manipulation. So how could we prevent a reputation system from being manipulated for or against certain people?

kevingadd4• 7 years ago

Historically removing anonymous posting and sharing from services like facebook and youtube hasn't done much to solve these problems. Why would government-backed IDs and reputation scores attached to posts fix it, specifically? One person one upvote? Does that really help if thousands of people are getting duped by the fake content and upvoting it?

For a reputation system to actually work, you end up having to limit it to hand-picked "experts", and now it's a question of how you pick them and how you stop regulatory capture from ruining the system (spoiler alert: it will)

tomjen3• 7 years ago

You don't need the government to provide an ID. Have people make their own keypair, then publish what they want. In the beginning it will be nearly worthless, but as others start to trust it it will become more and more trusted, even if nobody knows who is behind it.

friendlybus• 7 years ago

It will force the rise of the editor again. We replaced him with aggregation, but now that is mostly noise we need the editor again to ensure quality. Journalism will have changed but it will come back in walled off online gardens or whatever.

2sk212• 7 years ago

I think the model will be the way Kosher certification works. There are multiple certification authorities and observant Jews can select which certifier they agree with.

joe_the_user2• 7 years ago

I was experimenting with this stuff and you can too here [1]. It's kind of impressive but not convincing. The main impression it gives is it doesn't know what subjects affect which objects, what one kind of relation implies about another relation and so-forth. Still, it gives a sequence of words with a consistent "feel" which is impressive.

However, I would still only find it's text convincing for producing ... a marketing blog since such things just seem like a contentless stream of buzzwords to begin with. If anything, it gives a certain idea of how marketing speech require something, a stream of words with certain feeling, but not real logic.

[1] https://talktotransformer.com/

jeffshek• 7 years ago

I built https://writeup.ai to help with that, but while it helps, it still feels like it's missing "something" at times.

joe_the_user2• 7 years ago

The thing is that I think language over a longer term is about actually communicating a structure to world - in a way that requires knowledge of the world. It is just that over a shorter period, a good portion of language isn't about this communication but about just certain coloring of communication. Which is to say that I think this lacks more than it seems at first blush.

adeeds• 7 years ago

I tried using the new 1558 model on the headlines of news articles and the results were surprisingly decent. Maybe 1/5 of the articles were plausible, and 1/5 were comically awful (Things got pretty zany when I plugged in an LA Times article titled "Yes, Elsa and Anna wear pants in 'Frozen 2.' And yes, it’s a big deal."). Of the remaining 60% of the articles, a lot could be made plausible with some light human editing. I also didn't fine-tune the model or tweak many parameters, which I'm assuming would lead to even better results.

I did notice that some types of articles tended to be easier to fake. A sports article asking a coach about his team's performance would end up spouting the same handful of platitudes, and some random sports statistics that sound realistic unless you look into them. Some articles that were more on the wonk side of things (lots of statistics and not a lot of context) also seemed to produce decent results, although I'm sure they're ridiculous if you're in expert in the subject matter of unemployment figures or auditing a city for overtime charges.

shams93• 7 years ago

This has been true long before ai. Writing and journalism have always been weaponized. The opposite could be true in that it's easier to recognize automated fake news than well crafted hand done human deception.

YarickR2• 7 years ago

Well, this probably means end of unsigned content ; every line of text, every article, etc should be / will be signed by living person's key, or it will be heavily penalized in search engine output; governments will run keystores with citizens' keys, and content signatures will be checked against such keystores to ensure content authenticity (or lack thereof) . Time to reopen GPG , I guess.

daxfohl• 7 years ago

How do we know this article was not generated by a bot?

greggman2• 7 years ago

I often wonder if Ranker, Thrillist, Collider, Vulture are all AI based. The seem to show up in every search

unityByFreedom2• 7 years ago

Doubtful. It's easier to photoshop fake content and we haven't seen that get out of control.

EGreg2• 7 years ago

Wow that AI-generated blog text actually made sense! The best I have ever seen. How did they do it?

100011• 7 years ago

I am going to take the contrary opinion here. AI-generated fake content will inflate away the informational value transmitted by whatever it is trying to fake. It's like 'deep fakes', they'll just destroy trust to video.

achow• 7 years ago

OTOH: I'm pretty excited that these technologies are maturing so that they can be harnessed for empowering common people, or workers in enterprises to make their content beautiful, simple & into effective stories.

One example: Pentagon's slide decks. https://archive.org/details/MilitaryIndustrialPowerpointComp...

Abishek_Muthian• 7 years ago

Considering video, audio are accepted as evidence in most courts without any independent verification; I'm seriously worried about the implications of deep fake on justice.

There is an urgent need gap[1] on detection of deep fakes.

[1] https://needgap.com/problems/21-deep-fake-video-detection-fa...

bostik2• 7 years ago

Risky Business did a really good interview on the subject early last year[0]. Law profession is already aware of the potential problems.

Me? I welcome the future where audio and video evidence are just another piece of evidence.

0: https://risky.biz/RB489/

hertzdog2• 7 years ago

I try to take a different direction. Let's suppose some ai generated content is better than human created content (IMHO we are quite there). Let's go further: maybe in the future we will trust again only some "trusted sources" (newspapers? HN?) while everything else will be not taken into account because the quality will be low (like some comments saying the source is not in the industry...).

account73466• 7 years ago

> maybe in the future we will trust again only some "trusted sources" (newspapers? HN?) while everything else will be not taken into account because the quality will be low (like some comments saying the source is not in the industry...)

Do you realize that current conversational NNs are better at making comments than you?

hertzdog2• 7 years ago

Yes. That’s the point :)

zahrc• 7 years ago

I have personally yet to be convinced of AI generated media content (read articles, videos, photos) maybe the bias that I know that they are AI-generated, but to me it’s equivalent buying a cheap knockoff iPhone from China: it’ll work if you don’t really think about it, or do not know the difference.

We have to top-up education and teach media-awareness in school, while giving badly researched and generally toxic content the cold shoulder.

nightnight• 7 years ago

All tech demos without strong use cases yet. Machine-generated content, spinning content, etc. are black hat tactics employed for decades in order to game Google. Works (just look at what crap ranks high) but the foundation for new huge industries? No.

r0h1t4sh• 7 years ago

Looks like this would be the new form of spam we will have to fight.

seddin• 7 years ago

I might be wrong, but on some social networks as Reddit, many comments or shared links seem too weird, like if they were not real, and some posts that get resposted always end up with the same comments or similar words.

this_was_posted• 7 years ago

We shouldn't talk too much about our skepticism on this becoming problematic. Otherwise believable skeptic text can be generated by malicious actors through AI once it does become problematic so that they can drown out real concerns with virtual trolls.

lordgrenville• 7 years ago

We've had fake photographs for decades and it hasn't seemed to make a big difference in politics. But I think that's because in the past you had gatekeepers, like the editors and factcheckers of "respectable" publications, who would ascertain the legitimacy of a picture before using it. They'd make mistakes sometimes, but got it right 99% of the time.

Now news spreads horizontally through social media and group chats. It's common to see, say, a clip purportedly of police brutality right now in country X, which is actually 7 years old and from country Y. Someone will correct it, someone will dispute the correction, whatever - the damage is done. So I don't think deepfakes will move the needle much. The real damage is the end of gatekeeping, and that's already happened.

QuantumGood2• 7 years ago

We haven't had high-velocity media for decades, and information is easier to make extremely false and get believers than photographs. You can't create a complete narrative through photos alone. You need associated information.

HocusLocus• 7 years ago

muching virtual popcorn

jon_akimbo• 7 years ago

People very concerned about this should spend some time reading ${opposing political group} social media. As you'll discover, people will believe what suits them. Veracity is of remarkably little interest to a remarkably high percentage of the population. Most people, and this is not an exaggeration, would sooner kill/die than change their mind. And if that's true, then consider the mental acrobatics individuals are willing to go through before they even reach that point.

QuantumGood2• 7 years ago

The effects of an ever-higher velocity of fake news isn't clear, but there is no "solution".

Real news not believed, fake news believed has been an unsolved problem for a long time. For example, the history of medical advances show doctors not believing exceptionally solid science in many cases.

There are a number of quotes about progress along the lines of "First they say it's impossible, they they fight it, then they say they believed it all along".

This is a people problem and a media velocity problem going back to the famous quote "A lie travels around the globe while the truth is putting on its shoes."

You can't stop people from believing a lie after it has been released. Removing the lie doesn't help. "Reputable" sources not repeating the lie doesn't help.

AI-generated fake content could unleash a virtual arms race

Related Stories