'Fiction is outperforming reality': how YouTube's algorithm distorts truth

https://www.theguardian.com/technology/2018/feb/02/how-youtubes-algorithm-distorts-truth

by sus_007 • 8 years ago

51 40 8 years ago

graeme2• 8 years ago

Interesting results. Though I wonder to what extent some of them reflect the outside world. (Not reality. Just what people outside youtube focus on.)

The Clintons have been the subject of conspiracy theories since the 90s at least. So, it's not surprising that this would extend on to Youtube.

(I haven't made an extensive study of this, but my impression is that the American right wing is prone to conspiracies about people or events, whereas the left wing is prone to conspiracies about institutions.)

The article also mentions Trump speeches were more recommended. There's an innocent recommendation: people find Trump more engaging. Of the two candidates, Clinton was far less charismatic - she even admitted she wasn't a natural like Bill Clinton was. So, it's not exactly surprising that Trump was more recommended and more watched.

This isn't to take away from the broader thesis. The recommendation algorithm is certainly worthy of study. And the fact that the algorithm quickly moves from normal to polarizing is disconcerting.

But, it does seem as though the recommendation algorithm to some extent mirrors what we see in talk radio and other media segments in America.

pulisse3• 8 years ago

But, it does seem as though the recommendation algorithm to some extent mirrors what we see in talk radio and other media segments in America.

I'm not sure what you think the significance of that fact is. YouTube's recommendation system is specifically trying to maximize time spent by users watching videos, and what's turned out to be the case is that sensationalistic, politically polarizing, conspiratorial, etc. videos are excellent for promoting engagement.

It sounds as if you're saying that because such extremist content exists and engages people in other media, there's no real significance to the fact that YouTube's recommendation system creates a feedback loop for this content. But YouTube most definitely has agency there: First, promoting videos to users isn't just holding up a mirror to reality, but is an overt act. Second, the design of the recommendation system--what aspects of user behavior to cue off of and what metrics to optimize--reflects decisions on YouTube's part; they don't fall out of the sky.

graeme2• 8 years ago

I guess I meant Youtube is augmenting an existing tendency within American society, rather than creating a new phenomenon out of whole cloth.

XorNot• 8 years ago

It's to some extent self-destructive for YouTube - the I've noticed that when looking for entertainment I wind up being rapidly bored by the recommendations because they're only ever a mirror of exactly what I've seen before - they're not an attempt to challenge or expand on my preferences.

Which of course is what you get from blind correlation driven user feedback - "did you like this?" - if I'm not sure because I'm now exploring outside my comfort zone then that goes back into the algorithm as "no".

roenxi• 8 years ago

There's an innocent recommendation: people find Trump more engaging.

100% this. There seems a to be a presupposition in the 'fake news' debate that somehow there are right and a wrong choices in politics and anyone making the 'wrong' choice must be re-educated by the powers that be.

I still recall the NSA scandal where it turned out they were indeed spying on everyone; that would undoubtedly have been branded 'fake news' pre-Snowden. One can easily imagine a world where a presidential candidate in favour of winding back the NSA apparatus was shut down, with videos talking about overreach being condemned as conspiracy theories.

The only solutions here are communication and engagement of people with different ideas. Trying to police the truth will not end well; somebody will end up corrupting the systems in ways that end badly.

pulisse3• 8 years ago

Trying to police the truth will not end well; somebody will end up corrupting the systems in ways that end badly.

YouTube's recommendation policy is currently, to a zeroth-order approximation, "push content that gets people riled up". Trying to get them to not follow that policy isn't policing truth; it's asking them to refrain from an affirmative act.

CandidlyFake6• 8 years ago

(I haven't made an extensive study of this, but my impression is that the American right wing is prone to conspiracies about people or events, whereas the left wing is prone to conspiracies about institutions.)

No. Both the right wing and left wing are prone to conspiracies about people, events and institutions. The right wing sees conspiracies in institutions like the UN, EU, etc. The left wing sees conspiracies in people like Trump.

They are both the same. 10 years ago, it was the right wing claiming the saudis got obama elected. Now, it's the left wing claiming the russians got trump elected. 10 years ago, the right wingers claimed obama was born in kenya and serving the saudis. Now, the left wingers are claiming trump is working for putin and the russians.

The article also mentions Trump speeches were more recommended. There's an innocent recommendation: people find Trump more engaging.

This is so funny. 10 years ago, mccain supporters were claiming that youtube was biased towards obama. And the media mocked mccain, palin and their supporters for being paranoid. Now it's the left wing media claiming youtube was biased towards trump. The hypocrisy of the media.

It's amazing how things flipped in a matter of a few years. 10 years ago, it was foxnews and the tea party ranting and raving about obama. Now it's the left leaning media ranting and raving about trump and social media.

thirdsun• 8 years ago

That was my first thought as well.

The algorithm likely shows controversial content for the same reason we rarely see good news on TV: Dramatic, sensational and outrageous content is simply much more appealing and entertaining to a large audience (which doesn't necessarily mean that they agree with it).

thoaway1• 8 years ago

The Guardian calling YouTube out for distorting truth? This is the pot calling the kettle black.

w_t_payne2• 8 years ago

The guardian certainly has a strong editorial position, but I've never found its' reporting to be anything other than honest, well researched, and written with integrity.

I think that they believe what they write with a high level of conviction, which is more than you can say for some of the more egregious videos floating around on YouTube.

ghostcluster• 8 years ago

The Guardian plays with heavy distortions, biased selective reporting, and dishonest ideological clickbait tactics all the time.

CandidlyFake6• 8 years ago

The guardian certainly has a strong editorial position, but I've never found its' reporting to be anything other than honest, well researched, and written with integrity.

You got to be kidding me. If you think the guardian has integrity then it's because you agree with their viewpoints.

They have no more integrity than foxnews does.

It's ridiculous how much certain news organizations get on a "hacker" forum.

nullc• 8 years ago

I put on some science video and walked away recently... came back latter and the autoplay had navigated itself deep into k00ky flat earther stuff.

People raising concerns about superhuman AI grey-gooing the planet have their priorities wrong. Incompetent narrow AI will happily navigate man kind into a wall when the pattern of bricks by chance happens to look like a tunnel.

Don't be evil... we have automation for that.

zzzzzzzza2• 8 years ago

the stereotypical grey goo is usually an example of the kind of narrow AI you are talking about

dogma1138• 8 years ago

Grey goo doesn’t require intelligence of any sort tho just the ability to consume resources and replicate.

And it doesn’t even have to be nano machines, HzD is effectively a grey goo scenario with drones that can replicate and can produce essentially biodiesel.

CandidlyFake6• 8 years ago

I put on some science video and walked away recently... came back latter and the autoplay had navigated itself deep into k00ky flat earther stuff.

That's the point of youtube autoplay. You leave it running long enough it'll go through all the videos. That's what makes youtube interesting. At least that's what made youtube deep diving interesting before all the censorship.

You go through enough books in the library and you will eventually find books on alchemy.

Also, you could turn off autoplay with a click of a button...

arbie• 8 years ago

That's the point of youtube autoplay. You leave it running long enough it'll go through all the videos. That's what makes youtube interesting. At least that's what made youtube deep diving interesting before all the censorship.
You go through enough books in the library and you will eventually find books on alchemy.

This might be true if there were only a finite number of videos _before_ getting to the kooky ones, but YouTube tends to get to them disturbingly fast.

analogic• 8 years ago

pro tip: never fall asleep to youtube without disabling autoplay. conspiracy nut vids make for some interesting dreams.

jwhitlark• 8 years ago

Or suddenly the volume jumps.

smacktoward• 8 years ago

Not as interesting as in-depth documentaries on the Donner Party.

(source: personal experience. yikes!)

firasd• 8 years ago

Youtube personalization is eerily well-tuned from a tech perspective, and has undoubtedly been good for their business. But one of the least expected side-effects of new technology has to be the fact that Youtube’s ‘related videos’ algorithm has given rise to a generation of Flat Earth believers.

Despite that, I gotta say, I'm uncomfortable about the series of media 'hits' on Youtube over the past year. It is based on self interest as a tech professional (I want to be able to create a social network without having to highly moderate everything!) but also, everytime it happens, a bunch of Youtubers who are innocent of what the media is alleging (objectionable content, etc.) lose their ad revenue. It's true that the 2016 US election has prompted the Anglosphere to subject social networks to more scrutiny, but when clamping down on user-generated content (or how it's displayed and monetized) we gotta look at the collateral damage, too.

pulisse3• 8 years ago

I want to be able to create a social network without having to highly moderate everything!

No one (in the context of this debate) is asking YouTube to ban videos.

CandidlyFake6• 8 years ago

Youtube’s ‘related videos’ algorithm has given rise to a generation of Flat Earth believers.

That's nonsense. There have been flat earthers long before youtube and there will be flat earthers long after. Also, the amount of flat earthers is so insignificant, but of course the media has to exploit that to push their agenda.

The guardian and the news media caused far more damage with their fake campus rape culture nonsense than silly flat earth videos.

Despite that, I gotta say, I'm uncomfortable about the series of media 'hits' on Youtube over the past year.

The media has been attacking youtube, facebook and social media because it is marginalizing their effectiveness.

The news industry was used to having a captive audience to whom it force fed propaganda. What to read, what to think, how to vote, who to believe, etc were their domain.

Social media is allowing people to co-opt that system and seek out what they want to read, watch, etc.

I'm awestruck at how coordinated the supposed "independent" and disparate news companies are working together in this propaganda campaign. And how british media and our media are working in tandem.

Also, if anyone has the time, it would be interesting to see how many "anti social media" stories have been on HN. I'm guessing we've had an anti social media story on the frontpage of HN everyday for the past year.

It's ridiculous.

poisonous_dart• 8 years ago

I don't know, guys. The world before complex algorithms generating clicks on the Internet was pretty biased and stupid. I don't think autoplay is going to be the thing that destroys our collective intelligence for us.

If we want our society to be less prone to fictitious nonsense, education that encourages critical thinking and a rudimentary of statistics should be encouraged, but that's often not interesting enough to write about for many journalists, nor is it an easy problem to tackle.

i_feel_great• 8 years ago

"If we want our society to be less prone to fictitious nonsense, education that encourages critical thinking and a rudimentary of statistics should be encouraged,"

That's the chicken right there. The egg is: the more people look at this stuff, the less critical thinking they are going to do.

skywhopper• 8 years ago

I would say that we should start by critiquing the implicit claim by Google, Facebook, Waymo, and others that new means of computational statistical analysis can serve as a reasonable replacement for human intuition and judgment. Turns out some of the world’s most valuable companies are betting on lots of poor assumptions and easily-gamed algorithms, and we’re just starting to see the fallout of our failure to call BS on the ever-louder hype machine.

I certainly agree that deep learning has delivered some impressive results, but the acceptance by VCs and the media that his new wave of AI will bring a revolution in Internet time to established real-world infrastructure and processes is a major failure of basic critical thinking on a broad scale.

CandidlyFake6• 8 years ago

The world before complex algorithms generating clicks on the Internet was pretty biased and stupid

The way the news has been ranting about social media, you'd think it was responsible for all the world's ill. The hyperbole and just pathetic nonsense from the media the past few years has been ridiculous. From the pewdiepie nonsense to the insane coverage of trump to the current hysteria over facebook. I wish we could go back to a time when the news media didn't have such obvious agenda. It would be nice if the news industry pushed out news instead of their agenda.

but that's often not interesting enough to write about for many journalists

If everyone was educated, thought critically and understood stats, where would news companies get their journalists from?

aesto• 8 years ago

Don't blame algorithms for human nature. That thing is ultimately just minimizing a loss function and if recommending garbage gets the best results, that's what it will do.

dawhizkid• 8 years ago

The fact that we’re all so easily influenced in ways we don’t even realize makes me believe AI deities are only a matter of time. I’m legit terrified of a future where some programmer creates a cult AI figure that will direct people to either harm themselves or others.

ryanwaggoner• 8 years ago

People have been harming themselves and others in the name of religion for millennia. You could argue that it’s easier to destroy or modify an actual AI than an imaginary god.

CandidlyFake6• 8 years ago

It's already begun.

https://news.ycombinator.com/item?id=15347067

bornonline12• 8 years ago

All the needles suffering would call for an AI to save us from ourselves.

mythrwy• 8 years ago

This whole "fake news" and "big tech is horrible" trope is tiresome.

That being said there is a bunch of nonsense on YouTube and it is annoying when trying to find legitimate information. Watch an informative video on history or archeology or something and two seconds later they are pushing a video about aliens creating the pyramids and the worst part is, because it's a video, you might have to watch it for a few minutes to find out it's utter rubbish unlike text where you can scan quickly and determine.

I don't really expect (nor want) YouTube to curate though, the moral hazard in that seems even more undesirable. Some kind of an expert rating layer would be cool though for objective topics (lets leave politics out, I'm not interested in that kind of a rating layer and don't watch those kind of videos). I can't follow every legit poster on every topic I might become passingly interested in and it's troubling to see nonsense presented right alongside informed commentary.

0xBA5ED2• 8 years ago

If the algorithms are designed to maximize engagement, then the opposite is true from a certain point of view. The result is an _undistorted_ reflection of our nature.

bornonline12• 8 years ago

Can't follow you

0xBA5ED2• 8 years ago

I'm saying it reveals truth about ourselves because it gives us more of what we want. The growth of certain communities is an exposure of something that was already there.

Bitcoin_McPonzi• 8 years ago

An ex-YouTube insider reveals how its recommendation algorithm promotes divisive clips and conspiracy videos. Did they harm Hillary Clinton’s bid for the presidency?

So the reason people didn't vote for Hillary was that they were fooled by videos they saw on the Internet? Is this what people who are upset their candidate didn't win believe, even one year after the election?

sqdbps• 8 years ago

It's not a given that social media had anything to do with the result of the election, the few scientifically researched reports out there suggest that good ol’ mainstream media was by far the prime channel of influence: https://cyber.harvard.edu/publications/2017/08/mediacloud

That aside, so what if “fiction was outperforming reality”? it’s a marketplace of ideas and people choose to spend their time watching what they may, stop infantilizing them.

I realise that the media has been under “attack” lately (if you really consider name calling as serious attack) but they would benefit from some introspection, not only are they scapegoating social media and calling for it to be regulated - freedom of speech be damned - they also keep advancing the invented notion of a “backlash against tech firms” projecting their own aversions onto the rest of the world.

See: no real evidence of any such backlash:

https://www.wired.com/story/davos-big-tech/

https://www.edelman.com/trust-barometer/

'Fiction is outperforming reality': how YouTube's algorithm distorts truth

Related Stories