YouTube’s Algorithm Incentivizes the Wrong Behavior

https://www.nytimes.com/2019/06/14/opinion/youtube-algorithm.html

by furcyd • 7 years ago

203 243 7 years ago

0815test• 7 years ago

Quite true, but let's not pretend that Twittr, Tumbler and Fakebook aren't also "incenting" all sorts of distorted behaviors of their own! These sites are "algorithms" all the same, even if the workings of these algorithms are in some ways more transparent. We need open and widespread federation via technologies like Mastodon, Matrix and ActivityPub, so that if you don't like one "algorithm" you can easily switch to another that's more appropriate to your use case.

nkozyra• 7 years ago

We need open and widespread federation via technologies like Mastodon, Matrix and ActivityPub, so that if you don't like one "algorithm" you can easily switch to another that's more appropriate to your use case.

This always sounds good, but decentralized is nearly impossible to commoditize or make appealing to the general public. Outside of evangelism and word-of-mouth, how are people going to escape the Youtube advertising budget and instead choose - en masse - the product that is better for their privacy?

There's just so much money and inertia to fight.

swiley• 7 years ago

YouTube removing harmless content over copyright etc is one way.

phreeza• 7 years ago

If the ranking algorithm is open for all to see, won't that encourage even worse gaming of the system? I am trying to think of comparable situations in existing open systems, but none come to mind.

tqi• 7 years ago

I think the author's issue is not that her recommendations are bad, but that other people are getting recommendations for things she disagrees with (ie conspiracy theory videos, child-unsafe content, etc). So I don't think she would view decentralization as a win.

thephyber• 7 years ago

We need open and widespread federation via technologies like Mastodon, Matrix and ActivityPub, so that if you don't like one "algorithm" you can easily switch to another

We already have them, yet FB, IG, Twitter, YT are the social media behemoths.

Are you making a plea for the average internet person to care about the values of the platforms they use over the platform content? You are likely preaching to the choir here on HN, but I would guess that the audience here is only 1% of 1% of the audience you need to message.

Corps make good use of psychological experiments to optimize their utility function. "Evil is efficient." The problem is that companies optimize for money without taking into account any other factor in any significant way.

In 1970, Nobel Prize–winning economist Milton Friedman published an essay in The New York Times Magazine titled “The Social Responsibility of Business Is to Increase Its Profits.” [1]

Arguably this quote incentivized the destruction of "good corporate citizenship" (although I admit it's possible that concept never existed in a broad sense).

[1] https://www.newsweek.com/2017/04/14/harvard-business-school-...

kauffj• 7 years ago

http://archive.is/6lbCR

(Archive link for those who prefer non-broken web experiences)

umvi• 7 years ago

Does the algorithm incentivize bad behavior or simply reflect the desires of the viewers?

Someone watching lots of DIY home repair videos will start seeing more. In that case it seems like it's incentivizing good behavior. Likewise, someone watching lots of soft porn on YouTube will be recommended more soft porn.

The algorithm isn't responsible for helping you make good life choices. The algorithm is responsible for recommending videos that you would like, and it seems like it does a good job of that, generally.

Unfortunately, some people like bad things and that's an age old problem that is hard to fix.

That said, it would be nice if users could CTOA (choose their own algorithm) instead of letting Google be the sole gatekeeper.

posterboy2• 7 years ago

It is not the case that someone watching a certain topic will see videos exclusively tailored to their taste. More over it is rarely the case that someone watches something specifically definable to the exclusion of anything less specific, because that desire will ideally be quickly saturated. And if it isn't then the recommendations are still rubbish.

detaro• 7 years ago

In my experience, some ~20 % of recommendations almost always go towards noticably more clickbaity/low-quality content (with rest often fitting quite well, or at least being of a similar level, just on topics that happen to not interest me right now), and as soon as you make the mistake to click one of them it shoots up dramatically. I've taken to open videos I'm not sure about in a private tab to avoid 2 weeks of crap recommendations.

bubblewrap5• 7 years ago

I guess you could compare it to criminals using the telephone. Invention of the telephone helped a lot of people, but unfortunately it also helps criminals.

Likewise the YouTube algorithm helps many people, but criminals or unwanted people (like pedophiles) can also use it.

It's ok to think about ways to prevent it, but I don't think it should be the first concern. Imagine outlawing telephony, because criminals could benefit from its use.

DanBC• 7 years ago

Telephony is an interesting example because for many many years it had very tight restrictions.

Telephone service providers were monopolies, sometimes government monopolies. There was only one type of telephone you could use, and that was supplied by that same monopoly. It was illegal to attach anything else to the line either directly or indirectly. There were even (outside the US) laws on what you could say when talking on the phone.

Here's an article from 1994 about a modem maker who had products that were not officially licensed to connect to the network. https://www.newscientist.com/article/mg14219263-000-technolo...

rosterface• 7 years ago

Ah, the classic “think of the children!” argument. It is no one’s responsibility other than the parent to ensure their child isn’t watching inappropriate content (which will be different for every family and individual).

This article suggests that machine learning and collaborative filtering are incapable of producing healthy recommendations. I beg to differ, the New York Times may not like the result but they work for the vast majority of users on any service with too much content to manually curate.

rspeer• 7 years ago

There are healthy recommender systems, like Spotify.

YouTube is a _disastrously_ unhealthy recommender system, and they've let it go completely out of control.

ariwilson• 7 years ago

Spotify has 40M tracks total. On YouTube, more than 5B videos are watched by users every day. Different scales of problem demand different solutions.

ihuman• 7 years ago

How is Spotify's different from Youtube?

posterboy2• 7 years ago

Spotify simply does not have the content over which an algorithm could loose control.

aukust• 7 years ago

I disagree in some sense. I personally have found the recommending system on YouTube pretty good for the main page of the site. The thing that bugs me is the recommended bar right (or bottom right) of the videos, which can be really annoying and infested with clickbait etc.

throw20102010• 7 years ago

Spotify's recommendation system is dealing mostly with artists that have recording contracts and professional production- their problem shouldn't be compared to YouTube's which has to deal with a mix of professional, semi-pro, and amateur created content. Also there's more of a "freshness" aspect to a lot of YT videos that isn't quite the same as what Spotify has to deal with (pop songs are usually good for a few months, but many vlogs can be stale after a week). Not only that, but many channels have a mix of content, some that goes stale quickly and some that is still relevant after many months- how does a recommendation engine figure that out?

It's better to compare Spotify's recommendations to Netflix's recommendations, which also deals with mostly professional content. Those two systems have comparable performance in my opinion.

restingrobot10• 7 years ago

I completely disagree, my children have a wonderful time following the recommended videos that youtube provides. I'm interested to hear your reasoning on why it is "disastrous".

jasode3• 7 years ago

YouTube is a _disastrously_ unhealthy recommender system,

Can you explain with more details?

I use Youtube as a crowdsourced "MOOC"[0] and the algorithms usually recommended excellent followup videos for most topics.

(On the other hand, their attempt at matching "relevant" advertising to the video is often terrible. (E.g. Sephora makeup videos for women shown to male-dominated audience of audiophile gear.) Leaving aside the weird ads, the algorithm works very well for educational vids that interests me.)

[0] https://en.wikipedia.org/wiki/Massive_open_online_course

AbrahamParangi• 7 years ago

Setting aside the personal responsibility angle for the moment (which I agree with you on!) don't you think that negative externalities should generally be managed?

YouTube is a paperclip maximizer (where paperclips correspond to eyeball-hours spent watching YouTube) and at some point optimizing paperclips becomes orthogonal to human existence, and then anticorrelated with it.

I think it's a perfectly fair thing to say that maybe the negatives outweigh the positives at the present.

(This argument doesn't apply solely to YouTube, of course)

bengotow• 7 years ago

I generally agree with you, but I think YouTube being safe for kids became their problem when they launched a version specifically for kids and marketed it as safe.

taeric• 7 years ago

This implies there is not a society benefit from healthy options.

The parents are the most well placed to know at an individual level. But responsibility is a cop out, if you are just dropping it on someone.

Granted, I agree it is a hard problem. Not even sure it is solvable. :(

sneakernets2• 7 years ago

It is no one’s responsibility other than the parent

Yes, but you _must_ understand that most (no, ALL) of the millennial generation grew up with public content over the airwaves that was curated and had to pass certain guidelines. So many parents think that the YouTube Kids app is the same thing. it's not!

If YouTube want to be the next Television, they're going to have to assume the responsibilities and expectations surrounding the appliances they intend to replace. Pulling a Pontius Pilate and tossing the issue to another algorithm to fail at figuring out is not going to fix the problem.

Thankfully, there's much more out there than YouTube when it comes to children's entertainment, actually curated by human beings with eyeballs and brains, and not algorithms. The problem is that parents don't know these apps even exist, because YouTube has that much of a foothold as "place to see things that shut my kid up, so I can see straight."

RHSeeger• 7 years ago

It is no one’s responsibility other than the parent to ensure their child isn’t watching inappropriate content

Society has had laws in place to prevent children from viewing things they should not be (inappropriate movies, magazines, etc).

scarface746• 7 years ago

What law is there to prevent a kid from going on the internet and going to “inappropriate” sites? Watching video on cable? Finding their Dad’s Playboy magazine back in the day?

Nasrudith7• 7 years ago

No we don't - not in the US. Short of literal pornography that could fall afoul of corruption of a minor the state isn't involved. That is just from ratings cartels and pressure groups.

If nobody gives a fuck enough to affect business you can give the complete SAW series to 3 year olds and all the offended can do is yelp indignantly.

manfredo• 7 years ago

Nope. This only applies to pornography if I recall correctly. There's not laws against showing R rates movies to kids, it's just the theaters that refuse to admit them. In 2011 the courts struck down a California law prohibiting selling I'd M rates games to minors, too.

Miner49er• 7 years ago

I don't think that's the point. It is false advertising for YouTube to create YouTube Kids for kids, and then not have content that is appropriate for kids on it.

la_barba3• 7 years ago

I want to place a kid in front of a screen, press a button and walk away. How am I supposed to do that now?

avip• 7 years ago

It's easier, and more profitable, to write a book than confront your kids about screen time.

FabHK3• 7 years ago

This article suggests that machine learning and collaborative filtering are incapable of producing healthy recommendations. I beg to differ,

The article cites actual instances and recurring problems showing that "machine learning and collaborative filtering are incapable of producing healthy recommendations.": Even when YouTube tried to produce child friendly content, they failed. You can't just say "it's fine" after the article shows it not being fine.

oh_sigh• 7 years ago

What about when youtube marketed a specific product for children, but then it turned out they were letting really, really weird stuff in there.

strikelaserclaw• 7 years ago

"If YouTube won’t remove the algorithm, it must, at the very least, make significant changes, and have greater human involvement in the recommendation process.", man does this person know how many videos and how many users YouTube has? They cannot use anything except an algorithm to recommend videos. They cannot use anything except an algorithm to detect videos inappropriate for children. It seems YouTube is working on this, and this opinion seems like a ill thought out fluff piece to enrage readers and sell this persons book.

andrewvc• 7 years ago

Maybe they can't make editorial recommendations for the long tail but they absokutely could do so for the top few thousand videos each week.

Would that yield an improvement? I don't know, but it would have an impact.

scj5• 7 years ago

I'm kind of wondering if a "Ned Flanders" user-detector is possible.

Search for users who stop videos at "offensive" moments, then evaluate their habits. It wouldn't be foolproof, but the "Flanders rating" of a video might be a starting metric.

Before putting something on YouTube for kids, run it by Flanders users first. If Flanders users en masse watch it the whole way through, it's probably safe. If they stop it at random points, it may be safe (this is where manual filtering might be desirable, even if it is just to evaluate Flanders Users rather than the video). But if they stop videos at about the same time, that should be treated as a red flag.

Of course, people have contextual viewing habits that aren't captured (I hope). Most relevantly, they probably watch different things depending on who is in the room. This is likely the highest vector for false positives.

The big negative is showing people content they obviously don't want for the sake of collecting imperfect data.

mtgx• 7 years ago

All true. But all of this is making me wonder - what are the people thinking who say they can't wait for our society to be run by AI? The apex of AI capability can't even recommend videos properly right now, and we want it to run all the aspects of our society?! No, thanks.

icebraining2• 7 years ago

What those people actually mean is "I can't wait for AI to be so good that it'll be obvious that it should run all the aspects of our society". The current state is irrelevant, nobody wants to put those in charge.

jodrellblank• 7 years ago

What are the people thinking who say they can't wait for our society to be run by humans? The most common state of human government capability can't even put human suffering before numbers in a virtual bank account, can't prioritise truth over falsehood, can't restrain themselves from bribery, can't reliably turn up to hearings or ballots, can't organise projects and complete them, can't compromise when millions of people depend on it. We want to dismiss alternatives which haven't even been developed yet for not being good enough?

v7p1Qbt1im4• 7 years ago

The argument is that a hypothetical benevolent ASI can't be corrupted like literally all humans can. Those people are likely referring to AI's as they appear in Ian Banks The Culture series.

scarface746• 7 years ago

Just to add on, a Youtube executive was recently on a podcast and she said there are 500 videos uploaded per second.

RugnirViking• 7 years ago

thats.... actually shockingly few

v7p1Qbt1im4• 7 years ago

Probably Neal Mohan on Recode right? The current public number is 500 hours per minute. But that number has been floating around for a while. It's probably higher now.

blueboo• 7 years ago

Such an effort would cost literally millions of dollars and surely sink this fledgling startup

ggggtez• 7 years ago

I don't think sarcasm with no substance behind it is very insightful.

Humans are involved in the process. To suggest otherwise is to be willfully ignorant.

moomin• 7 years ago

You are 100% not thinking big enough. These algorithms identify clusters. These clusters can be examined through random sampling. It doesn’t take a genius to spot that a cluster that involves children and pornography might have some problems.

Of course, the system doesn’t expose these kinds of outputs, because no-one has any interest in designing such a system and taking responsibility for the content.

gibrown2• 7 years ago

The total number of videos really doesn't matter, it is the total number of creators, which at least this site claims is a total of 50m for all time: https://mediakix.com/blog/youtuber-statistics-content-creato... (first result I found)

Just start banning certain creators from showing up in recommendations if their content crosses the line. Not that hard if you are willing to do it.

cortesoft• 7 years ago

But how would that solve the problem that the article opened with? There is nothing wrong with the videos of children playing, the wrong part was recommending them to pedophiles

jgalt212• 7 years ago

man does this person know how many videos and how many users YouTube has

While that might be true, 99% of the views are a very small subset of the videos posted. It's completely doable, or at the very least the problem can be greatly mitigated by putting more humans into the process and not letting the algos recommend videos that haven't been viewed by someone in Youtube's equivalent of "standards and practices". All that being said, I fear the primary reason this is not done is because such actions would reduce the number of hours of viewed videos and ad revenues. In fact, I've read articles supporting this theory.

Google under Pichai is basically like Exxon under Lee Raymond--solely focused on revenue growth and completely blind to any number that doesn't show up on the current and next quarter's income statement.

sharcerer• 7 years ago

pichai doesn't come off as enthusiastic. I am a heavy Google product user. Watch all the hardware, I/O events etc, I have seen him use the same sentences multiple times over the past 2 years across events. I get that he won't exude the same charm, excitement as a founder-CEO, nevertheless a lot is left to be desired. A lot of his public statements feel like carefully crafted PR responses. Nothing wrong with crafted responses. When you are a 800 Billion$ company, you gotta be careful, but at least try to give off the perception of being authentic. Google is really bad at the perception game. Apple's really good at that. But I have a strong dislike for stupid moves, even more so than bad moves and Google has made lots of those stupid ones.

kartan• 7 years ago

They cannot use anything except an algorithm to recommend videos.

I agree that with the current business model it is not possible for YouTube to sort it manually.

When I was a kid, a long long time ago, it would have been impossible to conceive that a TV channel showed that kind of content regularly and continue open. If their answer would have been that they cannot fix it because it costs money there would have been an outraged response.

If YouTube cannot keep things legal, cannot respect people rights, cannot be a good responsible part of society because it is not cost effective for me the way to go is clear. And that is true for YouTube, Facebook or any other business digital or not.

d1zzy• 7 years ago

Youtube is not a TV channel, it's a video crowdsourced sharing site.

If we want to have a "free" (as in no subscription and no money required to be payed for the service) video sharing/uploading site, what model would that make it work and still have human reviewing? I consider the fact that there may be undesirable videos as the cost of having such a site, similarly how to the "cost" of having a free Internet is that there's going to be lots of hate online and free access to tutorials to make bombs and what not. It's part of the deal and I'm happy with that, YMMV. If you worry about what kids might access then don't let them access Youtube but please don't create laws that would make free video sharing sites illegal/impossible to run.

This is true for pretty much any free Internet service that allows for user content. If all of Internet content production will go back to just "official" creators (because they are the only ones where the cost/benefit math would make sense) I think that would be a huge loss/regression over what we have gained since the age of the Internet.

seanmcdirmid• 7 years ago

When I was a kid in the 80s, cartoons were basically 30 minute toy commercials. My toddler loves watching videos on YouTube of Russian kids playing with toys, so I guess things haven’t changed much.

CamperBob22• 7 years ago

How about actually demonstrating harm to children (or to anyone else) before launching a moral panic?

Is that an option?

pythonwutang• 7 years ago

They cannot use anything except an algorithm to recommend videos

That’s assuming recommendations need to be personalized. They could recommend at a higher level to groups of people using attributes like age range or region.

I’m not a fan of their personalized recommendations. It’s algorithm overfits my views to recommend videos extremely similar to videos I’ve recently watched, which isn’t really aligned with my interests.

If they took a completely different approach (not personalized) it could really impact the UX in a positive way.

icebraining2• 7 years ago

No thanks. You try logging out and see the generic recommendations. It's the lowest common denominator, just like anything else targeted at large masses of people.

robbrown4512• 7 years ago

I would think having humans more involved in training the algorithm could scale much better.

Also, detecting videos that are inappropriate for children is a lot harder than determining certain content creators that are trustworthy to post videos that are appropriate (and to tag them correctly). That can be learned from the user's history, how many times their stuff has been flagged, getting upvotes from users that are themselves deemed credible, and so on. The more layers of indirection, the better, a la PageRank.

So even without analyzing the video itself, it would have a much smaller set of videos it can recommend from, but still potentially millions of videos. You still need some level of staff to train the algorithm, but you don't have to have paid staff look at every single video to have a good set of videos it can recommend. The staff might spend most of their time looking at videos that are anomalous, such as they were posted by a user the algorithm trusted but then flagged by a user that the algorithm considered credible. Then they would tag that video with some rich information that will help the algorithm in the future, beyond just removing that video or reducing the trust of the poster or the credibility of the flagger.

ehsankia2• 7 years ago

The algorithm works really damn well for 99.999% of the cases. It manages to show me great recommendations from very niche things I'm interested in. But it's the very same behavior that can, in some cases, lead to issues.

PaulAJ• 7 years ago

The trouble with depending on user flags is that it creates opportunities for blackmail.

https://www.theverge.com/2019/2/11/18220032/youtube-copystri...

razius• 7 years ago

They don't care, they want to push them into approved content rather than recommended content. Aka "these are the topics that you are allowed to speak of".

See current Pinterest scandal and banning from Youtube of any video mentioning this.

restingrobot10• 7 years ago

I don't think this is incentivizing bad behavior. It's merely showing the viewer more of what they are already watching with a gradual introduction to broader material. The example of a youtube serving content to "pedophiles" is borderline asinine. The neural network is just making suggestions on viewing, it's not telling people to watch. In regards to the complaint that "adult" content is being served to adolescents, there is an option to filter out sensitive content all together.

Also, as a parent to 4 children myself, the idea of letting my kids loose on the internet completely devoid of any supervision is ridiculous. When did it become youtube's responsibility to parent the children in its audience? Should we also ban HBO, Amazon, and Netflix from providing recommendations because it might be a child in front of the screen?

This is just another pointed attempt to censor free speech via the abuse of technology companies. The idea being that the platform will be restrictive if they are constantly badgered about it.

masklinn• 7 years ago

with a gradual introduction to broader material.

It doesn't gradually introduce broader material, it gradually introduces more "engaging" material.

restingrobot10• 7 years ago

I would argue that your point is semantics, but even so you still have a choice of whether or not to watch the recommended more "engaging" material. It doesn't change the overall point of my statement.

hrktb3• 7 years ago

just making suggestions on viewing, it’s not telling people to watch

I’m not sure I get the difference between suggesting content and telling people what content to watch. Were you trying to drive a different point ?

That aside, it seems your argument is that youtube being neutral in recommending videos shelters them from blame, while the article is basically about why being neutral is harmful.

I personaly think anything dealing with human content can’t be left neutral, as we need a bias towards positivity. Just as we don’t allow generic tools to kill and save people in the same proportion, we want a clear net positive.

restingrobot10• 7 years ago

To make my first point clear, here is a scenario:

I walk up to you on the street and suggest you give me a dollar.

I walk up to you on the street and take a dollar from you by force.

Youtube is a platform, in order remain a platform it MUST remain neutral. You cannot have an open forum with bias. There are certain mutually agreed upon rules, (no nudity, extreme violence, etc.), those limitations are more than enough to handle the vast majority of "negative" content.

I whole heartedly disagree that we need a bias towards positivity. Who determines what that definition is? Something you see as negative, I might happen to enjoy. If Youtube begins to censor itself in that way it is no longer a platform and is now responsible for ALL of its content.

SomeOldThrow• 7 years ago

The broader material is the problem. It’s not a natural way of using recommendations: it’s just an ad at that point.

sneakernets2• 7 years ago

In addition to this, seeing content creators being slaves to the algorithm is an eye-opening experience. Especially when it comes to the children's videos. It's all computer generated garbage powered by responses to changes in algorithms. If kids suddenly watch more content with alligators, prepare for that being the only thing created, recommended or playing. It's wild.

Analemma_• 7 years ago

I don't know if YouTube's problems are so bad that the argument applies in this case, but in general, "We can't comply with this regulation, it would be too difficult at our scale" is not considered a valid defense. Just as banks shouldn't be allowed to get so large that they can't fail without wreaking havoc on the economy, if algorithmic recommendation and moderation can't work, then maybe social networks shouldn't be allowed to get so large that human moderation is not possible.

aaomidi2• 7 years ago

Agree with you. If you can't do your job then maybe you'll have to be shut down.

andromeduck• 7 years ago

Since when did it become YouTube's responsibility to police speech?!

skybrian• 7 years ago

The queue for getting your video posted on YouTube would grow infinitely. (Or, more realistically, people would give up and not bother once it takes years.)

But I guess they could charge money to get to the head of the line?

ilikehurdles5• 7 years ago

The queue for having your video uploaded and public does not at all have to be the same queue for getting your video included in others' recommendations.

PretzelFisch2• 7 years ago

That's not true you can upload a video and not allow it to be recommended until some human review was done. Most youtube channels don't need the recommendation engine.

hrktb3• 7 years ago

There could be a middle ground where videos have limited visibility until getting vetted, or a karma system to fast track regular uploaders etc.

I think there’s a ton of ideas to be tried.

restingrobot10• 7 years ago

That is an apples to oranges comparison, Youtube is a platform not an institution. It is open to all videos, provided they meet certain agreed upon guidelines, and should not be responsible for censoring content based on individual opinions.

I don't think that the recommendation is broken at all, in fact it works astonishingly well for the vast majority of people. The fact that there are a few bad actors is also present in the banking industry, (Wells Fargo for instance), to use your own bad comparison.

munk-a2• 7 years ago

YouTube is asserting editorial and publishing rights when it promotes certain videos, if it were a pure video hosting site (providing a link to uploaded videos for people to do with as they please) then I'd agree they were just a platform, but a newspaper isn't a platform and neither is YouTube.

nradov• 7 years ago

YouTube needs no defense in this case because video recommendations are protected free speech. In the US at least it would be impossible to outlaw video recommendations in a way that would pass Constitutional review.

peteretep• 7 years ago

but in general, "We can't comply with this regulation, it would be too difficult at our scale" is not considered a valid defense

This is a great point that I was going to phrase slightly differently: if YouTube is too large to be able to prevent harm, YouTube needs to be regulated. YouTube get the benefit of being so large, so they should also get the cost.

everyoneisbias• 7 years ago

It's all about advertising money. TV and newspapers are dying and they need someone to blame.

restingrobot10• 7 years ago

I personally think this has deeper political motives as well, but yes I completely agree with you!

jaydz• 7 years ago

I'm sure Google and Facebook understand this, hopefully they won't cower any further. Big Media wants its "fair share" and they will keep attacking until they do.

eganist• 7 years ago

This seems like it takes some notes from Veritasium's theory on YouTube's recommendation algorithm which he posted after his initial reservoir shade balls video went viral. (edited for clarity)

https://www.youtube.com/watch?v=fHsa9DqmId8 for his theory.

ilikehurdles5• 7 years ago

Google absolutely can do all of those things without an algorithm. What they can't do is accomplish that without impacting profit margins (or at the minimum, executive bonuses). "If it impacts business as usual, then it is impossible" is a naive/flawed/libertarian stance.

scarface746• 7 years ago

I’m usually a proponent of the “wall garden” when it comes to applications and strict sandboxing for most users, since software can harm your computer.

But in the case of YouTube, there is absolutely no way that they can curate it and it still being as open as it is.

ilikehurdles5• 7 years ago

There is no need to curate every video, only the ones qualified enough to be recommended/showcased to the public who is not explicily looking for them.

bsder• 7 years ago

But in the case of YouTube, there is absolutely no way that they can curate it and it still being as open as it is.

So?

If YouTube exits the space and allows oxygen back into the video sharing market, we might actually get some different video sharing services that do different things (a la NicoNicoDouga).

physics515• 7 years ago

YouTube does human curation already. They are refered to as "playlist" and every user has the ability to create and share them. So what you are asking for is Google to create their own playlist? Would this also entail removing that ability from other users?

xondono2• 7 years ago

You do realize that to cover current needs (400h uploaded every minute), YouTube would need to employ more than 72000 people working full time right?

emmp• 7 years ago

And these people would inevitably make some number of mistakes in categorization too, or miss something, or just be unable to quite hit some baseline universal standard that doesn't upset a group. Then YouTube still gets the bad press.

bjourne• 7 years ago

But 99.9% of all videos uploaded never gets more than a few handfuls of views so those are irrelevant. Of the remaining 0.1%, you don't need to watch every second of every frame - speeding it through at twice the speed should be doable. So by your own calculations, 72 000 * 0.001 * 0.5 = 36 people working full time.

throwaway287391• 7 years ago

You made me curious so I did some back-of-the-envelope math. An average of 576K hours of video is uploaded to YouTube every day [1], which is 4.032M hours per week. If the reviewers watch all the video at 1x speed and work 40 hours per week, you'd need about 100K reviewers to do the job. (This is just to watch the video -- not including any additional work done to annotate the video with whatever information you want out of your reviewers.) If each one costs $30K a year (probably a lowball estimate including salary, insurance, etc.) it would cost a total of $3B per year. YouTube makes $4B in revenue per year and roughly zero profit AFAICT, so there's no way this is feasible.

[1] https://www.quora.com/How-many-videos-are-uploaded-on-YouTub...

jiveturkey• 7 years ago

Wow, I'm conflicted. First, an obvious idiot statement, which helps us ground our analysis:

Human intuition can recognize motives in people’s viewing decisions, and can step in to discourage that — which most likely would have happened if videos were being recommended by humans, and not a computer. But to YouTube’s nuance-blind algorithm — trained to think with simple logic — serving up more videos to sate a sadist’s appetite is a job well done.

So this person is advocating that a human (ie, another human besides oneself, an employee at youtube), have access to the click stream of individual users? This proposal, in 2019??? Of course this would have to be compulsory to be effective. Why would I want a megacorp to be making moral decisions for me? I'm ok with them making amoral algorithmic decisions.

The author is generalizing the problem of YT Kids, which should be human curated, to all of youtube.

OTOH, yeah feeding our worst impulses is kind of a problem. But, tweaking the algorithm isn't the solution. The machine itself is designed to thrive on attention.

kodz4• 7 years ago

It's not complicated.

They need to stop showing people the upvote and view COUNTS. Behind the scenes they can still use it to make recommendations.

Those numbers are pseudo signals of quality to people who encounter content they have never encountered before.

Even when they have doubts that are watching something unhealthy the mind goes "well if the rest of the world thinks this dumbass is important I better pay attention..."

If a dumbass hurting people on video gets 10 million views other dumbasses worldwide automatically get triggered looking at the count. "hey I can do this maybe I should run for President..."

Remove the counts and you remove the pseudo signal of quality.

jakear• 7 years ago

Fully agree. Instagram is removing like counts (or at least looking into it). I think this a great path forward for the industry. Too often people see “popular” as “correct and not needing question”.

Edit: the Instagram motivation is admittedly a bit different, but a good path regardless

verall3• 7 years ago

It is complicated. I think thats a bad solution.

I want to see the counts. I feel it is far more transparent to see the counts than for things to just be surfaced or not opaquely. Youtube is not a discussion site and it does not work as one. How popular things are is a part of the context of pop culture, and most youtube content is pop culture.

smt883• 7 years ago

How popular things are is a part of the context of pop culture, and most youtube content is pop culture.

I can't think of any traditional medium that tells you the popularity of something before you consume it. Movie theaters, TV stations, radio stations, etc. have no concept of "view counts" telling you whether or not to consume something.

cheez3• 7 years ago

Every single day, I watch the channel of a guy who has put out < 15 minute videos going back to nearly the founding of YouTube.

He gets an average of 10-15 views per day.

The value this guy adds to my day is literally measurable in $$$.

If I could find more people like him, that would be great, but instead these are my recommendations:

    - 5 ways to do X
    - Bill Gates breaks down blah blah blah
    - Something about Tesla
    - One video by a guy I discovered outside of YouTube who is similar to the guy I watch every day. I don't watch this one that much though.

YouTube's algorithm is not designed for discovery. It's designed for engagement. So I keep separate accounts:

    1. Account for actually useful stuff where YT's recommendations are useless
    2. Account where YT's recommendations are OK: white noise like things. Howard Stern interviews, etc

I wish you could configure the algorithm for discovery somehow.

naravara• 7 years ago

How popular things are is a part of the context of pop culture, and most youtube content is pop culture.

Only with respect to people you know talking about it. Not just arbitrary metrics. Rating systems are part of the context of putting valuations on ads, not part of culture. Whatever impact they do have is based on advertisers trying to reel you in by applying the bandwagon fallacy and stoking your sense of FOMO. It's not something edifying.

hellbanner• 7 years ago

https://www.youtube.com/watch?v=jdaPJLJCK1M&t=6s remember that anyone can manipulate these algorithms

merpnderp• 7 years ago

People aren't children needing information withheld from them. Give them the information and let them make up their own minds. This kind of coddling is how we ended up here in the first place.

rossdavidh• 7 years ago

Except, you know, for the ones who are children, who are on YouTube a lot.

pier252• 7 years ago

Give them the information and let them make up their own minds.

That only works for rational people.

notenoughbeans2• 7 years ago

That only works for a specific usecase. I've been looking at videos on how to drywall. Views and upvotes helped me find the most useful instructionals and skip the bad ones.

lstamour2• 7 years ago

I often find the upvote to downvote ratio to be a higher sign of quality than purely the number of upvotes. If they showed me the ratio, I still might get the same value from it.

bubblewrap5• 7 years ago

I would expect most of the time the counts actually are a pretty good indicator. There may be good content that is overlooked, but if something is successful, it probably has something going for it.

module0000• 7 years ago

If YouTube won’t remove the algorithm, it must, at the very least, make significant changes

It must? No, it doesn't have to do a damn thing. It's a product from a publicly traded company, therefore it "must" return value for stockholders. That means more behavior that increases ad revenue. The author is out of touch with reality. Stop feeding your kids youtube if you don't want them exposed to youtube. It's a private service(youtube), not a public park.

fzeroracer4• 7 years ago

Incentivizing value for stockholders above all else is a good way to ensure incredibly anti-consumer practices grow in popularity. Something you might only begin to notice when your kids start getting recommended games from their friends that require you to gamble with IAP to get a new dance or something.

drewbug01• 7 years ago

It must? No, it doesn't have to do a damn thing.

Subject to the laws of the jurisdiction in which it operates, of course. We could - if we so wanted - pass laws to regulate this behavior. That is perhaps the best option, in my own opinion.

It's a product from a publicly traded company, therefore it "must" return value for stockholders.

The dogma that it "must" return value for shareholders is not an absolute rule[1]; rather it's a set of market expectations and some decisions from Delaware (which have an outsize impact on business law) that encourage it. But it's not required. In fact, many states allow a type of corporation that specifically and directly allows directors to pursue non-shareholder-value goals - the benefit corporation[2].

The author is out of touch with reality.

Please re-read the HN guidelines[3].

Stop feeding your kids youtube if you don't want them exposed to youtube. It's a private service(youtube), not a public park.

This is the doctrine of "caveat emptor," essentially - that a consumer is ultimately responsible for all behavior. However, a wealth of regulation exists because that's unworkable in practice. The FDA and the EPA come to mind, but we also regulate concepts like "false advertising." Your stance here ignores the realities of life in service of ideological purism.

[1] http://web.archive.org/web/20190327123200/https://www.washin...

[2] https://en.wikipedia.org/wiki/Benefit_corporation

[3] https://news.ycombinator.com/newsguidelines.html

Nasrudith7• 7 years ago

No we cannot pass laws that do that no matter how indignant we may be. The whole bloody point of the constitution is that no matter how pissed off the majority (or "the majority" which is just a noisy minority as it may be) is that you cannot simply legislate away rights.

The vague "do something!" regulation push has all of the marks of a moral panic and all participants should slap themselves hard enough to leave a mark and repeat "It is never too import to be rational."

FabHK3• 7 years ago

No. As you yourself recognise (presumably, as you put the "must" in scare quotes and italics), that companies "must" maximise shareholder value is a goal contingent on our decisions and policies, not some natural law.

Of course, it is incumbent on us individually to behave responsibly. But there is space for public policy and regulation, even of YouTube.

annadane• 7 years ago

It's pretty interesting to see the increase lately of comments defending various companies. Not saying there's a conspiracy but it's a bit weird

wtracy2• 7 years ago

Hacker News has always been like this. The site is specifically targeted at people associated with venture-capital-funded startups.

A very large percentage of those people are former FAANG employees, employees of companies trying to get acquired by FAANG, or employees of companies doing exactly the same things that FAANG are getting criticized for. (Some are all three!)

I doubt any of them are getting paid to AstroTurf, but they know what side their bread is getting buttered on.

dang7• 7 years ago

The site is specifically targeted at people associated with venture-capital-funded startups.

Quite un-so! HN is targeted at one group only: the intellectually curious. So curiosity—keeping HN interesting—is the only thing we care about and optimize for.

As for VC-funded startups, since only about 10% of HN users are located in Silicon Valley, the number working for such startups is certainly far less than that. 50% of HN users are outside the US. This community is orders of magnitude more diverse than your image of it here.

People work for many different employers. Are their views impacted by their work experience? Of course they are—for every user here. There's nothing sinister about that. If someone posts a view that's wrong, others will correct it. As long as they do so thoughtfully, that's great. The cycle of life on HN.

jstanley2• 7 years ago

Causation could run in either direction here.

It's not necessarily the case that people sympathise with companies because they're involved with running them. It might be that they involve themselves with running companies because they sympathise with them.

duxup• 7 years ago

Is it really an increase, or are you just seeing it?

dang7• 7 years ago

Saying you're not saying there's a conspiracy is a way of saying there's a conspiracy. There's a site guideline against this: the one that asks you not to post insinuations of astroturfing without evidence. Experience has shown that these are imaginary the overwhelming majority of the time, and meanwhile the "cure" poisons the community more than the alleged disease does. So please don't do that on HN

https://news.ycombinator.com/newsguidelines.html

There's a vast history about this for anyone who wants more explanation: https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...

bubblewrap5• 7 years ago

This site used to be called "startup news", it is run by a startup accelerator.

Many people here maybe have or hope to have their own company one day. It seems normal to me to defend the concept of companies.

Personally I think blaming big companies is just lazy thinking that prepares the path to socialism. Of course companies, or the economy in general, aren't perfect. People are still trying to figure out how to do things better. That's normal. There is no need to assume evil greed behind everything.

la_barba3• 7 years ago

I think YouTube has just exposed the kind of content people were already interested in, and possibly consuming outside of the public eye. We find it frightening that people readily click on abhorrent content. When they probably were doing it over other platforms earlier. The internet had gore videos for the longest time. I remember a shotgun suicide video that kids in my school used to shock each other with. If Google as a private company chooses to ban content, than that is their right, but an apriori expectation that an entertainment platform should control peoples social behavior and enforce morality is harmful in a free society IMHO.

zanny• 7 years ago

People were fueling industries of creatively bankrupt content well before the Internet came around, just look at the long term existence of tabloids.

Youtube is optimizing for the underlying psychological mechanisms that put people in that mood because it makes them suggestive and because none of this stuff has substance or meaning they can graze on it like how junk food producers want to promote.

FabHK3• 7 years ago

I think the analogy to junk food is instructive. Both fast food and YouTube maximise revenue while minimising costs by exploiting human flaws and foibles, and do so much more effectively than was possible 100 years ago. It is creating an environment that is radically different than the one we evolved in.

Watching hours of YouTube - obesity of the mind. Kind of.

la_barba3• 7 years ago

Youtube is optimizing for the underlying psychological mechanisms that put people in that mood because it makes them suggestive and because none of this stuff has substance or meaning they can graze on it like how junk food producers want to promote.

Well, YouTube (or any advertising platform) also wants people clicking on ads and actually buying things, not just graze. AFAIK they already demonetize content that is not advertiser friendly, and thus de-prioritize it. Busy professionals with limited free time are your best bet for people with a lot of disposable income. If anything YouTube optimizes for content that is safe-for-work, and will quickly lead to you opening your wallet. But yes, I think this is a large scale multi-variate problem, and individual simple metrics don't cut it.

raz32dust• 7 years ago

YouTube's incentives are the best among such platforms IMO. They allow a simple profit sharing model where a part of the ad-revenue goes to the content creator. This is unlike instagram, for example, where the content creators have to peddle products in their ads to make money. Take a fitness channel for example - on YouTube, the content creator can just be honest, and the views alone will guarantee income. On the other hand, on instagram, they have to resort to selling snake oil. I love YouTube for this, and I am constantly amazed by how YouTube has become a livelihood earner for so many.

vgetr• 7 years ago

I feel like things were mostly fine until the 2016 election, after which journalists became _very_ concerned. If I had a nickel for each, “The algorithms are coming! The algorithms are coming!”, I’d be rich. I mean, I didn’t like the outcome either, but these types of articles seem to motivated by a) finding a scapegoat and b) wanting to use “algorithm” in a sentence.

edoo• 7 years ago

We know we have to keep kids sheltered from people which may have unscrupulous neural networks at play looking for a way to further their own goals at the expense of a child's well being and overall health.

Engagement on the internet is also being driven by neural networks that are learning to adapt to the users brain chemistry to statistically modify behavior for maximum engagement/profit. Perhaps it is time to realize that these services are going to be analogous to a random stranger offering your kid candy for their own twisted goals that are unlikely compatible with a child's well being. If you consider a service like YouTube as an untrusted source of interaction perhaps you'll be as likely to block or monitor it the same as random chat rooms.

tunesmith• 7 years ago

Still looking for recommendations that are influenced by people's best-self intentions of who they want to be, rather than influenced by their worst-self behaviors.

bjt2n3904• 7 years ago

What a pleasant way of stating that humans are basically good. We just keep passing the buck. "We'd be fine if it weren't for this algorithm!"

    We believe that man is essentially good.
    It’s only his behavior that lets him down.
    This is the fault of society.
    Society is the fault of conditions.
    Conditions are the fault of society.

If you ask me, "YouTube's algorithm" is simply exposing the way humanity is. And trying to get an algorithm to "shepherd" humanity to be better is simply Orwellian.

jerrac• 7 years ago

Two ideas come to mind. First, make the engine recommend a few videos that it thinks you probably won't watch. That could help break up the echo chamber effect.

Second, allow users to blacklist, or whitelist, different kinds of content. If someone is struggling with sexual attraction to minors, let them blacklist content with minors in it. If I don't want to see the latest anti(or pro)-<insert political figure here> videos, I should be able to filter them out. I have no interest in Minecraft, so why should I have to keep scrolling past Minecraft videos just because I watch a lot of game related videos?

That said, all the calls for regulation or censorship concern me. I haven't seen the video, but Steven Crowder saying mean things isn't exactly something that should be censored. Any more than all the videos calling President Trump names. What I'm seeing signs of is a society that silences any speech that doesn't fit in a specific, politically correct, box. And that box is being defined by advertising companies who don't want to be associated with topics that their potential customers find uncomfortable. That's not a direction any of us should support...

michaelbuckbee• 7 years ago

The NYTimes uses a very human "algorithm" to determine what to report on and if you look at the comparison of causes of death to what's reported it's wildly off:

Data: https://ourworldindata.org/uploads/2019/05/Causes-of-death-i...

This isn't a knock against the NYTimes so much as it is of humanity, we're all fascinated by the lurid and sensational (note that the Google searches are similarly off) and this permeates all levels of life.

CountHackulus• 7 years ago

To quote Jim Sterling, YouTube has a YouTube problem.

KoenDG• 7 years ago

I doubt this person does not care about the subject they wrote about.

And if the algorithm is producing negative side effects, then, of course, it should be looked at and changed.

I'm no expert myself, but to my understanding: any algorithm is limited by its data set.

Based on its data set, an algorithm comes to conclusions. But one can then, of course, ask: what's the basis for these conclusions?

I recall reading that a certain AI had been fooled into thinking a picture of a banana was showing a toaster or a helicopter, after a few part of the image were changed to contain tiny bits of those items.

It turned out that the AI used the apparent texture on places in the image to determine what was on the image, rather than doing a shape comparison.

Which sounds like a time-saving measure. Though it may very well have been the method that most consistently produced correct results, for the given dataset.

Frankly, the attitude of "we don't know how it works and we don't care" cannot possibly end well.

Neither the attitude "oh well make a better dataset then".

I get that we're all excited about the amazing abilities we're seeing here, but that doesn't mean we shouldn't look where we're going.

I recall a story of an AI researcher who didn't want to define anything because he was afraid of introducing bias. Upon hearing this, his colleague covered up his eyes. When asked why he did this, he replied: "The world no longer exists". And the other understood.

Because of course the world still exists. And just the same way: it's impossible get rid of bias.

Some human intervention is needed. Just like constant checks and comparison against human results.

dbt00• 7 years ago

The problem of the dataset is not just that AI will pick shortcuts and naive heuristics, because humans will too.

The problem of the dataset is that you're not in control of who populates the dataset and what their intentions are. There's no understanding of an adversarial model and threat handling.

diogenescynic• 7 years ago

I mostly watch movie reviews on YouTube and I'm constantly being recommended either weird Joe Rogan, alt-right content, or makeup videos. I don't get it. I've never clicked or watched anything remotely associated with it. I suspect a lot of the popular YouTube channel's are gaming the algorithms or SEO their videos to get more recommendations.

restingrobot10• 7 years ago

The neural network takes into account videos that other people who watched the same one you did watched. It's quite possible that the movie trailer you watched was popular among demographics that also watched those recommendations. If you don't have a lot of data for yourself, you will see a heavier bias towards other people's videos.

toss1• 7 years ago

YouTube can be a source of astonishingly great education and entertainment that will help grow society, as well as astonishingly horrid corruptions of nearly anything that will corrode society at it's roots.

Most of these discussion posts seem to miss the point that 'engagement' or 'upvotes' does NOT equal value.

Also missing is the concept that a company with a massive platform has any social responsibility to at least not poison the well of society.

And claiming "it's the parent's responsibility" may have some truth value, but it does not and should not be an excuse to absolve the platform owner of responsibility.

The key to longer term success of the platforms is to abandon the low-hanging-fruit of "engagement" as a measure value and develop more substantitive metrics that actually relate to value delivered, both to the individual watcher and society as a whole.

As one audience member, I find their recommendations to be basically crap, nearly never leading me to something more valuable than what I just watched (sure, they'll occasionally put up a recommendation that has enough entertainment value to watch, but much of the time I want my 5min back). To find any real value, I need to search again. That already tells us that their "engagement"-based algos are insufficient to serve the needs.

not_that_noob• 7 years ago

YouTube also leads you down radical rabbit holes as that keeps the algorithm happy. How many of the recent terror attacks (ISIS or New Zealand type incidents) were fostered by YouTube watching?

bubblewrap5• 7 years ago

My guess would be zero.

gersh• 7 years ago

I think there is an inherent in optimizing for retention time. Ideally, the recommendation should help find stuff which improve people's health, make them happier, or more informed about the world. However, it doesn't seem like YouTube has metrics on those things. Furthermore, things like that probably can't be determined very quickly on new content.

Buge2• 7 years ago

PewDiePie, a skinny, fast-talking Swede

Is he really fast-talking? He seems kind of slow talking to me, when I watch his videos I use 2x speed.

dredmorbius• 7 years ago

"Fast talker" is not merely descriptive but idiomatic:

https://idioms.thefreedictionary.com/fast+talker

https://www.logicallyfallacious.com/tools/lp/Bo/LogicalFalla...

Cf: fast and loose:

https://idioms.thefreedictionary.com/play+fast+and+loose+wit...

bubblewrap5• 7 years ago

There seem to be some extreme cases where the algorithm fails. That doesn't imply in general it doesn't work well.

Sounds again like hyperbole from the NYT.

I find it more interesting to consider what would actually be a good outcome for the viewers. I suppose originally all those recommender algorithms simply optimized for viewer engagement. Obviously that may not be the best outcome for consumers. Perhaps enraging content makes people stick on a platform longer, for example. But it would be "better" for a viewer to see more educational content and even to disconnect after a certain while.

But how would you even quantify that, for the algorithm to be able to train for it?

The son of a friend of mine taught himself programming from YouTube videos, which YouTube had recommended to him. I wouldn't complain about a result like that.

alt_f4• 7 years ago

yet another NYT anti-tech hit piece

emilfihlman• 7 years ago

I mean PewDiePie's info is rather public but what's with the need to "dox" him right in the beginning?

cyrksoft• 7 years ago

Big Media is dying and they are desperate to shut down competition. This is a political article and nothing more.

JDiculous• 7 years ago

The "wrong behavior" that Youtube incentives is promoting and encouraging clickbait garbage content (just look at the default homepage). The holy metric now is "watch time", the result being that creators stretch out their content to 10 minutes because then Youtube is more likely to promote it (and midroll add = twice the revenue). Yesterday Youtube recommended me a 10 minute video of some guy explaining how he made this simple drone shot that could've been condensed down to a single sentence - "Turn sensors off". What a waste of time.

But hey they're a corporation and thus have no accountability to the public good.

YouTube’s Algorithm Incentivizes the Wrong Behavior

Related Stories