Tech firms must tame toxic algorithms to protect children online

https://www.ofcom.org.uk/news-centre/2024/tech-firms-must-tame-toxic-algorithms-to-protect-children-online

by w14 • 2 years ago

52 117 2 years ago

ofcom.org.uk

w14• 2 years ago

- Ofcom sets out more than 40 practical steps that services must take to keep children safer - Sites and apps must introduce robust age-checks to prevent children seeing harmful content such as suicide, self-harm and pornography - Harmful material must be filtered out or downranked in recommended content

fidotron• 2 years ago

It's curious how aligned this is with similar moves in Canada discussed here: https://news.ycombinator.com/item?id=40298552

For those unfamiliar Ofcom is basically the UK telecoms regulator.

johnea3• 2 years ago

No intention of protecting adults?

onemoresoop3• 2 years ago

Adults could potentially protect themselves, children absolutely can't.

com2kid• 2 years ago

When facing off against an army of ML engineers and psychologists?

It is obvious that even adults are suffering negative consequences from social media, and if it isn't obvious, take a look at any of the studies showing the negative impact on adults of social media usage.

Every single consumer is outmatched. The only winning choice is to not play at all.

avgDev• 2 years ago

I'm an adult, history of ADHD. Youtube, facebook, reddit is a disease for me. I am a moderator of some niche subreddits but if I stray away I will find myself stuck reading comments and engaging in conversations for hours.

I take steps by not using my phone during work or after a certain time but it is extremely difficult and I still fall into the trap and watch some shorts for 2-3 hours at a time. I'm not stupid either, graduated with honors in CS and people consider me really smart. My brain just likes how it makes me feel.

Now, I have a fear what all this stuff will do to my child, if even I have trouble with it.

I hate almost everything about the modern commercialized web and really miss irc and forums.

8note• 2 years ago

Children have legal guardians, who are responsible for their safety.

If the adult an protect themselves, they can protect the children they're responsible for too

Dig1t• 2 years ago

As an adult, I do not want to be "protected". I can decide for myself if I like the algorithm or product/service and choose not to use it if I don't like it.

mantas• 2 years ago

The problem is in modern societies the society is footing the bill for those who choose, let's say, unwisely.

Why should my mandatory social and medical insurance taxes go fix issues created by such wrong choices? Should the society not cover them? E.g. smokers with lung cancer should pay out of pocket? Or should society try to minimize wrongdoing instead?

johnea3• 2 years ago

Given that statement, I'm not sure you're fully an adult.

Every single person living in modern society is "protectected":

- You're protected from people deciding which side of the street to drive on

- You're protected from a bank who tells you the money you depisited is "all belong to us"

- even a "market" can't exist without "protection"

Once every person has their own army of ML engineers and psychos at their disposal, then there is an oportunity for personal choice in anti-social media.

Or much simpler, the anti-social corps can be required to allow each user to configure their own feed algorithm. This would be a useful "protection".

The negative consequences of not having _any_ "protection" in anti-social networks are clear across society.

These are analagous to the damage done in the US by lack of gun protection. Guns are now the leading cause of childhood death in the US. Combine guns and 4chan/Discord and you get the rash of idiots shooting up the populace and then themseelves.

gedy2• 2 years ago

We want children to enjoy life online. But for too long, their experiences have been blighted by seriously harmful content which they can’t avoid or control. Many parents share feelings of frustration and worry about how to keep their children safe. That must change.

Yes, stop letting kids stare at screens all day. Yes, you are a bad/lazy parent letting the firehose of the Internet pipe into their heads.

JohnMakin• 2 years ago

I agree with you but the inevitable counter argument is that not letting kids have unfettered access to screens and the internet is impractical to the point of comedy. I personally do not buy this argument, and although I do not have a child, my sister has kept my nephew screen-free so far and he's well at the age where kids start to use them independently. What does he do instead? He reads, he plays games (he's allowed some supervised access to video games), he plays sports. Stuff kids used to do and were totally fine.

The other argument is that this makes kids socially isolated from their peers, because they all have and use these devices. If that truly is the case there definitely still is a way to monitor your kids device and internet consumption without giving them free-reign. There are internet safety settings for parents on every device out there. Kids are perfectly capable of communicating with each other through a variety of mediums, they don't need tiktok (or whatever app is the trendy one of the decade).

The last argument, when you present the former argument, is that kids are clever and will find a way to get around controls. Yea, no shit, that's what kids do. Your role as a parent is to monitor that and teach/correct them.

gedy2• 2 years ago

Yeah I'm not talking about no access ever, I'm talking about the (very sad imho) situation where you see toddlers mindlessly staring at their tablet watching random videos/tiktoks while ignored by parents for hours, even if the mom or dad is nearby and and could be interacting with them.

narenst• 2 years ago

Children at school use a screen for school work starting in middle school (and sometimes even in elementary school). It is very difficult for parents or teachers to always supervise this. I think the adults should educate the children of safe online behavior but like other real world experiences they need have the independence when being online too.

mrmetanoia• 2 years ago

Yes, and stop using screens as a pacifier. People don't like to be told how to parent, but I'd settle for them just parenting. Shoving shit in your kids hand to shut them up isn't parenting.

I used to blame the shitty influencers and internet at large for the selfish greedy brats, but it's their parents. I've met too many exceptions where the difference was just "We limit their screen time." Maybe they end up shitty in some other way, I dunno, but at least they're going to be more functional than my nephews who my SIL is constantly trying to figure our how they get sucked into racism and greed and what to do about it. Stop letting them get all their views from greedy racists online would be a start - can't do that, she'd "never have a moments peace." Idiot human.

Side note, if you let your kids play roblox unsupervised, you're fucking up hard.

ricardo814• 2 years ago

It's quite obvious that Twitter/Google/Facebook/whoever do not have algorithms that scale where they can genuinely curate their content. Seems that obvious since Google bought YouTube.

Isn't it quite obvious that it's never been their prerogative. Nor protecting copyright.

Aerroon3• 2 years ago

And it shouldn't be their prerogative!

If we're going to randomly throw blame around, then why not throw it on the ISP too? They're the ones ultimately serving the content. But I don't think anybody wants to open up that can of worms.

ricardo814• 2 years ago

The blame is not random.

ISPs don't know what content they're serving up other than what host it's coming from 99% of the time.

kwhitefoot2• 2 years ago

The thing that always strikes me in all the reporting and discussion of the problems that Ofcom is trying to solve is that no one seems to ask if the problems are equally bad in other countries, especially in non-English speaking countries. And if it isn't then can, and should, whatever helps there be implemented in the UK?

I live in Norway and it doesn't seem that the problem is so severe here. Or is it simply that English speaking media is more willing to latch on to extreme events and make out that they are the norm?

cynicalsecurity3• 2 years ago

US/UK really like spying on their citizens. Privacy is not their thing.

Their culture regards anything sexual as Taliban regards women not covering their head. As an extreme taboo. As a fellow European, yes, I know, it's crazy.

whimsicalism10• 2 years ago

If you are european, this is basically glass houses and stones

rustcleaner6• 2 years ago

Eric Blair's Anti-Sex League is coming into full swing, innit?

With a century of public policy fuggery, the domestic stock has been induced into a situation where it refuses to effectively breed its next generations in comparable numbers to other competing stocks. It could be mistaken for accidental if the elite weren't then back-stopping tax and investment losses with importing supply from those other stocks.

If The West is purely cultural/ideal, then mere assimilation should be enough for its persistence. If The West is its people then this situation spells certain doom if not arrested.

The West's place at the top of the world is going to be toppled by a nation with less feminism and anti-natalism this century, unless The West can destroy all other effective competition.

red_admiral• 2 years ago

There's a general election coming up in the UK, probably later this year. No date announced yet but they're starting to get TOUGH ON CRIME and other things in preparation.

ricardo814• 2 years ago

Quite likely that most AI is better suited to English and all the big tech companies are heavily into AI with their algos.

wouldbecouldbe• 2 years ago

I think you're just missing it, or not enough media attention.

Youtube is full of weirdass kid content. The ones I've seen are mostly weird / scary, on the border.

It's all garbage, but youtube algorithm recommends it more and more the kids I've seen because it notice the longer screen time.

I wouldn't say it's downright illegal content, but definitely not tasteful.

DaiPlusPlus• 2 years ago

if the problems are equally bad in other countries, especially in non-English speaking countries

Well, Myanmar had a genocide blamed on social-media (at least in-part), so arguably things are worse there, not just equally-bad.

mschuster912• 2 years ago

no one seems to ask if the problems are equally bad in other countries, especially in non-English speaking countries.

Oh hell yes it is. Everywhere that Russia has interest in has been fraught with serious issues. Germany, Poland, France come to my mind - there have been reports concerning especially the spread of far-right and/or pro-Russian content for years now.

I live in Norway and it doesn't seem that the problem is so severe here. Or is it simply that English speaking media is more willing to latch on to extreme events and make out that they are the norm?

You guys are simply too small to matter and have been in NATO from the start. In Sweden and Finland though, I had read reports of pro-Russian propaganda problems when they were in the process of joining NATO.

The unifying link is always Russia.

marcosdumay• 2 years ago

On most countries, the stuff this article is proposing to fight against "for the children" is already illegal to push to anybody.

So, on most countries this proposal would be completely useless.

whimsicalism10• 2 years ago

Is it possible that perhaps people just have differing views on Russia and great power competition than you do? I find the Russia influence story overstated

rustcleaner6• 2 years ago

far right this
far right that
*crickets* far left *crickets*

Man I am glad years ago I watched Yuri Bezmenov's interview with G. Edward Griffin.

whimsicalism10• 2 years ago

The anglosphere media is much more anti-tech than most other countries, so they focus on this stuff - and it’s also a double US/UK election year

drak0n1c• 2 years ago

It's a difference of scale. English is dominant in the number of countries that speak it and is even more dominant among the online population (people from countries that don't otherwise use English who are "overly-online" tend to use it).

There is a larger population of bad actors, a greater variety of underlying cultural/philosophical differences and thus conflict, and the algorithms that seem fine for a smaller contained country like Norway can produce a different quality of topics at a larger scale. It's not just algorithm thresholds either - people are simply more naturally prone to follow fads when there is multinational scale affecting the quantity and rapidity of the content and replies/likes that they see (dopamine and confirmation bias).

Unsure what the solution is - maybe more location-based weighting of suggestions? Conversely you don't want to empower local predators. So far attempts at moderating the entire English speaking world by the standards of SF-cloistered young professionals and PhDs has also been unwieldy and led to backlash.

cynicalsecurity3• 2 years ago

Isn't it the parents' job? Why introduce authoritarianism under the disguise of caring for children?

frereubu2• 2 years ago

The internet is a big place. How am I supposed to manage what Google / Facebook / YouTube etc. serves to my daughter?

devmor2• 2 years ago

Is there a reason you can't supervise your child's time online and/or use appropriate parental control software?

polski-g• 2 years ago

Did someone do that to your internet experience in 1999? Why do you think it needs to be done now?

cynicalsecurity3• 2 years ago

Are you also leaving your daughter unattended in a big city at a McDonald's alone and just go about your business, expecting them to take care of her?

causal4• 2 years ago

This is a pretty weak argument IMO. Try applying this argument to any other regulated context:

- Should McDonalds be allowed to put crack in their happy meals or is it the parents responsibility to keep those meals away from their children?

- Should kids clothing be free of asbestos or is it the parents job to avoid buying that?

- Should baby formula be free of lead or is that the parents responsibility to check for?

If a company is deliberately pushing a product that is harmful and (arguably in this case) addictive to children, that is a problem regardless of the parental role.

jimbob452• 2 years ago

If your problem is with the class of product, then you should know to stay away/keep your kid away. If the product has broken with the class (say, adding asbestos to kids' clothes), then we would want to regulate that. Really, as long as the product is staying within the general confines of the definition of the class, then it's up to the consumer to educate themselves and know better.

For a fun and controversial example, one could look at the RNA vaccines for COVID. Those had some properties that separated them from traditional vaccines such that people relying on the class of "vaccine" might have felt misled. As such, you would have expected government regulation to inform the consumer on the difference to expect in that scenario (which the government did).

mitthrowaway22• 2 years ago

Hmm, I don't think that they're allowed to put cocaine or lead into food marketed for adults, either.

Aurornis6• 2 years ago

People love these proposals until they read the details and think of the consequences. Anything that requires "robust age-checks" means that everyone using the site must go through an ID check and validation process. No more viewing anything without first logging in via your ID-checked account

1. Carry out robust age-checks to stop children accessing harmful content
Our draft Codes expect much greater use of highly-effective age-assurance[2] so that services know which of their users are children in order to keep them safe.
In practice, this means that all services which do not ban harmful content, and those at higher risk of it being shared on their service, will be expected to implement highly effective age-checks to prevent children from seeing it. In some cases, this will mean preventing children from accessing the entire site or app. In others it might mean age-restricting parts of their site or app for adults-only access, or restricting children’s access to identified harmful content.

Before people try to brush aside these regulations as only applying to sites you don't think you use, the proposal is vague about what is included in the guidelines. It includes things like "harmful substances", meaning any discussion of drugs or mushrooms could be included, for example.

Think twice before encouraging regulations that would bring ID checking requirements to large parts of the internet. If you enjoy viewing sites like Reddit or Hacker News or Twitter without logging in or handing over your ID, these proposals are not good for you at all.

whimsicalism10• 2 years ago

This might only be marginally relevant, but California’s digital id has a way of verifying age without revealing anything else about your identity called “TruAge”

throwup238• 2 years ago

I always figured it'd be implemented Stripe style where completing age verification just gives the site a token that they can use to validate the third party age check.

The problem is how to make the provider side anonymized so that they don't know what sites your visiting, but that could be probably solved with legislation. In California, at least. I wouldn't trust Congress with a bill like this.

brevitea• 2 years ago

Agree. The idea is to verify your age, not harvest all your PII data across every login and viewing session. Companies can easily implement this privacy-preserving step, they just won't until it is strictly enforced.

Aurornis6• 2 years ago

California’s digital id has a way of verifying age without revealing anything else about your identity called “TruAge”

I'm not familiar with the system, but I assume it would necessarily have to reveal the sites you're verifying with to the State of California.

So it's less of a big deal, as long as you're okay with sending a record to the government about what site you're visiting every time you want to sign up somewhere or re-verify your age.

I'm sure someone in the comments will propose some cryptographic solution where neither party knows anything other than the fact that someone, somewhere, possesses a token associated with a person over the age of 18. If you think this is viable, you're not thinking like a kid trying to get around this system, nor a blackhat trying to take advantage of it: Many people would immediately set up a service that handed out age verification tokens in exchange for viewing some ads (the file sharing site model) if there were no limits and nobody could trace it back to the source. Any ID verification system must necessarily have some party able to verify the person to avoid abuse like this.

A4ET8a8uTh05• 2 years ago

Oddly, I think I agree with you. What we really need, and that is the reason why it does not even enter the conversation, is standard default that we can agree upon.

By this I mean, no more fucking around with timeline. We agree that any corporation introducing a feed based on anything other than date from newest to oldest is subject to penalties and sanctions.

Will it stop 'innovation'? God I hope so. I am tired of innovation that farms up rage.

edit: From here we can start working on what algos CAN be included in customer facing crapola.

whimsicalism10• 2 years ago

maybe we shouldn’t use government intervention and force as the default in all of these things.

infecto• 2 years ago

Such a rule is impossible to write.

jorvi• 2 years ago

Already happening in the EU. Give your phone number, credit card number, or government ID number if you want to watch age-gated YouTube.

The best solution would be a government or banking API that emits a one-time token. No logging, and it self-destructs upon verification.

But aside from the user, no other party has detriment from the current situation.

s1artibartfast8• 2 years ago

That's a really interesting idea. I was just wondering how any of this could be prevented in a way that preserves user choice.

rustcleaner6• 2 years ago

These kinds of proposals will hand the 'net to the darknet [its proper successor]. I can't wait for Freenet/Hyphanet to eventually get onion routing!

manlobster• 2 years ago

It's not clear to me that every site would have to perform age verification themselves. Seems like the "I am a minor" flag could be managed at the client operating system level (e.g. as a property of the cell account on mobile devices, or as a property of the user account on a laptop or desktop machine), and transmitted per-request (e.g. in a HTTP header).

Aerroon3• 2 years ago

Then we would have mandatory online-only accounts for the OS "for your safety". This sounds even worse.

ricardo814• 2 years ago

Agree in your vision of the ramifications, but I feel that it's their response to a basically unregulated big tech. Their powers are limited.

dyauspitr• 2 years ago

Who cares. I want the internet to be an ecosystem without any privacy (which is a drastic change from what I believed for most of my life). I believe anything of value will be available even without anonymity. For something that absolutely needs anonymity people will find workarounds and I appreciate the extra technical barrier when they have to do so so it’s not available to very pleb.

1vuio0pswjnm7• 2 years ago

The so-called "tech" companies love to tell the world, especially their advertiser customers, how they know "everything" about the people who use their websites, with ridicuouls claims such as knowing more about users than users' own friends and family. Certainly the knowledge they claim to have would include the age. If not, then any claims by so-called "tech" companies that they can serve targeted advertising to people in a certain age bracket are false.

Whereas if their claims are true, and they do know the age of their website users, then these so-called "tech" companies can solve this problem without needing to do age verfication. By not targeting people in certain age brackets with certain content, they can stop the politicians from proposing legislation that requires age verification. But they refuse to do so.

Interesting.

em-bee2• 2 years ago

wouldn't they have to guarantee no false positives ever? so any child being misidentified as an adult would be a problem. the result would be that the targeting is so strict that to many adults would be excluded. and that matters even more if we consider that young adults are among the most lucrative demographic, which makes false positives more likely.

you can easily tell a 30 year old from a 10 year old. but can you tell a 12 year old from a 15 year old? or a 15yr old from a 20yr old?

so tech companies want a system that is approved so that they are not responsible if it fails

sudofail• 2 years ago

In my view, we need legislation to step in and enforce some level of algorithmic tuning. Modern algorithms drive engagement at all costs, regardless if it's healthy for the individual. I want to be able to tune the algorithm to potentially use a timeline feed instead, or limit content to only come from topics I subscribe to, etc. We probably need parental controls that allow parents to enforce algorithm tuning as well.

A recent example of an algorithm going wrong is Reddit. Home used to show you strictly a feed of reddits you subscribed to, and it was shown as a timeline. The most recent changes not only removed the timeline approach to the feed, it's now injecting subreddits you don't subscribe to and asks if you're interested in them.

causal4• 2 years ago

Something important to keep in mind: most people never experience just how twisted these recommendation algorithms can get, because each of us gets an experience tailored to our developed tastes.

But these algorithms will totally curate wildly disturbing playlists of content because it has learned that this can be incredibly addicting to minds unprepared for it.

And what's most sinister is how opaque the process is, to the degree that a parent can't track what is happening without basically watching their kids activity full-time.

Idk if OFCOM is implementing this right or not, but I think there would be a much greater outcry if more people saw the breadth of these algorithms' toxicity.

geor9e• 2 years ago

If, instead of a machine algorithm, we wired newborn human brain in a jar and "trained it" to choose the next thing to show in a feed, to get nothing but reward for when someone clicks/comments/lingers on something, and nothing but punishment when they leave, plus the basic human functions of walking and talking etc, then put that brain into an adult human body, what would we describe that person as? Probably a psychopath with no moral compass? A sicko willing to do indescribable horrors? Evil incanate? A calculating manipulator for sure. But somehow when we "train" a machine model with singular goal of profit alignment, people think that's just the free and efficient market. And the idea of aligning the models with human good instead is seen as overregulation.

deadbabe2• 2 years ago

What if we just ban all recommendation systems?

mitthrowaway22• 2 years ago

You'll have to be more specific... Does that ban cover everything down to alphabetical ordering?

deadbabe2• 2 years ago

If the user did not actively specify what they want to search for or subscribed to a content source, it is banned.

kwhitefoot2• 2 years ago

Just forbid recommendations altogether. Let the user subscribe to posts by specific users.

yawboakye• 2 years ago

next week: music studios must tame toxic lyrics to protect children.

surfingdino• 2 years ago

Interesting... if you sob and moan to YT or Instagram about not having enough followers or views they'll tell you to replace the word "algorithm" with "audience", so people. It makes sense, if your content is not popular with people no algorithm will surface it (recent tweaking of Instagram algo notwithstanding). But if we follow that interpretation we have to admit that it's not the algorithms that are toxic, but people. So what Ofcom is asking tech companies is to "tame" toxic people. Good luck with that. Parents have to realize that computers, phones, or tablets help sometimes unsavoury characters get in touch with their children. We do not allow strangers into daycare centres, schools, or children's hospitals, so why do we allow strangers unrestricted access to our children via the devices we give them? Parents need to be told to take responsibility for who has access to their children.

eddof13• 2 years ago

No. The parents are responsible for their children, not tech firms or anyone else.

Digit-Al• 2 years ago

Personally, I am against the idea of adults having to prove their age before being able to access certain types of content - particularly if that means giving their identity. I am not, however, adverse to the idea that big tech companies should be more responsible for what they are serving to youngsters.

Yes, I know their are plenty of tools to allow parents to restrict what sites their children visit, etc... but not all parents are tech savvy enough to be able to set this stuff up, plus you could still allow a child to access Youtube, for example, but then find they are getting unsavoury recommendations from the algorithm.

This made me think about the fact that the major platforms (Alphabet, Amazon, Apple, Meta, and Microsoft) gather enough data on their users that they almost certainly know roughly how old someone is, even if no age has been provided to them. They can use all the signals they have available to provide a score for how certain they are that an individual is, or is not, legally an adult.

(As an example, if you have a credit or debit card in your Google or Apple wallet then you are almost certainly an adult because it would be very difficult for a child to obtain a card and get it into a digital wallet due to the security procedures that are in place.)

Given that, if these companies get forced to discern whether users are adults or not in order to serve appropriate content then it seems a no brainer for them to provide free age verification as well.

My vision would be for the UK government to provide an anonymised age verification router service. When a website requires you to verify your age in order to access some particular content it could ask you which age verification service you wish to use. It then sends a request to the government "middleman" that includes only the URL of the verification service. The router forwards the request anonymously to the specified server (no ip address logs are stored). If you are logged in to the account already then it will immediately return true or false to verify that you are or are not an adult. If you are not logged in then you will be prompted to login to your account with the service and then it will return the answer. The government server will then return the answer to the original website.

That way, we can get free, anonymous verification.

I'm sure people will have issues with this idea, such as "do you trust the government server to not log details fo your request instead of being anonymous?" - to which I do not have a definitive answer, but I feel like it is potentially a little better than having Google or Facebook knowing what sites I am visiting that need verification.

Anyone out there have any thoughts on this? I have only just had the idea pop into my head, so no serious thought has gone into it. There are probably issues that I have not thought about.

az09mugen• 2 years ago

I have a better solution : tech firms must stop using toxic algorithms for everyone, not just childrens. Why are they allowed to use these practices in the first place ? Why do we have to endure/tolerate this stuff that makes internet a worse place ?

Tech firms must tame toxic algorithms to protect children online

Related Stories