Three governments agree on something the AI industry doesn't want to hear

https://www.theattachmenteconomy.com/p/three-governments-agree-on-something

by mikelgan • 7 days ago

34 15 7 days ago

mikelgan• 7 days ago

The governments of China, California and New York agree on three harmful aspects of companion AI chatbots.

elmer2• 7 days ago

"The logic is that free, user-built companions inside mass-market general-purpose apps are impossible to moderate at scale."

I think they misspelled censor.

watwut• 7 days ago

Not really. They are censored easily and successfully.

However, moderation is hard and the companies are motivated to moderate them toward maximum outrage with some plausible deniality on top of it.

holgerschurig2• 6 days ago

Na, it's just that people these days aren't that educated anymore.

A censor was a government employed entity where you had to present your text BEFORE you could publish it. Book, newspaper articles, things like. It was in heavy use in the European kingdoms and republics (e.g. during Napoleons reign).

You had no choice there: would you not censor and publish, then hefty fines would be one you. Sometimes even leading to death, e.g. look up "Der Hessische Landbote" from Georg Büchner, a tract against monarchy and feudalism.

So censorship has two keys: mandatory prior check, fines if you publish a text without that.

Moderation however is to keep your channel in a sane tone. It is something you do out of your own will, e.g. to cut down on flaming. And there is no government entity involved that checks the text in question. It is, after all, not identical to censorship.

My guess is that anarchist minded people, that (wrongly) believe that human groups develop organically in a nice way, have a problem with moderation. And then they seek some word with an extremely bad connotation and just use that instead. Just like some politicians use these days "terrorist" in plethora, even when "murder" would describe it better.

IMHO users of "censorship" term often want to manipulate us.

dTal• 5 days ago

People are most certainly not less educated than in the 1800s.

Meanings do indeed shift, especially when terms become euphemisms for other less savory terms. "Moderation" sounds much nicer than "censorship", and you may be right to fight a rearguard action at preserving the nuances of each term, but in the end they're both subsets of "information control". People say "moderation" when they want to imply that the control is good, and "censorship" when they want it to sound bad.

Neither of your definitions of "censorship" amd "moderation" fit cleanly into a discussion about controlling the dialogue between a human and their private LLM. There is no publishing of information, and no channel whose tone must be maintained. It is about control of what the LLM says in private - in particular, it is about preventing it from saying certain things, before it says them. It is close to "censorship" by your definition, but because the thing being censored is not human, we need not speak of fines, or pre-publication vetting - it is simply a question of controlling the output of a computer program, impossible without a total lockdown on general purpose computing.

I think we should default to "information control is bad" and only allow exceptions in well supported exceptional circumstances, because the consequence of allowing systems more control over information than humans is that the systems fail to reflect the interests of the humans that constitute them, in favor of their own self preservation.

chews• 7 days ago

With respect to GDP the #2, #4, and #8 all agree that artificial companionship is probably bad for humans should tell you something.

g42gregory• 7 days ago

It begs the conclusion that these 3 are similar Governments in the way they govern.

Havoc• 7 days ago

I liked this article. Does a good job of framing some of the concerns around this in clear language.

e.g.

Attachment Economy, which is an extractive business model that uses AI to lure people into forming emotional attachments to chatbots

jhbadger3• 6 days ago

Eh, this hysteria reminds me a lot of the early 1980s when people seriously believed that role-playing games like Dungeons & Dragons were bad for mental health - one of Tom Hanks' early roles was in "Mazes & Monsters", a 1982 TV movie that supposedly exposed the sanity-destroying nature of RPGs. Now it looks just silly. Maybe, just as we now get that people can enjoy pretending to be an elf wizard without going insane, others enjoy pretending that a chatbot is their boyfriend and girlfriend and still be well adjusted members of society.

soulofmischief2• 6 days ago

It's important to ground ourselves in the differences: https://en.wikipedia.org/wiki/Deaths_linked_to_chatbots

Not to say D&D hasn't surely been linked to a few deaths as well... get a large enough sample and you'll see just about anything.

jhbadger3• 6 days ago

The plural of anecdote is not data - and it was exactly these sort of anecdotes that led to the anti-D&D hysteria as well. Often something else was going on - for instance "Mazes & Monsters itself was very loosely based on an incident where a child prodigy who was known to be an avid D&D player commited suicide. The media was quick to blame D&D, but further investigation revealed he was suffering depression because he realized he was gay and didn't know how to tell his conservative religious parents.

krapp• 6 days ago

Few if any of the people playing D&D actually thought they were elf wizards, or that they were doing magic, or communing with eldritch gods.

Meanwhile many if not most of the people who have chatbots as their significant other actually have that relationship, and believe it to be as real as a relationship with a human being.

The former are playing a game for entertainment, and suspending their disbelief, the latter are using a machine to satisfy emotional and psychological needs in ways that anyone would consider harmful, and having their beliefs altered by proprietary commercial software sometimes past the point of psychosis.

jhbadger3• 6 days ago

I think there needs actual statistical evidence (rather than sensationalist hysteria) that very many people think their chatbot lover is real. Before chatbots, there were (and still are as far as I know) phone sex lines where people paid to talk dirty to an actual person. There isn't any reason to believe that most users of sexy chatbots see them as any different from those.

Three governments agree on something the AI industry doesn't want to hear

Related Stories