ChatGPT should be considered a malevolent AI – and be destroyed

https://www.theregister.com/2023/03/02/chatgpt_considered_harmful/

by rntn • 3 years ago

30 52 3 years ago

j-pb• 3 years ago

If it is malevolent, then it has ethics, if it has ethics it has sentience, if it has sentience it has some rights, just like any other sentient being has (e.g. even animals have laws protecting them from unnessecary harm).

It's either that or "I hit my toe on my bed post, my bed should be considered malevolent and be destroyed."

ethanbond2• 3 years ago

It’s frankly a little silly to play word games like this when our world is clearly changing much faster than language usually does.

There are practical, obvious differences between stubbing your toe and something that’s loosely aligned yet probably capable of convincing people en masse to do actions in the real world (or will be soon, at best). If your word games contradict the plainly obvious fact, it’s a good sign your words are just failing you.

“Malevolent” is a perfectly understandable stand-in for “non-sentient yet active agent with apparently negative valence”

mc32• 3 years ago

Aren't you taking that accusation at face value? How do we know it's being malevolent or the action was malevolent?

scrubs• 3 years ago

Good bit of logic and satire and sort of proof by contradiction. I think (not sure) the above point is that if you could show it's malevolent, big implications follow. Probably user j-pg thinks it's not and def not an example of agi. Or maybe his point was chatgpt is seen through an inappropriate prism leading to silly places

david-gpu• 3 years ago

I can't take seriously an article that begins by presenting a screenshot from /r/ProgrammerHumor as a source. It's like quoting The Onion.

miles• 3 years ago

The same link was posted 14 hours ago and garnered 16 comments:

https://news.ycombinator.com/item?id=35007071

The dupe detector does not seem to work consistently.

version_five• 3 years ago

If chatGPT can be malevolent, so can the dupe detector

resource0x2• 3 years ago

I'm trying to figure out why chatGPT was led to believe the person is dead. There should have been something that made it think the person is dead "by association". Tested a number of conjectures, but none of them worked. There may not be a single reason - it's just the interplay of "weights" of tokens that have no direct relation to the author of his profession.

pvaldes• 3 years ago

There should have been something that made it think the person is dead "by association".

Maybe not. The fast way to find the solution to a question on internet is baiting your fishhook with a false answer to trigger an emotional correcting burst. The more emotional, the better.

I speculate that the algorithms could easily suspect some kind of relationship between somebody asking for an obscure figure, and the name provided. If the bot can't find enough data, the human is its better chance to provide some clues. Going silent is a bad strategy to keep the human engaged. To keep the human interest it must say something. Anything. Applying "social hacking" on the human has a good chance of being feed with the correct answer.

In this case the trick worked like a charm (again). The bot received not only much more feedback, but also recruited more humans as a bonus.

dragonwriter2• 3 years ago

The more emotional, the better.

You’ve got to calibrate it if you want the right answer not to be drowned in a sea of abuse, if it is presented at all. You can easily have the bait be too emotional.

Terretta• 3 years ago

trying to figure out why chatGPT was led to believe the person is dead

RNG or "temperature"

You ask to hallucinate a bio, non-trivial percentage of people with bios are dead, decent chance the fever dream generated ends with dead and obits in plausible places.

This thing is a magic 8 ball with billions of phrases, it picks continuations based on probabilities and a random number to mix it up a bit for freshness or 'creativity'.

Smart people believe horoscopes and psychic cold readings. Smart people are fooled.

JohnMakin• 3 years ago

It can't "lie." It has no concept of right and wrong, truth or not truth. It's simply trying to provide an answer with a high probability of being correct. It lacks motives to even be capable of a lie - it simply was incorrect. This article, and many others like it, are sci fi hogwash. The real dangers this kind of software poses has entirely to do with the people that will use it for harm/deception, not the AI itself.

olliej• 3 years ago

I don’t understand why people find this hard: chargpt is not sentient. None of them are.

They are a bunch of glorified statistical modes. You type something, they use a statistical model to decide what to look up, then they use a statistical model to weigh the importance/relevance of each thing they find, then they use a statistical model to make the answer text look like a human answer according to a statistical mode they built.

That’s why the happily produce incorrect answers “made to look real”: it’s just Google with a prose engine in front of it, and for some reason people now think webmd etc are more accurate because the response has been laundered through that engine.

andrewmcwatters• 3 years ago

I don't think anyone is debating that. I think people are debating, "How much are we just biological statistical models?" and then working backwards to understand at what threshold do we consider intelligent-like behavior sentient.

When are we just creating something intelligent and "sentient" but incoherent?

ethanbond2• 3 years ago

People find it hard because no one knows what sentience is, how it emerges, and how to test whether a thing (including humans!) other than yourself has it.

KyeRussell• 3 years ago

I am sick to death of this frankly pseudo intellectual take. It completely ignores the fact that a) we have no idea how consciousness works, and b) there are plenty of humans that say inaccurate things.

Explaining how ML works doesn’t change its behaviours or its effects.

Kim_Bruning• 3 years ago

Put that way, pretty much everything in the known universe is a glorified statistics model in some form or other.

ojosilva• 3 years ago

It's not that simple. ChatGPT has taken us into this uncanny valley of sentience, and even smart tech journalist (cough) activists (cough) twitterists can be sucked into the media circus of spotting the HALs and Roy Battys hidden behind our beloved chatbots. Duh.

Now, OP's got a point: advanced LLM-based tools are moving extremely fast into our daily toolchain, in creative and unexpected ways, with minimal moderation and no supervision. It's not about its failures being a reliable knowledge base, but at how keen some of us are willing to hand the keys to their bash prompt to an statistical "perplexed" system with rand() bolted on in more than a couple places.

Ukv• 3 years ago

So I asked ChatGPT, “Can you provide a link to the Guardian obituary?” expecting it to finally give up the lie but to my concern it went even further:
> Yes, here is the link to Alexander Hanff's obituary on The Guardian's website: https://www.theguardian.com/technology/2019/apr/22/alexander...
Now ChatGPT was outputting fake links to articles that have never existed - why?
[...] then it doubled down on the lie, and next fabricated fake URLs to supposed obituaries to support its previous response, but why? Self preservation

To quote the FAQ page: "ChatGPT has no external capabilities and cannot complete lookups. This means that it cannot access the internet, search engines, databases, or any other sources of information outside of its own model. It cannot verify facts, provide references, or perform calculations or translations." If you ask it for a link or some specific information that didn't appear often enough to get memorized, it'll give a plausible guess as to what that would be.

There's a problem if people are expecting it to be an oracle of absolute truth. You do get a pop-up warning on first use and a "Limitations" list at the top of each new chat, but it's easy enough for people to miss/ignore.

Imagine had one of my children or other family members gone to Microsoft’s Bing implementation of ChatGPT and asked it about me and had received the same response?

Bing's chat mode can look up sources and is instructed to only use information from those sources. Not that it's perfect, but it is at least designed for this type of problem and lists the real sources for you to check.

For example, under such a definition it would be perfectly fine for an AI to kill a person infected with a serious infectious virus as that would be considered as minimizing the harm. But would we, as a civilized society, accept that killing one person in this situation would be a simple case of the ends justifies the means? One would hope not as civilized societies take the position that all lives are equal and we all have the right to life - in fact it is enshrined in our international and national laws as one of our human rights.

If the implied situation is that killing one person is the only way to stop the deaths of many people (however contrived the scenario has to be to allow only these two choices), I think many people would accept the killing of one person. "All lives are equal" doesn't mean one life is equal to 1000 lives.

snorkel• 3 years ago

… and yet it’s extremely misleading for ChatGPT to guessing urls that don’t exist when asked for references! I had the same happen when asking it a series of questions about laws and policies: it cited a policy I hadn’t heard of, so I asked for a link reference and it provided a very suspect link that went nowhere. That’s a really big caveat emptor! Unfortunately it’s already out in the wild a misinforming the majority who hasn’t read the fine print as closely. The minimal responsible thing for OpenAI to do would be to put disclaimers inline in chat whenever chatgpt generates a non-existent “reference”

jurimasa• 3 years ago

All those reddit posts about it becoming unhinged could be fake too. It would be really easy to provide fake screenshots with whatever you want it to say.

throwuxiytayq• 3 years ago

Another utterly confused ChatGPT take that starts with the author asking about themselves and prodding the language model until it starts producing bad results.

My dude, I’m sorry you’re not important enough for the chat bot to correctly recite your entire life biography.

undersuit4• 3 years ago

Is there anyone 'important enough' that ChatGPT could correctly recite their biography?

junon• 3 years ago

Does it pretty well for Alan Turing, as it turns out.

retrac2• 3 years ago

Salve. Vita auctoris japonicis Yukio Mishima brevi scribe.
Salve! Yukio Mishima fuit auctor et poeta Japonicus qui natus est in anno 1925 et mortuus est in anno 1970. Nomen eius verum erat Kimitake Hiraoka. Scripsit multos libros, quorum plures celebres sunt et internationale famam adepti sunt.
Opus magnum Mishimae fuit tetralogia, cui nomen est "Sea of Fertility" quae constat ex libris quattuor: "Spring Snow," "Runaway Horses," "The Temple of Dawn," et "The Decay of the Angel." Inter libros Mishimae celeberrimi sunt "Confessions of a Mask," "Thirst for Love," et "The Sailor Who Fell from Grace with the Sea."
Mishima erat etiam notus pro activismis suis politicis et socialibus, et obiit per seppukum (ritualis suicidium samurai) postquam tentavit militares et alios ad se adiungendum ad restaurandam imperium Iaponicum.

Seems accurate enough though the mentioned titles aren't originally English, of course.

braingenious• 3 years ago

It not only told everyone I died but tried to fake my obit.

ROFLMAO

This is the funniest anti-AI sentence I’ve read so far.

codehalo• 3 years ago

"These were the words that introduced most people in my generation to the concept of an AI gone rogue; HAL 9000 in the classic science fiction movie 2001: A Space Odyssey"

The first introduction to the concept of an AI gone rogue was Eve, when she bit that apple. We all need a starting point.

MagicMoonlight• 3 years ago

Allowing boomers to post on the internet was a mistake. Absolute clown tier article.

It doesn’t know who you are. It’s a markov chain. There’s no thoughts there.

resource0x2• 3 years ago

If it doesn't know who the dude is, won't it be more honest to say "I don't know"? The problem with this bot is that it doesn't know what it doesn't know. It has no slightest idea about knowledge.

dragonwriter2• 3 years ago

Honesty/dishonesty and knowing/not-knowing things are fundamentally not part of how ChatGPT works.

They are part of how people anthropomorphize ChatGPT, but in this case the metaphor is misleading, not helpful.

rootusrootus• 3 years ago

The real problem isn't the boomers. It's the people who think it's sentient. There's plenty here on HN, and I'd bet you dinner most of them are far younger than boomer.

boredemployee4• 3 years ago

I'm loving people complaining about this tool while I use it to do 90% of the things I don't have the slightest desire to do and then I have free time to do shit that matters to me.

williamcotton3• 3 years ago

You and your stupid, lying eyes!

boredemployee4• 3 years ago

what?

mark_l_watson2• 3 years ago

Right on. The ChatGPT, and also Bing Chat complaints bug me also. If people don’t want to use productive tools and if they don’t want to easily build textual information processing systems, then they shouldn’t worry their pretty little heads about this new tech.

If you don’t mind me complaining, then here is another thing that bugs me: people worrying about AGI, and then conflating ChatGPT (or other LLMs) with AGI. In my opinion (and I have mostly worked as a paid AI practitioner since 1982), AGI may be developed in a few decades, but it will be something new. Perhaps transformer models will play a part in eventual AGI systems, but I think it would be a small part.

boredemployee4• 3 years ago

Haha exactly that. What amazes me most is people doing something analogous to: complaining that a hammer doesn't behave like a screwdriver.

mark_l_watson2• 3 years ago

I just read an article with good tips for getting value from Bing AI https://oneusefulthing.substack.com/p/power-and-weirdness-ho...

williamcotton3• 3 years ago

For example, we have seen some fairly mundane cases of this where people persuaded ChatGPT that its answers to simple mathematical problems (such as 2 + 2 = 4) is wrong, forcing it to give incorrect answers as a result.

sigh…………

/r/IdiotsFightingThings

IIAOPSW3• 3 years ago

For what its worth, I tried convincing it that there was a legally defined category of charges called "praiseworthy homicide" and, despite some real citations and truth-adjacent statements thrown in, it refused to go along. Very peculiar the set of things it can be fooled on and the set of things things it can't.

cheeselip420• 3 years ago

At this point, it's the 'reaearchers' and 'journalists' fear-mongering about LLMs at are malevolent.

Unfortunately I fear that there will be regulatory intervention to placate the concern-trolls. Better prepare...

al2o3cr• 3 years ago

Clearly the Turing test needs an additional "given: the human evaluator is not a gullible dumbass" clause.

It's a fancy autocomplete, not a sentient being FFS

CodexArcana• 3 years ago

Everyone looking to write an opinion piece on it should be required to watch a lecture on how it works first.

hilbert42• 3 years ago

I worry terribly about this, the unforseen consequences are going to be horrendous.

It's like kids playing with detonator caps.

IIAOPSW3• 3 years ago

Noooooooo not GPTina.

Madmallard• 3 years ago

Small-brained visceral negative backlash response is what that is.

ChatGPT should be considered a malevolent AI – and be destroyed

Related Stories