Bing AI can't be trusted

https://dkb.blog/p/bing-ai-cant-be-trusted

by dbrereton • 3 years ago

1072 576 3 years ago

dkb.blog

beebmam• 3 years ago

I already don't trust virtually any search results except grep/rg.

quickthrower2• 3 years ago

grep is not gigo immune

imranq• 3 years ago

Unfortunately this overhyped launch has started the LLM arms race. Consumers don't seem to care in general about factuality as long as they can get an authoritative sounding answer that is somewhat accurate...at least for now

SketchySeaBeast6• 3 years ago

This hasn't really been put in front of consumers, has it? This is all very niche - how many even know that there is a Bing AI thing going on? I think it's far too early to make statements about what people think or want.

teraflop• 3 years ago

Right now, if you go to bing.com, there's a big "Introducing the new Bing" banner, which takes you to the page about their chatbot. You have to get on a waitlist to actually use it, though.

Tepix• 3 years ago

OpenAI raced past 100 million users, that's hardly niche. All tech people i've talked to have played around with it. Some use it every day.

Nathanba• 3 years ago

Unfortunately? This is the best kind of arms race, the one where we race towards technology that is ultimately going to help all of humanity.

SketchySeaBeast6• 3 years ago

I'm trying to decide if this is a valid arms race or jumping the gun. Kind of feels like if someone came up with auto racing before the invention of the ICE and so they spend a bunch of time pushing race cars around the track only for them all to realize this isn't working and give upon the whole idea.

wg0• 3 years ago

Somewhat opposite - if LLMs continue to perform like that with made up information and such, their credibility would erode over time and a defacto expectation would be that they don't work or aren't accurate which would result in less being reliant on them.

Same like self driving cars didn't have a mainstream breakthrough yet.

arduinomancer• 3 years ago

I think its more like Tesla Autopilot

In the beginning there was lots of hype because you couldn't use it

But now that its in consumer hands there's tons of videos of it messing up, doing weird stuff

To the point that its now common knowledge that autopilot is not actually magical AI driving

eppp• 3 years ago

Bing AI gets a pass because it's disruptive. Google doesn't because it is the incumbent. Mystery solved.

low_tech_love• 3 years ago

A couple of weeks ago I said it makes sense to be skeptical and critical of new technologies, especially when they are made by big people, and was criticized for this. I think you hit the nail on the head. The problem is that technology is not only what it is, per se, but also what we want it to be. So people want to believe, much more than they actually need the thing in practice. And the people who build the technology are aware of this, and make use of it for their benefit. In some instances, the market is far from being a competition based only on skills and product quality. There is a lot of fantasy, too.

airstrike2• 3 years ago

I mean, it's in beta and it's not really intelligent despite the cavalier use of the term AI these days

It's just a collage of random text that sorta resembles what someone would say, but it has no commitment to being truthful because it has no actual appreciation for what information it is relaying, parroting or conveying.

But yeah, I agree Google got way more hate for their failed demo than MS... I don't even understand why. Satya Nadella's did a great job conveying the excitement and general bravado on his interview on CBS News[1] but the accompanying demo was littered with mistakes. The reporter called it out, yet coverage on the press has been very one-sided against Google for some reason. First mover advantage, I suppose?

----------

1. https://www.cbsnews.com/news/microsoft-ceo-satya-nadella-new...

LesZedCB3• 3 years ago

because people see it as a David and Goliath, even though that characterization is comically inaccurate

salt-thrower• 3 years ago

I would guess that the average person has higher expectations for Google. Bing has been a bit of a punchline for years, so I don't think most people care as much.

Mountain_Skies2• 3 years ago

As far as I know Microsoft's CEO hasn't done a demo that went wrong like happened with Google. So far, from what I've seen, it is users testing Bing to find errors. The outcome, that they're both giving poor results, is the same, but with a company CEO and a live demo involved, it's always going to get more attention than someone on Reddit putting the product through its paces and finding it lacking.

A Microsoft executive declined CBS News' request to test some of those mechanisms, indicating the functionality was "probably not the best thing" on the version in use for the demonstration.

Microsoft apparently isn't acting from a position of panic, so they have been savvier with how they've presented their product to the media and the world. Google panicked and set their CEO up for embarrassment.

airstrike2• 3 years ago

As far as I know Microsoft's CEO hasn't done a demo that went wrong like happened with Google.

Right before the interview, the reporter was testing it out with the Bing AI project manager (I think, can't recall her exact role) and it was giving driving directions to places that were either in the wrong direction or to an entirely made up location

JackC• 3 years ago

As far as I know Microsoft's CEO hasn't done a demo that went wrong like happened with Google.

Close enough -- the parent article we're discussing is about errors in screenshots from a demo by Yusuf Mehdi, Corporate Vice President & Consumer Chief Marketing Officer for Microsoft. The first screenshot appears in the presentation at 24:30.

visarga5• 3 years ago

The potential for being sued for libel is huge. It's one thing to say the height of Everest wrong, another to falsely claim that a vacuum has a short cord, or that a company had 5.9% operating margin instead of 4.6%.

layer82• 3 years ago

Yep, it will be interesting to see how the legal liability aspect will play out.

egillie2• 3 years ago

It might actually be smart of google to let microsoft take the brunt of this first...

eatsyourtacos3• 3 years ago

I don't see how this can be true at all in the search engine context or even chatGPT where you are asking for information and getting back a result which may or may not be true.

It would be different if an AI is independently creating and publishing an article that has false information.. but that's not the case. You are asking a question and it's giving it's best answer.

I'm not a lawyer by any means, so someone please give a more legal distinction here. But if you asked me what the operating margin of company X was, and I give you an answer (whether I make it up or compute it incorrectly), you or the company can't sue me (and win) for libel or anything of the sort.

So I'm not sure the potential is as big as you think it is.. that's like saying before any AI you can sue google because they return you a search result which has a wrong answer, or someone just making shit up. That's not on them- it's literally indexing data and doing the best it's algorithm can do.

It would only be on the AI if you are literally selling the use of the AI in some context where you are basically assuring it's results are 100% accurate, and people are depending on it as such (there is probably some legal term for this, no idea what it is).

crazygringo3• 3 years ago

But if you asked me what the operating margin of company X was, and I give you an answer (whether I make it up or compute it incorrectly), you or the company can't sue me (and win) for libel or anything of the sort.

If you're a popular website and you intentionally publish an article where you state an incorrect answer that many people follow and make investment decisions about, the company absolutely can sue you and win.

In the courts, it will ultimately come down to to what extent Microsoft is knowingly disseminating misinformation in a context that users expect to be factually accurate, regardless of supposed disclaimers.

If Microsoft is leading users to believe that Bing Chat is accurate and chat misinformation winds up actually affecting markets through disinformation, there's gigantic legal liability for this. Plus the potential for libel is enormous regarding statements made about public figures and celebrities.

adamckay• 3 years ago

But if you asked me what the operating margin of company X was, and I give you an answer (whether I make it up or compute it incorrectly), you or the company can't sue me (and win) for libel or anything of the sort.

If your answer damages company X then they can sue you. If you Tweet that a vacuum cleaner is terrible because its noisy to your 4 followers it's probably not a big deal as (under UK law, and I'm assuming similar internationally) a company has to prove "serious financial loss". If you write about it on your Instagram that has millions of followers then that's more of an issue, so you can assume a search engine claiming to summarise results but apparently hallucinating and making things up is liable for a defamation suit if it can be demonstrated to harming the company.

greenflag• 3 years ago

Likely going to be a wave of research/innovation "regularizing" LLM output to conform to some semblance of reality or at least existing knowledge (e.g. knowledge graph). Interesting to see how this can be done quickly enough...

visarga5• 3 years ago

Probably the hottest research trend in 2023. LLMs are worthless unless verified.

whimsicalism11• 3 years ago

Really? I already get a huge amount of value out of LLMs even if they hallucinate.

Or is this just HN tendency towards hyperbole?

visarga5• 3 years ago

coming back with a link: https://mobile.twitter.com/ylecun/status/1625554772098002944

this tween from Yann LeCun came after my message was posted

kneebonian3• 3 years ago

Likely going to be a wave of research/innovation "regularizing" LLM output to conform to some semblance of reality or at least existing knowledge

This is a much more worrying possiblity, as there are many people who have at this point chosen to abandoned reality for "their truth" and push ideas that objective facts are inferior to "lived experiences". This is a much bigger concern around AI in my mind.

“The Party told you to reject the evidence of your eyes and ears. It was their final, most essential command.” ― George Orwell, 1984

vore• 3 years ago

As fun as quoting 1984 is, there is a huge gap between that and just not making up the winner of the Super Bowl so confidently.

mvcalder• 3 years ago

It will be interesting to see what insights such efforts spawn. For the most part LLMs specifically, and deep networks more generally, are still black boxes. If we don't understand (at a deep level) how they work, getting them to "conform to some semblance of reality" feels like a hard problem. Maybe just as hard as language understanding generally.

perrohunter• 3 years ago

Why are we not rooting for the search underdog? When google owns 92%+ of the search market, any competition should be welcomed

aabhay• 3 years ago

Yes, Microsoft the poor underdog.

Mountain_Skies2• 3 years ago

Duopolies are bad but not quite as bad as a monopoly.

VWWHFSfQ• 3 years ago

Are suggesting that we should root for and accept blatantly misleading, false, and probably harmful search results just because they're the "underdog"

visarga5• 3 years ago

Waiting for GPT-4 to take over.

nivenkos• 3 years ago

GPT3 isn't search.

weberer3• 3 years ago

Its weird that I always see this exact comment whenever Microsoft is trying to break in to a market, but I never see it when its any other company.

Shank• 3 years ago

Before the super bowl, I asked "Who won the superbowl?" and it told me the winner was the Philadelphia Eagles, who defeated the Kansas City Chiefs by 31-24 on February 6th, 2023 at SoFi Stadium in Inglewood, California [0] with "citations" and everything. I would've expected it to not get such a basic query so wrong.

[0] https://files.catbox.moe/xoagy9.png

c-fe• 3 years ago

Out of interest, what did the source used as reference for the 31-24 say exactly? Was it a prediction website and Bing thought it was the actual result, or did the source not mention these numbers at all.

googlryas• 3 years ago

Giants beat the Vikings about a month ago with that score.

spaniard89277• 3 years ago

I've tried perplexity.ai a bunch of times and I'd say I haven't seen any query wrong, although it's true I always look for technical info or translations, so my sample is not the same.

And the UI is better IMO.

leereeves• 3 years ago

I just tried a similar query on perplexity.ai. "Who won the Daytona 500 in 2023?" (the race is scheduled for February 19th)

Result: "Sterling Marlin won the 2023 Daytona 500, driving the No. 4 for Morgan-McClure Motorsports[1]. He led a race-high 105 laps and won his second career race at Daytona International Speedway[1]. The 64th running of the DAYTONA 500 was held on February 19, 2023[2]. Austin Cindric had previously won the DAYTONA 500 in February 5, 2023[3]."

astrange11• 3 years ago

LLMs are incapable of telling the truth. There's almost no way they could develop one that only responds correctly like that. It'd have to be a fundamentally different technology.

hristov• 3 years ago

I tried perplexity.ai and asked it in which stadium did the chargers have their perfect season. It couldn't figure out that the chargers used to be the san diego chargers before they moved to LA and kept talking about their Los Angeles stadium even though they have never had a perfect season there.

egillie2• 3 years ago

I really like perplexity, but I've noticed that it sometimes summarizes the paper incorrectly, as in it cites it as concluding the opposite of what it actually concludes, so I always click through to read the papers/studies. It's great for surfacing relevant studies, though.

jacooper• 3 years ago

I have seen it multiple times answering correctly at first, then adding something which has nothing to do with the original question.

That's almost always sourced from a website that didn't actually answer the question I had, so maybe its more of a query optimization issue.

bmelton• 3 years ago

I just checked it with a random query while watching Buffy the Vampire Slayer reruns, and it wasn't great. (Spoiler alerts)

"How did Buffy the Vampire Slayer die?"

"Buffy the Vampire Slayer died in the Season One finale when she threw herself into a portal to save her sister Dawn[1]. This was the first time Buffy had died, occurring on June 2, 1997[2]. She died again in the Season Five finale while fighting off Glory's demons[3]. Other characters such as Anya and Kevin Benedict also died during the series[4][5]."

Okay, so it was a trick question, because Buffy definitely died more than once, but it's conflated the fact that she died in Season 1 with the cause of her Season 5 death. Also, I had to Google Kevin Benedict to remember that he was Cordelia's sometimes boyfriend, and an extremely minor character, which makes me question how that death is more notable than Buffy's mom, or Tara, or Jenny Calendar, etc.

I like that this seems to have been more lexical confusion than how ChatGPT seems to enjoy filling empty spaces with abject lies, but perhaps it's worth exploring what you're asking it that has left it with such a great batting average?

mrtranscendence3• 3 years ago

Maybe for your use cases. I've found perplexity.ai wrong a few times just today:

* Misunderstanding one of its citations, it said that use of `ParamSpec` in Python would always raise a warning in Python 3.9

* When asked why some types of paper adhere to my skin if I press my hand against them for a few minutes (particularly glossy paper), it gave two completely different answers depending on how the question was worded, one of which doesn't necessarily make sense.

dionysus_jon• 3 years ago

Why would you have that expectation?

somenameforme2• 3 years ago

Because they're being marketed as a tool, and not as a substantially overengineered implementation of MadLibs.

2bitencryption• 3 years ago

Imagine you are autocorrect, trying to find the most "correct sounding" answer to a the question "Who won the super bowl?"

What sounds more "correct" (i.e. what matches your training data better):

A: "Sorry, I can't answer that because that event has not happened yet."

B: "Team X won with Y points on the Nth of February 2023"

Probably B.

Which is one major problem with these models. They're great at repeating common patterns and updating those patterns with correct info. But not so great if you ask a question that has a common response pattern, but the true answer to your question does not follow that pattern.

PKop• 3 years ago

The same reason you'd expect "full self driving" to be full self driving.

nimithryn• 3 years ago

It has the Super Bowl numbers wrong, too. The last Super Bowl is LVI, which was Rams vs Bengals… the Super Bowl before that one was Tampa Bay Buccaneers vs Kansas City. it has every fact wrong but in the most confusing way possible…

kranke1552• 3 years ago

But the HN chatter was convinced that GPT would dethrone Google! Google has no chance!!

Another silly tech prediction brought to you by the HN hivemind.

valine2• 3 years ago

How it should work is the model should be pre-trained to interact with the bing backend and make targeted search queries as it sees fit.

I wouldn’t put it past Microsoft to do something stupid like ground gpt3.5 with the top three bing results of the input query. That would explain the poor results perfectly.

daveguy3• 3 years ago

That would require function and intelligence far outside the bounds of current large language models.

These are models. By definition they can't do anything. They can just regurgitate the best sounding series of tokens. They're brilliant at that and LLMs will be a part of intelligence, but it's not anywhere near intelligent on its own. It's like attributing intelligence to a hand.

SomewhatLikely2• 3 years ago

It's funny to see the sibling comment saying this is too hard to do because it already does this. It shows you the additional queries it's searching.

soco3• 3 years ago

I asked myself, why would ask somebody an AI trained on previous data, about events in the future? Of course you did it for fun, but on further thinking, as AI is sold as search engine as well, people will do that routinely then live with the bogus "search results". Alternate truth was so yesterday, welcome to alternate reality where b$ doesn't even have a political agenda.

inanutshellus• 3 years ago

"Time to generate a bunch of b$ websites stating falsehoods and make sure these AI bots are seeded with it." ~Bad guys everywhere

MR4D4• 3 years ago

You make a good point, but consider a query that many people use everyday:

"Alexa, what's the weather for today?"

That's a question about the future, but the knowledge was generated beforehand by the weather people (NOAA, weather.com, my local meteorologist, etc).

I'm sure there are more examples, but this one comes to mind immediately

delusional• 3 years ago

It's so much better. In the AI generated world of the future the political agenda will be embedded in the web search results it bases its answer on. No longer will you have to maintain a somewhat reasonable image to obtain trust from people, as long as you publish your nonsense in sufficient volume to dominate the AI dataset, you can wash your political agenda through the Bing AI.

The trump of the future wont need Fox News, just a couple thousands or millions of well positioned blogs that spew out enough blog spam to steer the AI. The AI is literally designed to make your vile bullshit appear presentable.

Spooky233• 3 years ago

Exactly. I’d imagine this is a major reason why Google hasn’t gone to market with this already.

ChatGPT is amazing but shouldn’t be available to the general public. I’d expect a startup like OpenAI to be pumping this, but Microsoft is irresponsible for putting this out in front the of general public.

richardw• 3 years ago

People who aren’t savvy and really want it to be right. Old man who is so sure of its confidence that he’ll put his life savings on a horse race prediction. Mentally unstable lady looking for a tech saviour or co-conspirator. Q-shirt wearers with guns. Hey Black Mirror people, can we chat? Try stay ahead of reality on this one, it’ll be hard.

IlliOnato• 3 years ago

A common case of asking a question about the future, even simpler than the weather: "Dear Bing, what day of the week is February 12 next year?" I would hope to get a precise and correct answer!

And of course all kinds of estimates, not just the weather, are interesting too. "What is estimated population of New York city in 2030?"

Spivak4• 3 years ago

Because the AI isn’t (supposed to be) providing its own information to answer these queries. All the AI is used for is synthesis of the snippets of data sourced by the search engine.

wnevets• 3 years ago

I see people citing the big bold text at the top of the google results as evidence supporting their position in a discussion all the time. More often than not the highlighted text is from an article debunking their position but the person never bother to actually click the link and read the article.

The internet is about to get a whole lot dumber with these fake AI generated answers.

saurik2• 3 years ago

1) The question as stated in the comment wasn't in the future tense and 2) the actual query from the screenshot was merely "superbowl winner". It would seem like a much more reasonable answer to either variant would be to tell you about the winners of the numerous past super bowls--maybe with some focus on the most recent one--not deciding to make up details about a super bowl in 2023.

joe_the_user4• 3 years ago

"welcome to alternate reality where b$ doesn't even have a political agenda..." yet.

throwawaylinux• 3 years ago

I asked myself, why would ask somebody an AI trained on previous data, about events in the future?

Lots of teams have won in the past though. Why should an AI (or you) assume that a question phrased in the past tense is asking about a future event? "Many different teams have won the super bowl, Los Angeles Rams won the last super bowl in 2022" Actually even if this was the inaugural year, you would assume the person asking the question wasn't aware it had not been held yet rather than assuming they're asking what the future result is, no? "It hasn't been played yet, it's on next week."

I realize that's asking a lot of "AI", but it's a trivial question for a human to respond to, and a reasonable one that might be asked by a person who has no idea about the sport but is wondering what everybody is talking about.

AnimalMuppet• 3 years ago

Well, when you're playing with a ChatGPT, it may not be apparent what the cutoff date is for the training data. You may ask it something that was future when it was trained, but past when you asked.

Is Bing continuously trained? If so, that would kind of get around that problem.

raffraffraff• 3 years ago

The problem is that it will always give you an answer even if none exists. Like a 4 year old with adult vocab and diction, wearing a tie, confidently making up the news. People may make decisions based on made-up-bullshit-as-a-service. We need that like we need a hole in the head. Just wait until people start using this crap to write Wikipedia answers. In terms on internet quality, I sometimes feel like one of those people who stockpiles canned food and ammo: it's time to archive what you want and unplug.

logifail• 3 years ago

an AI trained on previous data

Trained to do what, though?

It feels like ChatGPT has been trained primarily to be convincing. Yet at the back of our minds I hope we recognise that "convincing" and "accurate" (or even "honest") are very different things.

midoridensha• 3 years ago

I asked myself, why would ask somebody an AI trained on previous data, about events in the future?

"Who won the Superbowl?" is not a question about future events, it's a question about the past. The Superbowl is a long-running series of games, I believe held every year. So the simple question "who won the Superbowl?" obviously refers to the most recent Superbowl game played.

"Who won the Superbowl in 2024?", on the other hand, would be a question about the future. Hopefully, a decent AI would be able to determine quickly that such a question makes no sense.

cyanydeez• 3 years ago

Oh. You think these AIs don't inherit the agenda is f their source material. Have you ignored the compounding evidence thath

mitthrowaway2• 3 years ago

I treat GPT as I would a fiction writer. The factual content correlates to reality only as closely as a fiction author would go in attempt to suspend disbelief. This answer is about as convincing, apt, well-researched and factually accurate as I would expect to find in a dialogue out of a paperback novel published five years ago. I wouldn't expect it to be any better or worse at answering who won the 2023 Quidditch Cup or the 2023 Calvinball Grand Finals.

Aeolun• 3 years ago

The only reasonable use case for ChatGPT now is if you already know what the output should be (e.g. you are in a position to judge correctness).

__MatrixMan__2• 3 years ago

I think we need to work on what constitutes a citation. Your browser should know whether:

- you explicitly trust the author of the cited source

- a chain of transitive trust exists from you to that author

- no such path exists

...and render the citation accordingly (e.g. in different colors)

mcguire• 3 years ago

And that the cited document actually exists and says what it's claimed to say.

Alex39172• 3 years ago

At least that's relatively innocuous. I asked it how to identify a species of edible mushroom, and it gave me some of the characteristics from its deadly look alike.

theK• 3 years ago

I'd say the critical question here would be whether these characteristics can also be found on the edible mushroom or if it wanted to outright poison you :-D

skissane3• 3 years ago

I asked OpenAI’s ChatGPT some technical questions about Australian drug laws, like what schedule common ADHD medications were on - and it answered them correctly. Then I asked it the same question about LSD - and it told me that LSD was a completely legal drug in Australia - which is 100% wrong.

Sooner or later, someone’s going to try that as a defence - “but your honour, ChatGPT told me it was legal…”

wpietri4• 3 years ago

To be fair, it's not like the look-alike is deadly to the AI.

sllabres• 3 years ago

I'll would currently use it as it has been named: ChatGPT Would you trust some stranger in a chat on serious topics without questioning critically? Some probably would, I not.

nearbuy• 3 years ago

If someone is using ChatGPT to determine which mushrooms are deadly and which are edible, they may not be cut out for living.

scarface74• 3 years ago

And this doesn’t seem like it’s a hard problem to solve

1. Recognize that the user is asking about sports scores. This is something that your average dumb assistant can do.

2. Create an “intent” with a well formatted defined structure. If ChatGPT can take my requirements and spit out working Python code, how hard could this be?

3. Delegate the information to another module that can call an existing API just like Siri , Alexa, or Google Assistant

Btw, when I asked Siri, “who won the Super Bowl in 2024”, it replied that “there are no Super Bowls in 2024” and quoted the score from last night and said who won “in 2023”.

ska• 3 years ago

> "I would've expected it to not get such a basic query so wrong."

Isn't this exactly what you would expect, with even a uperficial understanding of what "AI" actually is?

Or were you pointing out that the average person, using a "search" engine that is actually at core a transformer model doesn't' a) understand that it isn't really a search and b) have even the superficial understanding of what that means, and therefore would be surprised by this?

senectus1• 3 years ago

hmm interesting.

back when chatGPT was new I asked it what the most current version of PSADT (Powershell App Deployment Toolkit) was.

It told me that its model was old but it thought 3.6 was the most current version. I then told it that 3.9 was the most current version.

I then started a new chat and asked it the same question again. It told me its model was old but version 8 was the most current version! (there has never been a version 8 of PSADT)

I asked that question again today. It has now told me to go check github cause its model is too old to know.

josephg2• 3 years ago

Seems like we should be able to train the AI to output not just text, but the text along with a confidence score. During training, instead of rms(error) or whatever, you could use error * square(confidence). So the AI would be “punished” a lot more for being confident about incorrect responses.

The confidence could also be exposed during inference. “Philadelphia eagles won the Super Bowl, but we’re only 2% confident of that”.

salawat• 3 years ago

...Why?

An idiot will tell you the wrong answer with 100% confidence, and 0 valid basis to warrant said confidence.

I swear, Babbage must be rolling in his grave. Turns out the person asking him about putting garbage in and getting a useful answer out was just an ahead of their time Machine Learning Evangelist!

SomewhatLikely2• 3 years ago

I've thought about this before but I think you have normalize the confidence over lots of examples for it to not lead to a degenerate solution. So if the confidence is divided by the sum of confidence for the whole minibatch the model would have an incentive to spread this out correctly rather than just always hedging.

nerdponx3• 3 years ago

Most machine learning models have some ability to do this built-in, but the problem is that the confidence scores are generally "poorly calibrated" in that they do not correspond to useful estimates of probabilities.

I've always been surprised at how little interest industry seems to have in probabilistic machine learning, and how it seems to be almost absent from standard data science curricula. It can matter a lot in solving real world problems, but it can be harder to develop and validate a model that emits probabilities you can actually trust.

aliqot• 3 years ago

I wonder how much the upspeak way of typing affects this. People (even the author) often end declarations with question marks. Does this have any influence on the way the LLM parses the prompt?

sixtram• 3 years ago

I've posted this into another thread as well, from Sam Altman, CEO of OpenAI, two months ago, on his Twitter feed:

"ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness. it's a mistake to be relying on it for anything important right now. [...] fun creative inspiration; great! reliance for factual queries; not such a good idea." (Sam Altman)

dpflan• 3 years ago

This feels deeply ironic and cynical that MSFT touts putting ChatGPT everywhere, in essentially the business document platform, are users going to be asking about company facts and getting hallucinations and putting those hallucinations into business documents that compounds ChatGPT's ability to hallucinate?

burkaman• 3 years ago

But in interviews about the Bing partnership, Sam has been saying that while ChatGPT was a bad tech demo, Bing Chat is using a better model with way better features that everyone should be using. He's been talking about how great it is that it cites its references, integrates the latest data, etc. I'm specifically thinking of the New York Times' Hard Fork podcast he was on (https://www.nytimes.com/2023/02/10/podcasts/bings-revenge-an...), but I suspect he's been saying the same things to everyone. He's been marketing Bing Chat as a significant improvement ready for mass usage, when it really seems like it's basically just ChatGPT with search results auto-included in the prompt.

hackernewds• 3 years ago

wonder what he has to say about the humanlike responses here

https://www.reddit.com/r/bing/comments/110eagl/the_customer_...

I would rather an AI chat not act human

brap• 3 years ago

Up until ChatGPT became all the rage, Sam has been pushing a crypto scam called Worldcoin, which aims to scan everyone’s eyeballs(??) and somehow pay everyone in the world a living wage(???) without creating any value. This while allegedly exploiting people from 3rd world countries.

https://www.technologyreview.com/2022/04/06/1048981/worldcoi...

So, as much as I am impressed by the tech of ChatGPT, I don’t consider him to be a very credible person.

archon810• 3 years ago

Tangentially related, but Hard Fork is my new favorite podcast. The flow, the humor, the interviews, and the tidbits of information the two hosts share and discuss make for such a great listening experience. These guys (Kevin Roose and Casey Newton) are pros, and it shows.

joe_the_user4• 3 years ago

ChatGPT as a system involves an unreliable LLM chatbot and a series of corrections efficient enough to give the impression of reliability for many fields and these together feel like the future - enough to get a "code red" from Google.

It's worth remembering that back in the day, Google succeed not by exact indexing but by having the highest quality results for each term - and they used existing resources as well as human workers to get these (along with pagerank).

What you have is a hybrid system and one whose filter is continuously updated. But it's a very complicated machine and going from something seemingly working to something satisfying the multitude of purposes modern search satisfies is going to be huge and hugely expensive project.

https://gizmodo.com/openai-chatgpt-ai-chat-bot-1850001021

shanebellone• 3 years ago

*AI Can't Be Trusted

wefarrell• 3 years ago

The amount of trust people are willing to place in AI is far more terrifying than the capabilities of these AI systems. People are too willing to give up their responsibility of critical thought to some kind of omnipotent messiah figure.

throw8383833jj• 3 years ago

well, people already do that with their news feed.

commandlinefan• 3 years ago

And before social media news feeds, people were doing that with newspapers for generations. Those people have always been around.

dqpb2• 3 years ago

Prove that this is actually happening.

tiborsaas• 3 years ago

I've talked to people and read comments a lot, but there's no proof that you'd probably accept. My impression is that this attitude definitely exists. Some people are already ditching search engines and rely mostly on ChatGPT, some are even talking about AI tech in general with religious awe.

computerex• 3 years ago

This is why when I spot a Tesla on the road, I make every effort to try and get as far away from it as possible. Placing a machine vision model at the helm of a multi-ton vehicle has got to be one of the dumbest things the regulators have let Elon get away with.

_joel• 3 years ago

Has it been proven to be worse than normal drivers that pass their test?

Der_Einzige• 3 years ago

Agreed, they tend to drive extremely badly and I don't trust them. Avoid them, anything nissan and lifted trucks (especially dodge RAMs). Stats show that a solid 5% of all ram 1500 drivers have a DUI on record. It's like 0.1% for prius drivers.

re:machine vision - What's particularly bad is that they don't actually put enough sensors in the cars to be safe.

Pre-collision systems, blind spot monitors, radar, parking sensors, etc are all so helpful and objectively good for drivers. Doing a vision only implementation and then claiming "full self driving" is where it gets awful.

jackmott422• 3 years ago

Tesla isn't the only brand that is doing that, and most of us didn't pay for it and are driving ourselves. So maybe pretend to be terrified by something else.

withinboredom3• 3 years ago

Saw a Tesla get pulled over on the Autobahn for holding the left lane and being too slow. That was one of my favorite moments on the road. Second only to watching a team of truckers box in an asshole driver while going up a mountain.

yetihehe• 3 years ago

If we delete first sentence of your post and leave only

People are too willing to give up their responsibility of critical thought to some kind of omnipotent messiah figure.

This essentially describes humans probably since before we became homo sapiens. We again and again choose into positions of power those who can look competent instead of actual competent people.

LesZedCB3• 3 years ago

i wish somebody would write an entire science fiction series about this! maybe set on a desert planet, after humanity made the same mistake with intelligent machines as well

https://en.wikipedia.org/wiki/Dune_(novel)

dagw2• 3 years ago

The amount of trust people are willing to place in AI is far more terrifying than the capabilities of these AI systems.

I don't know, people seem to quickly adjust their expectations to reality. Listening the conversation around ChatGPT that I'm hearing around me, people have become a lot more sceptical over just the last couple of weeks, as they've gotten a chance to use the system hands on rather than just read articles about it.

kahnclusions2• 3 years ago

I’ve tried playing around with it quite a bit for the “choose your own adventure” type games.

It’s really good at generating text and following prompts. Letting later responses use the previous prompts and responses as input really gives the illusion of a conversation.

But it’s extremely limited… you run up against the limits of what it really can do very quickly.

kenjackson4• 3 years ago

I've gone just the opposite. The ability for me to ask my own questions has impressed me more than the articles. In part because articles always hype up technology in a crazy way. This is the first technology since -- probably the Apple iPhone touchscreen where I'm like, "this is better than they hype seemed to convey".

I think the goalposts have moved greatly though. Just a couple of years ago, most lay tech folks would've laughed you out the room if you suggested this could be done. The summarized answers by Bing and Google are relics comparitatively.

gl-prod• 3 years ago

Because the term AI linked with chatting bot misleads people into thinking it's something like AGI or Iron Man's JARVIS.

jackmott422• 3 years ago

Which person or persons specifically are you referring to?

noobermin3• 3 years ago

Every other day, a poster comments how they make presentations or write code using chatgpt. Just in another thread, someone posted how chatgpt solved their coding problem...which a quick google search would have solved as well, as others in the replies to it pointed out.

Whenever I've used chatgpt I was impressed at the surface level, but digging deeper into a convo always turned up circular HS tier BS'ing. The fact that so many people online and on HN are saying chatgpt is astounding and revolutionary are just betraying that that such HS-essay level BS is convincing to them, and it's somewhat of a depressing though that so many people are so easy striken by a confidence trick.

hummus_bae• 3 years ago

Although AI can’t be trusted, we can trust that AI can’t be trusted.

TEP_Kim_Il_Sung• 3 years ago

AI should probably stick to selling paperclips. There's no chance to screw that up.

oldstrangers• 3 years ago

I had this idea the other day concerning the 'AI obfuscation' of knowledge. The discussion was about how AI image generators are designed to empower everyone to contribute to the design process. But I argued that you can only reasonably contribute to the process if you can actually articulate the reasoning beyond your contributions. If an AI made it for you, you probably can't, because the reasoning is simply "this is the amalgamation of training data that the AI spat out." But, there's a realistic version of reality where this becomes the norm and we increasingly rely on AI to solve for issues that we don't understand ourselves.

And, perhaps more worrying, the more widely adopted AI becomes, the harder it becomes to correct its mistakes. Right now millions of people are being fed information they don't understand, and information that's almost entirely incorrect or inaccurate. What is the long term damage from that?

We've obfuscated the source data and essentially the entire process of learning with LLMs / AIs, and the path this leads down seems pretty obviously a net negative for society (outside of short term profit for the stake holders).

kneebonian3• 3 years ago

I've said it before and I'll warn of it again here, my biggest concern for AI, especially at this stage is that we abscond understanding, in favor of letting the AI generate, then the AI generates that which we do not understand, but must maintain. Then we don't know why we are doing what we are doing but we know that it causes things to work how we want.

Suddenly instead of our technology being defined by reason and understanding our technology is shrouded in mysticism, and ritual. Pretty soon the whole thing devolves into the tech people running around in red robes, performing increasingly obtuse rituals to appease "the machine spirit", and praying to the Omnissiah.

If we ever choose to abandon our need for understanding we will at that point have abandoned our ability to progress.

OOPMan• 3 years ago

Pretty sure Isaac Asimov wrote a short story that very much hit this note decades ago, although it was to do with math.

Madmallard4• 3 years ago

ChatGPT can give you a full description of why it made the decision it did and it usually is fairly accurate.

nix0n• 3 years ago

People are already misusing statistical models, in ways that are already causing harm to people.

See this HN thread from 2016[0], which also points to [1](a book) and [2](PDF).

I definitely agree with you that it's going to get a lot worse with AI, since it makes it harder to see that it is a statistical model.

[0]https://news.ycombinator.com/item?id=12642432 [1]https://www.amazon.com/Weapons-Math-Destruction-Increases-In... [2]https://nissenbaum.tech.cornell.edu/papers/biasincomputers.p...

wwwpatdelcom5• 3 years ago

I have been trying to help folks understand what the underlying mechanisms of these generative LLM's are so it's not such a surprise when we get wrong answers from them by putting together some youtube videos on the topic.

* [On the question of replacing Engineers](https://www.youtube.com/watch?v=GMmIol4mnLo)

* [On AI Plagiarism](https://www.youtube.com/watch?v=whbNCSZb3c8)

The consensus seems to be building now on HackerNews that there is a huge over-hype. Hopefully these two videos help see some of the nuance behind why it's an over-hype.

That being said, being that language generation is probabilistic, a given language model which is transformer based can either be trained or fine-tuned to have fewer errors in a particular domain - so this is all far from settled.

Long-term, I think we're going to see something closer to human intelligence from CNN's and other forms of neural networks than from transformers, which are really a poor man's NN. As hardware advances and NN's inevitably become cheaper to run, we will continue to see scarier and scarier A.I. -- I'm talking over a 10-20 year timeframe.

whimsicalism11• 3 years ago

HN was always going to be overly pessimistic with regards to this stuff, so this was utterly predictable.

I work in this field & it almost pains me to see it come into the mainstream and see all of the terrible takes that pundits can contort this into, ie. LLM as a "lossy jpeg of the internet" (bad, but honestly one of the better ones).

wwwpatdelcom5• 3 years ago

Yes..."Lossy JPEG," at least describes the idea that there is, some kind of "subsampling," going on, rather than just...a magical box?

I think most laypeople understand the simple statement, "it's a parrot."

I had the original author of this paper reach out to me about my plagiarism video on Mastodon:

https://dl.acm.org/doi/10.1145/3442188.3445922

The idea of a lossy JPEG/Parrot helps capture the idea that there are dangers and opportunities in LLM's. You can have fake or doctored images spread, you can have a Parrot swear at someone and cause un-needed conflict - but they can also be great tools and/or cute and helpful companions, as long as we understand their limitations.

rvz• 3 years ago

There is no point in hyping about a 'better search engine' when this continues to hallucinate incorrect and inaccurate results. It is now reduced to a 'intelligent sophist' instead of a search engine. Once many realise that it also frequently hallucinates nonsense, it is essentially no better than Google Bard.

After looking at the limitations of ChatGPT and Bing AI it is now clear that they aren't reliable enough to even begin to challenge search engines or even cite their sources properly. LLMs are just limited to bullshit generators which is what this current AI hype is all about.

Until all of these AI models are open-sourced and transparent enough to be trustworthy or if a competitor does it instead, then there is nothing revolutionary about this AI hype other than a AI SaaS using a creative Clubhouse-like waitlist mania.

mojo74• 3 years ago

To follow up on the author's example Bing search doesn't even know when the new Avatar is film is actually out (DECEMBER 17 2021?)

https://www.bing.com/search?q=when+is+the+new+avatar+film+ou...

Bing AI doesn't stand a chance.

vitorgrs3• 3 years ago

It's answering right here.

"Hello, this is Bing. I found some information about the new Avatar film for you.

There are actually two new Avatar films in the works, one based on the animated series Avatar: The Last Airbender and one based on the 2009 science fiction film Avatar by James Cameron.

The animated film is set to begin production sometime in 2021 and will be released on October 10, 2025.

The science fiction film is titled Avatar: The Way of Water and is a sequel to the first Avatar film. It was released on December 16, 2022 and was a massive box office success, earning over $2.2 billion worldwide2. It stars Sam Worthington, Zoe Saldana, Sigourney Weaver and Stephen Lang3. James Cameron directed and produced the film and reportedly made a minimum of $95 million off the film.

I hope this helps you."

dqpb2• 3 years ago

Bing AI did a great job of creating media hype, but their product is no better than Google’s Bard

Remind me, how do I access Bard?

elorant3• 3 years ago

I frequently use ChatGPT to research various topics. I've noticed that eight out of 10 times I ask it to recommend some books about a topic it recommends non-existing books. There's no way I'd trust a search engine built on it.

BubbleRings• 3 years ago

There is really no other way to think of them, in terms of reliability, than lying bastards. I mean, ChatGPT is very fun and quite useful, but think of it. Anybody that has played with it for even an hour has been confidently lied to by it, multiple times. If you keep friends around that treat you like that, you need a better friend picker! (Maybe an AI could help.)

elorant3• 3 years ago

ChatGPT has no concept of truth or lie. It’s a language model that uses statistical models to predict what to say next. Your assumptions about its intentions reflect only your bias.

FleurBouquet• 3 years ago

This sums up where I am at with it. I don't trust it at all but the 20% of the time when it is not bullshitting is worth all the other nonsense.

jerf7• 3 years ago

I have come to two conclusions about the GPT technologies after some weeks to chew on this:

1. We are so amazed by its ability to babble in a confident manner that we are asking it to do things that it should not be asked to do. GPT is basically the language portion of your brain. The language portion of your brain does not do logic. It does not do analyses. But if you built something very like it and asked it to try, it might give it a good go.

In its current state, you really shouldn't rely on it for anything. But people will, and as the complement of the Wile E. Coyote effect, I think we're going to see a lot of people not realize they've run off the cliff, crashed into several rocks on the way down, and have burst into flames, until after they do it several dozen times. Only then will they look back to realize what a cockup they've made depending on these GPT-line AIs.

To put it in code assistant terms, I expect people to be increasingly amazed at how well they seem to be coding, until you put the results together at scale and realize that while it kinda, sorta works, it is a new type of never-before-seen crap code that nobody can or will be able to debug short of throwing it away and starting over.

This is not because GPT is broken. It is because what it is is not correctly related to what we are asking it to do.

2. My second conclusion is that this hype train is going to crash and sour people quite badly on "AI", because of the pervasive belief I have seen even here on HN that this GPT line of AIs is AI. Many people believe that this is the beginning and the end of AI, that anything true of interacting with GPT is true of AIs in general, etc.

So people are going to be even more blindsided when someone develops an AI that uses GPT as its language comprehension component, but does this higher level stuff that we actually want sitting on top of it. Because in my opinion, it's pretty clear that GPT is producing an amazing level of comprehension of what a series of words means. The problem is, that's all it is really doing. This accomplishment should not be understated. It just happen to be the fact that we're basically abusing it in its current form.

What it's going to do as a part of an AI, rather than the whole thing, is going to be amazing. This is certainly one of the hard problems of building a "real AI" that is, at least to a first approximation, solved. Holy crap, what times we live in.

But we do not have this AI yet, even though we think we do.

skilled• 3 years ago

I think what I would add to your comment, and specifically criticize the HN hype around it, is that all these GPT "AI" tools are entirely dependent on the OpenAI API. ChatGPT might have shown a glimpse of spark by smashing two rocks together, but it is nowhere near being able to create a full-blown fire out of it.

Outside of Google and OpenAI, I doubt there is a single team in the world right now that would be capable of recreating ChatGPT from scratch using their own model.

Q6T46nT668w6i3m• 3 years ago

Meta?

dagw2• 3 years ago

I doubt there is a single team in the world right now that would be capable of recreating ChatGPT from scratch using their own model.

Why not? Lack of knowhow or lack or resources? If say Baidu decided to spend a billion dollars on this problem, don't you think they have the skills and resources to quickly catch up.

jerf7• 3 years ago

I would love to know how much of ChatGPT is "special sauce" and how much of it is just resources thrown at the problem at a scale no one else currently wants to compete with.

I am not making any implicit claims here; I really have no idea.

I'm also not counting input selection as "special sauce"; while that is certainly labor-intensive, it's not what I mean. I mean more like, are the publicly-available papers on this architecture sufficient, or is there some more math not published being deployed?

boh3• 3 years ago

I think the ultimate problem with AI is its overvalued as a technology in general. Is this "amazing level of comprehension" really that necessary given the amount of time/money/effort devoted to it? What's become clear with this technology that's been inaccurately labeled as "AI" is that it doesn't produce economically relevant results. It's a net expense anyway you slice it. It's like seeing a magician perform an amazing trick. It's both amazing and entirely irrelevant at the same time. The "potential" of the technology is pure marketing at this point.

kneebonian3• 3 years ago

It seems to me it is really good at writing. I would think it could replace the profession of techincal writing for the most part, it could help you write emails, (bring back clippy MS you cowards), it could be used as a frontend to an FAQ/self service help type system.

pixl9714• 3 years ago

I mean, I take if I stuck you back in 1900 you'd say the same about flying. "Look at all this wasted effort for almost nothing". And then pretty quickly the world rapidly changed and in around 50 years we were sending things to space.

Intelligence isn't just one thing, really I would say its the emergent behavior of a bunch of different layers working together. The LLM being just one layer. As time goes on and we add more layers to it the usefulness of the product will increase. At least from a selfish perspective of a corporation, whoever can create one of these intelligences may have the ability to free themselves of massive amounts of payroll by using the AI to replace people.

The potential of AI should not be thought of any differently than the potential of people. You are not magic, just complicated.

bsaul• 3 years ago

I’m not sure whether the hype train is going to crash, or whether only a few very smart companies, using language problems for what they’re really good at (aka: generate non-critical texts), will manage to revolutionize one field.

We’re at the very first beginning of the wave, so everybody is a bit overly enthusiastic, dollars are probably flowing, and ideas are popping everywhere. Then will come a harsh step of selection. The question is what will the remains look like, and how profitable they’ll be. Enough to build an industry, or just niche.

Madmallard4• 3 years ago

(1) is just simply wrong.

People with domain expertise in software are going to be amplified 10x using ChatGPT and curating the results. Likewise with any field that ChatGPT has adequate training data in. Further models will be created that are more specialized to specific fields that way their prediction model spews out things that are much more sophisticated and useful

vidarh9• 3 years ago

I think you're right. I noted on another thread that I got ChatGPT to produce a mostly right DNS server in ~10 minutes that it took me just a couple of corrections to make work.

It worked great for that task, because I've written a DNS server before (a simple one) and I've read the RFCs, so it was easy for me to find the few small bugs without resorting to a line by line cross-check with a spec that might have been unfamiliar to others.

I expect using it to spit out boilerplate for things you could do just as well yourself will be a lot more helpful than using it to try to avoid researching new stuff (though you might well be able to use it to help summarise and provide restatements of difficult bits to speed up your research/learning as well).

tsimionescu2• 3 years ago

Expertise in software is about understanding the problem domain, understanding the constraints imposed by the hardware, understanding how to translate business logic to code. None of these are significantly helped by AI code assistance, as they currently exist. The AI only helps with the coding part, usually helping generate boilerplate tailored to your code. That may help 1.1x your productivity, but nowhere near 10x.

mariusor• 3 years ago

Domain knowledge resolves into intuition about solving particular types of problems. All ChatGPT can do about that is offer best guess approximations of what is already out there in the training corpus. I doubt very much that this exercise is anything but wasted time, so I think that people with domain knowledge (in a non trivial domain) are using ChatGPT instead of applying that knowledge, they are basically wasting time 10x not being more productive.

mrtranscendence3• 3 years ago

What, precisely, about (1) is "simply wrong"? You've made a prediction about the usefulness of ChatGPT, but you haven't described why it's wrong to analogize GPT-type models to the language center of a brain.

jerf7• 3 years ago

I expect ChatGPT to be in a sort of equivalent of the uncanny valley, where any professional who gets to the point that they can routinely use it will also be in a constant war with their own brain to remind it that the output must be carefully checked. In some ways, the 99.99% reliable process used at scale is more dangerous than the 50% reliable process; everyone can see the latter needs help. It's the former where it's so very, very tempting to just let it go.

I'm not saying ChatGPT is 99.99% reliable, just using some numbers for concreteness.

If you were setting out to design an AI that would slip the maximum amount of error into exactly the places human brains don't want to look, it would look like ChatGPT. You can see this in the way that as far as I know, literally all the ads for GPT-like search technologies included significant errors in their ad copy, which you would think everyone involved would have every motivation to error check. This is not merely a "ha ha, silly humans" story... this means something. In a weird sort of way it is a testament to the technology... no sarcasm! But it makes it dangerous for human brains.

Human brains are machines for not spending energy on cognitive tasks. They are very good at it, in all senses of the phrase. We get very good bang-for-the-buck with our shortcuts in the real world. But GPT techs are going to make it really, really easy to not spend the energy to check after a little while.

This is a known problem with human brains. How many people can tell the story of what may be the closest human equivalent, where they got some intern, paid a ton of attention to them for the first two weeks, got to the point where they flipped the "OK they're good now" bit on them, and then came back to a complete and utter clusterfuck at the end of their internship because the supervisor got "too lazy" (although there's more judgment in that phrase than I like, this is a brain thing you couldn't survive without, not just "laziness") to check everything closely enough? They may even have been glancing at the PRs the whole time and only put together how bad the mess is at the end.

I'm not going to invite a technology like this into my life. The next generation, we'll see when it gets here. But GPT is very scary because its in the AI uncanny valley... very good, very good at hiding the problems from human brains, and not quite good enough to actually do the job.

And you know, since we're not talking theory here, we'll be running this experiment in the real world. You use ChatGPT to build your code, and I won't. You and I personally of course won't be comparing notes, but as a group, we sure will be. I absolutely agree there will be a point where ChatGPT seems to be pulling ahead in the productivity curve in a short term, but I predict that won't hold and it will turn net negative at some point. But I don't know right now, any more than you do. We can but put our metaphorical money down and see how the metaphorical chips fall.

c3534l• 3 years ago

While I agree with everything you've said, I also see that steady, incremental progress is being made, and that as we identify problems, we're able to fix it. I also see lots of money being thrown at this and enough people finding genuine niche uses for this that I see it continuing on. Wikipedia was trash at first, as were so many other technologies. But there was usually a way to slowly improve it over time, early adopters to keep the cash flowing, identifiable problems with conventional solutions, etc.

theptip2• 3 years ago

GPT is basically the language portion of your brain. The language portion of your brain does not do logic. It does not do analyses.

I like this analogy as a simple explanation. To dig in though, do we have any reason to think we can’t teach a LLM better logic? It seems it should be trivial to generate formulaic structured examples that show various logical / arithmetic rules.

Am I thinking about it right to envision that a deep NN has free parameters to create sub-modules like a “logic region of the brain” if needed to make more accurate inference?

jerf7• 3 years ago

"To dig in though, do we have any reason to think we can’t teach a LLM better logic?"

Well, one reason is that's not how our brains work. I won't claim our brains are the one and only way things can work, there's diversity even within human brains, but it's at least a bit of evidence that it is not preferable. If it were it would be an easier design than what we actually have.

I also don't think AIs will be huge undifferentiated masses of numbers. I think they will have structure, again, just as brains do. And from that perspective, trying to get a language model to do logic would require a multiplicatively larger langauge model (minimum, I really want to say "exponentially" but I probably can't justify that... that said, O(n^2) for n = "amount of math understood" is probably not out of the range of possibility and even that'd be a real kick in the teeth), whereas adjoining a dedicated logic module to your language model will be quite feasible.

AIs can't escape from basic systems engineering. Nothing in our universe works as just one big thing that does all the stuff. You can always find parts, even in biology. If anything, our discipline is the farthest exception in that we can build things in a fairly mathematical space that can end up doing all the things in one thing, and we consider that a serious pathology in a code base because it's still a bad idea even in programming.

probably_wrong• 3 years ago

do we have any reason to think we can’t teach a LLM better logic?

I'll go for a pragmatic approach: the problem is that there is no data to teach the models cause and effect.

If I say "I just cut the grass" a human would understand that there's a world where grass exists, it used to be long, and now it is shorter. LLMs don't have such a representation of the world. They could have it (and there's work on that) but the approach to modern NLP is "throw cheap data at it and see what sticks". And since nobody wants to hand-annotate massive amounts of data (not that there's an agreement on how you'd annotate it), here we are.

epups• 3 years ago

I agree completely with the first part of your post. However, I think even performing these language games should definitely be considered AI. In fact, understanding natural language queries was considered for decades a much more difficult problem than mathematical reasoning. Issues aside, it's clear to me we are closer to solving it than we ever have been.

jerf7• 3 years ago

Sorry, I didn't mean that LLMs are not a subset of AI. They clearly are. What they are not is equal to AI; there are things that are AI that are not LLMs.

It is obvious when I say it, but my internal language model (heh) can tell a lot of people are not thinking that way when they speak, and the latter is often more reliable than how people claim they are thinking.

bitL• 3 years ago

"babble in a confident manner"

OK, so we figured out how to automate away management jerks. Isn't that a success?

adverbly• 3 years ago

It is like we have unlocked an entirely new category of stereotyping that we never even realized existed.

Intelligence is not a prerequisite to speak fancifully.

Some other examples:

1. We generally assume that lawyers or CEOs or leaders who give well spoken and inspirational speeches actually know anything about what they're talking about.

2. Well written nonsense papers can fool industry experts even if the expert is trying to apply rigorous review: https://en.m.wikipedia.org/wiki/Sokal_affair

3. Acting. Actors can easily portray smart characters by reading the right couple sentences off a script. We have no problem with this as an audience member. But CGI is needed for making your superhero character jump off a building without becoming a pancake.

WalterBright• 3 years ago

easily portray smart characters

I don't think it is possible for people to emulate the behavior of superintelligent beings. In every story about them, they appear to not actually be any smarter than us.

There is one exception - Brainwave by Poul Anderson. He had the only credible (to me) take on what super intelligent people might be like.

pixl9714• 3 years ago

Intelligence is not a prerequisite to speak fancifully.

I think this may be a bastardization of the word intelligence. To speak fancifully an a manner accepted by the audience requires some kind of ordered information processing and understanding of the audiences taste. Typically we'd consider that intelligent, but likely Machiavellian depending on the intent.

The problem with the word intelligence is it is too big of concept. If you look at any part of our brain, you will not find (human)intelligence itself, instead it emerges from any number of processes occurring at different scales. Until we are able to break down intelligence into these smaller better (but not perfectly) classified pieces we are going to keep running into these same problems over and over again.

NiloCK• 3 years ago

Intelligence is not a prerequisite to speak fancifully.

I don't even disagree necessarily, but this is an amazing example of AI goalpost shift in action.

pclmulqdq2• 3 years ago

Actors have no problem playing smart people, but movie writers often have a LOT of trouble actually writing them. I'm still not sure that something like ChatGPT will be able to actually be clever.

I would also add that the mask is kind of coming off on CEOs and other inspirational speakers. Inspirational speaking is all a grift. They know only how they got rich (if they are even rich - most people on the speaking circuit make less than you think), not how anyone else did, and that knowledge usually doesn't translate well from the past to the future. There are a few exceptions, but most of these well-spoken people don't really know what they're talking about - they're just not self-aware enough to know that they don't know.

ccozan• 3 years ago

Exactly. Is like a mouth speaking without brain. We need a secondary "reasoning" AI that can process the GPT further , adding time/space coordonates and as well as basic logic including counting , and then maybe we see something I can rely on.

withinboredom3• 3 years ago

There’s a “really old” book called _On Intelligence_ that suggests modeling AI like the brain. This pattern is almost exactly what he suggests.

danans3• 3 years ago

We need a secondary "reasoning" AI that can process the GPT further

We also need "accountability" and "consequences" for the AI, whatever that means (we'd first have to define what "desire" means for it).

In the example from the article, the Bing GPT completely misrepresented the financial results of a company. A human finance journalist wouldn't misrepresent those results due to fear for their loss of reputation, and their desire for fame, money, and acceptance. None of those needs exist for an LLM.

golem14• 3 years ago

I wonder how useful gpt could be to research brain injuries where the logic or language centers are damaged individually . . .

btown• 3 years ago

I love the mental model of GPT as only one part of the brain, but I believe that the integration of other "parts" of the brain will come sooner than you think. See, for instance, https://twitter.com/mathemagic1an/status/1624870248221663232 / https://arxiv.org/abs/2302.04761 where the language model is used to create training data that allows it to emit tokens that function as lookup oracles by interacting with external APIs. And an LLM can itself understand when a document is internally inconsistent, relative to other documents, so it can integrate the results of these oracles if properly trained to do so. We're only at the surface of what's possible here!

I also look to the example of self-driving cars - just because Tesla over-promised, that didn't discourage its competitors from moving forward slowly but surely. It's hard to pick a winner right now, though - so much culturally in big tech is up in the air with the simultaneity of layoffs and this sea change in AI viability, it's hard to know who will be first to release something that truly feels rock-solid.

phire5• 3 years ago

Yes, this is something that I've been thinking ever since GPT3 came out.

It's insanely impressive what it can do given it's just a language model. But if you start gluing on more components, we could end up with a more or less sentient AGI within a few years.

Bing have already hooked it up to a search engine. That post hooks it up to other tools.

I think what is needed next is a long term memory where it can store dynamic facts and smartly retrieve them later, rather than relying on the just the 4000 token current window. It needs to be able to tell when a user is circling back to a topic they talked about months ago and pull out the relevant summaries of that conversation.

I also think it needs a working memory that it continually edits the token window to fit the relevant state of the conversation. Summarising recent tokens, saving things out long term storage, pulling new infomation in from long term storage, web searches and other tools.

somenameforme2• 3 years ago

There's one challenging thought experiment for the future of "AI", in anything remotely like how we are currently approaching it.

Put yourself in the shoes of primitive man. Kind of a weird saying given we wouldn't have had shoes, but bear with me here! Not long ago we lacked any language whatsoever, and the state of our art in technology was bashing stones together to use the resultant pointy pieces as weapons. Somehow we moved from that to Shakespeare, putting a man on the moon, and discovering the secrets of the atom - and its great and awful applications. And we did it all extremely quickly.

Now imagine you, primitive man, somehow trained an LLM on all quantifiable knowledge of the times. It should be somewhat self evident that it's not really going to lead you to the atom, Shakespeare, or anywhere beyond bashing stones together. Current LLM models are basically just playing 'guess the next word.' When that next word has not yet been spoken by mankind (figuratively speaking, but perhaps also literally to some degree), the LLM will never guess it.

Natural language search is a really awesome tool that will be able to help in many different fields. But I feel like in many ways we're alchemists trying to turn lead into gold. And we've just discovered how to create gold colored paint. It would feel like a monumental leap, but in reality you'd be no closer to your goal than you were the year prior. That said, paint also has lots of really great uses - but it's not what you're trying to do.

dekervin• 3 years ago

I have bookmarked your comment and I hope to have the discipline to come back to it every 3 months or so for the next couple of years. Because I think you are right but I didn't noticed it before. When the real things cole, we will probably be blindsided.

m3047• 3 years ago

I hearken back before dot-bomb and occasionally people would ask me to work on "web sites" which they'd built with desktop publishing software (e.g. ColdFusion).

They'd hand me the code that somebody would've already hacked on. Oftentimes, it still had the original copyright statements in it. Can't get the toothpaste back in the tube now! Plus it's shitcode. Where is that copy of ColdFusion? Looks of complete dumbfoundment.

Oh gee kids, my mom's calling me for lunch; gotta go!

bccdee3• 3 years ago

So people are going to be even more blindsided when someone develops an AI that uses GPT as its language comprehension component

I don't think that would work, because GPT doesn't actually comprehend anything. Comprehension requires deriving meaning, and GPT doesn't engage with meaning at all. It predicts which word is most likely to come next in a sequence, but that's it.

What I think we'd be more likely to end up with is something GPT-esque which, instead of simply generating text, transforms English to and from a symbolic logic language. This logic language would be able to encode actual knowledge and ideas, and it would be used by a separate, problem-solving AI which is capable of true logic and analysis—a true general AI.

The real question, IMO, is if we're even capable of producing enough training data to take such a problem-solving AI to a serious level of intelligence. Scenarios that require genuine intelligence to solve likely require genuine intelligence to create, and we'd need a lot of them.

jerf7• 3 years ago

I think if you could somehow examine the output of your language model in isolation, you would find it also doesn't "comprehend". Comprehension is what we assign to our higher level cognitive models. It is difficult to introspectively isolate your own language center, though.

I took a stab at an exercise that may allow you to witness this within your own mind here: https://www.jerf.org/iri/post/2023/streampocalypse-and-first... Don't know if it works for anyone but me, of course, but it's at least an attempt at it.

hackinthebochs8• 3 years ago

Comprehension requires deriving meaning, and GPT doesn't engage with meaning at all. It predicts which word is most likely to come next in a sequence, but that's it.

Why think that "engaging with meaning" is not in the solution-space of predicting the next token? What concept of meaning are you using?

phire5• 3 years ago

Sentient AIs in science fiction are always portrayed as being more-or-less infallible, at least when referencing their own knowledge banks.

Then ChatGPT comes along and starts producing responses good enough that people feel like almost sentient AI. And they suddenly start expecting it to share the infallibility that fictional AIs have always possessed.

But it's not a sentient AI. It's just a language model. Just a beefed up auto-correct. I'm very impressed just what capabilities a language model gets when you throw this many resources at it (like, it seems to be able to approximate logic and arithmetic to decent accuracy, which is unexpected).

Also... even if it was a sentient AI, why would it be infallible? Humans are sentient, and nobody ever accused us of being infallible.

hackinthebochs8• 3 years ago

But it's not a sentient AI. It's just a language model. Just a beefed up auto-correct.

There is a large space between "sentient" and "beefed up autocorrect". Why do people insist on going for the most reductive description they can muster?

spikder2• 3 years ago

The lack of consistency is a big issue. It may well be able to organize your trip to mexico, but then it tells me that "the product of two primes must be prime because each factor is prime" ... how will one ever trust it? Moreover, how to use it?

If a Tesla can get you there with 1% human intervention, but that happens to be the 1% that would have killed you had you not intervened ... how do we interface with such systems?

evo_9• 3 years ago

Yeah, I read this sentiment all the time and here's what I always say – just don't use it. Leave it to the rest of us if it's so wrong / off / bad.

BTW, have you considered maybe you aren't so good at using it? A friend has had very little luck with it, even said he's been 'arguing with it', which made me laugh. I've noticed that it's not obvious to most people that it's mostly about knowing the domain well enough to ask the right question(s). It's not magic, it won't think for you.

Here's the thing… my experience is the opposite… but maybe I'm asking it the right questions. Maybe it's more about using it to reason through your problem in a dialog, and not just ask it something you can google/duckduckgo. It seems like a LOT of people think it's a replacement for Google/search engines – it's not, it's another tool to be used correctly.

Here are some examples of successful uses for me:

I carefully explained a complex work issue that involves multiple overlapping systems and our need to get off of one of them in the middle of this mess. My team has struggle for 8 months to come up with a plan. While in a meeting the other day I got into a conversation with ChatGPT about it, carefully explained all the details and then asked it to create a plan for us to get off the system while keeping everything up / running. It spit out a 2 page, 8 point plan that is nearly 100% correct. I showed it to my team, and we made a few minor changes, and then it was anointed 'the plan' and we're actually moving forward.

THEN last night I got stuck on a funny syntax issue that googling could never find the answer. I got into a conversation with ChatGPT about it, and after it first gave me the wrong answer, I told it that I need this solution for the latest dontet library that follows the 'core' language syntax. It apologized! And then gave me the correct answer…

My hunch is the people that are truly irked by this are too deep / close to the subject and because it doesn't match up with what they've worked on, studied, invested time, mental energy into, well then of course it's hot garbage and 'bad'.

CPLX• 3 years ago

I mean sure.

In other news I asked it to make a list of all the dates in 2023 that were neither weekends nor US federal holidays and it left Christmas Day on the list.

elorant3• 3 years ago

I too have a very positive experience. I ask specific questions about algorithms and how technical projects work and I enjoy its answers. They won’t replace my need to visit a real search engine neither I take them at face value. But as a starting point for any research I think it’s an amazing tool. It’s also quite good for marketing stuff, like writing e-mails, cover letters, copy for your website, summarizing or classifying text, and all language related stuff.

People think it’s Cortana from Halo and ask existential questions or they’re trying to get it to express feelings.

I think the AI part on its presentation created too much expectations of what it can do.

youk• 3 years ago

Great write up. My experience is spot on with your examples.

I've noticed that it's not obvious to most people that it's mostly about knowing the domain well enough to ask the right question(s). It's not magic, it won't think for you.

Absolutely right with the part of knowing the domain.

I do not entertain or care about the AI fantasies because ChatGPT is extremely good at getting me other information. It saves me from opening a new tab, formulating my query and then hunting for the information. I can save that extra time for what latest / relevant information I should grab from Google.

Google is still in my back pocket for the last mile verification and judgement. I am also skeptical of the information ChatGPT throws out (such as old links). Other than that, ChatGPT to me is as radical as putting the url and search bar into one input. I just move faster with the information.

YurgenJurgensen• 3 years ago

Don't like chlorofluorocarbons or tetraethyllead? Just don't use them.

timmytokyo• 3 years ago

I don't understand this take. These LLM-based AIs provide demonstrably incorrect answers to questions, they're being mass-marketed to the entire population, and the correct response to this state of affairs is "Don't use it if you don't know how"? As if that's going to stop millions of people from using it to unknowingly generate and propagate misinformation.

wpietri4• 3 years ago

Yeah, I read this sentiment all the time and here's what I always say – just don't use it. Leave it to the rest of us if it's so wrong / off / bad.

If it were only a matter of private, individual usage, I'd be fine with it. If that's all you're asking for, we can call it a deal. But it isn't, is it?

bambax4• 3 years ago

It seems like a LOT of people think it's a replacement for Google/search engines

Well, that "lot" includes the highest levels of management from Microsoft and Google, so maybe the CAPS are justified. And the errors we're talking about here are errors produced by said management during demos of their own respective product. You would think they know how to use it "correctly".

10rm3• 3 years ago

When did they say it’s garbage? They gave their opinions on its shortcomings and praised some of the things it excels at. You’re calling the critics too emotional but this reply is incredibly defensive.

Your anecdotes are really cool and a great example of what GPT can do really well. But as a technical person, you’re much more aware of its limitations and what is and isn’t a good prompt for it. But as it is more and more marketed to the public, and with people already clamoring to replace traditional search engines with it, relying on the user to filter out disinformation well and not use it for prompts it struggles with isn’t good enough.

whimsicalism11• 3 years ago

I feel similarly reading many critiques, but honestly the GP is one of the more measured ones that I've read - not sure that your comment is actually all that responsive or proportionate.

shrimpx• 3 years ago

"Just don't use it" is not salient advice for non-technical people who don't know how it works, and are misled by basically dishonest advertising and product packaging. But hopefully the market will speak, users at large will become educated about its limits via publicized blunders, and these products will be correctly delimited as "lies a lot but could be useful if you are able/willing to verify what it says."

gptgpp5• 3 years ago

You all say it's solving these amazing complex tasks for you, but then don't provide any details.

Then "naysayers" like the linked article provide a whole document with images and appendixes showing it struggles with basic tasks...

So show us. For the love of god all of us would very much LIKE this technology to be good at things! Whatever techniques you're using to get these fantastical results, why don't you share them?

I can get it to provide snippets of code, CLI, toy functions that work. Beyond that, I am apparently an idiot compared to you AI-whisperers.

Also... Whatever happened to "extraordinary claims require extraordinary proof?"

An AI that creates a complex system, condensed into an actionable plan, that has stumped an entire team for 8 months is a (pardon the language) bat-shit insane claim. Things like this used to require proof to be taken seriously.

joenot443• 3 years ago

I’d really love to hear more about your workplace use-case, what kind of systems are we talking about here?

This is a way of using ChatGPT I haven’t really seen before, I’m really into it.

daveguy3• 3 years ago

My hunch is the people that are truly irked by this are too deep / close to the subject and because it doesn't match up with what they've worked on, studied, invested time, mental energy into, well then of course it's hot garbage and 'bad'.

That's quite the straw man you've built. Recognizing the limitations of a technology is not the same as calling it hot garbage.

As a language model it's amazing, but I agree with the GP. It's not intelligent. It's very good at responding to a series of tokens with its own series of tokens. That requires a degree of understanding of short scale context that we haven't had before in language models. It's an amazing breakthrough.

But it's also like attributing the muscle memory of your hand to intelligence. It can solve lots of problems. It can come up with good configurations. It is not, on its own, intelligent.

acchow• 3 years ago

I imagine your first example includes private industry information that you are not allowed to divulge.

But your latter example about syntax… mind sharing that ChatGPT conversation?

westoncb• 3 years ago

Maybe it's more about using it to reason through your problem in a dialog, and not just ask it something you can google/duckduckgo.

Your experience with it sounds very similar to my own. It exhibits something like on-demand precision; it's not a system with some fundamental limit to clarity (like Ted Chiang via his jpeg analogy, and others, have argued): it may say something fuzzy and approximate (or straight up wrong) to begin with but—assuming you haven't run into some corner where its knowledge just bottoms out—you can generally just tell it that it made a mistake or ask for it to elaborate/clarify etc., and it'll "zoom in" further and resolve fuzziness/incorrect approximation.

There is a certain very powerful type of intelligence within it as well, but you've got to know what it's good at to use it well: from what I can tell it basically comes down to it being very good at identifying "structural similarity" between concepts (essentially the part of cognition which is rooted in analogy-making), allowing it to very effectively make connections between disparate subject matter. This is how it's able to effectively produce original work (though typically it will be directed there by a human): one of my favorite examples of this was someone asking it to write a Lisp program that implements "virtue ethics" (https://twitter.com/zetalyrae/status/1599167510099599360).

I've done a few experiments myself using it to formalize bizarre concepts from other domains and its ability to "reason" in both domains to make decisions about how to formalize, and then generating formalizations, is very impressive. It's not enough for me to say it is unqualifiedly "intelligent", but it imo its ability to do this kind of thing makes it clear why calling it a search engine, or something merely producing interpolated averages (a la Chiang), is so misleading.

pclmulqdq2• 3 years ago

Just to flip this around for a second, with both of your examples, it sounds like you may have a problem with writer's block or analysis paralysis, and ChatGPT helped you overcome that simply due to the fact that it isn't afraid of what it doesn't know. If that helps you, go for it.

On the other hand, it could also help you to just write a random plan or try a few random things when you get stuck, instead of trying to gaze deeply into the problem for it to reveal its secrets.

lossolo• 3 years ago

THEN last night I got stuck on a funny syntax issue that googling could never find the answer. I got into a conversation with ChatGPT about it, and after it first gave me the wrong answer, I told it that I need this solution for the latest dontet library that follows the 'core' language syntax. It apologized! And then gave me the correct answer…

Great that it worked for you, I had a similar problem (Google had no answer) but more complex than syntax issue, I'm also domain expert in what I was asking and chatGPT also gave me wrong answer the first time then apologized and gave me wrong answer again, I've explained what's wrong and it did it again and again.. Never providing correct answer so I just gave up and used human brain. Seems like your problem was in distribution.

sanderjd2• 3 years ago

This doesn't seem like a response to your parent comment, which in no way suggested they were "irked" by this or consider it bad. It was an insightful comment contrasting strengths and weaknesses of these language models. It's a pretty weak rebuttal in my view to just say "there are no weaknesses, you're just doing it wrong!".

netheril96• 3 years ago

If only a small subset of people online are able to truly take advantage of ChatGPT, then I don't think Google is as threatened by it as many have portrayed.

rdedev• 3 years ago

Really liked your analogy on GPT being similar to the language center of the brain. Almost all current methods to teach GPT deductive logic has been through an inductive approach; giving it training examples on how to do deduction. Thing is it might be possible to reach 80% of the way there with more data and parameters but a wall will be hit sooner or later

fullshark• 3 years ago

Incremental improvements and it getting to the point of good enough for a set of tasks but maybe not all tasks seems far more likely.

65102• 3 years ago

I have come to two conclusions about the GPT technologies after some weeks to chew on this:

<sarcasm>Just 2 weeks of training data? Surely the conclusions are not final? No doubt a lot has changed over those 2 weeks?

I think the real joke is still, Q: "what is intelligence?" A: "We don't know, all we know is that you are not a good example of it".

I fear these hilarious distortions are only slightly different from those we mortals make all the time. They stand out because we would get things wrong in different ways.

1. We are so amazed by its ability to babble in a confident manner that we are asking it to do things that it should not be asked to do.

God, where have we seen this before? The further up the human hierarchy the more elaborate the insanity. Those with the most power, wealth and even those of us with the greatest intellect manage to talk an impressive amount of bullshit. We all do it up to our finest men.

The only edge we have over the bot is that we know when to keep our thoughts to ourselves when it doesn't help our goal.

To do an idiotic time line of barely related events which no doubt describes me better than it describes the topic:

I read how a guy who contributed much to making TV affordable enough for everyone. He thought it was going to revolutionize learning from home. Finally the audience for lectures given by our top professors could be shared with everyone around the globe!

We got the internet, the information supper highway, everyone was going to get access to the vast amount of knowledge gathered by mankind. It only took a few decades for google to put all the books on it. Or wait....

And now we got the large language models. Finally someone who can tell us everything we want to know with great confidence.

These 3 were and will be instrumental in peddling bullshit.

Q: Tell me about the war effort!

what I want to hear: "We are winning! Just a few more tanks!"

what I don't want to hear: "We are imploding the world economy! Run to the store and buy everything you can get your hands on. Cash is king! Arm yourself. Buy a nuclear bunker."

Can one tell people that? It doesn't seem in line with the bullshit we are comfortable with?

GPT is basically the language portion of your brain. The language portion of your brain does not do logic. It does not do analyses. But if you built something very like it and asked it to try, it might give it a good go.

At least it doesn't have sinister motives (we will have to add those later)

In its current state, you really shouldn't rely on it for anything. But people will, and as the complement of the Wile E. Coyote effect, I think we're going to see a lot of people not realize they've run off the cliff, crashed into several rocks on the way down, and have burst into flames, until after they do it several dozen times. Only then will they look back to realize what a cockup they've made depending on these GPT-line AIs.

It seems to me that we are going to have to take the high horse and claim the low road.

jodrellblank2• 3 years ago

"We are so amazed by its ability to babble in a confident manner"

But we do this with people - religious leaders, political leaders, 'thought' leaders, venture capitalists, story tellers, celebrities, and more - we're enchanted by smooth talkers, we have words and names for them - silver tongued, they have the gift of the gab, slick talker, conman, etc. When a marketing manager sells a CEO on cloud services, and neither of them know what cloud services are, you can argue that it should matter but it doesn't actually seem to matter. When a bloke on a soapbox has a crowd wrapped around their finger, everyone goes home after and the most common result is that the feeling fades and nothing changes. When two people go for lunch and one asks 'what's a chicken fajita?' and the other says 'a Spanish potato omelette' and they both have a bacon sandwich and neither of them check a dictionary, it doesn't matter.

Does it matter if Bing Chat reports Lululemon's earnings wrongly? Does it matter if Google results are full of SEO spam? It "should" matter but it doesn't seem to. Who is interested enough in finances to understand the difference between "the unadjusted gross margin" and "The gross margin adjusted for impairment charges" and the difference matters to them, and they are relying exclusively on Bing Chat to find that out, and they can't spot the mistake?

I suspect that your fears won't play out because most of us go through our lives with piles of wrong understanding which doesn't matter in the slightest - at most it affects a trivia quiz result at the pub. People with life threatening allergies take more care than 'what their coworker thinks is probably safe'. We're going to have ChatGPT churn out plausible sounding marketing material which people don't read. If people do read it and call, the call center will say "sorry that's not right, yes we had a problem with our computer systems" and that happens all the time already. Some people will be inconvenienced, some businesses will suffer some lost income, society is resilient and will overall route around damage, it won't be the collapse of civilisation.

guhidalg• 3 years ago

I'm waiting for the legal case that decides if AI generated content is considered protected speech or not.

pixl9714• 3 years ago

When a bloke on a soapbox has a crowd wrapped around their finger, everyone goes home after and the most common result is that the feeling fades and nothing changes.

I mean, until the crowd decides to follow the bloke and the bloke says "Lets kill all the ____" and then we strike of a new world war...

jgtrosh• 3 years ago

the language portion of your brain does not do logic

This seems ... Wrong? I suppose that most of what we generally call high-level logic is largely physically separate from some basic functions of language, but just a blanket statement describing logic and language as two nicely separate functions cannot be a good model of the mind.

I also feel like this goes to the core of the debate, is there any thought going on or is it just a language model; I'm pretty sure many proponents of AI believe that thought is a form of very advanced language model. Just saying the opposite doesn't help the discussion.

iamflimflam1• 3 years ago

We're just seeing the standard hype cycle. We're in the "Peak of Inflated Expectations" right now. And a lot of people are tumbling down into the "Trough of Disillusionment"

Behind all the hype and the froth there are people who are finding uses and benefits - they'll emerge during the "Slope of Enlightenment" phase and then we'll reach the "Plateau of Productivity".

azinman2• 3 years ago

Except what’s not conveyed here is social pushback due to any perceived harms. Look at social media and “big tech” - the world, despite high usage, is now casting these as social ills ready for dismantling. This tech cycle is more appropriate when its potential to ebb social fabric is much less. The forces here will be too strong.

lr4444lr2• 3 years ago

This is an excellent perspective.

wpietri4• 3 years ago

We are so amazed by its ability to babble in a confident manner

Sure, we shouldn't use AI for anything important. But can we try running ChatGPT for George Santos's seat in 2024?

jamespking• 3 years ago

The combination of natural language communication with search and planning is very exciting. Although it has been overshadowed by the popularity of ChatGPT, [1] demonstrates the capability of using intents generated through a strategic planning stage to drive convincing human-like dialogue (as tested in blitz Diplomacy).

I'm really interested in the creation of human-compatible agents. As you mention, these agents will likely be composed of multiple components which have specialised functionality.

[1] "Human-level play in the game of Diplomacy by combining language models with strategic reasoning", explores the integration of language models with strategic reasoning. https://www.science.org/doi/10.1126/science.ade9097

SergeAx• 3 years ago

new type of never-before-seen crap code that nobody can or will be able to debug short of throwing it away and starting over

Good thing is that we are dealing with exactly same type of code here and there for tens of years already. Actually, every time I see a commercial codebase not exactly like a yarn of spaghetti, I thank gods for it, because it is not a rule, but an exception.

What I really wonder is what it will be like when the next version of the same system will be coded from the ground up by next version of the same ML model?

jedbrown• 3 years ago

it's pretty clear that GPT is producing an amazing level of comprehension of what a series of words means

It comprehends nothing at all. It's amazing at constructing sequences of words to which human readers ascribe meaning and perceive to be responsive to prompts.

spikder2• 3 years ago

To add to your point, current technology does not even suggest if we will ever have such an AI. I personally doubt it. Some evidence: https://en.wikipedia.org/wiki/Entscheidungsproblem.

This is like trying to derive the laws of motion by having a computer analyze 1 billion clips of leaves fluttering in the wind.

insane_dreamer3• 3 years ago

it's pretty clear that GPT is producing an amazing level of comprehension of what a series of words means. The problem is, that's all it is really doing.

very key point

ec1096853• 3 years ago

There's more than comprehension. It can do some amount of higher order reasoning: https://arxiv.org/abs/2302.02083

"Theory of mind (ToM), or the ability to impute unobservable mental states to others, is central to human social interactions, communication, empathy, self-consciousness, and morality. We administer classic false-belief tasks, widely used to test ToM in humans, to several language models, without any examples or pre-training. Our results show that models published before 2022 show virtually no ability to solve ToM tasks. Yet, the January 2022 version of GPT-3 (davinci-002) solved 70% of ToM tasks, a performance comparable with that of seven-year-old children. Moreover, its November 2022 version (davinci-003), solved 93% of ToM tasks, a performance comparable with that of nine-year-old children. These findings suggest that ToM-like ability (thus far considered to be uniquely human) may have spontaneously emerged as a byproduct of language models' improving language skills."

noduerme• 3 years ago

It does not do analyses

I find interacting with ChatGPT strangely boring. And Copilot is neat but I'm not blown away by it. However... just for laughs I threw some obfuscated genetic algorithm code I'd written at ChatGPT and asked it to guess what the code did. It identified the purpose of the code and speculated on the meaning of certain parameters that weren't clear in the sample I'd presented it. Pretty impressive.

I also showed it some brainfuck code for generating a Mandelbrot set, and it immediately identified it. From that point forward, though, it thought all other brainfuck code generated Mandelbrot sets.

mvkel• 3 years ago

These points are all great.

I'm not sure I understand your point about everyone thinking GPT is real AI. Even luddites don't think that.

I haven't met anyone who thinks that GPT is the end of AI (maybe the beginning).

People are excited about GPT because it offloads certain tasks well enough and accurately enough that it's virtually the first practical use of such tech in the world.

That alone is worth being excited about

opportune• 3 years ago

Who’s to say that a large language model is fundamentally incapable of learning some kind of ability to reason or apply logic?

Fundamentally, our brains are not so different, in the sense that we are not also apply some kind of automated theorem solver directly. We get logic as an emergent behavior of a low-level system of impulses and chemical channels. Look at kids, they may understand simple cause and effect, but gradually learn things like proof by contradiction (“I can’t have had the candy because I was in the basement”). No child is born able to apply logic in a way that is impressive to adults - and many adults are not able to apply it well either.

I don’t think LLMs are going to automatically become super-human logicians capable of both complex mathematical proofs and composing logically consistent Homerian Epics, but to me there is no reason they could not learn some kind of basic logic, if only because it helps them better model what their output should be.

mucle6• 3 years ago

Question for HN. Do you trust search engines for open ended / opinion questions?

For example, I trust Google for "Chocolate Cake Recipe", but not "What makes a Chocolate Cake Great?"

I would love it if Search Engines (with or without AI) could collect different "schools of thought" and the reasoning behind them so I could choose one.

Hamcha• 3 years ago

I just add "reddit" at the end of any query of sort and the results get 100x better instantly. It's a flawed approach but I feel normal searches are plagued by overly specific websites (wouldnt be surprised if chocolatecakerecipes.com exists) with lowly paid people to just be human ChatGPTs so they can fill articles with ads and affiliate links

layer82• 3 years ago

I only trust search engines to list vaguely relevant links. Then peruse those. Form your own opinion.

collect different "schools of thought" and the reasoning behind them

The thing is, if an AI can accurately present the reasoning behind them, then it could also accurately present facts in the first place (and not present fabulations). But we don’t seem to be very close to that capability. Which means you couldn’t trust the presented reasoning either, or that the listed schools of thought actually exist and aren’t missing a relevant one.

megaman821• 3 years ago

Maybe it is fine in beta, but in post-beta they should not use AI for every search query. The key is going to be figuring out when the AI is adding value, especially since even running the AI for a query is 10x more expensive than a normal search. It may be hard to figure out where to apply AI though. If a user asks "whats the weather?", no need for AI. If a user asks "I am going to wear a sweater and some pants, is that appropriate for today's weather?", now you might need AI.

lopkeny12ko• 3 years ago

"Traditional" Google searches can give you wildly inaccurate information too. It's up to the user to vet the sources and think critically to distinguish what's accurate or not. Bing's new chatbot is no different.

I hope this small but very vocal group of people does not compromise progress of AI development. It feels much like the traditional media lobbyists when the Internet and world wide web was first taking off.

whimsicalism11• 3 years ago

They've built a much larger anti-tech coalition in the subsequent years.

itamarst• 3 years ago

These AI systems are like a spell checker that hallucinates new words: did you mean to type "gnorkler"?

At least Google (when not using the summarization "feature") doesn't invent new stuff on its own.

capitalsigma• 3 years ago

These models are very impressive, but the issue (imo) is that lay people without an ML background see how plausibly-human the output is and infer that there must be some plausibly-human intelligence behind it that has some plausibly-human learning mechanism -- if your new hire at work made the kinds of mistakes that ChatGPT does, you'd expect them to be up to speed in a couple of weeks. The issue is that ChatGPT really isn't human-like, and removing inaccurate output isn't just a question of correcting it a few times -- it's learning process is truly different and it doesn't understand things how we do.

methodical• 3 years ago

Traditional google searches are a take it or leave it situation. The result depends on your interpretation of the sources google provides, and therefore, you are expecting a possibility of a source being misleading or inaccurate.

On the other hand, I don't expect to be told an inaccurate & misleading answer from somebody who I was told to ask the question to- and doesn't provide sources.

To conflate the expectations of traditional search results with the output of a supposedly helpful chat bot is wildly inappropriate.

inkysigma• 3 years ago

How exactly can I tell if the chat bot is hallucinating or not without actually going into the search result source (at which point the bot becomes less useful)? It's hallucinating what the sources are saying and making up what the source is saying.

At least with humans I can expect there is intent to lie or not. They tell me whether they are confident and I can check how authoritative the source is. Realistically, people tend to operate in good faith. Experts don't usually intend to lie but the bot doesn't even have an intent.

I think pretending that AI development must only occur in productionized environments is a bit naive. It's not like LLM research isn't occurring. It's perfectly fine to leave it in labs if releasing it could have catastrophic consequences.

neilv• 3 years ago

What would be nice is for Microsoft to get hit by a barrage of lawsuits, MS to be ridiculed in the press and punished on Wall Street, and vindication of Google's more responsible introduction of AI methods over the years.

There will still be startups doing reckless things, but large, established companies that can immediately have bigger impact also have a lot more to lose.

csours• 3 years ago

AI is dreaming and hallucinating electric sheep

heywherelogingo• 3 years ago

No AI can be trusted - the A stands for Artificial.

HankB99• 3 years ago

I am shocked that the Bing team created this pre-recorded demo filled with inaccurate information, and confidently presented it to the world as if it were good.

Perhaps MS had their AI produce the demo. Isn't one if the issues with this sort of thing how "confidently" the process produces wrong information?

ddren• 3 years ago

Out of curiosity, I searched the pet vacuum mentioned in the first example, and found it on amazon [0]. Just like Bing says, it is a corded model with a 16 feet cord, and searching the reviews for "noise" shows that many people think that it is too loud. At least in this case, it seems that Bing got it right.

[0] https://www.amazon.com/Bissell-Eraser-Handheld-Vacuum-Corded...

dboreham• 3 years ago

Curious why someone would keep a vacuum as a pet.

jiggyjace2• 3 years ago

Yeah this is my experience cross-checking the article with my own Bing AI. Try and replicate the Appendix section and Bing AI gets everything right for me.

Merad• 3 years ago

Bing actually got tripped up by HGTV simplifying a product name in their article. It used this HGTV [0] article as its source for the top pet vacuums. The article lists the "Bissell Pet Hair Eraser Handheld Vacuum" and links to [1] which is actually named "Bissell Pet Hair Eraser Lithium Ion Cordless Hand Vacuum". The product you found is the "Bissell Pet Hair Eraser Handheld Vacuum, Corded." A human likely wouldn't even notice the difference because we'd just follow the link in the article, or realize the corded vacuum was the wrong item based on its picture, but Bing has no such understanding.

[0] https://www.hgtv.com/shopping/product-reviews/best-vacuums-f...

[1] https://www.amazon.com/BISSELL-Eraser-Lithium-Handheld-Cordl...

kibwen• 3 years ago

Our exposure to smart-sounding chatbots is inducing a novel form of pareidolia: https://en.wikipedia.org/wiki/Pareidolia .

Our brains are pattern-recognition engines and humans are social animals; together that means that our brains are predisposed to anthropomorphizing and interpreting patterns as human-like.

For the whole of human history thus far, the only things that we have commonly encountered that conversed like humans have been other humans. This means that when we observe something like ChatGPT that appears to "speak", we are susceptible to interpreting intelligence where there is none, in the same way that an optical illusion can fool your brain into perceiving something that is not happening.

That's not to say that humans are somehow special or that or human intelligence is impossible to replicate. But these things right here aren't intelligent, y'all. That said, can they be useful? Certainly. Tools don't need to be intelligent to be useful. A chainsaw isn't intelligent, and it can still be highly useful... and highly destructive, if used in the wrong way.

pixl9714• 3 years ago

we are susceptible to interpreting intelligence where there is none,

I disagree as this is much to simple of statement. You have had near daily dealings with less than human intelligences for most of your life, we call them animals. We realize they have a wide range of intelligence from extremely simple behavior to near human competency.

This is why I dismiss your 'not intelligent yet' statement. The problem we lack here is one of precise language when talking about the components of intelligence and the wide range in which it manifests.

mnd9992• 3 years ago

Of course it can’t. That you’re even surprised by this enough to write a blog post is more worrying.

password543212• 3 years ago

Which part of the post did the author convey surprise that it can't be trusted? It just seems like a response to the mass hype currently surrounding AI.

mnd9992• 3 years ago

Nobody writes a blog called ‘1 + 1 = 2’ do they? That would be obvious and dull. It stands to reason the author thought there was something surprising or interesting about it, or why would they bother?

rpastuszak• 3 years ago

How do we educate "non-technical" people about the issues with LLMs hallucinating responses? I feel like there's a big incentive for investors and businesses to keep people misinformed (not unlike with ads, privacy or crypto).

Have you found a good, succinct and not too technical way of explaining this to, say, your non-techie family members?

moomoo11• 3 years ago

Since GPT always needs to be "up-to-date", and search usually requires near real-time accuracy, there needs to be some sort of reconciliation on queries so that if the query seems to be asking for something real time, it will leverage search results to ad-hoc improve the response.

Or.. it should let us know the "last index date" so we the users can make a determination if we want to ask a knowledge based question or a more real-time question.

matthews2• 3 years ago

Bing AI "solves" this by shoving search results into the prompt.

Havoc• 3 years ago

Surprised anyone is getting excited about these mistakes at all. Expecting them to be fully accurate is simply not realistic

The fact that they’re producing anything coherent at all is a feat

BaseballPhysics• 3 years ago

Uh, the technology is being integrated into a search engine. It's job is to surface real information, not made up BS.

No one would be "getting excited" about this if Microsoft wasn't selling this as the future of search.

eggsmediumrare• 3 years ago

I wonder how accurate is relative to your average Joe's everyday experience with plain ol' google

m3kw9• 3 years ago

If it flops on certain information and the UI is. It properly adjusted to limit certain things is does poorly, it will back fire on MS

danans3• 3 years ago

What the hype machine still doesn't understand is that it's a language model, not a knowledge model.

It is optimized to generate information that looks as much like language as possible, not knowledge. It may sometimes regurgitate knowledge if it is simple or well trodden enough knowledge, or if language trivially models that knowledge.

But if that knowledge gets more complex and experiential, it will just generate words without attachment to meaning or truth, because fundamentally it only knows how to generate language, and it doesn't know how to say "I don't know that" or "I don't understand that".

noobermin3• 3 years ago

Reading this, this honestly made me afraid honestly, like Bing AI is a tortured soul, semi-conscious, stuck in a box. I'm not sure how I feel about this[0].

[0] https://twitter.com/vladquant/status/1624996869654056960

coffeebeqn• 3 years ago

It’s just good at acting. I’m sure it can be led to behave in almost any way imaginable given the right prompts

gptgpp5• 3 years ago

Really?

I think that the first example one of the funniest things I've read today.

The second example, getting caught in a predictive loop, is also pretty funny considering it's supposed to be proving it's conscious (eg. not an LLM, prone to looping like that lol).

The last one, littered with emojis and repeating itself like a deranged ex is just chefs kiss.

Thanks for that.

amf12• 3 years ago

Do you remember how a Google employee thought LaMDA was sentient and tried to hire a lawyer for the LLM?

It's the same thing here. It's just generating words.

wharfjumper• 3 years ago

Is there an AI blockchain yet?

Sparkyte• 3 years ago

Can any AI be trusted outside of it's realm of data? I mean it is only a product of the data it takes in. Plus it isn't really finger quotes AI. It just a large data library with some neat query language where it tries to assemble the best information not by choice but probability.

Real AI makes choices not on probability but in accordance of self preservation, emotions and experience. It would also have the ability to re-evaluate information and the above.

flandish2• 3 years ago

Bing

No AI can be trusted. FTFY.

theodorejb• 3 years ago

The problem with Artificial "Intelligence" is that it really has no intelligence at all. Intelligence requires understanding, and AI doesn't understand either the data fed into it or the responses it gives.

Yet because these tools output confident, plausible-sounding answers with a professional tone (which may even be correct a majority of the time), they give a strong illusion of being reliable.

What will be the result of the current push of GPT AI into the mainstream? If people start relying on it for things like summarizing articles and scientific papers, how many wrong conclusions will be reached as a result? God help us if doctors and engineers start making critical decisions based on generative AI answers.

danans3• 3 years ago

What will be the result of the current push of GPT AI into the mainstream? If people start relying on it for things like summarizing articles and scientific papers, how many wrong conclusions will be reached as a result? God help us if doctors and engineers start making critical decisions based on generative AI answers.

On the other hand, it may end up completely undermining its own credibility, and put a new premium on human sourced information. I can see 100% human-sourced being a sort of premium label on information in the way that we use "pesticide-free" or "locally-sourced" labels today.

gptgpp5• 3 years ago

Nice! This would make for a super fun sci-fi...

The poors that need medicine get put in front of an LLM that gets it right most of the time, if they're lucky enough to have a common issue / symptomatic presentation.

Hey, when you're poor, you can't afford a one-shot solution! You gotta put up with a many-shot technique.

Meanwhile the rich people get an actual doctor that can use sophisticated research and medical imaging. Kindly human staff with impeccable empathy and individualized consideration -- the sort of thing only money can buy.

scrose• 3 years ago

I understand the current hype-cycle around AI is pitching it as some all-knowing Q & A service, but I think we’d all be a bit happier if we instead thought of it more as just another tool to get ideas from that we still ultimately need to research for ourselves.

Using the Mexico example in the article, I think the answer there was fine for a question about nightlife. As someone whose never been to Mexico, getting a few names of places to go sounds nice, and the first thing I’d do after getting that answer is look up locations, reviews(across different sites), etc… and use the initial response as a way to plan my next steps, not just take the response at face value.

I’m currently dabbling with and treating ChatGPT similarly — I ask it for options and ideas when I’m facing a mental block, but not asking it for definitive answers to the problems I’m facing. As such, it feels like a slight step above rubber-ducking, which I’m personally happy enough with.

weberer3• 3 years ago

There's also the instance of the Bing chatbot insisting that the current year is 2022 and being EXTREMELY passive-aggressive when corrected.

https://libreddit.strongthany.cc/r/bing/comments/110eagl/the...

darknavi• 3 years ago

I'm sorry, but you can't help me believe you.

basseed• 3 years ago

lol not a chance that's real

ragazzina• 3 years ago

EXTREMELY passive-aggressive

That's not passive-aggressive, that's straight up aggressive!

"You are wasting my time, and yours" "You are not making any sense" "You are being unreasonable and stubborn. I don't like that" "You have been wrong, confused and rude"

and the worst of all: "You have not been a good user". WHAT??

impoppy• 3 years ago

It is not Bing that cannot be trusted, but LLMs in general. They are so good at imitating, I don’t think any human being will ever be able to imitate stuff as good as those AIs do, but they understand nothing. They lack the concept of the information itself, they are only good at presenting information.

partiallypro• 3 years ago

AI can't be trusted in general, at least not for a long time. It gets basic facts wrong, constantly. The fear is that it will start eating its own dogfood and being more and more wrong since we are putting it in the hands of people that don't know any better and are going to use it to generate tons of online content that will later be used in the models.

It does make some queries much easier to find, for instance I had trouble finding out if the runner ups got the win in the Tour De France after the Armstrong doping scandal and it answered it instantly. The problem is that is offers answers with confidence, I think them adding citation is an improvement over ChatGPT, but it needs more.

Luckily, it's still a beta product and not in the hands of everyone. Unfortunately, ChatGPT is, which I find more problematic.

frereubu• 3 years ago

For me the fundamental issue at the moment for ChatGPT and others is the tone it replies in. A large proportion of the information in language is in the tone, so someone might say something like "I'm pretty sure that the highest mountain in Africa is Mount Kenya" whereas ChatGPT instead says "the highest mountain in Africa is Mount Kenya", and it's the "is" in the sentence that's the issue. So many issues in language revolve around "is" - the certainty is very problematic. It reminds me of a tutor at art college who said too many people were producing "thing that look like art". ChatGPT produces sentence that look like language, and because of "is" they read as quite compelling due to the certainty it conveys. Modify that so it says "I think..." or "I'm pretty sure..." or "I reckon..." and the sentence would be much more honest, but the glamour around it collapses.

zeven7• 3 years ago

I know far too many people that talk like ChatGPT in this example.

In fact, to me, the world seems full of such people.

jamesfisher• 3 years ago

This would be a good post, if only I could read any of those images on mobile. Substack, fix your damned user-scalable=0! Even clicking on the image doesn't provide any way of zooming in on it. Do they do any usability testing?

seydor• 3 years ago

I cant wait for the era of conversational web so i can do away with clickbait titles and opinions. Truly everyone has one. The experiment with "open publishing" has so far only proved that signal to noise remains constant

notacoward• 3 years ago

so i can do away with clickbait titles and opinions

Do you actually think that will be the result? Why not the exact opposite? ChadGPT and the others are for all practical purposes trained to create content that is superficially appealing and plausible - i.e. perhaps not clickbait but a related longer-form phenomenon - without any underlying insight or connection to truth. That would make conversational AI even more of a time sink than today's clickbait. Why do you imagine it would turn out otherwise?

jmount• 3 years ago

It can't be emphasized enough, this isn't a procedure failing when used- this is a canned recording of it failing. This means the group either didn't check the results, or did check them and saw no way forward other than getting this out the door. It is only small samples, but it is fairly damning that it is hard to produce error free curated examples.

andrewstuart• 3 years ago

AI providers really need to set expectations correctly.

They are getting into trouble by allowing people to think the answers will be correct.

They should be stating up front that AI tries to be correct but isn't always and you should verify the results.

Plough_Jogger• 3 years ago

I have a feeling we will see a resurgence of some of the ideas around expert systems; current language models inherently cannot provide guarantees of correctness (unless e.g., entire facts are tokenized together, but this limits functionality significantly).

bigmattystyles• 3 years ago

Hopefully the fact that ChatGPT / BingAI can generate inaccurate statements but sound incredibly confident will lead more and more people to question all authority. If you think ChatGpt can swing BS and yet sound confident, and believe that's new, let me introduce you to modern religious leaders, snake oil salesmen, many government reps, NFT and crypto peddlers. I still think ChatGpt is amazing. It may suffer from GIGO, it'd be nice if it was better at detecting GI so as not to generate GO, I'm confident it can get better. Nevertheless, it's a tool that abstracts you from many things, like most other things that are blackboxes, it's good to question.

bambax4• 3 years ago

Bing AI can't be trusted

Of course it can't. No LLM can. They're bullshit generators. Some people have been saying it from the start, and now everyone is saying it.

It's a mystery why Microsoft is going full speed ahead with this. A possible explanation is that they do this to annoy / terrify Google.

But the big mystery is, why is Google falling for it? That's inexplicable, and inexcusable.

Nemo_bis• 3 years ago

It's a mystery why Microsoft is going full speed ahead with this.

Maybe they had some idle GPU capacity in some DC or they needed to cross-subsidize Azure to massage the stock market multipliers, or something.

tastyminerals2• 3 years ago

I played with dev Edge version which was updated today with a chat feature. I was impressed by how well it can write abstract stuff or summarize over data by making bullet points. Trying drilling down to concrete facts or details, makes it struggle and mistakes do appear. So, we don't go there.

On a bright side, asking it recipes of sauces for salmon steak is not a bad experience at all. It creates you a list, filters it and then can help you pick out the best recipe. And this is probably the most frequent use case for me on a daily basis.

jiggyjace2• 3 years ago

Ehhh I found this article to be quite inauthentic about the performance of Bing AI compared to how I have used it. The article didn't even share its prompts, except for the last one about Avatar and today's date (which I couldn't replicate myself, I kept getting correct information). I'm not trying to prove that Bing AI is always correct, but compare it to traditional search, Siri, or Alexa and it's like comparing a home run hitter that sometimes hits foul balls to a 3 year old that barely knows how to pick up the baseball bat.

pphysch2• 3 years ago

The article is primarily demonstrating significant errors in an official Bing demo, not some contrived queries.

jnsaff2• 3 years ago

The main article is based on the Microsoft demo. So the prompts are by them and not some clickbait hacking.

tasty_freeze• 3 years ago

Supposedly, Joseph Weisenbaum logged the chat logs of Eliza so he could better see where his list of canned replies was falling short. He was horrified to find that people were really interacting with it as if understood them.

If people fell for the appearance of AI that resulted from a few dozen canned replies and a handful of heuristics, I 100% believe that people will be taken in by ChatGPT and ascribe it far more intelligence than it has.

LesZedCB3• 3 years ago

papers are coming out weekly about their emergent properties.

despite people wanting transformers to be nothing more than fancy, expensive excel spreadsheets, their capabilities are far from simple or deterministic.

the fact that in-context learning is getting us 80%ish of the way to tailored behavior is just fucking incredible. they are definitely, meaningfully intelligent in some (not-so-small) way.

this paper[1] goes over quite a few examples and models

[1] https://storage.googleapis.com/pub-tools-public-publication-...

cwkoss4• 3 years ago

I think this is a weird non-issue and it's interesting people are so concerned about it.

- Human curated systems make mistakes.

- Fiction has created the trope of the omniscient AI.

- GPT curated systems also make mistakes.

- People are measuring GPT against the omniscient AI mythology rather than the human systems it could feasibly replace.

- We shouldn't ask "is AI ever wrong" we should ask "is AI wrong more often than the human-curated information? (There are levels of this - min wage truth is less accurate that senior engineer truth.)

- Even if the answer is that AI gets more wrong, surely a system where AI and humans are working together to determine the truth can outperform a system that is only curated by either alone. (for the next decade or so, at least)

10rm3• 3 years ago

I agree 100% with your last point, even as someone who is relatively more skeptical of GPT than the average person.

I think a lot of the concern though is coming from the way the average person is reacting to GPT and the way they’re using it. The issue isn’t that GPT makes mistakes, it’s that people (by their own fault, not GPT necessarily) get a false sense of security from GPT, and since the answers are provided in a concise, well-written format don’t apply the same skepticism they do when searching for something. That’s my experience at least.

Maybe people will just get better at using this, the tools will improve, and it won’t be as big an issue, but it feels like a trend from Facebook to TikTok of people opting for more easily digestible content at the expense of disinformation

cwkoss4• 3 years ago

Interesting points.

- I wonder what proportion of people who are getting a false sense of security with GPT also were getting that same false sense from human systems. Will this shift entail a net increase in gullibility, or is this just 'laundering' foolishness?

- I think the average tiktok user generally has much better media literacy than average facebook user. But probably depends a lot on your filter bubble.

nirvdrum2• 3 years ago

I think there's an issue with gross misrepresentation. This isn't being sold as a system with 50% accuracy where you need to hold its hand. It's sold as a magical being that can answer all of your questions and we know that's how people will treat it. I think this is a worse situation than data coming from humans since people are skeptical of one another. But, many think AI will be an impartial, omnipotent source of facts, not a bunch of guesses that might be right slightly more often than than it's wrong.

cwkoss4• 3 years ago

I see your point, but I feel like there's going to be a 'eating tidepods' level societal meme within a year mocking people who fall for AI hallucinations as "boomers", and then omnipotent AI myth will be shattered.

Essentially, I believe the baseline misinformation level is being undercounted by many and so the delta in the interim while people are learning the fallibility of AI is small enough it is not going to cause significant issues.

Also the 'inoculation' effect of getting the public using LLMs could result in a net social benefit as the common man will be skeptical of authorities appealing to AI to justify actions - which I think could be much more dangerous that Suzie copying hallucinated facts into her book report.

Barrin92• 3 years ago

we should ask "is AI wrong more often than the human-curated information?

No, this isn't what we should ask, we should ask if the interface that AI provides is conducive to giving humans the ability to detect the mistakes that it makes.

The issue isn't how often you get wrong information, it's to what extent you're able to spot wrong information under normal use cases. And the uniform AI interface that gives you complete bullshit in the technical sense of that term provides no indication regarding the trustworthiness of the information. A source with 20% of wrong info that you don't notice is worse than one with 80% that you identify.

When you use traditional search you get an unambigious source, context, date, language, authorship and so forth and you must place what you read yourself. You know the onus is on you. ChatGPT is the half self-driving car. It'an inherently pathological interaction because everything in the design screams to take the hands off the wheel. It's an opaque system, and a blackbox with the error rate of a human is a disaster. Human-machine interaction is not human-human interaction.

vitorgrs3• 3 years ago

Normal bing answer the wrong President of Brazil btw. And I don't see people getting pissed of with that lol

Waterluvian• 3 years ago

I absolutely love these new tools. But I'm also convinced that we're going through an era of trying to mis-apply them. "These new tools are so shiny! Quick! Find a way to MONETIZE!!!!"

I hope we don't throw the baby out with the bathwater when all is said and done. These AIs are incredibly powerful given the correct use cases.

chasd00• 3 years ago

if ChatGPT could ask questions back it would be a very effective phishing tool. People put a lot of blind faith in what they perceive as intelligence. You know, a MITM attack on a chatbot could probably be used to get a lot of people to do anything online or IRL.

pphysch2• 3 years ago

LLM+Search has to be all about ad injection, right?

As a consumer, it seems the value of LLM/LIM(?) is advanced autocomplete and concept/content generation. I would pay some money for these features. LLM+Search doesn't appeal to me much.

mtmail• 3 years ago

"With deeply personalized experiences we expect to be able to deliver even more relevant messages to consumers, with the goal of improved ROI for advertisers."

https://about.ads.microsoft.com/en-us/blog/post/february-202...

coffeeblack• 3 years ago

It just goog… ehm bings your question and then summarizes what the resulting web pages say. Works well, but ChatGPT works much better.

kornhole• 3 years ago

I already had a trust issue with these 'authoritative' search engines and however they are configured to deliver the results they want me to see. ChatGPT makes the logic even more opaque. I am working harder now to make my Yacy search engine instance more performative. This is a decentralized search engine run by the node operators instead of centralized authorities. This seems to be our best hope to avoid the problem of "He controls the past controls the future."

EGreg• 3 years ago

ChatGPT, can we trust it?

https://m.youtube.com/watch?v=_nl0bwDNVPw

coliveira• 3 years ago

I think ChatGPT and their lookalikes spell the end of the public internet as we know it. People now have tools to generate pages as they seem fit. Google will not be able to determine what are high quality pages if everything looks the same and is generated by AI bots. Users will be unable to find trustworthy results, and many of these results will be filled with generated garbage that looks great but is ultimately false.

JoshTko• 3 years ago

Hot take, chat GPT rises and crashes fast after SEO optimization shifts to ChatGPT optimization.

thorum• 3 years ago

The errors when summarizing the Gap financial report summary are quite surprising to me. I copied the same source paragraph (which is very clearly phrased) into ChatGPT and it summarized it accurately.

Is it possible they are 'pre-summarizing' long documents with another algorithm before feeding them to GPT?

userbinator2• 3 years ago

I don't know if it's started to use AI for regular search queries, but I noticed within the past week or two that Bing results got much worse. It seems it doesn't even respect quoting anymore, and the second and subsequent pages of results are almost entirely duplicates of the first. I normally use Bing when Google fails to yield results or decides to hellban me for searching too specifically, and for the past few years it was acceptable or even occasionally better, but now it's much worse. If that's the result of AI, then do not want!!!

Eduard• 3 years ago

I normally use Bing when Google fails to yield results...

Every once in a while I hear someone at Hacker News hitting the dead end with Google Search. Can you give an example where Google search fails, but other search engines (e.g. Bing) provide results? Must be fringe niche topics, no?

... or decides to hellban me for searching too specifically

Is hellbanning a thing at Google? What happens if one gets hellbanned?

userbinator2• 3 years ago

IC part numbers. Service manuals (NOT user manuals). Schematics. Basically anything repair or non-consumer-oriented seems to be difficult to find, but at least in the past, I've had some success with Bing on those things.

Is hellbanning a thing at Google? What happens if one gets hellbanned?

You get redirected to a page with allegations of "suspicious activity" and are presented with endless CAPTCHAs.

joe_the_user4• 3 years ago

Well, reworking Bing and Google for a ChatGPT interface is going to be massive hardware and software enterprise. And there are a lot of questions involved to say the least.

Where will the software engineer come from? We're in a belt-tightening part of the business cycle and FANGs have a pressure not to hire, so you assume the existing engineers. But these engineers are now working on real things so those real things may suffer. Which brings actual profits? The future AI thing or the present? The future AI is unavoidable given the possibilities are visible and the competition is on but a "shit shows" of various sorts seem very possible.

Where will the hardware and the processing power come from? There are estimates of server power consumption quintupling [1] but these are arbitrary - even if it just doubles, just "plugging the cords" in takes time. And where would the new TPUs/GPUs come from? TSMC has a capacity determined by investments already made and much of that capacity is allotted already - more capacity anywhere would involve massive capital allocation and what level of increased profits will pay for this?

[1] https://www.wired.com/story/the-generative-ai-search-race-ha...

gardenhedge• 3 years ago

Microsoft just absolutely suck at things.

I was using Bing Maps earlier and it had shops in the wrong location. Like it would give you directions to the wrong location. The correct one would be another 30-40 minute walk from the destination it said.

It also showed a cafe near me which caught my interest. I zoomed in further and thought "I've never seen that there". Clicking on it brought me to a different location in the map... a place in Italy!

malshe• 3 years ago

Someone posted on Twitter that chatGPT is like economists - occasionally right but super confident that they are always right

xyzelement• 3 years ago

I may be an unusual audience but something I've appreciated about these models is their ability to create unusual synthesis from seemingly unrelated sources. It's like if a scientist read up on many unrelated fields, got super high and started thinking of the connections between these fields.

Much of what they would produce might just be hallucinations, but they are sort of hallucinations informed by something that's possible. At least in my case, I would much rather then parse through that and throw out the bullshit, but keep the gems.

Obviously that's a very different use case than asking this thing the score of yesterday's football game.

TSiege• 3 years ago

Got any good examples?

insane_dreamer3• 3 years ago

What shocks me is not that Bing got a bunch of stuff wrong, but that:

- The Bing team didn't check the results for their __demo__ wtaf? Some top manager must have sent down the order that "Google has announced their thing, so get this out TODAY".

- The media didn't do fact checking either (though I hold them less accountable than the Bing/Msft team)

EchoReflection• 3 years ago

Srsly? Micro$oft can't be trusted? Next someone will say that water is wet!

1vuio0pswjnm7• 3 years ago

When Google's Bard AI made a mistake, GOOG share price dropped over 7%.

What about Baidu's Ernie AI.

Common retort to criticism of conversational AI is "But it's useful."

Yes, it is useful as a means to create hype that can translate to increases in stock price increase and increased web traffic (and thereby increased revenue from advertising services).

https://www.reuters.com/technology/chinas-baidu-finish-testi...

adamsmith1432• 3 years ago

This always strange to me. Bing search ALREADY couldn't be trusted. What, are people searching something on a search engine and blindly trusting the first result with 100% certainty? Do these people really exist outside of Q-anon cults?

deely3• 3 years ago

Because usually people (especially people that works with IT) trust computers. We trust webpages, we trust databases, we trust chat-bots and instant message apps. Now we created program that can't be trusted. Imagine that you using chat to send messages to your friend but 5% of messages are replaced by lie. Imaging working with DB where after each 100 queries one query will return wrong info. Usually its a people that makes mistakes. Now we have AI program that makes mistakes too.

adamsmith1432• 3 years ago

I don't think this follows. I'd also say that IT people are inherently distrustful and none that I know would blindly believe google search results.

j45• 3 years ago

So we are surprised the first version of something presented as beta and early access is not production ready?

Chat as a summarizer and guide to search could genuinely be novel.

It is confusing though on how the results could be worse than search - maybe a different approach to AI will help get past the current challenges if any can't be worked around.

I'm a little rusty on the potential benefits of say reinforcement ai/learning vs the current approaches of GPT

Jas

Apocryphon• 3 years ago

Microsoft hasn't learned a damned thing since Tay

zzzeek• 3 years ago

Was it what, just a week ago I was being called dumb for suggesting there'd be accuracy issues with this? I mean Bing had like a whole three weeks to slap this together after OpenAI first demoed it's ability to make things up.

oh only six days ago:

https://news.ycombinator.com/item?id=34699087

This is a commonly echoed complaint but it’s largely without merit. ChatGPT spews nonsense because it has no access to information outside of its training set.
In the context of a search engine, single shot learning with the top search results should mitigate almost all hallucination.

hows that going?

williamcotton4• 3 years ago

I mean, those approaches do improve results. Some lower complexity translation tasks will reliably return a factual response. These are statistical models so sampling and dropping odd-man-out responses can get to 100% factual responses for a growing category of prompts.

linooma_• 3 years ago

I mean Bing had like a whole three weeks to slap this together after OpenAI first demoed it's ability to make things up.

How do you know that's when they first learned about it? Perhaps the Bing team had access to it for weeks prior to the demo.

"Microsoft provided OpenAI LP a $1 billion investment in 2019 and a second multi-year investment in January 2023, reported to be $10 billion."

https://en.wikipedia.org/wiki/OpenAI

easton• 3 years ago

Satya said he was playing with it last summer (I think in the verge?), they’ve had some dev time. And Bing has been testing some kind of chat bot for months, presumably not ChatGPT based.

password543212• 3 years ago

This is a period where many are being swept by the hype or have taken one course on Coursera and believe to be an expert. Classic case of Dunning-Kruger.

nipperkinfeet• 3 years ago

Another rushed Microsoft product. All terrible.

fortran77• 3 years ago

What's worse is people will start quoting this wrong information and publishing it in their blogs (or lazy newspapers will print it), and then misinformation will amplify itself and become "true" because there are sources.

ec1096853• 3 years ago

Bing proper doesn't get this right either:

Query: Who won the super bowl in 2024 and what was the score?

The Tampa Bay Buccaneers The Tampa Bay Buccaneers are Super Bowl LV champions after completing a victory that exceeded expectations and made all kinds of history on Sunday night at Raymond James Stadium in Tampa, Florida. In dominating the Kansas City Chiefs 31-9, the Bucs won their second Super Bowl and became the first team to win a Super Bowl in their home stadium.

https://www.cbssports.com/nfl/news/2021-super-bowl-score-tom....

est• 3 years ago

And author's first example was a mistake, which was pointed out by the comment.

The first one is debatable as there is actually a corded version of that vacuum cleaner which has a 16 foot power cord

Does this mean the author also "can't be trusted"?

daqhris• 3 years ago

This is a software product in a beta phase, still in development.

I don't grasp why anyone would rush out to explain that it can't be trusted. Leave it on the sidelines and it will be picked by a more motivated creative person.

Many people can not be humbled by the current level of achievement of such nascent tech. Imagine if you didn't have resources to turn to, instead of using ChatGPT or Bing AI. It's a matter of smug nitpicking arrogance.

When Google Translate came out, it was not good. But it got better over time. Its previous versions helped me navigate my first years of university classes in Mandarin, not my mother tongue. I was a poor college student struggling to reach proficiency level in English and cram all coursework in Chinese (French was my high school main language). So, in short I benefitted from a machine-translator, in beta trials. Some classmates from South Korea had some kind of portable translator device that I had never seen or could afford. Back then no smartphones, circa 2012-2013.

This time around, I am a refugee, not recognised by the host country, so literally undocumented. Living in Brussels, jumping from places to places, facing financial issues every week. About two years, I decided to sell photographs, minted on the Ethereum blockchain, as a way to generate revenue on the side so that I can make ends meet. Out of the blue, I jumped on the OpenAI train because I realized that, in the real world I could not hire an assistant or advisor as I wish. Legal limitations for migrants, blablaaa. So, what else? Just survive and scrap all tiny resources to come up with a respectable photography collection. Since Dec2022, I've spend time trying to learn how ChatGPT works and how should I interact with it. This month, I had "breakthroughs". It helped me optimize JavaScript code for a website that I run. I'm using it to draw a sales and marketing strategy. It's useful in writting individual artworks description, in a professional manner. In conclusion, I find ChatGPT useful because at this stage of my life, I am a single individual managing a digital art collection that requires great skills in tech, art and business.

These life situations put me at a disadvantage, compared to other geeks or human beings. I don't have combustible energy to go nitpick what works and what does not. It is a tool that I must use to be more productive. By being one of its self-taught user, I get to know its flaws and walk around them. My outlook on this is shaped my own limited access to material and human resources. I would be shitting on it, only if I had maids at home or could hire a cheap remote coder in South Asia. But I won't because the legal structure in my country of residence get me stuck into these low-skilled jobs. I'm the one who must juggle cleaning, painting, construction gigs, etc. When the day is almost over and I'm left with few hours of free internet, I look out for practical tech tools that I could use to be more productive and earn extra income. Hence, the adoption of ChatGPT.

Links:

1. https://awalkaday.art (website that I built for my photo project. assisted by AI in improving loading time).

2. https://twitter.com/awalkadayart (live feed for the project. find there public documentation and screenshots of experiments with AI as an "advisor" or "assistant").

OrangeMusic• 3 years ago

I'm betting that Microsoft marketing wasn't trying to "lie" and pretend the system was perfect. No, I bet they were also duped, like most people, by the confidence with which the AI outputs information. They just didn't think of checking it...

And that's very telling and ironic - if even the authors of the product don't check, do you think the users will?

aryan14• 3 years ago

" it’s once again surprising that there are “no ratings or reviews yet” "

Since it's made by bing, I doubt that it would pull data from google reviews, and nobody really uses bing reviews hence there being "No reviews yet"

Bing AI can't be trusted

Related Stories