Publishing AI Slop Is a Choice

https://daringfireball.net/linked/2024/05/24/publishing-ai-slop-is-a-choice

by mpweiher • 2 years ago

167 76 2 years ago

skilled2• 2 years ago

The AI Overviews outrage is only the beginning. It has been 11 days since they made it public in the US, and it had been in experimentation for over a year before that.

Already now, a lot of users will associate Google AI Overviews with nonsense; eat rocks, cook spaghetti with gasoline. Google also showed how easily these AI Overviews can be manipulated, since they use RAG.

I think the Slopception is only going to get worse. Slop in. Slop out.

And also, Google Research just happened to be sitting on AGREE[0]:

a learning-based framework that enables LLMs to provide accurate citations in their responses, making them more reliable and increasing user trust

Which was published yesterday.

I tried to submit it[1] but it didn’t get much love.

[0] https://research.google/blog/effective-large-language-model-...

[1] https://news.ycombinator.com/item?id=40469518

tomrod2• 2 years ago

<3 from me.

We are in the rough and tumble time of new tech. As an analogy, trains didn't always have the distance between their tracks as uniform (called gauge). It strikes me that we are in a similar time of invention. If you have ever read old patents (my experience being older pop book like "Strange Stories, Amazing Facts" and random blogposts) you'll see the truly bizzare.

shh_labs• 2 years ago

But the trains with differently sized tracks didn’t fall off the tracks because (and I imagine there is some obscure exception to this, but for the very most part) someone had made sure that the systems were safe. The distinction here is that Google is rolling out half-baked rubbish that is palpably unreliable. No sane person is going to eat rocks or put glue on their pizza, but the race is on to find some genuinely dangerous answer from this system.

It would be regrettable if the result of rolling out this type of product too soon has the effect of eroding trust in AI-based search, and delaying people from engaging with it when it actually is ready.

viking123• 2 years ago

It's going to get much much worse with all types of media tbh.

skilled2• 2 years ago

Scam callers are going to have a field day with audio for sure. Maybe I am not thinking about it right but a lot of this does feel surreal and I just can’t see the light at the end this tunnel.

No regulation, rapid development, no protective measures even for the very basic attack vectors.

fullshark• 2 years ago

Maybe people will finally be willing to pay for it.

polski-g• 2 years ago

Google AI was so much better when they used it to do useful things like create the best starcraft player in the world. Imagine if strategy game companies could license/extend DeepMind game AI.

adolph• 2 years ago

Enjoy the nonsense, this is the “egg freckles”[0] stage of the technology.

raydev• 2 years ago

Already now, a lot of users will associate Google AI Overviews with nonsense

A lot of tech-minded folks who actively follow this stuff on Twitter et al make this association, but the average person was already defaulting to reading the first (likely incorrect) hit on Google and moving on with their day. I'm not sure they'll notice a dip in quality until it directly contradicts something they know about confidently.

drpossum2• 2 years ago

I agree "Companies need to move really fast, even if that includes skipping a few steps along the way. The user experience will just have to catch up." is insane. There's a difference between a bad "user experience" and "harmful with no value" (the latter being unacceptable). The mentality that we must cut as close to the latter at the altar of market share is why we're in this state and why it's going to get much, much worse before anything gets better.

CPLX• 2 years ago

Matt Stoller’s theory is that Google at this point is like Boeing was maybe a decade ago — an incredible product company that has already been taken over by finance types, and on the way to irreversible crash.

nojvek• 2 years ago

Google has a good decade ahead of it. They own major browser and Mobile OS. Default is Google.

Most of the world is primed to Google and Bing isn’t any better. Bing is doing the same shenanigans.

A new player could beat them at their Game, but Google has a huge ML team and millions of TPUs. They could either buy them or quickly copy them.

Google did this to defend against OpenAI and Bing.

The big question is whether their bottom line will suffer. Google like every other big corp is beholden to their ever growing market cap. A few negative quarters and they face the fire.

ein0p• 2 years ago

I wonder what Gruber will be saying when his beloved Apple inevitably steps on the same rake. Somehow I think his human slop will be markedly more exculpatory

mcphage• 2 years ago

Maybe. It also may be that he likes Apple because they're less likely to step on that particular rake.

insin• 2 years ago

We must do it, promo packets are riding on this!

sunir• 2 years ago

It could be that Google has a professional CEO. In many of those cases, the customer is the board of directors. AI is driving share prices.

asadotzler2• 2 years ago

The customer isn't even the board, it's the stockholders. This has always been true though. I was at Netscape when the end came and for the last couple years of their life, before Mozilla spun out as independent, they would have quarterly layoffs to bridge the gap between what they told analysts they were targeting for the quarter and the numbers they actually produced.

The users and even the employees took a back seat to making Wall St. happy.

It was ever so, and so only the naive put their faith in mega corporations. Google was extremely useful for about 10 years from around 2002 through about 2012, but even then they were conspiring with Apple to suppress wages for top web browser and other programmers, and they were abusing their overwhelming search monopoly to gain illegal advantage in email, web browsers, and office productivity apps.

Fuck them. They were always evil, but the trade off was better back then when their evil gave us useful stuff and not AI slop. For over a decade that trade off hasn't been worth it.

nox101• 2 years ago

Google is fighting for it's life. I'm not saying that excuses things but it puts them in context. IIRC, 60% of google's profit is ads on search. Chat GPT like results remove much of the need for "search". I find myself more and more trying to find an answer on Google, not finding it, asking ChatGPT, getting an answer. And I'm NOT suggesting Google search sucks. It might but that's not my point.

I recently need to find some details of hardware implementations for a feature. I used Google to find specs for this hardware. The specs were hard to read or didn't explain things as I needed. So I asked ChatGPT and it gave me what I needed. It first regurgitated info from the specs but I was able to ask it to explain pieces in more detail and it was great! At what point do I just stop using Google search to find an answer?

Or course AI answers have their issues. ATM I wouldn't search for reviews of some new video game or movie on ChatGPT. I also wouldn't search for product reviews. Even though it will happily spew out recommendations they'll be old at best. Maybe that will get fixed but it will have all the same issues as search and more. People will write their SEO type techniques to try to get the AI to surface their product just like they try to get them to the top of search results. I guess we'll need a new name. ARO (AI Results Optimization). Searching for products at least I get the illusion of lots of various opinions. With ChatGPT I just get the bot's one opinion.

Anyway, the point is, Google has to do something. Too late and everyone will switch. Which pushes them into too soon and hence the issues we're seeing.

ToucanLoucan2• 2 years ago

I hope either the information you needed was just personal curiosity or that you verified with sources what ChatGPT was reporting. The hallucination problem is still terrible with no real solutions from OpenAI apart from "we're working on it."

IMHO, fully cosigning that google search has gotten worse and the AI shit is only going to make that even worse, using ChatGPT as a replacement feels crazy to me. Like yeah, Google gets it wrong sometimes, sure, but ChatGPT just makes shit up. Is a dumbass better or worse than a confident liar? Guess that's up to the user to decide.

llm_trw2• 2 years ago

Reading posts like this I'm reminded of the teachers who told me not to cite wikipedia because it's not reliable who 20 years later are spouting whatever right wing nonsense is the flavour of the month on facebook.

Close enough is good enough. It doesn't need to be perfect, it needs to just not be catastrophically wrong.

YetAnotherNick• 2 years ago

False information problem is terrible as well on the internet. And if I were to guess based on usage, GPT generates an order of magnitude less error than a random reddit or stackoverflow or quora answer.

drpossum2• 2 years ago

They're entitled to their business model. They're not entitled to it being successful.

mezyt• 2 years ago

I recently need to find some details of hardware implementations for a feature. I used Google to find specs for this hardware. The specs were hard to read or didn't explain things as I needed. So I asked ChatGPT and it gave me what I needed. It first regurgitated info from the specs but I was able to ask it to explain pieces in more detail and it was great! At what point do I just stop using Google search to find an answer?

But was it accurate ? Chat GPT just provided you with the statistically most likely specs. If the actual doc for this hardware is lacking, this is 100% hallucinations.

My experience is that ChatGPT is simply bullshitting you (sometime accurately), Google is drowning the info in 20 links (where they try to make you buy "Hardware you're searching for"), and you have to go to DuckDuckGo to find the thing you're actually searching for.

telesilla2• 2 years ago

Honestly I see a ton more ads on YouTube than search as I use Kagi but I can't avoid YouTube. I'm sure they have other future revenue streams for ad placement.

seankurtz• 2 years ago

Check out FreeTube

Nutcase5753• 2 years ago

I wonder how much of that pattern (Google won't give me shit, better ask GPT) is Google's own fault. Their search engine has been degraded so much that, if their LLM results are based on its search, it won't match the specificity of GPT's results. They seem to have pushed people into ChatGPT's arms. If they had consistent search services, I think less people would flock to GPT, which would give them more room to develop their own LLMs properly instead of just rushing crap like Bard or Gemini's inconsistent stuff. The gamble to rush things means you run the risk of shooting yourself in the foot and letting the other runners catch up

squigglydonut• 2 years ago

I agree that trust is eroding with a certain user demographic. As time moves on, they are competing to win trust with the next generations. Does anyone remember going to a library? Used to be the place to get info. Same thing.

legel• 2 years ago

I’ve been using the AI Overviews in Google Search for months now and overall find more utility in them.

I’ve developed an instinct for when to ignore them. Typically it is when “edge information” is involved: facts / data that are only very sparsely published in time / web space, perhaps by only one or a few web authors, perhaps only very recently.

As an AI developer for over a decade, I feel like if we’re calling this “slop”, I fear what these authors think of human error. I’m also doubtful that these people who are damning this project have ever in fact innovated or developed anything significant from scratch themselves.

Makes me want to start a company called Slop.

throwup238• 2 years ago

Makes me want to start a company called Slop.

There is precedent. You can start a modern lifestyle brand like Goop.

rickdicker• 2 years ago

I don't think you ever have to have innovated or developed anything significant to take issue with a major consumer product losing some utility - I don't know if you used Twitter ten years ago, but it was pretty sad to see the site slowly downgrade year by year, every new feature making it less functional and more addictive, fueling toxicity, becoming a black hole of individuals' attention. It makes you wonder what would have happened if the people put up more of a fight whenever some new stupid feature was rolled out, or a useful feature taken away to make the whole thing just a little more like a slot machine. (Hmmm... maybe Elon Musk should buy Google, so people will finally start complaining en Masse.)

As a defender of the new LLM-thingies, do you think they're doing a reasonable job of promoting AI-output literacy? I think it's their job to do so when they are the ones generating the content, whereas general media-literacy was not really their problem when Google was just a directory for the web.

lubujackson• 2 years ago

From my perspective, the issue isn't with AI but with UI - Google is promoting these as "answers", pushing them to the top of their results and giving no indication that the results might ever be inaccurate.

Google has been pushing to "answer questions" over return web sources since at least 2010 (when their public messaging changed), so that is nothing new. But it is a seismic, human difference between aggregated results from trusted web sources and reformatted comments from Reddit posts.

AI is great for targetted tasks and even for general web usage, with caveats. I would love to see something like an accuracy/trust score associated with the results or at a minimum a beta flag with a "hide results like this" option.

These are all basic UI procedures any reasonable person would make, but Google has been riding high on hubris for a while now (zero human support, just see the Google Cloud issue on the frontpage...) and this won't change until the stock starts getting slammed, which seems imminent.

dialup_sounds• 2 years ago

If a human offered to confidently answer every question you asked without regard to whether it is true or correct, you wouldn't say they're merely mistaken when they tell you falsehoods. You would call them a bullshitter or a charlatan.

Calling it slop at least acknowledges that the AI isn't trying to lie to you. It's not merely making a error, either. It just doesn't care either way.

Retr0id4• 2 years ago

I was born in the same year that Google Search launched. Half-decent search has been here my whole life. When someone asks me "how did you learn X", the answer usually starts with "I googled it".

I've never looked something up in an encyclopedia. I've never visited a library in pursuit of a research goal. There's no "going back to how things were before" for me - there was no "before"!

Life will go on, but the demise of search engines is quite a terrifying prospect for me. Generative AI is a search engine killer but not in any positive sense.

lionkor• 2 years ago

I'm convinced people like you and me will keep using search engines, and building them if all good ones shut down.

animaomnium• 2 years ago

There's a big number of things you can search for, but it's a finite number. There are certainly people interested in curating resources for their specific niche in exchange for curated resources in other niches, across the entire internet. Perhaps through a social web of trust. We need a new search engine resistant to sybil attacks. One that takes the social aspect of searching—connecting you with the relevant experts—into account.

k__• 2 years ago

Most people have a singular image of the world.

Google, WhatsApp, Facebook, Tinder.

As if those services are the only options. No they are not.

The world isn't one dimensional nor binary. Everything is a multidimensional spectrum.

I don't use Google search and I don't have to. There are reasonable alternatives like DuckDuckGo and Brave Search. They aren't as good as Google was in the past, but there is also Wikipedia, Reddit, GitHub, Phind, StackExchange, Discord, IRC, forums, newsgroups, Claude, ChatGPT, LLama, and what not.

Sure, it would be awesome to get all the answers from a single input field, but honestly, even though Google was better in the past, this has never been the case for any sufficiently complex question.

Retr0id4• 2 years ago

Google has never been the only tool in my toolbelt, but it's always been the biggest and sharpest. Less so these days.

II2II• 2 years ago

There's no "going back to how things were before" for me - there was no "before"! (...) Life will go on, but the demise of search engines is quite a terrifying prospect for me.

Speaking as someone who was born long before the public Internet: you can never go back to how things were before. Similarly, you can never really stay with the way things are now. The world changes. Even if you stick to the ways of your lifetime, or go back to the ways prior to your lifetime, the ways have change simply because the context has changed.

I'm not saying that we should jump the AI bandwagon. I'm just saying that we need to recognize the world is in constant flux.

Retr0id4• 2 years ago

Time marches onwards and people adapt, but that doesn't mean things are getting better.

card_zero3• 2 years ago

Well, how are we going to search the fluxing internet?

bigallen• 2 years ago

This is so interesting. I have learned more from a years’s use of chatGPT than from two decades of googling. Yeah, AI has drawbacks. But AI has never displayed an ad to me and only once displayed a cookie banner, and that’s worth quite a bit to me

jprete2• 2 years ago

Aside from the apples-to-oranges claim that a chatbots teaches you more than a search engine, it's extremely unlikely that AI will remain ad-free. Ads seem to be at a local minimum of the information economy and I don't see how chatbots would be immune to it.

fire_lake• 2 years ago

I have learned more from a years’s use of chatGPT than from two decades of googling

I simply don’t believe you. The human mind can only take in information so fast.

noman-land• 2 years ago

I have gotten great use from various AI tools but don't be naive. The "ads" are invisibly embedded in the training data and get expressed in the subtleties of the model's output. When you control the model and there's no transparency around what's in it, you can do whatever you want with the training data. This is only the beginning.

cess11• 2 years ago

I'd like to recommend that you start using libraries. You'll find that they often contain vastly more information than you'll find through web search, and you're already paying for them.

card_zero3• 2 years ago

I've never looked something up in an encyclopedia

Does that include Wikipedia? I mean, the problem with encyclopedias is that they were small (despite physically taking up entire bookcases). I didn't use them much either, even back in prehistoric times when the internet wasn't invented. They tended to disappoint by giving a crappy, superficial overview of something that was only vaguely similar to the thing you wanted to know about.

But I use wikipedia instead of doing an internet search a lot of the time. Or sometimes archive.org, for things probably in books. Cut out the middleman, go directly to where you know the information is.

cfn3• 2 years ago

For me, having an encyclopedia at home was a huge help with my studies and even finding out what I wanted to do/be later in life. This was 40+ years ago and I also had or had access to a lot of other books but never found them disappointing at all.

Retr0id4• 2 years ago

I meant physical encyclopedias, I use Wikipedia all the time. Although I reach ~half of those wikipedia articles via a google search query.

uptownJimmy• 2 years ago

Google's market share was suffering because they have been using their dominance to force everyone to participate in the "game the search engine" scam that Google was selling, which exists only to cheat the Google search engine.

Once enough of us realized that Google search results are usually just paid ads, what's the point anymore?

It's insane, and "AI" isn't going to fix any of it.

The rot in our tech economy is astonishing.

vouaobrasil2• 2 years ago

The trust Google has built with users over the last 25 years is the most valuable asset the company owns.

That gave me a laugh. I don't trust Google. Their entire purpose for many years now has been simply to enable SEO trash in such a way that users see as many ads as possible while providing a minimum amount of useful information so that users are not totally frustrated.

If Google could legally sell your organs, they would. They are that nefarious, and deserve all the hate they get. AI is just the next step. Frankly, if Google went bankrupt today, I think civilization would benefit immensely.

acheron• 2 years ago

Seriously. Google built trust from maybe 1999 through 2008, and rapidly burnt through all that long ago.

vouaobrasil2• 2 years ago

I never trusted them, ever. Large tech company = evil.

Havoc• 2 years ago

Users have no allegiance to google and this 25 years of trust he reckons google has is ephemeral.

I've stopped using google search entirely in favour of Phind and OpenAI.

There’s no reason Google had to enable this feature now.

Given that Bing beat them to the punch I'd say they're already late

llm_trw2• 2 years ago

Phind is astonishingly good for how simple it is.

If google had any sense they'd just buy them and use them as their new front end.

meiraleal2• 2 years ago

My view of Google during these 25 years went from amazed to disgusted.

joe8756438• 2 years ago

Google is in a tough position, but the track they’re on isn’t helping. People are already publishing massive amounts of meaningless AI generated content which is making results poor. So they short-circuit the madness by introducing their own poor result content, it’s bad but at least they control it.

The big question is: can it get better fast enough to save search? I think the era of search is ending, and it doesn’t look like LLMs are a good replacement.

beloch• 2 years ago

Google thinks they're untouchable. That's why they serve up so many ads that most adblockers can't keep up. That's why they've done very little to combat bad actors who manipulate google search results. That's why they think they can toss half-baked AI out there.

AI could be something that helps prevent google bombing. AI could be something that lets them serve up one or two ads per page and get a similar number of click-throughs.

Google's lunch is spread out on a picnic blanket, just waiting for someone to come along and eat it.

asadotzler2• 2 years ago

Any competent ad blocker keeps up fine. Give uBlock Origin a try, for example, rather than the ad blockers that take cash to let shit slip through and you won't see ad blockers failing to keep up with Google or YouTube.

arnaudsm2• 2 years ago

Google search quality has been decreasing not because of a lack of AI, but because of AI itself. LLM spamming and a decade of SEO maxxing has made pagerank completely obsolete. No one has a personal site anymore, backlinks don't mean anything now.

Maybe we need a need a new paradigm for search ranking. Voting, experts, introduce organic feedback back into the loop ?

meiraleal2• 2 years ago

Google shot their own foot. Starting with discontinuing Google Reader, the biggest ecosystem of blogs at the time. Could have evolved into a moat against social media walled garden.

p3rls2• 2 years ago

Gotta imagine that Conde Nast, Hearst etc are not going to take these changes lying down and the last ten years of SEOmaxxing have given them literally all the money, power etc in the web scene. They will be able to clone any in-depth "expert" site with their firepower and then come in with backlinks, socials etc and we're back to the status quo ante.

With the March updates I thought there was a chance that Google could pull it off, but now I am leaning towards it's all doomed. Personally I wish I never got into building websites when I have to compete with ugly wordpresses from SEO companies in India taking my #1 spot for English content.

I wish I could get angry about the forbes articles in my niches but at least that's written ostensibly by a professional-- when you start getting beaten by center-aligned text from South Asia you know there was -never- a chance for quality to win.

arnaudsm2• 2 years ago

Provocative proposition: what about search engine that bans everything with ads and affiliate links?

Wikipedia won over competitors because it has an anti-business-model. Only the passionate write for it, which paradoxically ensures quality. My favorite content on the internet is always non-profit, sincere stuff. Like this comment thread :)

That's a scary prospect for creators that live off the internet, but if the ecosystem is already doomed it may be the last option.

mediumsmart• 2 years ago

I say the mistrust Google has built with me in 25 years can totally accommodate ai search. Bring it on.

transcriptase• 2 years ago

It feels like at some point their goal reversed. From “be the tool that shows the user what they’re looking for as accurately and efficiently as possible” to “be the tool that has them clicking between Google search results and maybe relevant pages showing Google ads”

Publishing AI Slop Is a Choice

Related Stories