The trust collapse: Infinite AI content is awful

https://arnon.dk/the-trust-collapse-infinite-ai-content-is-awful/

by arnon • 9 months ago

237 211 9 months ago

huijzer2• 9 months ago

Yes nice article. Interesting point.

One small one I do not agree with is "Are you burning VC cash on unsustainable unit economics?". I think it's safe to conclude by now that unsustainable businesses can be kept alive for years as long as the investors want it.

arnon8• 9 months ago

I guess that's true for some but not for all. I wouldn't say that's the most common scenario

pessimizer• 9 months ago

Nothing safer than financial scams in the West these days. Never short Herbalife.

rufus_foreman• 9 months ago

Looks like you could have made a lot of money shorting Herbalife over the last 5 years.

piker• 9 months ago

The observations in this article about the insane signal-to-noise ratio are valid.

bgwalter• 9 months ago

It is becoming unbearable. YouTube now has "AI" slop ads for Freenow (Lyft brand in the EU) with fake cars that move without the wheels turning and "AI" "actors" that look like plastic.

This of course means that Freenow is now on the personal blacklist. People should not engage with companies who advertise with "AI" slop.

symfoniq• 9 months ago

Agreed. I’m increasingly using the presence of AI slop as a primary signal about whether to avoid a company and its products.

Applejinx3• 9 months ago

Which suggests the next human dirty trick will be to put out AI slop 'supporting' a company and its products just to make 'em deny it was them making it :)

SideburnsOfDoom2• 9 months ago

Youtube is intentionally blunting this signal by making many videos look slightly slop-y, I mean "enhancing low-res content with AI upscaling"

https://www.theatlantic.com/technology/archive/2025/08/youtu...

https://www.nbcnews.com/tech/tech-news/youtube-dismisses-cre...

SideburnsOfDoom2• 9 months ago

When it comes to ads on Youtube, Freenow is at the classy end of them. i.e. it's not that they're good, but that many others are much worse.

mv4• 9 months ago

I was just looking at Instagram's promoted shopping ads, and 9 out of 10 are AI slop, pretending to be actual physical products. It is quite shocking.

HarHarVeryFunny• 9 months ago

Not just ads - there is TONS of AI-generated YouTube shorts now, and the quality is so good that it's tough to always tell whether it's real as long as it's plausible (which not all of it is).

It's annoying because the whole point of a lot of this stuff is that it's real, and one can be informed, entertained or have an emotional response to it. When you distrust everything because it's maybe fake, then the fun of the internet as a window into human nature and the rest of the world just disappears.

everdrive2• 9 months ago

Doesn't matter. We must keep building more and more technology no matter the cost. Have an idea for a business? Build it. Does your business make the lives of people worse? Doesn't matter, keep pushing. Could some new technology ruin the lives and relationships that people have? Doesn't matter, just build it. We always need more, need to do more. Every experiment is valid, every impulse must be followed. More complexity, more control, more distraction, more outrage, more engagement. Just keep building forever no matter the cost.

ricogallo• 9 months ago

It sounds like the "The City" in "Blame!"

throwmeaway307• 9 months ago

move fast and break things!

nevermind if the things are people or their lives!!

arnon8• 9 months ago

build things """"people"""" want

palmotea• 9 months ago

move fast and break things!
nevermind if the things are people or their lives!!

Breaking things is ok. If people are things then it's ok to break them, right? Got it. Gotta get back to my startup to apply that insight.

wartywhoa232• 9 months ago

Yours truly,

Larry Fink and The Money Owners.

drakythe• 9 months ago

Turns out the Torment Nexus was just democratizing Venture Capital's desire for infinite growth.

giraffe_lady• 9 months ago

The paperclip maximizer is us.

tdeck• 9 months ago

That's an uncharitable take. Those VCs care very deeply about society, that's why they're funding so much research into the Torment Singularity (and giving so many talks about it) and making sure that the Right People get the Torment Nexus first so "we" can decide how it gets used.

DougN7• 9 months ago

I misread your comment as being about Vulture Capital. I think I like that :)

lioeters• 9 months ago

Seeking VC funding now for OpenTormentNexus: Democratize the nightmare.

Eddy_Viscosity2• 9 months ago

Has any big-tech company every used "Torment Nexus" unironically as some new project name, or any startup used is their company/product name? I feel its close if it hasn't happened already. I mean the unironic use.

edaemon• 9 months ago

Nothing is true, everything is permitted.

SeanDav• 9 months ago

One of the potential upsides to this as that people just might start taking time to engage in a bit of critical thinking before reacting. Is this real? How likely is this AI nonsense? What is the source? Is this the full picture? etc.

Perhaps I am too optimistic...

cantor_S_drug2• 9 months ago

Neil deGrasse Tyson said a quote expressing a concern about the future impact of AI on information credibility.

The exact quote is: "I foresee the day where AI become so good at making a deep fake that the people who believed fake news as true will no longer think their fake news is true because they'll think their fake news was faked by AI."

area51org• 9 months ago

Quote may be cliched, but still it's valid.

"Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should."

WhyOhWhyQ• 9 months ago

The world will be a soulless hell, but Dario Amodei promises we'll live forever in it.

swader999• 9 months ago

And in those days shall men seek death, and shall not find it; and shall desire to die, and death shall flee from them.

vesche• 9 months ago

Relevant: https://geohot.github.io/blog/jekyll/update/2025/07/05/are-w...

AndrewKemendo5• 9 months ago

Oooh I recognize this, he’s entering his midlife crisis. Smart guy I wish him well, and hope he comes out the other side groovy.

avhception3• 9 months ago

You better maximize engagement or you will lose engagement this is a red queen’s race we can’t afford to lose! Burn all the social capital, burn all your values, FEED IT ALL TO MOLOCH!

Wow. A new profile text for my Tinder account!

8474_s• 9 months ago

Release the hypnodrones

If you are not building the next paperclip optimizer the competition already does!

fellowmartian• 9 months ago

It’s quite rich coming from this guy, but I’m glad he’s seen the truth nonetheless.

m0llusk2• 9 months ago

This is ignoring the Marketing to Engineering ratio. For most recent history technology companies have had to spend at least as much on marketing as engineering in order to survive, and two to ten times as much spent on marketing as engineering is common for successful companies. Who is going to buy the thing is the most important question and without solid answers there is nothing, no matter how much technology was engineered.

Now this formula has been complicated by technological engineering taking over aspects of marketing. This may seem to be simplifying and solving problems, but in ways it actually makes everything more difficult. Traditional marketing that focused on convincing people of solutions to problems is being reduced in importance. What is becoming most critical now is convincing people they can trust providers with potential solutions, and this trust is a more slippery fish than belief in the solutions themselves. That is partly because the breakdown of trust in communication channels means discussion of solutions is likely to never be heard.

heddycrow11• 9 months ago

I want to keep reading your thoughts on this. How can I get more or can you source your influences?

ssalka• 9 months ago

Eric Weinstein refers to this as an Embedded Growth Obligation (EGO), whereby organizations and economies at large assume perpetual growth, and that things really start to unravel when that growth inevitably slows. It is pretty mindblowing how we have basically accepted growth as the default state, it is not at all a given that things always grow and get better.

mayhemducks• 9 months ago

We will achieve essentially zero-cost infinite exponential scalability! The cloud has no limits! InfiniDum enterprises will operate in billions of markets across time space and dimensions of probability!

zahlman4• 9 months ago

It is pretty mindblowing how we have basically accepted growth as the default state

It is completely to be expected, exactly because it is not new.

It's been scarcely a generation since the peak in net change of the global human population, and will likely be at least another two generations before that population reaches its maximum value. It rose faster than exponentially for a few centuries before that (https://en.wikipedia.org/wiki/World_population#/media/File:P...). And across that time, for all our modern complaints, quality of life has improved immensely.

Of all the different experiences of various cultures worldwide and across recent history, "growth" has been quite probably the most stable.

Culture matters. People's actions are informed by how they are socialized, not just by what they can observe in the moment.

ninetyninenine• 9 months ago

But the difference this time is now is that AI is heading in the direction where it can do ALL of WHAT you mentioned FOR you and everyone else.

Der_Einzige• 9 months ago

This is what you get for not reading the writings of https://en.wikipedia.org/wiki/Paul_Virilio

alexpotato• 9 months ago

Yuval Noah Harari, Sapiens fame [0], has a great quote (paraphrasing):

Interviewer: How will humans deal with the avalanche of fake information that AI could bring?

YNH: The way humans have always dealt with fake information: by building institutions we trust to provide accurate information. This is not a new phenomenon btw.

In democracies, this is often either the government (e.g. the Bureau of Labor Statistics) or newspapers (e.g. the New York Times) or even individuals (e.g. Walter Cronkite).

In other forms of government, it becomes trust networks built on familial ties e.g. "Uncle/Aunt is the source for any good info on what's happening in the company" etc

0 - https://amzn.to/4nFuG7C

pron8• 9 months ago

The problem is that too many people just don't know how to weigh different probabilities of correctness against each other. The NYT is wrong 5% of the time - I'll believe this random person I just saw on TikTok because I've never heard of them ever being wrong; I've heard many stories about doctors being wrong - I'll listen to RFK; scientific models could be wrong, so I'll bet on climate change being not real etc.

ajaisjsbbz• 9 months ago

Trust is much more nuanced than N% wrong. You have to consider circumstantial factors as well. ie who runs The NY Times, who gives them money, what was the reason they were wrong, even if they’re not wrong what information are they leaving out. The list goes on. No single metric can capture this effectively.

Moreover, the more political a topic the more likely the author is trying to influence your thoughts (but not me I promise!). I forgot who, but a historian was asked why they wouldn’t cover civil war history, and responded with something to the affect of “there’s no way to do serious work there because it’s too political right now”.

It’s also why things like calling your opponents dumb, etc is so harmful. Nobody can fully evaluate the truthfulness of your claims (due to time, intellect, etc) but if you signal “I don’t like you” they’re rightfully going to ignore you because you’re signaling you’re unlikely to be trustworthy.

Trust is hard earned and easily lost.

nsxwolf2• 9 months ago

COVID ended my trust in media. I went from healthy skepticism to assuming everything is wrong/a lie. There was no accountability for this so this will never change for me. I am like the people who lived through the Great Depression not trusting banks 60 years later and keeping their money under the mattress.

stocksinsmocks3• 9 months ago

5% wrong is an extremely charitable take on the NYT.

I once went to a school that had complementary subscriptions. The first time I sat down to read one there was an article excoriating President Bush about hurricane Katrina. The entire article was a glib expansion of an expert opinion who was just some history teacher who said that it was “worse than the battle of Antietam” for America. No expertise in climate. No expertise in disaster response. No discussion of facts. “Area man says Bush sucks!” would have been just as intellectually rigorous. I put the paper back on the shelf and have never looked at one since.

Don’t get emotionally attached to content farms.

tsunamifury• 9 months ago

Once a problem demands second order thinking you immediately lose a significant portion of the population.

It’s simply reality, or else propaganda wouldn’t work so well.

mvdtnz3• 9 months ago

The problem isn't "The NYT is wrong 5% of the time". It's that institutions are systematically wrong in predictable ways that happen to benefit their point of view. It's not random. It's planned.

wartywhoa232• 9 months ago

by building institutions we trust to provide accurate information

Except those institutions have long lost all credibility themselves.

lazide• 9 months ago

This was done intentionally, over decades, to try to push ‘trust’ closer to where it can be controlled. Religion, family ties (through propaganda), etc.

Cthulhu_2• 9 months ago

For sure. He then goes on to mention "in democracies", but a lot of democracies are now failing, in part because the institutions like the free press are being directed by their billionaire owners, or suppressed. And the family ties are being impacted heavily by mass misinformation and propaganda campaigns online, where their publishers are actively pushing it themselves (major worldwide social networks are now state influenced and / or their top brass has subjugated themselves to the reigning parties and / or any countermeasures have been removed).

saulpw4• 9 months ago

Yes, the existing ones were captured, and so we have to build new ones.

AtlasBarfed• 9 months ago

The New York Times.

Wall Street, financier centric and biased in general. Very pro oligarchy.

The worst was their cheerleading for the Iraq war, and swallowing obvious misinformation from Colin Powell at face value.

myth_drannon• 9 months ago

Unfortunately that's not what happens. BBC, Al-Jazeera, RT, CBC are all propaganda sources and are not sources of information. The other family members will get the information from those sources so family will not be trusted as well. And the sources I consider as trustfull, my opinion of them most likely skewed by my bias and others will consider it propaganda as well.

ravetcofx• 9 months ago

CBC and BBC aren't perfect, but I trust them leagues over any billionaire owned media like anything from Post Media the Murdochs, or Bezos. Really any for profit news isn't to be trusted.

NoMoreNicksLeft• 9 months ago

Our familial ties have been corrupted, supposing they were ever anything a sane person should've relied upon. And if humans can build institutions they trust, what happens when AI can build fake, simulated institutions that hit all the right buttons for humans to trust just as if they were of the human-created variety? Do those AIs lock in those pseudo-institution followers forever? Walter Crondeepfake can't not be trusted, just listen to his gravitas!

Sharlin2• 9 months ago

How very inconvenient it is, then, that at the same time intentional efforts to spread uncertainty and to erode trust in traditional institutions are at an all-time high! Must be a coincidence.

Cthulhu_2• 9 months ago

It's a feedback loop; you need things like freedom of speech and press to get a functional and free democracy, but you need a functional and free democracy to have freedom of speech / press. Infringe on one and you take down the other. But you need to strip down the legal branch of a free democracy first, because the democracy and freedom of speech/press is protected by a constitution in most cases.

pjc50• 9 months ago

The thing is, building such institutions and maintaining trust is expensive. Exploiting trust is lucrative (fraud, etc.) It's also expensive to not trust - all sorts of opportunities don't happen in that scenario if, say, you can't get a friend or relative in the right place.

There are many equilibrium points possible as a result. Some have more trust than others. The "west" has benefited hugely from being a high trust society. The sort of place where, in the Prisoner's Dilemma matrix, both parties can get the "cooperate" payoff. It's just that right now that is changing as people exploit that trust to win by playing "defect", over and over again without consequence.

https://en.wikipedia.org/wiki/High-trust_and_low-trust_socie...

devsda• 9 months ago

Trusting institutions is fine but you have to trust people or institutions for the right things, blind trust is harmful.

I'll trust my doctor to give me sound medical advice and my lawyer for better insights into law. I won't trust my doctor's inputs on the matters of law or at least be skeptical and verify thoroughly if they are interested in giving that advice.

Newspapers are a special case. They like to act as the authoritative source on all matters under the sun but they aren't. Their advice is only as good as their sources they choose and those sources tend to vary wildly for many reasons ranging from incompetence all the way to malice on both the sides.

I trust BBC to be accurate on reporting news related to UK, and NYT on news about US. I wouldn't place much trust on BBC's opinion about matters related to the US or happenings in Africa or any other international subjects.

Transferring or extending trust earned in one area to another unrelated area is a dangerous but common mistake.

johnnienaked• 9 months ago

That's not how they did it though. Trusted institutions are only really needed in a trustless society and reliance on them as a source of truth is a really new trend. Society used to be trustful.

huijzer2• 9 months ago

YNH: The way humans have always dealt with fake information: by building institutions we trust to provide accurate information. This is not a new phenomenon btw.

Funny that he doesn’t say that the institutions have to provide accurate information, but just that we have to trust them to provide accurate information.

profstasiak• 9 months ago

such an uninformed take, Yuval Noah Harari must be the most overrated thinker on earth (I have all of his books on my shelf)

Chinjut2• 9 months ago

AI-esque blog post about how infinite AI content is awful, from "a co-founder at Paid, which is the first and only monetization and billing system for AI Agents".

arnon8• 9 months ago

So I can't have opinions?

Also, this is entirely hand-written ;)

dkdcio• 9 months ago

this is a funny phenomenon that I keep seeing. I think people are going through the reactionary “YoU mUsT hAvE wRiTtEn ThIs oN a CuRsEd TyPwRiTeR instead of handwriting your letter!1!!”

hopefully soon we move onto judging content by its quality, not whether AI was used. banning digital advertisement would also help align incentives against mass-producing slop (which has been happening long before ChatGPT released)

ambicapter• 9 months ago

The authentic nature of opinions means that sometimes they suck. Maybe GP is commenting that your opinion sucks?

gwd• 9 months ago

I think this basically proves your point. There were things about it that made me think it may have been at least "AI-assisted", until I saw your "guaranteed bot-free" thing at the bottom. Anyone doing entirely hand-written things from now on are going to be facing a headwind of skepticism.

titanomachy• 9 months ago

This didn’t seem AI-generated to me, although it follows the LinkedIn pattern of “single punchy sentence per paragraph”. LinkedIn people wrote like this long before LLMs.

I do love the irony of someone building a tool for AI sales bots complaining that their inbox is full of AI sales slop. But I actually agree with the article’s main idea, and I think if they followed it to its logical conclusion they might decide to do something else with their time. Seems like a great time to do something that doesn’t require me to ever buy or sell SaaS products, honestly.

arnon8• 9 months ago

You're right - it isn't

This is just how I write in the last few years

Chinjut2• 9 months ago

I'm not saying it's actually written with AI (and indeed, I don't think that's the case; hence my calling it "AI-esque" rather than actually AI generated). It's just that it's a particular style of businessy blog writing that, though originated by humans, AI is now often used to crank out. Lots of bullet points, sudden emphases, etc.

It's just funny, even by hand, to be writing in the infinite AI content style while lamenting the awfulness of infinite AI content while co-founding a monetization and billing system for AI agents.

tetris112• 9 months ago

I needed to get some builder quotes for my home. It did not enter my mind to go online to search for any.

I just reached out to my family for any trustworthy builders they've had, and struck up conversations with some of my fancier neighbors for any recommendations.

(I came to the conclusion that all builders are cowboys, and I might as well just try doing some of this myself via youtube videos)

Using the internet to buy products is not a problem for me, I know roughly the quality of what I expect to get and can return anything not up to standard. Using the internet to buy services though? Not a chance. How can you refund a service

miloignis2• 9 months ago

When we needed some work done, we asked family and friends too, and ended up with a cowboy. When the work needed to be re-done, we looked up local reviews for contractors, and ended up with someone who was more expensive but also much more competent, and the work was done to a higher standard.

bluGill5• 9 months ago

ended up with someone who was more expensive but also much more competent, and the work was done to a higher standard.

How do you know that? Or is it just that your bias is coybows are bad and so you assume someone who dresses and acts better is better?

Now step back, I'm not asking you personally, but the general person. It is possible that you have the knowledge and skills to do the job and so you know how to inspect it to ensure it was done right. However the average person doesn't have those skills and so won't know the well dressed person who does a bad job that looks good from the poorly dressed person who does a good job but doesn't look as good.

arnon8• 9 months ago

I know roughly the quality of what I expect to get

because you know the brands and trust them, to a degree

you have prior experience with them

stonogo• 9 months ago

     all builders are cowboys

What does this mean?

franktankbank2• 9 months ago

They don't really do rework they'll rather keep plowing on assuming some slack in an ethereal safety margin.

tetris112• 9 months ago

Every house is different and so every job is custom. Whatever standards you think the builder is enacting to get the job done to an agreeable price is likely an ad-hoc solution that you yourself could have done as an amateur if you had the tools.

For every standard to be met, you compromise either on cash or time.

mvdtnz3• 9 months ago

Maybe this is regional. "Cowboy" in reference to a tradie means someone who is untrustworthy and out to rip you off or cheap out on work. They'll do the work but often to a low standard or with major defects they try to hide.

bee_rider3• 9 months ago

What I get from the article is that, proving that a company will stick around for a while after you’ve subscribed is hard now, because anybody can AI generate the general vibe of the marketing department of a big established player. This seems like it’ll be devastating for companies whose business model requires signing new users up for ongoing subscriptions.

Maybe it could lead to a resurgence of the business model where you buy a program and don’t have to get married to the company that supports it, though?

I’d love it if the business model of “buy our buggy product now, we’ll maybe patch it later” died.

arnon8• 9 months ago

that's exactly my point - yes

you need to prove beyond a doubt that YOU are the right one to buy from, because it's so easy for 3 Stanford dropouts in a trenchcoat to make a seemingly successful business in just a few days of vibecoding.

gnarlouse3• 9 months ago

3 stanford dropouts in a trenchcoat

I'm using this

gnarlouse3• 9 months ago

We already got your money, what do we need to work for again?

bee_rider3• 9 months ago

Yeah.

The modern software market actually seems like a total inversion of normal human bartering and trade relationships, actually…

In Ye Olden Days, you go to the blacksmith, and buy some horseshoes. You expect the things to work, they are simple enough that you can do a cursory check and at least see if they are plausibly shaped, and then you put them on your horse and they either work or they don’t. Later you sell him some carrots, buy a pot: you have an ongoing relationship checkpointed by ongoing completed tasks. There were shitty blacksmiths and scummy farmers, but at some point you get a model of how shitty the blacksmith is and adjust your expectations appropriately (and maybe try to find somebody better when you need nails).

Ongoing contracts were the domain of specialists and somewhat fraught with risk. Big trust (and associated mechanics, reputation and prestige). Now we’re negotiating an ongoing contracts for our everyday tools, it is totally bizarre.

ethmarks• 9 months ago

I think the point is that nobody will give companies money unless the product already works. No more "but in a month this'll get a really cool update that'll add all these features". If you can't trust that a company will continue to exist, you have to be confident that what you're buying is acceptable in its current state.

realitydrift• 9 months ago

What you’re describing is basically the Drift Principle. Once a system optimizes faster than it can preserve context, fidelity is the first thing to go. AI made the cost of content and the cost of looking credible basically zero, so everything converges into the same synthetic pattern.

That’s why we’re seeing so much semantic drift too. The forms of credibility survive, but the intent behind them doesn’t. The system works, but the texture that signals real humans evaporates. That’s the trust collapse. Over optimized sameness drowning out the few cues we used to rely on.

adammarples• 9 months ago

I think this "drift principle" you're pushing is just called bias or overfitting. We've overfit to engagement in social media and missed the bigger picture, we've overfit to plausible language in LLMs and missed a lot.

renegat0x02• 9 months ago

I use RSS to follow these I want to hear. I do not follow recommendation, since RSS reader is my window to the world.

I follow even AI slop via reddit RSS.

I control however what comes in.

erpigna• 9 months ago

I'm looking for a practical and possibly Open Source way to filter noise from RSS feeds, have you got any recommendation?

robin_reala• 9 months ago

Delete noisy feeds. I’m not joking: RSS only becomes a truly great experience when actively curated.

renegat0x02• 9 months ago

I have seen a lot recommends for karakeep.

I disagree with people suggesting removing noisy RSS feeds, as some are noisy, yet sometime useful. I think RSS client needs advanced filtering and search.

I use my own project for RSS https://github.com/rumca-js/Django-link-archive, but it should be considered 'alpha' as I move fast and break things. It provides me functionality that I miss in most of the clients, and it is a web client, so I can access it through mobile and PC, no need for sync. I am not a front-end dev so it does not look that much appealing.

Applejinx3• 9 months ago

I'm already seeing this. I very much fall into the category of 'delete all email offers' as I'm a small youtuber, big enough to be targeted by AI sponsor deals, so I'm just buried with it.

The last five times I've looked at something in case it was a legitimate user email it was AI promotion of someone just like in the article.

Their only way to escalate, apart from pure volume, is to take pains to intentionally emulate the signals of someone who's a legitimate user needing help or having a complaint. Logically, if you want to pursue the adversarial nature of this farther, the AIs will have to be trained to study up and mimic the dialogue trees of legitimate users needing support, only to introduce their promotion after I've done several exchanges of seemingly legitimate support work, in the guise of a friend and happy customer. All pretend, to get to the pitch. AI's already capable of this if directed adeptly enough. You could write a script for it by asking AI for a script to do exactly this social exploit.

By then I'll be locked in a room that's also a Faraday cage, poking products through a slot in the door—and mocking my captors with the em-dashes I used back when I was one of the people THEY learned em-dashes from.

One thing about it, it's a very modern sort of dystopia!

AaronAPU2• 9 months ago

YouTubers and other social media influencers are a sort of royalty now, getting to decide by fiat which companies live or die.

But you can’t really even make the case to them anymore because like you said they can’t/won’t even read your email.

What mostly happens is they constantly provide free publicity to existing big players whose products they will cover for free and/or will do sponsored videos with.

The only real chance you have to be covered as a small player is to hope your users aggregate to the scale where they make a request often enough that it gets noticed and you get the magical blessing from above.

Not sure what my point is other than it kinda sucks. But it is what it is.

Applejinx3• 9 months ago

Or know people personally. There's two people I promote because they actively helped me do my own work, pitched in open source code and did development to support my project. There's another guy who lives in my town, so he gets some mention just because of that (his work's good, but that's my angle for mentioning it). And a microphone company got a shout out not because they give me microphones, but because the guy running the company noticed and liked my ethos and did a repair for free for me. That counts as a form of sponsoring so I talked him up, am already a fan of his work.

Make friends and work with people where possible. I get that some of this only works for us open source types, but the microphone guy isn't, he just did good work. I initially heard of his company through a pro sound engineer website, and ran with it when the advice turned out to be good.

ModernMech2• 9 months ago

Trust collapse is real, I don't trust anything anymore. Take this article for instance, I don't trust it because of the random bolding. Does that mean it's AI generated? I don't know but I've seen lots of AI generated content and it has random bolding, so when I see it, I immediately don't trust it. And I don't have the time to verify anything, so whether or not this article was written by the author or AI, it's gone on the "not credible" heap for me, just because of the bolding. It's not a strong signal but it's a signal, and due to the volume of slop, I must filter on whatever signals I have to maintain any chance of finding genuine human work product. Maybe I miss something genuine and important by filtering this way, but it's the best I can do.

rho4• 9 months ago

Well said. Sad how that reflex starts kicking in for HN comments as well (ps I'm not getting any signals from your comment).

ChrisMarshallNY• 9 months ago

Will you still be here in 12 months when I’ve integrated your tool into my workflow?

This is the biggie; especially with B2B. It's really 3 months, these days. Many companies have the lifespan of a mayfly.

AI isn't the new reason for this. It's been getting worse and worse, in the last few years, as people have been selling companies; not products, but AI will accelerate the race to the bottom. One of the things that AI has afforded, is that the lowest-tier, bottom-feeding scammer, can now look every bit as polished and professional as a Fortune 50 company (often, even more).

So that means that not only is the SNR dropping, the "noise" is now a lot riskier and uglier.

tablet• 9 months ago

One of the things that AI has afforded, is that the lowest-tier, bottom-feeding scammer, can now look every bit as polished and professional as a Fortune 50 company (often, even more).

Made my day. So true.

stevetron• 9 months ago

You can't build trust in your OS (operating system) when your OS spies on the entire customer base, and you spin it off as telemetry. Or you remotely target the OS to implement a radical change, and force it to be installed as an 'update'.

I stopped accepting telephone calls before 2010. They still ring the phone.

slightwinder• 9 months ago

What if this is the plan all along? People losing trust in media, so the rich and powerful can continue doing shit without getting exposed any more, because now they always can say it's just AI, and didn't really do this or that?

heddycrow11• 9 months ago

I wish we were talking about what's next versus what's increasingly here.

How can infinite AI content be strictly awful if it forces us to fix issues with our trust and reward systems? Short term, sure. But infinite (also) implies long term.

I wish I had a really smart game theorist friend who could help me project forward into time if for nothing other than just fun.

Don't get me wrong, I'm not trying to reduce the value of "ouch, it hurts right now" stories and responses.

But damned if we don't have an interesting and engaging problem on our hands right now. There's got to be some people out there who love digging in to complicated problems.

What's next after trust collapses? All of us just give up? What if that collapse is sooner than we thought; can we think about the fun problem now?

arnon8• 9 months ago

I suggest we go back to before and be human about things - and build trust in-person.

trylist• 9 months ago

This is childish thinking. Whatever we do, we cannot go back to "before". Which "before"? How do we go back?

You can't regress back to a being a kid just because the problems you face as an adult are too much to handle.

However this is resolved, it will not be anything like "before". Accept that fact up front.

heddycrow11• 9 months ago

Dunbar's number leaps to mind. I wonder what our systems look like at large when we have cause to strengthen our 150 meaningful connections.

Would this truly be a move back? I've met people outside my social class and disposition who seem to rely quite heavily on networking this way.

AndrewKemendo5• 9 months ago

Unfortunately there’s no “roll back to last stable” - the current version is actually still the most stable

If you try to “go back” you’ll just end up recreating the same structure but with different people in charge

Meet the New boss same as the old boss - biological humans cannot escape this state because it’s a limit of the species

kyoob• 9 months ago

From a game-theory perspective, if players rush the field with AI-generated content because it's where all the advantages are this year, then there's going to be room on the margins for trust-signaling players to advance themselves with more obviously handspun stuff. Basically, a firm handshake and an office right down the street. Lunches and golf.

The real question to ask in this gold rush might be what kind of shovels we can sell to this corner of hand shakers and lunchers. A human-verifiable reputation market? Like Yelp but for "these are real people and I was able to talk to an actual human." Or diners and golf carts, if you're not into abstractions.

tompccs• 9 months ago

Long airlines and corporate credit cards

heddycrow11• 9 months ago

That gets my brain moving, thanks. What do you think those who are poor/rich in a trust economy look like? How much of a transformation to trust economy do you think we make?

oblio2• 9 months ago

How can infinite AI content be strictly awful if it forces us to fix issues with our trust and reward systems?

You're assuming they can be fixed.

But damned if we don't have an interesting and engaging problem on our hands right now. There's got to be some people out there who love digging in to complicated problems.

I'm sure the peasants during Holomodor also thought: "wow, what an interesting problem to solve".

heddycrow11• 9 months ago

I don't have the time to read all four stories that ChatGPT turned up right this minute, but I now have cause to believe that at least some minority of those peasants you refer to did find fun in solving their problems.

I'm with that group of people. What was your point in bringing this up?

Wait, was I just trolled? If so, lol. Got me!

cowboylowrez2• 9 months ago

how can crime be bad if it forces us to police crime?

Sharlin2• 9 months ago

In other words, many new people now get to know the "using an internet dating service as a woman" experience.

heddycrow11• 9 months ago

If that's what people wanted, they could have created a woman account or a western white male account on a rest-of-the-world dating site.

If it's not? Oh well, suffer. It's still better than the "average western male on a dating site" experience.

Note: I really like the metaphor. My apologies if I abused it or stretched it to far or in the wrong direction.

jjoe• 9 months ago

Do you think usury is to blame for this aggressive and excessive push to make more money? Synonym for excessive here could be unethical

JCM9• 9 months ago

We see business go through this cycle a lot. Some new “better cheaper” thing comes along. Everyone implements it to keep up with the Jones’s. Suddenly there’s no differentiation because everyone has it and everyone thinks it sucks. Suddenly going back to some reworked version of the old thing is the new black.

One such example was call centers. In the 2000s implementing a call center in India was all the rage on cost cutting. The customer experience was terrible and suddenly having a US-based call center (the thing companies just abandoned) was now a feature.

I think we’ll see similar things with AI. Everyone will get flooded with AI slop. Folks will get annoyed and suddenly interacting with a real human or a real human writing original content will be a “feature” that folks flock to.

bluGill5• 9 months ago

Problem is how do you find real humans in the first place if you don't know them. Easy enough to walk/drive around my city and talk to them. However there are a lot of topics that the experts live elsewhere and so that won't work.

cantor_S_drug2• 9 months ago

We need PageRank like algorithm for "Trust / Human Content" to be applied directly to the source of such content. E.g. following all three channels are AI made. But all these content can be liked to an advanced AI version of audio based videos of Wikiarticles. If a video is providing just a summary based on established historical facts, even though it is AI based, how is it different than refering a thesaurus or dictionary? Aren't such videos making "knowledge" accessible.

FINAL Financial hours of U.S.A. just before the 1929 crash

https://www.youtube.com/watch?v=dxiSOlvKUlA&t=1008s

The Volcker Shock: When the Fed Broke the Economy to Save the Dollar (1980)

https://www.youtube.com/watch?v=cTvgL2XtHsw

How Inflation Makes the Rich Richer

https://www.youtube.com/watch?v=WDnlYQsbQ_c

_the_inflator• 9 months ago

It didn't help documentation at all. I had to work with auth0 for example and their documentation is such a bloat, that I am already prototyping with better-auth.

No structure, outdated stuff marked as "preview" from 2023/2024, wikipedia like in depth articles about everything but not for simple questions like: how to implement a backend for frontend.

You find fragments and pieces of information here and there - but no guidance at all. Settings hidden behind tabs etc.

A nightmare.

No sane developer would have done such a mess, because of time constraints and bloat. You see and experience first hand, that the few gems are from the trenches, with spelling mistakes etc.

Bloat for SEO, the mess for devs.

returnInfinity• 9 months ago

Yes sick of AI generated "illustrations".

guzik• 9 months ago

So I went on X after a long break from social media, and my feed is full of tips like this one:

Growing on X is so simple I’m shocked it works.

100x comments a day

10x posts a day

15x DM’s a day

1x thread a day

1x email a day

This is how you grow your presence on X.

Even if having a presence matters, how can you actually say something meaningful if you post 10 times a day - there's no way (unless you just repeat yourself). Hopefully my algorithm's just gone weird but sadly the people I used to follow stopped posting.

Lerc• 9 months ago

I don't think this shows that you can't trust things. I think it means trust should be earned.

We might be transitioning to a world where trust has value and is earned and stored in your reputation. Clickbait is a symptom of people valuing attention over trust. Clickbait spends a percentage of their reputation by trading it for attention.

In a world of many providers, most people have not heard of any particular individual provider. This means they have no reputation to lose, so their choice to act in a reputation losing manner is easy.

Beyond a certain scale when everyone can play that game we end up with the problem that this article describes. The content is easy but vacuous. There are far more people vying for the same number of eyballs now.

The solution is, I believe, earned trust. Curators select items from sources they trust. The ones that do a good job become trusted curators. In a sense HackerNews is a trusted curator. Reddit is one that is losing, or has lost, trust.

AI could probably take on some of the role of that curation. In the future perhaps more so. An AI can scan the sources of an article to see if the sources make the claims that the article says it makes. I doubt it can do so with sufficient accuracy to be useful right now, but I don't think that is too far off.

Perhaps the various fediverse reddit clones had the wrong idea. Maybe they should in a distributed fashion where each point is a subreddit analogue operated each with their own ways of curation, then an upper level curation can make a site of the groups they trust.

This makes a multi level trust mechanism. At each level there are no rules governing behaviour. If you violate the values of a higher layer, they lose trust in you. AI could run its own curation nodes. It might be good at it or it might be terrible, it doesn't really matter. If it is consistently good, it earns trust.

I don't mind there being lots of stuff, if I can still find the good stuff.

avhception3• 9 months ago

This isn't limited to sales. The trust collapse is also coming for the public debate, interpersonal relationships and probably more stuff than I can imagine right now.

I predict a renaissance of meeting people in person.

tartoran2• 9 months ago

I predict a renaissance of meeting people in person.

I hope that will come to fruition.

avhception3• 9 months ago

Let's call it the glass-half-full scenario :)

TedDallas• 9 months ago

Three words to solve this problem: direct mail marketing.

Just kidding, that just goes into my RL trash can.

wazoox• 9 months ago

As Hannah Arendt said : "The result of a consistent and total substitution of lies for factual truth is not that the lie will now be accepted as truth, and truth be defamed as lie, but that the sense by which we take our bearings in the real world - and the category of truth versus falsehood is among the mental means to this end - is being destroyed."

visarga• 9 months ago

I think the claim is wrong, depends on many things. In most cases AI content is better than web content, especially if you use Deep Research to ground it in multiple sources. It is a problem when content is generated to capture traffic (SEO, content feeds), but in those venues we were already neck deep in slop before 2022. If anything AI slop might improve the quality of old SEO and feed bait.

When I need to be sure about something, I check it manually in trusted sources. If I am not sure, I check all the sources I can. More recently I run deep research on 3 agents (Claude, ChatGPT and Gemini) and then compare between the reports.

turnsout• 9 months ago

We're finally getting the "pure" capitalism that economists dream about. An infinite number of sellers with perfectly commoditized products, driving margins asymptotically to zero.

If you want to get ahead, you'll need to find the 1% edge and exploit it for 15 minutes until a competitor erases your lead.

projektfu• 9 months ago

My screening inbox is full of the same exact form of engagement, almost identical to the one mentioned in the article. "I'm curious..." and then some interval later a follow up and then a "I don't seem to be reaching you" e-mail and by that point I have noticed and blocked them. It is fine for me, I have a system to handle it, but my receptionists often forward me these things from their inbox which bypasses my controls.

It's not just that it is 0 effort, it also sucks, and it is increasingly not relevant because their agents are just scooping up stuff to reach out about and they aren't even selling something that you would need to buy.

I just wish that we could go back to the old way. There should be a cost to attempt to get a sales lead.

hexator3• 9 months ago

The cost to create content is not zero. People across the board are underestimating the cost to run AI.

aarondf• 9 months ago

I think you're conflating the cost to consumers to use AI and the cost to run AI. Sure, it's a house of cards, but the cost to consumers still rounds to zero.

hexator3• 9 months ago

Sure, but that can't last.

hexator3• 9 months ago

From reading some of the reactions to this post, this quote comes to mind by Upton Sinclair: “It is difficult to get a man to understand something, when his salary depends on his not understanding it.”

There are a lot of people here who have spent a lot of time, money, and effort building AI products. And they may be good and worthwhile! I'm literally one of them. But you still see some people totally underestimate this public-facing trust collapse and the growing anti-AI sentiment in general.

yndoendo• 9 months ago

Anyone else see AI as nothing more than a logging system for citing real world experience and expertise?

In order for any AI/ML content to have value it must cite where the accumulation of information came from. By not doing so it nothing more than a custom Wikipedia-esque with a motto _Trust Me Bro_.

Citations and lack there of should be a simple key factor in evaluating trust. What are your sources for this idea / answer?

BeFlatXIII• 9 months ago

Ironic coming from a blog about B2B sales. Who was answering cold calls and reading LinkedIn profiles before AI?

eddythompson80• 9 months ago

I'm honestly all for it. As AI keeps poisoning all aspects of online content and online interactions, people who care about that sort of thing will have to move back to in person interaction more.

The person you're physically interacting with might be using AI in their workflow, that's fine. I use AI too. I just don't want to "build a relationship" with AI. I don't care for AI "content". Art, blogs, articles, advertisements, even detestable things like sales and marketing are all forms of human relationships to me. It's fine if you wanna autogenerate it. I'm even "in the market" for autogenerates stuff as I use AI bots too, but you can't try to sell me a 100% automated-burger when I have the Fabricator 3000 too.

If I'm hungry and just want a burger, I'll get my Fabricator 3000 to generate one for me. If I'm in the mood for a human touch on food and a dining experience, I'll cook, go to a (reputable) restaurant or a friend's place who likes to cook. Maybe there is a market for you to run your Fabricator 3000 to generate a burger for me. maybe. I don't know why I'd buy it though when I can just get your prompt and feed it into my own Fabricator 3000...

hbn• 9 months ago

AI slop has tarnished the one remaining pure thing I do on the internet which is watch videos of cute animals. I've spotted a good amount in my Instagram feed lately which I only happened to notice it was fake at the last second. I'm sure some of them I've watched were indeed AI. It feels betraying to get tricked like that.

scotty79• 9 months ago

Which means all of it is now suspect.

That's actually wonderful result. Humans and their messages are not to be trusted. It's a bit late that we had to make AI to show us that.

obscurette• 9 months ago

The problem isn't that I don't trust AI. The problem is that I don't trust anything or anyone any more.

The trust collapse: Infinite AI content is awful

Related Stories