95% of Companies See 'Zero Return' on $30B Generative AI Spend

https://thedailyadda.com/95-of-companies-see-zero-return-on-30-billion-generative-ai-spend-mit-report-finds/

by speckx • 11 months ago

418 412 11 months ago

AI is a deflationary technology, sort of like how most of the cost of TVs is the cost of shipping them from where they are made to where they are used. So wouldn't the returns show up in 'less work needs to be done' slowly over time?

lazide• 11 months ago

Except generative AI is mostly an arms race, with content being generated being processed by some other AI on the other side. And any humans unfortunate enough to be in the middle regretting being born.

seydor• 11 months ago

I'm confused, which point of the hype cycle is this?

jameskilton• 11 months ago

The Trough of Disillusion. Though we definitely aren't down there yet, it's coming.

m_fayer• 11 months ago

You just wait, when we achieve agi it’ll drive hype cycles at a speed humans have no chance of keeping up with. We will always effectively be in all points of the hype cycle at once. Money will wash randomly through the economy like soap through your laundry. You ain’t seen nothing yet.

dr_dshiv• 11 months ago

Man, if people aren’t on the plateau of productivity by now with genai, it’s their own damn fault. Trough of disillusionment was like, 2020 AI projects.

spogbiper7• 11 months ago

Sounds like 95% of companies are potential clients for my consulting services

uncircle2• 11 months ago

Do you sell overpriced generative AI solutions, or do you consult them on how to pivot away from idiotic generative AI?

onlyrealcuzzo• 11 months ago

Can't tell if you're in on it, but he's implying this waste is no different than average for consulting

ehutch792• 11 months ago

YES!

nextworddev• 11 months ago

Good thing consulting is one thing AI is decent at

jryio• 11 months ago

Started an entire consulting practice to get engineering teams and founders out of vibe coded pits. Even got a great domain for it - vibebusters

So far business is booming and clients are happy with both human interactions with senior engineers as well as a final deliverable on best practices for using AI to write code.

Curious to compare notes

AstroBen• 11 months ago

The link to their referenced study doesn't seem to work?

This is confusing.. it's directly saying AI is improving employee productivity, but that's not leading to more business profit... how does that happen?

macintux• 11 months ago

Possible conclusion: most of the work that employees do has no direct impact on earnings.

AznHisoka• 11 months ago

Maybe they’re measuring productivity by flawed metrics. One could write 10x as much code, but that doesnt mean that will equate to more profit

addaon• 11 months ago

AI is improving employee productivity, but that's not leading to more business profit... how does that happen?

One trivial way is that the increase of productivity is less than the added cost of the tools. Which suggests that (either due to their own pricing, or just mis-judgement) the AI companies are mis-pricing their tools. If the tool adds $5000 in productivity, it should be priced at $4999, eventually -- the AI companies have every motivation to capture nearly all of the value, but they need to leave something, even if just a penny, for the purchasing company to motivate adoption. If they're pricing at $5001, there's no motivation to use the tool at all; but of course at $4998 they're leaving money on the table. There's no stable equilibrium here where the purchasing companies end up with a /significant/ increase in (productivity - cost of that productivity), of course.

bilbo0s2• 11 months ago

the AI companies are mis-pricing their tools

Sounds like the AI companies are not so much mispricing, as the companies using the tools are simply paying wayyy too much for the privilege.

As long as the companies keep paying, the AI companies are gonna keep the usage charges as high as possible. (Or at least, at a level as profitable to themselves as possible.) It's unreasonable to expect AI companies to unilaterally lower their prices.

a-posteriori• 11 months ago

This study was led by a PI in the MIT Media Lab who studies cameras and their impact on society.

There's a reason it has a catchy headline, there's a reason you needed to fill in an email form to get access to the study, and there's a reason why the 2nd author has an agentic AI startup.

I thought it was a low quality article with no data or in detail methods. MIT needs to do better. This is the second article of this type in the last few months that caught headlines.

philipodonnell• 11 months ago

Anyone have a link to the actual report?

theonionspeaks• 11 months ago

https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Bus...

cstejerean• 11 months ago

Why do I have a feeling that most of that $30B was spent on paying for consultants, most of which were also essentially making things up as they went along.

bilbo0s2• 11 months ago

I don't know?

For some reason, I'm thinking most of the money went to either inferencing costs or NVidia.

wood_spirit• 11 months ago

Are there really that many consultants floating around still? I remember the heyday of the 2000s and kind of thought the outside consultants were largely disappearing and nowadays companies have trained themselves to jump on the hype train without paying consultants to push them?

varispeed2• 11 months ago

The funniest part isn’t that AI hasn’t delivered profits. It’s that the only “value” most people got from LLMs was accidentally rediscovering what Google used to be before it turned into an ad-riddled casino.

Executives mistook that novelty for a business revolution. After years of degraded search, SEO spam, and “zero-click” answers, suddenly ChatGPT spat out a coherent paragraph and everyone thought: my god, the future is here. No - you just got a glimpse of 2009 Google with autocomplete.

So billions were lit on fire chasing “the sliced bread moment” of finally finding information again - except this time it’s wrapped in stochastic parroting, hallucinations, and a SaaS subscription. The real irony is that most of these AI pilots aren’t “failing to deliver ROI” - they’re faithfully mirroring the mediocrity of the organisations deploying them. Brittle workflows meet brittle models, and everyone acts surprised.

The pitch was always upside-down. These things don’t think, don’t learn, don’t adapt. They remix. At best they’re productivity duct tape for bored middle managers. At worst they’re a trillion-dollar hallucination engine being sold as “strategy.”

The MIT study basically confirms what was obvious: if you expect parrots to run your company, you get birdshite for returns.

xpe• 11 months ago

Grain of truth acknowledged, though much exaggerated -- to the point of forgetting history.

No - you just got a glimpse of 2009 Google with autocomplete.

Go look at the kinds of "answers" you got in 2009 with Google's autocomplete.

Time to drop this convenient nicely-packaged narrative.

These things don’t think, don’t learn, don’t adapt.

Perhaps some don't -- for some intentionally constructed definitions of these terms. Adding a feedback loop (chain of thought, etc) challenges even such definitions though.

if you expect parrots to run your company, you get birdshite for returns.

If this is a reference to "stochastic parrots" metaphor, then I would encourage you to read these articles and HN comments below, which push-back against misunderstandings or overuse of that metaphor:

[1] https://www.lesswrong.com/posts/HxRjHq3QG8vcYy4yy/the-stocha...

[2] https://www.lesswrong.com/posts/7aHCZbofofA5JeKgb/memetic-ju...

[3] https://news.ycombinator.com/item?id=44299996

[4] https://news.ycombinator.com/item?id=44926540

[5] https://news.ycombinator.com/item?id=44967655

[6] https://news.ycombinator.com/item?id=43676755

[7] https://news.ycombinator.com/item?id=43585572

[8] https://news.ycombinator.com/item?id=42351348

kmarc• 11 months ago

While you are being downvoted here, I would like to express how much I liked how this thought summarized some of the pain points I have to deal with: (regardless of AI):

they’re faithfully mirroring the mediocrity of the organisation

This is happening with to technologies / tools / etc...

aqme28• 11 months ago

This is why it's so good to sell shovels, so-to-speak.

In this case, that's NVDA

tovej2• 11 months ago

The shovel business is good as long as the gold rush lasts. Once the gold rush is over, you're going to have to deal with a significant decrease in volume, unless you can find other customers.

Crypto's over, gaming isn't a large enough market to fill the hole, the only customers that could fill the demand would be military projects. Considering the arms race with China, and the many military applications of AI, that seems the most likely to me. That's not a pleasant thought, of course.

The alternative is a massive crash of the stock price, and considering the fact that NVIDIA makes up 8% of everyone's favorite index, that's not a very pleasant alternative either.

It seems to me that an ultra-financialized economy has trouble with controlled deceleration, once the hypetrain is on it's full-throttle until you hit a wall.

taormina2• 11 months ago

There aren’t enough GPUs for average gamers to buy anything vaguely recent and they would love to be able to. Making the best GPUs on the planet is still huge and the market is quite large. Scalping might finally die at this rate, but NVDA wasn’t making any of the scalping money anyway so who cares? Data centers and gamers still need every GPU NVDA can make.

lgats• 11 months ago

or steel and wood for making shovels, TSMC

abrinz• 11 months ago

Skill issue

mushufasa• 11 months ago

1 in 20 doesn't sound great. But you have to mediate that with

- everyone and their mother are doing a "generative ai program" right now, a lot of times just using the label to try to get their project funded, ai being an afterthought

- if the 1 out of 20 projects is game-changing, then you could argue right now people should actually be willing to spend even more on the opportunity, maybe the number should actually be 1 in 100. (The VC model is about having big success 1 in 10 times.)

- studies of ongoing business activities are inherently methodologically limited by the data available; I don't have a ton of confidence that these researchers' numbers are authoritative -- it's inherently impossible to truly report on internal R&D spend especially a private companies without inside information, and if you have the inside information you likely don't have the full picture.

JCM92• 11 months ago

We are entering the “Trough of disillusionment.” These hype cycles are very predictable. GPT-5 being panned as a disappointment after endless hype may go down as GenAI’s “jump the shark” moment.

It’s all fun and games until the bean counters start asking for evidence of return on investment. GenAI folks better buckle up. Bumps ahead. The smart folks are already quietly preparing for a shift to ride the next hype wave up while others ride this train to the trough’s bottom.

Cue a bunch of increasingly desperate puff PR trying to show this stuff returns value.

highwaylights• 11 months ago

I wouldn’t be surprised if 95% of companies knew this was a money pit but felt obligated to burn a pile of money on it so as not to hurt the stock price.

JCM92• 11 months ago

FOMO on the way up to the peak is a powerful force. Now that we’re sliding down the other side FOMO turns into “WTF did we just spend all that money on again?”

lenerdenator• 11 months ago

I also wouldn't be surprised if bean counters were expecting a return in an unreasonable amount of time.

"Hey, guys, listen, I know that this just completely torched decades of best practices in your field, but if you can't show me progress in a fiscal year, I have to turn it down." - some MBA somewhere, probably, trying and failing yet again to rub his two brain cells together for the first time since high school.

Just agentic coding is a huge change. Like a years-to-grasp change, and the very nature of the changes that need to be made keep changing.

generic920343• 11 months ago

In Germany there is the additional issue of companies only really starting to invest into the hype when the hype cycle is already at its end, in other parts of the world. And do not imagine that would lower the investments or shorten the amount of time spent on the hype. The C level can never admit errors, the middle management only sees a way to promotion by following the hype.

pgwhalen• 11 months ago

It's hard to define what it means for a company to know something, but as a person inside a company spending on gen AI efforts, I'm pretty confident that we're not investing in it just to maintain an elevated valuation (we're a mature, privately owned company).

empath752• 11 months ago

There were similar headlines in the late 80s and early 90s as IT in general was widely seen to have been a money wasting bust. Most people who try to use new technologies, especially early adopters waste a shitload of money and don't accomplish very much.

runarberg• 11 months ago

My favorite conspiracy theory at the moment is that this is a way for the rich to literally burn the excess money to prevent it from getting back to the working classes in an effort to keep the exploitation machine running.

Now, I don’t believe this is an actual conspiracy, but rather a culture of hating the poor. The rich will jump on any endeavor—no matter how ridiculous—as long as the poor stay poor, even if they loose money in the process.

frozenport• 11 months ago

Yo what’s the next hype cycle that smart folks like us should be working on?

belter4• 11 months ago

The US mix of government and enterprise. You want Capital you need to be MAGA

"Donald Trump and Silicon Valley's Billionaire Elegy" - https://www.wired.com/story/donald-trump-and-silicon-valleys...

"Secret White House spreadsheet ranks US companies based on loyalty to Trump" - https://www.telegraph.co.uk/business/2025/08/15/secret-white...

quotemstr• 11 months ago

Defense will increasingly become a national priority over the next few decades. Pax Americana is teetering.

Terr_3• 11 months ago

The next hype-cycle might not relate to software. I'm thinking of the (smaller, shorter) phase where "nanotechnology" was getting slapped onto everything including laundry detergent.

zzzeek2• 11 months ago

mRNA was set to be huge but US voters apparently didnt want it.

rpcope12• 11 months ago

Probably defense and robotics.

bitmasher9• 11 months ago

That’s the $100M question. Pick the answer correctly and you’ll get record breaking compensation levels.

no_wizard• 11 months ago

Gemini keeps being rather impressive though, even their iterative updates have improvements, though I'm seeing a significant slowdown in the improvements (both quantity and in how much they improve) suggestion a wall may be approaching.

That said, technologies like this can also go through a rollercoaster pattern itself. Lots of innovation and improvement, followed by very little improvement but lots of research, which then explodes more improvements.

I think LLMs have a better chance at following that pattern than computer vision did when that hype cycle was all the rage

belter4• 11 months ago

Prepare for the crash:

"Spending on AI data centers is so massive that it’s taken a bigger chunk of GDP growth than shopping" - https://fortune.com/2025/08/06/data-center-artificial-intell...

herval3• 11 months ago

This is such a huge repeat of the early 2000s. All the bust startups spent billions in infrastructure. Everyone built their own datacenters, no matter what your business is.

We'll either see a new class of "AWS of AI" companies that'll survive and be used by everyone (that's part of the play Anthropic & OpenAI are making, despite API generating a fraction of their current revenue), or Amazon + Google + Microsoft will remain as the undisputed leaders.

seatac763• 11 months ago

Good point. I wonder if the Windsurf folks saw the writing on the wall and cashed out when they could.

herval3• 11 months ago

I was a big Windsurf fan. What they did with their team became a massive cautionary tale around the current wave of startups. It's creating a distrust culture that's gonna be very hard to repair (not that it matters for the handful of newly-minted billionaires)

deadbabe4• 11 months ago

If GPT-5 had been released last year instead, they could have probably kept the hype manageable. But they waited too long and too greedily, and the launch fell flat. And in some cases even negative as I’m seeing some bad PR about people who got too attached to their GPT-4o lovers and hate the new GPT-5.

mattlondon• 11 months ago

Perhaps just a trough of disillusionment for OpenAI. Anthropic and Google keep delivering.

chriskanan3• 11 months ago

Sam Altman way oversold GPT-5's capabilities, in that it doesn't feel like a big leap in capability from a user's perspective; however, the a idea of a trainable dynamic router enabling them to run inference using a lot less compute (in aggregate) to me seems like a major win. Just not necessarily a win for the user (a win for the electric grid and making OpenAI's models more cost competitive).

pseudosavant• 11 months ago

When OpenAI went from GPT-3.5-Turbo to GPT-4 it seemed massive, and there were no other steps in-between. And nobody else had a meaningful competitive model out yet.

When GPT-5 came out, it wasn't going from GPT-4 to GPT-5. Since GPT-4 there has been: 4o, o1, o3, o3-mini, o4-mini, o4-mini-high, GPT-4.1, and GPT-4.5. And many other models (Llama, DeepSeek, Gemini, etc) from competitors have been released too.

We'll probably never experience a GPT-3.5 to GPT-4 jump again. If GPT-5 was the first reasoning model, it would have seemed like that kind of jump, but it wasn't the first of anything. It is trying to unify all of the kinds of models OpenAI has offered, into one model family.

neom• 11 months ago

hahaaa! I lead a growth team for a genAI company and on Monday I said to the team "we need to start to put content together that proves our customers are having return and finding value, because we're entering the trough of disillusionment"

...I'll try not to sound desperate tho.

bicepjai• 11 months ago

I would bet on Robotics in general, humanoids and other type robots

CharlesW• 11 months ago

Duplicate discussion from this week (162 comments): https://news.ycombinator.com/item?id=44941118

Here's the source report, not linked to by this content farm's AI-written article: https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Bus...

pawelduda• 11 months ago

I blame bad prompts /s

eldenring3• 11 months ago

This is how America ends up being ahead of the rest of world with every new technology breakthrough. They spend a lot of money, lose a lot of money, take risks, and then end up being too far for others to catch up.

Trying to claim victory against AI/US Companies this early is a dangerous move.

ta202405282• 11 months ago

This is how America ends up being ahead of the rest of world with every new technology breakthrough.

Too young to remember GSM?

eldenring3• 11 months ago

I don't think its controversial to say that in net, the US has had a good streak so far with new technologies. Yeah there have been some pretty big misses, but obviously the wins more than make up for them.

vdupras4• 11 months ago

[flagged]

onlypassingthru• 11 months ago

Wasn't half of Napoleon's army gone by the time he reached Moscow?[0] It's the sunk cost fallacy writ large with dead bodies, no?

[0]https://www.researchgate.net/figure/Napoleon-march-graphic-C...

dang2• 11 months ago

"Eschew flamebait. Avoid generic tangents."

"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize."

https://news.ycombinator.com/newsguidelines.html

9dev• 11 months ago

I’m not sure you can even generalise that much anymore. The US banking system is stuck in the dark ages compared to other countries, China is leading in electric cars, photovoltaics. and lots of other industries.

You could make that claim for the software industry, but I’m pretty sure a big part of the US moat is due to oligopolies, lock-in effects, or corruption in favour of billionaires and their ventures.

hiddencost• 11 months ago

Like solar power? Or electric cars? Or drones?

pepinator• 11 months ago

That's an awfully simplistic way of seeing things. It always surprises me how disconnected from reality Americans are.

stephankoelle2• 11 months ago

In some products, certain AI features have become expected. If a product doesn’t include them, it risks losing customers, making it a net negative for the market. At this point, companies either invest in AI or risk falling behind.

Scarblac• 11 months ago

Can you name some of those products?

stephankoelle2• 11 months ago

A/B Testing: Automate test setup and simplify result analysis. E-commerce: Use AI to generate synonyms for more relevant search results. Issue Tracker: Implement natural language search to find tickets more easily.

Sorry, this are only the categories. But i have actual products in mind.

phplovesong• 11 months ago

No shit. AI is not "AI", its just a meme word for getting VC cash, because the original term (ML) did not raise any capital. Play stupid games, win stupid prizes.

IgorPartola3• 11 months ago

What are the actual use cases that can generate revenue or at least save costs today? I can think of:

1. Generate content to create online influence. This is at this point probably way oversaturated and I think more sophisticated models will not make it better.

2. Replace junior developers with Claude Code or similar. Only sort of works. After all, you can only babysit one of these at a time no matter how senior you are so realistically it will make you, what, 50% more productive?

3. Replace your customer service staff. This may work in the long run but it saves money instead of making money so its impact has a hard ceiling (of spending just the cost of electricity).

4. Assistive tools. Someone to do basic analysis, double check your writing to make it better, generate secondary graphic assets. Can save a bit of money but can’t really make you a ton because you are still the limiting factor.

Aside: I have tried it for editing writing and it works pretty well but only if I have it do minimal actual writing. The more words it adds, the worse the essay. Having it point out awkward phrasing and finding missing parts of a theme is genuinely helpful.

5. AI for characters in video games, robot dogs, etc. Could be a brave new frontier for video games that don’t have such a rigid cause/effect quest based system.

6. AI girlfriends and boyfriends and other NSFW content. Probably a good money maker for a decade or so before authentic human connections swing back as a priority over anxiety over speaking to humans.

What use cases am I missing?

wedn3sday• 11 months ago

One use case I'd love to see an easy plug-and-play solution for is a RAG build around companies vast internal documentation/wikis/codebase to help developers onboard and find information faster. I would love to see less of people trying to replace humans with language models and more of people trying to use language models to make humans jobs less frustrating.

OutOfHere2• 11 months ago

In all the companies I have worked at and have looked at such docs, unfortunately this doesn't really work because those internal documentation sites are statistically never up to date or even close. They are hilariously unclearly written or out of date.

As for relying on the code base, that is good for code, although not for onboarding/deployment/operations/monitoring/troubleshooting that have manual steps.

arahman4710• 11 months ago

Check out https://asksolo.ai/

We connect with slack/notion/code/etc so that you can do the following:

1. Ask questions about how your code/product works 2. Generate release notes instantly 3. Auto update your documentation when your code changes

We primarily rely on the codebase since it is never out of date

jaimebuelta• 11 months ago

Stock images. I’ve already seen trining courses (for compliance reasons) using AI videos. A bit cringey, but I imagine cheaper than shooting real people.

wredcoll4• 11 months ago

but I imagine cheaper than shooting real people

How much does that cost these days? Do you still have to fly to remote islands?

plantain• 11 months ago

+50% productivity for 200$/mo is outstanding value! Most countries have 0-2% productivity growth per year!

IgorPartola3• 11 months ago

This is a one time gain. Think of it like instead of doing the work yourself you are just pair programming with a junior developer whom you cannot trust to write secure and bug free code. You can probably optimize your interactions such that this “intern” does work while you do other work. But if it’s work blocks what you are doing, it’s just that now you work differently.

I toyed with it and found it to be less frustrating to set up the latest layout for a VueJS project, but having it actually write code was… well I had to manually rewrite large chunks of it after it was done. I am sure it will improve but how long until you can tell it the specs, have it work for a few minutes or hours or days, and come back to an actual finished project? My bet is decades to never.

spogbiper7• 11 months ago

I am working on a project that uses LLM to pull certain pieces of information from semi-structured documents and then categorize/file them under the correct account. it's about 95% accurate and we haven't even begun to fine tune it. i expect it will require human in the loop checks for the foreseeable future, but even with a human approval of each item, its going to save the clerical staff hundreds of hours per year. There are a lot of opportunities in automating/semi-automating processes like this, basically just information extraction and categorization tasks.

beepbooptheory2• 11 months ago

How will you know in practice which 5% is wrong?

kjkjadksj• 11 months ago

Isn’t that something you can do with non ai tooling to 100% accuracy?

systemerror• 11 months ago

The big issue with LLMs is that they’re usually right — like 90% of the time — but that last 10% is tough to fix. A 10% failure rate might sound small, but at scale, it's significant — especially when it includes false positives. You end up either having to live with some bad results, build something to automatically catch mistakes, or have a person double-check everything if you want to bring that error rate down.

whatever1• 11 months ago

All of the AI projects promise that they just need some fine tuning to go from poc to actual workable product. Nobody was able to fine tune them.

Sorry this is some bull. Either it works or it doesn’t.

LPisGood4• 11 months ago

its going to save the clerical staff hundreds of hours per year

How many hundreds of hours is your team spending to get there? What is the ROI on this vs investing that money elsewhere?

kingkawn2• 11 months ago

#5 is an enormous use case that when well implemented will permanently replace prescribed character arcs

It is uniquely susceptible because the gaming market is well acclimated to mediocre writing and one dimensional character development that’s tacked on to a software product, so the improvements of making “thinking” improvisational characters can be immense.

Another revenue potential you’ve missed is visual effects, where AI tools allow what were previously labor intensive and expensive projects to be completed in much less time and with less, but not no, human input per frame

shantara• 11 months ago

#5 is an enormous use case that when well implemented will permanently replace prescribed character arcs

I mostly disagree. Every gaming AI character demo I've seen so far is just adds more irrelevant filler dialogue between the player and the game they want to play. It's the same problem that some of the older RPG games had, thinking that 4 paragraphs of text is always better than 1.

b8• 11 months ago

AI Customer Service is very frustrating to work with as a end user.

siliconc0w• 11 months ago

The problem is that a lot of friction is intentional. Companies want there to be friction to return an item or cancel a subscription. Insurance companies want there to be friction to evaluate policies or appeal denied claims. Companies create legal friction to make competition harder. The friction is the point so AI isn't a solution. If you were to add AI they'd just find a way to create new friction.

Spivak4• 11 months ago

I mean look, if the customer service department is just trying to frustrate you into not contacting them then AI is just a new tool in the belt for that. Sad, but improvement for them is explicitly worse for you.

But if you're actually trying to provide good customer service because people are paying you for it any paying per case then you wouldn't dare put a phone menu or AI chat bot in-between them and the human. The person handles all the interaction with the client and then uses AI where it's useful to speed up the actual work.

jmkni• 11 months ago

Yep.

The thing is, you aren't contacting customer services because everything is going well, you are contacting them because you have a problem.

The last thing you need is to be gaslit by an AI.

The worst ones are the ones where you don't realise right away you aren't talking to a person, you get that initial hope that you've actually gotten through to someone who can help you (and really quickly too) only to have it dawn on you that you are talking to a ChatGPT wrapper who can't help you at all.

timeinput• 11 months ago

So are call trees where you have to answer yes / no to a decision tree (and can't press 0 / 1, you have to verbalize "yes" or "no"). Those continue to exist, so I expect AI customer 'service' to be very frustrating to work with for quite some time.

SalmoShalazar• 11 months ago

Even “only sort of works” is too generous for point #2. A dozen Claude Code agents spamming out code is… something. But it still does not replace a human, at all, even a junior one. It’s something else entirely.

empath752• 11 months ago

Replace junior developers with Claude Code or similar.

I don't know why everyone goes to "replacing". Were a bunch of computer programmers replaced when compilers came out that made writing machine code a lot easier? Of course not, they were more productive and accomplished a lot more, which made them more valuable, not less.

LPisGood4• 11 months ago

A lot of companies have upper limits on the value add of more programmers.

jsmith99• 11 months ago

MS Copilot is quite useful for meeting minutes and summaries etc. Still not nearly as useful as good handwritten notes but saves loads of time.

IgorPartola3• 11 months ago

Sure useful. But we are talking making money useful, not just nice. Like will it create 20% more revenue? 1% more? Or is it just a nice to have?

mrweasel2• 11 months ago

Part of the issue with all this AI hype is that I can't tell if you're joking or not. Most of those suggestions are horrible, 4 and 5 makes sense.

carlosjobim• 11 months ago

The most important use for AI is for translation. Now anybody can communicate and serve customers and clients from everywhere in the world, and you can also translate all your customer-facing material into any language in the world.

That means you expand from millions to billions of potential customers.

mbb70• 11 months ago

In healthcare, notes are directly correlated with $$$ for the hospital, because everything that is billed for must be documented with a mix of metrics (O2%, temp, lab results), events (orders, prescriptions, procedures) and notes (consultation notes, imaging interpretations, discharge summaries).

Billions get spent annually in administrative overhead focused on squeezing the most money out of these notes as possible. A tremendous expense can be justified to increase note quality (aka revenue, though 'accuracy/efficiency' is the trojan horse used to slip by regulators).

GenAI has a ton of potential there. Likewise on the insurance side, which has to wade through these notes and produce a very labor intensive paper trail of their own.

Eventually the AIs will just sling em-dashes at each other while we sit by pool.

m3kw9• 11 months ago

Is hard to measure return when you prompt a question and it comes out with a partial solution.

cm2012• 11 months ago

The report itself can only be viewed by filling out a form - this article is so details light as to be useless.

lacy_tinpot6• 11 months ago

Why do people so desperately want to see AI fail?

lkramer• 11 months ago

I think people want to see the mindless application of crappy LLM chatbots and AI summaries everywhere fail. At least that's my position. I would also like the notion that "development can be sped up 6-700% by applying AI" would go away.

taormina2• 11 months ago

Why do people so desperately want to see AI succeed? The financial investment explains it for some.

lacy_tinpot6• 11 months ago

Why do want a technology that's able to automate human drudgery? Feels like an absurd question.

snozolli2• 11 months ago

Because it's going to destroy knowledge work and the entire middle class.

lacy_tinpot6• 11 months ago

I thought we hated doing the menial office jobs though.

recallingmemory2• 11 months ago

Humans like knowing their next paycheck is safe. AI is a disruption to that feeling of security.

rusted1• 11 months ago

We dont. People just tired of listening to the pipe dreams of these weirdos CEOs who sell AI.

lacy_tinpot6• 11 months ago

We do this every round for every new technology.

Here's the truth: NO ONE KNOWS.

What part of No One Actually Knows do people not understand? This applies to both the "AI WILL RULE THE WORLD MUAHAHA" and "AI is BIG BIG HOAX" crowd.

exasperaited2• 11 months ago

[removed]

lacy_tinpot6• 11 months ago

The real evil people here are the "artists" that have this kind of attitude. I still remember the "Digital Art isn't real Art" people, which btw still exist among your snobby "Fine Arts" crowds.

I think we should actually ban all digital art platforms, no Photoshop, no special effects, all hand drawn. And I'll use some weird weaponized empathy calling out for the human soul and human creativity.

What a toxic bunch.

jampa• 11 months ago

The biggest mistake people are making is treating AI as a product instead of a feature.

While people are doing their work, they don't think, "Oh man, I am really excited to talk with AI today, and I can't wait to talk with a chatbot."

People want to do their jobs without being too bored and overwhelmed, and that's where AI comes in. But of course, we cannot hype features; we sell products after all, so that's the state we are in.

If you go to Notion, Slack, or Airtable, the headline emphasizes AI first instead of "Text Editor, Corporate Chat etc".

The problem is that AI is not "the thing", it is the "tool that gets you to the thing".

jameshart• 11 months ago

Agreed. We’ve got the potential to build real bicycles for the mind here and marketing departments are jumping right in to trying to sell people spandex cycling shorts.

the_snooze2• 11 months ago

I wouldn't mind it if it were presented as yet another tool in the box. Maybe have a one-time popup saying "Hey, there's this thing, here's a cool use case, go check it out on your own terms."

In reality, AI sparkles and logos and autocompletes are everywhere. It's distracting. It makes itself the star of the show instead of being a backup dancer to my work. It could very well have some useful applications, but that's for users to decide and adapt to their particular needs. The ham-fisted approach of shoving it into every UI front-and-center signals a gross sense of desperation, neediness, and entitlement. These companies need to learn how to STFU sometimes.

ryandrake• 11 months ago

I wouldn't even call it a feature. It's enabling technology. I've never once said "I would like AI in [some product]." I say: "I would like to be able to [do this task]." If the company adds that feature to a product, I'll buy it. I don't care if the company used AI, traditional algorithms, or sorcery to make the feature work--I just care that it does what I want it to do.

Too many companies are just trying to spoon AI into their product somehow, as if AI itself is a desired feature, and are forgetting to find an actual user problem for it to actually solve.

ponector4• 11 months ago

> I've never once said "I would like AI in [some product]

But that is exactly what we got. AI washing machine! AI espresso machine! And many more AI tools.

TimCTRL• 11 months ago

I like this take, in fact i feel a little uneasy when i see startups mention "MCP" on their landing pages! Its a protocol and its like saying we use HTTP here.

I could be wrong but, all in all, buy a .com for your "ai" product, such that you survive the Dot-ai bubble [1]

I Love LLM's though!! Amazing math and tech.

[1] https://en.wikipedia.org/wiki/Dot-com_bubble

biophysboy2• 11 months ago

1000% agree, so many AI "applications" right now are solutions looking for a problem.

rank0• 11 months ago

Seriously! The product itself is supposed to be the valuable thing…regardless of the underlying technology.

rpcope12• 11 months ago

All true, but then there goes your stratospheric valuations and all the crazy hype. This come to jesus moment may very well deflate one of the few remaining hot areas around software engineering..I could see people being reluctant to stop the hype train as then we'd really have to come to terms with the fact that the "industry" as a whole is kind of in the shitter and it's a less good time to be a software engineer across the board than 5 or 10 years ago.

TrackerFF• 11 months ago

I'm guessing every VC firm out there is hoping that they'll be the ones that picked the winners - the AI companies that will rule the world in N years.

os2warpman• 11 months ago

TIL 5% of companies are lying about their financials. (or 5% of companies are selling the pickaxes to the unfortunate miners)

ApeWithCompiler• 11 months ago

For the future I will relabel "AI" as "Ain't Interested". But despite the missing return of investment, it only needs a manager or a few, invested enough. They will push layoffs and restructuring regardless of better advice.

md3911027514• 11 months ago

It’s interesting how self-reports of productivity can be wrong.

For example a study from METR found that developers felt that AI sped them up by 20%, but it empirically it slowed them down by 19%. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...

throitallaway2• 11 months ago

Sometimes it's like an autocomplete on steroids and reads my mind. Other times it suggests things that make no sense and completely gets in the way.

sdeframond• 11 months ago

An interesting study indeed. Not enough data points but still way better than anecdata and self reporting!

rapind2• 11 months ago

I suspect there's a lot of nuance you can't really capture in a study like this.

How you use AI will depend on the model, the tools (claude-code vs cursor vs w/e), your familiarity and process (planning phases, vibe coding, etc.), and the team size (solo dev versus large team), your seniority and attention to detail, and hard to measure effects like an increased willingness to tackle harder problems you may have procrastinated on otherwise.

I suspect we're heading to a plateau. I think there's a ton of polish that can be done with existing models to improve the coding experience and interface. I think that we're being massively subsidized by investors racing to own this market, but by the time they can't afford to subsidize it anymore, it'll be such a commodity that the prices won't go up and might even go down regardless of their individual losses.

As someone who knows they are benefitting from AI (study shmuddy), I'm perfectly fine with things slowing down since it's already quite good and stands to be much better with a focus on polish and incremental improvements. I wouldn't invest in these AI companies though!

didibus3• 11 months ago

As someone who knows they are benefitting from AI (study shmuddy)

Look, I get it, I still use it, but you have to admit, people also think that various bogus home remedy totally helps them get over a cold faster. There's absolutely a possibility it in no way makes us faster.

Now, you did say "benefit", that's more broad, and you implied things like polish, I've seen others mention it just makes the work easier, that could be a win in itself (for the workers). Maybe it's about accessibility. Etc.

I do think though, right now, we're all in the "home remedy" territory, until we actually measure these things.

biophysboy2• 11 months ago

I sort of wonder if AI pushes people to stay "in the weeds". I've noticed that the speed of my software development hinges a lot on decisions that require me to take a step back (e.g. Am I using the right tool? Is this feature necessary? etc)

xyst• 11 months ago

I am not surprised at all. The capabilities of AI have been vastly oversold. The only winners here are the billionaires that have had early equity/investment in genAI.

They got a majority of the country hooked into AI without truly understanding its current limitations. This is just like digital currency bubble/fad that popped a couple of years ago.

What most companies got out of it is a glorified chatbot (ie, something that was possible in 2014…) at 1000X the cost.

What a sad joke. Innovation in this country is based on a lie, fueled by FOMO.

jawns• 11 months ago

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value.

Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually.

We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls.

We recently switched to using AI to transcribe and write these summaries. Not only are the summaries better than those produced by our human agents, they also free up the human agents to do higher-value work.

It's not sexy. It's not going to replace anyone's job. But it's a huge, measurable efficiency gain.

vasco• 11 months ago

I wonder if the human agents agree the AI summaries are better than their summaries. I was nodding as I read and then told myself "yeah but it wouldn't be able to summarize the meetings I have", so I wonder if this only works in 3rd person.

dymk2• 11 months ago

Have you tried having it summarize the meetings you have?

mbStavola• 11 months ago

Part of me also wonders if people may agree that its better simply because they don't actually have to do the summarization anymore. Even if it is worse by some %, that is an annoying task you are no longer responsible for; if anything goes wrong down the line, "ah the AI must've screwed up" is your way out.

jcims• 11 months ago

I built a little solution to record and transcribe all of my own meetings. I have many meetings (30hr week+) and I can't keep pace with adequate note-taking while participating in them all.

I'm finding that the summarization of individual meetings very useful, I'm also finding that the ability to send in transcripts across meetings, departments, initiatives whatever to be very effective at surfacing subtexts and common pain points much more effectively than I can.

I'm also using it to look at my own participation in meetings to help me see how I interact with others a (little) bit more objectively and it has helped me find ways to improve. (I don't take its advice directly lol, just think about observations and determine myself if it's something that's important and worth thinking about)

kenjackson2• 11 months ago

AI definitely summarizes meetings better than me and _almost_ anyone else I've seen do it (there is one exception -- one guy was a meeting note taker god. He was so good that he set up a mailing list because so many people wanted to read his meeting notes.) I could probably do better than AI if I really tried, but I've only ever done that a few times.

doubled1122• 11 months ago

At work we've tried AI summaries for meetings, but we spent so much time fixing those summaries that we started writing our own again.

Is there some training you applied or something specific to your use case that makes it work for you?

nsxwolf• 11 months ago

We stopped after it kept transcribing a particular phrase of domain jargon as “child p*rn”, again and again.

cube004• 11 months ago

Unless a case goes down the legal road, nobody is ever bothering to read old call summaries in a giant call center.

When was the last time you called a large company and the person answering was already across all the past history without you giving them a case number first?

shawabawa3• 11 months ago

My guess is that the summaries are never actually read, so accuracy doesn't actually matter and the AI could equally be replaced with /dev/null

mrweasel2• 11 months ago

We tried Otter.ai, someone complained and asked: "Could you f-ing not? I don't trust them" and now Otter is accused of training their models on recorded meetings without permission. Yeah, I don't even care if it works, I don't trust any of these companies.

butlike3• 11 months ago

How can you double-check the work? Also, what happens when the AI transcription is wrong in a way that would have terminated the employee. You can't fire a model.

Finally, who cares about millions saved (while considering the above introduced risk), when trillions are on the line?

throitallaway2• 11 months ago

I presume they're not using these notes for anything mission or life critical, so anything less than 100% accuracy is OK.

PaulRobinson• 11 months ago

Having a human read a summary is way faster than getting them to write it. If they want to edit it, they can.

AI today is terrible at replacing humans, but OK at enhancing them.

Everyone who gets that is going to find gains - real gains, and fast - and everyone who doesn't, is going to end up spending a lot of money getting into an almost irreversible mistake.

pluc• 11 months ago

You could do that offline 20 years ago with Dragon Naturally Speaking and a one-time licence.

dymk2• 11 months ago

You could get a transcript, not a summary

loloquwowndueo• 11 months ago

Only if your audio was crystal clear, you spoke like a robot very slowly, and each customer has to do a 30-minute “please read this text slowly to train the speech recognition software” preamble before talking to the actual human.

glimshe• 11 months ago

This makes sense. AI is obviously useful for many things. But people wouldn't invest tens of billions to summarize data center calls and similar tasks. Replacing data center workers isnt where the money is - it's replacing 100K-200K/year workers.

generic920343• 11 months ago

It's not going to replace anyone's job.

Is it not, in the scenario you are describing? You are saying the agents are free now to do higher-value work. Why were there not enough agents before, especially if higher-value work was not done?

hobs2• 11 months ago

Because call centers are cost centers - nobody pays a dime more than they have to in these situations and its all commodity work.

cube004• 11 months ago

It's such a useless platitude. The "higher value work" is answer more calls so we can have less staff on queue.

positron262• 11 months ago

Given that people skimp on work that is viewed as trash anyway, how were you getting value out of the summaries in the first place?

thrown-08252• 11 months ago

they weren't

its likely a checkbox for compliance or some policy a middle manager put in place that is now tied to a kpi

Terr_3• 11 months ago

I think the biggest issue is accurately estimating the LLM failure risk, and what impacts the company is willing to tolerate in the long term. (As distinct from what the company is liable to permit through haste and ignorance.)

With LLMs the risk is particularly hard to characterize, especially when it comes to adversarial inputs.

hobs2• 11 months ago

That is a good thing, but that's also just a training gap - I worked with tech support agents for years in gigantic settings and taking notes while you take an action is difficult to train but yields tangible results, clarity from the agent on what they are doing step by step, and builds a shared method of documenting things that importantly focuses on the important details and doesn't miss a piece which may be considered trivial by an outsider but (for instance) defines SOP for call escalation and the like.

dsr_3• 11 months ago

Pro-tip: don't write the summary at all until you need it for evidence. Store the call audio at 24Kb/s Opus - that's 180KB per minute. After a year or whatever, delete the oldest audio.

There, I've saved you more millions.

lotsofpulp• 11 months ago

The summaries can help automate performance evaluation. If the employee disputes it, I imagine they pull up the audio to confirm.

anoojb• 11 months ago

Also entrenches plausible deniability and makes legal contests way more cumbersome for plantiffs to resolve.

paulddraper2• 11 months ago

This works unless you want to automate something with the transcripts, stats, feedback.

alooPotato2• 11 months ago

you want to be able to search over summaries so you need to generate them right away

andix• 11 months ago

Imagine a follow-up call of a customer. They are referring to earlier calls and the call center agents needs to check what it was about. So they can skim/read the transcripts while talking to the customer. I guess it's really hard to listen to transcripts while you're on the phone.

doorhammer12• 11 months ago

Sentiment analysis, nuanced categorization by issue, detecting new issues, tracking trends, etc, are the bread and butter of any data team at a f500 call center.

I'm not going to say every project born out of that data makes good business sense (big enough companies have fluff everywhere), but ime anyway, projects grounded to that kind of data are typically some of the most straight-forward to concretely tie to a dollar value outcome.

FirmwareBurner• 11 months ago

Store the call audio at 24Kb/s Opus - that's 180KB per minute

Why OPUS though? There's dedicated audio codecs in the VoiP/telecom industry that are specifically designed for the best size/quality for voice call encoding.

sillyfluke3• 11 months ago

You also will have saved them all the cost of the AI summaries that are incorrect as well.

The parent states:

Not only are the summaries better than those produced by our human agents...

Now, since they have not mentioned what it took to actually verify that the AI summaries were in fact better than the human agents, I'm sceptical they did the necessary due dillengence.

Why do I think this? Because I have actually tried to do such a verification. In order to verify that the AI summary is actually correct you have to engage in the incredibly tedious task of listening to original recording literally second by second and make sure that what is said does not conflict with the AI summary in question. Not only did the AI summary fail at this test, it failed in the first recording I tested.

The AI summary stated that "Feature x was going to be in Release 3, not 4" whereas the in the recording it is stated that the feature will be in Release 4 not 3, literally the opposite of what the AI said.

I'm sorry but the fact that the AI summary is nicely formatted and has not missed a major topic of conversation means fuck all if the details that are are discussed are spectacularly wrong from a decision tracking perspective, as in literally the opposite of what is stated.

And I know "why" the Ai summary fucked up, because in that instance the topic of conversation was about how there was some confusion about which release that feature was going to be in, that's why the issue was a major item of the meeting agenda in the first place. Predicably, the AI failed to follow the convoluted discussion and "came to" the opposite conclusion.

In short, no fucking thanks.

tux35• 11 months ago

it's a huge, measurable efficiency gain.
It's not going to replace anyone's job

Mechanically, more efficiency means less people required for the same output.

I understand there is no evidence that any other sentence can be written about jobs. Still, you should put more text in between those two sentences. Reading them so close together creates audible dissonance.

flkiwi• 11 months ago

You're not accounting for teams already being understaffed and overtasked in many situations, with some AI tools allowing people to get back to doing their jobs. We aren't expecting significant headcount changes, but we are expecting significant performance and quality improvements for the resources we have.

missedthecue3• 11 months ago

"Mechanically, more efficiency means less people required for the same output."

Why can't it mean more output with the same number of people? If I pay 100 people for 8 hours of labor a day, and after making some changes to our processes, the volume of work completed is up 10% per day, what is that if not an efficiency gain? What would you call it?

It really depends on the amount of work. If the demand for your labor is infinite, or at least always more than you can do in a days work, efficiency gains won't result in layoffs, just more work completed per shift. If the demand for the work is limited, efficiency gains will likely result in layoffs because there's no point in paying someone who was freed up by your new processes to sit around twirling a pen all day.

belter4• 11 months ago

Are the summaries reviewed by the agents? And if not how do you handle hallucinations, or transcribe of wrong insurance policy id for example? Like, customer wants to cancel insurance policy AB-2345D and transcribe says wants to cancel insurance policy AD-2345B

pedrocr• 11 months ago

agents were previously spending 3-5 minutes after each call writing manual summaries of the calls

Why were they doing this at all? It may not be what is happening in this specific case but a lot of the AI business cases I've seen are good automations of useless things. Which makes sense because if you're automating a report that no one reads the quality of the output is not a problem and it doesn't matter if the AI gets things wrong.

In operations optimization there's a saying to not go about automating waste, cut it out instead. A lot of AI I suspect is being used to paper over wasteful organization of labor. Which is fine if it turns out we just aren't able to do those optimizations anyway.

nulbyte• 11 months ago

As a customer of many companies who has also worked in call centers, I can't tell you how frustrating it is when I, as a customer, have to call back and the person I speak with has no record or an insufficient record of my last call. This has required me to repeat myself, resend emails, and wait all over again.

It was equally frustrating when I, as a call center worker, had to ask the custmer to tell me what should already have been noted. This has required me to apologize and to do someone else's work in addition to my own.

Summarizing calls is not a waste, it's just good business.

Shank• 11 months ago

Who reads the summaries? Are they even useful to begin with? Or did this just save everyone 3-5 minutes of meaningless work?

vosper• 11 months ago

AI reads them and identifies trends and patterns, or answers questions from PMs or others?

doorhammer12• 11 months ago

Not the op, but I did work supporting three massive call centers for an f500 ecom.

It's 100% plausible it's busy work but it could also be for: - Categorizing calls into broad buckets to see which issues are trending - Sentiment analysis - Identifying surges of some novel/unique issue - Categorizing calls across vendors and doing sentiment analysis that way (looking for upticks in problem calls related to specific TSPs or whatever) - etc

False positives and negatives aren't really a problem once you hit a certain scale because you're just looking for trends. If you find one, you go spot-check it and do a deeper dive to get better accuracy.

Which is also how you end up with some schlepp like me listening to a few hundreds calls in a day at 8x speed (back when I was a QA data analyst) to verify the bucketing. And when I was doing it everything was based on phonetic indexing, which I can't imagine touching llms in terms of accuracy, and it still provided a ton of business value at scale.

recallingmemory2• 11 months ago

So long as precision isn't important, I suppose. Hallucination within summaries is the issue I keep running into which prevents me from incorporating it into any of our systems.

thrown-08252• 11 months ago

I have seen ai summaries of calls get people into trouble because the ai hallucinated prices and features that didn't exist

trevor-e• 11 months ago

This is a great use-case of AI.

However I strongly doubt your point about "It's not going to replace anyone's job" and that "they also free up the human agents to do higher-value work". The reality in most places is that fewer agents are now needed to do the same work as before, so some downsizing will likely occur. Even if they are able to switch to higher-value work, some amount of work is being displaced somewhere in the chain.

And to be clear I'm not saying this is bad at all, I'm just surprised to see so many deluded by the "it won't replace jobs" take.

varispeed2• 11 months ago

What’s the actual business value of a “summary” though? A transcript is the record. A tag or structured note (“warranty claim,” “billing dispute,” “out of scope”) is actionable. But a free-form blob of prose? That’s just narrative garnish - which, if wrong or biased, is worse than useless.

Imagine a human agent or AI summarises: “Customer accepted proposed solution.” Did they? Or did they say “I’ll think about it”? Those aren’t the same thing, but in the dashboard they look identical. Summaries can erase nuance, hedge words, emotional tone, or the fact the customer hung up furious.

If you’re running a call centre, the question is: are you using this text to drive decisions, or is it just paperwork to make management feel like something is documented? Because “we saved millions on producing inaccurate metadata nobody really needs” isn’t quite the slam dunk it sounds like.

amluto2• 11 months ago

I’d like to see a competent AI replace the time that doctors and nurses spend tediously transcribing notes into a medical record system. More time spent doing the actual job is good for pretty much everyone.

simmerup• 11 months ago

Until it hallucinates and the AI has written something wrong about you in your official medical record

nottorp• 11 months ago

Competent, yes. But the current ones are likely to transcribe "recommend amputating the left foot" as "recommend amputating the right foot". Still want it?

beart• 11 months ago

But... that is the actual job. A clear medical history is very important, and I'm not ready yet to cut out my doctor from that process.

This reminds me of the way juniors tend to think about things. That is, writing code is "the actual job" and commit messages, documentation, project tracking, code review, etc. are tedious chores that get in the way. Of course, there is no end to the complaints of legacy code bases not having any of those things and being difficult to work with.

jordanb• 11 months ago

We use Google meet and it has Gemini transcriptions of our meetings.

They are hilariously inaccurate. They confuse who said what. They often invert the meaning "Joe said we should go with approach x" where Joe actually said we should not do X. It also lacks context causing it to "mishear" all of our internal jargon to "shit my iPhone said" levels.

ricardonunez• 11 months ago

I don’t know how it can confuse because input on mic is relatively straight forward to get. I use fathom and others and they are accurate, better than manual taken. Interesting take, that I don’t memorize 100% on the calls anymore since I rely on note takers, I only remember the major points but when I read the notes, everything comes clear.

rowanseymour• 11 months ago

Same here. It's frustrating that it doesn't seem to have contextual awareness of who we are and the things we work on so things like names of our products, names of big clients, that we use repeatedly in meetings, are often butchered.

orphea• 11 months ago

Oh, that's what happening. I thought my English is just terrible :(

thisisit• 11 months ago

I found that if you have people with accents and they emphasize certain words then it becomes very difficult to read. One example, I find is "th" is often D because how people pronounce it. Apart from that it is a hit or miss.

sigmoid10• 11 months ago

That's the difference between having real AI guys and your average linkedIn "AI guys." The other post is a perfect example for a case where you could take a large but still manageable, cutting-edge transcription model like Whisper and fine-tune it using existing hand made transcriptions as ground truth. A match made in heaven for AI engineers. Of course this is going to work way, way better for specific corporate settings than slapping a random closed source general purpose model like Gemini on your task and hoping for the best, just because it achieves X% on random benchmark Y.

nostrademons• 11 months ago

I also use Gemini notes for all my meetings and find them quite helpful. The key insight is: they don’t have to be particularly accurate. Their primary purpose is to remind me (or the other participants) of what was discussed, what considerations were brought up, and what the eventual decision was. If it inverts the conclusion and forgets a “not”, we’re going to catch that, because we were all in the meeting too. It’s their to jog our memory of what was said, because it’s much easier to recognize correct information than recall it, it’s not the authoritative source of truth on the meeting.

This gets to a common misconception when it comes to GenAI uses: it functions best as “augmented intelligence” rather than “artificial intelligence”. Meaning that it’s at its best when there’s still a human in the loop and the AI supplements the parts the person are bad at rather than replacing the person entirely. We see this with coding, where AI is very good at writing scaffolding, large-scale refactoring, picking decent libraries, reading API docs and generating code that calls it appropriately, etc but still needs a human to give it very specific directions for anything subtle, and someone to review carefully for bugs and security holes.

actsasbuffoon2• 11 months ago

That’s the thing. There’s value in AI, it’s just not worth half a trillion dollars to train a new model that’s 0.4% better on benchmarks. Meta is never going to get a worthwhile return on spending $100M on individual engineers.

But that doesn’t mean AI is without its uses. We’re just in that painful phase where the hype needs to die down and we treat LLMs as what they really are; an interesting new tool in the toolkit that provides some new ways to solve problems. It’s almost certainly not going to turn into AGI any time soon. It’s not worth trillions. It’s certainly worth something, though.

I think the financials on developing new frontier models are terrible. But I’ve already built multiple AI projects for my company that are making money and we’ve got extremely happy customers.

Investors thought one company was going to win the AI Wars and make a quadrillion dollars. Instead it’s probably going to be 10,000 startups that will build interesting products based on AI, and training new models won’t actually be a good financial move.

Imanari• 11 months ago

Could you broadly describe the AI projects you have built?

nuker• 11 months ago

recently switched to using AI to transcribe and write these summaries

Did users knew that conversation was recorded?

prophesi• 11 months ago

You would be hard-pressed to find a call center that _doesn't_ start every phone call with a warning that the conversation may be recorded.

creaturemachine• 11 months ago

Yeah the standard "this call may be recorded for quality or training purposes" preamble shouldn't cover for slurping your voiceprint to further the butchering of client service that this call centre is here for.

watwut• 11 months ago

Typical call center call is recorded and you are told so by the start of the conversation. I had quite a few of those.

Capricorn2481• 11 months ago

Not only are the summaries better than those produced by our human agents

We have someone using Firefly for note taking, and it's pretty bad. Frequently gets details wrong or extrapolates way too much from a one-off sentence someone said.

How do you verify these are actually better?

ghalvatzakis3• 11 months ago

I lead an AI engineering team that automated key parts of an interviewing process, saving thousands of hours each month by handling thousands of interviews. This reduced repetitive, time-consuming tasks and freed human resources to focus on higher-value work

the_snooze2• 11 months ago

I'm under the impression that one of the most critical responsibilities a lead has is to establish and maintain a good working culture. Properly vetting new additions feeds directly into that. Why offload it to AI?

pjmorris• 11 months ago

I wonder how this change affects what the agents remember about the calls, and how that affects their performance on future calls. And I wonder whether agent performance, as measured by customer satisfaction, will decline over time, and whether that will affect the bottom line.

stronglikedan• 11 months ago

I wouldn't allow myself to be held accountable for anything in a summary I didn't write.

MangoToupe• 11 months ago

We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls.

This is a tiny fraction of all work done. This is work people were claiming to have solved 15 years ago. Who cares?

didibus3• 11 months ago

Where is the money being saved? Are you reducing the number of agents? Otherwise, it should actually cost more, before you simply had customers wait longer to speak to the next agent no? Or do you sell "support calls" so you're able to sell more of them given the same number of agents?

ozgune• 11 months ago

Previously discussed here: https://news.ycombinator.com/item?id=44941118

It's also disappointing that MIT requires you to fill out a form (and wait for) access to the report. I read four separate stories based on the report, and they all provide a different perspective.

Here's the original pdf before MIT started gating it: https://web.archive.org/web/20250818145714/https://nanda.med...

doorhammer12• 11 months ago

I'm curious, have you noticed an impact on agent morale with this?

Specifically: Do they spend more time actually taking calls now? I guess as long as you're not at the burnout point with utilization it's probably fine, but when I was still supporting call centers I can't count the number of projects I saw trying to push utilization up not realizing how real burnout is at call centers.

I assume that's not news to you, of course. At a certain utilization threshold we'd always start to see AHTs creep up as agents got burned out and consciously or not started trying to stay on good calls.

Guess it also partly depends on if you're in more of a cust serv call center or sales.

I hated working as an actual agent on the phones, but call center ops and strategy at scale has always been fascinating.

lljk_kennedy• 11 months ago

Thank you, I came to say this too. You're mushing your humans harder, and they'll break. Those 5 mins of downtime post-call aren't 100% note taking - it's catching their breath, trying to re-compose after dealing with a nasty customer, trying to re-energise after a deep technical session etc.

I think AI in general is just being misused to optimise local minima in detriment to the overall system.

ethagknight2• 11 months ago

This highlights the potentially unrealistic spend on AI, where the proposal is to spend 10s of millions to save... marginal millions... that could have also been saved by a change in process with limited additional spend.

I also would assume that there are far more significant behavioral or human factors that consume the time writing those minutes, i.e. an easy spot to kill 5-10 min before opening the line for the next inbound call, but the 5-10 minute break will persist anyway.

I fully believe AI will create a lot of value and is revolutionary, especially for industries where value is hidden within data. Its the pace of value creation that stands out to me (how long til its actually useful and better and creates more value than it costs??) but the bubble factor is not ignorable on the near term.

apwell23• 11 months ago

i was pretty sure you were going to say "meeting summaries" ( which apparently is poster child of LLM application) .

my guess was wrong but not really.

chasd002• 11 months ago

so you're saving 3-5 minutes per agent per call. I'm guessing calls come into a queue and then the next available agent starts to handle it. If an average call takes about 20min until the agent hangs up and is free for another then after about 5 calls they've saved enough time to take an extra call they wouldn't have before. 5 calls is 1.4 hrs on the phone, i'm guessing with breaks and call center reps not being 100% on the ball all the time then your agents probably will take maybe 3-4 more calls per day with the AI than without (assuming call volume is such that there are always more calls than agents can handle)

Is that really millions of savings annually? Maybe it is but I always hesitate when a process change that saves one person a few minutes is extrapolated all the way out to dollars/year. What you'll probably see is the agents using those 3-5 minutes to check their phone.

croes• 11 months ago

But I doubt it justifies the billions of dollars getting burned for training language models and building power plants.

And are full transcriptions not the better option?

tempodox• 11 months ago

That’s an important point. Real-life use cases are not sexy. And they don’t lend themselves to overblown hype generation and “creative marketing”.

ponector4• 11 months ago

Your company saved millions means they fired many agents and hiked amount of cases per agent?

dvfjsdhgfv2• 11 months ago

What I found is that although some tools are good for transcription and the final result is quite accurate, this is not necessarily true for summaries.

Yes, I routinely read meeting summaries spit out by the so-called SOTA tools and they almost always contain major errors. So if you actually need them for anything serious, it would be wise to keep the audio.

dsr_3• 11 months ago

"I'm sure my company is among the 5% of superusers," said CEO Chad "Bro" Brochad, who later admitted he had not checked.

Terr_3• 11 months ago

Can you really put a price on a good excuse for layoffs that deflects rancor from employees while sounding like positive news to investors? :p

... Well, probably yes, but I don't have the data to do it.

tqi• 11 months ago

The link to the report pdf redirects to a landing page[1] that makes me think this "study" is run-of-the-mill content marketing. The CTA at the bottom of the page says "Join the Revolution / Be at the forefront of creating the agentic web. Get exposure and faster adoption for your AI products and tools," which certainly doesn't give the impression that this is an objective report. Either way I can't speak to the quality of the study directly because it is walled off behind a contact information form, which is another bad sign.

From the article though:

But researchers found most use cases were limited to boosting individual productivity rather than improving a company’s overall profits.

What does that even mean?

[1] Website: https://nanda.media.mit.edu/, FAQ: https://projnanda.github.io/projnanda/#/faq_nanda

leetharris• 11 months ago

Try to click the author name. It doesn't work. This isn't a real author or a real site.

HN is turning into reddit, where people look at the title, come to the comments, and post if they agree with the title or not.

kingstnap• 11 months ago

I mean of course they aren't. So many people are offering so much stuff for free or at a pittance.

I honestly don't think it matters though. Feel free to disagree with me but I think the money is irrelevant.

The only thing that actually matters is the long run is the attention, time, and brain space of other people. After all that's where fiat currency actually derives it's value. These Gen AI companies have captured a lot of that extremely quickly.

OpenAI might have "burned" billions but they way they have wrung themselves into seemingly every university student's computer, every CEOs mind, the policy decisions of world leaders, ever other hackernews post, is nothing short of miraculous...

justinator• 11 months ago

It HAS lead to widespread enshitification of products and services though!

mannyv• 11 months ago

My daughter, as an intern, created a whole set of prompts for a metal fab that extracted all the metal parts out of a CAD file (or PDF) and dimensions so it's easier for them to bid.

Saved them hours of work.

Of course, they didn't spend on "AI" per se.

Most people don't know how to meta their job functions, so AI won't really be worth it. And the productivity gains may not be measurable ie: "I did this in 5 minutes instead of 500, so I was able to goof off more."

phlipski• 11 months ago

This sounds fascinating! I'd like to learn more about using AI for CAD file generation - since freecad is miserable to use! Can you share more on this?

ryeguy_24• 11 months ago

Does anyone have this mystical report?

nerevarthelame• 11 months ago

https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Bus...

rconti2• 11 months ago

I'm no AI apologist, but for one, this is how investments, particularly speculative investments, work. They're investing money now in the _hopes_ of a future return. It's pretty early days still. Secondly, of _course_ the huge AI players are doing everything they can to overpromise and convince corporations to throw cash at them to keep the party going.

I think the real problem is, it's just a bit too early, but every CEO out there dreams of being lauded for their visionary take on AI, and nobody wants to miss the bus. It's high-leverage tech, so if it (some day) does what it's supposed to do, and you miss making the investment at the right time, you're done.

Thrymr• 11 months ago

It is rare that a third of the capital in the US stock market (by some recent estimates) is going for essentially the same speculative bet, though.

rconti2• 11 months ago

Sure-- I'm not saying ANY of this is good!

maerF0x0• 11 months ago

If everyone does it, then no one gets an advantage, because all see a productivity boost.

If you do not do it, you get left behind and cannot compete in the marketplace.

I took a business systems administration course like 20 years ago, and they knew this was the case. As far as we can tell it's always been the case.

IT doesn't create massive moats/margins because price competition erodes the gap. And yet if you do not keep up you lose.

It's definitely a boon for humanity though, in the industries where technology applies things have been very obviously getting much cheaper over time.

(Most notably American housing has been very very resistant to technological change and productivity gains, a part of the story why housing has gone way up) - https://youtu.be/VfYp9qkUnt4?si=D-Jpmojtn7zV5E8T

manishsharan• 11 months ago

I am not interested in the results of the 95%.

I want to know more about the 5% who got it right. What are their use cases ?

revskill• 11 months ago

Because they are doom to fail.

morelandjs• 11 months ago

I’d have to guess that new startup founders are building leaner teams leading to slower burn rates and longer run ways. That has value. I think share skepticism is warranted is whether or not old behemoths can retrofit AI efficiency gains into their existing organizational structures.

crowcroft• 11 months ago

IMO 'Zero return' is really just saying that these companies never had a plan for their AI implementation in the first place, and so never had any idea what they were trying to achieve and measure in the first place.

The article does call out clear issues companies have with AI workflows etc. and those are likely real problems, but if you're saying *zero* return those aren't the root cause problems.

seatac763• 11 months ago

Ed Zitron was right all along eh.

ChrisArchitect• 11 months ago

Earlier discussion on the report:

AI is predominantly replacing outsourced, offshore workers

https://news.ycombinator.com/item?id=44940944

PDF report that was taken down/walled: https://web.archive.org/web/20250818145714/https://nanda.med...

pjmlp• 11 months ago

I can hardly wait for the bubble to burst, now everyone is forced to meet OKRs on using AI at work, while at the same time, being forbidden to use the data that actually makes the tools usable.

turnsout2• 11 months ago

The AI wave is so reminiscent of the early days of the internet. Right now we're in about 1999, a time of tremendous hype. Business that weren't even in tech were wondering how much they need to know about the HTTP spec. People from the marketing team were being pulled off their print ad campaigns, given a copy of Dreamweaver, and told to make the company website. We hadn't even figured out the roles.

It 100% turned out to be a bubble and yet, if anything, the internet was under-hyped. The problem in 1999 was that no one really knew how it was going to play out. Which investments would be shrewd in retrospect, and which ones would be a money pit?

When an innovation hits, it takes time to figure out whether you're selling buggy whips, or employing drivers who can drive any vehicle.

Plenty of companies sunk way too much money into custom websites back in 99, but would we say they were wrong to do it? They may have overspent at a time when a website couldn't justify the ROI within 12 months, but how could they know? A few short years later, a website was virtually required for every business.

So are companies really seeing "zero return" on their AI spend, or are they paying for valuable lessons about how AI applies to their businesses? There may be zero ROI today, but all you need to do is look at the behavior of normal people to see that AI is not going anywhere. Smart companies are experimenting.

llmllmkom• 11 months ago

The internet is about sharing actual information. LLMs can't share real information, just a flimsy derivative. It's in their DNA. How so many supposedly smart people who understand machine learning and its fundamental entropy failed to acknowledge this reality from day zero, is beyond me.

turnsout2• 11 months ago

Not every revolution is about sharing information. I have plenty of healthy skepticism of companies adding "AI sauce" to everything, but if you can't see the genuine utility of LLMs, it might just be a failure of imagination.

foxfired• 11 months ago

Atlassian added this new AI feature to create audio summaries of Confluence documents. It's really impressive. No one reads documentation. I know it because I'll send someone a documentation, then they'll ask me a question that is answered in the second paragraph, meaning they didn't read it.

So their feature is not just text to speech, but a reading of a summarized version of the articles. But here is the problem. The documentation has no fluff. You don't want a summary, you want the actual details. When you are reading the document that describes how the recovery fee is calculated, you want to know exactly how it is calculated.

I've ran it on multiple documents and it misses key information. An unsuspecting user might take it at face value. So this feature looks impressive, but it misses the entire point of documentation. Which is *preserving the details*.

swader999• 11 months ago

AI reads the docs. For the first time in twenty years I'm actually advocating for more documentation.

GloriousMEEPT• 11 months ago

One thing I ponder after remembering previous tech booms, is that they often left behind something extremely valuable (fiber). With all the GPU datacenters rolling out, what next bubble can take advantage of this boon, if it is one?

eulgro• 11 months ago

It will probably all be used to mine bitcoin.

OutOfHere2• 11 months ago

For many years now, Bitcoin is mined only with custom ASICs, no longer with GPUs.

wheelerwj• 11 months ago

My guess is that this title could also be written as, “The value of AI projects are being captured by just 5% of companies.”

It’s pretty clear to anyone who’s using this technology that it’s significant. Theres still tons to work out and the exact impact is still unknown. But this cat isn’t going back in the bag.

LPisGood4• 11 months ago

It’s pretty clear to anyone who’s using this technology that it’s significant

I disagree entirely. It’s neat, and it’s a marginal improvement over current-year google, but significant is an overstatement.

resiros• 11 months ago

Here is the report: https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Bus...

The story there is very different than what's in the article.

Some infos:

- 50% of the budgets (the one that fails) went to marketing and sales

- the authors still see that AI would offer automation equaling $2.3 trillion in labor value affecting 39 million positions

- top barriers for failure is Unwillingness to adopt new tools, Lack of executive sponsorship

Lots of people here are jumping to conclusions. AI does not work. I don't think that's what the report says.

baal80spam• 11 months ago

Lots of people here are jumping to conclusions. AI does not work. I don't think that's what the report says.

Well...

"It is difficult to get a man to understand something when his salary depends upon his not understanding it"

didibus3• 11 months ago

affecting 39 million positions

Wow, that is crazy. There's 163 million working Americans, that's close to a quarter of the workforce is at risk.

rawgabbit• 11 months ago

This stood out to me in the report:

      A corporate lawyer at a mid-sized firm exemplified this dynamic. Her organization invested $50,000 in a specialized contract analysis tool, yet she consistently defaulted to ChatGPT for drafting work: "Our purchased AI tool provided rigid summaries with limited customization options. With ChatGPT, I can guide the conversation and iterate until I get exactly what I need. The fundamental quality difference is noticeable, ChatGPT consistently produces better outputs, even though our vendor claims to use the same underlying technology." This pattern suggests that a $20-per-month general-purpose tool often outperforms bespoke enterprise systems costing orders of magnitude more, at least in terms of immediate usability and user satisfaction. This paradox exemplifies why most organizations remain on the wrong side of the GenAI Divide.

kami23• 11 months ago

That's a similar story for me at $DAYJOB. We have copilot for our IDEs and it is so much worse than Claude code or any other CLI integration option. We're so restricted on adopting the features as fast as they are turned on. I try to use it during the day and end up frustrated that the agent mode is restricted and returns "I can't complete that for you" or something similar when asking for pretty reasonable actions.

I've been cranking out personal apps with Claude Code in contrast and my brain is exploding with ideas for day job, but this is such an organic space that the speed that corporations move at they are using the cool tool from last year has left me demoing personal work to coworkers and hoping that starts to move the needle on getting better tooling. I understand the governance and privacy concerns for $DAYJOB, and as such every tool needs to get approved by a slow human process.

We also have OpenAI access and I have found myself using that for research more so than copilot as well, maybe we just picked the worst tool because of that MS vertical integration...

johnnyanmac3• 11 months ago

This pattern suggests that a $20-per-month general-purpose tool often outperforms bespoke enterprise systems costing orders of magnitude more

$20/month? Is "mid-sized" different than I imagined, or was this 3-4 years ago? We're already seeing model subscriptions balloon.

I wouldn't be surprised if these approach typical enterprise prices in a few more years.

johnnyanmac3• 11 months ago

AI would offer automation equaling $2.3 trillion in labor value affecting 39 million positions

But

Current automation potential: 2.27% of U.S. labor value

Given the US GDP right now is 27 trillion, I'm not sure if this is really mathing out in my head. Wwe're going to potentially optimize 61 billion dollars of US labor value while displacing some 15% of the American labor force, and return back 2.3 trillion in value? Who's purchasing all this (clearly not the workforce)? Meanwhile, investments in AI as of 2025 is already hitting half of that.

Granted, GDP is an odd indicator to measure on for this situation. But I'm unsure how else we measure "labor value" here.

therealpygon2• 11 months ago

I’m not sure how you got that 2.2% of 18.5 trillion in GDP attributed to labor is 61 billion, so I’d agree that math doesn’t seem accurate.

Additionally, you seemed to have pulled the cherry-picked quote and compared with the “current” impact and ignored the immediately following text on latent automation exposure (partially extracted for quote) that explains how it could have a greater impact that results in their 2.3t/39m estimate numbers. Seems odd to find those numbers in the report but not read the rest of the same section.

chriskanan3• 11 months ago

That's my assessment of the report as well.... really, some news truly is "fake" where they are pushing a narrative that they think will drive clicks and eyeballs, and the media is severely misrepresenting what is in this report.

The failure is not AI, but that a lot of existing employees are not adopting the tools or at least not adopting the tools provided by their company. The "Shadow AI economy" they discuss is a real issue: People are just using their personal subscriptions to LLMs rather than internal company offerings. My university made an enterprise version of ChatGPT available to all students, faculty, and staff so that it can be used with data that should not be used with cloud-based LLMs, but it lacks a lot of features and has many limitations compared to, for example, GPT-5. So, adoption and retention of users of that system is relatively low, which is almost surely due to its limitations compared to cloud-based options. Most use-cases don't necessarily involve data that would be illegal to use with a cloud-based system.

franktankbank• 11 months ago

Absolutely, its a failure of the workers.

hakfoo• 11 months ago

My team has been chewed out for "just because it didn't work once, you need to keep trying it." That feels, to be blunt, almost religious. Claude didn't bless you because you didn't pray often enough and weren't devout enough.

Maybe we need to not just say "people aren't adopting it" but actually listen to why.

AI is a new tool with a learning curve. But that means it's a luxury choice-- we can spend our days learning the new tool, trying out toy problems, building a workflow, or we can continue to use existing tools to deliver the work we already promised.

It's also a tool with an absolutely abysmal learning model right now. Think of the first time you picked up some heavy-duty commercial software (Visual Studio, Lotus 1-2-3, AutoCAD, whatever). Yes, it's complex. But for those programs, there were reliable resources and clear pathways to learn it. So much of the current AI trend seems to be "just keep rewording the prompt and asking it to think really hard and add more descriptive context, and eventually magic happens." This doesn't provide a clear path to mastery, or even solid feedback so people can correct and improve their process. This isn't programming. It's pleading with a capricious deity. Frustration is understandable.

If I have to use AI, I find I prefer the Cursor experience of "smarter autocomplete" than the Claude experience of prompting and negotiation. It doesn't have the "special teams" problem of having to switch to an entirely different skill set and workflow in the middle of the task, and it avoids dumping 2000 line diffs so you aren't railroaded into accepting something that doesn't really match your vision/style/standards.

What would I want to see in a prompt-based AI product? You'd have much more documented, formal and deterministic behaviour. Less friendly chat and more explicit debugging of what was generated and why. In the end, I guess we'd be reinventing one of those 1990s "Rapid Application Development" environments that largely glues together pre-made components and templates, except now it burns an entire rainforest to build one React SPA. Has anyone thought about putting a chat-box front end around Visual Basic?

bishnu• 11 months ago

The illustrative example to me is the construction of railroads at the beginning of the Industrial revolution. Unquestionably useful, unquestionably important there were nevertheless at least 3 speculative financial bubbles centered on railway construction.

paul7986• 11 months ago

Present day AI (chatGPT 5) is not a complete graphic/web designer or front-end developer. My job and skills are safe from it as it can't...

- Make edits to the solid looking logos & web designs it spits out. Instead it creates brand new logos and designs ..not what i asked it to do!

- Front end code it doesn't spit a zip file with all the images. It does speed up my design/development (I use code to design) process where I use to design/develop in the browser using a bootstrap template.

Maybe it will finally figure it out and or maybe its just the Wizard of Oz .. a facade that grabs designs off the web and mixes some up but can never make edits.

mwkaufma• 11 months ago

Levels of cope in this thread many previously thought impossible.

ozgung• 11 months ago

So, on August 21, 2025, everyone in the world suddenly decided that AI was a bubble, no matter what day had said the day before. And they decided to start a PR campaign to announce this to the general public. CEOs were overselling, now they are underselling (their job is to sell).

What's going on? I find all of these pretty sus.

chriskanan3• 11 months ago

Where is the actual paper that makes these claims? I'm seeing this story repeated all over today, but the link doesn't actually seem to go to the study.

I am not going to trust it without actually going over the paper.

Even then, if it isn't peer-reviewed and properly vetted, I still wouldn't necessarily trust it. The MIT study on AI's impact on scientific discovery that made a big splash a year ago was fraudulent even though it was peer reviewed (so I'd really like to know about the veracity of the data): https://www.ndtv.com/science/mit-retracts-popular-study-clai...

Workaccount22• 11 months ago

An alternative headline is "90% of employees report using LLMs regularly"

The story is a "Pick your narrative" one.

https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Bus...

yalogin• 11 months ago

This is looking more like an enterprise cycle instead of a consumer one. I may be a bit jaded or over simplifying things, but I see this AI cycle as redoing enterprise workflows using AI. Not all of them will benefit from it and even then the incremental benefit or ROI from switching to an genAI automation is being questioned here.

Aperocky• 11 months ago

If there are zero return, then something is not being done right.

AI had led to significant operational improvements. The speed of root causing has increased by multiple times and the time spent head-desking or chasing leads have reduced significantly - even if the AI is wrong the first few times, it is recognizable that it would sometime be a rabbithole that I myself were likely to spend time - a lot longer time in.

It obviously cannot do these things by itself because it can arrive at the wrong conclusion all the time and be pretty stubborn about it, but at the same time to say there are no benefit is like throwing a bike away because you fell riding it the first time and go back to walking.

czhu12• 11 months ago

As much as I also want to jump on the ai bubble bandwagon (who doesn’t love a good bit of pessimism), I’m still mind blown daily by how these models operate.

I recently ported over a fastapi app to Django with Claude code and it was at least twice as fast as I probably would’ve been able to do myself, and I only had to somewhat pay attention. What would’ve been a pretty intense few days turned into about 2 hours of mindless work while tens of thousands of lines were ported over, tested, and refactored

Workaccount22• 11 months ago

How unfortunate would it be if people actually read the report

https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Bus...

While only 40% of companies say they purchased an official LLM subscription, workers from over 90% of the companies we surveyed reported regular use of personal AI tools for work tasks. In fact, almost every single person used an LLM in some form for their work. In many cases, shadow AI users reported using LLMs multiples times a day every day of their weekly workload through personal tools, while their companies' official AI initiatives remained stalled in pilot phase.

Corporate initiatives are failing, but people are using LLMs like crazy at work. This story is not the bombshell it's made out to be, in fact it could even go in the other direction.

1vuio0pswjnm7• 11 months ago

Solution looking for problem

kin• 11 months ago

I'm sick and tired of seeing similar headlines and articles that reference the MIT report. It is incredible lazy journalism and borderline click bait.

The MIT report was created by project NANDA, a very pro-AI group. Read about them here: https://nanda.media.mit.edu/

The original Fortune article here: https://fortune.com/2025/08/18/mit-report-95-percent-generat... cites specifically generative AI pilot programs.

From the article: "But for 95% of companies in the dataset, generative AI implementation is falling short. The core issue? Not the quality of the AI models, but the “learning gap” for both tools and organizations. While executives often blame regulation or model performance, MIT’s research points to flawed enterprise integration."

You could have a pile of cash and 95% of companies will fail to see zero return because they don't know how to pick up the money.

Havoc• 11 months ago

What I found most interesting about the stat is how media spun it. At a glance they're similar...but the closer you look the more you notice writers are playing it fast & loose:

only 5 percent of custom enterprise AI tools reach production
95% of Enterprise AI Pilots Fail to Boost Revenues
Why are 95% of GenAI pilot projects failing?
95% of Companies See 'Zero Return'
5% of integrated AI pilots are extracting millions in value
95% of generative AI implementations in enterprise 'have no measurable impact on P&L'

IT4MD• 11 months ago

Happy to hear this.

95% of Companies See 'Zero Return' on $30B Generative AI Spend

Related Stories