AI Does $58k of Engineering Work for $15, but Enterprises Can't Use It

https://softwarecentury.com/blog/architecture

by sdan • 1 year ago

5 5 1 year ago

Meanwhile, McDonald's just killed their IBM Watson drive-thru because it kept adding bacon to ice cream orders. The AI worked fine. The architecture was broken.

It is killing me that this statement isn't followed-up on. I can't find any more technical details about this online.

Come to think of it, the whole article is like this. Broad statements without any details to support its claims:

A team I know needed a code review tool last week. Instead of evaluating vendors, they described what they wanted to Claude. Four hours later they had a custom application integrated with their workflow. Cost: $3.40.

That's cool, I'm glad they could vibecode something in a week. What does it actually do? How well will it work next month, next year?

In March 2025, Claude 3.5 completed what amounted to $58,000 of freelance SWE tasks for about $15 in API credits. Real engineering work, benchmarked against OpenAI's SWE-Lancer.

Is there a link to a source on this? It'd be cool to read more.

This entire blog post, like the one before it, feels like a big ad for ... Prava, I think? I have no idea what they do. (And the fact that they continually cite DOGE isn't inspiring confidence.)

pavel_lishin2• 1 year ago

Unrelated to my other comment, but the double-click-to-switch-color-themes & the constant animated highlights are driving me insane. It's like trying to read a book while someone else is trying to flip pages and add their own highlighting.

stego-tech• 1 year ago

I disagree with the pie-in-the-sky sales pitch (“post-code”? Really?), but the gist is something I’ve been fighting in my career for fifteen years, now.

The cold truth is your leadership team has just been bolting on new products and subscriptions for decades. They have no vision, no architecture expertise, no understanding of how the work gets done. This means they’re easy prey for the kind of marketing slop AI, blockchain, and SaaS offer (savings! Modern! Automation! Layoffs!).

Say it with me, now: your infrastructure and architecture are the problem. The article - marketing, DOGE-boosting, and sales pitch aside - is technically correct. Your enterprise, by not having competent architects and engineers in positions of leadership, is letting millions of dollars of your cash stores get devoured by unnecessary product spend. You don’t need AI, you need discipline and vision to build modern infrastructure for a modern organization, and the will to suffer political blowback as you cut products that don’t deliver meaningful returns to the company or aren’t compatible with said open architecture.

desktopninja• 1 year ago

I was PIP'd out within 48hrs after vocalizing this sentiment. Now they are selling an enslopified AI centric product. Employee moral evaporated and yet the powers be deniably (sp) claim everything is fine. Fun times.

taylodl• 1 year ago

Big talk and lots of hand-waving.

I call BS.

Those integrations? That is the problem in the enterprise. I don't see how you can wave your magic AI wand and make it all go away. In fact, one of the limitations I see in AI is it doesn't understand those integrations. That's the local training I want to augment my AI system with: I need a service catalog and a data catalog, with services defined in terms of data nomenclature. Once the AI knows that for my particular enterprise, then it can go to town and start producing useful code. Then my prompt can be "I need to get <such and such data> and cross-correlate with <some other data> and filter out based on parameterized criteria and send the result to <blah>." Then the AI has everything it needs to actually generate the needed code.

THAT is how you revolutionize enterprise development.

AI Does $58k of Engineering Work for $15, but Enterprises Can't Use It

Related Stories