HN user
Writing code helps me think.
If the tools are making you worse, don't use the tools.
The faster you realize this will never pop, the faster you realize that you too can make money in the biggest gold rush in human history.
Gold worth tens of billions of today's US dollars was recovered, which led to great wealth for a few, though many who participated in the California gold rush earned little more than they had started with.
The human and environmental costs of the Gold Rush were substantial. Native Americans, dependent on traditional hunting, gathering and agriculture, became the victims of starvation and disease, as gravel, silt and toxic chemicals from prospecting operations killed fish and destroyed habitats.[0]
"When it comes to a field I'm not an expert in, AI is a great tool."
Every time.
No. If anything I’m understating the risk.
What you're building should be illegal. Educators who use these tools on children should face jail time.
It's hard to overstate the harm of the system you are building. Please understand, there is no way this idea is salvageable or any tweaks or safeguards can make it safe. If you've actually tested this on real children you've already risked extreme harm.
Please, stop.
I don’t think “I know plenty of people who commit fraud anyway so who cares if it’s right” is the ringing endorsement of AI you think it is.
I think close to 100% of new ambitious projects are going to leverage AI at least to some degree.
Once the free money dries up that number will rapidly tend towards 0%.
So how much AI usage does it make it an “AI rewrite”?
Any amount.
That "nearly" is doing an awful lot of heavy lifting. It doesn't matter if your AI model is 99% or 99.99% accurate. For a tax return it has to be perfect every time or someone is at best getting a fine or at worst going to prison.
Sure, human error happens too, but humans take accountability. That's why accountants are a regulated profession. Until an AI company CEO is willing to go prison if the output of their model is wrong, these tools are worthless.
But don't take my word for it, head on over to Toot's own terms of service https://toot-books.pages.dev/terms#ai-not-advice
Toot uses automated and AI systems to generate classifications and reconciliation suggestions. Output may be incomplete or wrong and must be reviewed by you.
Toot is a software tool. It does not provide accounting, tax, legal, audit, or financial advice, and nothing it produces is a substitute for a qualified accountant or tax adviser. You are responsible for checking Output before approving it or relying on it, and for any decision you make based on it. To the extent permitted by law, we are not responsible for outcomes arising from automated Output you approve without review.
Ironically it is you who is most at risk from the direction this is taking.
When this house of cards collapses, AI research dries up, and companies pivot to the next hype cycle there will be a generation of people left with atrophied skills and lingering addiction and psychosis. The most flexible will bounce back just fine, but many will never recover from this damage.
Sincerely, I hope you're in the former category.
Why do all arguments from AI boosters boil down to this same cycle:
A new model is released, AI fans hail it as huge shift in whatever metrics the AI vendor has gamed this time, and all criticism is shrugged off as "not up to date" and met with "try the new model!" Then, once level heads actually put the claims to the test and find it wanting, criticism is met with "you're just not using it right, you have to learn how to prompt/context/loop engineer for best results" until the next model comes out and this argument repeats.
It's obvious to more people every day that "AI" has never delivered on what it promised. Until now nobody cared because the costs were cheap and it pays to appear cutting edge. Now the price is rising, and the delayed cost of AI-addiction and broken outputs are starting to sting. Not that employers care about the individual harms of course, but a workforce so deep in collective delusion that they can't see the train coming is only useful when the market is delusional too.
Once this snaps, and it will snap suddenly, companies will be climbing over each other to rip out AI as fast as possible. They won't call it that, of course, you're not going to get a CEO on the news talking about how they made a mistake and it was wrong to invest so much in AI tech. But they'll mean it.
The win scenario is that the crash reduces "AI" use to near zero. Spat out into the graveyard of VC hype like blockchain and metaverse before it. Banished to an eternal unlife of scammers running call centre scams, deepfake porn producers, and the occasional "we made AI safe!" startup trying to reignite the bubble again. While companies with their business on the line clamber to announce that they don't use it.
we thank ChatGPT, Claude, Gemini, and Grok for tirelessly traversing the combinatorial space of this manuscript.
This is a premise so fanciful this might as well start with "assume a goose that lays golden eggs" that's then been extrapolated into 100+ pages of AI generated text.
Tried what? Using LLMs to produce software? I'd say the industry has collectively "tried" to do this for a while now and the results have been an unmitigated disaster.
Yes, we should "give up" on using LLMs, but certainly not for a lack of "trying".
What AI does is the exact polar opposite of "democratizing". We're going from a world where high-quality learning content is plastered all over the internet for free, where open-source projects are desperate for contributors and full of "good first issue" tickets, and where the tools you need to code can be run on any crappy laptop from the past 15 years—to a world where the majority of content about programming is unverified AI slop, where open-source projects are locking down access to protect against an avalanche of drive-by AI garbage, and where you need a $200/month subscription or a $3,000 GPU to run AI models for coding.
They don’t even need to be deterministic checks - a shell script that wraps a `claude -p` - and maybe fetches some online resources to stuff into the context - can do your agent’s legal check for it.
No. It can't. If you think that injecting legal text into the context window and appending "make sure the output complies with this law" will solve your problems you have not understood how an LLM works. The kind of checks you are talking about cannot be codified.
Yes connect the 3rd party company with a vested interest to capture as much of your data as possible with a direct pipe into all your internal sensitive discussions about your product, business, and market positioning in real time.
Has everyone collectively lost their minds? Suggesting anything like this even five years ago would get you laughed out of the room, and actually doing this would be a career ending mistake.
outsourcing their decision making and thinking to AI and not really about using AI itself
I use AI a ton and I'm having more fun every day than I ever did before
With respect, this is what makes me worry.
If someone is a user of AI, can they really tell the difference between "outsourcing" and "using"? I worry that a lot of people will start out well-intentioned and end up completely outsourced before they realise it.
So?
The research on AI is showing again and again that people that use AI are losing their skills not just in the specific task but more generally. This isn't a "change of skills" it's a fundamental reduction in the skills of knowledge workers.
With my newly-acquired superpowers I could knock out the last two pieces in a few days’ work
From the linked post:[0]
I left an employer that is years behind adopting AI to one actively supporting and encouraging it. As of March, in my professional capacity I no longer write code myself. My current situation was unimaginable to me only a year ago. Like it or not, this is the future of software engineering. Turns out I like it, and having tasted the future I don’t want to go back to the old ways.
It's deeply distressing to watch people fall into AI psychosis. Being smart, accomplished, or experienced is no defence.
After the bubble pops and the industry realises the damage these tools can do to people, folks like the author will have to confront that they were taken in by a lie. Many won't be able to confront that.
We're not talking about replacing "writing code", we're talking about replacing your ability to think about the problems critically at all.
None of this requires new technology. It requires treating the database as a defensive layer that assumes the caller might be wrong, might retry, and might not be watching the results.
This is one of those takes that is so close to understanding the problem, and then drawing an insane conclusion.
The problem is that AI agents and the code they output is untrustworthy, buggy, insecure, and lacking in any of the standards the industry has developed over the last 30 years. The solution to this is "don't use AI agents", not "change the rest of the stack to accommodate garbage".
Any company that has become dependant on AI will struggle to survive from here on. By the time many teams realise it'll be too late.
At least it'll make it easy to audit and replace it all in a few years.
I'm going to take your comment at face value, and I'm also going to assume that you're US-based.
You need to take a step back and look at the economic reality of the majority of Americans today. Many live paycheck-to-paycheck, even those with "middle class" incomes. For many a $200 one-off bill is debilitating, yet alone a recurring subscription. If you don't know that, you have a dangerously narrow view of the economy.
The median per capita income in the United States is $37,683/year.[0] Depending on your state, after taxes, that's something like ~$2,600/month. You're asking almost 10% of their post-tax income to this just for the opportunity to create software. With rent, food, and other living expenses many households at that income level simply cannot afford this.
This is the median income. If it's a struggle for someone on this income then it's worse for half of all Americans, and American incomes are higher than most of the rest of the world.
[0] https://en.wikipedia.org/wiki/Per_capita_personal_income_in_...
I don't know what to say to you. More people are coding now with AI than ever coded before. If your argument was true, then that would just mean that there are more elites than ever. Obviously that's not what's happening.
I don't know how I can explain this any more clearly.
If you need AI to create software, and the cost of AI is $200/month, then only people who can afford $200/month can create software.
Costs will increase. The current cost is substituted by investor funding. Sell at a loss to get people hooked on the product and then raise the price to make money, a "high-growth business model" as you say.
The cost to make a competitor to Anthropic or OpenAI is tens or hundreds of billions of dollars upfront. There will be few competitors and minimal market pressure to reduce prices, even if the unit costs of inference are low.
$200/month is already out of reach of the majority of the population. Increases from here means only a small percentage of the richest people can afford it.
I don't know what definition of "elite" you're using but, "technology limited so that only a small percentage of the population can afford it" is... an elite group.
This is fun and all, but I think we've reached the end of the productive discussion to be had and I don't have much more to say. Charitably, we're leaving in completely different realities. I just hope when the bubble pops the fall isn't too hard for you.
Democratizing is defined as "the process of making technology, information, or power accessible, available, or appealing to everyone, rather than just experts or elites."
Your definition only supports my point. The transfer of skill from something you learn to something you pay to do is the exact and complete opposite of your stated definition. It turns the activity from something that requires you to learn it to one that only those that can afford to pay can do.
It is quite literally making this technology, information, and power available to only the elite.
Uhhh. Maybe you don't know any AI investors, but the payout is coming NOW.
What payout? Zero AI companies are profitable. If you're invested in one of these companies you could be a billionaire on paper, but until it's liquid it's meaningless. There's plenty of investors who stand to make a lot of money if these big companies exit, but there's no guarantee that will happen.
The only people making money at the moment are either taking cash salaries from AI labs or speculating on Nvidia stock. Neither of which have much do with the tech itself and everything to do with the hype.
AI has none of these things.
1. As I said before, we've long since reached diminishing returns on models. We simply don't have enough compute or training data left to make them dramatically better.
2. This is only true if it actually pans out, which is still an unknown question.
3. Just... not using it? It has to justify its existence. If it's not of benefit vs. the cost then why bother.
4. The public hates AI. The proliferation of "AI slop" makes people despise the technology wholesale.
On a SWE salary maybe. If the baseline cost of doing business is a $5k GPU you've excluded like a quarter of the US working population immediately.
This is what everyone says when technology democratizes something that was previously reserved for a small number of experts.
What part of renting your ability to do your job is "democratizing"? The current state of AI is the literal opposite. Same for local models that require thousands of dollars of GPUs to run.
Over the past 20 years software engineering has become something that just about anyone can do with little more than a shitty laptop, the time and effort, and an internet connection. How is a world where that ability is rented out to only those that can pay "democratic"?
When the printing press was invented, scribes complained that it would lead to a flood of poorly written, untrustworthy information. And you know what? It did. And nobody cares.
A bad book is just a bad book. If a novel is $10 at the airport and it's complete garbage then I'm out $10 and a couple of hours. As you say, who cares. A bad vibe coded app and you've leaked your email inbox and bank account and you're out way more than $10. The risk profile from AI is way higher.
Same is even more true for businesses. The cost of a cyberattack or a outage is measured in the millions of dollars. It's a simple maths, the cost of the risk of compromise far oughtweights the cost of cheaper upfront software.
You cut out the part where I said it only popped economically, but the technology continued to improve.
The improvement in AI models requires billions of dollars a year in hardware, infrastructure, end energy. Do you think that investors will continue to pour that level of investment into improving AI models for a payout that might only come ten to fifteen years down the road? Once the economic bubble pops, the models we have are the end of the road.