maxnevermind

I'm surprised it is that low. Are not all top AI labs "cheating" and workaround LLMs's low sample efficiency by hiring people to generate more data points - similar problems with answers, so they can train models on those and improve scores? A good benchmark for general intelligence probably should be a complete black box, no sample data given/leaked at all.

As Steve Eisman said: Mag7's businesses becomes capital intensive and they also seems to lack a strong moat, that will make investors more reluctant to buy keep buying their stock.

synthetic verifiable traces

What does it mean, Is it like when somebody used some coding agent to develop a feature and later input prompts and a resulting PR can be used for training by a presumption that final PR was a correct implementation of a prompt?

everything is now attached to a watercooled block

Does it increases manufacturing and operational cost of such racks?

Are logs somehow used for the purpose of training their own models or something else?

Don't big tech ask meaningless LeetCode problems because it serves as a proxy for IQ?

these types of training data might be key to teaching models to reason beyond what they could gather from static data.

I was under impression that every time LLMs try to be truly novel and they need to assume things in the area where they didn't have enough data points that there were trained on, results are not good, has that changed?

That is informative, I was suspecting that is how models improve their performance on some convoluted "non-googlabe" benchmarks like SimpleBench, that is how, they just got the taste of those those questions from publicly available samples and then hired people to generate similar questions and provide answers for them.

I wonder if extracting those static reasoning chains make sense given a Rich Sutton's "The Bitter Lesson" and Geoffrey Hinton's "People should stop training radiologists now.". I guess until participants make money they won't stop, not sure if they do, so far it is more about expectation of profitability as I understand.

Given the enormous cost of training, would it be worth training new models then?

I'm not an expert, but is this even an option? I mean models must be refreshed with latest knowldge base periodically even without algo/design improvements, otherwise the lag become too noticeable and it will hurt users and their use-case.

I guess that is good, competition should lower the margins with the time as it doesn't seem like any of AI labs have a particularly strong moat.

The reported 2025 figures include $7.5 billion in cost of revenue, $19.18 billion in research and development, $5.73 billion in sales and marketing, and $1.57 billion in general and administrative expense

Does training of new models go into RnD or cost? And subscription plans' subsidies, are those cost or sales and marketing?

Plot twist: we already are through the usage of AI lab's API.

"Now you see why, when I meet a founder, the first thing I ask about is their growth rate."

This obsession with growth instead of progress or value rubs me the wrong way, given the trend for enshittification of services sooner or later, also remind me of a video I watched yesterday when a founder gives examples of private equity people trying to force growth no matter what: https://www.youtube.com/watch?v=k4vNIsVY-0Y&t=412s

I thought usually founders try to pivot till they run out of money. I wonder if that is good or bad for a serial entrepreneurs if they decide to shut it down instead of pivoting?

I was also wondering if it will result in Helium shortage and even higher prices the next few years, then/if it will be all over, new fabs might also come online and it will be the opposite - a glut. I believe that follows the historic pattern of boom and bust cycles of chip production.

What is this, Series for ants? It barely covers Andrej's sign-on bonus.

What type of projects you work on, in particular how rich it is in novelty, non-googlable data points and non-trivial project-specific deviations from industry standards?

I wonder if it is reasonable to assume the propagation of shortages further. At first it was GPUs, then RAM, then what?

Customers are finding value.

Where I can find confirmation of that in public sources?

Andrej has decided to become a billionaire. Anthropic keeps preparing for the IPO. I wish they IPO soon, let everyone see how the earnings look like.

Defining exactly what the product is supposed to do is the hard part, writing code is the easy part.

There is a massive difference between a spec, which defines what the product should do, and code, which defines exactly how it should do it.

He states: The difficult part is figuring out the details so LLM doesn't save much time. You state: If LLM is able to correctly assume the details that saves you a lot of time.

Case 1: Part of the spec describe some basic feature based on a popular framework and industry standards, everything is trivial. You are right, he is wrong.

Case 2: Part of the spec describe some niche feature and/or uses some not popular framework and/or require deviation from industry standards and/or cutting edge performance/latency requirements and/or uses a bunch of proprietary non-googlable data. You are wrong, he is right.

The more senior engineer are the less time they spend on case 1, those are easy, they don't spend much time on it, it is the 2nd which is much more time consuming.

I'm also confused with that as well as the moral of the story as the whole. I get the sentiment but what is the lesson here, leave production capacity keep going as well as a "hiring pipeline" and just stock pile the output forever? Also, given article's take on the current situation of AI assisted coding, it seems to suggest that we need to apply that same logic to other industries too, so just don't let any industry/practice die and keep it alive? I would appreciate some inputs into some sort of actual solution or at least ideas of the solution but that is not present in the article.

Galactic Empire has agreed to acquire a local lemonade stand in exchange for 10 death stars.

Anthropic will kick and scream as those are often distilled from their latest models and is cutting into their margin. Though it is not like their hands are clean neither, it is just a different type of stealing, an approved one :-)

Any publicly available evidence to back that up? There have been post-exit blog posts from OpenAI employees on HN before and it did sound like the only black magic they use there is that many employees work 16 hrs a day during launch of new features. I know that some current Claude Code devs are doing interviews where they claim that they use Claude Code extensively but they clearly have a conflict of interest while they are still employed at Anthropic, so it would be like asking a barber if you need a haircut.

I wonder if this ends up like Tesla - China copies it and makes it cheaper - GG if not protected by huge tariffs/bans. It seems like US these days is just a testing ground for new tech that later scaled further and optimized in China. Are there any hard moats protecting SpaceX from that?

Does anybody know what their Enterprise offering actually offers? I read through it but still don't get it.

Yeap, I recently came to realization that is useful to think about LLMs as assumption engines. They have trillions of those and fill the gaps when they see the need. As I understand, assumptions are supposedly based on industry standards, If those deviate from what you are trying to build then you might start having problems, like when you try to implement a solution which is not "googlable", LLM will try to assume some standard way to do it and will keep pushing it, then you have to provide more context, but if you have to spend too much time on providing the context, then you might not save that much time in the end.

verified with human use

Quality of that verification matters, people who might use AI tend to cut corners. This does not completely solve problem with AI slop imo and solution quality. You ask Claude Code to go and implement a new feature in a complex code base, it will, the code might even work, but implementation might have subtle issues and might be missing the broader vision of the repo.

I was trying to find some more context on this but all I could find is that Rob Pike seems to care a lot about efficiency of software/hardware and against bloat which is expressed in his work on Golang and in related talks about it.