nr378

The frontier LLM labs run on a huge fixed cost and very low marginal cost.

Imagine that you want to buy a few B300s to run GLM 5.2 and rent the service out to other people. How could this business be viable and sustainable in the first place?

My understanding is the frontier labs have huge fixed costs and relatively low marginal costs because they have to bear the cost of training the model/R&D, and then amortise that cost over their userbase.

By contrast, if I buy a few B300s and run GLM5.2 and rent the service out to other people, I can be profitable at a comparatively very small scale because I got the model for free.

Arguably Google is both (with GCP and Gemini).

Claude Teams and Claude Enterprise are 2 distinct plans. Simon is right that Enterprise seats have no included usage (and so all usage is charged at API billing rates), whereas Teams seats do.

Yep I think you can reasonably argue that immutability + strong conventions are the most important dimensions (as opposed to FP vs. OOP, as much as I like FP and dislike OOP):

Immutable by convention + Strong conventions: 91.3% - Elixir 97.5%, Kotlin 90.5%, Racket 88.9%, C# 88.4%

Immutable by convention + Fragmented: 78.4% - Scala 78.4% (n=1)

Mutable + Strong conventions: 77.5% - Ruby 81.0%, Swift 78.5%, Julia 78.5%, Dart 78.0%, Go 71.7%

Mutable + Fragmented: 67.9% - Java 80.9%, R 75.8%, C++ 75.8%, Shell 72.9%, Python 65.3%, Perl 64.5%, TS 61.3%, JS 60.9%, PHP 53.8%

(my grouping is somewhat subjective)

The data doesn't well support the claim that FP is best. Elixir tops the table at 97.5%, but C# (88.4%) is OOP and scores almost identically to Racket (88.9%), and Ruby (81.0%) and Java (80.9%) both outscore Scala (78.4%), which is explicitly functional. If FP were the driver, Scala should beat those languages, but it doesn't.

It's tempting to argue that a more constrained language helps, but Rust (62.8%) vs Elixir (97.5%) is an interesting data point here. Both are highly constrained, but in different directions. Elixir's constraints narrow the solution space because you can't mutate, you can't use loops, and you must pattern match, so every constraint eliminates options and funnels you toward fewer valid solutions that the LLM has to search through. Rust adds another constraint that must independently be satisfied on top of solving the actual problem, where the borrow checker doesn't eliminate approaches but adds a second axis of correctness the LLM has to get right simultaneously.

Overall, it seems like languages with strong conventions and ecosystems that narrow the solution space beat languages where there's a thousand ways to do something. Elixir has one build tool, one formatter, one way to do things. C#, Kotlin, and Java have strong ceremony and convention that effectively narrow how you write a program. Meanwhile JS, Python, PHP, and Perl offer endless choices, fragmented ecosystems, and rapidly shifting idioms, and they cluster at the bottom of the table.

3. Storing it the way this article presents makes it usable for agents, but not humans. Whereas the point of knowledge graph, ontology, etc is to create the same layer for both humans and AI to interact with

If storing it this way makes it usable for agents, then why don't humans just use agents when they need to interact with it?

Dario has made a specific cohort argument here. His numbers (from various interviews) are: you train a model in 2023 for $100M, deploy it, and it earns $200M over its lifetime. Meanwhile you train the 2024 model for $1B, which goes on to earn $2B. Each vintage returns 2x on its training cost.

However, the GAAP P&L tells the opposite story. You book $200M revenue in the same year you spend $1B training the next model, so you report an $800M loss. Next year you book $2B against $10B in training spend, reporting an $8B loss. The business looks like it's dying when every individual model generation actually generates a healthy profit.

That's actually Dario's answer to your depreciation question. If each cohort earns back its training cost within its natural lifespan (however short that lifespan is), the depreciation schedule is already baked in. The model doesn't need to live forever, it just needs to return more than it cost before the next one replaces it. Whether that's actually happening at Anthropic is a different question, and one we can't answer without audited financials, but it's the claim Dario makes (and seems entirely reasonable from a distance).

Based on the docs and API surface, I think the filesystem abstraction is probably copy-on-mount backed by object storage.

I suspect it works as follows: when a task starts, filesystem contents sync down from S3/R2/GCS to a local directory, which gets bind-mounted into the container. The agent reads and writes normally - no FUSE, no network round-trips per file op. On task completion or explicit sync, changes flush back to object storage. The presigned URL support for upload/download is the giveaway that object storage is the source of truth.

This makes way more sense than FUSE for agent workloads. Agents do thousands of small reads (find, grep, git status) that would each be a network call with FUSE. With copy-on-mount it's all local disk speed after initial sync.

Cross-task sharing falls out naturally - two tasks mounting the same filesystem ID just means two containers syncing from the same S3 prefix. Probably last-write-wins rather than distributed locking, which is fine since agents rarely have concurrent writes to the same file.

Oh that's quite a nice idea - agentic context management (riffing on agentic memory management).

There's some challenges around the LLM having enough output tokens to easily specify what it wants its next input tokens to be, but "snips" should be able to be expressed concisely (i.e. the next input should include everything sent previously except the chunk that starts XXX and ends YYY). The upside is tighter context, the downside is it'll bust the prompt cache (perhaps the optimal trade-off is to batch the snips).

Nice work.

It strikes me there's more low hanging fruit to pluck re. context window management. Backtracking strikes me as another promising direction to avoid context bloat and compaction (i.e. when a model takes a few attempts to do the right thing, once it's done the right thing, prune the failed attempts out of the context).

Here's a concrete example of what composition looks like in practice.

Say your team has an internal `infractl` CLI for managing your deploy infrastructure. No LLM has ever seen it in training data. You add `--mtp-describe` (one function call with any of the SDKs), then open Claude Code and type:

  > !mtpcli
  > How do I use infractl?

  > Write a crontab entry that posts unhealthy pods to the #ops Slack channel every 5 minutes

  */5 * * * * infractl pods list --cluster prod --unhealthy --json \
    | mtpcli wrap --url "https://slack-mcp.example.com/v1/mcp" \
        postMessage -- --channel "#ops" --text "$(jq -r '.[] | .name')"

Looks like another Claude App/Cowork-type competitor with slightly different tradeoffs (Cowork just calls Claude Code in a VM, this just calls Codex CLI with OS sandboxing).

Here's the Codex tech stack in case anyone was interested like me.

Framework: Electron 40.0.0

Frontend:

- React 19.2.0

- Jotai (state management)

- TanStack React Form

- Vite (bundler)

- TypeScript

Backend/Main Process:

- Node.js

- better-sqlite3 (local database)

- node-pty (terminal emulation)

- Zod (validation)

- Immer (immutable state)

Build & Dev:

- pnpm (package manager)

- Electron Forge

- Vitest (testing)

- ESLint + Prettier

Native/macOS:

- Sparkle (auto-updates)

- Squirrel (installer)

- electron-liquid-glass (macOS vibrancy effects)

- Sentry (error tracking)

100 SWEs running Claude Code generating 400mn tokens/mo = 400mn * $25/mn = $10,000/mo of revenue for Anthropic

10 SWEs running Claude Code generating 400mn tokens/mo = 400mn * $25/mn = $10,000/mo of revenue for Anthropic

If AI can make 10 engineers a productive as 100, then AI companies bank at least the same revenue.

The simple evidence for this is that everyone who has invested the same resources in AI has produced roughly the same result.

I think this conflates together a lot of different types of AI investment - the application layer vs the model layer vs the cloud layer vs the chip layer.

It's entirely possible that it's hard to generate an economic profit at the model layer, but that doesn't mean that there can't be great returns from the other layers (and a lot of VC money is focused on the application layer).

In python, ..., calling shell commands or other OS processes requires fiddling with the subprocess module, writing ad-hoc streaming loops, etc - don't even start with piping several commands together.

You inspired me to throw something simpler together - https://pypi.org/project/shell-pilot/

I suspect that the asymptotic price of consumer facing LLMs will be 0, much like Search - although just like Search, monetisation potential from ads will be high (perhaps even higher given the ability to truly integrate ads into the result itself using LLMs).

It potentially looks like Google and OpenAI will take this new market.

You can hide fields in Python with a little bit of gymnastics:

  class EncapsulatedCounter:
      def __init__(self, initial_value):
          _count = initial_value

          def increment():
              nonlocal _count
              _count += 1
              return _count

          self.increment = increment


  counter = EncapsulatedCounter(100)
  new_value = counter.increment()
  print(f"New value is: {new_value}")

The "bubble" will burst if it turns out that the demand for OpenAI/Anthropic's services is primarily driven by investment-dollars (i.e. VC money) rather than revenue-dollars.

If OpenAI/Anthropic's customers are themselves generating real revenue with reasonable margins, then it's not a bubble at all.

Today, we’re pleased to announce a new round of financing: our Series D of $2.3B at a $29.3B post-money valuation.

We’ve also crossed $1B in annualized revenue

A 30x revenue multiple on (presumably) relatively low-margin revenue is certainly punchy.

One wonders how much of their $1bn of ARR they're paying straight through to Claude/Anthropic.

Palantir is trading at 80x revenue (NTM), whereas Nvidia is only trading at 19x revenue (NTM).

Both companies are growing revenue at a similar rate (~50% YoY), and Nvidia has a higher net margin, however Palantir's share price is up 717% over 18 months, whereas Nvidia is only up 124%.

It's hard to argue Palantir's valuation reflects its fundamentals, even if you believe Palantir will be benefit from lucrative government contracts for years to come.

Buying companies at 80x revenue has not historically been a great way to make money, unless they're growing revenue at several hundred percent per year.

Remind me again why we need investors to fund bad ideas?

Early stage investors generally fund a portfolio of multiple ideas, where each idea faces great uncertainty - some investments will do tremendously well, some won't. Invstors don't need every investment to do well due to the assymmetry of outcomes (a bad investment can at worst go down 100%, a good investment can go up 10,000%, paying off many bad investments).

The whole premise of western capitalism is that investors can better align with the needs of the society and the current technological reality.

This is not the premise of capitalism, it's the justification for it - it's generally believed that capitalism leads to better outcomes over time than communism, but that doesn't mean capitalism has 0 wasteage or results in 0 bad decisions.

money is just a competitive resource we use to outbid each other for _real_ things

That's true, but the thing that's lost is the economic/productive capacity that the money was spent on, that could have been used for other (better) purposes.

For example, if I raise $100mn in a frothy market, and spend it on employing 100 Engineers on $1mn/yr salaries for 1 year before ultimately going bankrupt, it's true that the money doesn't disappear, as it was simply transferred from the VCs to the Engineers, but what's spent/consumed is the Engineers' time. Society can never get those 100 person-years back, and the VCs have to write their capital investment to 0.

The other comments are separately true - money is created by bank borrowing and destroyed by loans being repaid or going bad. Periods of speculation often result in increasing leverage (e.g. borrowing to buy stocks/houses), which does result in the destruction of money when it unwinds (as well as damage to bank's balance sheets, which can become problematic when it happens at a large enough scale - see 2008).

Interest rates set the exchange rate between future cashflows (i.e. assets) and cash today. Lower interest rates mean higher asset values, higher interest rates mean lower asset values. Higher asset values generally disproportionately benefit those that own assets (wealthy people) over those that don't (average people).

Of course, this is just one way that interest rates affect the economy, and it's important to bear in mind that lower interest rates can also stimulate investment which help to create jobs for average people as well.

People are sending you shit for free.

They aren't sending you stuff for free, they're sending you stuff in exchange for the right to receive future cashflows back from you (i.e. they're sending you stuff in exchange for taking ownership of your financial assets). This isn't necessarily a bad trade for you, but it's far from free.

No, the deposit is a liability with a matching asset that comes with it. The asset is needed to balance the asset. They certainly don't lend it out to other customers

When a customer (customer A) deposits $10 of cash at a bank, the bank has a new $10 asset (the cash), and a new $10 liability (the deposit it owes to the customer).

If another customer (customer B) then comes in and asks for some cash in the form of a loan, the bank can loan that customer the $10 cash, at which point the bank goes from having a $10 asset in the form of cash, to a $10 asset in the form of an outstanding loan.

This new asset is less liquid than the cash, but the bank's balance sheet still balances.

which always happens through new money creation

You are correct here - bank lending is the process through which the vast majority of money is created. Before the loan, customer A thinks they have $10. After the loan, customer A and customer B both think they have $10. In this sense, $10 of new money is created.

Interest rates moderate the rate at which banks lend and the rate at which money is created, and the Central Bank acts as one of the most important price-setters in the economy.

They do buy short duration government bonds with it because they're not stupid and can get a better return than on reserves, and bonds are acceptably liquid. Nothing they do can be construed as investment. Savings really are savings and are effectively removed from the economy.

Even if you believe that banks only buy short duration government bonds (which is provably not the case[1]), this is still a form of lending, as it effectively finances government borrowing, and the Government can spend their borrowed money as they see fit (such as for building infrastructure).

[1] See JP Morgan's consolidated balance sheet as an example, page 206 - https://www.jpmorganchase.com/content/dam/jpmc/jpmorgan-chas...

A bank deposit is both an asset and a liability to the bank - a customer depositing money in a bank expands both sides of the bank's balance sheet.

Banks can and do then lend out deposited money to other customers, who can then spend it in the economy as they see fit.

The Federal Reserve targeting "price stability" literally does mean that they target prices increasing exponentially by an average of 2% per year[1]. The mathematical form is P(t) = P₀(1.02)ᵗ, where P₀ is the initial price level and t is time in years.

[1] https://www.federalreserve.gov/economy-at-a-glance-inflation...

Like the U.S. Government prints it and spends it, so its not clear to me that there is a balance sheet of cash being someone elses debt.

The US government prints it (“quantitative easing") by creating new money and buying its own debt. In this sense it’s still correct to say this Government printed money is backed by debt.

Nb. That this is only a small proportion of the overall money supply though. Commercial bank deposits (created through bank lending) represent the vast majority.

1. Stock buybacks are always anti-dilutive regardless of the value of the stock, and so investors always benefit as future earnings are concentrated amongst fewer shares (all other things being equal). It's true that a lower price/value increases the magnitude of this effect, but the effect is always present.

2. Zero income stocks can still do buybacks when they're returning capital raised from shareholders to shareholders (this is essentially just partially reversing a funding round, which again undoes some of the dilution).

3. Companies can be deemed to be significantly valuable without paying any dividends. Companies that have a long history of paying dividends but then suddenly stop tend to be in distress, which is then reflected in their share price.

Businesses are always greedy. The fact corporate profits are up is an effect of inflation, not a cause.

A good essay on the topic is: https://www.economicforces.xyz/p/greedflation-lets-try-this-...