typ

There are some benchmarks I personally trust. But the most reliable indicator other than public benchmarks is the quality of products people are working on, and the model they use for the work. Not a benchmark scoreboard or a one~few shot demo, but the product they have been building for weeks. The reality is, even the diehard advocates who build and sell tooling for open weights, still use proprietary frontier models for their jobs.

Tokens will surely become commodities, but models don't need to.

The point is that electricity becomes a commodity, rather than, say, nuclear reactors or gas turbines also need to be commoditized.

Contrary to most people in the software-minded circle, I am not very excited about running local LLMs. Most of what I have seen involves selling wrappers that call APIs on top of an LLM AI, similar to how traditional SaaS has generated revenue from new technologies. So, this would certainly make those people who are working on that sensitive about the pricing. LLM pricing is treated as a continuous cost in the COGS. They don't really use AI to create much value (consumer surplus) other than replacing the existing boring products, like just building the good old CRUD apps, but faster.

What I find REAL interesting about the potential of LLM AI instead is to create new technology out of it, or revolutionize something old to be order-of-magnitude more efficient. In this regard, the expenses on those tokens are more akin to an upfront CapEx.

Cheaper tokens would surely be nice. But if what we are talking about is, like, solving self-driving, curing cancer, or making air-conditioners 100x efficient, the narrow focus on running a cheaper model in my home so I can write my SaaS apps for my get-rich-quick business looks really unpromising and a waste of time for human civilization in a near-singularity horizon.

Abstraction doesn't necessarily lead to a smaller binary. Much of the bloat in modern software is indeed due to (bad) abstractions.

Unlike the belief that frontier AI is expensive due to a high margin, and going to be expensive if there is no competition. My understanding is that, under certain circumstances (which is most likely true), the price will be driven down just because of profit seeking.

The frontier LLM labs run on a huge fixed cost and very low marginal cost. They need the economies of scale to make sense of the business (an incentive to expand their user base as large as possible). Imagine that you want to buy a few B300s to run GLM 5.2 and rent the service out to other people. How could this business be viable and sustainable in the first place? You need as many customers as possible. If you charge everyone $1000, you find fewer customers who can afford it. It rots the ROA if the servers are not utilized 100% (you would better buy less compute instead).

Also, the marginal cost for onboarding a new customer is low. And it's getting even lower when you have more customers. You wouldn't leave money on the table (especially for your competitors) if you want to maximize your profit.

By this logic, all frontier AI labs are incentivized to lower the price to maximize their customer base, profit, and ROA.

I don't understand how knowledge, either public or private, could get damaged.

Though the income of the individuals and businesses that rely on the expertise of the knowledge would be damaged. Is that what you meant?

Edit: At this stage, the revenue made by the AI labs is almost entirely spent on the formation of fixed capital and opex. The demand is mobilizing physical resources with money. Atoms are relocated and reconfigured into compute racks. But eventually, the created productivity will perhaps make supply-elastic goods extremely cheap and abundant, while the supply-inelastic goods will be worth even more relative to the elastic ones.

After all, money is simply a token for the transmission of physical resources. It doesn't create stuff out of thin air. When new stuff is created, it just makes money cheaper, so that the banks can respond with more money to counteract it. More stuff -> deflation -> more money creation allowed to undo the deflation.

But the "exchange rates" between different goods and services will diverge. That's also why I don't think a direct money transfer like UBI would fix the problem, when it doesn't change the divergence of relative economic values of different goods. Let's say, extremely cheap software and entertainment, but unaffordable healthcare and housing. More money for everyone doesn't make limited resources available. So, what I am leaning into is some sort of Georgist policy. That could hopefully mitigate the price divergence, assuming that AI cannot make every commodity equally abundant.

We cannot always want to capture only the (temporary) winners whenever we see a lucrative business and expect to share a free ride. I'd also assume that most of the revenue these AI labs are making is turned into depreciating fixed capital (hardware) and OPEX at this point.

Why don't we capture Meta and Google as they allegedly take advantage of more publicly available information for profit? Let alone the truly valuable knowledge, like mathematics, has nothing to do with the majority of garbage posts that an average person would "contribute" on social media.

If we really want to tax or nationalize some economic activity, then, in my opinion, the target should be what it takes from society, not what it produces for society. By this logic, we should tax all labs, including those lagging ones, that utilize the public knowledge.

However, if everyone can access the public knowledge without rendering it less useful or reducing its available quantity, there should be no reason to tax it.

Low latency is nice. But it would be more interesting if they could demonstrate the efficiency of energy consumption.

an open training source model

It's always funny to see people tempted to call open-blobs/open-weights, which are literally shareware like WinRAR or Adobe PDF Viewer, open source, and then need to invent a new term for what is actually open source.

A thousand 90 IQ cannot do what a 145 IQ can do.

Similarly, some bosses might believe that they can hire 100 cheap, unmotivated SWEs to replace Linus Torvalds or Fabrice Bellard and achieve something slightly worse. But in certain areas, it doesn't work like that.

It takes billions of investments for infrastructure, and a high-paying, top-notch team for R&D and operations. Not just a bunch of torrents of pirated books. Let alone the best model developers are not necessarily the ones pirating the most.

It's funny that Google, Meta, TikTok, OnlyFans, PornHub, and many other lucrative businesses never open-source their core business software, and people just don't bother about it with that moral standard, simply because we don't need to pay for the service (paid by ads, actually). To me, that is the hypocrisy.

I don't think that learning from textbooks to take an exam and learning from the answers of another student taking the exam are the same.

Joking aside, I also don't believe that maximum access to raw Internet data and its quantity is why some models are doing better than Google. It seems that these SoTA models gain more power from synthetic data and how they discard garbage.

6% compared to the post-2000s is mediocre, especially given the low baseline. Not remarkably better than other high-income democratic countries like Japan and West Germany. Even the US can have ~4% growth at the time.

American business leaders have (had?) an obsession with gross margin and tech "advancedness." They thought they would be the winner as long as they occupied the high-tech sectors in the supply chain. So they discarded the high-volume, low-margin, low-growth, low-tech businesses like assembly lines and outsourced them. But the reality is that the proximity of the assembly lines creates a cost advantage that attracts more upstream suppliers to surround it. Even Intel was seeking to build more fabs in China before being stopped by the US government.

If that were the true secret sauce of the economic success in China, why had it not taken off before the 2000s? Like, they have been that "aligned" and "want the same thing" and "run by engineers" since the 50s, no?

The silicon shield became a slogan that has only been popularized in recent years. The potential crisis of war has been there for more than half a century (even before semiconductors became a thing). The real value proposition of the status quo is the freedom of navigation between the northeastern Asian countries and the SEA (the Strait of Malacca, aka the lifeline of energy imports), and the consequential domino effect of the entire western Pacific.

Also, not sure why everyone forgets about it. People should have learned from the experience of the pandemic that the cutting-edge foundry nodes are not really the crucial ones, as being the bottleneck of industrial infrastructure. A delay of the next-gen iPhone or RTX gaming card isn't that catastrophic. But a shortage of embedded MCUs, which are actually fabricated by mature nodes, could stall the entire industrial base of a country.

I'd bet, on average, the quality of proprietary code is worse than open-source code. There have been decades of accumulated slop generated by human agents with wildly varied skill levels, all vibe-coded by ruthless, incompetent corporate bosses.

reduce energy requirement by 10-50 times

This is only relevant to the compute productivity (how much useful work it can produce), but it's irrelevant to the heat dissipation problem. The energy income is fundamentally limited by the solar facing area (x 1361 W/m^2). So the energy output cannot exceed it, regardless useful signals or just waste heat. Even if we just put a stone there, the equilibrium temperature wouldn't be any better or worse.

That should be better than a sphere. Though I imagine there could be some fancier 3D geometry designs.

Even for a simple sphere, if we give it different surface roughnesses on the sun-facing side and the "night" side, it can have dramatically different emissivity.

Assuming that we place an iron ball (ideal sphere-shaped and thermal conductivity) on the SSO (solar synchronous orbit), how hot can the object be?

Given the solar constant 1361 W/m^2, you can calculate the temperature range based on the emissivity and absorptivity. With the right shape and “color”, the equilibrium temperature can be cooler than most people thought.

I suppose that a space data center powered 100% by solar is no different than this iron ball in principle.

The radiator area is probably not what they need to worry about that much as we thought. When the energy input comes from solar 100%, they just need to optimize the ratio of the sectional area facing the sun over the total surface area of the satellite. If the ratio is low enough, like a fin or cone shaped object, it will be harder to be hot.

Not sure about the effectiveness of a heat pump in this use case.

Heat radiation works better the higher the temperature?

The power output is proportional to T^4 according to the Stefan-Boltzmann law.

It makes sense to target a higher operating temperature, like 375K. At some point, the energy budget would reach an equilibrium. The Earth constantly absorbs solar energy and also dissipates the heat only by radiative cooling. But the equilibrium temperature of the Earth is still kind of cool.

I guess the trick lies in the operating temperature and the geometry of the satellites.

The label 'open source' has become a reputation reaping and marketing vehicle rather than an informative term since the Hugging Face benchmark race started. With the weights only, we cannot actually audit that if a model is a) contaminated by benchmarks, b) built with deliberate biases, or c) trained on copyrighted/privacy data, let alone allowing other vendors to replicate the results. Anyways, people still love free stuff.

Rather than API/ABI stability, I think the problem is the lack of coherence and too many fragile dependencies. Like, why should a component as essential as Systemd have to depend on a non-essential service called d-bus? Which in turn depends on an XML parser lib named libexpat. Just d-bus and libexpat combined takes a few megabytes. Last time I checked, the entire NT kernel, and probably the Linux kernel image as well, has no more than single-digit MBs in size. And by the way, Systemd itself doesn’t use XML for configurations. It has an INI style configuration format.

The difference is that you can customize/debug it or not. You might say that a .EXE can be modified too. But I don't think that's the conventional definition of open source.

I understand that these days, businesses and hobbyists just want to use free LLMs without paying subscriptions for economic motives, that is, either saving money or making money. They don't really care whether the source is truly available or not. They are just end users of a product, not open-source developers by any means.

It seems to come from a Plan 9 C idiom that GCC even has an extension for it.

https://gcc.gnu.org/onlinedocs/gcc-5.3.0/gcc/Unnamed-Fields....

Downstream assembly factories attract component manufacturers because of lower transport costs and shorter delivery times, which can lead to network effects. (Remember why Intel wanted to build more foundries in China a few years ago?) That's the success formula of Shenzhen, for example.

Maybe the README should mention whether reflection (the part that would disable dead-code elimnation) is used? That was a pain point for me when I had to investigate a binary bloat issue due to reflect.

Homes are not a homogeneous commodity. New homes cannot be perfect substitutes due to locations, amenities, access to jobs (commute time), etc. Also, government spending on infrastructure will be capitalized into the neighborhood's land (location) value. You cannot 'undo' this capitalization simply by building more homes.

C# is fine. But last I checked, the AOT compilation generates a bunch of .dll files, which are not suitable for a CLI program like Go's zero dependencies binary.