sfifs

Well OpenAI is offering equity to the US Government (and who knows who else privately) Tim Apple famously refused to bring manufacturing back to the US when the current president asked and play hardball on infosec. While this is a civil case, increasingly judiciary seems to be an extension of the executive. So it will be interesting to see how this plays out.

Materials science fundamentally has a math & computability problem. Rigorous materials simulations at a nanometre scale seem now feasible - so you could model physical properties of small groups of atoms - eg. Modeling molecular reactions at a sub-atomic level. Bulk property simulations at a centi and up scale also work and so you can do first principles based design and use tricks like finite element methods to deal with the underlying stochasticity to a degree. Materials properties however largely arise from features between nano and micro scale - grain boundaries, dislocations etc. These are computationally infeasible today - there are neither engineering solutions, not math tricks that make this tractable and so materials science and engineering becomes a grind of experimentation and metrology.

This is interesting in that it seems to be making the grind more efficient. I think the true breakthrough will likely be proper scale quantum computing to make the first principles design feasible

HaHa I was thinking just the same :-)

For those unfamiliar with the reference, in Hitchikers guide to the galaxy series, Mice are projections of hyper intelligent extra terrestrial beings from a higher dimension who commission the creation of planet earth complete with fossils as a giant organic computer

Used both. DeepSeek-4 Flash Q2 - last 6 layers Q4 quant with DwarfStar which just about fits in 128Gb is definitely superior IMO - my contexts tend to run typically 50-100k. Throughput tends to be about 12-13k tok/sec - just about acceptable.

Is this advice coming from personal experience of significant diabetes and weight reversal or just online reading informed opinion? Physiology & psychology are quite complex and individualized and "just do this.." doesn't quite work at any sustainable level when people have jobs, families, mortgage, sick children & sick parents to also manage at the same time.

It's quite interesting what incentives and culture being a privately held company creates.

You do see this working in many places. Singapore is probably the best example with an explicit scholar programme and competitive pay in much of the upper rungs of the government. This comes with its own problem culturally but does result in very well thought through programs and infra that actually helps people.

Even in the top echelon of the Indian government (Indian Administrative Services), a similar culture exists and has interestingly been strengthend as the power has fragmented across political parties. I personally know several of my top B School/Engineering college mates who joined and any of them would do very well in top management in private industry. You can see the result in the rapidity with which India is pulling people out of poverty and modernizing. Certainly alower than China, but pretty amazing seei living outside for a decade now - many urban services are already far superior in quality to what you get in Europe, US or even Singapore. The challenge in India is the bulk of the bureaucracy below them isn't held to the same standard.In India it's still a prestige thing only in the top echelon - the lower echlons are more motivated by job security, pensions and avenues for corruption.

The difference in Singapore is the government pays well enough to attract and retain higher quality talent deeper.

We're still in the "$5 airport Uber" era of LLMs. They're heavily subsidized, and everyone still complains about costs.

This is nonsense that AI providers want to peddle. Inference is wildly gross margin profitable - likely 90%+ gross margins. It's very easy to work out the cost structures bottoms up. All providers can drop costs to a third and still keep positive gross margins.

The problems are 1. It possibly still doesn't pay out on training investment in a reasonable time frame without a massive expansion of the 90% gross margin.

2. There is no moat. As we see Mac Mini & High End GPUs stock outs and the pricing offered by DeepSeek and Qwen, the performance of Open Weight models are good enough that people can and are already shifting many inference workloads out of these 90% margin players

Inference I estimate runs 90% plus gross margins. Just work out the math on these servers. I am pretty sure any player can price down. It wouldn't look good on an IPO prospectus.

So a lot depends on your specific use case but mid-sized open weight models are pretty actually good now, so this is realistic [1]

The first question to ask is does your use case require handling personal or sensitive data.

If you're using the LLM for OpenClaw or you want to handle sensitive or medical data, a local model generally is necessary.

if it's not so sensitive - Cloud providers with some sort of user agreement guarantee on not using your data for training would be the next bet. I personally generally use Gemini or Sonnet as my cloud backup. As I understand, OpenAI, Cloudflare (which bought replicate) and Qwen also seem to provide such guarantees and make SOTA models available. Others like DeepSeek seem to have an opt-out setting. Open router & co I avoid except for benchmarking models with public or dummy data as there is absolutely zero guarantee or ability to enforce terms on providers where your data might be sent.

Gemini and Anthropic (and OpenAI) tend to be expensive - it's very easy to run up 15 dollars a day or so bills which puts you solidly in 1 year pay out on Mac Mini territory - at this point I decided to buy. Gemini Flash Lite 3.1 is however surprisingly good value.

the next question is Mac or CUDA. If your expected use is serving LLM models for inferences, the latest large memory Macs give pretty good inference speed (better than DGX Spark) at a reasonable cost - I think there offer much better value than CUDA if the only use case is LLM inference & harnesses.

if you plan to also fine tune models, experiment with other types of ML on GPUs, do computer vision stuff etc. the development tooling on CUDA is far in advance of all other platforms.

Lastly if you choose CUDA, the question is GB10 family (DGX Spark - cluster able with 128Gb RAM et all) or dedicated GPUs workstations. What I found is practically any serious models weighs in requiring at least 96GB VRAM - Antirez's 2 bit quant of Deepseek 4 flash (my current daily driver) [2] , the Qwen 3.5 122B A10B 4-bit quant, the Qwen 3.6 27B Dense and 35B A3B 8 but quants etc. So you're well out of the consumer GPU territory into 1 or more RTX 6000 Pros or Data center grade devices. Yes you can try to hack away with multiple consumer cards or SSD streaming but it's very fiddly and you probably have better things to do with your life.

The GB10 system - which I ultimately went with - is certainly much cheaper and can be clustered through the Special NVLink cable to get 256, 384 or 512 GB setups but comes with severely constrained bandwidth. The Pro GPUs blast these out of water on performance but are expensive.

Lastly, renting a cloud GPU machine doesn't really make sense except to run already debugged fine tuning workloads. You'll probably spend at least 4 dollar an hour for sufficient capacity which if it's personal use, will mostly sit idle.

1. https://srinathh.medium.com/mid-size-local-models-are-now-co...

2. https://github.com/antirez/ds4

And companies from outside the US will outcompete those from the US forcing more protectionism, higher prices in US etc. I think there have been several cycles of several industries that have gone through this cycle (cars, shipping?), and mostly forced to roll back.

The problem here is that open weight models are already good enough for a majority of process automation and intelligence tasks and that is where a good chunk of efficiency corporate dollars are. So there's an ever shrinking slice of inference that will hit the frontier models and inference is where the insane margins are. Now to be fair, Claude co-work and Claude code/Codex do seem magical today and these potentially will continue to be high margin/leverage plays. Frontier models are also likely to push towards decision making - so we'll have to see how it shakes out but the bottom end is already commoditized and it is getting bigger and bigger.

As an update, turns out Antirez created a brilliant 2 bit quant of Deepseek to fit into 128Gb systems along with a custom highlight tuned server. I've been running this the last few days and if I turn off envelope on OpenClaw, the performance is brilliant. Still to try with coding harnesses. It's a bit slow compared to the other models but so good that I'm willing to put up :-) https://github.com/antirez/ds4

I think Israel's leadership has very unwisely lost the Strategic plot here in favour of tactical political advantages of survival.

The history of middle East for the last 5000 years (since Sargon of Akkad) is replete with 'the king X "pacified" (the most commonly used euphemism) the people in the conquered territory'. It has never gone well for the victors under successors of king X, often within a generation or two.

In today's age when access to technology and information is such that any small sufficiently competent and motivated group can cause massive destruction, is it wise to keep creating motivated enemies and expect they will somehow never become competent or that the competent won't become motivated? It is doubly ironic given Israel's own defense industrial complex is filled with such small motivated and competent groups and the evidence of Ukraine/Russia conflict is staring in the face. This situation will blow up I fear within a generation unless Israeli society chooses different leaders.

Deepseek Flash v4 actually runs on 128Gb systems (about 14 tok/sec). Antirez created a fabulous 2 bit quant and a highly tuned LLM server

https://github.com/antirez/ds4

Pretty sure this ruling will be used as a precedent in cases against the other providers really soon. As I understand it, there is a legal cottage industry that brings digital related cases (eg. Copyright) in Germany.

I found cloudflare zero trust excellent for this and it works perfectly well on the free tier (I do use cloudflare as my registrar)

Deepseek is too large for me to self host on Spark. I was actually using Deepseek as my cloud backup and it performed well but then read the T&C which doesn't give as strong data protection guarantees unlike Google and Alibaba. Kimi is again massive and cloud hosted APIs are fairly expensive compared and it also has weak T&C, so have only benched but not tested. In general I found that with OpenClaw it works better to turn Reasoning off.

I think there's possibly value to try fine tuning Qwen 3.5 on my OpenClaw turns log to see if performance improves. The one recent model I haven't tested yet is Nemotron 3 Super which I might bench soon.

Picketty's analysis has been empirical. The assumption that is likely mistaken is there is such a thing as market economy.

Qwen is definitely the model to beat as of Mid 2026. While I didn't benchmark with SWE as my use cases are OpenClaw [1]. I found both Qwen 3.6 35B A3B and more impressively Qwen 3.5 122B A10B starting to be competitive with closed flash models. The NVFP4 quant of the latter is what I'm running now on DGX.

[1] https://srinathh.medium.com/mid-size-local-models-are-now-co...

What is "finish the job"? Iran is a large populous country that's mostly been a unified polity since Cyrus the Great. They are a different people and culture from their Arab neighbours - how will the Arabs rule and why will the Persians follow them? A colonial rule by the US will be more disastrous than the experience in Iraq or Afghanistan - in those days cheap autonomous drones didn't drop grenades on soldiers.

The present US President is smart enough to realise that now. In this case he let himself be misled by Israelis & their supporters that Iranians would rise up and replace their own Government. That indeed might have happened if the US had intervened when the Iranians were actually protesting some months before the present crisis. Now the US administration is looking for a way out without putting boots on the ground and Iran is looking to haggle on the price and for the US. this very business like cutting of ones losses is almost certainly the right move.

Given their precarious domestic supply situation, it's been surprising they haven't chosen to lead the world on electrification of transport. China had much the same equation and have clearly gone through other way. In my recent visit, I was told in big cities buying a gasoline powdered vehicle takes months or years in permits but an electric vehicle can be bought instantly.

Only if you insist on passing on global prices into your domestic economy. Not doing this is very difficult if you're a net importer but very much possible if you're a producer. This "leaves money on the table" but it is very much a choice (with different consequences), just one that rarely occurs to people who have grown up in highly capitalist economies.

This is also why historically companies have preferred being vertically integrated to avoid having their supply chains exposed until American economists and brokerages started pushing the cult of specialization. Outside of the US, big conglomerates still operate this way.

Concord was very old technology. I am quite sure a clean sheet now

1. Would have much lower sonic booms thanks to recent research (quite a bit of it by NASA on wing geometry) and more importantly computer simulation available now

2. The engines would be far more fuel efficient

3. The flights would be able to have better efficiency in the subsonic regime as well. Just see what winglets and the like have done to fuel economy .

I fly 14 to 18 hour routes maybe 4-5 times a year on business paying 5x the economy cost and it still sucks. Breaking the flight with a connection (IMO) sucks more. My management flies such routes every month. There is a lot of revenue headroom in that fare gap for something that flies maybe 3x-4x as fast which military aircraft already do.

What will hold back the idea is conservatism among the business managers in aircraft manufactures and incumbent airlines who will "draw lessons" from a 50 year old experiment

Benchmarking the kind of cost savings I'm seeing moving from sonnet and gemini flash to local models, inference runs at least 90-95% gross margins. So they are probably still gross margin profitable.

BTW form my benchmarking, open weigh models are good enough for many agentic tasks starting with Qwen 3.5/6 family and Deepseek v4 family, so it's likely we'll see displacement of api usage from the premium priced providers. Yes trainingis expensive, this isn't training

The frontier labs have fantastic margin on inference. You do not understand how fantastic. And they have license to change inputs at will based on profitability.

This. The gross margin on inference is at least 95% if not higher - several open weight models on my tiny consumer DGX Spark easily replace the 15 dollars a day I was paying in tokens for Claw usage with a dollar a day electricity. You add data centre overhead and depreciation, the theoretical net margin will trend lower but depreciation is always far more aggressive than actual product degradation. The old NVIDIA GPU on a 9 year old second hand gaming PC I bought still serves up a small Gemma 4 variant quite reasonably.

Very nice! I also saw you have a vllm branch and I validated it works on my system. There is bugfix which sent you a PR for to auto-discover served-model-name which vllm hard validates for.

Not infeasible, just allows lower net margins.

With lower margins of course. Walmart, Indian Railways, major airlines etc all support massive user bases comparable to or bigger than the paid tiers of these apps. But of course the cult of Big Shareholder value creation means the CEO that does this, especially in the US will be fired.

Visa and Master have basically lost the enormous India market outside of credit card business when UPI was more or less mandated by the Indian Reserve Bank and really took off during COVID. Now these regional payment networks are starting to be interconnected like UPI with Singapore's NETS etc. This must be driving intense paranoia among payment networks.