woadwarrior01

Also, people seem to be oblivious to the existence of the neo-cloud: Nebius whose services and servers, a lot of western AI companies use. Nebius was spun out of Yandex 2 years ago to assuage the "supporting Russia" allegations and is still run and managed by an ex-Yandex crew. By that measure, I suppose using a lot of western AI tools is also equivalent to "supporting Russia".

Krea-2-Turbo and Z-Image-Turbo, both offline.

Thanks for bringing this up. You're 100% right. But most people, even technical ones are oblivious to how much of a difference modern samplers and higher quality quantization algorithms make for on-device LLM inference and are stuck with good old top-p, top-k samplers and RTN quantization.

TBF mlx-vlm does support min-p sampling, but none of the other modern samplers that you list. Ollama and LM Studio are even worse with only top-p and top-k samplers.

It is a reasonably non-opinionated model. My usual test is to ask these models to generate comic book and cartoon characters that hosted image generation models refuse to generate. I think that text is just CYA legalese.

I've seen this pattern repeated in a number of large production codebases using optional booleans.

Krea-2-Turbo. I've even got it working locally on my M5 iPad Pro.

Ironic that the app is named Nativ(e) and yet bundles a full Python runtime. Nonetheless, still less bloated than LM Studio, which bundles a full Python runtime and electron.js (which in turn bundles a whole browser runtime).

Works with the Bonsai-27B-1bit-mlx model on my mac.

Edit: this is running with MLX, and not with WebGPU on the browser, also it is a fail.

"""

You should *walk*.

Here’s why:

- *Distance*: 50 meters is very short — about 0.3 miles or 0.5 kilometers.

- *Time*: Walking takes roughly 2–3 minutes. Driving would take longer due to parking, starting the engine, and maneuvering.

- *Effort*: Walking is light exercise and avoids parking hassle.

- *Safety*: Less risk of misjudging parking space or damaging the car.

Unless you have a disability that makes walking difficult, walking is the most practical and efficient choice.

"""

I think the "8.9 million developers use ollama" figure they throw around is contrived. I found this 2025 article which posits that there were 47.2 million software developers in the world. Even the number of developers has doubled in a year (plausible if you account for the rise in vibe coding), I think it's very unlikely that 1 in 10 developers use ollama.

https://www.slashdata.co/post/global-developer-population-tr...

To me, it's the cli.

It's unlikely that a clear single winner will emerge in this competitive market.

Yeah, I learnt about this from the amazing Max Bennett book: A brief history of intelligence.

BaaS: Bhatta as a Service.

nb: Bhatta is Sanskrit for priest.

I think NERVA and its Soviet equivalent RD-0410 were much more practical and plausible. Unlike Orion which was complete vaporware, both these projects reached the ground test prototype stage.

https://en.wikipedia.org/wiki/NERVA

https://en.wikipedia.org/wiki/RD-0410

Next update? Have you read their privacy policy?

https://lmstudio.ai/app-privacy

In any case I am not sure pivoting from running local models to "cloud offering" (as in providing llm inference at their severs) is a sensible choice granted there is already competition in that space and they have no leverage there.

I agree. Incidentally, this is exactly what ollama are doing too.

The desktop app's GUI is electron.js based.

Ultimately, the onus at every VC backed local LLM startup is to launch a cloud based offering, because that's the only potential path in sight for venture scale returns.

Small encoder-only transformers are excellent at classifying LLM-Generated Text. I built an on-device iOS app using a custom small encoder that achieves an AUROC of 99.81 on RAID-bench.

Cursor will rebrand it as Composer 3.0 to assuage any such concerns, as they did with the previous Kimi models.

$56 premium for the OpenAI skin. :)

Agree. OTOH, people have been using plotters for simulating handwriting. Also, nothing preventing someone from hand-copying an AI written piece as I'm sure lots kids are doing these days for take-home assignments.

I've been thinking about something similar for a while now. Although my scheme involves cryptographic attestation and typing stuff with hands.

Perf should be fairly straightforward to ballpark. You'll need to transfer roughly 2 . hidden_size . num_shards bytes over the network per token during autoregressive decoding. And divide that number by chunk size during prefill.

Apple ripped of the phone that won the iF Design Award in 2006, the LG Prada.

Nice, albeit implausible story. Apple had been working on multi touch screens for a long time before that. They applied for a patent on it from 2004[1]. And TBF, neither Apple nor LG invented capacitive touch screens. Multiple discovery is a thing.

[1] https://patents.google.com/patent/US7663607B2/en

Granted LLMs aren't very good at the ML family of languages. Isn't vibecoding a lot safer with a typed language? Going from Haskell to Python is going from one end of the typing spectrum to the other.

because Android re-skinned to use BUTTONS.

No. Steve's rage was justified, IMO. It was because Eric Schmidt was on Apple's board while simultaneously being Google's CEO and Google was surreptitiously building Android at the time. Mother of all conflict of interests.

There was a recent story that reminded me of it. Mike Krieger was on Figma's board and Anthropic's CPO, while Anthropic was surreptitiously building Claude Design.

If they were wary of dying, they wouldn't be in the business of terrorism in the first place. Also, they almost certainly believe in a better afterlife. Reminds of the old animated short: Saga of Bjorn[1].

[1] https://www.youtube.com/watch?v=MV5w262XvCU

some providers will ride on the good reputation of some local model, selling you a terrible quant instead.

Quants in popular local inference apps (Ollama, LM Studio, etc) are the worst possible quants (RTN).

It'd be terribly compute inefficient to not share prefix caches (KV cache) across customers.