For most of the world, open-source AI is the only way forward

https://techstrong.ai/articles/for-most-of-the-world-open-source-ai-is-the-only-way-forward/

by CrankyBear • 28 days ago

235 144 28 days ago

pmontra• 28 days ago

Agreed but I want to see how it plays out. Historically a good Windows computer cost $1000 and it was all it took to start programming. How much does it cost a computer with enough resources to run a good enough AI model for agentic workflows and a reasonable time to first token? Can "most of the world" afford buying one?

bensyverson• 28 days ago

Yes, between Moore's Law and more efficient model architectures, we just have to let time do its work.

Danox• 28 days ago

Software models and hardware are getting better all the time—and that’s where some big companies spending billions might stumble! In fact, Microsoft recently announced that they’re scaling back a bit on their AI investments.

ktallett4• 28 days ago

Hence why brute force needs to be replaced with examples such as neuromorphic methods. It could realistically could be combined with mesh networking as well to utilise the capabilities of all computers locally.

mbgerring3• 28 days ago

About $2k in 2026 dollars and falling.

simonw5• 28 days ago

... or rising, at least as long as there's a RAM shortage.

Kim_Bruning• 28 days ago

Roughly about Eur 3-4K right this minute I think? The graphics card, ram and storage are punishing. Under more normal circumstances (hopefully late 2027) it'd be 1500-2500 depending on what you think is realistically useful.

Possibly it's the same price range, allowing for inflation.

wizee• 28 days ago

Qwen 3.6 27B is quite good for agentic coding, and practical to run on consumer hardware. You need a system with either 32+ GB VRAM, or a unified memory system with 48+ GB VRAM and a decent integrated GPU. While not cheap, such a setup is still attainable for much of the world, and will eventually get cheaper over time. Open models hosted on non-American clouds also remain an option with a much lower barrier to entry, for cases where privacy is less critical.

jochem9• 28 days ago

There was an article on HN a few weeks ago where someone detailed how they managed to get an old datacenter GPU to run in their consumer PC, getting decent performance with qwen. He spent something like $200 on the GPU (second hand of course).

So yeah, I think models on local hardware will be quite common soon among the tech savvy (such as people creating software).

schmuhblaster• 28 days ago

Indeed, and with some tinkering around the harness it can even punch way above its weight.

thewebguyd3• 28 days ago

You need a system with either 32+ GB VRAM

I do hope you're right that it will get cheaper over time (it should), but right now 32GB of VRAM is not affordable to a lot of people. You're talking ~$4500 just for the GPU, or $800 ish used if you can find one.

yowlingcat• 28 days ago

I've seen folks make it work with a 3090 on 4 bit quant using turboquant for KV cache. That's key because 3090s remain the most cost effective gpu metal for enthusiasts (albeit 24g) and the jump to 5090 (32g) is quite expensive and not always worth the LLM specific performance; sadly, good 32g metal is somewhat lacking in the price point at or above the 3090.

Chu4eeno6• 28 days ago

Open weights/source doesn't necessarily mean running on local hardware, though.

I imagine having multiple providers competing will drive down hosted versions of open weight models drastically.

Tuna-Fish2• 28 days ago

You are describing OpenRouter. And yes, it does.

mncharity• 28 days ago

Open weights/source doesn't necessarily mean running on local hardware, though.

And we've barely started to scratch the surface on helping open-weight models "be the best they can be", with cloud burst parallel sampling and prompt mutation. Looking for best probabilistic results for a prompt, and looking for best prompt variants for a task. Adaptively scaling computes at generation, not just training.

And speculatively, if agentic coding is naturally a multiplicity, what UX might enable human devs to dance with that quantum superposition? Rather than quickly collapsing to one monkey and its keyboard.

giancarlostoro2• 28 days ago

Before the AI "crisis" it used to take about $3500 to get a prebuilt with a 5090 which can run good enough LLMs. I run reasonable LLMs on just 16GB of VRAM on my Mac, and the 5090 has double that.

dgellow3• 28 days ago

What model (and quant) do you run with 16GB? I assume you have a 24GB model, and dedicate 16GB to the model?

majormajor• 28 days ago

Historically a good Windows computer cost $1000 and it was all it took to start programming.

Gotta remember inflation here.

$1K in 1995 was roughly equivalent to $2K now and wouldn't have been a particularly "good" machine then.

In 1982 the Commodore 64 started at about $600 bucks, also roughly around $2K today.

If you outgrew that, beefier machines back then were A LOT. It was easy to find $2k+ towers and (especially) laptops even into the 2000s, and a lot of those would be $5K+ equivalent today.

SoftTalker2• 28 days ago

And a unix workstation in those days could be high 4 or even 5 figures, depending on configuration.

rayiner• 28 days ago

Isn’t this just a bet that I’ll have an AI data center in my iPhone within 10 years? Why is that a bad bet?

abetusk2• 28 days ago

Moore's law or one of its generalizations still holds, so it will only be a short matter of time before a $1k computer will be able to train and run a powerful enough model.

Windchaser3• 28 days ago

I thought Moore's Law came to an end in the last decade?

Certainly the transistors/chip or transistors/$ or flops/$ have not been progressing at the same exponential rate as during 1970-2010. There is still progress, but it's rather slower.

crazycracker• 28 days ago

Historically the cost of compute has also gone down. Like just look at it as compared to a year ago. We have amazing open source models that can run on consumer hardware and if we go away from our obsession of using opus 4.8 or mythos for everything then it actually is super amazing to see what these open source models could do. I use qwen3.6:27b as a daily driver and I am heavily impressed with it.

ssivark• 28 days ago

I don't understand the justification for local hardware with cost as the motivation. The same (or bigger/better) open weights models can served by third parties at much higher resource utilisation, and will therefore be much cheaper!?

Especially because the world is likely to persist, at least for a while, in state where computing hardware demand drastically exceeds supply resulting in high prices for hardware. So why wouldn't you want to max out utilisation and amortize costs, at least for typical (non sensitive) use cases.

layer83• 28 days ago

IMO the more useful distinction is in analogy to VPS versus SaaS/PaaS. Open models allow you to use any inference provider you like, including local ones, similar to running open-source software using VPS providers. You’re not bound to a particular SaaS/PaaS as you are with closed model providers. That same freedom also allows you to self-host when you care about that.

skydhash• 28 days ago

Historically a good Windows computer cost $1000 and it was all it took to start programming

Started with computers around 2009 and later bought an oldish computer (a pentium 4 PC) for the equivalent of 50 usd. Codeblocks and Python Idle were free at the time (C and Python were the first languages I learned). The barrier to programming has always been low as the only thing you needed was books (the internet made things easier) and access to a PC (I had friends with laptop and my school lab).

Supermancho• 28 days ago

In 1998 I had a second PC for the first time in my life. It was an old i486DX2-66 that my dad gave to me. He said it cost him $50 secondhand and was bought so I could put Linux on it.

epolanski• 28 days ago

If we had 256-512 GB ram unified memories at 2022 prices, we'd be talking 1500 computers.

nicoburns• 28 days ago

I think AI is still in it's "mainframe era". Once upon a time, computers that ran useful workloads filled a whole room. And now that's true of computers that run (state of the art) AI models. My guess would be that this won't always be the case, although I certainly don't claim to have a clear idea of how things will play out.

Thraway1982• 28 days ago

With the free tiers available now you can make your own document processors and other such office software, in one pass.

orbital-decay• 28 days ago

How far back are we talking? Historically there were just a few computers in the world, personal computing revolution happened much later. And a private inference rig costs roughly the same in 2026 as a personal computer did in the early 80's, if you want the performance comparable with a personal workstation of that time (vs a mainframe/supercomputer).

onel• 28 days ago

Even if you go with an open source AI model, it will still make a lot more financial sense to pay a provider then actually invest in the hardware to self-host it.

It's definitely worth investing in self-hosting the agent infrastructure around the model though: all the documents, knowledge base, all the connectors, the agent itself to run on your hardware

echelon6• 28 days ago

We don't need rinky-dink RTX models that budget VRAM.

We need large scale open weights models just as capable as what's at the frontier.

And we need the ability to rent compute and spin up the weights easily. One-click, easy enough for anyone. Easier than nerd tools like ComfyUI, Claw, and node graph garbage.

Freedom is owning very large scale weights. Anything less is subsistence.

ktallett4• 28 days ago

We need to improve the waster and energy usage and this method doesn't. Most are not reinventing the wheel, a shared AI repository, communicated between online local computers would save a lot of need for these large models.

echelon6• 28 days ago

NO!

This is the wrong approach that will turn us into serfs. We need big honking models that do what the leading foundation hyperscaler models do to within a few percentage points of measured performance.

The small-scale models are not productive, and the duct tape solutions built on top of them are hobbyist-tier "year of Linux on desktop" toys.

I imagine fedora-wearing, crypto-shilling, coupon-cutting boffins every time I see small weights thing lauded as the future. This is the Pine Phone F-Droid of AI.

"SMS works most of the time on my phone, I swear! I don't really need my banking app!"

That is not big model energy.

Nothing outside of the top ten is worth spending any time on, and we need to focus on models that bridge the gap.

You're talking about impractical toys for highly technical people wasting their own time. That doesn't move the needle or have any economic impact on the competitive landscape.

We need sharp teeth that bite at the legs of the top-tier foundation labs and hold them back from running away with the prize.

We've been through this time and time again over the last thirty years. It's the same shaped problem as before. We don't need toys - we need real infra for real people paying money to do work. Not freeware for freeloaders who don't spend and invest in the problem space.

Large models fit that precisely, because it forces investment into a wide variety of open infra, routers, inference engines, etc. Not to mention the weights ecosystem itself.

simonw5• 28 days ago

I'd love to see credible numbers on the energy usage of thousands of people running models on their own devices compared to sharing data center resources to run big models that serve many different people at the same time.

My hunch is that the energy/water usage of the data centers is a whole lot more efficient than everyone running at home, but I'd be interested in seeing real data on that.

xtracto• 28 days ago

we need this: https://news.ycombinator.com/item?id=48516751

distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive. So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments.

Also The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it.

echelon6• 28 days ago

Just rent an H200.

You rent your fiber optic internet. You're doing just fine. The world isn't collapsing because you don't own the hardware racks, routers, and fiber lines.

This crazy zany P2P communal infra is Arch Linux coded - too much work, aimed at the 0.001% of users, and the juice isn't worth the squeeze.

It sounds the exact same as being mad at your ISP, so you want to build a mesh internet protocol over microwave dishes and share with everyone in your neighborhood. That was a thing in the 00's. And it predictably got nowhere and delivered no value.

Nobody's got time for this kind of stuff except for hobbyists. It's not solving a real problem. Nor does it attract business investment to grow into a healthy offering.

Just build OpenRunPod instead.

The problem is that the world doesn't have access to frontier open weights. That's the only problem to focus on and solve.

The quickest way to get there is to use large scale open weights instead of little rinky dink RTX distillations. And the easiest way to run them is by renting H100s or using a managed service offering. There's nothing wrong with either of those options.

The infrastructure is the easy part anyway.

blakesterz• 28 days ago

There's a video of the entire session here:

https://webtv.un.org/en/asset/k14/k14ej1ucqu?kalturaStartTim...

(if that link doesn't work, it starts about 12 minutes into the start)

verdverm3• 28 days ago

had to click the play button, but it keyed to the 12m mark

mbgerring3• 28 days ago

There is no reason we should accept the enclosure of the digital commons represented by AI. The data these models are trained on amounts to the total intellectual and artistic output of human kind through recorded history. It belongs to all of us, and accordingly, so should the models and weights produced by it.

logancbrown4• 28 days ago

The effort of humans who had to toil through training models belongs to everyone? Do they no longer have any ownership over their hard work?

petcat2• 28 days ago

Not too mention the unbelievable cost of actually doing all that training.

riddlemethat• 28 days ago

They got paid. That’s what the money was for. It’s the investors who backed these foundational model companies who will hold the bag as more open source models come along and consume more market share.

tencentshill• 28 days ago

Some consider copyright to be unethical. "Information wants to be free".

cgannett• 28 days ago

A drop in the bucket compared to the value of the collective human work that was stolen to train it.

edit: come to think about it I think the ratio of one drop to one bucket is vastly over estimating the ratio of the trainer's effort.

dofm• 28 days ago

Paying them may now be impossible. There might be some legal settlements still.

Preventing a handful of massive companies from continuing to be the only ones able to make money off that, not only unimpeded but with overt or covert state assistance (regulatory capture, ownership, whatever), at least puts an end to the worst of the abuse.

If we have broken the idea of copyright, and we do indeed appear to have broken the idea of copyright, why should trillion dollar companies owned and controlled by strange or psychopathic weirdos and their circle of investors be the only ones benefiting? Why do Sam and Dario or the US government get to decide when and for whom the tap is turned on?

failbuffer• 28 days ago

Are the going to pay for the societal harms they cause?

cyanydeez• 28 days ago

ok, but government is how you do that. and as should be evident, its easy to year down and corrupt

hootz• 28 days ago

Companies aren't really that better. Actually, sometimes it's even worse, because they are not even formally required to serve the public.

nxm• 28 days ago

It belongs to not us, but the copyright holders of that work

dwa3592• 28 days ago

and that is millions of people; hence the use of the word 'us'.

stackedinserter2• 28 days ago

It's closer to "that output belongs to everyone, everyone's free to train your own model"

Otherwise, everything you do belongs to all of us, because you learned on total intellectual and artistic output of human kind too.

jazzyjackson3• 28 days ago

No I did not illegally download every book ever published. anyone who went to college paid good money for their textbooks.

robwwilliams5• 28 days ago

Yann is on the mark. Almost amusing to see the EU along with its many former “subjects” realize they are at great risk of joint Chinese-American hegemony in AI. We should all be more terrified of a few nation states defining the agendas and policies of AI use than current Ai variants that a inherently without purpose or autonomy.

Great analogy to the fear of the printing press being really bad news in that it enabled the rabble to get aroused.

oceanplexian2• 28 days ago

AI is the canary in the coal mine. They don't have an AI problem they have an everything problem. Inability to maintain energy security, declines in manufacturing, their social programs are no longer sustainable (Pension age rises and reforms), German car industry is in decline, increased spending demands for defense, and so on.

All that's needed is another sovereign debt crisis to spark what is essentially dry tinder and I think the EU is a lot closer to collapsing than anyone even remotely realizes.

blitzar• 28 days ago

EU is a lot closer to collapsing than anyone even remotely realizes

I have been hearing that will happen next year for the last 20 years. No this time it isn't different.

robwwilliams5• 28 days ago

Not how I see it. And I have seen Europe close up for 60 years. It is wealthier and happier than it was even 15 years ago. Visit Spain, Portugal, Greece, and of course the other larger northern states; no comparison. Immigration has served them well in wealth and health. Compare Europe to the hollowed out core of the USA and you will shudder. Visit rural America.

The European “problem” is they live to have a life whereas in the USA we seem to live to work.

paxys2• 28 days ago

We aren’t going to have Open Source AI without Open Source hardware specs and Open Source manufacturing. Software has been solo driving open computing for far too long, and with AI now the bottlenecks are finally moving up the stack.

OsrsNeedsf2P• 28 days ago

What..? We have open weights AI already, and even if we didn't, I don't see how Open Source hardware is a pre-requisite

paxys2• 28 days ago

Weights are meaningless if you can’t run the model from a computer under your desk.

wmf• 28 days ago

I don't see any evidence of this. Open source software thrived on proprietary hardware for decades.

dippogriff• 28 days ago

Edge models will get much better after the current insane capex and organic data for pre-training is dried out. But hard to see how the best open source models will ever come close to the best closed ones.

dbolgheroni• 28 days ago

It's already happening. GLM-5.2 ranks quite close to SOTA models. Some might argue that benchmarks can't measure the real effectiveness for day-to-day usage but that's another discussion.

peterlk2• 28 days ago

Over the long term, it seems like open models must win out. This feels like it rhymes with the story of operating systems. Despite the enormous financial contributions of Microsoft and Apple, linux still won because control matters over the long term.

I predict that mech interp and things like Neuronpedia will matter more and more over time, and the frontier providers are disincentivized from providing those tools

petcat2• 28 days ago

linux still won because control matters over the long term.

what has Linux won? Servers? sure

peterlk2• 28 days ago

Yeah. I would expect that Samsung or Apple’s bundled AI things (whatever those end up being) will be good enough for consumers, and open models will be the things that actually get used in the products of the future because they will allow technical people to alter models in ways that specialize them for whatever use case they’re building for. They will also be the onramp for engineers/data scientists of the future, and thus will have more available, specialized knowledge over time.

The bitter lesson would claim that we only need one model with more data thrown at it, but I’m a bit skeptical that we’ll end up with “the one true way” of building a model, and I think there will be model tradeoffs that we can pick from as the industry matures.

seanclayton• 28 days ago

Laptops with Chromebooks. Mobile with Android phones. Game consoles in Steam decks and now Steam Machines. The things linux appears on are growing. The things proprietary OSes appear on are shrinking. Losers and victors in a war are often defined by who loses and who gains the most.

sademo• 28 days ago

depends on what you mean by winning out most of the profits will be hold by closed proprietary models and open sourced models will be barely able to fund their own research and development cost just like how apple has barely 10-15% of market share of smartphones but takes away 80-85% of all profits. I kinda see a similarity of this may turn out for AI as well.

prmoustache• 28 days ago

What is Open-Source AI? Has it been defined?

By all accounts, all AI companies starting with open are doing proprietary stuff. All models delivered for free as "open-models" are just freeware as no source is really provided.

layer83• 28 days ago

The talk is about this: https://thealliance.ai/projects/tapestry

Collaboratively trained open-weight models, is my understanding.

mossTechnician• 28 days ago

LeCun framed AI as an infrastructure-level platform that will soon mediate "all of our interaction with the digital world with information more generally"

This doesn't read as revolutionary, it reads as accepting a premise without explaining how and why it will and must be the case.

Some of my favorite parts of the internet - wikis and forums and blogs - are special to me because there is no procedurally generated mediator between me and the information. I don't want a summary that was never intended by the author. I don't want the rough edges shaved off every blog until they all sound like they were written with an identical voice. Maybe some people would like that some of the time, but all?

cesarvarela• 28 days ago

As AI amplifies everyone’s output, you’ll need AI to amplify your input capacity—or you’ll fall behind.

dminik• 28 days ago

Slop out, slop in.

627467• 28 days ago

I thought this line of argument ended with the fall of the iron wall. Just because you can put every citizen digging holes to increase gdp it doesnt mean you should

grim_io• 28 days ago

This is already the reality for a large portion of the population.

They(we) don't consume directly from the firehose of the internet, but from whatever "The Algorithm" has curated for them(us).

wbl• 28 days ago

Ever since stumble upon.

Thraway1982• 28 days ago

I'm pretty sure they mean that AI will become the infrastructure you view the digital world though. Like, instead of using a browser, just tell the LLM what you want to go to and it will bring you there, or give you the link, or even just the info.

shark1• 28 days ago

Yes. Very precise way to put that.

It's the same way the algorithm telling us what to see when we just want to see the raw feed.

In all cases, we could and can have both.

snootypoot• 28 days ago

not only is he correct, but this path will result in openai going the way of enron. i hope in the case of openai and their collaborators in nvidia we see more people going to prison.

dgellow3• 28 days ago

Would need changes in the US government before that can happen

For most of the world, open-source AI is the only way forward

Related Stories