HN user

robotswantdata

750 karma
Posts11
Comments136
View on HN

Was “only” $2k in its previous form but even in this updated box the mem bandwidth is woefully inadequate. There’s a few models with space for a dedicated GPU for hybrid inference but imo not worth it. Save your money for a Xeon or EPYC build

It doesn’t cover the reality that nearly all your extra pay in a progressive tax system goes to the government. UK is hitting 70% marginal rates , or even 1000% for people with children.

Very interesting.

The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents.

Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, developers are forced to build janky code that chops PDFs into individual pages, processes them one by one, and glues the text back together.

Unlimited OCR uses Reference Sliding Window Attention (R-SWA) to split the AI's focus into two paths:

Global Reference: The AI keeps full, uncompromised sight of the original document image so it never loses context.

Local Generation: The AI restricts its memory of its own typed text to a tight, moving window (like the last 128 words) and safely forgets the rest.

Will be very interesting for local AI and can’t wait to see what the community builds and extends with it!

Granite or sapphire rapids are very under rated for MoE inference loads. But you need a GPU for the KV cache.

Plus many boards also support CXL for RAM expansion over PCI 5!

Source: building a hybrid inference business for regulated industry workloads.

MCP is dead? 2 months ago

I agree, codex app’s computer use agent feels sci-fi. Closest we’ve seen yet of a general purpose virtual worker.