Ask HN: If I cancel Codex today whats the next best local inference agent?

https://news.ycombinator.com/item?id=48326267

by Bulbasaur2015 • 2 months ago

11 9 2 months ago

news.ycombinator.com

better place to ask over /r/LocalLLaMA

bigyabai• 2 months ago

For local inference? It entirely depends on what your hardware is.

verdverm• 2 months ago

OpenCode + vllm, model will depend on your hardware, but OpenCode also has a killer $10/m plan with quotas for some top tier open weight models.

I'm using qwen3.6 on a DGX spark, llama-cpp has prompt cache bugs for qwen/gemma models (among more being reported). Using my OpenCode-go sub when I want a bigger / more capable model

JojoFatsani• 2 months ago

Check llmfit

kevinsmith51• 2 months ago

Would use a capability based routers so you can use a blend of OSS models. I.e. use the least capable model per prompt that includes the appropriate tooling capability, etc. Can even include a frontier provider subscription and get almost as many tokens at very close to the benchmarking on a $20/mo subscription as a $200/mo subscription. Easier with Claude's bearer token setup but I have seen people do it with OpenAI subscriptions as well.

caterama• 2 months ago

Zed agent

amazingamazing• 2 months ago

Do you have a b200 or integrated graphics in your home? Basically it depends on hardware.

Tanxsinxlnx• 2 months ago

exactly,great responsibility comes with powerful hardware

nulledy• 2 months ago

The Qwen models seem to be the best currently in my experience.

But if you're thinking about buying new hardware just for local inference: don't.

You'll spend more upfront for rapidly out of date and depreciating hardware while running inferior models compared to the latest from Anthropic and OpenAI.

Unless your requirement is prompt privacy, or you have idle hardware sitting around, the ROI just isn't there compared to a monthly subscription.

fircelik• 2 months ago

freebuff or Opencode! Codex or Claude Code is overhyped and commercial hype cli's or apps.

Ask HN: If I cancel Codex today whats the next best local inference agent?

Related Stories