HN user

justaboutanyone

138 karma
Posts1
Comments20
View on HN

At this point, it might be moot. Too many people are assuming it's still a closed-source thing and will dismiss it.

Due to the closed source nature, every mojo announcement I see I think "whatever, next"

If the actual intent is to open-source, just do it, dump out whatever you have into a repo, call it 'beta'

Shell programming is high density inter-language glue. You simply have more options of implementations to call out to and so less to write.

I can trivially combine a tool written in rust with one written in js/java/C/whatever without writing bindings

Qwen3-Coder-Next 6 months ago

Running llama.cpp rather than vLLM, it's happy enough to run the FP8 variant with 200k+ context using about 90GB vram

For Qwen2.5-72B-Instruct-Q5_K_M at 32k context, I fed it a 26k token file (truncated fiction novel) asking it to summarize, and it input processed at 224 tok/s and output generated at 3 tok/s. Not really good enough for interactive use without frustration. Not just from watching it reply, but also the long wait for it to actually read the book.

On the same hardware gpt-oss-120b at 128k context, I fed it a longer version of the input (a whole novel, 97k tok), and it input processed at 1650 tok/s and output generated at 27 tok/s. Just fast enough IMO

You can run large-ish MoE model at good speeds, like gpt-oss-120b, it's snappy enough even with big context.

But large and dense at the same time is a bit slow.

Running a local LLM will be a load of money for something much slower than the api providers though.

[dead] 6 months ago

What is the article text for those stopped by the paywall