HN user
GIS, fullstack, AI/ML stuff in Austin, Texas
Buf.build, never again. Had a proto evangelist get us ingratiated into their system a few years ago and then they introduced a bunch of rent seeking behavior just to be able to to the code generation, and getting it ripped out was a huge PITA. https://github.com/betterproto/python-betterproto2 was a decent (async, type hinted) python implementation last I checked.
273 GB/s. Same ballpark as M4 Pro and Strix Halo.
The RTX 5000 Pro 72GB seems like kind of a sleeper to me, and sips < 300W of power, approx 1/2 that of its big bro the RTX 6000. Kind of dream about installing it in a 10" rack, it seems like it might be able to work? @jeffgeerling you out there?
https://www.microcenter.com/product/709071/pny-nvidia-rtx-pr...
Nope, have both these machines, can confirm the M5 max blows the M4 mini away. It does get hot, but I use it mostly with an external monitor and keyboard. Conceptually I like the headless model better with a workstation, but work was buying the M5 and can't get it in any other form factor at the monute.
I don’t think there’s a GPU component so it’s probably a much lower power profile. Also, per this cloudflare write up the Turin gen of the AMD Epycs is very efficient: https://blog.cloudflare.com/gen13-launch/
Gas is the cheapest, fastest zero-to-production choice for onsite power generation, and has been for a long time.
Last I heard the wait time for turbines was ~5 years at the moment. I'm sure MSFT has some inside baseball with Chevron but it doesn't work as a general rule of thumb.
I'm using the DS4 flash IQ2 2-bit quant, per Salvadore's recommendations for my hardware in the repo. I haven't messed with the cloud hosted variant. The only other paid API I have messed with is a $20 Anthropic sub, primarily with whatever the latest version of Sonnet is. For the most part, this local configuration feels on par with that.
With this configuration (set up over the last month) I have been working on Python data processing tools, an internal Svelte 5/SvelteKit data intensive BI app, and some smaller Rust projects. It's been doing really well there.
I'm on the verge of cancelling my anthropic $20 plan since it's come out. On an M5 Max 128GB, hooked up to the pi.dev harness, I get in the neighborhood of 400-450tps prefill and 30-35tps generation. It is imminently usable and at times feels more stable than my previous CC setup. Occasionally there are things it struggles with that I will bounce back over to CC for, but it is highly usable. The future is bright for local models! As a tinkerer, it makes me really happy to have a local setup I can be just as productive in, and not have the token overlords ready to shut me down at any time.
Trying to run them on a unified memory Mac
but still not quite in the realm of Sonnet or DeepSeek 4 Flash
these are not mutually exclusive anymore. DS4 has set the bar for me these days. https://github.com/antirez/ds4
Launching a new product in SvelteKit! Coming from Vue and it has been mostly great.
It's already greatly improved over previous generations due to M5s having tensor cores (higher compute capacity for matmul operations, the bottleneck for prefill).
Sounds like it means you don't have to wire up the RPC server yourself anymore? Just build a docker container that invokes this quack server command, expose it over the network and connect to it from remote clients using your own access controls?
Ducklake handles the metadata and storage, but a local duckdb instance connected to it still has to do the compute itself. This lets you federate access to the compute.
Fun for me, I just finished a big streaming implementation doing essentially the same thing in Go-gRPC with arrow table record batches. It was fun though.
uh, doing analytics type queries on large datasets that postgres would choke on, as an RPC? I'm using it (ducklake specifically) to build a lakehouse RPC server that can scale horizontally based on resource utilization in k8s.
Dang, fun memories of when I was first getting in to geo/data stuff and doing a lot of web mapping stuff with D3, Leaflet and friends. Seems as tools like Vector tiles/PMTiles have supplanted topojson for a lot of visualization oriented use cases.
what has meta ever done that would instill trust in you? From the very article you cited:
The best thing you can do to preserve your privacy and security with your Meta messages is to use end-to-end encryption (E2EE) whenever possible. WhatsApp has E2EE built-in, and Meta has automatically started rolling it out for Messenger, but you might need to manually start an E2EE chat for existing conversations in the app. The same goes for Instagram: Meta offers E2EE, but you need to enable it yourself. In either app, tap the name of the chat to check whether or not that conversation is currently E2EE.
Just set up Pi after listening to Marios talk at AIE Europe[0] and have solid initial impressions! Especially on limited hardware like a MB Air, seems a lot more resource efficient
the Stepchange show went fairly deep on this topic in their first episode (listened to it recently). https://www.stepchange.show/coal-part-i
Yeah, but then I'm heavily coupled to their proprietary infrastructure. Maybe a good thing for them, but a nonstarter for thinking about building a real business on, for me and many others I'd presume.
DB seems like the main shortcoming in the stack for them. I don't want to deal with the limitations of D1. Seems like a serverless postgres setup a la Neon/Supabase would be a slam dunk.
our open source system. We use this tool to serve a custom routing engine at day job. Handles 100req/s djikstra in a 2GB pod, due to precalculation of contraction hierarchies.
And I only mentioned options. How do you store "every stock quote and options trade in the past 4 years" in 263 GB!?
I think this would be pretty straightforward for Parquet with ZSTD compression and some smart ordering/partitioning strategies.
DuckDB and SQL FTW.
Doesn’t matter. The point is that DuckDB can operate well on a wide range of infrastructure and is well suited for operating in resource constrained environments.
Post to HN apparently
The changelog is remarkable. Thanks to this team for creating such an amazing tool. It's genuinely the technology I've been most excited about in a long time. Makes the ergonomics of working with large data a joy and extremely fast.
IDK, they were sending around stacks of Mac Studios to tinkerer youtubers messing with EXO clustering like @geerlingguy.
How do you ascribe a revenue number like that based on one collection of changes in a huge system? Presumably there were a bunch of other features being released around the same time as it. Was there a lot of A/B testing around it?
Ok, but prefill/prompt processing was definitely the weak point before. They were already solid in raw tokens/sec after TTFT
my mac mini m4 is getting to be a good substitute for claude for a lot of use cases. LM Studio + qwen3.5, tailscale, and an opencode CLI harness. It doesn't do well with super long context or complexity but it has gotten production quality code out for me this week (with some fairly detailed instructions/background).
I hope so! I already have the M4 mini pro - would like to bump up the prompt processing time and the memory bandwidth at the same time with the new M5 matmul changes.