HN user
https://nickmonad.blog/
I haven't written any serious Rust in a while, but I assume there is some kind of lifetime annotation involved. The "objects" allocated in the arena have to be explicitly given the same lifetime as the arena itself. I have no idea how that looks syntactically.
Zig still offers a lot of great additions that make systems work more reliable. Optionals and comptime are two helpful things C does not have, and there are plenty others. One of their core devs addressed this in a thread over on Lobsters [1]
Bounds checks, checked arithmetic, strongly-typed alignment, strongly-typed error codes, tagged unions, explicit undefined are some rather important language features. If Zig didn't exist, TigerBeetle would probably have been written in C, and the safety gap between C and Zig is just gigantic. And safety is one aspect of the language, Zig has a lot of going on for it elsewhere!
He's not.
"There's a dichotomy being presented here where you have to either choose a "style guide" or a programming language feature in order to avoid bugs. The sleight of hand misdirects the reader away from the main way bugs are eliminated: by dedicating engineering resources to it. You're not giving TigerBeetle nearly enough credit. Quite simply they put in the time to find and eliminate the bugs, they make an effort to maintain a healthy relationship with ZSF, and Bun did not do that."
The reference to TigerBeetle is important, and a bit under-explained for the point he's trying to make. They have consistently attributed things like design and deterministic simulation testing to their success and reliability, which has nothing to do with Zig as a language. Some things might be _easier_ in Zig, such as static memory allocation, but ultimately, a holistic approach brings success, not one individual tool used "right".
Andrew doesn't strike me as someone who does any marketing at all. He just wants to make the language he wants to use, and does it well.
Sometimes its just right time, right place. But also, Zig has received attention via projects like Ghostty, TigerBeetle, and Bun (prior to rewrite of course)
Yep. He mentioned recently in his JetBrains interview he wants Zig to be a language for the next 50 years. Rushing 1.0 for the sake of signaling to the wider industry today would be actively harmful to that goal.
Yeah or anybody who can still actually read code.
Sure. I'm just saying in the context where fully-managed SaaS was already decided not to be an option, and a customer is deploying vendor code in their environments, the update mechanism can in fact be a problem. It's not just poor CISO management.
I agree that keeping things up to date is a good practice, and it would be nice if enterprise CISOs would get on board with that. One challenge we've seen is that other aspects of the business don't want things to be updated automatically, in the same way a fully-managed SaaS would be. This is especially true if the product sits in a revenue generation stream. We deal with "customer XYZ is going to update to version 23 next Tuesday at 6pm eastern" all the time.
So you're stuck debugging a system you don't control, through screenshots and copy-pasted logs on a Zoom call.
This is very real.
I work with a deployment that operates in this fashion. Although unfortunately, we can't maintain _any_ connection back to our servers. Pull or push, doesn't matter.
The goal right now is to build out tooling to export logs and telemetry data from an environment, such that a customer could trigger that export on our request, or (ideally) as part of the support ticketing process. Then our engineers can analyze async. This can be a ton of data though, so we're trying to figure out what to compress and how. We also have the challenge of figuring out how to scrub logs of any potentially sensitive information. Even IDs, file names, etc that only matter to customers.
On the streaming side, are you looking for Change Data Capture?
Did you try asking the model?
Spam bots.
It's a type of team building, improves employee morale and humanizes management. All lead to improved productivity in the long term.
Yes, but the important distinction is that its intention is to bring people together face-to-face, not isolate them to their desks for continued work. Just because the end goal is "productivity" broadly speaking, doesn't mean the mechanisms are socially/morally equivalent.
I doubt anyone is forcing the employees to take the stimulants.
I agree, and I hope my comment didn't imply I thought that was the case.
I agree to a point. Although, alcohol (when consumed responsibly) has a social element to it, so companies having a "beers on Friday after 4pm" just feels different than "here's nicotine so you can be more productive and make us more money." They are serving different functions.
Not allowed.
Yeah definitely something that would've been posted as a joke in a "HN front-page 10 years from now" kind of thing.
You turn it on, and it scales right up.
Unless you're doing OLTP. Then, TigerBeetle ;)
Trying to blog more frequently with shorter posts!
Narrowing in on background color is an extreme oversimplification of what Tailwind provides. I found it to be a great tool for working with CSS, especially for layout. Business viability can be debated, but the value is way beyond what you suggested.
I agree with the sentiment that companies should help fund open source they depend on, but I think it's a stretch to say those business succeeded "only" because of Tailwind. It's a great project, although I'm pretty sure they would have figured out a way to work with CSS without it.
Gotcha. Thanks for clarifying! I guess I wasn't super concerned about the 'try' failing here since this code is squarely in the initialization path, and I want the OOM to bubble up to main() and crash. Although to be fair, 1. Not a great experience to be given a stack trace, could definitely have a nice message there. And 2. If the ConnectionPool init() is (re)used elsewhere outside this overall initialization path, we could run into that leak.
The allocation failure that could occur at runtime, post-init, would be here: https://github.com/nickmonad/kv/blob/53e953da752c7f49221c9c4... - and the OOM error kicks back an immediate close on the connection to the client.
Would love to read this, although I'm seeing some pretty horrific code formatting issues in both Firefox and Chrome.
Hey matklad! Thanks for hanging out here and commenting on the post. I was hoping you guys would see this and give some feedback based on your work in TigerBeetle.
You mentioned, "E.g., in OP, memory is leaked on allocation failures." - Can you clarify a bit more about what you mean there?
This is the fundamental question which motivated the post. :)
I think there are a few different ways to approach the answer, and it kind of depends on what you mean by "draw the line between an allocation happening or not happening." At the surface level, Zig makes this relatively easy, since you can grep for all instances of `std.mem.Allocator` and see where those allocations are occurring throughout the codebase. This only gets you so far though, because some of those Allocator instances could be backed by something like a FixedBufferAllocator, which uses already allocated memory either from the stack or the heap. So the usage of the Allocator instance at the interface level doesn't actually tell you "this is for sure allocating memory from the OS." You have to consider it in the larger context of the system.
And yes, we do still need to track vacant/occupied memory, we just do it at the application level. At that level, the OS sees it all as "occupied". For example, in kv, the connection buffer space is marked as vacant/occupied using a memory pool at runtime. But, that pool was allocated from the OS during initialization. As we use the pool we just have to do some very basic bookkeeping using a free-list. That determines if a new connection can actually be accepted or not.
Hopefully that helps. Ultimately, we do allocate, it just happens right away during initialization and that allocated space is reused throughout program execution. But, it doesn't have to be nearly as complicated as "reinventing garbage collection" as I've seen some other comments mention.
Nice! Will definitely take a look :)
Author here! Overcommit is definitely a thing to watch out for. I believe TigerBeetle calls this out in their documentation. I think you'd have to explicitly disable it on Linux.
For the second question, yes, we have to keep track of what's in use. The keys and values are allocated via a memory pool that uses a free-list to keep track of what's available. When a request to add a key/value pair comes in, we first check if we have space (i.e. available buffers) in both the key pool and value pool. Once those are marked as "reserved", the free-list kind of forgets about them until the buffer is released back into the pool. Hopefully that helps!
I have a few cases in this (proof of concept) codebase that require knowledge about allocation strategy, even in Zig, but that's on me and the design at this point. Something I wanted to touch on more in the post was the attempt to make the components of the system work with any kind of allocation strategy. I see a common thing in Zig projects today where something like `gpa: std.mem.Allocator` or even `arena: std.mem.Allocator` is used to signal intent, even though the allocator interface is generic.
Author here! That's totally fair. I did learn this is a common technique in the embedded world and I had a whole section in the original draft about how it's not a super well-known technique in the typical "backend web server" world, but I wanted to keep the length of the post down so I cut that out. I think there's a lot we can learn from embedded code, especially around performance.
TigerBeetle: https://tigerbeetle.com/blog/