I'm too dumb for Zig's new IO interface

https://www.openmymind.net/Im-Too-Dumb-For-Zigs-New-IO-Interface/

by begoon • 11 months ago

206 311 11 months ago

stop50• 11 months ago

I have never understood libraries or imterfaces that want me to allocate buffers for their type. I can't parse them (no need for the lib then) or write to them (would probably break the exchange).

The weird interface of go is probably due the fact that some interfaces can be used to extemd the writer like the hijacker interface (ResponseWriter.(http.Hijacker)) and the request object is used multiple times with different middlewares interacting with it. In short: request does not need to be extended, but the response can be an websocket, an wrapped tcp connection or something else.

benreesman3• 11 months ago

It's just a different convention like radians and degrees.

You can lift/unlift in or out of arbitrary IO, in some languages one direction is called a mock, in other languages the opposite is called unsafeFoo.

Andrew Kelley independently rediscovered on a live stream 30 years of the best minds in Haskell writing papers.

So the future is Zig. He got there first.

pjmlp9• 11 months ago

Only if use after free story actually gets fixed, and not by repurposing what has already existed in the C and C++ ecosystems for the last 30 years, like PurifyPlus or VC++ debug allocator.

kllrnohj2• 11 months ago

So the future is Zig. He got there first.

The future is many things, but a love letter to C is definitely not it.

Zig is cute and a fun hobby project that might see above average success for a hobby project. But that's about it. It doesn't address the problems people actually have with C++, not like Rust or Swift do, and it certainly isn't going to attract any attention from the Java, JavaScript, C#, Python, etc... of the world.

geon5• 11 months ago

Isn’t the whole point of an external buffer that the function won’t need to allocate?

kelnos4• 11 months ago

I have never understood libraries or imterfaces that want me to allocate buffers for their type.

That doesn't seem that odd to me. It's a trade off: more flexibility, but more manual work. Maybe I have a buffer that I've allocated that I'm not using anymore (say I have a buffer pool) and want to use it again. If the type allocates its own behind the scenes, I can't do that. Or maybe I'm working in an environment where I need to statically allocate all of my resources up-front, and can't allocate later.

The big downside is that if 90% of people are just going to allocate a buffer and pass it in, it sucks that 90% of people need to do more work and understand more minutiae when only 10% of the people actually need to. The holy grail is to give lots of flexibility, but make the simple/common case easy.

A simple improvement to this interface might be to allow the caller to pass a zero-length buffer (or Zig's version of null), and then the type will allocate its own buffer. Of course, there's still a documentation burden so people know they can do that. Another option could be to have second constructor function that takes no buffer arguments at all, which allocates the buffers and passes them to the fully-flexible constructor function.

jeroenhd4• 11 months ago

If the type allocates its own behind the scenes, I can't do that.

Isn't that the reason why Zig passes around allocators everywhere? If you're using a buffer pool, you should probably be handing out some kind of buffer pool allocator.

Requiring all allocation to have happened before execution is still a good reason to pass buffers around, but I feel like the other situations you describe can be solved by just passing the right allocators.

Galanwe2• 11 months ago

The Zig's language is really good, but the standard library is really a big work in progress, constantly shifting, missing a lot of bits, overly abstracted at some places and too low level at other places.

I would say just stay away from the standard library for now and use your OS API, unless you're willing to be a beta tester.

omgtehlion• 11 months ago

Yup, exactly how I use zig most of the time: just use good old OS apis, it is very easy to use cImports, when I'm too lazy to create zig definitions )

norir• 11 months ago

From my perspective, Zig is trying to do far too many things to ever reach a baseline of goodness that I consider acceptable. They are in my view quite disrespectful to their users who they force to endure churn at the whims of their dictator. Now that enough people have bought in and accepted that a broken tool is ok so long as it hasn't been blessed with a 1.0 all of its clear flaws can be overlooked in the hope of the coming utopia (spoiler alert: that day will never arrive).

Personally, I think it is wrong to inflict your experiments on other people and when you pull the rug out from underneath say, well, we told you it was unstable, you should't have depended on us in the first place.

I don't even understand what zig is supposed to be. Matklad seems to think it is a machine level language: https://lobste.rs/s/ntruuu/lobsters_interview_with_matklad. This contrasts with the official language landing page: Zig is a general-purpose programming language and toolchain for maintaining robust, optimal and reusable software. These two definitions are mutually incompatible. Moreover, zig is clearly not a general purpose language because there are plenty of programming problems where manual memory management is neither needed nor desirable.

All of this confusion is manifest in zig's instability and bloated standard library. Indeed a huge standard library is incompatible with the claims of simplicity and generality they frequently make. Async is not a feature that can be implemented universally without adding overhead and indirection because of the fundamental differences in capabilities exposed by the various platforms. Again, they are promising a silver bullet even though their prior attempt, in which they publicly proclaimed function coloring to be solved, has been abandoned. Why would we trust them to get it right a second time?

There are a very small number of assembly primitives that every platform provides that are necessary to implement a compiler. Load/store/mov/inc/jeq/jump and perhaps a few others. Luajit implements its parser in pure assembly and I am not aware of an important platform that luajit runs on that zig goes. I do the vast majority of my programming in lua and _never_ run into bugs in the interpreter. I truly cannot think of a single problem that I think zig would solve better than luajit. Even if that did exist, I could embed the zig code in my lua file and use lua to drive the zig compiler and then call into the specialized code using the lua ffi. But the vast majority of code does not need to be optimized to the level of machine code where it is worth putting up with all of the other headaches that adopting zig will create.

The hype around zig is truly reaching llm levels of disconnection from reality. Again, to believe in zig, one has to believe it will magically develop capacities that it does not presently have and for which there is no plan to actually execute besides vague plans of just wait.

eps• 11 months ago

Sounds mostly like a documentation issue, or the lack of thereof.

jeroenhd4• 11 months ago

Because of how many parts of Zig change every other month, writing documentation doesn't seem to be a priority with Zig. The same way the Zig tutorial is just "here's a bunch of examples of Zig code" (that doesn't always compile on the latest version of the compiler), you're expected to read the source code for a lot of Zig standard library definitions.

For a lot of simple calls, that works out pretty well, once you know all the tricks to Zig's syntax. A lot of requirements and implications are generally written out in very simple and short functions that are often logically named. Things like Allocators are pretty easy conceptually even if you probably don't want to write one yourself.

It all breaks down when you start dealing with complex concepts. Zig's new I/O system looks a lot like Java's streams and wrappers and readers and writers, all wrapping around each other to make sending encrypted text over a secure channel as simple as output.write("hello").

I think the new I/O system combined with the lack of documentation about how to use it was a mistake. I'm not even sure if expressing a typing system as complicated as this in the Zig standard library is a good idea. The entire language runs on clear, concise, short and readable methods, and the new system doesn't seem idiomatic in that way.

hardwaresofton6• 11 months ago

I'm not a Zig PM but the first obvious fix for the issues the OP wrote about is to write better documentation, including usage examples (the more the better, almost to a fault). Also doubles as a good time to reflect on whether the user is having to do too much.

If the tradeoff was absolute performance/avoiding introducing load-bearing performance-lowering abstraction I think that goal was achieved, but DX may have gone out the window.

brabel2• 11 months ago

You’re not familiar with Zig’s culture, I guess. Complain about the lack of documentation and be prepared for the flood of “just read the stdlib code” helpful comments by pretty much everyone who writes Zig right now. Because most APIs are just as hard to use as in this post (check things like HTTP and even basic file system operations) only the strongest survive.

hardwaresofton6• 11 months ago

Yeah, thinking about this attitude positively, maybe it’s a feature — if only hard core people can comfortably figure it out, you get higher quality contributions?

Not trying to imply that’s an explicit goal (probably instead just a resource problem), but an observation

virtualritz• 11 months ago

I do not think this is a viable excuse any more.

I am just editing docs now that Claude Code writes for me. I am fanatic about developer docs (and I guess an exception as I love writing them) but with a set of concise instructions for CC and some writing style examples I get 90% there, sometimes 99%.

If you believe you don't have time for the last 1--10% you should not be in charge of writing any API used by anyone but yourself. Just my two c.

keyle2• 11 months ago

That would hurt adoption. I understand things move fast but if you want people to make the switch other than hello world, it has to be at a minimum cosy. Sending them to hell and find your way out isn't a good move long term.

I tried Zig a couple of times and I got that feeling: very powerful and clever language but not really for me, I don't have the headspace, sorry. I need something I can debug after an 8 hours dayjob, a commute and having put the kids to bed. It better be inviting & fun! (Hi, C).

littlestymaar13• 11 months ago

The key problem with Zig nowadays is how much of its community and adoption is driven by anti-Rust sentiment. As a result, while Rust puts beginner onboarding and documentation at the center of its culture, as opposed to the “C neckbeard”'s culture, Zig is going the other way around.

(Loris Cro being a key community figure isn't helping in any way, and it's a good remainder that if you don't clear up your community from bullies from the beginning, they will turn your entire community to a miserable place. And that's a shame because from what I've seen, Andrew Kelley seems to be a very cool guy in addition to being very smart).

ulbu• 11 months ago

i find that zig is too oriented at doling out directives for what not to do instead of just collecting and teaching variants of how and what to do. the lack of documentation on this interface is a sore case in point.

geon5• 11 months ago

You can’t expect documentation this early. The new interface was just released.

0x696C69617• 11 months ago

Writing good docs/examples takes a lot of effort. It would be a waste considering the amount of churn that happens in zig at this point.

geon5• 11 months ago

Yes. For now, that effort is better spent writing clear test cases that can serve to illustrate the intended usage.

While tests aren’t quite as good documentation as actual documentation, they are guaranteed to not be out of date.

dns_snek5• 11 months ago

That's a false dichotomy, I'll take minimal bitrotten docs in a community wiki over no docs. There's no excuse not to at least have a stub that says "these features are evolving quickly but here are 10 open source projects that should serve as good references".

Something - anything. As much as I like Zig, I dread returning to it after a few months of being out of the loop.

DannyBee2• 11 months ago

Only in a world where features are made without thought. Documentation is not just for your users. Writing the development documentation helps you make better features in the first place.

The zig users on this thread seem to not understand this, and all seem to think documentation is a thing you write later for users when everything settles down. Or is somehow otherwise "in the way" of feature and API development speed.

That is a very strange view.

If writing developer documentation is having serious affect on your language feature velocity, you are doing something very wrong. Instead, writing it should make things move faster, becuase, at a minimum, others understand what you are trying to do, how it is going to work, and can help. Also, it helps you think through it yourself and whether what you are writing makes any sense, etc.

Yes there are people who can do this all without documentation, but there are 100x as many who can't, but will still give high quality contributions that will move you along faster if you enable them to help you. Throwing out the ability to have these folks help you is, at a minimum, self-defeating.

I learned this the hard way, because i'm one of those folks who can just stare at random undocumented messy code and know what it actually does, what the author was probably trying to do, etc, and it took years till i learned most people were not like this.

jakobnissen• 11 months ago

There is a cost to writing documentation - it takes time, which could be used to improve Zig in other areas. For code that is work-in-progress, it can make sense to not document until things are more settled.

Of course documentation is good. But if you have to prioritize either a new feature, or a critical bugfix, or documentation, you often can't have it all

hardwaresofton6• 11 months ago

This is a good point, but one could make a point for the usefulness of documentation in “thinking like a user” which is a valuable exercise.

I do very much prefer moving fast though, so I get it, docs-later is obviously a very valid way of doing things.

If someone is excited about Zig and wanted to make a difference I guess it’s now obvious where they could have outsized impact!

HeartofCPU• 11 months ago

Very good point

aDyslecticCrow5• 11 months ago

Then they should not have relead the new api at all. Why release half finnished library.

simonask17• 11 months ago

I tend to actually disagree with this attitude, because I see writing documentation as really effective "rubber-ducking". If it's hard and time-consuming to properly document, it's probably hard to use, so extra effort should be spent to actually justify the design, not least to yourself in 6 months. If you can't justify it, it's probably wrong.

wordofx2• 11 months ago

Na. It’s flat out laziness. Don’t make excuses. Either write docs or stop worrying code.

7bit2• 11 months ago

Your argument implies that good documentation is not an improvement, which of course is wrong. It also belongs to the task of improving code. Why would you move away after half-assing the API, when you can add the docs and whole-ass it instead?

crote• 11 months ago

On the other hand, not writing documentation also has a cost.

Code is rarely written from start to finish in a single session. Would you rather spend 5 minutes before you do a git commit to write down some very basic documentation, or spend an hour rediscovering it when you pick up development two weeks later?

Nobody is expecting extensive and well-polished documentation here, but is a "Flags X, Y, and Z are mandatory, buffer A is required: I tested with 10MB, smaller might be fine too" really too much to ask for?

Is the time you saved by not writing that one line worth someone else spending several hours figuring out how to do a "hello world"?

pharrington3• 11 months ago

I'm not a Zig developer, but I imagine one reason why the Zig documentation is so spartan is because the language is still young and constantly evolving. It's really hard to devote the time and energy to writing documentation when you know that what you've written will just be wrong at some uncertain point in the future.

baranul4• 11 months ago

Many would argue that Zig is not to be considered a young language anymore. It's been out for 9 plus years already.

A different answer to that question, could be that its still unstable and making breaking changes (not 1.0), thus a reluctancy to focus on documentation to a level many consider satisfactory.

8s2ngy• 11 months ago

I’m sorry, but any non-trivial Zig code gives me PTSD flashbacks of C. I don’t understand who Zig is targeting: with pervasive mutability, manual allocation, and a lack of proper sum types, it feels like a step back from languages such as Rust. If it is indeed a different way to write code, one that embraces default memory unsafety, why would I choose it over C, which has decades of work behind it?

Am I missing some context? I’d love to hear it.

ozgrakkurt3• 11 months ago

Compared to C:

Discriminated unions, error handling, comptime, defer.

Better default integer type casting, ability to choose between releaseSafe/releaseFast

And probably other things.

As for comparison to Rust, you do want very low level memory handling for writing databases as an example. It is extremely difficult to write low level libraries in Rust

simonask17• 11 months ago

I think the argument is that it is also extremely difficult to write low level libraries in Zig, just as it is in C. You will just only notice the difficulty at some later point after writing the code, potentially in production.

kstenerud2• 11 months ago

It is extremely difficult to write low level libraries in Rust

Really? I've not found it at all difficult to write low level libraries in Rust.

LAC-Tech3• 11 months ago

Zig is a systems programming language. I think that's probably who it's targeting.

People do systems programming in rust, but that's not really what most of the community is doing. And it's DEFINITELY not what the standard library is designed for.

konart• 11 months ago

People do systems programming in rust, but that's not really what most of the community is doing.

As someone who haven't done any systems programming after university: wait, what?

I was under impression that this is exactly what people where doing with Rust.(system apps, even linux kernel, no?)

If not - what do they (most if the community) are doing with Rust?

simonask17• 11 months ago

Which part of the Rust standard library are you referring to here?

As far as I can tell, it contains many, many features that are irrelevant outside of systems programming scenarios with highly particular needs.

sothatsit5• 11 months ago

I love Zig precisely because it is so similar to C. Honestly, if you don't like C, I can totally understand why you wouldn't like Zig. But I love C, and I love Zig.

Zig has become my go-to for projects where I would previously have reached for C, largely because Zig has such good compatibility with other C projects.

Rust, on the other hand, is a completely different beast. It is very different from C, and it is far more complicated. That makes it harder to justify using, whereas Zig is a very easy choice as an alternative to using C itself.

simonask17• 11 months ago

C is entirely as complicated as Rust, if your goal is to write correct software that doesn't crash all the time. It's only a syntactically simple language. Actually making anything interesting with it is _not_ simple.

flohofwoe7• 11 months ago

a lack of proper sum types

Do you consider Rust enums 'proper sum types'? If yes what are Zig's tagged unions missing?

E.g.:

    const Variant = union(enum) {
        int: i32,
        boolean: bool,
        none,

        fn truthy(self: Variant) bool {
            return switch (self) {
                Variant.int => |x_int| x_int != 0,
                Variant.boolean => |x_bool| x_bool,
                Variant.none => false,
            };
        }
    };

kllrnohj2• 11 months ago

Zig is for people who want to write C, that's really it. It's not a replacement for C++ or Rust or Go or Swift or anything "serious".

As for why you would choose it over C, because C has too many problems for even the C lovers to ignore. Zig fixes a tiny amount of them, just enough to pretend it's not problematic, but not enough to be useful in any non-hobby capacity. Which is fine, very few languages do achieve non-hobby status after all.

danieltanfh95• 11 months ago

its bad because they are mixing what was supposed to just be execution boundaries into the overall runtime engine without making it explicit how to bridge between one and another.

JaggerJo• 11 months ago

Zig would be my go to language for low level stuff.

I think fact that Zig can be used as a C/C++ cross compiler is brilliant.

latch• 11 months ago

Author here.

I finally got it working. I had to flush both the encrypted writer and then the stream writer. There was also some issues with reading. Streaming works, but it'll always return 0 on the first read because Writer.Fixed doesn't implement sendFile, and thus after the first call, it internally switches from streaming mode to reading mode (1) and then things magically work.

Currently trying to get compression re-enabled in my websocket library.

(1) https://github.com/ziglang/zig/blob/47a2f2ddae9cc47ff6df7a71...

dchest2• 11 months ago

Ha, "Don't Forget to Flush" https://www.youtube.com/watch?v=f30PceqQWko

jenadine• 11 months ago

That's why I like RAII.

tempodox• 11 months ago

Whatever happened to the principle of least surprise?

DrewADesign• 11 months ago

Unsurprisingly, it’s occasionally disregarded—seemingly when you least expect it.

selfmodruntime• 11 months ago

Going from the previous interface to what ever this is, is certainly something. Yeesh.

jedisct1• 11 months ago

The new I/O interface makes printing a simple “Hello, world!” more complicated, but once you get used to it, the design is actually very clean, versatile, and future-proof.

Since 0.15, though, I feel too dumb for Zig’s ArrayList.

simonask17• 11 months ago

Is it future-proof though? Last I saw, it relied on some yet-to-be-determined design for compiling async variants of everything that uses IO, and it was still unclear whether it was possible at all to support dynamic dispatch.

My info could be outdated - I don't follow Zig very closely, but I am curious.

kristoff_it5• 11 months ago

The Reader/Writer changes are perfectly compatible with the upcoming async I/O stuff and you won't need to change any code that just deals with streams.

No promises about potential future changes though :^)

throwawaymaths11• 11 months ago

i think hello world is actually easier? getstdout doesnt exist anymore and its equivalent is nonfailable.

xmorse• 11 months ago

If you are too dumb for it I don't have any hope

laserbeam• 11 months ago

I… will not update my zig side projects to 0.15.x. I can see why Andrew wanted to release this, and I can appreciate getting the new Io in people’s hands… but it’s merely a few weeks after merging massive amounts of breaking changes on readers and writers.

For those working on the standard library, it’s a great thing. For one like me who casually uses zig, it feels like waiting for 0.16.0 for most of the IO dust to settle is the right thing to do.

debo_• 11 months ago

If you're a language called Zig, I guess you sometimes have to Zag.

audunw• 11 months ago

Even Loris Cro one of the Zig core members, mentioned in a very recent interview that he’ll wait for all the fallout of the IO changes to settle before updating and continuing work on his project using Zig.

The future looks bright after that though. Both Andrew and Loris seem to think this is the last major breaking change, so I hope we can see a 1.0 not too long after. The biggest uncertainty now is what the consequences of (re-)introducing stack-less coroutines will be.

Galanwe2• 11 months ago

Both Andrew and Loris seem to think this is the last major breaking change, so I hope we can see a 1.0 not too long after.

Well I mean the standard library still needs a _lot_ of work IMHO. Process management for instance is at a stage where it's barely usable for simple use cases.

j1elo• 11 months ago

To convert the Stream.Reader to an std.Io.Reader, we need to call its interface() method. To get a std.io.Writer from an Stream.Writer, we need the address of its &interface field. This doesn't seem particularly consistent.

That made me think of how that change would be received in Go (probably would be discarded). They way they approach changes in extremely deep analysis and taking as much time as it needs to avoid mistakes and reach a consistent solution (or as close as possible).

This has been my favorite for a while: https://github.com/golang/go/issues/45624

4 years to decide on something relatively minor, that right now can be done with a bit of a one-liner extra work. But things need to be well thought out. Inconsistencies are pointed out. Design concerns are raised. Actual code usage in the real world are taken into account... too slow for some people, but I think it's just as slow as it needs to be. The final decision is shaping out to be very nice.

josephg3• 11 months ago

Rust is the same. It grinds my goat a little how many useful features are implemented - but only available in nightly rust. Things like generators.

But when rust ships features to stable, they’re usually pretty well thought through. I’m impatient. But the rust language & compiler teams probably have the right idea.

hu3• 11 months ago

I beg to differ. Rust async implementation is contentious and criticized often. Sometimes you just miss the mark despite pondering about it for a while. Same with Go.

dchest2• 11 months ago

This wasn't exactly the case before Go 1.0 -- it changed quite rapidly, although in a less fundamental way (for example, removing semicolons, changing error types, etc.), and they usually provided an option to transform code automatically.

It works now that way because they promised it to be stable after 1.0.

noobermin• 11 months ago

Having seen the posts about the new IO interface, I decided to steer clear from Zig. Looks like that fortuitious instinct was proven valid as this looks more and more like the verbosity from pre C++11 for different reasons but with a similar result.

This pattern (new language evolves to be as complex as the languages it was supposed to replace) seems familiar.

ivanjermakov• 11 months ago

As being one of the post authors you mentioned, I don't think such posts should scare people from trying it out. Zig team is eager to discuss and break things if a better solution is found.

Such approach is not a great fit for those who treat Zig as a future job opportunity, but for personal and small-team projects it's _already_ a neat language with clear goals and great tooling.

I'm too dumb for Zig's new IO interface

Related Stories