Why does an extraneous build step make my Zig app 10x faster?

https://mtlynch.io/zig-extraneous-build/

by ojosilva • 2 years ago

231 98 2 years ago

Rygian• 2 years ago

The TL:DR; is that the build step masks the wait for input from a shell pipe. With a side dish of "do buffered input" and then a small "avoid memory allocation for fun."

FreeFull• 2 years ago

You can easily hit a similar problem in other languages too. For example, in Rust, std::fs::File isn't buffered, so reading single bytes from it will also be rather slow.

aforty• 2 years ago

There is general wisdom about bash pipelines here that I think most people will miss simply because of the title. Interesting though, my mental model of bash piping was wrong too.

alas44• 2 years ago

We are two!

Joker_vD3• 2 years ago

There were several reasons why pipes were added to Unix, and the ability to run producer/consumer processes concurrently was one of them. Before that (and for many years after on non-Unix systems) indeed the most prevalent paradigm were to run multi-stage pipelines with the moral equivalent of the following:

    stage1.exe /in:input.dat /out:stage1.dat
    stage2.exe /in:stage1.dat /out:stage2.dat
    del stage1.dat
    stage3.exe /in:stage2.dat /out:result.dat
    del stage2.dat

jakogut• 2 years ago

Pipes are so useful. I find myself more and more using shell script and pipes for complex multi-stage tasks. This also simplifies any non-shell code I must write, as there are already high quality, performant implementations of hashing and compression algorithms I can just pipe to.

fuzztester• 2 years ago

Sometimes you want the intermediate files as well, though. For example, if doing some kind of exploratory analysis of the different output stages of the pipeline, or even just for debugging.

Tee can be useful for that. Maybe pv (pipe viewer) too. I have not tried it yet.

adql• 2 years ago

...how ? It's called pipe, not "infinitely large buffer that will wait indefintely till the command ends to pass its output further"

zamfi• 2 years ago

Can’t speak for OP, but one might reasonably expect later stages to only start execution once at least some data is available—rather than immediately, before any data is available for them to consume.

Of course, there many reasons you wouldn’t want this—processes can take time to start up, for example—but it’s not an unreasonable mental model.

hawski6• 2 years ago

I know this about Unix pipes from a very long time. Whenever they are introduced it is always said, but I guess people can miss it.

Though now I will break your mind as my mind was broken not a long time ago. Powershell, which is often said to be a better shell, works like that. It doesn't run things in parallel. I think the same is to be said about Windows cmd/batch, but don't cite me on that. That one thing makes Powershell insufficient to ever be a full replacement of a proper shell.

lylejantzi3rd• 2 years ago

Pipe, |, was also commonly used as an "OR" operator. I wonder if the idea that you could "pipe" data between commands came later.

arp242• 2 years ago

Usually mental models develop "organically" from when one was a n00b, without much thought, and sometimes it can take a long time for them to be unseated, even though it's kind of obvious in hindsight that the mental model is wrong (e.g. one can see that from "slow-program | less", and things like that).

m000• 2 years ago

That is called a sponge!

  SPONGE(1)                          moreutils                         SPONGE(1)

  NAME
         sponge - soak up standard input and write to a file

  SYNOPSIS
         sed '...' file | grep '...' | sponge [-a] file

  DESCRIPTION
         sponge reads standard input and writes it out to the specified file.
         Unlike a shell redirect, sponge soaks up all its input before writing
         the output file. This allows constructing pipelines that read from and
         write to the same file.

shiomiru• 2 years ago

DOS also has a "pipe", which works exactly like that. (Obviously, since DOS can't run multiple programs in parallel.)

Symmetry• 2 years ago

My first guess involved caching but I was thinking about whether the binary itself had to be read from disk or was already cached in RAM. Great linux-fu post.

OJFord2• 2 years ago

I was so confused about why this mattered/made such a difference - until I went back and re-read from the top: OP does the benchmark timing in `main`, in the Zig app under test.

If you don't do that, if you use the `time` CLI for example, this wouldn't have been a problem in the first place. Though sure you couldn't have compared to compiling fresh & running anyway, and at least on small inputs would've wanted to do the input prep first anyway.

But I think if you put the benchmark code inside the DUT you're setting yourself up for all kinds of gotchas like this.

hawski6• 2 years ago

I don't want to belittle the author, but I am surprised, that people using a low-level language on Linux wouldn't know how Unix pipelines work or that reading one byte per syscall is quite inefficient. I understand that the author is still learning (aren't we all?), but I just felt it is a pretty fundamental knowledge. At the same time author managed to have better performance that the official thing had. I guess many things feel fundamental in the retrospect.

mtlynch4• 2 years ago

Author here.

Thanks for reading!

I am surprised, that people using a low-level language on Linux wouldn't know ... that reading one byte per syscall is quite inefficient.

In my defense, it wasn't that I didn't realize one byte per syscall was inefficient; it was that I didn't realize that I was doing one syscall per byte read.

I'm coming back to low-level programming after 8ish years of Go/Python/JS, so I wasn't really registering that I'd forgotten to layer in a buffered reader on top of stdin's reader.

Alex Kladov (matklad) made an interesting point on the Ziggit thread[0] that the Zig standard library could adjust the API to make this kind of mistake less likely:

I’d say readByte is a flawed API to have on a Reader. While you technically can read a byte-at-time from something like TCP socket, it just doesn’t make sense. The reader should only allow reading into a slice.
Byte-oriented API belongs to a buffered reader.

[0] https://ziggit.dev/t/zig-build-run-is-10x-faster-than-compil...

joelfried• 2 years ago

Thank you for sharing it!

Articles like this are how one learns the nuances of such things, and it's good for people to keep putting them out there.

hawski6• 2 years ago

Zig certainly needs more work. That part is more on familiarity with Zig and how intuitive it is or isn't.

In any case I would recommend anyone investigating things like that to run things through strace. It is often my first step in trying to understand what happens with anything - like a cryptic error "No such file or directory" without telling me what a thing tried to access. You would run:

$ strace -f sh -c 'your | pipeline | here' -o strace.log

You could then track things easily and see what is really happening.

Cheers!

amluto• 2 years ago

As a very general tip:

execution time: 438.059µs

That’s a rather short time. (It’s a lot of cycles, but there are plenty of things one might do on a computer that take time comparable to this, especially anything involving IO. It’s only a small fraction of a disk head seek if you’re using spinning disks, and it’s only a handful of non-overlapping random accesses even on NVMe.)

So, when you benchmark anything and get a time this short, you should make sure you’re benchmarking the right thing. Watch out for fixed costs or correct for them. Run in a loop and see how time varies with iteration count. Consider using a framework that can handle this type of benchmarking.

A result like “this program took half a millisecond to run” doesn’t really tell you much about how long any part of it took.

vlovich1234• 2 years ago

FWIW, the current version of the code (i.e. with the buffered reader), on my machine at least, runs identically fast with and without the tmp file.

Here's a possibly more detailed reason as to why https://news.ycombinator.com/item?id=39764287#39768022.

nebulous1• 2 years ago

I feel like you are underestimating how many fundamental misunderstandings people can have (including both of us) even though they have deep understanding of adjacent issues.

0x000xca0xfe2• 2 years ago

There are many fundamental things People Should Know™, but we are not LLMs that have ingested entire libraries worth of books.

Exploratory programming and being curious about strange effects is a great way to learn the fundamentals. I already knew how pipes and processes work, but I don't know the Ethereum VM. The author now knows both.

cpuguy83• 2 years ago

It's one of those things that you don't usually need to think about, so you don't.

Not too long ago I hit this same realization with pipes because my "grep ... file | sed > file" (or something of that nature) was racey.

I took the time to think about it and realized "oh I guess that's how pipes would _have_ to be implemented".

boffinAudio• 2 years ago

This deleterious effect is a factor in computing. We deal with it every few years: kids graduate, having ignored the history of their elders in order to focus on the new and cool - and once they hit industry, find that they really, really should have learned POSIX or whatever.

Its not uncommon. As a professional developer I have observed this obfuscation of prior technology countless times, especially with junior devs.

There is a lot to learn. Always. It doesn't ever stop.

franciscop• 2 years ago

This is exactly what surprised me as well. I'm literally now learning in depth WebStreams[1] in JS (vs the traditional Node Streams) and I've seen too many times the comparison of how "pipe() and pipeTo() behave just like Unix's pipes |". Reading this article makes me think this might not be the best comparison, specially since for many webdevs it's the first time for approaching some CS concepts. OTOH, the vast majority of webdevs don't really need to learn WebStreams in-depth.

[1] https://exploringjs.com/nodejs-shell-scripting/ch_web-stream...

LAC-Tech• 2 years ago

Most people have gaps somewhere in their knowledge. I learned very on, as a general superstition, to always try and batch things that dealt with the world without like file writes, allocations, network requests etc. But for years I had no idea what a syscall even was.

0x000xca0xfe2• 2 years ago

Also a great reminder to always benchmark with different data set sizes.

masto• 2 years ago

I realize one needs a catchy title and some storytelling to get people to read a blog article, but for a summary of the main points:

* This is not about a build step that makes the app perform better

* The app isn't 10x faster (or faster at all; it's the same binary)

* The author ran a benchmark two ways, one of which inadvertently included the time taken to generate sample input data, because it was coming from a pipe

* Generating the data before starting the program under test fixes the measurement

meowface• 2 years ago

Another semi-summary of the core part of the article:

"echo '60016000526001601ff3' | xxd -r -p | zig build run -Doptimize=ReleaseFast" is much faster than "echo '60016000526001601ff3' | xxd -r -p | ./zig-out/bin/count-bytes" (compiling + running the program is faster than just running an already-compiled program)
When you execute the program directly, xxd and count-bytes start at the same time, so the pipe buffer is empty when count-bytes first tries to read from stdin, requiring it to wait until xxd fills it. But when you use zig build run, xxd gets a head start while the program is compiling, so by the time count-bytes reads from stdin, the pipe buffer has been filled.
Imagine a simple bash pipeline like the following: "./jobA | ./jobB". My mental model was that jobA would start and run to completion and then jobB would start with jobA’s output as its input. It turns out that all commands in a bash pipeline start at the same time.

anonymous-panda4• 2 years ago

That doesn’t make sense unless you have only 1 or 2 physical CPUs with contention. In a modern CPU the latter should be faster and I’m left unsatisfied by the correctness of the explanation. Am I just being thick or is there a more plausible explanation?

karmakaze• 2 years ago

I would definitely classify the title as clickbait because the app didn't go "10x faster".

underdeserver• 2 years ago

If I were trying to optimize my code, I would start with loading the entire benchmark bytecode to memory, then start the counter. Otherwise I can't be sure how much time is spent reading from a pipe/file to memory, and how much time is spent in my code.

Then I would try to benchmark what happens if it all fits in L1 cache, L2, L3, and main memory.

Of course, if the common use case is reading from a file, network, or pipe, maybe you can optimize that, but I would take it step by step.

jedisct1• 2 years ago

This is exactly the thing that feels obvious once you realize it, but that can be puzzling until you don't.

john-tells-all• 2 years ago

This is an excellent writeup, with interesting ideas and clear description of actions taken. My idea of pipelines, also, was flawed. Well done!

Nothing to do with Zig. Just a nice debugging story.

WalterBright4• 2 years ago

Back in college, a friend of mine decided to learn how to program. He had never programmed before. He picked up the DEC FORTRAN-10 manual and read it cover to cover.

He then wrote a program that generated some large amount of data and wrote it to a file. Being much smarter than I am, his first program worked the first time.

But it ran terribly slow. Baffled, he showed it to his friend, who exclaimed why are you, in a loop, opening the file, appending one character, and closing the file? That's going to run incredibly slowly. Instead, open the file, write all the data, then close it!

The reply was "the manual didn't say anything about that or how to do I/O efficiently."

TylerE4• 2 years ago

I firmly believe that teaching how to idiomatically do both character and line oriented file IO should be the first thing any language tutorial teaches, almost. Just as soon as you’ve introduced enough syntax.

PoignardAzur• 2 years ago

FWIW, the Epitech cursus starts your very first C programming lesson by making you write a "my_putchar" function that does a "write" syscall with a single character. Then you spend the next few days learning how to create my_putnbr, my_putstr, etc, using that single building block.

I think that's the right choice, by the way. Baby developers don't need to learn efficient I/O, they need to learn how you make the pile of sand do smart things.

And if you've spent weeks doing I/O one syscall per character, getting to the point you write hundreds of lines that way, the moment some classmate shows you that you can 100x your program's performance by batching I/O gets burned in your memory forever, in a way "I'm doing it because the manual said so" doesn't.

everforward2• 2 years ago

I would argue that they should be teaching how file IO works at a low level at some point (preferably as one of the first "complicated" bits).

Everybody should, at some early point, interact with basic file descriptors and see how they map to syscalls. Preferably including building their own character and line oriented abstractions on top of that, so they can see how they work.

I'm convinced that IO is in the same category as parallelism; most devs understand it poorly at best, and the ones who do understand it are worth their weight in gold.

WalterBright4• 2 years ago

The FORTRAN-10 manual was not a tutorial, it was a specification.

michael1999• 2 years ago

If you want create something like the pipe behaviour the author expected (buffer all output before sending to the next command), the sponge command from moreutils can help.

styfle• 2 years ago

By adding a benchmarking script to my continuous integration and archiving the results, it was easy for me to identify when my measurements changed.

This assumes CI runs on the same machine with same hardware every time, but most CI doesn’t do that.

boesboes• 2 years ago

And that the hardware is not overbooked. I found that my ci/cd runs would vary between 8 and 14 minutes (for a specific task in the pipeline, no cache involved) between reruns.

And it seemed correlated to time of day. So pretty sure they had some contention there.

Edit: and that was with all the same cpu’s reported to the os atleast

mcguire2• 2 years ago

There seems to be a small misunderstanding on the behavior of pipes here. All the commands in a bash pipeline do start at the same time, but output goes into the pipeline buffer whenever the writing process writes it. There is no specific point where the "output from jobA is ready".

The author's example code, "jobA starts, sleeps for three seconds, prints to stdout, sleeps for two more seconds, then exits" and "jobB starts, waits for input on stdin, then prints everything it can read from stdin until stdin closes" is measuring 5 seconds not because the input to jobB is not ready until jobA terminates but because jobB is waiting for the pipe to close which doesn't happen until jobA ends. That explains the timing of the output:

    $ ./jobA | ./jobB
    09:11:53.326 jobA is starting
    09:11:53.326 jobB is starting
    09:11:53.328 jobB is waiting on input
    09:11:56.330 jobB read 'result of jobA is...' from input
    09:11:58.331 jobA is terminating
    09:11:58.331 jobB read '42' from input
    09:11:58.333 jobB is done reading input
    09:11:58.335 jobB is terminating

The bottom line is that it's important to actually measure what you want to measure.

mtlynch4• 2 years ago

Author here.

Thanks for reading!

All the commands in a bash pipeline do start at the same time, but output goes into the pipeline buffer whenever the writing process writes it. There is no specific point where the "output from jobA is ready".

Right, I didn't mean to give the impression that there's a time at which all input from jobA is ready at once. But there is a time when jobB can start reading stdin, and there's a time when jobA closes the handle to its stdout.

The reason I split jobA's output into two commands is to show that jobB starts reading 3 seconds after the command begins, and jobB finishes reading 2 seconds after reading the first output from jobA.

dsm9000• 2 years ago

This post is another example of why I like zig so much. It seems to get people talking about performance in a way which helps them learn how things work below today’s heavily abstracted veneer

Why does an extraneous build step make my Zig app 10x faster?

Related Stories