I Improved My Rust Compile Times

https://benw.is/posts/how-i-improved-my-rust-compile-times-by-seventy-five-percent

by excsn • 2 years ago

139 107 2 years ago

eikenberry• 2 years ago

https://web.archive.org/web/20240320042828/https://benw.is/p...

Very cool, still kind of disappointing the author was only able to get incremental compilation down to 4.5s, no matter the baseline. Hopefully Cranelift lands in stable at some point.

dclowd9901• 2 years ago

That’s pretty good. Our frontend app built with TS with SWC recompiles in about that time.

boxed3• 2 years ago

Depends on the code size though. Is this 1k lines? 10k lines? 100k lines?

Just looking at the blog this seems like a fully static site that should be able to be built in milliseconds. Some input from the developer here on what takes all this time would be interesting.

denysvitali3• 2 years ago

Their benchmarks are based on a machine with 72GB of RAM and another with 128GB of RAM - not that it does matter, but I'm really curious if they're outliers or if I should bump my computer specs. That's a lot of Electron web apps running in parallel!

mixedCase• 2 years ago

Some people need it, some don't. When I upgraded to a 7950X I was unable to find an excuse to buy more than 32GB. "That one time a script compiled clang by default with LTO and OOMed" was not a good enough reason.

If you use a lot of VMs, it comes in handy. If you do a lot of heavyweight compilation in parallel, same thing. But at least on Linux, popping 3 VS Code projects and 4 more browsers doesn't even hit 50% utilization.

rakejake2• 2 years ago

Funnily enough, RAM I find I can use no matter how much I have, especially now that you have LLMs. I can run a quantised Mixtral 8x7B on 64GB RAM.

But the 7950X is grossly underutilised, I don't know what to do with it.

db48x• 2 years ago

Yea, very workload dependent. Reposurgeon needed hundreds of gigabytes of ram to convert the GCC repository from SVN to Git, which was a lot of fun, but most of the time I would be fine with 32GB.

yaantc• 2 years ago

You can try running tests in parallel with the memory sanitizer (MSAN) enabled. MSAN uses a lot of memory, and on a 64 GB machine I have to reduce the amount of parallelism not to trigger Linux OoM killer.

    MSAN: https://github.com/google/sanitizers/wiki/MemorySanitizer

pcthrowaway• 2 years ago

My understanding is that processor performance is more important for rust compilation than available memory, but ultimately the atrocious compilation times are why I struggled so much with Rust.

lolinder3• 2 years ago

Definitely outliers. The Steam hardware survey [0] is obviously not a perfect measurement, but in its February results only ~3% had 64GB and only ~0.3% had more than that.

I recently upgraded to 64GB to run quantized LLM models, but when I'm not using an LLM I rarely use more than 16GB, much less 32GB.

[0] https://store.steampowered.com/hwsurvey

IshKebab7• 2 years ago

I have 32GB on my Linux laptop (plus 12GB swap) and regularly run out of RAM with just a handful of VSCode's open (it's generally the language servers using all the RAM) and a couple of dozen Firefox tabs (Firefox seems to be much worse than Chrome RAM-wise). If I run my actual work project, which needs a few gigs then it often just completely runs out of memory and hard reboots.

Never have the same problem on Windows even though I only have 16GB there. Feels like Linux is just really bad at memory management. That may be skewing the Steam survey since almost everyone there will be using Windows.

On Linux I wish I had 64GB.

nindalf5• 2 years ago

I benchmarked Rusts performance on 2, 4, 8 and 16 cores on several widely used crates (https://arewefastyet.pages.dev/). Performance generally improved with the number of cores, but memory made no difference. All of these crates topped out at around 1GB and additional memory didn’t improve performance.

So no, 72GB or 128GB makes no difference.

denysvitali3• 2 years ago

Yes, maybe I didn't make it clear but I'm pretty sure this is not impacting for Rust. I was just wondering if it's normal to have these specs since my computer has probably "just" 32GB of RAM

drewtato• 2 years ago

I agree that memory size isn't going to matter much, but memory speed should be noticeable, especially on large projects.

lifthrasiir• 2 years ago

If you are like me and opening too many browser tabs (and other applications, of course), you need a lot of RAM. My desktop has only 48 GB of RAM right now and a typical VM usage is around 80--100 GB. I will probably go for 128 GB or possibly 192 GB of RAM for the next desktop.

Rust builds, in comparison, don't actually reach that far, mainly because it is proportional to the maximum number of parallel jobs. In my experience, 48 GB was more than plenty for 8 parallel jobs if I wasn't doing anything else.

nextaccountic• 2 years ago

Note: Discord User @PaulH(and a couple others) let me know about this bug that prevents the server build from using the dependencies built for the previous compilation of the server/webassembly frontend build due to changes in the RUSTFLAGS. You can fix this by having cargo-leptos > 0.2.1 and adding this to you Cargo.toml under your Leptos settings block. I did not test with this enabled.
[package.metadata.leptos]
separate-front-target-dir = true

This option is deprecated since cargo-leptos 0.2.3 (now it's enabled unconditionally), it's going to be removed in 0.3

https://github.com/leptos-rs/cargo-leptos/pull/216

https://github.com/leptos-rs/cargo-leptos/issues/217

https://github.com/leptos-rs/cargo-leptos/commit/b0c19a87cff...

rui3145• 2 years ago

The sold linker is no longer commercial software; it's been relicensed under the MIT license.

mhaehnel2• 2 years ago

But it is no longer updated, is it?

Thanks anyway for the work on sold.

umanwizard• 2 years ago

Sold was just mold under a commercial license. Mold is still maintained and now licensed permissively.

rui3145• 2 years ago

But it is no longer updated, is it?

Essentially, yes, but that’s because Apple’s new linker is so much faster than it was. You can use it instead.

stefanos82• 2 years ago

Rui, if I ever get rich, I will seriously give you 1 million US dollars, $100K the minimum, for everything you have done so far, with all your projects.

That's the least I can do to show my appreciation for your incredible work; I can't thank you enough brother!

I also wish to see that chibicc compiler plus the book completed. I want to learn everything around such topics.

rui3145• 2 years ago

Thanks, I really wish you become a billionaire :)

dan_can_code3• 2 years ago

As a front-end developer dabbling with rust, the compiler being slow is not a problem for me. Whilst I appreciate the benchmark, it's not likely I'll have 72GB of ram laying around to speed up the process.

Yes, hot module reloading is nice and quick. But it reloads with errors included.

The primary benefit of the rust compiler, at least from my point of view, is telling me what's wrong, where. It's a worthwhile sacrifice for a few seconds of my time, when at the end of addressing all of the obvious errors, I have something that works. I find this miles better than HMR.

boxed3• 2 years ago

I use Elm at work, and the confidence from really strict compilation and typing is very worth it.

With that said, Elm is super fast to compile too, which is very nice.

pjmlp5• 2 years ago

I have a couple of projects where it is measured in minutes, on a dual core with 8GB and a SSD.

If only devs with beefy machines are able to use Rust properly, that hinders adoption.

guappa• 2 years ago

Developers in USA: "nobody in the entire world has less than 32GB of RAM!"

enqk• 2 years ago

This has definitely be one big reason for me why I did not get into rust. I don’t want to have to buy a new machine just to get the compile times I’m used to

Starlevel004• 2 years ago

The primary benefit of the rust compiler, at least from my point of view, is telling me what's wrong, where.

Incredible! Rust developers have invented "IDEs"!

dan_can_code3• 2 years ago

I'm not sure if you're familiar with front-end developer tooling or the development process, but my editor will not tell me what error a browser is going to throw.

boxed3• 2 years ago

Your sarcasm missed the mark. You meant "static typing".

oefrha3• 2 years ago

You’re a front end developer but you don’t need to make small tweaks all the time? “At the end of addressing all of the obvious errors, I have something that works.” is irrelevant to large swaths of front end code, which is basically always in a “working” state (displays something without error), but needs to be continuously expanded and refined.

TFA talks about reload times of 20-30s before their optimization, on a powerful computer. It’s certainly not “a few seconds” especially for people on less powerful hardware. That’s a ridiculous wait time for, say, tweaking a CSS class.

I find this comment highly suspect.

dan_can_code3• 2 years ago

JavaScript errors only occur at runtime. It's very common, especially in legacy codebases, for issues to just fly under the radar, even if they are obvious to the seasoned dev. There is a reason why one of the most popular JS books is called 'Javascript: the good parts'.

With rust, it's very refreshing in that it's idiot proof, which JavaScript is not. CSS is not relevant, since it is not actual programming.

nindalf5• 2 years ago

So no, 72GB or 128GB makes no difference.

b33j0r2• 2 years ago

Working on a couple of ideas that flip that logic on its head, but you’re more likely to be right because of finite time.

The thing about rust is that the overhead is all upfront.

Not to be sacrilegious, but bleeding at the move or &wtf borrow boundary is a price one pays to have confidence, subsequently.

Your cms/customer acronym probably doesn’t need it, but you’ll love the —fast option in some stacks that merely means “compile the rust extension.”

Plus… it’s a gainful hobby. I’m fine if other people have hobbies that don’t resemble x86 in 1998 on slackware ;)

b33j0r2• 2 years ago

I will never understand who hated my innocent and light-hearted thought that much. Hope it was worth making me go, “aw” for no reason.

doubloon2• 2 years ago

People are impatient. I expect in a few years someone will”solve” this problem by removing the borrow checker and then by removing the unsafe keyword, then allowing dynamic linking and shared libraries…. And this person will be lauded as a genius. And some group of cranky bitters will make a new language with a new borrow checker,,,, cycle repeats

pkolaczk• 2 years ago

The borrow checker is not responsible for even 1% of compilation time.

The elephant in the room is monomorphisation and macro expansion. Those generate a ton of code that LLVM has to process. In return you get much faster programs at runtime.

winrid2• 2 years ago

The coolest thing about this to me IMO is the comparison between the two CPUs.

Is there a project that publishes benchmarks compiling projects across different CPUs? I know passmark is kinda the go-to but this would be cool.

Someday I'll feel the need to upgrade from a 2700x...

canu7• 2 years ago

Phoronix does a lot of CPU benchmarks, including code compilation, but mostly focused on Linux. More result are also available in the OpenBenchmark page, which is also part of the same project. Take a look at the timed compilation test-suit: https://openbenchmarking.org/suite/pts/compilation

oynqr• 2 years ago

Have you looked at https://openbenchmarking.org?

winrid2• 2 years ago

I haven't, thanks!

beeb• 2 years ago

Super interesting! I would have loved to see included in the potential solutions the use of sccache too

SushiHippie• 2 years ago

Looking at some benchmarks online and the 7900x always beat the 5950x even though the 5950x has more cores.

But in this article the 7900x performed worse than the 5950x, is it because of the core count, or are there some other factors?

daghamm• 2 years ago

First time I hear about mold.

The performance looks good, but are there any plans for any other improvements such as LTO?

rui3145• 2 years ago

mold supports LTO out of the box.

rob74• 2 years ago

I really wish people wouldn't use AI-generated images for blog posts, it's so distracting! I clicked the link wanting to read the article, but instead spent several minutes looking at the image to find errors: what's shown on the screen (apparently it's "Jot Commeditin") looks like something between a hex dump, git blame, and occult incantations written in a long-forgotten alphabet. Probably you need a keyboard like the one on the desk (with keys arranged in a fractal pattern) to write something like that. And a mouse with a cable going nowhere...

doubloon2• 2 years ago

He got three out of four of “manic pixie dream girl”

deanWombourne• 2 years ago

I did the same! Then didn't read the article.

klabb34• 2 years ago

Same, but what’s worse is that people never put in the caption that it’s AI nor what tool is used. Transparency goes a long way.

KingOfCoders• 2 years ago

From my experience, faster SSD have a suprising strong effect for some (simple) languages (probably not Rust), like Go [0]

[0] https://www.octobench.com/

klabb34• 2 years ago

Is it an unpopular opinion to say 75% of “a lot” is still “a lot”, plus now you have to keep track of the knobs and constantly monitor for and be conscious about not wrecking build times as you’re maintaining and developing.

I’ve found in 10+ years of software development that speed of iteration cycle is highly correlated with productivity. Compile times is not the only input into this cycle time, but it’s a big one, and importantly, it’s within the control of the language tooling itself to solve. The human idle attention time of 1-2 seconds should be the gold standard to strive for, even if not always achievable.

There seems to be quite a bit of cope around Rust build times in the community, which was natural a few years ago (a lot of people used to “blame” llvm, but it doesn’t seem to be as big of a culprit) but things are different now, no? Given the maturity, growing ecosystem and corporate investment, I would expect incremental build speedup to be prioritized, and steadily improving. But clearly it isn’t moving very fast in that direction. So why not?

IshKebab7• 2 years ago

I’ve found in 10+ years of software development that speed of iteration cycle is highly correlated with productivity.

I agree, though in practice with many Rust programs that iteration cycle is actually not "edit, compile, run", it's "edit, save, wait for rust-analyzer to update" which is generally much quicker (even if it is sometimes still a little slow).

In most cases, Rust's very strong type system means you spend a lot less time actually running your program.

There are definitely exceptions though. E.g. if you're making a web app and editing CSS or whatever then the Rust compiler isn't going to tell you if you've got the colour wrong.

a lot of people used to “blame” llvm, but it doesn’t seem to be as big of a culprit

It still is; there was a post here really recently where someone broke down the time spent in various phases. LLVM is still the vast majority, though apparently that is partly due to Rust generating a lot of work for it.

I would expect incremental build speedup to be prioritized, and steadily improving.

It is.

But clearly it isn’t moving very fast in that direction. So why not?

Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.

rockwotj• 2 years ago

I agree, though in practice with many Rust programs that iteration cycle is actually not "edit, compile, run", it's "edit, save, wait for rust-analyzer to update" which is generally much quicker (even if it is sometimes still a little slow).

This is one area that I think Dart did a good job on. They support both a JIT (I am not sure if it’s tiered) and AOT modes of compiling, the JIT targeted for development and AOT for production. This is the ultimate extreme of optimizing for both use cases, obviously it’s a ton of work, but I would love more languages to be like this. (I believe the JIT mode works directly on the source so there isn't an intermediate compilation step like Java, but it’s been a few years since I’ve used Dart)

[1] https://dart.dev/overview#native-platform

physicsguy• 2 years ago

It is a single file, but when you include header files which get dropped in they can get to many hundreds of thousands of lines of code in practice per corresponding .o file.

flohofwoe2• 2 years ago

In most cases, Rust's very strong type system means you spend a lot less time actually running your program.

That just sounds wrong to me TBH. IME it never pays off to 'front-load' too much, you might get a working implementation of your original idea without ever running the code, but that original idea will almost certainly be "wrong" once you see it in action and you just wasted a lot of time agonizing over your type structure which turned out to implement a quickly outdated idea.

Nothing beats quick iteration on a running program when it comes to productivity.

Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.

If the a C++ file includes any C++ stdlib header it will also also become very large (tens of thousands of lines of gnarly template code for common headers like <vector> or <string>). If the unit of compilation is a crate in Rust it should actually be faster to build than typical C++ projects which often put the implementation for one class into one source file (and each of those source files is expanded to tens of thousands of lines of code after includes are resolved).

C mostly doesn't suffer from this problem though, since C header typically are only a couple dozen or hundred lines of API declarations.

dist1ll3• 2 years ago

in practice with many Rust programs that iteration cycle is actually not "edit, compile, run", it's "edit, save, wait for rust-analyzer to update" which is generally much quicker

Rust-analyzer is struggling significantly on larger repos, to the point of being barely usable.

ladyanita223• 2 years ago

Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.

What do you mean exactly here?

klabb34• 2 years ago

In most cases, Rust's very strong type system means you spend a lot less time actually running your program.
There are definitely exceptions though.

I mean.. I know what you mean but this is kinda what I meant with cope. Putting “running code” aside as an edge case seems like a dangerous moving-the-goalpost kind of Rustism that kinda grew out of the “Rust is so safe that if it compiles it’s correct”-meme. Once we’re doing serious and boring good old development, it’s a redeeming quality at best, imo. Not to mention that rust analyzer also suffers slowdowns in similar ways.

LLVM is still the vast majority

That’s interesting. So why the meager numbers from cranelift then, judging by the post? Granted, I’m not doing Rust day-to-day anymore so I’ve been ootl.

Because Rust's unit of compilation (a crate) is very large. In C/C++ it's a single file.

Isn’t this the same for Go? Sure, it’s a simpler language but we’re talking many, many times on similar sized projects (wish I could link to something but unfortunately anecdotally only). Is it monomorphization, macros, or just everything combined?

pjmlp5• 2 years ago

In C++ it can be a module nowadays, which is similar to being a crate, yet it is faster, from my clang/vc++ experience.

nindalf5• 2 years ago

Is it an unpopular opinion to say 75% of “a lot” is still “a lot”

I don’t know if it’s unpopular, but it may be inaccurate. OP claims that they improved by 75%, so you probably meant “25% of a lot is still a lot”.

I would expect incremental build speedup to be prioritized, and steadily improving. But clearly it isn’t moving very fast in that direction. So why not?

Is this assessment based on something?

For what it’s worth, people have been working on it for years, with varying approaches and levels of success. The recent spate of articles you’ve seen is precisely because these approaches are bearing fruit. Several folks have worked on making a previously single threaded frontend multithreaded, others have added an option to swap the LLVM backend out for a much faster one (cranelift), and yet others are working on moving to a faster linker by default.

The net gain from all of these once they land could be as much as 30–50%, depending on the project, mode of compilation (clean or incremental) and hardware (single/multiple cores). That is a pretty fantastic improvement, considering the whole year improvements in the last 4 years have been 7%, 17%, 13% and 15%, for an overall speed gain of 37%.

If all of this doesn’t sound impressive, I’d encourage you to read nnethercote’s annual reports which explain how much effort went into making compile times faster. 2023’s report - https://nnethercote.github.io/2024/03/06/how-to-speed-up-the...

Ultimately, all of these improvements might not be enough for the people who need HMR to be productive. That’s possible. But the idea isn’t to get to a point where every line of code written in the world is Rust. There’s plenty of Rust development happening today. When the compile times drop as hoped, more people join and that grows the ecosystem.

dist1ll3• 2 years ago

I have a feeling many of the common multithreading optimizations carry the same flaw as incremental compilation: they give you good speed-ups in the short term, but potentially mask deeply flawed architectural decisions. Essentially improving perf by digging yourself into a deeper grave.

klabb34• 2 years ago

you probably meant “25% of a lot is still a lot”

Yeah exactly. This was afaik a fairy small sized application (where the majority was probably framework code) which went from 20s to 5s for incremental compilation on a very beefy desktop rig. Those absolute numbers are important to grok the situation.

That is a pretty fantastic improvement, considering the whole year improvements in the last 4 years have been 7%, 17%, 13% and 15%, for an overall speed gain of 37%.
If all of this doesn’t sound impressive, I’d encourage you to read nnethercote’s annual reports which explain how much effort went into making compile times faster

It’s not at all that it’s unimpressive. It’s abundantly clear that extremely bright minds are putting a ton of effort into it, consistently and over time. I’ve worked personally with some of these people and they are insanely talented and knowledgeable… Which is precisely why I’m concerned that we’re not seeing the big boosts that comes with picking low hanging fruit, perhaps there isn’t any left. Optimization tends to yield lower results over time (although the recent advances are great to see and I wish it continues). From the outside though, it appears like either inherent constraints that limit how far you can go, or at least initial compiler design reasons that would be cost-prohibitive to change.

dgellow• 2 years ago

I’m not sure I understand why editing an html template requires to recompile the rust project. I implemented a static site generator in rust for my own projects, html templates are simple liquid template files, it’s instant to recompile them. Same for stylesheets. Is the author generating the html from rust directly? If yes that sounds like a painful strategy

richrichie• 2 years ago

The older i get the more i am convinced that there-ain’t-no-free-lunch applies widely, beyond financial markets.

Software development: the art of redistributing aggregate lifecycle pain; who bears what, when and how much.

OJFord• 2 years ago

i am convinced that there-ain’t-no-free-lunch applies widely, beyond financial markets

That's amusingly ironic, since it doesn't originate in financial markets anyway!

https://en.wikipedia.org/wiki/No_free_lunch_theorem

rbalint• 2 years ago

75% is really good even if it requires changing the toolchain. Have you tried https://github.com/firebuild/firebuild ? It is a caching accelerator and it can cache the linking and the buildscripts, too in Rust builds. It can make your builds more than 90% faster, especially with low cores counts. https://balintreczey.hu/blog/improve-build-time-of-rust-java... The Mac port is experimental, though.

I Improved My Rust Compile Times

Related Stories