Speeding up C++ build times

https://www.figma.com/blog/speeding-up-build-times/

by skilled • 2 years ago

165 203 2 years ago

chipdart• 2 years ago

This blog entry is highly disappointing. The Sigma blog post reads as if they reinvented the wheel with basic information that is not only widely known and understood but it also featured in books published decades ago.

The blog post authors would do well if they got up to speed on the basics of working with C++ projects. Books such as "Large scale C++ vol1" by John Lakos already cover this and much more.

SuperV1234• 2 years ago

Shameless plug from my talk: https://youtube.com/watch?v=PfHD3BsVsAM

rileymat22• 2 years ago

It could be my bias but it seems a lot of inexperienced developers no longer read comprehensive books on topics but survive on Google, stack overflow, some documentation with examples/simple tutorials, blog posts and now Gpts.

All are useful tools but they are very poor in eliminating unknown unknowns like a book would.

bdowling• 2 years ago

You think only inexperienced developers have stopped reading books?

LeSaucy• 2 years ago

I use c++ on the daily and find that ccache and an m1/2/3 cpu go a very long way to reducing build times.

MathMonkeyMan3• 2 years ago

Not sure why this was downvoted. It's true that ccache and build parallelization (e.g. icecream) can grease the wheels enough that builds are no longer a dev cycle bottleneck.

What the article is about, though, is changing the source code so that it is intrinsically faster to compile. At some point you say "this program isn't complicated, why does it take so long to compile?" Then you start looking at unnecessary includes, transitive includes, forward declarations, excessive inlining, etc.

rjkaplan• 2 years ago

I'm guessing the comment was downvoted because the suggestions are mentioned in the first paragraph of the article...

After trying a few stopgap solutions—like purchasing M1 Maxs for our team—build times gradually reverted to their original pace; Ccache and remote caching weren’t enough either.

scheme271• 2 years ago

Yeah, I got 15 minute build times down to under 30s using ccache. Doesn't help with cold rebuilds but once you have a cache, it really does help things significantly.

ErikCorry4• 2 years ago

Ccache is great but then change one central .h file and you are back to square one.

gpderetta10• 2 years ago

Ccache + distcc (or similar caching/distributed build solutions) work very well.

juunpp• 2 years ago

Just another self-promotion blog post with near zero information density. "Behold, the vastness of our vanity, regurgitating old news like we just discovered something new." Do these posts really help with hiring?

CLION also highlights unused includes, nothing new here. Use a good IDE. A networked ccache also does wonders if your org allows it.

Slow build otherwise stem from a combination of: a) lack of proper modules in C++ (until recently) and b) unidiomatic or just terrible code bases. To help with the latter, hide physical implementations (PIMPL for class state, forward declarations for imports), avoid OOP-style C++ above all, minimize use of templates, design sound and minimal modules. No rocket science.

MathMonkeyMan3• 2 years ago

Three times I've joined a team that has a substantial C++ codebase, and three times I've been tempted to use libclang based tooling to automate changes, or at least to identify patterns that could be changed.

This article, while not the nerdy deep dive I'd like, does touch on what happens when you try to do that. You realize that the C++ standard library is really complicated, that your existing code is really fucked up, and that libclang is too limited a tool. You end up writing a XSLT engine in hacked up python, but by a different name.

[LibTooling][1] is probably The Right Thing ("in C++", as the article says), but I never spent the time to get it working.

Somebody write a DSL for C++ inspection and transformations that uses LibTooling as a backend. I bet there are many, but none close at hand.

edit: [this][2] is close...

[1] https://clang.llvm.org/docs/LibTooling.html

[2] https://clang.llvm.org/docs/LibASTMatchersTutorial.html#inte...

stefan_• 2 years ago

Is there anyone else that gets unreasonably angry at stuff like PIMPL? It is truly the most braindead, bereft of sense activity in this world. There was a comment in one of the many Rust threads that called C++ a respectable language now but then things like PIMPL snap you right back into the wasteland it is.

wakawaka287• 2 years ago

PIMPL is an elegant solution to multiple problems. Idk what you could possibly have against it besides the extra work involved. I don't think any language has solved the fundamental problem of hiding details better than PIMPL does.

Maxatar7• 2 years ago

No, maybe the acronym is ugly, sounds like pimple? But other than that the use of the technique is invaluable to writing stable ABIs which in turn makes distribution of C++ libraries a lot easier.

bun_terminator2• 2 years ago

I have never used or seen pimpl in my life, and c++ is all I do

iainmerrick7• 2 years ago

Yes! Why should I need to do extra work, messing up runtime performance by adding a pointer indirection, just to improve compilation time a bit?

In C, idiomatic C, you can forward-declare a struct and the functions that operate on it, and you don't need any indirection at runtime. C++ has plenty of nice features, and in general I'd reach for it rather than C, but for some reason it can't do that!

Scubabear68• 2 years ago

I find it hard to believe that this post indicates that C++ build times are proportional to included bytes, period.

I haven’t used C++ in quite a while, but aren’t templates a big part of this issue?

flohofwoe9• 2 years ago

It becomes believable when you consider that your own code is just a very tiny appendix dangling off the end of a massive chunk of included data. For instance just including <vector> results in a 24kloc compilation unit of gnarly template code in Clang with C++23:

https://www.godbolt.org/z/G18WGdET5

...add <string> and <algorithm> and you're at 45kloc:

https://www.godbolt.org/z/Whv73YPYh

...and those numbers have been growing steadily by a couple thousand lines in each new C++ version.

Multiply this with a few thousand source files (not atypical with the old 'clean code' rule to prefer small source files, e.g. one file per class), and that's already dozens to hundreds of million lines of code the compiler needs to process on a full rebuild, all spent on compiling <vector> over and over again.

TL;DR: the most effective way to improve build times in C++ is to split your project into few big source files instead of many small files (either manually, e.g. one big source file per 'system', or let the build system take care of it via 'unity' or 'jumbo' builds).

mgaunard4• 2 years ago

Except that doesn't improve the time of iterative builds, which are the only ones that really matter to software development.

gpderetta10• 2 years ago

From a quick test, including every single header in the standard C and C++ library is 180k loc in an otherwise empty .cc file and compiling it with g++13 in C++23 mode takes 1.6s on my machine. Not amazing, but not terrible either.

I also don't believe in tiny instantiation units, but when compiling real code, parsing the headers themselves is not necessarily the bottleneck.

wifijammer• 2 years ago

I really hate the repetition from separating out header and definition files so I've been writing my whole codebase headers only.

I feel like this kills my compile time but I'm not sure how to fix it. Precompiled headers?

flohofwoe9• 2 years ago

If you have all your application code in headers you'll get the fastest build times (for full rebuilds at least) by including all headers into a single main.cpp file and build just that, since that way there is no redundant code for the compiler to build at all.

Of course the downside is that every tiny code change triggers a full rebuild then, but it's quite likely that the most time is spent in the linker anyway, so maybe worth a try.

runevault2• 2 years ago

I think I've heard this called a Unity build where there's a precompile step that just dumps everything into a single file and compiles that so it doesn't have to re-include everything at different compilation units (when I first heard the term I got confused because it was in a game dev context but had nothing to do with the Unity engine lol).

asvitkine• 2 years ago

Except then there's no parallelism since you're only building one file. Ideally you'd split it into N files to take advantage of multiple cores, but then you have to decide how to split it...

ranger_danger• 2 years ago

ccache is one solution, or a script/IDE plugin that will create both the header/definition from a signature you provide?

pjmlp3• 2 years ago

C++20 modules.

Currently mostly usable on VC++ vlatest, and clang 17, with clang 18 bringing in support for c++23 import std (VC++ already does it).

Sadly GCC is still far behind, not to mention all the other ones still catching up with C++17.

mgaunard4• 2 years ago

DIWYDU sound like a better tool than IWYU.

snypehype46• 2 years ago

Coincidentally in the project I'm currently working I managed to reduce our compile times significantly (~35% faster) using ClangBuildAnalyzer [1]. The main two things that helped were precompiled headers and explicit template instantiations.

Unfortunately, the project still remains heavy to compile because of our use of Eigen throughout the entire codebase. The analysis with Clang's "-ftime-trace" show that 75-80% of the compilation time is spent in the optimisation stage, but not really sure what to do about that.

[1] https://github.com/aras-p/ClangBuildAnalyzer

celrod3• 2 years ago

What's the trick with explicit template instantiations? Including them in the precompiled header?

logicchains• 2 years ago

You leave them uninstantiated in the header file and explicitly instantiate them in the .cc file (only works if there's a small number of possible instantiations).

kjksf• 2 years ago

I wrote about how I keep build times sane in SumatraPDF at https://blog.kowalczyk.info/article/96a4706ec8e44bc4b0bafda2...

The idea is the same: reduce the duplicate parsing of .h files.

I don't use any tools, just a hard-core discipline of only #include'ing .h in .cpp files.

The problem is that if you start #include'ing .h in .h, you quickly start introducing duplication that is intractable, for a human, to avoid.

On another note: C++ compiler should by default keep statistics about the chain of #include's / parsing during compilation and dump it to a file at the end and also summarize how badly you're re-parsing the same .h files during build.

That info would help people remove redundant #include's.

But of course even if they do have such options, you have to turn on some flags and they'll spam your build output instead of writing to a file.

sfpotter3• 2 years ago

How much of a speedup did you get switching over to Rob Pike style includes?

cpeterso• 2 years ago

I had to look this up:

Rob Pike’s rule is that header files should not include other header files; they should only document which other header files they depend on so programmers can directly include those other header files in their .c files.

https://bytes.com/topic/c/answers/217074-rob-pikes-simple-in...

johannes1234321• 2 years ago

On another note: C++ compiler should by default keep statistics about the chain of #include's / parsing during compilation and dump it to a file at the end and also summarize how badly you're re-parsing the same .h files during build.

Not exactly that, but do you know clang's -ftime-trace and tools like https://github.com/aras-p/ClangBuildAnalyzer which help analyzing where time is actually spent? (In small repeated headers I don't see much of a problem, but they of course may contain not so small things ...)

touisteur• 2 years ago

Any idea whether gcc is roadmapping including this?

electroly• 2 years ago

If you use a "unity" build (all .cpp files concatenated into one compilation unit) with the normal '#pragma once' or guard in the headers, then you can be certain that every header file is parsed only once, with no discipline needed. CMake can do this for you with an option. You lose the ability to do incremental builds of individual changed files, but it may be fast enough that you don't care.

For my own personal projects, I just use ccache and precompiled headers. It's good enough for me. I don't want to have to apply "hard-core discipline" to my projects.

Maxatar7• 2 years ago

On another note: C++ compiler should by default keep statistics about the chain of #include's / parsing during compilation and dump it to a file at the end and also summarize how badly you're re-parsing the same .h files during build.

Clang does offer something very close to this, and if you use it you'll find parsing the same/duplicate header files contributes on the order of microseconds to the overall compile time. Your article is from 1989, which is likely before compilers implemented duplicate header file elimination [1], but nowadays all C++ compilers optimize the common #ifndef/#define header guard as well as #pragma once so that they entirely ignore parsing the same file over and over again.

[1] https://gcc.gnu.org/onlinedocs/cpp/Once-Only-Headers.html

WalterBright7• 2 years ago

True, but when you're compiling a.c and then b.c, the .h files get reparsed all over again.

jay-barronville• 2 years ago

As much as I hate long compilation times, I also value code discoverability, readability, and consistency. Your (i.e., Rob Pike’s) strategy seems like a nightmare to me. I love Rob, but I can’t follow this rule.

nextaccountic• 2 years ago

Shouldn't precompiled headers mitigate that?

eru• 2 years ago

Oh, the joys of using a language that thinks (automated) copy-and-paste coding is the same as a module system.

WalterBright7• 2 years ago

Having written, benchmarked, and maintained C and C++ compilers for decades, I know why the compiles are slow:

1. phases of translation

2. constant rescanning and reparsing of .h files

3. cannot parse without doing semantic analysis

4. the preprocessor has its own tokens - so you gotta tokenize the .h file, do the preprocessing, convert it back to text, then tokenize the text again with the C/C++ compiler. This is madness. (Although with the C compiler I did manage to merge the preprocessor lexer with the compiler lexer, this made it the speed champ.)

This experience fed into D which:

1. uses modules instead of .h files. No matter how many times a module is imported, it is lexed/parsed/semanticed exactly once.

2. module semantics are independent of who/what imports them

3. no phases of translation

4. lexing and parsing is independent of semantic analysis

kazinator3• 2 years ago

Having done some casual benchmarking recently, I found that GCC is about 15 times slower when optimizing than when not. In both situations, the compiler is scanning the same header files, so that activity is bound up within the 1/15th of the optimized compilation time.

It used to be a common wisdom that the character-level processing of code took the most time. Just like the old floating-point is slow; always use integer when possible.

Also note that the ccache tool greatly speeds up C and C++ builds. Yet, the input to ccache is the preprocessed translation unit! When you're using ccache, none of the preprocessing is skipped. ccache hashes the preprocesed translation unit (plus compiler command line options and the path of the compiler executable and such) in an intelligent way and the checks its cache. If there is a hit, it pulls the .o out of its cache, otherwise it invokes the compiler on the preprocessed translation unit.

If most of the time were spent in preprocessing, a much more modest speedup would be observed with ccache.

zerr• 2 years ago

Isn't `#pragma once` helpful for avoiding reparsing headers?

uecker3• 2 years ago

This is the same for C and C++ and C compile times are dramatically shorter than C++.

Also doing semantic analysis during parsing should save time compared to having additional tree walking later (and does in my experience).

pajko• 2 years ago

Precompiled headers eliminate some of these issues, if the usage is right. At one of my former companies any available build optimization stuff had been added to the code base over the years by the DevOps guys without any thought about how the thing would work together, and it just got slower and slower. At the end, the initial 5-minute complete build time increased to about 35 minutes, of which about 10 minutes could be attributed to the various refactors and extremely high amount of templating, but couldn't find a cause for the extra 20-minute increase. It was about 15 minutes to test just a single change.

kazinator3• 2 years ago

In modern compilers, the scanning of tokens is a tiny fraction of the compile time.

If the header files are properly protected with inclusion guards, then at worst the contents are tokenized by the preprocessor, and not seen by anything else. But for decades now, compilers have been smart enough to recognize include guards and not actually open the file. So that is to say, when the compiler has seen a file of this form once:

  #ifndef FOO
  #define FOO
  ...
  #endif

and is asked to #include that same file again, it knows that this file is guarded by FOO. If FOO exists, it won't even open that file.

There are reasons to avoid including .h files in .h files, but you're not going to get compile time gains out of it, other than perhaps through secondary effects.

By secondary effects I mean this: when in a project you forbid .h files including other .h files, it hurts to add new dependencies into header files. When you change a header such that it needs some definition in another, you have to go into every .cpp file and add the #include. So, you find ways to reduce the dependencies to avoid doing that.

When .h files are allowed to include others, the dependencies grow rampant. You want to compile a little banana, but it depends on the gorilla, which needs the whole jungle, just to be defined. Juggling the jungle definition takes a bit of time. C++ definitions are complicated, requiring lots of processing to develop. Not as much as optimizing the banana that is the actual subject being compiled to code, but not as little as skipping header files.

highfrequency2• 2 years ago

We try to mitigate it with #ifdef guards, #pragma once etc. but in my experience those band-aids don’t solve the problem.

Why don't include guards solve the problem of duplicate parsing of .h files within one compilation unit? I believe modern compilers can entirely optimize away even the opening and scanning of the file. And even without that, modern NVMe disks are so fast I would imagine the file opens would be negligible.

Curious to hear if anyone has data on whether duplicate header file parsing is still an actual performance issue even with modern compilers, modern SSDs, and #pragma once.

greenavocado2• 2 years ago

Only a handful of commenters have mentioned "#pragma once", which is alarmingly few people considering how many C++ practitioners are out there. "#pragma once" is the obvious way to deal with this problem.

Arech3• 2 years ago

`#pragma once` isn't standardized. Even though all major modern compilers do support it (hopefully, but not guaranteed in the same way), if you care about portability to some not even very esoteric platform, it most likely uses ancient compilers (or based on ancient version of a major compiler) that might not support it. But besides that I too don't see a reason why not use a better `#pragma once` instead if include guards...

flohofwoe9• 2 years ago

That's a somewhat unrelated problem, it just protects from parsing the same header multiple times. The rule that headers should not include other header is more of an organizational thing.

You can immediately see the include complexity of each source file by looking at the top of the file, all required includes are there as a flat list. Otherwise a single #include may hide dozens, hundreds or even thousands of other includes hidden in a deep dependency tree.

It also nudges the programmer away from "every class in a tiny header/source pair" nonsense.

timvdalen• 2 years ago

Thanks for building SumatraPDF! I just switched away from Windows, but I've been a happy user for a very long time, and it's hard to get that mix of user experience just right. Really appreciate your work.

intelVISA2• 2 years ago

The entire article's premise is misguided as slow C++ build times are usually a result of inexperienced SWEs unfamiliar with the language's "fun" build rules.

Arech3• 2 years ago

In enterprise you are almost always under a pressure to deliver something asap, so some bad practices are just a natural consequence of that. They could have liked to do it differently, but then they would only be half way to delivering what they have already shipped. There are always tradeoffs.

maccard2• 2 years ago

This is the equivalent of saying C is safe to use, as long as you use it correctly, or your iPhone is fine you’re just holding it wrong.

If build times are a wide spread problem with a language, that’s a language problem not a user problem.

petermcneeley• 2 years ago

The video game industry uses bulk builds (master files) which groups all .cc s into very large single .cc files. The speedups here are like 5-10x at least. These bulk files are sent to other developers machines with possible caching. The result is 12 min builds instead of 6 hours.

josephg3• 2 years ago

It makes sense. As projects grow, the average header file is included O(n) times from O(n) different .cc files - leading to O(n^2) parsed header files during compilation. And thus, O(n^2) work for the compiler.

Merging everything into one big .cc file reduces the compilation job back to an O(n) task, since each header only needs to be parsed once.

Its stupid that any of this is necessary, but I suppose its easier to hack around the problem than fix the problem in the language.

WalterBright7• 2 years ago

Those problems are fixable with C/C++, but nobody seems to want to do it. They are fixed with dlang's ImportC. You can do things like:

    dmd a.c b.c

and it will compile and link the C files together. ImportC also supports modules (to solve the .h problems).

It's all quite doable.

gpderetta10• 2 years ago

Which compiler parses the same header file multiple times in the same translation unit? Compilers have been optimizing around pragma once and header guards for multiple decades.

edit: ok, you meant that each header is included once in each translation unit.

forrestthewoods• 2 years ago

I'm starting to believe that one static/shared library should be produced by compiling exactly one cpp file. Go ahead and logically break your code into as many cpp files as you want. But there should then be a single cpp file that includes all other cpp files.

The whole C++ build model is terrible and broken. Everyone knows n^2 algorithms are bad and yet here we are.

sfpotter3• 2 years ago

Everyone: "O(n^2) algorithms are bad."

Also everyone: "Just do the stupidest thing in the shortest amount of time possible. We'll fix it later."

shanemhansen• 2 years ago

Chrome supported this for a long time and it really helped small developers outside of Google be able to build chrome without specialist machines.

But that feature was pulled by the chrome team with the stated justification being that since C++ guaranteed different things (iirc around variable scopes outside of a namespace) for one file vs multiple files, supporting the jumbo build option meant writing some language that was "not C++".

Unfortunate.

Shorel2• 2 years ago

When "best" is the enemy of "good enough"

celrod3• 2 years ago

If using cmake, you can try a unity build. https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.htm...

You can also specify a `-DUNITY_BUILD_BATCH_SIZE` to control how many get grouped, so you can still get some parallelism. However, I think it'd be more natural to be able to specify number of batches (e.g. `nproc`) than their size.

Code bases may need some updating to work.

WalterBright7• 2 years ago

How it works with D is you can do separate compilation:

    dmd -c a.d
    dmd -c b.d
    dmd a.o b.o

or do it all in one go:

    dmd a.d b.d

Over time, the latter became the preferred method. With it, the compiler generates one large .o file for a.d and b.d, more or less creating a "pre-linked" object file. This also means lots of inlining opportunities are present without needing linker support for intermodule inlining.

Twirrim• 2 years ago

sqlite uses something like this approach too, and there are additional optimisation advantages from keeping everything in a single file:

https://sqlite.org/amalgamation.html

    Over 100 separate source files are concatenated into a single large file of C-code named "sqlite3.c" and referred to as "the amalgamation". The amalgamation contains everything an application needs to embed SQLite.
    
    Combining all the code for SQLite into one big file makes SQLite easier to deploy — there is just one file to keep track of. And because all code is in a single translation unit, compilers can do better inter-procedure and inlining optimization resulting in machine code that is between 5% and 10% faster.

anybodyz• 2 years ago

Many of them use this tool: https://www.fastbuild.org/docs/home.html

It's was built and is regularly maintained by a Riot Games principal C++ architect, and automatically compiles files in large "unity" chunks, distributes builds across all machines in an organization, and creates convenient Visual Studio .sln files, and XCode projects. It's also all command line driven and open source.

This is industrial strength C++ builds for very large rapidly changing code bases. It works.

Shorel2• 2 years ago

Nice, that's like doing the link phase before the compilation.

jupp0r3• 2 years ago

Bazel + remote workers yields a great user experience with small infrastructure footprint per developer, but requires quite a bit of work to initially set up. You get reproducible builds, caching of test results and blazingly fast CI as a side effect.

bburnett44• 2 years ago

lol I initially read this as remote people workers and was super confused

jack_pp• 2 years ago

when you use tensorflow from python you're basically using c++

allpaca• 2 years ago

C++ is evolving so much, however I don't understand this thing: why people continue to develop AI projects with Python? I'd choose C++ instead...

bpicolo• 2 years ago

Because they're python libraries that just wrap C and C++. All the performance upside with better ergonomics

oivey• 2 years ago

Putting this another way, people use Python because it makes it way easier to compose together the underlying C++ code. Composition and polymorphism in C++’s static type system is rather weak.

Of course there is also the relative succintness of Python and other advantages, too.

VHRanger• 2 years ago

Because you need a dynamic language to do rapid iteration.

You don't want to recompile, re-parse a 12GB parquet file, etc. everytime you try a new parameter in a model.

diath• 2 years ago

Ever since I tried -ftime-trace in Clang to improve build times in a project a while ago, I've been very conscious about using forward declarations wherever possible, however I wish we had proper module support that actually worked well, having to keep this in mind whenever writing new code just so your project doesn't take forever to compile sucks, this shouldn't even be something we have to keep in mind in 2024.

gosub100• 2 years ago

I just checked and distcc [1] is still a thing. It's been around a while now, I remember messing with it in about 2010. It allows you to parallelize your builds across multiple machines. Not as relevant now with todays multicore CPUs, but if you or your employer can't afford to buy you a work station, distcc might be your answer.

[1] https://www.distcc.org/

hi-v-rocknroll• 2 years ago

Use icecream or sccache. sccache optionally supports distributed builds.

https://github.com/mozilla/sccache/blob/main/docs/Distribute...

saidinesh52• 2 years ago

Back in college days, I had to use icecream to compile some of KDE codebase. Our laptops were quickly thermal throttling and simply couldn't handle so much code. Add to that crazy summers here, icecream was a godsend.

gpderetta10• 2 years ago

Even if you have a beefy workstation, if your projects has thousands of translation units (i.e. any moderately large c++program), distcc (and ccache) still help significantly.

Unfortunately linking becomes the bottleneck.

curiousgal• 2 years ago

We use Incredibuild at work.

delta_p_delta_x2• 2 years ago

A big improvement in compile times with modern C++ can be had with C++20 standard modules.

This requires extremely bleeding-edge toolchains, though: VS 2022 17.10, Clang 18, and GCC 14.

sfpotter3• 2 years ago

I'm under the impression "big" here might mean a 10-20% improvement if you're lucky. Good to back this claim up with hard data.

celrod3• 2 years ago

I'm still waiting for clangd support, e.g. [0] before trying modules. But maybe I should just try it, as at least one person reports that it already works [1].

[0] https://github.com/clangd/clangd/issues/1293 [1] https://github.com/snu-sf-class/swpp202401/issues/21

jupp0r3• 2 years ago

You know you are talking about C++ when "bleeding edge" is compilers from 3 years ago which implement the version before the current standard from 4 years ago....

IAmLiterallyAB• 2 years ago

Clang 18 came out in March and GCC 14 isn't even out. So those are bleeding edge by just about any metric

Longhanks• 2 years ago

Well, ISO C++ is a standard, and the standard being finished does not mean the implementation being finished.

Also, modules is probably one of the biggest changes in terms of work required in build systems, compilers and tooling since at least C++11.

delta_p_delta_x2• 2 years ago

Clang 18 was released this March, barely two months ago[1]; VS 2022 17.10 is still in preview[2], and GCC 14 isn't even out yet. This can't be nearer the bleeding edge.

[1] https://github.com/llvm/llvm-project/releases/tag/llvmorg-18...

[2] https://learn.microsoft.com/en-us/visualstudio/releases/2022...

bdd8f1df777b• 2 years ago

I'm still waiting for CMake support to mature.

Kelteseth2• 2 years ago

I recently created https://arewemodulesyet.org/ to track module adoption in the c++ ecosystem. I used how often a vcpkg manifest file changed to get a rough estimate how popular a library is.

lastgeniusua• 2 years ago

Thank you very much for this resource! Reading through the article and the discussion here I was really surprised why nobody discussed using actually existing modules, this clarifies how far that is from a solution.

You should really post this as a separate Show HN story!

"Estimated finish by the year 5134" made me chuckle

Kelteseth2• 2 years ago

And we just improved the waiting time by about 1000 years, by adding flux to the list of supported projects. But sarcasm aside, yes this is a bit of criticism of the c++ ecosystem that moves a bit too slow. Many libraries pride themselves with only using c++11/14.

teunispeters• 2 years ago

(silly answer) use Visual C++. I've been coding across MacOS, Linux, and Windows 11, with roughly comparable machines in a lot of ways (all Intel). Visual C++ while awkward for compatibility - is very very fast for builds, and not bad for IDE interface for rapid fixes.

More serious - I moved to CMake presets and with that came a lot of cache optimization - including parallel builds. MacOS is now almost as fast as Windows for build, and Linux/gcc not far behind. Windows C++ seems to have the lowest modern feature compatibility, followed by MacOS/Clang, with Linux/recent GCC being the most complex. A lot of the newer features seem to add a lot to the build time..

... mind I've been working with C++ only for the last few months, and C for many years before, so consider it a beginner post in a lot of ways. Still, it was interesting to explore, and I'll be continuing to explore - I haven't yet enabled ccache for instance which I suspect will improve a lot.

jupp0r3• 2 years ago

My experience is the exact opposite. Moving a multimillion line C++ code base from msbuild to CMake/ninja on Windows cut the build time in half.

Chrome got even better speedups I believe by building with clang/ninja on Windows.

Bazel is where the real benefits lie by reusing other people's (or CI machine's) partial build artifacts via a centralized cache and by avoiding to run tests that are not affected by code changes.

spacemanspiff01• 2 years ago

How does bazel work with cmake builds?

gpderetta10• 2 years ago

Haven't used MSVC in a long while, but at last place where we were doing multi compiler builds, MSVC was always slower by far.

setheron• 2 years ago

Isnt this what header guards are for or the #pragma include

feverzsj• 2 years ago

On the contrary, I prefer putting everything in header file, and put related headers in same source file just for compile. It's basically unit build but without its drawbacks.

ramon1562• 2 years ago

The longest acronym I know is WYSIWYG (What You See Is What You Get)

8372049• 2 years ago

TIMTOWTDI (There Is More Than One Way To Do It)

ramon1562• 2 years ago

The longest acronym I know is WYSIWYG (What You See Is What You Get)

stockhorn• 2 years ago

I've also tried to optimize c++ compile times on large projects a few times. I never got IWYU working properly and I always hated the fact that I still have to care about header files at all. Then I switched to doing rust full time, which made all the fiddling with header files obsolete. This felt amazing. But now I'm facing the same problem, slow compile times :). Only this time I have to rely on the compiler team to make the improvements and I cant do much on my side AFAIK.

Nereuxofficial• 2 years ago

Well that's not quite true. You can do a few things: 1. Reduce dependencies and features of dependencies 2. Use a faster linker like mold 3. Use a faster compiler backend like cranelift(if possible) 4. Use the parallel compiler frontend(again if using nightly is possible) 5. Use sccache to cache dependencies But i do get what you mean. Especially in CI the build times are often long

fanf2• 2 years ago

Split up crates so your compilation units are smaller.

pietroppeter• 2 years ago

It is written purely in Python and does not use Clang, which makes it fast to run—usually in just a couple of seconds.

Oh, the irony

intelVISA2• 2 years ago

Indeed, reading this article is supposed to promote Figma's engineering team for 'solving' a solved problem: it does quite the opposite.

ErikCorry4• 2 years ago

My tip: Increase the size of .cc files. Since each .cc file is including tens of thousands of .h lines, you should not allow developers to check in < 100 line .cc files.

The OP was seeing build times increase faster than loc. Probably someone on the team likes small .cc files.

choppaface• 2 years ago

But translation units / small .cc files can build in parallel and cached, and with multi-core thus it's desirable to have many small translation units. Except of course when there's eventually one large translation unit that needs everything and then link time dominates ...

The article emphasizes a common issue about headers.. X-Code and Visual Studio work around this to some extent with pre-compiled headers, something that can be really hard to set up in ccache. If Figma's whole team is using macs (they mention getting everybody macbooks?) then I wonder if they could just switch to X-Code and use built-in pch support. While that introduces a dependency on X-Code :( maybe their whole C++ stack will get effectively re-written in the next couple of years anyways?

ErikCorry4• 2 years ago

OK, when you only have as many .cc files as you have CPU cores, you should stop making them bigger. This is not an issue for big projects.

The caching of compilation fails if you touch a central .h file. When you work on projects like that you start dreading a change to the .h files because development slows to a crawl as the compile times explode.

I worked on V8 and over time the .cc files got smaller and the build times got much worse. Some people felt this was neater, but if you are not in an office with 20 beefy workstations using distcc the effect is brutal.

tinganho• 2 years ago

One thing that crossed my mind when I was looking at the TypeScript compiler. Was that it was parsing header "d.ts" files even though it wasn't used in source. Although, it had some references of a type in a small main function.

Iirc, I think this was how most compilers did. The downside is that transitive deps can easily explode. Thus, compiling a super small main function can takes seconds.

I did suggest a solution to just lazily parse/check symbols if they are encountered in source. Instead of when including a type, you have to parse all the transitive header files of the file that defines the type.

jsbus• 2 years ago

I was thinking about applying at Figma a couple days ago. Seeing the engineering culture portrayed in this post, I do not any more.

saidinesh52• 2 years ago

Funnily enough, this topic was all they asked in one of the job interviews I had when I was younger.. they shared a bunch of sources files and asked me where/what I could do to improve the compilation speeds.

I didn't realize how big the impact of these little changes were (only include what you use, forward declare as much as you can, PIMPL etc..) until I worked on a large codebase in that company.

Gupie• 2 years ago

Link times are the main pain for our edit, build, debug cycle.

Liquid_Fire2• 2 years ago

Have you tried mold [1]? In my experience, it makes a dramatic difference to link times.

[1] https://github.com/rui314/mold/

nrclark3• 2 years ago

Back when I was doing C++ more often, I got a big speedup on my builds (and also smaller binaries) by using C-style forward declarations for everything that I possibly could, including class methods. Method definitions / template instantiations would be placed into a corresponding .cpp file.

I know that it's unpopular in a world of header-only libraries, but it really does make a dramatic difference to build-time. Especially if your code is heavy on templates.

chrisjj• 2 years ago

the perennial issue of slow build times

No, that's long build times.

andersa3• 2 years ago

It's frustrating to see the C++ committee spend year after year on pointless new over engineered libraries instead of finally fixing the compile times. On a high level view, with only one change to the language, we could entirely eliminate this problem!

Consider the following theoretically simple change:

A definition in a file may not affect headers included after it. If you want global configuration, define them at the project level, or in a header included by all files that need it.

i.e. we need to break this construct:

    #define MY_CONFIG=1
    #include "header_using_MY_CONFIG.h"

Thats really all we need to do to completely eliminate the nonsense that is constant re-parsing of headers and turn the build process into a task graph where each file is processed exactly one time, and each template is instantiated exactly one time, the intermediate outputs of which can be fully cached.

Most real-world large projects already practice IWYU meaning they are already fully compatible with this.

There are some videos by Jonathan Blow on how this is exactly how the Jai compiler is so fast. Why must we still suffer with these outdated design decisions from 50 years ago in C++? Why can't the tech evolve?

/end rant

affgrff2• 2 years ago

Aren't there modules since c++20 which solve this problem?

andoma• 2 years ago

Yes, but we are not exactly there yet: https://arewemodulesyet.org/

Edit: Saw after posting this was already posted in a top level comment.

andersa3• 2 years ago

Modules are a massively over engineered "solution" to the problem that require significant refactoring to actually make use of them. Have you tried to properly use modules (i.e. create ones in your software, not just import std)? It's super clunky and still hardly usable.

I doubt we'll see Unreal Engine get any benefit from that in a long time for example. It could be so much better, working fully automatically with almost all existing code so long as you use IWYU, which is already standard for large projects where this is needed the most.

James_K• 2 years ago

I used to think the same thing myself, but now I'm not sure that would solve much. The problem you describe is really just a matter of caching. I believe you should be able to process the tokens of a header to determine all the identifiers in it, at which point you can model the header compilation as a memoised function from the definition of those identifiers to the compiled file. So for your example, every include of the header where MY_CONFIG=1 could re-use the same results.

The real issue is just that C++ compilers are horrendously slow. They have been designed with the intention of producing fast executables rather than compiling quickly. Think of Rust, which has a high degree of structure in its compilation process and so a high degree of opimisability, yet it still suffers slow compilation due to its usage of a C++ compiler backend.

I think this really because C++ builds are fundamentally unstructured. Rather than invoking a compiler on the entire build directory and letting it handle each file, it is invoked once for every file in a way that might be non-trivial. Improving the build process almost always comes at the

Beyond that, C++ developers simply do not care about slow compilations times. If they did, they wouldn't be using C++. It's my personal theory that C++ as a language has effectively self-selected a user-base that is immensely tolerant of this kind of thing by driving off anyone who isn't.

andersa3• 2 years ago

I think this really because C++ builds are fundamentally unstructured. Rather than invoking a compiler on the entire build directory and letting it handle each file, it is invoked once for every file in a way that might be non-trivial.

True, this would also need to be fixed. Compilation would need to become a single process that can effectively manage concurrency and share data.

anthk• 2 years ago

Use ccache and stop reinventing the wheel.

demilich• 2 years ago

Use C++20 modules, take a look at this project: https://github.com/infiniflow/infinity

taylodl• 2 years ago

Back when I was doing a lot of C++ development, pre-compiled headers had become the solution to this problem. If memory serves me correctly, I think it was Symantec who first implemented this feature. Dramatically sped-up compilation teams on processors that were only running at 33 MHz - or slower!

Though I was an early adopter of STL at the time, it still hadn't enjoyed widespread use yet. Are templates now the problem with pre-compiled headers? If so, then that should be the problem we tackle.

DylanSp• 2 years ago

I'm curious how bad the build times were; minutes, tens of minutes, hours? I didn't see any absolute times in the post, just percentages.

bandika• 2 years ago

A bit of a shameless plug, I've created this Rust tool to visualise header dependencies (major contributor to build time) of C++ projects. https://gitlab.com/Akeras/header-dependency-visualiser

threesmegiste• 2 years ago

Whenever i tend to learn c++ this build compile issues prevent me. İ really dont understand how such a important language could still have tooling problems. Especially for beginners. I dont wanna learn a language in browsers, and dont wanna download visual studio. Is there a way or a light a complete solution or app to resolve this issue for beginners. İ really wanna learn this language to understand computer. Dont love rust or carbon or go or zig? any language called system language. I wanna learn c++. İ know basic, pascal yes you guessed right i am old

rurban• 2 years ago

The best thing would be to convert to C and use tcc. Other options would be dlang or go.

Cfront with tcc would sound nice enough to try

Speeding up C++ build times

Related Stories