LLVM 16.0.0 Release

https://discourse.llvm.org/t/llvm-16-0-0-release/69326

by mfiguiere • 3 years ago

208 69 3 years ago

discourse.llvm.org

dotnwat• 3 years ago

Very excited about this release. Just tried out clang-format on a code base with a lot of concepts, and finally, the formatting looks great.

ianlevesque• 3 years ago

It’s not a headlining feature but I love clang-format. Fast and reasonable output choices.

camel-cdr3• 3 years ago

Notably, function specialization optimization has been finally enabled by default, although it still seems to only be capable of constant propagating variables of register size [1].

The heuristics are, understandably so, somewhat conservative [2], and I presume there are some projects that could benefit from enabling `-mllvm -funcspec-for-literal-constant`, raising `-mllvm funcspec-max-clones=3` ad lowering `-mllvm -funcspec-min-function-size=100`.

[1] https://godbolt.org/z/jKo7K3jPd

[2] https://github.com/llvm/llvm-project/blob/main/llvm/lib/Tran...

pitaj• 3 years ago

I wonder if function specialization will help Rust, C++, etc as much as it will benefit C.

celrod• 3 years ago

I don't think it is likely to help C++, as templates and `auto` arguments (in C++20) were already specializing. These probably cover most of the cases you'd encounter there.

I'm less familiar with Rust, but I'd have assumed trait implementations specialize there as well.

charcircuit3• 3 years ago

The notable changes for me are faster linker times and C++17 (gnu++17) now being the default standard used.

TwentyPosts• 3 years ago

Any idea of how much faster we're talking here?

charcircuit3• 3 years ago

Around 10% faster.

In D130810 the speedup is given as:

Performance linking some programs with --threads=8 (glibc 2.33 malloc and mimalloc):

- clang: 1.05x as fast with glibc malloc, 1.03x as fast with mimalloc

- chrome: 1.04x as fast with glibc malloc, 1.03x as fast with mimalloc

-internal search program: 1.08x as fast with glibc malloc, 1.05x as fast with mimalloc

In D133003 the speedup is given as:

Speed-up with mimalloc and --threads=8 on an Intel Skylake machine:

- clang (Release): 1.27x as fast

- clang (Debug): 1.06x as fast

- chrome (default): 1.05x as fast

- scylladb (default): 1.04x as fast

Speed-up with glibc malloc and --threads=16 on a ThunderX2 (AArch64):

- clang (Release): 1.31x as fast

- scylladb (default): 1.06x as fast

Iwan-Zotow• 3 years ago

Did you try mold?

weitzj2• 3 years ago

Xtensa support (esp32). Will be interesting how this will be for Rust and Zig support for esp32

https://github.com/llvm/llvm-project/tree/main/llvm/lib/Targ...

https://www.phoronix.com/news/LLVM-Xtensa-Backend

https://github.com/espressif/llvm-project/issues/4#issuecomm...

https://github.com/ziglang/zig/issues/5467#issuecomment-1465...

jenadine• 3 years ago

But still not in that release afaik

weitzj2• 3 years ago

Edit:

More discussions:

https://github.com/espressif/llvm-project/issues/4#issuecomm...

Llvm is here:

https://github.com/llvm/llvm-project/tree/release/16.x/llvm/...

But I don’t know if there should be lld as well…

jws• 3 years ago

I see that after 50 years, C has finally gained an official null pointer value and type! "nullptr" and "nullptr_t".

That wouldn't be so interesting, but it highlights the complexity of specifying a language and implementing a specification. For what sounds at first hearing a lot like "#define nullptr 0" the Clang release notes identify 7 syntactic cases where the C standard and the C++ standard disagree about whether "nullptr" should be acceptable in a location. Clang takes each side about equally and it sounds like they are waiting for the standards groups to fix the standards.

pmarin• 3 years ago

It's a very common pattern to memset to zero structs with pointers to initialize them. What happend now with this new "feature"?

camel-cdr3• 3 years ago

Nothing changes. memseting a struct to zero has always been and still is implementation defined behavior, as there are no guarantees on the bit representation of a null pointer. [1]

The C11 standard says that a null pointer constant is defined as follows: "An integer constant expression with the value 0, or such an expression cast to type void , is called a null pointer constant."

The new definition is: "An integer constant expression with the value 0, such an expression cast to type void , or the constant nullptr are all a null pointer constant."

The reason for adding nullptr, is because NULL is roughly defined as a macro that expands to "0" or "(void*)0". This causes trouble with the addition of type detection trough _Generic and now typeof.

[1] https://c-faq.com/null/machnon0.html

saagarjha3• 3 years ago

Consider using = { 0 } instead.

kevin_thibedeau• 3 years ago

The null address isn't necessarily 0. When a literal 0 is assigned to a pointer it can be magically changed to a platform's true null. Memset() doesn't have this behavior. This isn't relevant on any modern systems but it is an affordance of the C standard. That is the true significance of nullptr. It obsoletes the magic 0 literals.

asb3• 3 years ago

I've written up a tour of some of the RISC-V related changes in the LLVM 16 development cycle https://muxup.com/2023q1/whats-new-for-risc-v-in-llvm-16

celeritascelery• 3 years ago

Thanks! Who is driving the development of RISC-V in LLVM? Are there companies investing in it or is it individual contributors?

asb3• 3 years ago

Primarily companies at this point, with some ongoing input from individual contributors or universities / research organisations. I'm at Igalia (an open source software consultancy), so contributions from me and my colleagues are paid for by companies within the ecosystem. Vendors like SiFive and Rivos have particularly active contributors on the RISC-V backend, but there's many more.

nevi-me• 3 years ago

From organisations, I understand LowRISC to be involved https://github.com/lowrisc. I'd also presume Espressif, as they have a growing number of risc32 chips.

saagarjha3• 3 years ago

https://github.com/llvm/llvm-project/commits/main/llvm/lib/T... is a start ;) Not all of these have affiliations listed but I can tell you that contributors hail from SiFive, Rivos, Google, and various universities.

diimdeep• 3 years ago

Is it slower again ? [1]

[1] https://www.npopov.com/2020/05/10/Make-LLVM-fast-again.html

akvadrako2• 3 years ago

Yeah, LLVM is a dog, about twice as slow and GCC and 20 times slower than TCC:

   gcc-11.2  -O0     0.02
   clang-12.0.1 -O0  0.04
   tcc               0.002

I wish TCC was still being maintained and supported by modern toolchains.

[1] https://forum.nim-lang.org/t/8677

camel-cdr3• 3 years ago

What makes you thing TCC isn't maintained anymore?

ComputerGuru• 3 years ago

You’re benchmarking clang 12, not clang 16.

w-m• 3 years ago

https://www.npopov.com/2022/12/20/This-year-in-LLVM-2022.htm...

From the article:

I think this is the first time in quite a few releases where we end up with an overall regression. However, the situation is not quite as bad as it looks.

In particular, the large regression on the right is due to enabling C++17 by default. The close to two times slowdown in 7zip O0 builds comes down to STL headers becoming 2-3 times as large in C++17.

While this is sad, and I had to edit out some choice words on the C++ standardization committee (I hear that this gets even worse in C++20), at least this does not affect non-clang compilers (e.g. Rust) and C code.

pjmlp3• 3 years ago

Meanwhile VC++ does just fine, thanks to how it handles incremental compilation, incremental linking and now modules.

IndoorPatio2• 3 years ago

Shouldn't people expect compile times to be slower at -O3? Presumably the improvements are better optimizations, many of which trade compile time for execution time. Now if it's getting slower at -O0, that might be a concern.

krastanov2• 3 years ago

There are dynamic interpreted languages that use LLVM as a way to compile fast paths (the details are more complicated). For such languages how fast the heavily-optimized code is compiled matters. Julia is a great example.

In this particular case though, no it does not seem it matters, as the slowdown is related to C++ headers in the newer standard lib, not to the low-level compilation.

nikic• 3 years ago

You can see the impact on Rust here: https://perf.rust-lang.org/compare.html?start=2d0a7def330ed9...

The tl;dr is that check builds are faster (i.e. optimization quality has improved) while debug/opt builds are mixed, but slightly slower when averaged over everything.

badsectoracula5• 3 years ago

AFAIK the original Clang 15 broke a lot of software, including pretty much all "configure" scripts generated by Autoconf due to some default options[0] that treated some of the code used for various feature checks invalid and the end result was checks failing even though they should have passed - often with barely any notification to the user (the user would see a "no" instead of a "yes" in some check). This was reverted in 15.0.1 but that was only to give people time to update until Clang 16 was released, the Clang developers wanted to get ahead with the change regardless and expected upstream developers to fix their code (this includes not only configure scripts but also projects using old style code - and AFAIK it wasn't only Autoconf that made such tests, though the focus was on it).

The problem however is that thousands of projects that still haven't updated. Gentoo developers are trying to patch the packages but as can be seen in [1] that tracks all the relevant bugs, not even half of them are fixed. Also those are fixes in Gentoo - AFAIK while the Gentoo devs are trying to upstream their patches, not everything is patchable. Note that i mention Gentoo since they were the ones that showed up in the discussions i've seen, but the same is case with all distributions (and projects like Homebrew, etc) - it is just that most likely Gentoo users were affected the most as they build everything from source and can use Clang for that.

But there seems to be two additional issues with Autoconf specifically (remember that this isn't just about Autoconf, it is just that due to how the existing Autoconf checks were written, configure scripts generated by it are the most likely to encounter the issue):

The first is that while there have been fixes for the failing checks, there hasn't yet been a new release of Autoconf that uses these checks - when i run a freshly regenerated configure scripts on my PC with the latest stable release as provided by my distribution i still get the failing checks if i force Clang 15 to use the options that were previously disabled. This means that any configure scripts generated right now will fail when those become enabled again.

The second is that some widely used macros in Autoconf[2], those to find if a library exists and to find which library exposes a symbol (e.g. if libm is needed to be linked to access the functions in math.h or they are part of the standard library - different OSes still need different options here) do it by writing and attempting to link a small C program that defines the function as (IIRC) "char the_function_name();" with code that calls that function (the function is not actually called, the check only tries to compile and link that program) and if that fails it assumes the library does not exist / does not support the function. Problem being is that the above fails with Clang 15.0.0 because of the "()" part (should have been "(void)" but when the check was written many years ago it was meant to work with pre-C89 compilers that didn't support the void keyword). This is fixed in Autoconf now but as mentioned above, no release yet so any currently made brand new configure script will have this check failing. But the more important issue is that this check seems to be a ticking bomb - at least as far as Clang is concerned - since it can trigger various errors in the future and judging from the responses by one of the Clang developers[2] (i actually recommend reading the entire thread) they seem to be of the opinion that treating this as an error in the future is something they might do as it is technically UB. As of right now there hasn't been any fix for this and from the discussions from that thread, the Autoconf developers do not seem to be able to figure out a generic solution (the only solution they found is to include prototypes for all known libc functions but this wont work if you, e.g., want to figure out if the system has curses or ncurses - as an example NetBSD needs -lcurses not -lncurses, at least last time i checked). So even if a new Autoconf release is made now, the configure scripts might still break in the future if Clang developers decide to make this an error (AFAIK even without a release some distros have patched Autoconf to fix the errors introduced and reverted in Clang 15.0.0 but even those do not have a fix for the above).

Also all these are about freshly generated configure scripts, in practice there are projects out there where their configure scripts haven't been regenerated for a long time (often because there hasn't been a new release for a long time). Though many of them can be brought up to date with "autoreconf -fi", assuming of course the fixes i mentioned above are available.

(FWIW from what i can tell this isn't a problem with GCC and the devs contemplated adding a special check for Autoconf generated code to skip the errors if they decide in the future to make these errors, but of course this only affects Autoconf - while i focus on it, Autoconf isn't the only affected program - and regardless the entire point of Autoconf is to allow whatever compiler and tools there are around instead of requiring specific ones from the user)

Considering the above i wonder if Clang 16 decided to proceed with the changes regardless. The release notes do mention some changes that might affect configure scripts[4] but i'm not sure if the notes there are just some heads up for people making configure scripts to check for potential breakage in the future or honeyed words for "technically not every configure script will break, so hey, we warned you". This is a bit amusing since if you check in the discussions i linked to, a concern others had was that the release notes in the version of Clang 15 barely made a mention of the potential for breakage.

(as a side note, all the above focus on Autoconf might sound like me ragging on Autoconf/Autotools, but this wouldn't be farther from the truth - having built a lot of software from source code, including on minimal non-Desktop non-Linux Unix-like environments like NetBSD, i absolutely love that Autotools-based releases do not require anything that isn't already there in a Unix-like environment to work and even recently decided to use it for some of my own projects because of that, which is how i found all the above. From a practical perspective it isn't that hard to work around the issues even with existing configure scripts - AFAIK some of Gentoo patches do exactly that - so despite all the above it is still my favorite build system to build stuff with as a user... as a developer, well, i haven't found it to be any worse than other build systems)

[0] https://discourse.llvm.org/t/configure-script-breakage-with-...

[1] https://bugs.gentoo.org/870412

[2] https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/a...

[3] https://lists.gnu.org/archive/html/autoconf/2022-11/msg00006...

[4] https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNot...

charcircuit3• 3 years ago

I would call this a design flaw with autotools. It would be ideal if you only had to update autotools to work around this. But it seems like autotools brittle configure scripts are spread out between thousands of projects instead of a single place. The amount of brittle code should be minimized and easily updatable.

badsectoracula5• 3 years ago

Not needing to have Autotools installed is certainly a feature and not a design flaw, it is the entire purpose of the project and as i wrote near the end it is the reason i personally prefer using it compared to other projects.

Also note that you can still regenerate configure scripts if you have Autotools installed, so it isn't like what you describe cannot be done with how Autotools are right now. According to a comment in the discussions i linked to, Debian already regenerates the configure scripts as part of their building process.

The difference is that unlike other tools, you don't need to do that - for example i can easily download the source of some program with an outdated configure script on my main Linux PC, regenerate the script with my PC's version of Autotools and then copy the resulting package to another machine running -say- NetBSD that doesn't have Autotools (or any of its dependencies) installed and be able to build it there.

rleigh2• 3 years ago

The autotools deserve their share of criticism here. They should be solving the portability problems of the present day, but they are still stuck in the past solving the portability problems of the 1990s.

Still catering for pre-C89 compilers in 2003 was questionable. Doing it 20 years later in 2023 is beyond questionable. It's not a portability problem any one of us has had to care about in decades. And it's now a source of problems because it's breaking with current tools. That defect is squarely upon them.

Portability problems are of their time, and old workarounds can be dropped once they are no longer required. Many of the Autotools checks and workarounds are for platforms which were retired decades ago. The whole lot could be deleted. When you look at the cost:benefit of them, they are utterly niche and are largely untested by anyone. When you actually look at which tools are portable in practice, the autotools are usually worse than the alternatives like CMake, because they ceased to keep up-to-date with contemporary systems.

As the other poster mentioned, the fact that the generated code is embedded in thousands of projects, all of which need independently updating is another aspect of its design which should also have been retired once the tools reached maturity. There is zero need for this, just require them to be installed like any other build dependency.

badsectoracula5• 3 years ago

I think your criticism is a bit outdated itself :-P, Autotools hasn't been supporting pre-C89 compilers for a long time now. C89 is the minimum requirement, the reason that the pre-C89 code was there was most likely because pretty much all compilers so far accepted K&R style and thus it still worked, so nobody bothered to update it. This code has now been replaced, the bigger issue with that is that there hasn't been a new release with the fixes (some distros do have the patches applied though - but sadly not mine, though i guess i can just download them myself).

Also as i replied to the other comment, you can still regenerate the configure script if you have Autotools installed, however the benefit of Autotools is that you don't have to have Autotools installed to build a project (you can always generate those scripts on another computer).

saagarjha3• 3 years ago

Apple turned this on several years ago when targeting Apple silicon, which might’ve influenced the decision. Obviously, software targeting macOS adapted; it was substantial effort on the part of packagers and open source projects. I wonder if they didn’t feel that pain and saw that things mostly ended up working in the end and used it to provide support for pushing the change through.

boris• 3 years ago

[...] treated some of the code used for various feature checks invalid and the end result was checks failing even though they should have passed - often with barely any notification to the user (the user would see a "no" instead of a "yes" in some check)

This is a perfect illustration of conceptual bankruptcy of the autoconf approach. Using compile/link tests and basing a decision on whether they succeeded or failed without distinguishing the reasons for failure will inevitably lead to silent false negatives.

If you are wondering what to do and what are the alternatives, here is one approach: https://github.com/build2/libbuild2-autoconf

badsectoracula5• 3 years ago

This is a perfect illustration of conceptual bankruptcy of the autoconf approach. Using compile/link tests and basing a decision on whether they succeeded or failed without distinguishing the reasons for failure will inevitably lead to silent false negatives.

The tests are actually fine as an idea, it is much better to perform tests to see if something is there instead of trying to guess based on other means, like OS name or whatever.

The problem isn't the idea of testing for the features you need (like checking if a header file or function is there), it is that the code generated for some of the tests was pre-C89. This was accepted by pretty much all C compilers from the 80s to today, until Clang 16 decided not to accept it anymore.

Technically they could have taken a different approach like checking the libraries themselves directly (for the case of libraries), but that'd require knowledge about the library formats and availability and i'm almost certain that the current approach gives the largest portability while requiring the least knowledge about that system.

Even now it is Clang 16 specifically that has the issue and it was already been fixed upstream last year.

hannob• 3 years ago

I've seen the bugs in Gentoo, so I am aware that this is annoying (and some of them I'm responsible to fix), but I think it's a good thing this is finally being cleaned up.

The issue here is that functions are called without a function prototype, aka the proper headers included. The compiler basically guesses what the function prototype is. This is certainly error-prone, and the fact that autoconf relies on tons of test code pieces that aren't really correct C code is if anything worrying.

It's also probably a good thing that this shows up first in clang, as most linux distros are built with gcc, so you have a testbed to fix all that stuff. gcc should eventually catch up and introduce the same default.

jahnu2• 3 years ago

Am I blind or is there still no proper/full PMR implementation? :(

dataangel• 3 years ago

What is PMR?

tomku• 3 years ago

Polymorphic allocators:

https://en.cppreference.com/w/cpp/memory/polymorphic_allocat...

https://en.cppreference.com/w/cpp/memory/memory_resource

nickysielicki• 3 years ago

https://youtu.be/q6A7cKFXjY0

csb6• 3 years ago

Implemented <memory_resource> header from C++17

Looks like they implemented it. Not sure about if they added all of the specialized containers though.

jahnu2• 3 years ago

Aha , well spotted.

Thanks.

amir734jj• 3 years ago

What is a good llvm book, not a beginner level?

hashtag-til• 3 years ago

Interested in this too.

pjmlp3• 3 years ago

Only Packt has a couple of them, they are mostly alright.

However they aren't recent and LLVM usually changes a bit between releases.

sesuximo• 3 years ago

Llvm has a lot of tutorial style docs. I’d recommend starting with the official llvm website

ChickeNES• 3 years ago

Damn, was really hoping we'd get C23 #embed support in this release

LLVM 16.0.0 Release

Related Stories