Current hardware trends make C++ exceptions harder to justify

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2544r0.html

by ingve • 4 years ago

229 506 4 years ago

open-std.org

Thaxll2• 4 years ago

That's why we disable exceptions in video games.

interroboink3• 4 years ago

For AAA / high-performance / console / close-to-the-metal video games, at least.

There are tons of games where exceptions are perfectly fine. Though still more often used for "probably going to crash soon" situations than otherwise, I'd wager.

CyberRabbi11• 4 years ago

Since exception handling in desktop runtime environments usually takes an unbounded amount of time, it’s not good practice to use them in your main loop since you have to guarantee a new frame at 60Hz.

You can definitely do it but you’d be conceptually allowing for frame skips in your codebase. It could be difficult to audit and remove this assumption if down the road you wanted to tighten up your main loop code.

Negitivefrags• 4 years ago

We disable exceptions in video games due to lock contention when throwing exceptions on multiple threads in high core count situations?

No.

We disable exceptions in video games for dumb historical reasons that no longer apply.

Thaxll2• 4 years ago

Dumb? https://www.youtube.com/watch?v=GC4cp4U2f2E

jmyeet2• 4 years ago

Root cause
Traditional C++ exceptions have two main problems:
1) the exceptions are allocated in dynamic memory because of inheritance and because of non-local constructs like std::current_exception. This prevents basic optimizations like transforming a throw into a goto, because other parts of the program should be able to see that dynamically allocated exception object. And it causes problems with throwing exceptions in out-of-memory situations.
2) exception unwinding is effectively single-threaded, because the table driven unwinder logic used by modern C++ compilers grabs a global mutex to protect the tables from concurrent changes. This has disastrous consequences for high core counts and makes exceptions nearly unusable on such machines.

That's really interesting. I've become very anti-exceptions in recent years far various much-discussed reasons (eg hard to follow, false economy, difficult if not impossible to write threadsafe C++ code in particular, use of exceptions as flow control is an anti-pattern).

One of the porposals is a value-or-error type object, which is basically what Rust has. I really like Rust's enums and match expressions.

It seems so difficult to make changes like this to C++ at this point, at what point do you just have to start again?

nly• 4 years ago

std::expected doesn't require language changes, it's a library type. If anything it shows how C++ is multi-paradigm

ModernMech11• 4 years ago

This is the great strength and weakness of C++. Increasingly the answer to C++'s rough edges is "We don't do things that way anymore. Everyone does X now", where X is the hot new thing. RAII is the best example I can think of, where some people insist that no one would ever use the "new" and "delete" keywords anymore. Except for all the C++ devs that do, and all the C++ code that exists that does and must be maintained.

It leads to the current situation where you have C++ "the language" which is everything, and then C++ "the subset that everyone uses" where that subset constantly changes with time and development context.

fbkr• 4 years ago

I've been using `expected`, i.e. value-or-error type, for a while in C++ and it works just fine, but the article shows it has some noticeable overhead for the `fib` workload for instance. Not sure if the Rust implementation has a different design to make it perform better though.

masklinn9• 4 years ago

Not sure if the Rust implementation has a different design to make it perform better though.

Prolly not, I expect the issue comes from the increase in branches since a value-based error reporting has to branch on every function return. Even if the branch is predictible, it’s not free.

And fib() would be a worst-case scenario as it does very little per-call, the constant per-call overhead would be rather major.

jeffbee3• 4 years ago

You don't have to start over with the whole language. Just use -fno-exceptions in your project and dictate the use of std::optional, or absl::StatusOr, or whatever your favorite variant return type may be. For the examples in the article, it may be perfectly fine to not support failure, to simply std::abort whenever the sqrt of a non-positive is requested and rename the function sqrt_or_die.

jmyeet2• 4 years ago

Except if that's incompatible with libraries you're using.

For the record, I like absl::StatusOr<>.

kevin_thibedeau• 4 years ago

You also have to abandon operator new and STL unless you want to pretend they never fail.

jkaptur• 4 years ago

I'm relatively new to C++ and I found absl::StatusOr<T> and its helper macros to be extraordinarily pleasant.

kirbyfan64sos• 4 years ago

It has helper macros now? I liked StatusOr but had to write my own helper macros for it at the time.

alskdjflaskjdhf• 4 years ago

Yep, these are used extensively at Google (which is where the abseil library came from, and which is famously anti-cpp-exceptions) and they work very well. If I somehow found myself writing a new C++ project I'd probably reach for abseil (and some of the other parts of the Google toolchain: GoogleTest for testing, bazel for builds).

layer85• 4 years ago

Regarding maintainability ("hard-to-follow"), I’ve become a big fan of Java’s checked exceptions (and also their causal chaining and adding of "suppressed" exceptions, which would be nice to have for C++ destructors). I effectively see them as a sum type together with the return type, just using different syntax. It’s an important reason why I stick to the language, because no other language has that kind of statically-typed exceptions.

As the article explains, the problems in C++ are more an ABI issue than a programming language issue (except for the by-reference vs. by-value semantics). You could implement exceptions internally by variant-like return values, for example, similar to how error passing is done in Rust, while still having it look like exceptions on the language level. It would be fun for future languages and runtimes to more easily be able to switch the underlying mechanism, or possibly to be able to use different implementation mechanisms for different parts of a program as needed.

dtech3• 4 years ago

Java's checked exceptions are generally regarded as a mistake. There's a reason no other languages has them, and newer JVM languages (Groovy, Clojure, Scala, Kotlin) treat all exceptions as runtime. Anders Hejlsberg (creator of Delphi, C# and Typescript) also has an excellent article on their problems [1]. In modern Java I see nearly only runtime exceptions used, especially because that's necessary for most Java 8+ lambda API's.

Especially when used as an error ADT they're awful because they mix everything from "you have to handle this 90% of the time" to "the universe physics just changed, sorry" into one construct. Much better to use something like Vavr's Try and explicitly propagate the error as a value.

[1] https://www.artima.com/articles/the-trouble-with-checked-exc...

vlovich1232• 4 years ago

Wouldn’t changing the global mutex into a read/write be a simple way to fix things? Shared libraries changing the exception table at the same time as exceptions being thrown seems rare. Might also be fixable in an API-preserving way…

Edit: nope. This idea is discussed later in the paper (not fully ruled out but the answer may still require ABI changes for more subtle reasons)

interroboink3• 4 years ago

I think this is addressed in the article in the section starting "A less radical change would be to change the global mutex into an rwlock, but unfortunately that is not easily possible either..."

plorkyeran• 4 years ago

A less radical change would be to change the global mutex into an rwlock, but unfortunately that is not easily possible either. Unwinding is not a pure library function but a back and forth between the unwinder and application/compiler code, and existing code relies upon the fact that it is protected by a global lock. In libgcc the callback from dl_iterate_phdr manipulates shared state, and switching to an rwlock leads to data races. Of course it would make sense to change that, but that would be an ABI break, too.

Findecanor• 4 years ago

Could the mutex be wrapped in a rwlock? New code that don't share state would be able to unwind concurrently while old code would be locked out.

vlovich1232• 4 years ago

That’s an interesting idea. Might be an elegant way. I was thinking that the exclusive lock could be updated to be RW by hiding the read information in unused bits of the exclusive lock state that’s mutated atomically.

I find the paper’s argument thin on why this could only be done in an ABI incompatible way.

tsimionescu2• 4 years ago

Very interesting that exceptions have much less overhead on the happy path than Rust/Haskell style Either types (std::expected):

For fib we see a slowdown of approx. 60% compared to traditional exceptions, which is still problematic.

yakkityyak• 4 years ago

I'm not convinced a recursive implementation of `fib` is a reasonable example to draw such a conclusion.

fwsgonzo• 4 years ago

It really isn't.

People need to stop making trivial benchmarks and draw ... silly ... conclusions.

http://www.eecs.northwestern.edu/~robby/courses/322-2013-spr...

It is a fact that C++ exceptions are largely low overhead, and that you also don't have to use them. In fact, C++ can make everyone happy, because you can choose which parts you like.

Personally, knowing how exceptions are implemented and having implemented small parts of the ABI, I can safely say that I will be using exceptions where appropriate in all my C++ projects. Parts where real-time behavior is needed we can use custom containers that don't randomly exit on failure. On low-memory hardware it is beneficial to have either tiny exception footprint or no exceptions at all.

It is true that C++ exceptions as they are implemented can be improved upon, breaking ABI. There are also other contenders (Herbceptions) that show great promise, bringing another way of handling exceptions that is not source compatible. Either way, many who work with custom C++ code do not care much about ABI, as everything is source compiled, and such would benefit from a new ABI generation.

Zero-overhead exceptions by Herb Sutter: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p070...

tsimionescu2• 4 years ago

The article explains that they close that as a simple example of a workflow that is heavy on function calls.

_0w8t6• 4 years ago

I like how Zig implements exceptions. It is similar to the restricted exception value type proposal for C++, but it also has a nice bonus of recording some stack trace on the unwind path. Plus in Zig any call to a function has to either catch the exception or annotate the call somewhat similar to ? in Rust.

masklinn9• 4 years ago

Very interesting that exceptions have much less overhead on the happy path than Rust/Haskell style Either types

C++ genuinely traded “error” cases (made them even costlier) so that the happy path of exceptions would be cheaper. I don’t remember whether that’s the case in C++, but in some language/implementations (used to be a big issue in V8) just having a try/except would drastically deoptimise a function, even if the exception never happened.

brundolf2• 4 years ago

I'm curious whether Rust's Result gets special optimization treatment from the compiler that std::expected doesn't

forrestthewoods2• 4 years ago

Nevertheless the overhead is so high that std::expected is not a good general purpose replacement for traditional exceptions.

This is basically the path Rust has chosen. I’m curious if it’s _actually_ too slow for C++. I feel like the answer must be no.

synergy206• 4 years ago

same as golang, I doubt that causes serious performance issues but have no proof.

ncmncm43• 4 years ago

It is always easy to find places where performance doesn't matter, and even Python is fast enough, including startup code in programs where performance otherwise does matter.

You cannot draw useful inferences from those cases.

forrestthewoods2• 4 years ago

I would also strongly argue you can’t draw useful inferences from do_fib.

To rephrase it another way. In how many contexts is std::expect insufficient? It’s easy to find places where nothing short of hand rolled SIMD is fast enough. You cannot draw useful inferences from those cases.

shiado• 4 years ago

This somehow reminded me, wasn't there a competition years back to see who could generate the most compiler error output with the least amount of C++? A few too many templates and you could generate terabytes. Edit: don't think it was this but this is still a fun read https://codegolf.stackexchange.com/questions/1956/generate-t...

gpderetta18• 4 years ago

I had ten of thousand lines C++ compilation errors that would crash emacs while trying to parse/colorize them.

Still prefer them to the average python error message.

CyberRabbi11• 4 years ago

I am glad more people are raising the issue with C++ exceptions. Unfortunately I don’t think the argument made in this article is compelling enough. Bjarne is against replacing the current model and has written an entire article responding to various criticisms of the current model. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p194...

In particular he has already responded to the efficiency argument with the counter argument that current implementations can be optimized. Even if that optimization process breaks ABI compatibility, it’s still better than breaking source compatibility, which is usually what is being proposed. He is right about that so I don’t think efficiency arguments are going to sway him.

I used to embrace C++ exceptions until I had to use C++ in non-conventional environments. For me C++ exceptions are wholly inappropriate for real-time programming since it’s difficult to statically quantify how much time an exception handling sequence may take. There’s also the issue of it requiring malloc() which has its own issues from an interface standpoint in the real-time context. To avoid unbounded malloc, you’d have to set aside a per-thread area for exception storage and require that you never throw an exception value past a certain size.

_0w8t6• 4 years ago

If the current model stays in place, then there would be no possibility to reconcile the de-facto fork of C++ into code disabling exceptions and code using it with the former can be running on more CPUs than the latter. In turn that leads to monstrosity like file system access API that try to have exception and exception-less versions of each function.

chippiewill3• 4 years ago

Probably worth mentioning that while Bjarne is the creator of C++ and his opinion holds a lot of weight, he's also one of the few language creators who doesn't hold a BDFL hat (and hasn't held that kind of control for a very long time).

rr8082• 4 years ago

I've never liked exceptions, its always awkward combination with returning null or throwing exceptions, or error codes. It always ends up breaking some abstraction making it leaky - eg read_doc throws a file exception, or web exception or higher level exception that has no useful detail.

I'm really heartened in the last few years that FP popularity has caused more people to avoid exceptions and even golang doesn't support them.

rossmohax• 4 years ago

If not exception, then how to fail constructor?

CyberRabbi11• 4 years ago

The “Herbceptions” proposal addressed this. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p070...

synergy206• 4 years ago

you can still set a static flag or something in ctor then check it, but you can't do anything about dtor indeed if you need throw there.

Koshkin19• 4 years ago

Except you are not supposed to throw in a destructor.

worik7• 4 years ago

It has been many years since I used C++

But returning null is an obvious choice.

Combined types which return the structure or an error are another. Has the advantage of returning error information, why failure.

There are many ways

roger10-4• 4 years ago

You can’t return null from a constructor (constructors have no return)

Calavar• 4 years ago

But returning null is an obvious choice.

It's not because C++ is not an everything-is-a-reference language.

Combined types which return the structure or an error are another. Has the advantage of returning error information, why failure.

Sum types are great, but changing constructors to return sum types would break all existing code. It would also lead to a whole slew of questions about how the construction of arrays of values would work. (Does an array of an objects now turn into an array of wrapper types? Byebye SIMD optimization!) It would also break the consistency of constructing POD vs non POD types, which would make writing templates that need to generalize over both a huge PITA.

_0w8t6• 4 years ago

Signal failure via an out parameter reference. For destructors store the reference in the object and call an error method on errors.

mannykannot• 4 years ago

I think that would be problematical in those cases where the special constructors are called, and complicate using constructors in expressions and argument lists.

ncmncm43• 4 years ago

That is the worst of all possible choices.

Just throw.

_b2• 4 years ago

Don't write constructors that can fail unless it is failure that would be appropriate to crash for. That works out a lot better than it might naively sound. It is hard for people to reason about the possibility of constructors/destructors failing, so actually rather nice to just forbid it.

addaon4• 4 years ago

And for those super-rare cases where it's appropriate to crash / fail hard in a constructor but you still have requirements about reporting or even recovery... setjmp/longjmp still exist.

_0w8t6• 4 years ago

Another problematic area is RTTI and dynamic_cast. Chromium disables both that and exceptions. As a replacement in few places where dynamic_cast can be useful Chromium adds a virtual function to the base class to return the subclass or null.

Chromium code replaces exceptions with boolean flags and logging or code to kill the current process for bad cases like out-of-bound access.

erwincoumans4• 4 years ago

I always avoid C++ exceptions, and also try to avoid dynamic memory allocations, smart pointers and RTTI etc whenever possible. This is pretty common in latency and (pseudo) real-time performance critical work, such as robotics control and 3d gaming.

worik7• 4 years ago

Why do you use C++ in those domains?

Curious as to the decision criteria.

cyber_kinetist• 4 years ago

If you need any sort of numerical programming, C++ is basically unmatched. It's fast and efficient (have total control over memory and heap allocations), while able to create complex math abstractions thanks to its template system and operator overloading.

For example, Eigen is the only library (not in C++, but in the entire programming space) that can optimize your math expressions at compile-time. You can also perform techniques like auto-differentiation by using templates. Meanwhile, Rust still doesn't have full const generics, which are needed to create a math library that's both efficient and easy to use (nalgebra still depends on typenum, which is much uglier than even the most esoteric C++ template stuff you can find!)

addaon4• 4 years ago

I work in the same domains, plus hard embedded real-time. Some projects end up choosing C, some C++. C has the advantage of simplicity and more mature tooling, C++ has the advantage of a stronger type system (able to move some things from correct-by-testing to correct-by-construction) and a medium-weight code generation system in the form of templates.

DoctorDabadedoo• 4 years ago

Ecosystem and performance.

There are other as performant languages out there that have a "small" footprint (C, Rust, Zig, etc.), but the combo with the libraries heavily used in robotics, mostly for image and mathematics computation, makes it hard to look away from it: C++ standard library, OpenCV, Eigen, Boost, PCL, ROS.

There may come a day when there is enough momentum and support to look elsewhere, but right now C++ is king in this domain. There is a lot of Python going on too, but usually not for the same type of things.

erwincoumans4• 4 years ago

C is fine too. I pick a C++ subset that is closer to C than to full modern C++17. Full control over memory access, hardware access and execution. I haven't learned enough Rust yet, that could be fine too, but Rust lacks still many libraries in Robotics and Game development.

What else do you suggest?

CyberRabbi11• 4 years ago

(pseudo) real-time performance critical work, such as robotics control and 3d gaming.

In reality all interactive applications are real-time performance critical. Any normal user would say that an unresponsive application is unacceptable. Sadly the entire stack of our contemporary desktop runtime environments grew out of background batch processing systems. This means that an application becoming unresponsive can be a sign of normal system operation.

Even if you avoid malloc, any memory access can trigger swapping which can block your main thread indeterminately. Or if you are running a high enough amount of CPU-hungry processes concurrently, your main thread can block indeterminately.

Millions of programmers carry on building applications for these systems blissfully unaware of these fundamental flaws. Application runtimes like Electron flourish, riddled with thousands of unbounded operations on every mouse click.

com2kid• 4 years ago

Even if you avoid malloc,

Way too many people believe the difference between malloc/new+free/delete vs a GC is that one is deterministic and the other isn't.

(It doesn't help that textbooks still teach this!)

Both are subject to the whims of the system's memory and, if swap is enabled, IO systems.

And unless you understand your call graph very well, constructing an object that constructs other objects is not something you are likely to be capable of calculating the performance of in a non-GC language.

Same goes for destructors.

The actual difference is that in language without GC's typically have explicit syntax for dynamic memory allocation (though it can happen by surprise in C++ from time to time!), which means if you are writing latency sensitive code, you can just avoid dynamic allocation altogether.

But there is no reason why GC languages can't do the same! In fact, newer GC languages sometimes do, and nowadays C# give you more control over stack allocation, so careful C# coding can also avoid using dynamic memory.

mhh__3• 4 years ago

Memory allocation is the big think that new players often miss.

A lot of time I see people saying D is unusable (excitedly) for a certain usecase, and you know what if you use the GC a lot it might be, but then I then see them write code that uses raw malloc and free willy nilly. GC can bite you in the ass, but so will basically all memory allocation if you don't understand the real trends of your program i.e. "Nature is a language, can't you read?"

RcouF1uZ4gsC• 4 years ago

It breaks the existing ABI, and all shared libraries would have to be compiled with the new model, as otherwise unwinding breaks.

I am convinced that maintaining ABI is a huge technical debt to C++. And that is largely occasioned by shared libraries.

Rust IMO made a fantastic decision to emphasize source distribution (and making it super easy via cargo and crates).

synergy206• 4 years ago

you do pay the compilation time price though, sometimes it takes really a long time to rebuild.

zozbot23410• 4 years ago

Rust just punts to the C ABI for shared libraries. Which is sensible since the whole point of those is to share code system-wide, and the C ABI is the de-facto common interface for "foreign" code on systems where shared libraries are widely deployed.

zabzonk5• 4 years ago

As illustrational example consider this small code fragment:

Consider this crap small code fragment. Any function that returns void is almost certainly wrong.

turminal• 4 years ago

Enlighten us, please

zabzonk5• 4 years ago

Any function that does not return a value must have a side effect, else why call it. Calling functions that have side effects (and particularly don't indicate if the side effect was succesful), is not a great idea.

worik7• 4 years ago

No it is not.

burnished• 4 years ago

What does 'wrong' mean in this context? I mean, if you're against side effects and thus believe functions should always return a value that seems reasonable. Except that for a toy example where the purpose is to exercise some hardware and measure results it does not seem to be applicable.

Koshkin19• 4 years ago

You are right. That's why in good old languages they were called 'procedures.'

zabzonk5• 4 years ago

And those languages were badly designed. Which is why we don't have "procedures" or "subroutines" in modern programming languages" - just functions.

MauranKilom3• 4 years ago

Good thing printf returns something, otherwise it might wrong or useless!

gpderetta18• 4 years ago

what should abort() return?

john567• 4 years ago

Why do we need to have ambient control flow? This is what exception handling is, it's a hidden control flow.

I don't use them. I just create an error type and pass that around.

The only legitimate exception I will accept is when you access invalid memory. That's a special case and depending on the environment something extraordinary must happen.

But exceptions and exception handling just creates annoying code. It doesn't add value, not really.

jdlshore• 4 years ago

There are some kinds of errors that can’t be handled locally, but do need to be handled globally, or generically higher in the call chain. Continuing execution after the error occurs will make the problem worse. Exceptions allow you to cease execution without putting an if statement after every function call.

jeffbee3• 4 years ago

That's what you get from disabling exceptions: a call to std::abort instead of a throw.

WkndTriathlete• 4 years ago

Haskell's IO monad says, "Hi! With me you don't need exceptions and you don't need to put an if statement after every function call."

wvenable7• 4 years ago

Exceptions come naturally from the realization that you mostly have to propagate errors to where they can be suitably handled or logged. And all that propagation code heavily detracts from the meaning of the code when you are writing it or reading it. And messing up the propagation is a common source of issues (historically).

With "exceptional" errors are are only 2 real recovery options: restart the operation or terminate the operation. Neither of these are typically decided on anywhere where an error might occur in the call stack.

mort96• 4 years ago

I also have to imagine there's a decent performance boost. With Rust's and Go's (and C's, usually) approach of having 'if (error) { return error; }' all over the place (with syntax sugar or not), there will be a lot of extra branches in the happy path. Sure, those branches are predictable and thus fast, but they're not instant, and they take up icache space in the happy path. Modern exception implementations can almost exclusively slow down the exceptional code, and code which propagates exceptions without catching or throwing will look identical to code with no error handling at all.

I'm sure the gains aren't tremendous, but lots of C++ design decisions are for slight performance improvements at the cost of less safety. Other examples are unchecked array access by default and unchecked overflow. Whether these are the right decisions or not is debatable, but at least it's consistent.

EDIT: Of course, as the article points out, if you have a high error frequency and many cores, exceptions cause significant performance issues. But in the case where exceptions are _actually_ exceptional, and especially in cases where the only real response to an exception is to log an error and exit, exceptions are exceptionally good (pardon the pun) from a performance perspective.

ithkuil• 4 years ago

Not all errors are "exceptional". For example, consider an API that lets you open a file (local or remote); it may be very common that a file doesn't exist and it may be natural to handle that case by checking a "file not found" error (checking if the file exists before accessing it incurs in an extra cost and it's also racy).

jstimpfle9• 4 years ago

Exceptions come naturally from the realization that you mostly have to propagate errors to where they can be suitably handled or logged.

I agree of course that errors must be handled and logged as appropriate, but the implication that this has to happen by popping from the call stack is not justified at all.

masklinn9• 4 years ago

One of the core problems is C++’s design basically requires it: there’s no other way to error from a ctor, and since ctors are used as hooks in many operations the dishonest rejoinder of “just use a factory” doesn’t work in any capacity.

mikepurvis4• 4 years ago

Doesn’t some of the reliance on constructors for everything come from how constness is fetishized in C++? Not that it's all bad to have those checks in place, but Python doesn't have this issue with out of control constructors in part because (almost) everything in Python is just unapologetically mutable.

rr8082• 4 years ago

Best to create object then initialize it in a separate method.

CyberRabbi11• 4 years ago

The only legitimate exception I will accept is when you access invalid memory.

If all memory accesses were done with a function call, e.g. read(void *addr), then by your logic there would be no legitimate need for exceptions.

If you extrapolate into the other direction, exceptions are convenient from a syntactic POV because it avoids littering every operation with an explicit error return mechanism.

edflsafoiewq2• 4 years ago

My problems with Result/expected/etc

1. They generate syntactic noise at every point they touch the call graph: function signatures, calls, returns.

2. In particular, if a change causes a deeply nested function that used to always succeed to be able to error, the entire path up the call graph needs to get Resultified.

3. Since the caller must be aware of them, generic code generally has to be Result-aware too.

4. They aren't a total solution, exceptions usually have to exist anyway (eg panic), for things like oom/assert/etc. So you're usually paying the cost of them anyway.

5. You only get what the callee gives you. With exceptions, you can get a backtrace to the cause of the error by default, with no effort needed on the part of the callee.

zozbot23410• 4 years ago

They generate syntactic noise at every point they touch the call graph: function signatures, calls, returns.

This is not "noise" but needed information. A function that can error out should not have the same signature as one that will never return an error. Similarly, call-site special syntax (like '?' in Rust) helps address the concerns raised by hidden control flow.

Since the caller must be aware of them, generic code generally has to be Result-aware too.

True, but the Rust standard library includes a zero type that can be used to mark a Result-aware function as infallible, making it easy to wrap it with a non-Result type.

kccqzy• 4 years ago

2. In particular, if a change causes a deeply nested function that used to always succeed to be able to error, the entire path up the call graph needs to get Resultified.

This is a pro not a con. It now shows clearly that all these function calls are now failable. Of course at higher levels you may have additional assumptions that you know won't make the low-level functions fail, and you are welcome not to change everything as well.

4. They aren't a total solution, exceptions usually have to exist anyway (eg panic), for things like oom/assert/etc. So you're usually paying the cost of them anyway.

Panics do not need to be caught and handled. You can (and should) transform panics into abort.

exdsq2• 4 years ago

An FYI that Monads are useful for removing that (left, right) boilerplate from composed functions. This was the issue I had where they finally clicked for me.

mjw1007• 4 years ago

One of the motivating examples for exceptions, at the time when they were beginning to appear in mainstream languages like Ada, was to allow people to write arithmetic expressions using familiar notation, while still having a place to put an error handler for the overflow case.

Perhaps the lesson of the last forty years or so is that this convenience wasn't worth adding such a heavyweight feature to the language, but it seems to me that modern languages are still weak at handling overflow.

pjmlp12• 4 years ago

C++ is the only language where exceptions are such an ideology war.

All the other ones that have born with exceptions don't have this issue, including Ada.

throw109202• 4 years ago

Why do we need to have ambient control flow?

Because, properly-used, it saves you a lot of effort.

Returning an error type unwinds the stack. If I have a low-level computation that has a number of error cases, and I want the high-level code to intelligently handle some of those error cases and continue processing, that's not doable using return values. Error codes bind the decision of which error-recovery strategy to take (which is only present at a high level) with the details of that strategy (which is only present at a low level).

As a trivial, obviously-fake example - if I have a high-level GUI library that makes use of a low-level division function, I might occasionally divide by zero. Depending on what the GUI library was doing, I might want my divideBy(x, y) function to return 0, 1, the first argument, the second argument, or not return anything because that section of the code will be completely aborted.

Without ambient control flow, if you just have return values, you have to check the return value for every single division operation you perform - and you'll have to re-implement code paths where a division operation failed.

What if you have a long-running operation? If you return an error value when that operation happens, but it turns out the nature of the operation allows you to ignore that error and continue, you'll have to re-start the entire computation. If you hard-code the lower-level logic to ignore errors and continue, then you'll also end up ignoring errors that you really shouldn't.

Error types do not solve these problems - in fact, they require that you duplicate lots of code to, say, make multiple variants of a library that are identical except when it comes to error-handling - or just cause your applications to lose lots of error-handling nuance and bail early on lots of exceptional circumstances that could be recovered from.

zozbot23410• 4 years ago

What if you have a long-running operation? If you return an error value when that operation happens, but it turns out the nature of the operation allows you to ignore that error and continue, you'll have to re-start the entire computation.

This is just as much of a problem with exceptions. "Fix the error and continue" needs some equivalent to resumable conditions, which are generally implemented at the "low level" using coroutines. My understanding is that async-await as a language feature might be able to express these with relative ease, but exceptions alone clearly do not suffice.

ziml772• 4 years ago

I find this analysis strange. Yes, C++ does need ergonomic ways to return an error without dynamic allocation (Rust's magic of ? combined with the From/Into traits is nice), but I don't know why you'd analyze the performance impact when you have many failures. If a failure is that common, then you aren't supposed to be using exceptions. It's not really an exceptional circumstance at that point.

masklinn9• 4 years ago

I don't know why you'd analyze the performance impact when you have many failures. If a failure is that common, then you aren't supposed to be using exceptions.

The problem is in that case you get into a probabilistic estimation, which is a nice way to say you roll the dice on your performances: what’s the ratio at which exceptions are too costly or sufficiently cheap? And how does that impact your level of service if e.g. exceptions are extremely rare but they tend to all affect the same request or workload?

HelloNurse• 4 years ago

If the program fails too often or too unfairly, why should you blame how such failures are handled? The only reasonable "probabilistic estimation" is that something that happens 10% of the time is normal and it shouldn't be treated as an "exception", even if actually thrown exceptions were fast.

xrikcus• 4 years ago

Adding to that once exceptions start being thrown, that ratio changes because the cost is so high. It's not hard for a service to reach a failure rate just high enough that it overwhelms the system.

tneumann5• 4 years ago

That argument is only true in single threaded applications, at least given today's exception implementations. The more threads you have, the more problematic exceptions become. On the large machine you start to see performance problems with 0.1% failure rate, which is not that much. An core counts continue to rise.

PaulDavisThe1st2• 4 years ago

Again, most experienced C++ programmers would say that even a 0.1% "normal" failure rate indicates a situation that should not be handled by exceptions. You seem to be describing a style of programming where there's a bunch of work to be done, most efforts to do the work will succeed, a few will predictably (albeit randomly, perhaps) fail. While I would concede that you might conclude that exceptions are the perfect tool to use to deal with the failures...

1) you're describing a fairly unusual application behavior 2) it's only because of the performance goals that the speed of exceptions matters

I'm the lead dev of a cross-platform DAW where our "inner loop" is real-time constrained by hardware. We use exceptions freely (in a multicore, multithreaded context) with no performance issues, because if they ever happen, things must stop.

JAlexoid6• 4 years ago

Errors and exceptions aren't necessarily low in all cases.

And in any case this is about how these exceptions impact parallel processing - and the impact is not negligible.

CJefferson2• 4 years ago

Why am I "not supposed to use exceptions"? I don't understand why exception should be used when errors occur occasionally, or when they occur 50,000/sec. They are a control flow method for when errors occur.

MauranKilom3• 4 years ago

For the same reason you may want to use or not use memory allocations when doing something 1 time per second or 1 million times per second. Performance is not amenable to arguments about degree (especially when talking about this many orders of magnitude in degree).

michaelnik• 4 years ago

I'd add a (major) inconvenience is problem investigation, often exceptions point you to root cause and all you need it to break on the first exception (think after running big codebase, for an 1h just to get where problem is), but if they are thrown willy-nilly this debugging strategy is MUCH less convenient...

jcelerier15• 4 years ago

. They are a control flow method for when errors occur.

no, not any errors, exceptional errors. Input data coming from "outside" being invalid is not exceptional. The filesystem suddenly becoming inaccessible during a copying or caching operation, the OS not being able to join a thread, your logging backend becoming unable to log because the disk is full, or a regular expression being incorrect are what comes to mind when I think of exceptional situations (unless your program allows users to input regexps). If your program runs on a normal computer in normal conditions, no exceptions should ever be thrown.

synergy206• 4 years ago

C++ STL is a strong selling point for c++, disabling exceptions mean you lose all of STL in the library, unless you're fine to use STL without any error reporting at all.

In the gaming case(no rtti, no smart pointer, no exceptions(thus meaning ctor|dtor are "unsafe")), what else do you leave with c++ then?

I use c++ but I'm always struggling with yes-or-no for exceptions.

c++ is deeply rooted with exceptions, bad or good.

Rarebox• 4 years ago

There's very little you need exceptions for with STL. Data structures, algorithms, etc. all work just fine without exceptions.

C++ without exceptions is great. Google doesn't use exceptions and they still use STL. Gamedevs usually don't use exceptions and they're fine. Just use some other mechanism for errors, like error codes, absl::Status, std::expected, etc.

_0w8t6• 4 years ago

Note Google explicitly configure runtime to instantly crash on memory allocation errors. With that STL works indeed nicely without exceptions.

synergy206• 4 years ago

Google did not do exceptions due to legacy code base reason, in its announcement it says for new code it will do exception, it just had too many old code and can't do exceptions.

Most STL operations and iteration etc will throw exception for errors, if you disable exception, any of those errors, be it recoverable or not, will just std::terminate, which might not be ideal.

kaetemi2• 4 years ago

Gamedevs usually don't handle any errors at all.

worik7• 4 years ago

Begs the question: Why C++?

Legacy projects (I am not being pejorative, they are important) which must be maintained aside, should not C++ be deprecated?

We have many new languages, we always had C. What does C++ give us in 2022 that makes up for the enormous cognitive load of understanding and keeping up with it.

criddell• 4 years ago

Believe it or not, there are a lot of us who enjoy C++ and think the language is getting better all the time. IMHO, it's a pretty good time to learn C++.

As for what do you get? Well, compared to C you can get higher programmer productivity and compared to any language other than C, you get great performance.

addaon4• 4 years ago

I find that it's not rare to get better performance from C++ than C. As a trivial example, generic container code in C is likely to be run-time generic and sit on memcpy etc; the same functionality in idiomatic C++ is likely to be compile-time generic and be able to use fixed-size copy/move operations instead.

freedomben2• 4 years ago

C++ is still a wonderfully performant language, and there are still domains where it is clearly a leader (networking, games/graphics, etc). Rust is slowly displacing it but there's still a lot of road left. The maturity, stability, prior art are also worth something. The pitfalls are really not that big and dangerous, although to someone who doesn't do C++ I can see why it has that perception. Additionally, if you aren't a functional programming lover (like I've become lately), C++ is one of the funnest languages to work in (as long as the codebase follows could principles).

That said while I used to use C++ for nearly everything, these days I use Elixir for app dev whenever I can, and Ruby and bash for scripts. However if I were going to write a desktop app today I'd most likely go for C++ so I could use Qt. I do really need to try out new GTK though, sounds like it's gotten really great.

ncmncm43• 4 years ago

More people start using C++ professionally in any given week that the sum total paid to code Rust.

Rust is not "displacing" C++ anywhere beyond the HN echo chamber.

physicsguy2• 4 years ago

If you don’t want to use LLVM then you don’t have much choice.

Koshkin19• 4 years ago

That's not true. GCC, for instance, is a GNU compiler collection. (True, it does not include Rust, the last time I checked.) Other than that, there's always Java, Common Lisp, Haskell, and D. Plenty of choice there.

trinovantes• 4 years ago

I still use C++ for the occasional projects that a quick Python script can't easily solve

- I've been using C/C++ since my first year of undergrad so I'm already familiar/comfortable with it

- Java has too much boilerplate (my mind is still stuck in Java 8 so maybe things have changed since then)

- I found myself spending more time fighting the Rust/Go compiler than actually solving my problems

- I don't want to package a web browser for a simple 1-2 UI application

ska• 4 years ago

We have many new languages, we always had C.

The short answer is probably that (unsurprisingly) C still does not scratch the particular itches that C++ was created for, and non of the new languages hit all of them well either. Plus network effect.

Koshkin19• 4 years ago

Little things... Destructors; overloading; namespaces; and yes, exceptions. Things I can no longer live without. Generics (templates) are nice to have, too.

worik7• 4 years ago

You will learn to live without overloading and exceptions.

Generics is an absolutely must have. In 1997 I was was giving up on C++ when I discovered the standard template library. I was lucky to be using Visual C++ on Microsoft. The (only) good thing good thing about that compiler is it implemented the STL using Stepinov's reference.

Stepanov defined STL in 1994. By 1997 there were SFA implementations that followed the standard. Most "improved" it and made a mess of it, like GCC.

(I may have remembered that wrong, mēh)

It completely changed the way I thought about computer programming. Before that I was always programming in a style suited to assembly programming

AnimalMuppet• 4 years ago

What does C++ give us in 2022 that makes up for the enormous cognitive load of understanding and keeping up with it.

If I were starting a greenfield project today, I might choose C++ for it, depending on what it was. (Pick the best tool for the job, and all that.) But if I picked C++, I would not use the full enormity of the entire C++ language specification. I would use the parts that helped with the program I was trying to write, and explicitly not use the rest of it.

fulafel• 4 years ago

There are still some environments where security/safety does not matter much and modern alternatives don't target. Like firmware for a RC project, or a non networked or sandboxed platform game where tooling for other languages is missing (like consoles). If you already know C++ and risks down the road very well.

usefulcat2• 4 years ago

The ecosystem is frequently more important than the language.

Someone• 4 years ago

FTA: The root cause is that the unwinder grabs a global mutex to protect the unwinding tables from concurrent changes from shared libraries

Is it unavoidable that that hurts performance badly for the common case? I would think such concurrent changes are rare, so if there’s an asymmetric way to protect against that that’s faster in the happy path exists, that would be a big improvement.

https://en.wikipedia.org/wiki/Readers–writer_lock mentions Read-preferring RW locks, but (from cursory reading) doesn’t say how much that can help.

chippiewill3• 4 years ago

The article discusses this.

RW lock is an option that helps but involves breaking the ABI of EVERY shared library.

superjan• 4 years ago

It is avoidable by using locks like you mention, the article actually says so, but it causes an ABI change. They go on to say that this makes it undesirable, but compared to what alternative exactly?

_b2• 4 years ago

I think the dilemma for C++ exceptions is that if the programmer actually thinks something should never happen, it is almost always better to just crash. But if the programmer thinks it might happen sometimes, then it is risky to predict it is will be super rare, as often this code will be called in different contexts in the future, so it is safer to use normal returns and flow control. As a result, throwing an exception is basically never the best thing to do.

grandinj2• 4 years ago

Counterpoint: I do a lot of perf work on LibreOffice, which makes extensive use of exceptions, and I have never ever even seen the exception throwing show up on a profile, let alone become a problem.

I think this paper started with a conclusion, and worked backwards to justify it.

henrydark2• 4 years ago

As per the piece, open 128 documents on a 128 core machine, and you'll see the difference.

netr0ute2• 4 years ago

How many people actually use a 128 core machine though? The absolute biggest AMD Threadripper you can get only has 64.

mhh__3• 4 years ago

This is mostly my opinion too.

My argument against exceptions is basically everything other than performance. To make a performance decision you have to build the code incrementally with feedback from how it is actually going to be used e.g. if almost every parse fails then you probably don't want to throw whereas if your code almost always succeeds you probably want your register back (i.e. use exceptions)

anarazel• 4 years ago

When I read the title, I was assuming it was going to be about worsened code generation when building with exception support. Not about, basically, throwing exceptions as fast as possible. That seems an odd concern given the current performance.

mlinksva• 4 years ago

Thanks a lot for your perf work on LibreOffice, which https://gerrit.libreoffice.org/q/Grandin seems to confirm!

addaon4• 4 years ago

I wouldn't necessarily expect to see throwing show up on a statistical profile. I would expect to see (and very much have seen) throwing showing up as a huge latency spike, leading to user-visible non-responsiveness. Especially in a cold situation (first throw after launch, or throwing after pages have been ejected), a surprising number of memory pages need to be populated to throw an exception.

tneumann5• 4 years ago

I am the the original author, and trust me, I am describing a real world problem. I run massive parallel data processing tasks on machines with 128 cores, and unfortunately some of them produce errors deep within the processing pipeline. From a programming perspective exceptions would be ideal for that scenario, but they cause severe performance problems.

Just think about this: If you have a 100 cores, and 1% of your tasks fail, one core is constantly unwinding. And due to the global lock you quickly get a queue of single threaded unwinding tasks. And things become worse, we expect to have machine with 256 cores soon, and there it is even more dangerous to throw.

If you do not believe me look here: http://wg21.link/p0709 It lists quite a few applications that explicitly forbid exceptions due to performance concerns.

jcelerier15• 4 years ago

I am the the original author, and trust me, I am describing a real world problem.

for a more-or-less niche part of "real world". The overwhelming majority of desktop GUI apps rely on some C++ system - Qt, gtkmm, Wx, Blink, Gecko, FLTK, etc etc... and it is not an issue for those, for what exceptions are commonly used for (a write failing because the user disconnected the USB drive while it was copying, a system resource limit exhausted.. things like that). As much as massive parallel data processing tasks matter, I'd really prefer my language to not side-step writing end-user apps for something that happens at $bigcompany or $bigresearchlab.

here's the list of binaries that link against libstdc++.so.6 in my /usr/bin: https://paste.ofcode.org/gfZJwx4puVx7Uxy9a7BBU3 - don't forget those please :)

tjungblut2• 4 years ago

If you run into contention issues have you tried scaling horizontally? Running on 128 single core VMs should mitigate what you see.

CyberDildonics4• 4 years ago

All of this hinges around the global lock, although I'm not sure what you mean exactly. Are you talking about memory allocation, a lock that exception handling takes or something else?

wvenable7• 4 years ago

and 1% of your tasks fail

That's a lot of failures. I'd question whether or not you should be using exceptions for something that happens 1% of the time all the time.

Admittedly you might not have any choice in the matter if it's a dependency that is failing.

ncmncm43• 4 years ago

If you are seeing 1% of operations failing, I can guarantee you are Doing It Wrong.

kfitch42• 4 years ago

Not only is this relevant to HPC environments, it also impacts the other end of the spectrum in the embedded space. When dealing with real-time requirements, you are much more concerned with the worst case performance as opposed to the average case or happy path performance.

This article makes it very clear that exceptions complicate analysis of performance. The non-local nature of exceptions mean I can't analyze performance in isolation. E.g. I can test that thread 1 always meets its deadlines (even with exceptions being thrown), then I can test that thread 2 always meets its deadlines.... but if thread 1 and thread 2 happen to throw exceptions at nearly the some time I might miss both deadlines. Who knows, maybe this means a thruster fires too long and that insertion burn fails ... and Mars has a new crater instead of a lander.

And, once you throw -fno-exceptions you are no longer using standard C++, which the standard library assumes. So, using anything that would throw exceptions on memory allocation failures is a no-go. You can work around this with extensive use of allocators (that reference enough static memory to avoid any possible out-of-memory situation)... but this is not looking like idiomatic C++ anymore, and most off-the-shelf libraries are unusable.

A completely local exceptions implementation (e.g. Herbceptions) would solve this.

zwieback• 4 years ago

You mention the problem of having to recompile for ABI-breaking improvements. Just out of curiosity - in the environment you're describing, how much of the code cannot be rebuilt? As I was reading I was assuming that most situations where exception handling causes performance issues are probably also situations where you have most if not all the source code available. This wasn't the case in my formative C++ years when 3rd party libs were expensive and distributed as binaries (on floppies).

Asooka• 4 years ago

I agree with the premise - C++ exceptions are not a good choice for signalling numerical errors in high-performance computing environments, where said errors can happen with some frequency.

However, I do not agree with the conclusion - what is the difference between removing exceptions and simply not using them in your project? All compilers already let you compile code without exceptions and the product I work on compiles all numerical code that way. I don't see what benefit you expect to reap from changing the fundamental way C++ exceptions are implemented and work. At the high end, you want a custom error solution fit for your particular task, I don't think we can have a generic exception framework that will work for every high-performance computing project.

xienze• 4 years ago

I run massive parallel data processing tasks on machines with 128 cores

So, not to be rude here, but this may be a real-world scenario in your world, but not in everyone else’s. I get the sense the kind of stuff you’re working on would benefit from things like using assembly as well. But for desktop apps, games, compilers, other command line tools… who cares about the performance of throwing exceptions?

cesaref3• 4 years ago

I've worked in low latency trading, and with large distributed back testing setups with hundreds of high core count machines, and not once felt the need to disable exceptions, or for that matter, felt the impact of them happening. A fair bit of noexcept stuff was useful to improve runtime performance, but that was about it.

I would suggest you need to address that 1% of failing tasks and determine what the issue is, as frankly, you are solving the wrong problem. If I might suggest a solution, it sounds like you are using threading when process farms might be a better solution.

(and if i'm way off the mark with my suggestion, apologies, i'm trying to help and have little information to work with).

brundolf2• 4 years ago

It probably depends a whole lot on whether exceptions are used only for exceptions, or for control-flow. The author seemed to be treating this question fairly pessimistically

downut• 4 years ago

In 30 years of c++ I've never seen error checking + throw in a tight numeric intensive loop like this. Sure, somebody could do it. I'd have a chat with a coworker who wrote something like that.

I only use exceptions for cleanly unwinding the stack when an unrecoverable error occurs. I design and implement my code so that is rare.

olvy02• 4 years ago

Totally agree, I do the same, up to and including having a chat with a coworker who wrote code like that. I just did that last week, in fact.

Code that uses exceptions like this is a code smell, especially when performance is important. Using exceptions as control flow instead of if/loops is not a good design, IMO.

Note that this is in C++. I consider this style of writing not idiomatic, despite the STL doing it. I use either error return codes or std::optional (itself having a set of problems but IMO better than exceptions).

I'm more willing to accept this kind of code in Java, where it's more or less idiomatic. Less so in C#, where the tendency in the last 15 years has been to use the Try__ method instead of throwing exceptions.

nicoburns• 4 years ago

I feel like the main problem with exceptions isn't the performance, it's the non-local control flow and the fact that it makes enumerating all possible failure modes almost impossible. IMO the only way that exceptions could be justified (other than being the status quo) is if they were much faster than Result/Maybe types. Performance parity doesn't make them worthwhile!

CJefferson2• 4 years ago

Counter-conterpoint: I've worked on two applications C++ which we had to "de-exception", as exception handling was taking >25% of the time when we profiled.

You could argue we were using "too many exceptions", but to me it's the obvious way to unwind in C++.. except it's not fast enough, so we had to switch to a proto-Rust (this was before Rust) style system.

passivate3• 4 years ago

Do you still have any notes from your profiling?

grandinj2• 4 years ago

I gave talks at the Libreoffice conference in 2019 https://www.youtube.com/watch?v=QnBIeeBZf70 and in 2021 https://www.youtube.com/watch?v=VbL6mXXnFdo&list=PL0pdzjvYW9...

GnarfGnarf2• 4 years ago

I manage 1.2MLOC C++ developed over 25 years. I am pleased to say we never had to use exceptions. Just check the return value.

aeldidi• 4 years ago

How do you guys handle using other libraries? Everything I can see uses stuff like std::vector and the like at API boundaries, and I'm not aware of a way to construct an std::vector without throwing an exception.

TheMonarchist4• 4 years ago

I'm not aware of a way to construct an std::vector without throwing an exception.

You use an allocator with enough pool for whatever you need to do and avoid out of bounds access.

GnarfGnarf2• 4 years ago

We use the STL. We have never observed an exception thrown.

cryptonector5• 4 years ago

(Exceptions are bad, full stop. It should all have been monadic all along, with Maybe/Either/Result.)

That said, exceptions should have been allocated on the stack, and catch handlers should have been closures that get called with the exception object. I guess this can't be implemented now.

As for multi-threading unwinding, the issue there is that unwinding tables are part of the shared objects whence the associated object code comes, and it often has to be possible to unload loaded shared objects, so now what? The situation shouldn't be bleak though: a shared object should never be unloaded while there are threads executing its code, so it should be possible to arrange a slow unload-time operation to update the the process-wide unwinding tables -- think of user-land RCU if you like.

The exception handling code should not have to synchronize around unwinding, except that when unwinding completes it might have to notice that there's a pending unload, so wake the thread that is waiting for unwinding. Or maybe not even, because maybe unloading could do something truly breathtaking like check that no thread's stack includes return addresses from the object to be unloaded, and then unwinders would never need to step into the unwinding tables from that object.

gumby13• 4 years ago

I find this paper quite unconvincing.

Exceptions are exceptional so in principle it doesn't matter (within reason) how long it takes to throw one as long as it costs nothing not to do so. So measuring the cost of repeated throws IMHO doesn't cast light on any useful case, and the approaches that add runtime cost for the path not taken, even Herb Sutter's, are not acceptable.

His code transformation example is simply the compiler behaving properly, unless it can't make that transformation even when foo() is declared noexcept.

The high core count is a real issue and a legitimate reason for an ABI break. Exceptions are the kind of below the surface plumbing that can't reasonably be implemented in regular code.

And in that regard the paper does make a good suggestion, though it then dismisses it! The tree approach described in section 3.4 seems like the right kind of fix.

Ultimately there's a spectrum of branching (`return` -> `break` -> `goto` -> `throw`) all of which need consideration in light of multicore deployment.

cryptonector5• 4 years ago

How exceptional exceptions really are depends on the code being run.

The serialization of stack unwinding issue is a real serious problem.

gumby13• 4 years ago

How exceptional exceptions really are depends on the code being run.

Well yes, but EH is basically trying to be the best way to do a highly non-local goto with a dynamic target. If that part is your bottleneck than indeed you may have a more specialized requirement.

I do on occasion build a homemade version of a standard library datatype because I have a specialized need and only need implement the subset our code will call. That doesn't invalidate the more general implementations in the standard library (which tend to be quite good, even corner cases).

Likewise sometimes I check for a null pointer being returned or other local error flag. That doesn't mean I don't think exceptions are a good idea.

The serialization of stack unwinding issue is a real serious problem.

That is the important part of the paper and as I wrote in my comment, I am disappointed that a possible solution, written and tested by the paper's author, was dismissed.

JoeAltmaier2• 4 years ago

But there's a design pattern where nearly everything is returned as an exception. It's the normal path in some code.

I deplore that - its side-effects writ large. But those that use it, find it reasonable and sensible.

It's become hard to distinguish right and wrong when it comes to CPU optimization. Different versions of the 'same' CPU can have wildly different sweet spots.

gumby13• 4 years ago

It's become hard to distinguish right and wrong when it comes to CPU optimization. Different versions of the 'same' CPU can have wildly different sweet spots.

Indeed, though I am shocked when people post benchmarks in which they are obviously unaware of this issue. It's why compilers have all those architecture switches.

If I understand your comment correctly: in the case of the posted paper, the compiler optimization issue the author mentioned was a semantics issue, not an architecture issue. If your comment was about optimizing for the multiprocessor case, I apologize.

gumby13• 4 years ago

But there's a design pattern where nearly everything is returned as an exception.

shudder The worst of cargo cults!

josephg3• 4 years ago

But there's a design pattern where nearly everything is returned as an exception. It's the normal path in some code.

The problem isn’t that C++ exceptions are slow when used the way the compiler expects. The problem is a mismatch between how C++ expects you to use the feature, and how people are actually using the feature.

I can’t find the story now, but I read about this happening with Ruby on Rails. Someone dug into why rails was so slow at Twitter, and found the way rails was looping through an array was that it would loop unbounded, and catch and discard the array out of bounds exception at the end of the loop. Whenever Ruby throws, it allocates 10k of memory and fills it with all sorts of information for debugging - including a full stack trace. And it was doing this work in the background every time someone iterated through an array. Fixing this resulted in a massive speed up at Twitter, and presumably across the entire rails ecosystem.

I saw the same thing at (big tech company) a decade or so ago. I was working on a project which used GWT. We brought some people in to help us optimize, because the program was too slow. The first thing they found was that the JS VM was spending most of its time in the exception handler for some reason. Turned out one of our engineers had a habit of using throw & catch as a way to do multi level returns in complex code in Java. But the exception was being converted to a javascript exception by GWT. And javascript exceptions are (were?) super slow.

Y’all gotta stop using exceptions like that. I know it feels clever. But in most programming languages, using exceptions for control flow will kill your performance.

NoSorryCannot• 4 years ago

Exceptions have some of the properties of a discriminated union, which is why they're used this way. Sometimes it seems parsimonious compared to the alternatives.

throwuxiytayq• 4 years ago

But it's not reasonable, sensible or performant. Thankfully, I've never seen anyone abuse exceptions like that in C# or any other language, is it really a thing in C++?

gpderetta18• 4 years ago

I'm certainly not an expert but as far as I understand, fixing the exception global lock need not break the ABI on gcc/glibc. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744

gpderetta18• 4 years ago

Turns out I need to read more carefully. Improvements can be done in a non-abi breaking way, but more radical changes are, according to the authors ABI breaking.

talaketu4• 4 years ago

"exceptions are exceptional" means what?

timClicks• 4 years ago

They're rare. Therefore their runtime costs have little impact on the overall running time. At least that's my understanding of the argument.

Jtsummers• 4 years ago

Exceptional means "unusual" (by the dictionary definition). In theory, exceptions should be thrown in in exceptional (rare, unusual) circumstances. Things like running out of memory, or dealing with random poorly constructed data inputs are reasonable circumstances to use exceptions. However, if your input is consistently mangled, an exception may not be the appropriate way to handle it since it becomes a normal thing (if for no other reason than performance) depending on how you want to handle the problem and whether that performance cost is worth it.

jcelerier15• 4 years ago

It means that in normal usage conditions, if you install your software on a clean computer and run it and nothing weird happens outside of your program, then no exception should ever be thrown.

pavon2• 4 years ago

In C++ the convention is that exceptions shouldn't be used for things you expect to happen in the normal execution of the program, in a way that would harm performance. For example, it is better to explicitly check if an item is in a map than to rely on exception handling to branch to the case where the item doesn't exist. Generating an exception for FileNotFound would be fine for a single file selected by the user in a UI, but you'd probably avoid it if checking for the existence of a large number of files based on a pattern. Most exceptions should either be a bug, or exhaustion of resources.

This is in contrast to say python where the convention is to rely heavily on exceptions as part of the normal flow of the code, sometimes described as "asking forgiveness, not permission". It is not uncommon for a method argument to support multiple types, and to discern them by treating the object like one type and if you get an exception, then try treating it like another type. Likewise, if you're not sure an item is in a dict, you just try to access it, and catch the exception if it isn't. This has performance impacts, but so does everything else about python, so it isn't worth optimizing.

gumby13• 4 years ago

The reason they care called "exceptions" is they are not part of the normal behavior of the function (/block, algorithm) and aren't something that can be handled locally. Something that is exceptional is unusual, out of the typical scope of things. In English there is a phrase, "the exception to the rule" -- because the rule is what normally happens.

So if you are trying to hold a lock you don't throw an exception, you just wait and try again. Perhaps you can't reach that host; try again a few tiles before giving up and throwing an exception. But if you try to write to removable media and the device won't open, all, your program isn't going to mount a tape itself: throw an exception and let the problem be handled at a higher level.

shrimpx2• 4 years ago

The problem is that in many practical situations you don’t know which situation is “exceptional”.

If you write a jpeg-processing service, it’s intuitive to raise an exception on a malformed jpeg, but there’s no guarantee that only 1% of the jpegs users upload to the service are malformed.

In other words, we treat exceptions as exceptions from our code expects, not what’s statistically unlikely in the input space, which is in many cases impossible to predict with accuracy. (E.g., even if only 1% of your inputs are malformed over all time, on some Thursday you may be hit with 80% bad inputs, making the performance drop across the service unacceptable.)

fivea• 4 years ago

If you write a jpeg-processing service, it’s intuitive to raise an exception on a malformed jpeg (..)

Not really. Having to parse a malformed doc is not an exceptional situation: it's a basic use case, and one which is very close to the happy path.

kazinator2• 4 years ago

This is a bit of a stretch. The purpose of the service has to be processing jpegs where they are expected to be bad. Moreover, they have to be mostly bad; i.e. the service is specifically intended for finding rare good jpegs in a deluge of bad ones, as quickly as possible.

In a situtaion where are more bad jpegs than good ones, we still don't necessarily care that they cause the slow path, if the purpose of the service is doing meaningful processing with good jpegs.

randyrand7• 4 years ago

The only way exceptions should be exceptional is that they should rarely be used in any code base. At best, the are a micro optimization that helps you a tiny bit in the success case. They should be used when standard return value errors are measured to be impacting perf (ie. exceedingly rarely). Unfortunately, C++ language authors made them basically a requirement for OOP and RAII.

Imagine writing a parser. Can you use exceptions? It's hard to say! In fact, often impossible to say! How often an exception is thrown is highly dependent on the input data. If you're in a controlled environment with nearly always well formed data, it might be a perf win! But there's a ton of complexity in just deciding which type of error to use, and thats on top of the complexity of now having multiple ways of returning errors!

No one likes code that has 2+ ways of indicating method failure. If you are a library author, do your users need to check for both Exceptions and Error codes? If you are a nice author, you'll just choose 1 way to indicate failure and wrap where necessary.

If you want to use exceptions, they should be used in very limited scope when perf measurements indicate they would help, tightly wrapped in a catch, and then immediately converted to some other standard error type.

Even better, we should just let the choice of when to use exceptions up to an optimizing compiler, and provide the compiler a function to convert between exceptions and our more general purpose error type of choice.

Traster• 4 years ago

Ok, I'll bite. Disclaimer: I'm not a software engineer, FPGA engineer by training. The use model I have in my mind is that my functions all work 99.999% of the time, but occasionally something happens where basically I want to throw up, I want the whole thing to collapse. Exceptions are great for me. I'm in an industry where "stop" is an okay solution in the rare occasion something goes wrong. So what's the alternative? I have to overload my return values annd then put in a load of special cases to propogate or return the exceptional cases.

I am aware there are functional languages that have other ways of dealing with this, or languages with things like optional values etc. But fundamentaly I like the idea that there's just a mechanism to just collapse, and collapse in a way that is traceable.

Having said that, my experience essentially does not involve catching exceptions, to me that's what is almost unthinkable (read: probably a bad hack). I guess my use case is not performance but separateing normal code function from catastrophic errors. Could you explain, do you think this is a fundamental mistake in the way I'm approaching this?

tremon• 4 years ago

They should be used when standard return value errors are measured to be impacting perf (ie. exceedingly rarely)

I disagree with this. I think they should be used whenever "standard" return value error handling starts to obfuscate the normal operation (happy flow) of the code. This impact on developer performance happens a lot sooner than the measurable machine performance.

ncmncm43• 4 years ago

This just spectacularly bad advice and policy. Point performance is the worst way to decide about error handling.

The overlap between "things that fail and the caller knows something useful to do", and "things that fail and somebody maybe six levels back in the call chain might know something to do" is remarkably thin. Most things are the latter. Once in a while, it makes sense to provide one for each: (1) open a file, and it had better succeed or you might have to bail out, or (2) see if a file is there.

The reason exception is the default, aside from that there are plenty of ways to return an error report, is that when you need code to clean up, the destructors are right there. They get exercised every time through the code, so are well-tested. How often is your special-snowflake error handler ever exercised? And the next one? And the next one? Even before you have given a moment's thought to error handling, your destructors are cleaning up, correctly. So, even if you haven't written code to handle it, your program has remained in a well-defined state. And, very often, you don't even need to write them: they are auto-generated from things the compiler already knows.

If you have set up a high-level catch block, you get there without fuss, and that code knows something correct to do, even if it is only to shut down cleanly. If you left a record of what was about to be attempted, that code knows what to try next.

tokumei• 4 years ago

I much prefer using a Result<T, E> type now, even in C++. Makes it easier to reason about the code in my opinion.

gumby13• 4 years ago

The article shows that this approach adds to runtime even in the non-erring case.

adrian172• 4 years ago

This paper confirmed what I suspected for some time: exceptions are still the only basically-zero-overhead solution available, which ironically completely justifies their existence in usage patterns where exceptions are actually exceptional. There are times when writing/profiling Rust when I wish I had access to exceptions instead of `Result` propagation.

I do agree with the mentioned design issues though.

cryptonector5• 4 years ago

The problem is that if you have exception-happy code, then it becomes a scalability limiter for threading.

adrian172• 4 years ago

Yes, if A: you have code that's expected to "fail" relatively many times, and B: this code is also multithreaded. A lot of code doesn't match one or both of these conditions while still being perf-sensitive.

Further, as you can see in other comments, it's not an uncommon belief that exceptions simply shouldn't be used for A (and thus it's reasonable that they aren't optimized for something they aren't supposed to be used for).

dureuill3• 4 years ago

There are times when writing/profiling Rust when I wish I had access to exceptions instead of `Result` propagation.

This introduces a whole bunch of design issues, but isn't `panic` (in unwinding mode) basically C++'s exceptions, implementation-wise? Couldn't we use `panic` + `catch_unwind` as a poor man's exception system, should the performance situation really require so?

Also, could we maybe add an attribute `#[exceptional]` to enum variants in a match, that would result in a match implementation closer to exceptions, implementation wise?

Dork1234• 4 years ago

It would be interesting to see a breakdown of exception overhead for C++/Rust/Haskell/Julia/Swift in multi-threaded workloads.

Sometimes that happens rarely but blocks many threads is a huge pain to deal with.

lanstin12• 4 years ago

Just write code with a bunch of threads that are all logging a few M / second and see if the CPUs are running equally hot or 1 is 100% and the rest are mostly idle.

asveikau• 4 years ago

2) exception unwinding is effectively single-threaded, because the table driven unwinder logic used by modern C++ compilers grabs a global mutex to protect the tables from concurrent changes.
The second problem could potentially be fixed by a sophisticated implementation, but that would definitively be an ABI break and it would require careful coordination of all components involved, including shared libraries.

This seems like a perfect application for an rwlock. Threads that throw acquire for reading, so can occur in parallel. Threads that load or unload shared libraries (a pretty rare occurrence outside of process start) acquire for writing and therefore block reads from the table from throwing threads. The shared library loader shouldn't throw during this piece.

Come to think of it, throwing should also be pretty rare. C++ exceptions are not really about common control flow, but truly exceptional circumstances. So the high cost serializing parallel throwers doesn't even sound like that huge of a deal. But as I've said, it seems pretty easy to improve upon it.

[Edit: It would seem the article does say something about how an rwlock is not feasible with the current implementation ... It doesn't sound terribly convincing]

zmmmmm2• 4 years ago

It's unclear to me if the author is saying that the global mutex for unwinding the stack interferes with the non-exception-throwing code paths, or are they just saying that the exceptions themselves bottleneck and are so inefficient that this becomes a significant impact on overall throughput?

It does seem like at least in theory it should be possible to create a non-locking / blocking exception unwinder, if there is no actual contention between the threads. If that can be done then it seems like the solution should be to do that rather than abandon a whole language feature. This is a bit like the Python GIL question. I would say if the language spec means you have to have a "GIL" in any context in a high performance language like C++ then it ought to be addressed at the spec level.

beached_whale• 4 years ago

One can mix the error abstracts, we have to anyways(cmath methods and NaN). When the distance between the error and the caller that can act on the error is large, it makes much more sense to throw in a large number of cases. Otherwise the cost is code that would have been error neutral becoming error opinionated and that is a software engineering cost.

But look at some of the costs we have to go to great lengths to get around. std::sqrt has to check for invalid inputs and that means a branch, often even when the code is checked prior or an ASSUME( f > 0.0 ) type thing is expressed. This inhibits auto-vectorization. On some compilers ASSUME( not std::isless( f, 0.0 ) ) can tell the compile that the number is real and >= 0 which elides the branch. But the math-error/errno issue is a big hinderance too.

A lot of the costs is the branch itself and telling the compiler it is not needed can boost hot path perf more than the rare cold paths.

dataangel• 4 years ago

The root cause is that the unwinder grabs a global mutex to protect the unwinding tables from concurrent changes from shared libraries.

OK, so replace the mutex with a reader writer lock. Shared library loading is incredibly rare, and dear god I hope nobody seriously does it and expects it to work during unwinding.

Current hardware trends make C++ exceptions harder to justify

Related Stories