Go: Sentinel errors and errors.Is() slow your code down by 3000%

https://www.dolthub.com/blog/2024-05-31-benchmarking-go-error-handling/

by ingve • 2 years ago

183 139 2 years ago

dolthub.com

leetrout• 2 years ago

I like posts like this.

I learn something about a tool I love and have used for 10 years and it isn't snarky, flamebait nor preachy.

richbell4• 2 years ago

errors.Is() is expensive. If you use it, check the error is non-nil first to avoid a pretty big performance penalty on the happy path.

Am I missing something, or could this be partially mitigated by adding a check for `err == nil && target != nil` and vice versa, rather than `err == target`?

https://github.com/golang/go/blob/9b43bfbc51c469ec13fca24960...

metadat• 2 years ago

I wonder why the compiler can't be smart enough to optimize such a case.

flexagoon• 2 years ago

It already does

Tomis02• 2 years ago

We probably don't need programmers either, the compiler should be smart enough to know what the stakeholders mean.

richbell4• 2 years ago

The answer is apparently that the code I linked isn't released yet.

The latest available release only checks if target is nil.

ReleaseCandidat2• 2 years ago

What he is missing is that if the max. 3 pointer comparisons in (together with the function call of `Is`)

    if err == nil || target == nil {
        return err == target
    }

are a "pretty big performance penalty", there isn't much work going on. I mean, of course, these 2 or 3 comparisons + the function call of `Is` are at least double that of

      err != nil

even if `Is` would have been inlined by the compiler.

richbell4• 2 years ago

Actually, the `err == nil` check doesn't exist in the latest release. It was added recently, which would explain the significant performance cost. If err is nil, it doesn't return early.

https://github.com/golang/go/blame/adbfb672ba485630d75f8b559...

eptcyka2• 2 years ago

There's a tangential issue in Rust where there's a concern around if a library should be exposing the error types of its dependencies. If they are exposed, semver is broken anytime a dependency is updated in a semver breaking fashion. I, in general, agree that this breaks encapsulation and is not good. However, and maybe I'm doing things wrong, but often enough there are edge cases where it is important to distinguish between different errors and adjust control flow. Often, library authors will not be able to foresee all possible usecases for their error types, so one can end up doing some horrible type casting or worse hacks to get at the actual underlying data. I'm still torn as to what is the better way forward.

paulmd2• 2 years ago

java has the "rootCause" constructor which wraps another Exception (actually Throwable) of any type passed in. so the idiomatic way would be to create your own error types, but to pass the lower-level exception in as the rootCause so you can trace the exceptions down the stack if desired. Since the type is Throwable rather than a dependency-specific exception, this isn't exposed in the interface unless you go out of your way to cast things in an obviously dangerous way.

Does that not exist in Rust?

Dylan16807• 2 years ago

When you say "obviously dangerous" do you mean "obviously breaking encapsulation" or is there a bigger issue? If there's a bigger issue then I'm not sure how it's useful for solving this problem of wanting the details.

tempaccount4202• 2 years ago

I don't think errors should have to follow semver. Please, break my match on non-major version bumps, if I didn't explicitly add a catch-all case.

tialaramex• 2 years ago

That's what the #[non_exhaustive] attribute is for. If a struct or enum is given this attribute then everybody else is treated as though they cannot exhaustively match it, they must write the catch all case. You (as the owner of the type) can still exhaustively match it, after all you're the one who'll get broken by your own change if you screw up - but everybody else gets told they need to write the "catch all".

So, no, errors aren't special and should obey semver but you deliberately can (and should) signal that your users can't exhaustively match a thing you know will change.

manlobster2• 2 years ago

Here's one of them, which follows a common recommendation in the Go community to use a sentinel error to represent the "value not found" condition.

Is this really a common recommendation in the Go community? Seems like returning `bool` to indicate if the value was found is somewhat of a no-brainer, since it follows the familiar approach used for map lookups and type assertions.

segfaltnh• 2 years ago

I've certainly seen and emulated this pattern in a lot of Go code. I don't know if I am representative of the larger community. I often use bool unless there are other error conditions (and there often are), but this article is definitely making me wonder if this is ideal.

abound• 2 years ago

There are at least two places in the standard library that behave like this: sql.ErrNoRows and the error thrown by os.Open when a file doesn't exist.

manlobster2• 2 years ago

True, but both of these are for relatively slow operations, where the performance of the error checking is relatively insignificant.

ReleaseCandidat2• 2 years ago

Sorry, that's still not a reason to use bools instead of "errors" and

    An error should by default be considered non-recoverable, to be returned when something goes very wrong. Semantically, a table not existing isn't really an error, it's something you expect to happen all the time.

remains a bad idea.

Seems like the reason for this article is the reaction to that one: https://www.dolthub.com/blog/2024-05-10-ok-considered-harmfu...

LoganDark5• 2 years ago

That quote conflates errors and exceptions. An error is an unsuccessful result. An exception, however, precludes any result at all. Sometimes they are the same, but in many cases they are different.

Asking to retrieve a table that doesn't exist is an error, because the answer is that there is no table; the operation was unsuccessful. There failed to be retrieved any table, but as the table always could have either existed or not, this is not an exceptional case. Nothing went wrong: you simply asked to retrieve a table and no table was found, so you get that back as an error.

However, trying to perform a table operation on that unsuccessful result generates an exception, because you violated a contract: you can't operate on something that doesn't exist. Sure, you can operate on the representation of nonexistence (that is, null), but you attempted to operate on a table and you don't have one.

The computation stops prematurely with an exception because the operation expects the table to necessarily exist. There is no unsuccessful result to be returned because the operation didn't even make it to the point of failing; the performance itself of the operation failed. The operation is entirely invalid and returning any result would be incorrect.

At least, that is my personal understanding of the terms.

For example, most things in Rust are errors and not exceptions, because exceptions (panics) generally terminate the thread, or abort the process, depending on user preference or runtime environment. Anything that isn't fatally unrecoverable deserves to be an error - in other words, a `Result::Err` - which can be handled by the caller, as opposed to panics, which are not guaranteed to be catchable (unwinding is not a required part of the language, it is an optional convenience).

Most failures in JavaScript are exceptions, because you are not made to check anything, and everything is dynamically typed, so if you make an assumption that isn't true, an exception has to be raised to stop you. Luckily, there is try/catch. It can also be difficult to represent errors without exceptions, you instead have things called "error states" which are basically certain non-errors being treated as errors by user code, when either that result is undesired even if perfectly normal, or that result is the only/easiest way for the error to be reported by the source.

Anyway, my point is that the article probably means to use the term exception and say that a boolean is better than an exception when the error isn't an exceptional case. And in that case I would agree. In Rust, I don't want my program to crash when I ask for a nonexistent table - I only want that to happen later if I decide that the table's nonexistence is a fatal error in the context of the whole thread.

randomdata10• 2 years ago

Sometimes they are the same

Your understanding otherwise seems on point, but I'm not sure they can ever be the same. An exception is the representation of programmer fault. An error, on the other hand, is the representation of faults outside of the programmer's control.

Like you point out, operating on a missing table is a mistake made by the programmer. This fault could have been avoided when the code was written. The table not being there is not within the control of the developer (with some assumptions of what table is meant to mean here) and no amount of perfect coding could have avoided the situation.

Most failures in JavaScript are exceptions

Idiomatic Javascript suggests handling errors using its exception handling mechanism, but the payload is still error in nature. In fact, the base object the idioms suggest you use to hold error data, passed using the exception handler, is literally called Error.

Javascript may be confused by having no official "Exception" type. Although arguably it does not need one as you can simply throw a string, which is all you need for an exception. The exception handler will tack on the rest of the information you need to know automatically. Java takes a different opinion, though, having two different, albeit poorly named, built-in classes for errors (Exception) and exceptions (RuntimeException).

tsimionescu3• 2 years ago

You are explaining the diferrence between error results and panics in Rust, and maybe this also applies to Go. But this is very much not how exceptions (a word these languages don't even use) are understood in languages which have them.

In Java, C#, C++, for example, exceptions are meant for any situation where the operation you asked for can't be completed successfully, for whatever reason. The most common example is IOException: if you asked to read 200 bytes from a socket, but the connection got interrupted after 100 bytes, that's an IOException. This would not be appropriate as a panic in Go or Rust, for a comparison of the difference in philosophy.

zachmu18• 2 years ago

Agree to disagree here. And yes, this article is a follow-up to the linked one.

The last section of the blog has a discussion of aesthetic and philosophical concerns around using errors for control flow. I'm against it, for the reasons I provide. YMMV.

sethammons3• 2 years ago

Your take away in the "ok is harmful" article is incorrect. There is nothing wrong with "ok" - your problem is with shadowing variables. The better organized code couldn't use shadowing. Your example of moving the ok-checks to the end makes as much sense as moving all your err checks to the bottom. If you overwrite a shadow variable before using the value, of course that's bad.

The real takeaway should be that one must always be careful with shadow variables and generally use them as immediately as possible.

aaomidi• 2 years ago

I agree with you.

The go standard library uses sentinel errors. ErrNotFound is actually good to use, because otherwise, a `Get` call can end up returning `nil, nil` (*Value, error). Which is just bad.

trustno2• 2 years ago

Go returns bool for map access, not errors...

zitterbewegung• 2 years ago

In error handling or even reporting status there seems to be a part of being unaware that using stdout or some other strategy is either designed incorrectly which leads to slowdowns or people don’t understand when to use it. Sure languages also have this slowdown or have some documentation about it but many people just use what seems to be correct.

L-four• 2 years ago

It shouldn't surprise anyone that a checking a bool is faster than calling a function and checking the bool it returns.

jay-barronville5• 2 years ago

The function call itself isn’t really the culprit (the cost of the function call itself is negligible in this case). It’s the implementation of the function [0].

[0] https://news.ycombinator.com/item?id=40542102

jchw• 2 years ago

I think this doesn't mean you shouldn't ever use sentinel errors, just that you should be mindful of the performance cost, as it isn't free. If you're doing this for a loop where the iteration of the loop is vastly slower, e.g. because it is performing some sort of I/O in each iteration, or if you're doing this for a loop where you know the worst-case amount of iterations is very low, then you can probably safely continue to eat the cost of using sentinel errors. The only time where it is genuinely better to avoid a performance pitfall at all costs is when scaling comes into play, since then it's very likely that it will creep up fast and make the performance absolutely unusably bad seemingly out of nowhere.

In practice, if you are really writing performance critical code, you kind of just need to benchmark and determine which bottlenecks are holding you back. Knowing the performance hit of some patterns is still useful, but it's not a substitute for actually measuring the performance.

That said, the even better solution would probably be if future Go versions could just make the idiomatic code faster somehow, so that it is less likely to be a bottleneck. But, this is the case for many patterns in Go, as Go and its stdlib doesn't really prioritize performance or zero (runtime) cost abstractions. (They justify this with data; in developers surveys, time and time again Go developers greatly express their priority in having robust, "safer" code over higher performance code.)

lenkite2• 2 years ago

Wish `errors.Is` immediately checked whether err is `nil` and returned `false`. Could have simply solved this issue I think.

jay-barronville5• 2 years ago

The first 3 lines of `errors.Is()` [0] are the following:

  if err == nil || target == nil {
    return err == target
  }

[0]: https://github.com/golang/go/blob/9b43bfbc51c469ec13fca24960...

lenkite2• 2 years ago

Glad to know they have fixed this. In Go 1.22.3 (the latest release), the first 3 lines of `errors.is` [0] are the following:

    if target == nil {
      return err == target
    }

[0]: https://github.com/golang/go/blob/go1.22.3/src/errors/wrap.g...

zachmu18• 2 years ago

Blog author here with a regretful correction. This result is not accurate, mea culpa. The headline should read: errors.Is() is 500% slower.

Basically: the critical functions in the benchmark are small enough they were being inlined by the compiler, which means it's possible for the compiler to further optimize the loop to avoid all comparisons in some cases, producing an inaccurate result for some of the benchmarks. You can fix this by adding noinline directives to the methods.

I'll be publishing an update to the article and a post-mortem on how this slipped past review. The rank ordering of techniques in the article is unchanged, but the magnitude of the difference is not nearly so large.

jay-barronville5• 2 years ago

Do you notice any difference if you update `BenchmarkFoundErrorsIs()` to the following?

  func BenchmarkFoundErrorsIs(b *testing.B) {
    var es errStore
    for i := 0; i < b.N; i++ {
      val, err := es.GetValue(true)
      if err != nil {
        if errors.Is(err, notFoundErr) {
          b.Fatal("expected found")
        } else {
          b.Fatal(err)
        }
      }
      if val == nil {
        b.Fatal("expected not nil")
      }
    }
  }

zachmu18• 2 years ago

That's the BenchmarkFoundErrorsIsNilCheck variation, and it eliminates the performance penalty on the happy path. Still just as bad on the error path.

rsc• 2 years ago

Thanks for posting the update here. I was going to point out that 0.5ns for anything is almost always a sign of benchmarks being optimized away.

But even if those numbers had been accurate, it is important to think about how they compare to the operations being performed. Taking 4 nanoseconds to check for an error sentinel is not a huge deal when you've spent microseconds or even milliseconds waiting for a network service.

zachmu18• 2 years ago

You'll never get those 4 nanoseconds of your life back though

fl0ki2• 2 years ago

It's a relatively bigger deal for many other potential sources of errors, like parsing a string as a number. Also, even if the caller does not unwrap the error values, producing errors often requires at least one heap allocation.

These are all fair tradeoffs for the simplicity and ergonomics of Go's error handling. They rarely matter for performance because the happy critical path of most programs does not produce lots of errors. These are just nuances worth recognizing in a thread specifically about the isolated overheads of errors.

zachmu18• 2 years ago

Corrections and new title are now live. Thank you for your forbearance.

hinkley2• 2 years ago

The thing with programming language evolution is that quite often everything except the error handling gets faster over time. If you have an app that makes aggressive use of errors, you’re not going to see the same speedups from version to version that others are seeing.

Particularly in JITed languages, if the multiplier is hyperbole today, there may be a day in the future where it’s accurate.

rafaelmn• 2 years ago

The thing with programming language evolution is that quite often everything except the error handling gets faster over time.

Funny I was just reading this the other day : https://learn.microsoft.com/en-us/dotnet/core/whats-new/dotn...

It's probably not the first thing you spend effort on when optimizing but it gets better over time.

samatman2• 2 years ago

The minimum cost of an error is one cold branch. Which is effectively free on the happy path, and as cheap as it possibly can be on the error path.

A well-designed error handling system can feasibly reach this goal, at which point no more optimization is possible.

Edit @hinkley: I am unfortunately in hacker jail, and have used up my post quota, so I have to edit my post rather than reply to yours. Reply follows.

Throwing, the way Java and C++ do it, exceeds the minimum cost of an error, yes. Not that this is some sort of show-stopping problem, I'm discussing the minimum cost, not what level of cost makes errors too expensive.

A branch-not-taken, which is the opposite of processing an error, caught or otherwise, is what I described as practically free. Not always. If the only path forward is, for instance, to reference a pointer, and the error would mean that the pointer is a null pointer, the chip would have to wait for the branch to coalesce, which might mean a minor pipeline stall.

But if the processor can do further work, such as checking for an error on malformed input, where it's legal to speculatively execute the next decode on the predicted branch, it really is practically free. It costs considerably less than one instruction. If the branch triggers, the processor spills all the speculative work and switches. Since errors are expected less frequently than success, more or less by definition, that's the correct tradeoff.

This is why I described the error branch as "cold". The processor should predict and speculate on the happy path.

Zig is an example of a language where errors can be as inexpensive as a superscalar check on a given register while the main path moves forward. This is also possible in C when checking errno(). I can't speak for Rust, but enormous amounts of work have gone into that compiler, and it should indeed be possible to compile a match-on-Result to the same efficient level. Perhaps trickier since Rust doesn't "know" what an error is, but I suspect that in practice it can get this right.

Not that it's even mostly up to the compiler: there are ASM hints for the branch predictor in some instruction sets, but modern branch predictors don't need those to handle the case where 99+% of the time execution proceeds on one branch. It's conventional for compilers to emit "branch on cold, proceed on hot", but it's been a long time since that has mattered for execution speed. You'll get a bit tighter on a chip with no predictor and a shallow pipeline, like AVR, but for anything which can run an operating system it's almost irrelevant now.

I see no a priori reason why Go shouldn't achieve the minimum level either. Languages which use exceptions and try/catch must do some bookkeeping, which imposes a cost on the code whether an exception is raised, or not. That can be minimized, but not to the limit I'm talking about.

You might find this link interesting: https://ziglang.org/documentation/0.12.0/#Implementation-Det...

Note that the costs described are for the ability to generate an error trace, not error handling itself. This cost is only paid in debug and ReleaseSafe modes.

SPascareli132• 2 years ago

How is this not a compiler bug? What would justify a nil check being many times more expensive then a bool check?

tedunangst2• 2 years ago

The nil check is not more expensive.

SPascareli132• 2 years ago

Did you read the article? It clearly show that it is. I just don't understand why.

jay-barronville5• 2 years ago

Anyone who’s curious why `errors.Is()` is so slow (relatively speaking) should read the implementation [0]. It’s pretty obvious once you read the code—it relies on reflection followed by a tree-walking algorithm.

[0] https://github.com/golang/go/blob/9b43bfbc51c469ec13fca24960...

esprehn2• 2 years ago

The happy path is still just nil comparison though:

  if err == nil || target == nil {
      return err == target
  }

So the overhead is the function call and 2 comparisons. I wonder if the `isComparable := reflectlite.TypeOf(target).Comparable()` line was moved into a separate function if the compiler would inline Is() reducing the perf impact.

Possibly related: https://github.com/golang/go/issues/61502

Looks like they're working on improving the inliner.

jamespwilliams2• 2 years ago

That err == nil check was added very recently and isn’t included in the latest Go release, which the author used in this article.

emmelaich• 2 years ago

Is the performance lost in the panic recover() not the actual panic?

Just a guess, I'm not a Go person.

If there's no attempt to recover in the code, then panic() wouldn't need to save any state and the compiler might just optimise out anything needed for recover()y.

Groxx• 2 years ago

For other fun surprises that are rather obvious from the implementation but still surprise people quite regularly: wait until you see how much slower context keys are when compared to thread locals, particularly since it's normal for contexts to contain MANY layers (as opposed to errors, which are frequently* returned without wrapping at almost any level). It's a glorified linked-list lookup, and it re-walks the whole path every time.

But for both contexts and errors: imo the tradeoff is overwhelmingly in favor of wrapping more, not less, in the vast majority of code. Context is useful and standard for many things, and wrapped errors carry a lot more useful information, and both of those are absolutely crucial for healthy interop with other code. Performance sensitive stuff can do whatever it needs, these are obviously terrible choices there so just don't even consider it.

* but not always! and it may change with time. Still, my contexts are regularly 10+ layers deep while my errors are rarely more than one or two.

neild• 2 years ago

There are some interesting results in here, but the slower cases are a bit misleading. The majority of time in the slow cases is spent constructing errors, not in errors.Is.

Some background for anyone not familiar with Go errors:

A Go error is an interface value with an Error method that returns the error's text. A simple error can be constructed with the errors.New function:

  var ErrNotFound = errors.New("not found")

A nil error indicates success, and a non-nil error indicates some other condition. This is the infamous "if err != nil {}" check. Comparing an error to nil is pretty fast, since it's just a single pointer comparison. On my laptop, it's about 0.5ns. Comparing a bool is about 0.3ns, so "err != nil" is quite a bit slower than "!found", but it's really unlikely the 0.2ns is going to be relevant outside of extremely hot loops.

We can also compare an error to some value: "if err == ErrNotFound {}". In this case, we say that ErrNotFound is a "sentinel" (some error value that you compare against). This is about 2.3ns on my laptop; there are two pointer comparisons in this case and a bit more overhead in comparing interface values. (You can actually make this check almost arbitrarily expensive; you could have an error value that's a gigabyte-large array, for example.)

It's common to annotate an error, adding some more useful information to it. For example, we might want our "not found" error to say what was not found:

  return fmt.Errorf("%q: not found", name) // "foo": not found

This is quite a bit more expensive than "return ErrNotFound". The fmt.Errorf function will parse a format string, produce the error text, and make two allocations (one for the error string, one for a small struct that holds it). This is about 84ns on my laptop--168 times slower than the fast path! But 84ns is still pretty fast, and you can't get away from the need for at least one allocation if you want to return an error that's varies based on the inputs of the function that produced it. (You can get faster than fmt.Errorf if it matters, but this comment is already getting large.)

A problem with using fmt.Errorf in this way is that you can't test the error against a sentinel any more. This was addressed a while back in Go 1.13 with the addition of error wrapping. You can return an error that wraps the sentinel (note the %w format verb):

  return fmt.Errorf("%q: %w", name, ErrNotFound) // "foo": not found

And you can then use the errors.Is function to ask whether an error is equal to ErrNotFound, or if it wraps ErrNotFound:

  if errors.Is(err, ErrNotFound) { ... }

On my laptop, producing a wrapping error like this and testing it with "err != nil" is about 91ns, and testing it with "errors.Is(err, ErrNotFound)" is about 98ns. So using Is is adding 7ns of overhead, which is not nothing, but is also pretty much lost in the noise compared to creating the error in the first place.

The example in this blog post went a step further, though, and created an error with not just a single layer of wrapping but one with four. The error text in the wrapped error cases is:

  GetValue couldn't get a value: queryValueStore couldn't get a value: queryDisk couldn't get a value: not found

(That is, by the way, a very difficult error to read. Don't hand users errors that look like that.)

Creating a stack of four wrapped errors like this on my laptop is 396ns, and inspecting it with errors.Is is another 21ns. 21ns is waaaaay more than the 0.5ns for a simple "err != nil" check, but again the runtime here is massively dominated by the expense of creating the error--which in this case involves repeatedly creating formatted strings and throwing them away, and two allocations for each layer in the stack.

In general, when doing low level optimization of Go code, avoiding allocations is the biggest bang for your buck. If microseconds matter, you absolutely should pay attention to the cost of constructing error values. But the cost of inspecting those values doesn't usually become an issue unless nanoseconds count, and will generally be dominated by the cost of construction.

Also, even the slowest cases here are running about 0.5-1.5μs, which absolutely matters in some cases, but is irrelevant in many others.

zachmu18• 2 years ago

This is true.

We have in the past eliminated sentinel errors and gotten measurable gains from doing so, but this is mostly because our errors were both common (occurring several times on every request) and expensive to construct. The direct cost of errors.Is() was minor compared to the cost of building the error itself.

And you might be surprised how common it is for people to insist on wrapping an error at every layer of the stack. I've gotten in arguments with these people online, they're out there.

sethammons3• 2 years ago

I insist on wrapping all errors each time and only removing that when performance testing shows it to be a bottleneck. A top concern of my systems is debugability which includes descriptive, wrapped errors with structured logging (this is a super power for system development and I am surprised when folks don't give love to structured logs and detailed, reproducible errors).

I want organizational velocity in the general case. If wrapping an error is in a hot path and shows up in metrics, yeah, remove the wrapping. Otherwise, wrap the error.

What is your argument against that? It would seem you find the compute savings of non-wrapped errors outweighs developer time and customer impact. If that is not what you are saying, please correct me.

My creds are using Go since 1.2 and writing massively scaled systems processing multibillion events daily for hundreds of thousands of users with 4 to 5 9s of uptime across dozens of services maintained by hundreds of developers earning the company hundreds of millions of dollars.

beautron• 2 years ago

In general, when doing low level optimization of Go code, avoiding allocations is the biggest bang for your buck.

I have found this to be the truth. I'm making a game in Go, and have learned that a smooth framerate depends on programming with allocation awareness. Always know where and when your memory is coming from.

eightnoteight• 2 years ago

The problem is that sentinel errors, as typically and idiomatically used, in fact are special, and are more expensive to deal with than other values. My suggestion to use boolean values outperforms them by a lot, 30x in fairly common idiomatic usage.

while I agree to some degree, but when performance comes into picture, what really matters more is normal path vs surprise path rather than happy path vs error path

it's hard to argue what is a happy path in the code, but it's not wrong to say that io.EOF check is a normal path of the code i.e. not a surprise in production. the bad performance of errors.Is is something to be improved upon but it's not a surprise in production when there are a large number of `errors.Is` checks during normal path of the code

now coming to the surprise path of the code, here's where performance gets really important, no one wants their code to suddenly hog 100% CPU because of some special error case - https://blog.cloudflare.com/cloudflare-outage . but such surprise paths often contain a large amount of business logic that weigh much more than how slow errors.Is function is compared to a boolean check

it would be interesting to see where this line of reasoning is valid but IMO performance isn't a good argument against why errors are not normal outcomes of operations in production

but thumbs up for the article, now I know what to reference for backing the below pattern that I often use, when I first saw the errors.Is it was pretty obvious that its going to be slow but just didn't have time to prove it and use below pattern

```

if err != nil && errors.Is(err, x) {

} else if err != nil && errors.Is(err, y) {

} else if err != nil {

    // handling unknown error

}

```

neonsunset8• 2 years ago

The numbers in the benchmark are unsurprising for Go.

Comparable code (for errors.Is()) written in C# takes about 4ns on M1 Pro:

(made unidiomatic and abstracion-heavy-ish for demonstration purposes)

https://gist.github.com/neon-sunset/65a8deeaac745ba159f990c7...

Of course were it rewritten to a TryGet or (object, bool), it would bring that to 1-3 CPU cycles, as everything would get inlined and lowered to a few branches.

randomdata10• 2 years ago

For gc. The was no attempt to benchmark under gccgo, tinygo, etc. so we don’t really know how Go fares. Go is, by explicit design, multi-implementation.

neonsunset8• 2 years ago

Do you have numbers for any of these on hand? I'm not holding my breath (after all, there's still issue of this being Go itself, which is inadequate at systems programming) but it's interesting to see how alternative implementations fare nonetheless.

(this does not make the point any less valid as the solution this is applied to discussed in the blog post uses "vanilla" runtime flavour)

12345hn6789• 2 years ago

Thanks for the updates and article.

If this is s big deal perhaps you should be choosing a different language :)

Also BTW the nested `errors.Is` bench is wrong. Each function is wrapping the errors instead of constructing them preemptively and this is benching the time to create nested errors. Don't do this, in the test or real code. It makes it tough to read as a client.

-mlv• 2 years ago

I really don't want to be that guy, but I think the author meant to say '96.77%' instead of '3000%', or '96.66%' instead of '30x'.

larsrc2• 2 years ago

You understood what they meant, though. But yeah, it's confusing. I argued with my PhD advisor over this several times. "50% slower" or "10% slower" is ready to understand, but "96% slower" is harder to figure out, and it's easy to dismiss the difference between 96% and 98% because the numbers are similar

samatman2• 2 years ago

As far as I can tell, this is an aspect of language which people merely pretend to have problems with, out of pedantry, rather than a barrier to understanding in any possible case.

Everyone understands that if A is 2x faster than B, then B is 2x slower than A. If you say either of these things to someone with a basic grasp of sums, and then tell them A takes one second, they'll know that B takes two seconds. It doesn't matter which one you supply. No one would get this wrong.

ongy• 2 years ago

@zachmu can you upload the code you ran to a public repo?

I'd like to play with it a bit, to compare patterns I've done before and am too lazy to copy+paste ;)

zachmu18• 2 years ago

There is a link in the blog to a public gist, it's just a single file you can download and run.

tommiegannert2• 2 years ago

Nice write-up.

It's a shame that errors.Is is slow for general use, and at least some of that seems attributable to the Comparable change requiring reflection. Multi-errors seems to have bloated the switch. And of course the lack of a happy-path that was fixed in [1].

Since Go already has two ways of handling exceptional state: return or panic, it does feel like a stretch to also introduce a "not found" path too. All bets are off in tight inner loops, but I think as a general coding practice, it'll make the language (de facto) more complicated/ambiguous.

But my take away is that the question has been kicked off: can wrapped errors be made more efficient?

1. https://github.com/golang/go/commit/af43932c20d5b59cdffca454...

boomlinde• 2 years ago

I would argue that "not found" is already present in the form of boolean "ok" return values, in cases where it really isn't an exceptional error state. On the other hand, the standard library implements things like errors.NewNotFound

Ferret7446• 2 years ago

This calls for the classic quote: "Premature optimization is the root of all evil"

I expect the vast majority of code spends so little time in errors.Is() that trying to optimize it saves nothing. Profile first, and change it later if needed; as Go is statically typed such a change is trivial.

jamespwilliams2• 2 years ago

https://go-review.googlesource.com/c/go/+/576015 this change to optimise errors.Is in the err == nil case was merged recently. It’s not included in the latest Go release, hence the stats here, but should end up in a release pretty soon I’d imagine.

zachmu18• 2 years ago

That's pretty cool, I'll try to remember to post a follow up when this makes it into a release.

liampulles• 2 years ago

This is an interesting experiment.

I have to say that nearly all the production code I write in Go is typically stuff interacting with external services like databases and SaaS APIs, so the difference in nanoseconds here is not going to move me away from using the safe idiomatic approach, but it is something to bear in mind for whenever I might use Go for very computationally intense tasks.

jakjak123• 2 years ago

Interesting enough, but this misses most of the point. In any case, its not really that interesting if the value you looked for is missing or not, its way more important that I can see _which_ value was not found. Any solution that does not capture which value was missing, is kind of besides the point for most practical applications.

zachmu18• 2 years ago

You can get that by implementing an error type, or otherwise constructing a new error every time one is returned. But doing this adds to the cost of returning the error.

This is how we use most errors in our code base. But we tend to not treat them as sentinels, we are only interested in the descriptive error messages.

amelius2• 2 years ago

Error handling is one of the reasons why I lost interest in Go.

randomdata10• 2 years ago

Regardless of language, error handling is my favourite part of programming. It’s where the most interesting problems lie.

What motivates the “happy path only” programmer?

amelius2• 2 years ago

Perhaps, but without the proper tools/primitives it is certainly not my favorite part of programming.

cedws• 2 years ago

This isn't really relevant unless you're error checking inside a hot function - in which case, you need to rewrite your hot function. The majority of code out there is I/O bound (network/disk) anyway.

marcrosoft• 2 years ago

I’m still using if err == something. I’ve never bothered to use error.Is. Looks like another reason to avoid

zachmu18• 2 years ago

The problem is error wrapping. Somebody deeper in the stack can break your equality check.

teepark2• 2 years ago

Comparing to the speed of a direct boolean check is a great way to sensationalize really small numbers.

Nobody's real-world code is being slowed down by 500% because all real world code is doing much more than just checking errors. All I see from these results is a 15-16ns cost to using errors.Is versus an additional boolean.

Even the examples ("GetValue") hint at an extremely common use case: reads from a data store. A case where single-digit milliseconds is considered "all good performance-wise" for the most common SQL databases, clocking in at 100000x the time scale of errors.Is.

janderland• 2 years ago

I thought this as well, but I also think it depends on how you’ve structured your code. If errors aren’t used by the algorithm (they’re just used for early exit) then yes it won’t make a big difference, but if your algorithm is handling errors nearly as frequently as the happy path then you’ll see a difference using the boolean.

teepark2• 2 years ago

Yeah, of course if you're just running through an in-memory slice and doing a little arithmetic on each item, sentinel errors and errors.Is might dominate your runtime. But the dominant use-cases (e.g. in the standard library) are syscalls, filesystem interactions, network APIs - you know, stuff that takes real time.

This reminds me of "latency numbers every programmer should know". Work with the standard library and other well-conceived go projects and you gain the intuition that error handling and bit-twiddling arithmetic don't belong together. That's the real story here, and OP's article is way wide of the mark.

kiitos8• 2 years ago

The absolute values here are in terms of single-digit nanoseconds.

The Venn diagrams of "situations where single-digit nanosecond performance differences are important" and "situations where one would use errors.Is" have no overlap.

Tempest in a teapot.

bool3max• 2 years ago

Towards the end of the article, what does the author mean by "sentinel errors are expensive to construct"? Aren't they called sentinel errors because they are constructed only once?

Go: Sentinel errors and errors.Is() slow your code down by 3000%

Related Stories