Error stack traces in Go with x/xerror

by sicromoft • 5 years ago

100 107 5 years ago

Hmmm...

  and (2) a way to cut down on if err != nil { ... }
  boilerplate that pervades all Go code.

Am I weird in liking the explicit error handling? :/

codegeek• 5 years ago

there are arguments against it but I frankly like it being this explicit as a newbie in Go. So you are not alone :)

throwaway894345• 5 years ago

I’ve been writing Go for a decade and I’ve liked it from the beginning, but stack trace support is welcome.

BobbyJo• 5 years ago

Explicit error handling and a lack of generics were a pain when I first started with Go, but after a few years I see them as amazing features.

OJFord5• 5 years ago

I've learned to consider it an excellent feature, but is Go's implementation/idiom really better than anything that came before it?

I have marginally more experience with Rust (not that much of either) which instead gives Result types, tuples of 'ok' and error values. And, crucially for this thread, they must be explicitly consumed.

That enforced error handling is novel to Rust afaik, and after a bit of getting used to, an excellent feature I think.

But I'm not sure that the idiomatic (because that's all it is really, rather than language feature?) Go is materially different from deciding all your Python functions will return Tuple[TOk, TErr] and sticking to it, or even that different from returning ok types only and raising exceptions, really.

andrenth• 5 years ago

That enforced error handling is novel to Rust afaik

Hardly, though Rust of course has merit in making it popular, to the point that people think it was introduced by the language.

suresk3• 5 years ago

I don't think you're weird - I understand that some people like it and I can kind of see some of the arguments as to why - but the error handling in Go is easily my least favorite thing about it. For two reasons:

1) The aforementioned boilerplate code that makes up a significant chunk of many Go projects and adds no value.

2) With explicit error handling, however deep your call stack is, you are relying on everything in that stack to have done the right thing with regards to error handling. With exceptions, you have to go out of your way to screw them up.

I can't tell you how many times I've gotten dumb error messages like "Invalid string" from a 3rd party library that take forever to debug, and accidentally swallowing an error is even worse. A simple (even crappy) exception message + a stack trace is much easier for me to use for debugging than even the best handcrafted error message in 99% of cases.

I write somewhat equivalent amounts of Python, Java, and Go lately, and each one has their good and bad parts, but there are few things I dislike as much as Go's error handling patterns.

ithkuil2• 5 years ago

With exceptions, you have to go out of your way to screw them up.

I remember well java codebases littered with:

    try { .. } catch() { // Todo }

Usually cheaply inserted by the IDE.

I'm pretty sure nowadays there are linters that will ensure you have to do some extra work to actually check in such code, but still...

radlad• 5 years ago

With explicit error handling, however deep your call stack is, you are relying on everything in that stack to have done the right thing with regards to error handling. With exceptions, you have to go out of your way to screw them up.

I don't think I agree with this argument. In many languages, there's no obvious indicator that any given function may or may not raise / throw an exception. And even if it doesn't throw an exception today, it might tomorrow, and it's easy to forget to update all callers.

Since Go makes errors a return value you have to actively discard the result (by replacing it with a underscore, e.g. `res, _ := getResults()`) or take the time to handle the error.

And just like an intermediary library in Go can swallow the error, so can an intermediary library in an exception language catch and discard an exception.

It seems to me that the result is more errors are properly handled in Go - because they are explicit - while uncaught exceptions often cause bugs that make it to production.

For context, I recently switched jobs from one where I wrote Python for 4.5 years to a job where I've been writing Go for about 5 months.

laumars7• 5 years ago

There’s two types or reason for a program to error:

- the code has failed and the developer needs to be informed what to fix

- the environment has failed and the user needs to be informed

Exceptions are for the former, which Go still supports via Panic(). Errors are for the latter. Which is why all the boilerplate Err code creeps in.

I don’t like getting stack traces from applications when I’m a user. It always feels to me like the application is only half finished. When the error is environmental (eg file system permissions) a stack trace just muddies the water with unnecessary output.

This is why I like Go’s distinction between errors and exceptions.

nightfly• 5 years ago

I prefer how Rust wraps it in a type. Makes it seems like "part" of the language, rather than something bolted onto every function.

iambvk• 5 years ago

It becomes painful to write in unit tests.

I really liked their `check` proposal. I hope they will bring it back to life.

politician• 5 years ago

Do you use stretchr/testify/assert? I find that it makes error handling in tests as natural as assertions on other properties.

benhoyt• 5 years ago

Aside from the "assertion" libraries, which I have mixed feelings about, I often use table-driven tests to avoid this. That way each test is generally one line, like `{"1234", 1234}` for testing an Atoi function. Or, I've seen little test helper functions that "fatal" the test on error -- that way it only takes one line instead of three: https://play.golang.org/p/WIALie5fsgd

ibraheemdev• 5 years ago

I think the main issue is not the boilerplate as is usually the source complaints, but the fact that not handling errors is so easy due to shadowing.

tsimionescu6• 5 years ago

Both are problematic. Boilerplate does wonders for mistakes hidden in plain sight - especially when you have that one rare piece of code that actually does something with an error, and it completely slips by code review because anyone who works with Go has long learned to glaze over anything that begins with if err!=nil

tomohawk• 5 years ago

In my experience, the people who complain about this the most are the least likely (in go or other languages) to pay proper attention to handling errors, and are the most likely to pay way too much attention to how code looks. Not just tidy, but a certain style. They're used to not seeing code in places where errors are emitted (exceptions) and putting magic try/catch blocks that often lead to imprecise error handling.

bccdee• 5 years ago

I like explicit error handling, but I wish there were a concise way to handle errors from the same line. A one-line function call `foo()` becomes four times as long once you handle errors.

    result, err := foo()
    if err != nil {
        return nil, fmt.Errorf("Error message: %w", err)
    }

This pattern is just so commonplace, but it's four times as long.

Contrast with Rust, which handles errors as `foo()?` or `foo().context("Error message")?`*. It's still explicit — I like it better than silently-propagating exceptions — but my functions don't wind up being 75% error handling.

* Where the `context` function comes from an error-handling library.

xpressvideoz• 5 years ago

I don't buy the argument that Go makes error handling explicit. It is too easy to forget to check for an error in Go. Did you know that `fmt.Printf` can also fail? Do you explicitly check for `err` when using it? Probably not, because what Go calls explicit error handling is merely a convention, not something supported by the language, e.g. [[nodiscard]] from C++ or #[must_use] from Rust. And who thought reassigning `err` multiple times was a good idea? That again strikes a lack of functionality in Go.

What saddens me is that these kinds of matters will never be fixed in Go. Go has a stubborn anti-feature mentality and while it does help preventing feature creep, it overall harms the language in the long run. For a language created and maintained by Google this is a huge missed opportunity.

makapuf2• 5 years ago

There has been a go2 proposal for better error handling, so there is no issue trying to improve things or adding needed features. But things go slowly and must be discussed and vetted before being accepted because the language has few features and they will stick so the filter is quite high. See generics, it took a long time but its coming.

thiht• 5 years ago

Did you know that `fmt.Printf` can also fail? Do you explicitly check for `err` when using it? Probably not, because what Go calls explicit error handling is merely a convention

I was curious about this and it’s true, I didn’t know these functions could err, TIL.

The Godoc states:

It is conventional not to worry about any error returned by Printf

xh-dude• 5 years ago

I’m not sure the mentality is exactly “oops, oh well, leave it broken forever” [0]; it is intentionally ponderous. As a stop-gap, linters [1] can get excellent insight from ASTs.

The stability of Go is something I value a lot, so I don’t mind stop-gaps like this too much on balance.

[0] https://github.com/golang/go/issues/20148

[1] https://github.com/golangci/golangci-lint

kortex• 5 years ago

I just want Result types I can map over. That would cut out a bunch of clutter. Even if I could pair one fallible function with one infallible function, that's 50% less err != nil

   val, err := int_or_err(foo)
   if err != nil { return nil, err}
   return val.add(bar), nil

   result := int_or_err(foo)
   return val.map(add,bar)

I'd even take some sort of

   result := int_or_err(foo)
   return val.map(add,bar).tuple()

to return a val,err pair in order to match conventional go.

Cthulhu_• 5 years ago

It looks more concise but if you're like me not into functional programming much, .map() is magic.

Consistency is more important than conciseness. Clear is better than clever. Plus, a function invocation / .map() is heavier than an err != nil check / condition.

Don't get me wrong, I appreciate the Either pattern as an alternative to if err != nil, but I also appreciate the really dumb and straightforward approach of non-clever Go.

obviouslynotme2• 5 years ago

I hate it. All it does is add if err != nil { return nil, err } to every other line. I love Go, but I have never understood the obsession with generics over putting try into the language. This is the biggest pain in Go, by far.

The vast majority of errors only stop or rollback the current action. An incredibly small amount of code uses errors to detect stop conditions or to retry.

aniforprez• 5 years ago

I know people on HN hate python to some extent because of performance issues but I love the error handling in python. Trying to learn go on the side this is the biggest thing by far that just halted all of that. No error handling? I was baffled. I hope they add this into the language. There's no way I'm wasting time writing `if err != nil` all over my code. try/catch blocks please

simiones6• 5 years ago

To be fair, generics can allow you to get rid of some of this boilerplate, in principle. Though I would still love to get some built-in error handling.

beltsazar• 5 years ago

Exception error handling is implicit because we don't know which line inside a try block can throw an exception.

However, Go's `if err != nil` is not more explicit than, for example, Rust's question mark operator. The former is more verbose than the latter, yes. But both are explicit in that we can know which line can return an error.

The `if err != nil` is probably okay if there are only a few lines that can return an error, but if most lines in a function can return an error, it will result in too much noise. A real world example where panic is abused as an exception-like approach because the "proper" error handling using `if err != nil` is way too verbose: https://pkg.go.dev/github.com/apple/foundationdb/bindings/go... (Actually Go's stdlib too sometimes abuses panic in a similar way.)

vips7L2• 5 years ago

Exception error handling is implicit because we don't know which line inside a try block can throw an exception

You just need to have some thought when writing your code:

  let a;
  try {
    a = someFailingFunc();
  } catch (e) {
    // handle
  }

  // use a
  // do work

masklinn• 5 years ago

Am I weird in liking the explicit error handling? :/

The issue is not explicit error handling it’s specifically Go’s, which is verbose and half-assed. Not entirely unlike java’s checked exceptions though unlike checked exceptions we have plenty of other (and I’d argue better) implementations of “explicit error handling”.

Go’s error handling is a relatively minor improvement on C’s, but we’ve gotten quite a ways beyond that since.

SamWhited• 5 years ago

I tend to agree, I dispute the premise that this is a problem a bit. I mean, I get not wanting to type the same thing repeatedly, but when reading code later it's really nice to have the logic explicitly in front of me and not hidden behind some other syntax or function. It could possibly be done better, but it doesn't need to be done better, IMO.

IshKebab• 5 years ago

I don't think so. I like that it strongly encourages you to actually add human readable context to errors too.

I think people's issue is that there's no "I don't care about errors, just show me a stack trace" option like you get with exceptions, or more or less with Rust's `?` if you don't use `.context()`.

alecthomas• 5 years ago

One extra tip when wrapping error return values is to use the wrapcheck (https://golangci-lint.run/usage/linters/#wrapcheck) linter. This will tell you when you're returning an error without wrapping it.

ollien5• 5 years ago

Like the article mentions, they didn't bring over stack traces (namely the `Formatter` interface) from xerrors. I wrote a library[1] around it that would generate true stack traces. I don't use it as much as I used to, because I don't want to depend on a package like xerrors I don't trust to remain maintained, but it was a fun exercise at the time, and very useful while I used it. I wish that we wouldn't have to depend on a tool like Sentry for bringing this about, like the author suggests.

[1] https://github.com/ollien/xtrace

durbatuluk• 5 years ago

github.com/pkg/errors also is a amazing option

ollien5• 5 years ago

Yeah, for sure! I mention it in the README of that library, but one of the motivations I had was to not require you to wrap the errors with my library, like pkg/errors requires you to.

siasia• 5 years ago

https://github.com/cockroachdb/errors

jrockway2• 5 years ago

People seem to fixate on stack traces, because other languages present nearly every error in stack trace form. I think you should think about why you want them and make sure you have a good reason before mindlessly adding them. I do collect stack traces in some Go code, because Sentry requires it for categorization, but in general, you can do a much better job yourself, with very little sorcery involved.

A common problem is that when multiple producers produce failing work items and send them to a consumer -- a stack trace will just show "panic -> consumer.doWork() -> created by consumer.startWork()". Gee, thanks. You need to track the source, so that you have an actionable error message. If the consumer is broken, fine, you maybe have enough information. If a producer is producing invalid work items, you won't have enough information to find which one. You'll want that.

The idea of an error object is for the code to make a decision about how to handle that error, and if it fails, escalate it to a human for analysis. The application should be able to distinguish between classes of failures, and the human should be able to understand the state of the program that caused the failure, so they can immediately begin fixing the failure. It's up to you to capture that state, and make sure that you consistently capture the state.

Rather than leaving it to chance, I have an opinionated procedure:

1) Every error should be wrapped. This is where all the context for the operator of your software comes from, and you have to do it every time to capture the state of the application at the time of the error.

2) The error need not say "error" or "failure" or "problem". It's an error, you know it failed. As an example, prefer "upgrade foos: %w" over "problem upgrading foos: %w". (The reason is that in a long chain, if everyone does this, it's just redundant: "problem frobbing baz: problem fooing bars: problem quuxing glork: i/o timeout". Compare that to "frob baz: foo bars: quux glork: i/o timeout".)

But if you're logging an error, I pretty much always put some sort of error-sounding words in there. Makes it clear to operators that may not be as zen about failures as you that this is the line that identifies something not working. "2021-08-23T20:45:00.123 PANIC problem connecting to database postgres://1.2.3.4/: no route to host". I'm open to an argument that if you're logging at level >= WARNING that the reader knows it's a problem, though. (I also tend to phrase them as "problem x-ing y" instead of "error x-ing y" or "x-ing y failed". Not going to prescribe that to others though, use the wording that you like, or that you think causes the right level of panic.)

3) Error wrapping shouldn't duplicate any information that the caller already has. The caller knows the arguments passed to the function, and the name of the function. If its error wrapping needs those things to produce an actionable error message, it will add them. It doesn't know what sub-function failed, and it doesn't know what internally-generated state there is, and those are going to be the interesting parts for the person debugging the problem. So if you're incrementing a counter, you might do a transaction, inside of which is a read and a write -- return "commit txn: %w", "rollback txn: %w", "read: %w", "write: %w", etc. The caller can't know which part failed, or that you decided to commit vs. roll back, but it does know the record ID, that the function is "update view count", etc.

The standard library violates this rule, probably because people "return err" instead of wrapping the error, and this gives them a shred of hope. And, it's my made-up rule, not the Go team's actual rule! Investigate those cases and don't add redundant information. (For example, os.ReadFile will have the filename in the error message, because it returns an fs.PathError, which contains that. net.Dial is another culprit. Make a list of these and break the rule in these cases.)

4) Any error that the program is going to handle programmatically should have a sentinel (`var ErrFoo = errors.New("foo")`), so that you can unambiguously handle the error correctly. (People seem to handle io.EOF quite well; emulate that.)

You can describe special cases of your error by wrapping it before returning it, `fmt.Errorf("bar the quux: %w", ErrFoo)`.

Finally, since I talked about logging, please talk about things that DID work in your logs. Your logs are the primary user interface for operators of your software, but often the most neglected interface point. When you see something like "problem connecting to foo\nproblem connecting to foo", you're going to think there's a problem connecting to foo. But if you write "problem connecting to foo (attempt 1/3)\nproblem connecting to foo (attempt 2/3)\nconnected to foo", then the operator knows not to investigate that. It worked, and the program expected it to take 3 attempts. Perfect. (Generally, for any long-running operation a log "starting XXX" and "finished XXX" are great. That way, you can start looking for missing "finished" messages, rather than relying on self-reported errors.)

(And, outside of HN comments, I would make that a structured log, so that someone can easily select(.attempt == .max_attempts) and things like that. It's ugly if you just read the output, but great if you have a tool to pretty-print the logs: https://github.com/jrockway/json-logs/releases/tag/v0.0.3)

Anyway, I guess where this rant goes is -- errors are not an afterthought. They're as much a part of your UI as all the buttons and widgets. They will happen to all software. They will happen to yours. Some poor sap who is not you will be responsible for fixing it. Give them everything you need, and you'll be rewarded with a pull request that makes your program slightly more reliable. Give them "unexpected error: success", and you'll have an interesting bug report to context-switch to over the course of the next month, killing anything cool you wanted to make while you track that down.

simiones6• 5 years ago

I like your argument in principle, but in my experience automatic exception stack traces in logs can get you ~80% of the way there with 0 effort, even more if there are enough happy path logs as well (e.g.

  $time, INFO, reading from /tmp/some-file
  $time, ERROR, FileNotFoundException at $whole-stack-trace

Conversely, the Go ecosystem is actually extremely bad with errors, using error strings almost everywhere, and adding very little context at that.

jrockway2• 5 years ago

I think it's a wash. Go programs emit "fatal error: success" less than C programs, which is nice. An exception is fine if it has the information you need to debug the problem, but the names of the functions along the way aren't enough on their own. Consider: "program failed: Get "https://does-not-exist.invalid/": dial tcp: lookup does-not-exist.invalid on 8.8.8.8:53: no such host" vs. "IOException: no such host" and then a bunch of stack frames that don't tell you someone put "does-not-exist.invalid" in the config file. (Usually the operator won't type that, they'll type correct-hostnmae.internal instead of correct-hostname.internal, in a config file with 100 other non-one-letter-typo'd hostnames ;)

mappu• 5 years ago

At $DAYJOB we had a Go dependency on a package (maybe pkg/sftp?) that used github.com/pkg/errors to capture a stack trace with the error. These errors were ultimately used in a loop for flow control (maybe testing a lot of files/servers?) where collecting all the stack traces caused a lot of slowdown, heap garbage, and GC pressure.

I used go mod replace to strip the stack trace collection out of pkg/errors, with a fork that does a no-op for that function call, and it was a significant improvement for our use case.

xyzzy_plugh• 5 years ago

Yes, I've gone back and forth on "all errors should be annotated" or "all errors should have a full stack trace" and frankly, I'm now in the "it's just an error" camp. Stack traces seem great but are kinda insane overhead. If you're bubbling up errors from a real deep place, having a $package.$method: %w wrap the error is nice, but beyond that, it's a headache.

derekperkins• 5 years ago

Insane overhead in what way?

Abishek_Muthian• 5 years ago

I used go mod replace to strip the stack trace collection out of pkg/errors, with a fork that does a no-op for that function call, and it was a significant improvement for our use case.

Is there a write-up or references on how one can achieve this? Sounds like a good practical use case for the replace directive.

gregwebs• 5 years ago

I believe the solution to this is to classify errors properly. Any "internal error" as in HTTP 500 Internal Error should be generating a stack trace. Most other expected errors (like your case) should not. I codified this practice in a library I created for Go called errcode [1] which is designed to attach error codes and other meta information where errors are generated.

[1] https://github.com/pingcap/errcode

SamuelHarris• 5 years ago

Individuals appear to focus on stack follows, in light of the fact that different dialects present practically every blunder in stack follow structure. I figure you should ponder why you need them and ensure you have a valid justification before thoughtlessly adding them. I do gather stack follows in some Go code, since Sentry requires it for order, yet as a general rule, you can improve work yourself, with very little magic included. https://www.nection.io/

ThePhysicist2• 5 years ago

I think you can just use the `Callers()` function from the "runtime" package to the get the call stack, though in general I think it's a bit overkill. I think what you should do instead is to wrap errors with '%w' [1], manually adding context to them as you pass them up your call stack. That leads to more readable code and won't incur any performance overhead. It's tempting to have the call stack available for debugging, but IMHO it creates too much overhead when doing it indiscriminately for all errors you generate, which for me also goes a bit against the "spirit" of Golang.

[1] https://go.dev/blog/go1.13-errors

tsimionescu6• 5 years ago

How many errors are you creating that you would start caring about overhead? And why do you think code is more readable if it's littered with guesses on what may be important to track down a problem, rather then the simple call stack?

marhee• 5 years ago

Because the call stack does not include the value of function call arguments. A good error message does include the call arguments values relevant to the error AND those of the relevant variables. But I agree it takes upfront effort and often also need adjustments when debugging an issue later (i.e. adding more info to the error message and rerun).

ThePhysicist2• 5 years ago

In my typical Go code many errors are handled internally, e.g. "not found" type of errors returned by database methods. It would be pretty wasteful extracting full stack traces for these and in high-performance scenarios this can really hurt you.

I think stack traces should be mostly reserved for interactive debugging and should not be included in user-facing errors. Why should the users of your program care that foo() called bar() called baz() which then produced an error? They want to know what went wrong and how to fix it (if it's "their" fault), and that is much easier if they get proper, context-specific errors (e.g. "CLI argument 'limit' must be between 1-10"). And if you need a stack trace for debugging a problem you should simply use panic().

shp0ngle• 5 years ago

From what I understand, go errors don't have stacktraces by default for performance reasons.

If all errors had stacktraces, everything would be much slower.

mozey• 5 years ago

At boundaries between our code and calls out to external packages, make an effort to always wrap the result with xerrors.Errorf. This ensures that we always capture a stack trace at the most proximate location of an error being generated as possible

I've been doing something similar for a while, using `errors.WithStack` from https://github.com/pkg/errors

The error can then be logged with https://github.com/rs/zerolog like this `log.Error().Stack().Err(err).Msg("")`

For human readable output (instead of the standard JSON) use a console writer, see https://github.com/mozey/logutil

samuell• 5 years ago

This end solution isn’t amazing, but it works. We’d of course far prefer if Go itself had a built-in equivalent.

Totally agree.

For a package like SciPipe, which we have managed to develop with zero dependencies, to maximize future reproducibility of scientific pipelines, it would very much hurt to bring in the first external (Go-) dependency, while at the same time, we really really would be much helped by stack traces.

Error stack traces in Go with x/xerror

Related Stories