Open-Source Go Libraries

https://www.spacemonkey.com/blog/posts/go-open-source

by jtolds • 12 years ago

81 22 12 years ago

While Go has very deliberately not implemented class hierarchies, a quick perusal of Go's net and os packages should indicate that sometimes error hierarchies are useful.

I just want to point out - I would hesitate from drawing too many conclusions from "net" and "os" (the latter in particular). Not only is "os" some of the oldest Go code, but it is also designed to wrap conveniently around external interfaces that follow very different idioms.

Another example would be the constant declarations[0], which use ALL_CAPS - this is actually not idiomatic in Go, but it is used here (a) because it was done very early on, and (b) because it more closely matches the C libraries it interfaces with[1].

I need to take a closer look at this errors package before drawing an opinion about the rest of it, but this sentence caught my attention.

[0] http://golang.org/pkg/os/#pkg-constants

[1] https://github.com/golang/lint/issues/32

tptacek2• 12 years ago

Adam Langley will disagree with me on this point, but if you were a client and put a choice of Golang crypto/tls and OpenSSL in front of me, I'd professionally recommend crypto/tls.

This doesn't come up that much, because most production-scale deployments terminate TLS at a cluster of load balancers, so you don't really get to choose which TLS library you use in prod. But if you have the option, I get warmer fuzzies from crypto/tls than from native bindings to the scariest library in cryptography.

This could change in a year or so when the forks settle out, but I hope not, because crypto/tls is a very nice piece of code.

jtolds6• 12 years ago

Totally. I'd love to see crypto/tls fixed. Until then, though, I'd rather not choose the option with known, existing attack vectors.

Too bad it took such a horrible bug to shake everyone into auditing these libraries.

Edit: fwiw, the hw acceleration is a pretty big deal for us. We have little ARM devices.

tptacek2• 12 years ago

Are you working from the presumption that Golang crypto/tls admits practical side-channel attacks and OpenSSL doesn't? I would question that assumption. Again: if I had to make a recommendation between the two, then from a security perspective I'd lean crypto/tls.

Regardless, if I was stuck on hardware acceleration, I'd probably stick a reverse proxy in front of my Golang code before I incorporated an OpenSSL Golang binding.

axaxs• 12 years ago

The problem seems to be that very few people understand timing attacks, and even fewer are capable or willing to fix it. This is evidenced by the fact I've seen this same argument made three or more times now, and all link to a completely technically void twitter post. Not that AGL isn't a brilliant man, mind you, but it seems people just link to the post with no real knowledge of whether the problem exists or what it means. I like go and have contributed where I can, but like most, my time is limited. You're a respected member of this community with a respect for go. Let's say we work to fix this by 1.4. Are you willing?

Xorlev• 12 years ago

Nice set of libraries. I wonder if they looked at log4go before writing their own logger, and if so what drove them to do so.

ominous_prime• 12 years ago

is there an active fork of log4go? The original hasn't been updated in 2 years, with a couple serious issues which have been open at least as long.

I would also love a flexible logging package (that isn't glog ;).

mratzloff• 12 years ago

I use seelog (https://github.com/cihub/seelog) and have been pretty happy with it.

kbd• 12 years ago

In their errors library - http://godoc.org/github.com/SpaceMonkeyGo/errors - they write:

It doesn't take long during Go development before you may find yourself wondering where an error came from. In other languages, as soon as an error is raised, a stack trace is captured and is displayed as part of the language's error handling. Go error types are simply basic values, and no such magic happens to tell you what line or what stack an error came from.

I was thinking the other day that Go's "if err != nil { return err }" is just the explicit version of what's done implicitly with exceptions, except the 'err' doesn't remember where it came from. This lack of information seems like a pretty big inconvenience. How do Go developers typically deal with this problem?

nknighthb• 12 years ago

I'm not sure what kind of inconvenience it seems like. If you do extensive logging (which you should), and you set Llongfile or Lshortfile (which you should), you will at least know where the error bubbled up into your own code. And you can always get a stacktrace from runtime/debug.PrintStack(), if you want, though I've only had real cause to use it once or twice. (In the event of a panic, you do automatically get a stack trace, though most of the time when that happens I barely have cause to glance at it.)

Idiomatic Go is not so deeply layered as some Java stacks I've seen. It is more horizontal than vertical. If you call a function in somebody else's library and get an error back, the source of the error is probably not ten layers further down.

grey-area• 12 years ago

If you're logging errors, you can use the runtime package to find out where a logging statement was called. So you can log on error and record the line, function name etc:

    pc, _, line, _ := runtime.Caller(1)
    name := runtime.FuncForPC(pc).Name()

Not that you need this much, but it is there if you need to return information on exactly where an error came from when logged, just put that in your logging function and whenever you log it'll trace where the log was called from, or you could store the info in your error object if you wish. log4go has some examples of this sort of thing I think. Typically I've only used that when there was an intractable bug though, normally you're not going to need it if your error objects are specific enough, it's easy to find where they came from and store whatever info you want in them, which I assume is what they're doing here.

_Soulou• 12 years ago

github.com/juju/errgo from Canonical is really great. 100% compatible with the error interface, allow you to get the source and the explanation of an error.

vendakka• 12 years ago

Thanks for pointing to this. I remember reading about it a while back, but when I needed a stack trace wrapper I couldn't remember the name. I ended up adding a function to an error library I maintain (github.com/surullabs/fault).

When I find some time I'll pull out my code and start using either errgo or the SpaceMonkey version.

jtolds6• 12 years ago

worth pointing out github.com/spacemonkeygo/errors is 100% compatible, allows you to get a source and explanation, allows for error hierarchies, allows for arbitrary user-defined error details, etc.

AYBABTME• 12 years ago

I find it good practice to add context to errors that I pass up.

    func drinkOrangeJuice() error {
        if !enoughCoffee {
           return errors.New("requires enough coffee before orange juice")
        }
    }

    func eatBreakfast() error {
       err := drinkOrangeJuice()
   
       if err != nil {
          return fmt.Errorf("eating breakfast, %v")
       }
       return nil
    }

    func prepareForDay() {
       if err := eatBreakfast(); err != nil {
          log.Printf("Couldn't prepare for the day: %v", err)
       }
    }

    // Outputs: 
    // Couldn't prepare for the day: eating breakfast, requires enough 
    // coffee before orange juice.

Most of the time, errors shouldn't bubble up very high, so these strings remain readable.

Supplemented with simple error types:

    type ErrTooMany error
    
    func take(n int) error {
       if n > 9000 {
          return ErrTooMany(fmt.Errorf("%n is over 9000"))
       }
    }
    
    func main() {
       err := take(wayTooMany)
       switch err.(type) {
       case ErrTooMany:
          log.Printf("Was too greedy, %v", err)
       case nil:
       default:
          log.Printf("Dunno what happened: %v", err) 
       }
    }

That's all you'll need until very long. I'm pretty sure that's all you need at all. I've been back and forth, and the simplest always win.

buro9• 12 years ago

That's all you need if your application isn't surfacing errors to end users.

If you are surfacing errors to an end user then you'll need to ensure that the error message is localised.

If your Go code is merely an API to a front-end, then you'll want to return error codes rather than a textual string. The string your errors are producing are for developers to use and not end users, and to localise them you'll want to return error codes will enable the front end to select and choose the right message to display to a user depending on their locale.

If you're a HTTP API, immediately you'll think, "Ah, HTTP codes suffice, I'll return 'HTTP 404 no coffee' or 'HTTP 428 not enough coffee'" but you'll notice straight away that neither code fitted the scenario of "not enough coffee" well. Additionally, if you tell a client that HTTP 404 means "not enough coffee" you're stuck when you want to explain that they can't have jam on toast without actually having toast.

Yes HTTP codes help to tell HTTP clients what type of error happened, but fails to tell a client which message should be shown to help the end user resolve the problem... besides, our error string isn't localised (and shouldn't be, perhaps someone implemented a Welsh client and we do not know Welsh) so would the end user understand the message? We should presume not.

We're already at the point where the correct thing to do would be to return a detailed error code along the lines of Twilio's: http://www.twilio.com/docs/errors/reference or Oracle's: http://www.ora-code.com/

With an error code we provide the ability for a user client to make a simple lookup to localised and actionable messages about an error, perhaps with additional information and detail to a developer too.

Error codes help us avoid string matching too, the strings may change in the message but the actual error remains the same.

Now it becomes obvious that Go's errors are too primitive. They may be adequate for telling a developer something happened, and with logging they can tell a developer where it happened... but they are woeful at communicating with a user to help correct an error.

Error libs need both a message, and an error code (which isn't a HTTP code).

Stack traces are nice, but I find logging is better (and log.go can tell you which file and line the message was logged from) and HTTP codes are not adequate error codes, an application should have it's own table of codes.

PS: Interesting to note that Go does have error codes and a set of constants for a large number of errors http://golang.org/pkg/syscall/#pkg-constants but it hasn't made them first class by having godoc document errors a package has, encouraging everyone to create and use error codes, etc.

tagrun2• 12 years ago

Go TLS does not support hardware accelerated crypto.

Go uses AES-NI instructions on amd64 when available http://tip.golang.org/src/pkg/crypto/aes/asm_amd64.s

Go TLS does not have wide support for backwards compatibility with less secure and older protocol versions (a good thing, until you have no other option).

LibreSSL people beg to differ about this "feature".

Go TLS does not give much control over the certificate validation process, which makes it difficult or impossible to add additional checks and validations.

What exactly is this about?

Oh, and Go TLS is vulnerable to timing attacks.

A mitigation is in WIP https://codereview.appspot.com/94850043

jtolds6• 12 years ago

AES-NI doesn't work on ARM, which is what we have.

> Go TLS does not have wide support for backwards compatibility with less secure and older protocol versions (a good thing, until you have no other option).
LibreSSL people beg to differ about this "feature".

Agreed agreed! Don't use old protocols!

> Go TLS does not give much control over the certificate validation process, which makes it difficult or impossible to add additional checks and validations.
What exactly is this about?

OpenSSL allows you to specify your own validation callback, to add additional checks to the certificate validation step. The only certificate validation option Go's TLS gives you is either InsecureSkipVerify or not.

> Oh, and Go TLS is vulnerable to timing attacks.
A mitigation is in WIP https://codereview.appspot.com/94850043

awesome!

tagrun2• 12 years ago

AES-NI doesn't work on ARM, which is what we have.

I'm afraid this falls into scope of the low-priority issue 4299 then: https://code.google.com/p/go/issues/detail?id=4299

OpenSSL allows you to specify your own validation callback, to add additional checks to the certificate validation step. The only certificate validation option Go's TLS gives you is either InsecureSkipVerify or not.

I'm not sure about the necessity or the details of the particular use case, but if it is general enough, you can file an issue.

Open-Source Go Libraries

Related Stories