Golang Diaries II

https://www.tbray.org/ongoing/When/201x/2013/07/15/Golang-Diaries-2

by espadrine • 13 years ago

96 46 13 years ago

espadrine• 13 years ago

Of course, anyone can make a Map function for any particular type combination, but Tim Bray has a point. The generic, functional map function cannot be written, and never will.

There is an issue of elegance. If generics were to be added, lists and maps would need to have similar syntax, because they feel and behave like generics. However, they cannot. Their syntax is… not generic enough.

Oddly enough, elegance through consistency was never reached when designing built-in functions like make(): those can be applied on any type, but user-defined functions don't have this freedom. Essentially, there is a Java-like divide between built-in types (which, for instance, you cannot create methods on) and user types. Their rules of elegance don't apply to the built-ins, which is quite odd.

I doubt that this impedes on programming efficiency compared to C. However, some of the duplicated code out there cannot be factored into a library, and map is an example of that.

pdeuchler2• 13 years ago

I took a couple minutes to whip up a quick map function, to be honest I didn't quite believe you. This may be my naivete, but is there any way it's not sufficiently generic for types? I realize it's not suitable for maps, but it satisfies the OP's request for a map that can convert arrays of type Foo to type Bar

http://play.golang.org/p/jxMFq5UYs1

burntsushi• 13 years ago

Here is something that is less onerous on the caller, but probably slower. [1] Here's the source. [2]

[1] https://github.com/BurntSushi/ty#examples [2] - https://github.com/BurntSushi/ty/blob/master/fun/list.go#L17

drewolson• 13 years ago

I think the point is, if you write map like this you might as well be using a dynamically typed language in the first place because you're deriving no benefit from the type checker. In fact, you're just using casting to completely circumvent it.

pcwalton2• 13 years ago

"I doubt that this impedes on programming efficiency compared to C." will not hold true if you're using reflection.

dragonwriter• 13 years ago

The generic, functional map function cannot be written, and never will.

https://github.com/droundy/gotgo/blob/master/example/slice.g...

comex4• 13 years ago

Isn’t it pathetic that so many allegedly-mainstream languages don’t have HTTP libraries that are this natural and idiomatic?

That's after 13 SLoC to make a single HTTP request - I like Go, but that seems almost like a parody. It's not that different from other languages, and the error handling makes it look longer, but for something described as "natural and idiomatic" I would expect it to be one function call plus one failure test, not six and three...

cmccabe6• 13 years ago

Experiencing a networking error is different than receiving Status 200 (or any other error status), which is different than getting a media-type different than "application/json". Personally, I don't see what would be gained by combining them all together in a library, except confusion. A lot of libraries do get this wrong and make it difficult to disentangle what is going on.

If you feel like golang needs one method to do all the above, feel free to write one and post it to the mailing list... I doubt that it does, but feel free.

comex4• 13 years ago

Depends on the use case. I'd say that for many, many cases, all errors should lead to the same result in the application, but with an error-specific informative message. Even if you want to have special handling for errors such as 4xx that might mean something is permanently wrong, getting a 500 is the same as experiencing a network error (except in a client side app where the latter might mean you're not connected to the Internet and should try again, but then you should use yet another API to determine when to connect), and getting a 200 with the wrong content-type has an excellent chance of meaning the same thing as a network error, in the form of a captive portal. To this extent, APIs with many separate error paths that encourage the user to write their own nonstandard error messages are harmful, because they will probably do worse explaining things than with a common method of triage.

Of course there are other cases where the difference matters. It just... doesn't seem very "natural and idiomatic" to me, that's all.

throwit1979• 13 years ago

When people complain about comprehensive error-checking, it makes my blood boil.

Checking and handling all possible error conditions is a GOOD thing.

comex4• 13 years ago

Only if you have a meaningfully different thing to do for each type of error, see my response to the sibling comment.

ballard• 13 years ago

Yep. Sometimes crashing isn't an option. There are places in network & systems code that have seemingly complicated but darn important many permutations of state that already exist. Sometimes the differentiation between error conditions requires a combination of conditions (i.e., talking to an existing legacy system.)

As for single-threaded map, there are at least two necessary behaviors that handle errors very differently:

1. batch all errors/exceptions/etc and return them as an array 2. return on first error

azth• 13 years ago

Checking and handling all possible error conditions is a GOOD thing.

I agree. But Go seems to have taken the most verbose way to achieve that. Pattern matching should have been the way to go if they did not want to use exceptions. Then again, Go does not have functional constructs, and you wouldn't be able to get far with it. Though it still prevents you from ignoring errors, unlike what exists in the language.

cgag• 13 years ago

Between that and the bit about the good type inference I thought the post was sarcastic until I saw the gripes.

cmccabe6• 13 years ago

With regard to the functional-style "map" function, is it really that much less typing than a simple for loop?

  output := make(output_type, len(input))
  for idx, e := range(input) { output[idx] = convert(e) }

compared to:

  output := map(input, func(in input_type) output_type {
      return convert(in)
    }
  )

There are cases where functional programming can be a lot terser (for better and worse), but I don't think this is one of them. My guess is that the original poster is just adjusting to different ways of doing things between Golang and Ruby-- that's natural.

My biggest complaint about Go so far is that you have to use an external library to get an ordered tree structure. Maps suffice most of the time, but sometimes you need ordering.

[edit: fixed map example]

comex4• 13 years ago

The second example should be

    output := map(convert, input)

After all, map generally handles fetching the element from the array.

pcwalton2• 13 years ago

There are cases where functional programming can be a lot terser (for better and worse), but I don't think this is one of them.

That's only true in Go because Go doesn't have type inference for closures. In ML for example this would be:

    let output = map (fn i -> convert i) input

or, more succinctly:

    let output = map convert input

cmccabe6• 13 years ago

Yes, that's true. I just wanted to address the specific suggestion that the original author made.

Languages make a whole set of related design choices and typically none of them makes sense without considering the others. (Sometimes none of them makes sense AFTER considering the others, but that's another story.)

jacques_chester5• 13 years ago

Loops impose, and guarantee, ordered traversal of a collection. Maps do not. This means that maps can sped up by dividing up the collection into smaller parts and mapping each of them in parallel.

For a language that has concurrency as a central design motivation, it's an unfortunate omission.

As we say in the database world: think in sets, not in arrays.

cmccabe6• 13 years ago

It's an imperative language, not a declarative one. The issue is not ordering (traversal over maps in golang is not ordered), but side effects.

SQL is great. Not all languages have to be SQL.

icebraining2• 13 years ago

It's an imperative language with side-effects, unordered mapping would break expectations and produce buggy code.

You can still achieve those speed ups, you just need to be explicit about the division (e.g., by creating a channel, launching goroutines and then collecting the results).

https://sites.google.com/site/gopatterns/concurrency/paralle...

NateDad• 13 years ago

I started writing this exact post before I saw yours. It boggles my mind that programmers complain about having to write four lines of code (if properly formatted, which yours is not... but I'm guessing the people that complain about not having a map function also complain about gofmt :)

zeckalpha• 13 years ago

gofmt is the one feature that I think every language designer should steal from go.

jlujan2• 13 years ago

Screw map, let me pass a comparator function into sort for interfaces.

cmccabe6• 13 years ago

That's not necessary. You can just create a type that embeds the type you're sorting, and define the Less function however you like. It's very efficient since you're just wrapping the entire slice object, not every element in the slice.

  package main
  import "sort"
  type RevIntSlice struct {
      sort.IntSlice
  }
  func (arr RevIntSlice) Less(i, j int) bool {
      return arr.IntSlice.Less(j, i)
  }
  func main() {
    myList := RevIntSlice { sort.IntSlice {1, 2 , 3 } }
    sort.Sort(myList)
    for _, elem := range(myList.IntSlice) {
        println(elem)
    }
  }

Note that if all you want to do is reverse the sort order, sort.Reverse is a better way than what I wrote here. This is just an example of the kinds of things you can do.

chongli2• 13 years ago

That's not necessary. You can just create a type that embeds the type you're sorting

But that's more work (and more boilerplate) than simply passing an anonymous comparator to the sort function.

jlujan2• 13 years ago

Yes. I am aware how to sort and can only read your comment as as informative sarcasm as to the ease.

buro93• 13 years ago

Isn’t it pathetic that so many allegedly-mainstream languages don’t have HTTP libraries that are this natural and idiomatic?

But at every point where Tim then returns a failure we lose insight as to why something failed.

My preferred idiom for handling all errors when it comes to web requests or responses is to return both the error (as it has the detailed message of what happened) and the http status code that matches the error (as it has translated the message into a code that describes the type of error).

This bit would change:

    if !strings.HasPrefix(mediaType, "application/json") {
        // bogus media type, deal
        return failure
    }

to:

    if !strings.HasPrefix(mediaType, "application/json") {
        // bogus media type, deal
        return HttpErr{Err: errors.New(""), Status: http.StatusUnsupportedMediaType}
    }

Whatever receives that knows there has been an error, and has the code available to do tell the developer or end user something meaningful without relying on the error message.

I'm sure others are also creating their own http error as well, so whilst Go does http really well it seems in practise that there are still a few places where it could benefit from a little extra to help standardise the code we write.

All of those Http codes are here: http://golang.org/pkg/net/http/#pkg-constants

Regarding the map stuff, the only other place the lack of generics has stood out has been within slices of a type of thing.

The scenario is: I have a slice of identifiers in an order important to me, for each of these concurrently fetch the record being identified and create a slice of those records ordered as per the initial slice.

We've ended up using this type of thing:

    // Envelope for making sorted concurrent requests for Comment
    type CommentRequest struct {
        Comment Comment
        Err     error
        Seq     int
    }

    type CommentRequestBySeq []CommentRequest

    func (v CommentRequestBySeq) Len() int           { return len(v) }
    func (v CommentRequestBySeq) Swap(i, j int)      { v[i], v[j] = v[j], v[i] }
    func (v CommentRequestBySeq) Less(i, j int) bool { return v[i].Seq < v[j].Seq }

    // Much later in the code...

    // Make a request for each id
    req := make(chan CommentRequest)
    defer close(req)

    for seq, id := range ids {
        go HandleCommentRequest(id, seq, req)
    }

    resps := []CommentRequest{}
    for i := 0; i < len(ids); i++ {
        resp := <-req
        if resp.Err != nil {
            return resp.Err
        }
        resps = append(resps, resp)
    }

    // Sort them
    sort.Sort(CommentRequestBySeq(resps))

    // Extract the values
    ems := []Comment{}
    for _, resp := range resps {
        ems = append(ems, resp.Item)
    }

In that example a Comment is a comment made on a forum web page and we're fetching lots concurrently, but needing to return them in the order that they will be displayed.

The problem stems from that envelope, and the need to implement that sort function for every type that we ever want to concurrently fetch and sort.

It's the only place where I think, if we just had generics this could be a lot better.

jalfresi2• 13 years ago

  if !strings.HasPrefix(mediaType, "application/json") {
        // bogus media type, deal
        return HttpErr{Err: errors.New(""), Status:    http.StatusUnsupportedMediaType}
    }

Sorry, but having dealt with a HTTP library in PHP that did something similar, I would have to hunt you down and kill you. The HTTP response status code from the original request was NOT 415 Unsupported Media Type. Do you have any idea how misleading this error is to the developer? The error you SHOULD be returning is an Application error code "Unexpected Media Type", NOT a HTTP error code. It's the application that has a problem with the returned media type.

buro93• 13 years ago

I would have to hunt you down and kill you

Attempts at off-the-cuff witty death threats don't tend to work too well online. I truly hope you were jesting.

dscrd• 13 years ago

Just recently, I needed a concurrent map for my project. I looked for a library and started using it. Fast forward a bit and I realized the map was completely untyped. Fast forward a slightly longer bit and I realized (perhaps wrongly? Enlighten me) that there was no way to implement a concurrent map library that wasn't untyped.

We need generics, or perhaps it's more goy to have the same features by some more clever way. But still. We need those features and bad.

I'll be looking at Rust with some more interest meanwhile.

ExpiredLink• 13 years ago

     if ... { return failure }

three(!) times in a row and then he writes:

Isn’t it pathetic that so many allegedly-mainstream languages don’t have HTTP libraries that are this natural and idiomatic?

Isn’t it pathetic that a wannabe mainstream language doesn’t provide exception handling?

jameskilton• 13 years ago

Exceptions are horrible if you're trying to make a fast, dependable, reliable, low-level, concurrent modern day programming language, as Go is. Exceptions are also horrible in that they never improve the readability and understandability of any code they exist in. The Go team, some of whom actually designed C back in the day, are well familiar with Exceptions, why they are bad, and explicitly chose to not implement them.

So no, using the word "pathetic" implies a gross oversight. This was a very deliberate decision, and a good one at that.

Laremere• 13 years ago

You're excluding the comments which would be actual code to handle the very different events which all result in failure. Also, arguable only the first case would be something you'd want to throw an exception in a language which has them anyways. (Not getting a json response from a general http library is far from exceptional)

illumen• 13 years ago

tbray works for Google as "Developer Advocate". Which means he markets to developers.

This is a marketing fluff piece from Google.

"I still haven't written 1000 lines of Go." Why do we care what this guy says then? Because lots of Googlers vote it up.

I suggest you flag this spam :)

Golang Diaries II

Related Stories