Go: Support for Generic Methods

https://github.com/golang/go/issues/77273

by f311a • 2 months ago

311 274 2 months ago

github.com

h1fra• 2 months ago

slowly implementing all the things they said we didn't need

TheChaplain• 2 months ago

It's not a bad thing to realize that one can be wrong and then strive for change.

a-french-anon3• 2 months ago

Maybe, but personally I've become quite tired of programming languages "organically grown" as opposed to properly designed the first time. After a good decade of C then C++, I found ANSI CL (despite being a massive compromise and unfinished) much more coherent and complete than both.

tux3• 2 months ago

I don't think anyone admitted any wrong or had any big change in philosophy. It's always a good thing to learn something along the way. But the current message seems to be that this was the plan all along, and it just took some time to design properly.

Of course adding generics is not something that every language needs to do. Scripting languages like Ruby don't really need this style of generics. It doesn't fit the design of the language, and it's not even clear what that would look like in Ruby.

But static typing with generics does solve a recurring problem, and we've seen some real convergence towards type hints and type systems even in staunchly dynamic scripting languages. Modern Javascript is now mostly Typescript, and they've successfully retrofitted a very advanced type system in the last place I would have expected 20 years ago.

maccard6• 2 months ago

There’s a fine line between being willing to change your mind and getting the basics wrong. Go has repeatedly gotten the basics wrong.

layer82• 2 months ago

It’s still annoying ~20 years after Java did the same mistake of not including generics, which was already clear to many people with C++ experience back then.

ivanjermakov3• 2 months ago

Worst thing a programming language can do is introduce core semantics changes after 1.0. See Python 2 -> 3 and Zig's *gates.

9rx18• 2 months ago

Of course, if you go back and watch the original Go announcement it said that it would need generics once they figured out how to do it. And when the first version of generics landed it was said that generic methods would be added later, once they figured out how to do it. So that isn't applicable here. The need was always recognized.

zarzavat2• 2 months ago

If Go had just taken an off-the-shelf implementation of generics in 2009 then they could have spent the last 16 years deliberating over something useful, rather than attempting to reinvent programming language theory.

The impression I have always gotten from Go's designers is that they are rather arrogant and averse to the idea of using other people's work. They want to develop everything from first principles, but by so doing end up with poor reinventions of well-studied concepts.

CamouflagedKiwi• 2 months ago

They didn't say they never wanted to do generics, but that they did want to take their time and do them right.

Debatable how much they have been "right", although this gets them somewhat closer. And I think they have not been "wrong" in the ways they wanted to avoid (they referenced some issues with Java generics as prior art, although I forget the details).

tines10• 2 months ago

From another commenter here:

The post quotes the Go FAQ as saying, "we do not anticipate that Go will ever add generic methods".

someone_197• 2 months ago

I agree that they were clearly not in a hurry. I disagree that they are doing everything right. I am interested to see how they will fix the 'million dollars mistake'.

thayne5• 2 months ago

Depends you who "they" is. If you mean the go development team, then yes, they said they wanted to "take their time and do them right"¹. But there are many "gophers" who did say that there was no need for generics, and that it shouldn't be added to the language.

¹ I would argue that it is really, really hard to add generics to a language after it has already matured, and still "do it right" than to add it in the beginning. At least if you care about backwards compatibility. Backwards compatibility adds a lot of constraints to your generics system that will almost certainly lead to a sub-optimal design. And you will be stuck with a standard library, and a lot of existing ecosystem code that would benefit from generics, but don't because generics didn't exist when they were written. This is a lesson I wish go had learned from Java's generics.

Cthulhu_2• 2 months ago

Where did "they" say "we" didn't need generics? That sounds like a bad faith / misinterpretation / straw man; as someone else pointed out, they postponed generics until they figured out the use cases and whatnot.

Remember that the generics implementations in other languages (like Java) take up half the spec + implementation - that's not something that Go wanted.

tines10• 2 months ago

From another commenter here:

The post quotes the Go FAQ as saying, "we do not anticipate that Go will ever add generic methods".

fhn2• 2 months ago

complaining about things given to you for free

doodpants2• 2 months ago

I frankly don't buy into this trope that a lack of monetary cost should shield something from criticism. Anything created by humans for other humans, especially tools meant for getting work done, should certainly be open to evaluation/judgement/critcism, regardless of whether the creator chooses to charge for it.

And it's not like Golang is some freshman student's hobby project; it was created by one of the world's largest tech companies, by people with a strong pedigree in programming language design.

zarzavat2• 2 months ago

Watching Go's development is like reliving the development of Java (which also didn't have generics at first), but over decades instead of years. Cannot wait for Go to implement an error handling system in the 2030s.

20k2• 2 months ago

Its very funny watching certain segments of the programming industry rediscovering incredibly basic programming principles, after railing against them for so long. The AI people are starting to try and create formal specs to force the AI to generate an exact output, which is absolutely hilarious to me

Dynamically typed/untyped languages finding that strict and visible typing is actually good is another

pjmlp12• 2 months ago

Thankfully we already have Java, so using Go is really for the scenarios where it cannot be avoided.

jimbokun4• 2 months ago

Because they refuse to do something until they figure out how to do it well.

I respect that.

tapirl2• 2 months ago

Go's generics design is the most clunky one among popular languages.

MrBuddyCasino6• 2 months ago

What did they say about proper enums.

someone_197• 2 months ago

You already have iota. Type safety is not needed by design:

Go intentionally has a weak type system... Go in general encourages programming by writing code rather than programming by writing types...

https://github.com/golang/go/issues/29649#issuecomment-45482...

kardianos2• 2 months ago

This is great. Will be useful for data access methods!

As for the detractors, from the first generics proposal this was called out as a "not now", not never. There were questions of implementation. They aren't a super large team, and they try to do things incrementally and do them well.

dude250711• 2 months ago

Gophers are usually quite fast, perhaps an elderly turtle would be a better mascot?

rob74• 2 months ago

In day-to-day usage, the (fast) compilation speed matters much more than the (slow) implementation of new features.

cookiengineer• 2 months ago

Gophers are usually quite fast, perhaps an elderly turtle would be a better mascot?

The slow turtle wins the race against the overly eager rabbit... so I'm okay with that

tczMUFlmoNk• 2 months ago

As for the detractors, from the first generics proposal this was called out as a "not now", not never.

What? The post quotes the Go FAQ as saying, "we do not anticipate that Go will ever add generic methods". There is also some similar discussion of the original generics proposal, with language like "then it's much less clear why we need methods at all". (I'm omitting some context, but I don't feel that it changes the meaning.) Those feel much closer to "never" than "not now".)

The post is also subtitled "A change of view".

kardianos2• 2 months ago

I could be mis-remembering it. I didn't look up and src it.

stouset10• 2 months ago

No, you’re clearly wrong; golang was always going to add support for generic functions.

Everyone also wanted and accepted the need for generics. It was always something they wanted to add to the language. Rob Pike never said that that kind of abstraction isn’t what golang is for. It was always just a matter of getting the design right.

Go has always been a systems language. It was one when we thought it was going to fit nicely for low-level, high performance use-cases. Given that the GC, runtime overhead, lack of control over memory layout, and other issues made it a poor fit for what we historically thought were systems language tasks, it’s still a systems language because we’ve grown to understand that the term “systems program” has always meant network middleware that shuttles around JSON and transforms it.

Dependency management too. Modules were something that nobody argued were unnecessary. None of the language developers ever claimed that “you should always build against HEAD, and if upstream breaks you, that’s a coordination problem to be solved socially”. The community didn’t need to independently invent godep, then glide, then govendor, then dep, before the core team finally shipped modules. That was just enthusiastic parallel exploration of a problem space that everyone agreed was a problem.

GOPATH was always understood to be an awkward temporary scaffold that everyone tolerated while the real solution was being designed. The single-workspace model was never defended as philosophically correct or a deliberate feature of the language. When modules arrived, everyone was simply relieved that this obvious stopgap was finally replaced.

The core team always intended to add builtins for min/max. Nobody ever told you to just write `if a > b { return a }; return b` yourself because it was “only two lines.” The fact that every Go codebase in existence had its own copy of this logic, typically buried in a file called util.go, was not evidence of anything being missing from the language.

Range was always a stopgap before iterators could be implemented. Nobody ever argued that iterators were needlessly complicated and went against the spirit of the language. The slices and maps packages provided important missing features that everyone using the language wanted.

Everyone agrees that errors were anemic from the outset. errors.Is/errors.As are nice additions but everything was Just Fine™ before they were added.

Speaking of errors, having two lines of error-handling boilerplate for every line of code is good, and right, and perfect. It’s not verbose; it’s “explicit”. But when that gets changed to be less verbose, we will all agree that it was always a pain and made reading code unnecessarily more difficult and that everyone always expected this to be fixed some day.

I personally can’t wait to see what next development will never have been “against the Go philosophy” and definitely not something that gophers argued was perfect the way it was any time misguided malcontents and rabble-rousers wrongly tried to suggest the language wasn’t perfect the way it was.

nasretdinov3• 2 months ago

Lack of generic methods was really surprising to me when I was first trying to use generics in Go. Nice to see it being actually implemented

ncruces3• 2 months ago

To be replaced by the surprise when you figure out these methods don't implement interfaces.

Still, in this case, half the feature is better than none at all, IMO.

nasretdinov3• 2 months ago

Generic interfaces are going to be implemented later too if I'm reading correctly. So no real surprises there :). I guess the only surprise yet is that generic interfaces aren't supported, so generic methods physically can't satisfy any interface

reactordev• 2 months ago

This resolves a big gap in generics for most people coming from other languages to go so I completely approve this direction. Not saying use it everywhere but if you must use it, it’s better to have it on the struct than call a module level generic func.

binary132• 2 months ago

Chasing a perceived gap between language features and user expectations has been and continues to be the greatest error in the leadership of Go.

pizzafeelsright• 2 months ago

The nagging imperative requires a stronger response than the capitulation of identity.

MrBuddyCasino6• 2 months ago

They should have blocked generics forever and instead have a proper enum. Unpopular opinion, I know.

throwpikerob• 2 months ago

A sad day for Go, the pHDs have won, simplicity has died.

valyala• 2 months ago

Agreed 100%. https://itnext.io/go-evolves-in-the-wrong-direction-7dfda8a1...

Cthulhu_2• 2 months ago

It only died if you actually applied it in your own codebase - as with any feature, using it is optional.

9rx18• 2 months ago

Code is written to be read, and typically for other people to read. So, in practice, it is only optional if everyone else agrees to not use it.

sdevonoes• 2 months ago

It certainly makes readability of Go code harder.

EdwardDiego2• 2 months ago

If generics replace code generation or hand coding a concrete implementation for specific types or... operating against interface{} and then switching based on runtime type, then I strongly disagree that it makes reading Go code harder, it makes it, IMO, easier.

But that's the thing, it's just IMO.

samber• 2 months ago

OMG. I'm going to recode some of my libraries.

mackross• 2 months ago

What a happy surprise today! The amount of times I’ve had to do weird janky package APIs so the API was still reasonable is more than I can count.

xena• 2 months ago

This will finally let me make the monad library I've been dreaming of for years. Be afraid.

nasretdinov3• 2 months ago

We already have monads at home (return X, err)

bonesss• 2 months ago

A monad library in go can really on have one name… …

oh_fiddlesticks• 2 months ago

Go nad or go home

zephen• 2 months ago

Mo-go?

throwaway894345• 2 months ago

Mongo

whaleofatw20222• 2 months ago

Strife

hennilu• 2 months ago

GoMad

dnnddidiej2• 2 months ago

Can we have Exception monads? Asking for friend.

AdieuToLogic6• 2 months ago

Can we have Exception monads? Asking for friend.

This is nonsensical. Monads define a strict set of behaviors formalized as "monad laws"[0].

Perhaps what you want is a container which adheres to monad laws capable of abstracting exceptions. Two exemplars of same are Haskell's Data.Either[1] and Scala's Either[2].

0 - https://wiki.haskell.org/Monad_laws

1 - https://hackage-content.haskell.org/package/base-4.22.0.0/do...

2 - https://www.scala-lang.org/api/3.8.3/scala/util/Either.html

assbuttbuttass2• 2 months ago

It's (sadly) still not possible to express monads with this change, since generic methods can't implement interfaces. You'd probably want something like:

    type Monad[T any] interface {
        Bind[U any](func(T) Monad[U])
    }

However this requires the Bind method to be generic, which still isn't allowed in an interface

_jackdk_2• 2 months ago

I am not very familiar with Go and especially not its generics support. Can you implement the "join" version instead of the "bind" version, where you turn a T[T[a]] into a T[a]?

9rx18• 2 months ago

Funnily enough, Go's generics were designed by the same guy who introduced monads to computer science. Everything comes fill circle.

AdieuToLogic6• 2 months ago

Funnily enough, Go's generics were designed by the same guy who introduced monads to computer science.

No contributor to Go is responsible for "introducing monads to computer science", as the Monad concept is a member of (or defined by if you prefer) Category Theory[0].

0 - https://en.wikipedia.org/wiki/Category_theory

thayne5• 2 months ago

Go doesn't support such generic interface methods because we don't know how to implement (calls of) them, or at least we don't know how to implement them efficiently.

I don't really understand this argument. I read the discussion linked to[1], and yeah, monomorphization approaches (whether at compile time, link time, or runtime with JIT) are obviously going to be difficult or impossible, but the reason against using runtime reflection is mostly that it's slow. But that runtime reflection is exactly how you would work around it today.

For the Identity example, could the interface be compiled to be basically equivalent to:

Identity(any) any

and then at the callsite add a cast of the return type to T?

I suppose primative non-pointer types add a bit of a wrinkle but even if it generic methods was restricted to pointer types, that's better than nothing. And the number of those types is relatively small, so when the implementation is compiled it could just instantiate method implementations for all the primative types, if they apply, and then maybe remove them if they aren't needed at link time.

Of course it's also possible there is some detail I've missed.

[1] https://go.googlesource.com/proposal/+/refs/heads/master/des...

whaleofatw20222• 2 months ago

Of course it's also possible there is some detail I've missed.

Can't speak too deeply for Go specifically, but I do know on .NET one of the big reasons generic methods where T is a structure gets monomorphized per type, is so that stack size is adjusted and potentially even arg passing (i.e. large struct) as far as the caller/callee.

Merovius5• 2 months ago

but the reason against using runtime reflection is mostly that it's slow.

More specifically, it is that it would introduce surprising performance cliffs – code becoming surprisingly slow due to seemingly unrelated changes.

Though BTQH I think an even more important argument is that you would need to have effectively two generics implementations, one working at runtime and one working at compile time. That's a lot of complexity, with surprising failure modes if these two are not bug-compatible.

But that runtime reflection is exactly how you would work around it today.

I think the overwhelming majority of people will "work around it" by just not trying to use generic methods.

thayne5• 2 months ago

you would need to have effectively two generics implementations, one working at runtime and one working at compile time

My understanding is that go already has a hybrid system works at compiletime and sometimes at runtime.

Boxxed• 2 months ago

BTQH?

MrBuddyCasino6• 2 months ago

Like Java‘s generic erasure? Primitive types aren’t supported at all, have to used boxed ones. Its slightly annoying but not too much. You can fall back to arrays to have performant primitive containers.

swisniewski• 2 months ago

The real problem is that Go produces interface implementations dynamically.

The determination wether type T implements interface I is made at runtime. So is generation of the necessary vtables to produce the interface implementation.

So you can do things like this in package a:

  type S struct {
      //...
  }

  func (s \* S) Foo() {
      //..
  }

and something like this in package b:

  type Foo interface {
      Foo()
  }

  func DoSomethingWithAFoo(f Foo) {
  }

and something like this in package c:

  func Stuff(obj any) {
      theFoo, _ := obj.(b.Foo)
      theFoo.Foo()
  }

And then do:

  var s a.S
  c.Stuff(s)

And everything works.

For generic functions, go uses a strategy similar to C++ templates: when you call a generic function the compiler statically produces a concrete specialization of the generic function based on the inferred types for generic parameters.

That is, if you do:

  func Bar[T any](x T) {
    //...
  }

And you do:

  var x int
  var y string
  var z float64

  Bar(x)
  Bar(y)
  Bar(z)

The compiler statically generates 3 versions of Bar, one that takes an int, one that takes a string, and another that takes a float64.

These two things don't work well together. If I have a variable typed as `any`, and I want to cast that to an interface, I need to dynamically determine 2 things:

1. The shape of the interface's vtable. The go runtime does this by iterating over the runtime metadata for the interface type.

2. For each named method in the interface's vtable, the address of the concrete function to stick in that vtable slot. This is done by accessing the reflection metadata for the implementing type. It verfies the method with name X for type T matches the required signature for the method with name X for interface I, then sticks that method pointer into the appropriate vtable slot.

The problem, however, is what happens when the method with name X is generic. There may, or may not, be an actual concrete method for the set of type parameters. It's possible that statically type T does implement interface I (via generic methods) but that dynamically it doesn't because the particular generic instantiation needed for the particular interface was never made statically.

Prior to go 1.27, this was never an issue, because methods could not declare their own type parameters. They could reference the generic parameters of the receiver, but once the receiver type was known, there was only ever one concrete method X for that receiver.

Once you allow methods to have their own generic type parameters, the compiler can introduced several different concrete implementations for a method X.

This is ok, when you do somethnig like:

  var x SomethingWithGenericMethods
  x.Foo(1)
  x.Foo("hello")
  x.Foo(1.2)

Because the compiler knows statically from the Foo call sites which concrete methods it needs to generate.

But, when you introduce a dynamic cast:

  var x SomethingWithGenericMethods
  var i SomeInterface

  i = x.(any).(SomeInterface)
  i.Foo(1)
  i.Foo("Hello")
  i.Foo(1.2)

It's entirely possible that the necessary Foo implementations don't actually exist in the binary.

So, go 1.27 introduces generic methods, but it gets around this problem by saying:

1. Interface types can't define generic methods

2. Generic methods can't be used to implement interfaces

Thus, it allows adding generic methods without introducing the issues that crop up with dynamic interface implementations.

pjmlp12• 2 months ago

Go becoming a proper 21st century language, is like pulling teeth.

It is Apple's school of design, think different, ah, actually, there are reasons why the fence is in the middle of nowhere.

Then the design ends up half way there versus being done properly from the beginning.

piekvorst5• 2 months ago

“Done properly from the beginning” means explaining why a particular feature is either included or not. In this sense, Go is done properly from the beginning. It would be wrong to add every popular feature uncritically.

pjmlp12• 2 months ago

"They are likely the two most difficult parts of any design for parametric polymorphism. In retrospect, we were biased too much by experience with C++ without concepts and Java generics. We would have been well-served to spend more time with CLU and C++ concepts earlier."

Yeah very critically.

zerotolerance• 2 months ago

Your comment could have been, "I've never agreed with the design philosophy behind Go." I've always appreciated the apparent Go design philosophy and feel like it most matches my lessons learned from 20 years in software. Feature minimalism is a feature for languages targeting organizations with thousands of programmers. If by 21st century language, you mean one that has become unrecognizable through multiple generations of fashionable feature and ecosystem thrash then I'm all for Go not becoming a 21st century language. Language should be boring if the target environment is large teams with varied skill sets. Plain speak. Low jumpy behavior. No cream, no syntactic sugar. Get the job done.

p0w3n3d• 2 months ago

what's wrong with `#define`? <laughing hysterically>

nah I'm kidding

Seriously, what's wrong with `#define`?

ethin3• 2 months ago

Watch as this gets rejected for nonsensical reasons just like the issue about adding uint128, which also has been open for an extremely long time and they keep moving the goalposts as to reasons why it can't be done...

piekvorst5• 2 months ago

It’s accepted.

ethin3• 2 months ago

Now there's a surprise. I've generally been very disillusioned with Go after they absolutely stonewalled everybody on uint128 (and continue to do so) for absolutely no reason (and ignoring that it would make many things in the language easier to express).

msaher662• 2 months ago

Since they can't implement interfaces, Generic methods are just syntax sugar for generic functions. I'm surprised they actually accepted this proposal for sugar.

tapirl2• 2 months ago

Yes, the sugar is just to make chain calls with parameter types possible. The sugar reflects the limitation of the basic of Go generics design. Now they would make the language even more complex for such a small need. In facts, there are more problems in Go generics need to be solved earlier than this: https://go101.org/generics/888-the-status-quo-of-go-custom-g...

msaher662• 2 months ago

Thanks for sharing. I had no idea Go's generics had this many limitations.

klik99• 2 months ago

I remember lack of generics being pitched as a feature of Go initially, not a lack. The original design goal was simplicity. I don’t use Go, so have no opinion on this, just interesting that it’s going in this direction.

matthewmc32• 2 months ago

Generics being omitted wasn't pitched as a feature that I ever saw. The discussion was mostly around "hic sunt dracones" with generics. The Go authors were very aware from the get-go that generics are an incredibly difficult thing to implement well, and the commitment was always to not do it until they were really ready to tackle it. The bizarre thing to me is that when they did tackle it, they left this massive implementation gap, and then what little they did implement didn't seem useful enough that anything meaningful was added to the standard library. So all this talk about generics for years was a big nothing-burger. That is, I hope, until 1.27 when this gets released. Flags is one in particular I think is in desperate need of proper generic methods.

apatheticonion• 2 months ago

I wrote Go professionally for years. I don't know how many hours I've spent debugging and reviewing nil pointer bugs and race conditions. Generics were sorely needed but the initial implementation was lacking.

I moved to Rust professionally 4 years ago and haven't looked back. Mutex<T> Option<T> Result<T, Err> are all phenomenal.

I've written everything from web backends, frontends (hurry up wasm, seriously), to Node.js and Python extensions.

Web backends use under 1mb of memory and can support hundreds of thousands of concurrent users on a $2/m VPS. Frontends can be beautifully multithreaded. Native extensions can dance between OS threads and multi-threaded runtimes.

When I review code I focus only on the logic, not sidetracked by reasoning about race conditions or anything. Great when you review the work of less experienced contributors.

The ultra strict compiler is extremely helpful with LLMs. You bounce back and forth until it compiles and, if it compiles, it's usually correct.

It's at the point where I can't really see a use case for another language - and yet, no one uses it! It's madness!

root-parent• 2 months ago

I have nothing to add to your comment, but it matches so eerily my exact experience, that I just wonder if your alias is a second account of mine :-)

davidatbu• 2 months ago

By frontend, do you mean wasm in the browser? I'd have expected multithreading there, especially in wasm, to be extremely unergonomic.

Maybe you mean to refer to concurrency?

axod• 2 months ago

And so the cycle will continue. Always a shame when languages cave like this and add extra unnecessary complexity and error prone hard to parse syntax.

It'll be interesting to see the next language that comes along rejecting bloat in favor of simplicity, and then we can all start again.

Go: Support for Generic Methods

Related Stories