Go's power is in emergent behavior

http://www.onebigfluke.com/2014/04/gos-power-is-in-emergent-behavior.html

by bslatkin • 12 years ago

222 133 12 years ago

pedrocr• 12 years ago

The second one just looks like a hack. You've defined a useless function to make an interface not implementable. It would look cleaner with a keyword in the interface definition.

jdiez17• 12 years ago

First of all: I agree. There are many things in Go that feel like a hack. For example, it is common (and idiomatic) to use a map with empty values when you want to use a set (unordered and unique values), because the map's keys are a set. I personally dislike this, and some aspects of Go are like this.

However this blog post is about how good core language design leads to unplanned patterns that are useful, like HandlerFunc and the private interface hack.

Iftheshoefits• 12 years ago

One person's "unplanned patterns that are useful" is another's "this is a potentially dangerous hack that could lead to all manner of headaches in any non-trivial program with even just a few hundred lines of code, let alone thousands or millions."

rectangletangle• 12 years ago

That's funny, that's how sets in early versions of CPython were implemented.

zaphar• 12 years ago

The function doesn't have to be useless. He uses a small and simple example to illustrate it. However the more common example of this is an interface with public piece your consumers can interact with and a private piece that only you can interact with.

seanmcdirmid3• 12 years ago

However the more common example of this is an interface with public piece your consumers can interact with and a private piece that only you can interact with.

Also known as...a class. Honestly, this is why classes exist: they are able to support both public and private interfaces, while interfaces and type classes are only able to support public interfaces. The closest we get to this essential aspect of OO programming in the H & M world is via existential types.

austinz2• 12 years ago

I'm not sure I understand why implicit interfaces are better than explicitly declared interfaces. (This is an honest question.) I do iOS stuff occasionally, and in that world one of the main purposes of interfaces (in Objective-C, protocols) is to explicitly declare that some entity adheres to a contract - it promises to implement some pre-agreed-upon functionality that other entities can use. What do implicit interfaces buy you? Better static type checking? Less typing?

/* Also, from the blog:

Nowhere in this code did we ever declare a relationship between MyType and Answer.

"Answer" isn't in the example. Was this something from an earlier draft? */

spydum• 12 years ago

It seemed to me clearly a autocorrect from Awesome to Answer.

austinz2• 12 years ago

Reading it again, that makes sense.

bslatkin3• 12 years ago

Thanks! Hooray for eyeballs. I fixed it

latch2• 12 years ago

With implicit interfaces you're not tied to a specific interface, but rather the method(s). If you have a type that implements Close() error, your type can now be used anywhere (include other people's code) which accepts an io.Closer.

Add a MarshalText() (text []byte, err error) function and it can now be used anywhere that expects a encoding.TextMarshaler.

Furthermore, your type can be used in someone's else code which has defined its own MarshalTextCloser interface.

I believe this is what the OP means by emergent. You wrote a type that has two methods, within your own package called "goku.Sayan", and it accidentally satisfies the interface "picard.MarshalTextCloser" which you'll never know about. "Sayan" was never built to satisfy _any_ interface. You just needed a Close and MarshalText function, but none the less, "Sayan" satisfies 3 distinct interface (which could be represented by an unlimited number of names)

I realise accidental sounds dangerous, but the only real risk is that the methods don't do what you think they should. But I don't see how that's different from explicit interfaces; in both cases the intent is only implied by the name. Overall, as the OP says, it's quite powerful when dealing with simple 1 or 2 function interfaces. I think most Go developers see this naturally emerge in their own code: they favor small interfaces specifically for this type of re-use.

pcwalton5• 12 years ago

I think the chances that people independently developing two separate libraries will accidentally give their methods exactly the same name, with the same types of their arguments in the same order and with the same return value, allowing this "emergent behavior" are very low in practice. Rather, Close() and MarshalText() are really people implementing explicit, well-known interfaces defined in the standard library. MarshalText() is a good example of this—I would have named it something like Serialize() if not for the well-known, named interface I explicitly wanted to conform to.

The primary benefit that structural interfaces give you, IMO, is less typing. Which has plenty of value, don't get me wrong; it's awesome that Go has successfully reduced the amount of syntactic overhead needed to implement interfaces. But I think the benefits of it shouldn't be overstated.

shasta• 12 years ago

But I don't see how that's different from explicit interfaces; in both cases the intent is only implied by the name.

Go's approach is theoretically more error prone. It relies on names matching in a way that can happen by coincidence. The important part of explicit interfaces here is referencing a common declaration site, that could e.g. give informal requirements in the form of comments. I doubt Go's approach is particularly error prone in practice, though.

zak_mc_kracken2• 12 years ago

Structural typing (sometimes referred to as "duck typing") has a few problems that make me unconvinced about the benefits.

First of all, I agree that the accidental argument is not very convincing (two developers writing a method Launch(), one that launches a football and another that launches nuclear missiles).

However, we lose two important things with implicit interfaces:

- Code readability. It becomes much harder for a human to interpret what types are actually implemented.

- More importantly, it severely limits the tooling available since the compiler has a lot less knowledge about the types you are dealing with. Automatic refactorings are all but impossible with structural typing.

Overall, I really don't see the harm in saying explicitly "My type is called Account and it implements Serializable and Entity" as opposed to me having to guess by reading all its methods and also having to remember which methods are necessary to be an Entity or a Serializable.

ori_b2• 12 years ago

Personally, I think it would be better if interfaces were explicit, but the list of interfaces on a type was open, allowing you to add interfaces over any type anywhere, and not just in the place where the type is defined.

    type foo = ...

and

    impl hashable(foo) = ...
    impl hashable(int) = ... // yep, adding to a builtin type

This is the way that Myrddin currently handles it.

pron6• 12 years ago

Personally I prefer Java's explicit interfaces to Go's implicit ones, but implicit interfaces have the advantage of allowing you define "partial interfaces" without changing the class hierarchy. For example, in Java, if an interface A declares methods foo() and bar(), and you think foo alone is useful, you can't define an interface B that declares only foo() and have all implementations of A satisfy it. The downside of implicit interfaces is that they preclude safe refactoring.

im3w1l• 12 years ago

Would be nice sometimes to be able to declare that interfaces are "extended by" or "implemented by".

e12e• 12 years ago

I suppose it might be "bad form", but couldn't you, if you had:

  public interface FooBar
  {
     int foo();
     int bar();
  }
  public Afob implements FooBar {...}

refactor FooBar to simply:

    public interface Foo { int foo() }
    public interface Bar { int bar() }
    public interface FooBar extends Foo, Bar {}

And keep FooBar the way it was, but now cast it to a Foo (or Bar) when you don't want a FooBar?

rtpg• 12 years ago

One advantage I can think of is that it lets you program in a duck-typing style without having to really think about it, but still get static checks.

midas0076• 12 years ago

It's essentially duck-typing iOS Objective-C protocols, but closer to how RubyMotion basically makes them optional.

skybrian2• 12 years ago

The advantage is that it allows interfaces to evolve without having to muck around with setting up shared dependencies. In a language like Java, if library A and library B want to share an interface, you have to create library C for the interface and modify A and B to depend on it, and if you want to add a method you have to modify all three at the same time. Furthermore, changing an interface isn't backward compatible so the correct way is to create a new interface every time you make a change.

In Go, they can both declare the same interface or even skip formally declaring it and just agree to implement the same methods. Furthermore, if one of them decides to implement a new method, the other can just copy the method signature and they stay compatible. This makes it a lot easier to coordinate API changes.

quotemstr• 12 years ago

Furthermore, changing an interface isn't backward compatible so the correct way is to create a new interface every time you make a change.

That's a feature, not a bug. Say we're using implicit interfaces and have library A that provides an interface Foo and an application B that uses A and passes a Foo to some callback system. Now A gets updated and adds a method to Foo, but existing B binaries still call A with an implementation of the old version of Foo, and things fall apart.

Java's model (and COM's and MS-RPC's) model here is better because it enforces good interface hygiene: incompatible types get incompatible names. Then you can choose whether to support the old name as well as the new name, but you're at least consciously making that choice. D's model worries me because it feels like it's easier to accidentally break things.

pcwalton5• 12 years ago

if you want to add a method you have to modify all three at the same time.

This is what defender methods in Java 8 are for.

kyrra• 12 years ago

One fun minor point about implicit packages. Since Go strictly forbids circular dependencies between modules, you can declare an interface in module A that module B inherits without B ever knowing about that interface. Sure you could do some refactoring and create a shared third module, but this is a neat trick.

aboodman3• 12 years ago

The general answer is that it puts swapability of implementation in the hands of the caller, not the callee.

Let's say I have a function A that depends on concrete type B, and that I only use a subset of B's functionality. Now say I decide, hey, I'd actually really like to be able to swap a different implementation of B at runtime.

With traditional interfaces, I would have to either:

a) define a subset of B's functionality as interface I and change B to implement I, or

b) define a subset of B's functionality as interface I and define a wrapper that implements I in terms of B.

If I choose (a) and many other callers end up doing the same thing to B, then B implements a mess of little micro-interfaces that it really should have no reason to care about. If I choose (b) then the implementation of A is way more complex than it seems like it ought to be.

===

One place this comes up all the time is testing: Object A relies on a subset of the interface exposed by service S, and service S is hard to create in test environment. With Go interfaces, A just defines an interface for the subset of S that it needs, and the test code for A implements that interface. S doesn't need to know anything about it.

eridius4• 12 years ago

You have to define a subset of B's functionality as interface I even with implicit interfaces. The extra step of formally declaring that B conforms to I is not an onerous burden. And in a properly-designed language, you don't even have to modify B, you can declare the conformance locally where you define I.

skywhopper• 12 years ago

So Go has duck-typing, and semantic capitalization, just like many other languages.

ryanobjc2• 12 years ago

for me, the built in goroutines is what moves the needle. The rest of the stuff is very nice, but built-in concurrency is pretty nice.

Yes I know about Erlang, no I won't touch Erlang again.

findjashua• 12 years ago

Elixir?

rdtsc4• 12 years ago

Yes I know about Erlang, no I won't touch Erlang again.

Curious, what makes you say that?

coolsunglasses2• 12 years ago

Haskell is statically compiled, has lighter weight green threads than Erlang, and a better type system than Go. You can even do CSP in Haskell.

Check it out :)

http://chimera.labs.oreilly.com/books/1230000000929

http://hackage.haskell.org/package/async

mseepgood2• 12 years ago

It's not duck-typing, it's structural typing.

coolsunglasses2• 12 years ago

There are nicer ways to do structural typing.

Here's just one example:

I have a post here of how you can partially define structure to be filled in by the consumer of the API: http://bitemyapp.com/posts/2014-04-11-aeson-and-user-created...

Can't do this without polymorphism and higher-kinded types.

biscarch• 12 years ago

I'm using this approach for a library that is currently in development. It works very well.

myg204• 12 years ago

Interesting to see surprise usages like that for such a small/simple language like Go; makes you wonder about the amounts of trickeries available in more complex languages.

philsnow• 12 years ago

the purpose is to not be able to use the interface outside the module, right? why wouldn't you just use an interface type with a lowercase name, which wouldn't be visible outside the module?

bslatkin3• 12 years ago

You want to be able to use the interface outside the module, but not allow anyone else to implement that interface.

Laremere• 12 years ago

The purpose was to not allow defining types which satisfy the interface outside the package. In this case they still want others to be able to use the values of the interface.

It's an uncommon thing to need to do, but this example shows that you can do it.

asuffield2• 12 years ago

Welcome to the world of languages with type systems that don't suck. It's going to be a fun ride, and if you think this is cool, you're going to be really excited by the other languages we've got in here.

It continually amazes me that people are willing to put up with Java.

midas0076• 12 years ago

The Oracle JVM is pretty versatile and has been beaten on in production on an extremely large scale, as much as M$ CLR.

Java as a language though was a primary response to over-correct and over-optimize for secure, correct, safe code based on the history of C's shortcomings. There's are many other lessons that have been learned since.

Java is really hard to beat apart from specialized formal methods verifiers (coq, CVC4), strongly-typed functional languages Haskell and similar derivatives for embedded industrial systems. If you're involved in safety critical systems, you should be using the simplest and easiest to understand formal methods tools as possible. If something's too esoteric, fewer people will be able to double-check the work.

For wider participation, it's a tradeoff to use one of the more popular languages that lack correctness aspects because of the absence of a learning curve.

yohanatan2• 12 years ago

JVM != Java so not sure why that point belongs in here.

zak_mc_kracken2• 12 years ago

If you are putting Go in the category of languages with a type system that doesn't suck, you probably need to study more languages.

Go has a few nice things going for it but its type system was clearly designed by people who stopped paying attention to type theory and compiler design in the late 90's.

yepguy• 12 years ago

Go's type system stays out of the way and adds some useful amount of safety. That's more than I can say of any mainstream language. No reason to tear it down just because it doesn't go further.

asuffield2• 12 years ago

I wouldn't say it's exactly brilliant - but it's just missing all the really cool stuff, rather than being buggy. It does what it's trying to do, and then stops. It's a bit like C in that regard (and unlike C++, Java, and C#, which tried to do something much bigger and didn't get it right).

It's also a bit like working with first-order propositional logic - well-founded and with its own unique simplicity, but you can't say everything with it and have alternatives.

So yes, I'd say that the type system of Go doesn't suck. It's not very good, and there are so many better things out there (rust, I'm looking at you), but it's a long way from being a disaster.

copergi• 12 years ago

Welcome to the world of languages with type systems that don't suck

Go is not in that world. It isn't even in the same star system.

shoo2• 12 years ago

For the second example, the non-implementable public interface, is there a way to allow implementations when testing?

naively, if i define a non-implementable public interface as described in the article inside `foo.go` and put some test code next to it in `foo_test.go` that attempts to implement the interface with a special test version, then as we might expect, that doesn't work. if i put test code inside `foo.go` itself, `go test` doesn't appear to collect the test.

(i have no experience with go, apologies if i am missing something very obvious)

helper• 12 years ago

Your example of `foo.go` and `foo_test.go` would actually work the way you want. In go, tests are in the same namespace as the code they are testing so they have access to private members.

shoo2• 12 years ago

Ah, thank you. Where i was going wrong was declaring `foo_test.go` to be a separate package from `foo.go`. When i write `package foo` at the head of both `foo.go` and `foo_test.go` this works fine.

mamcx• 12 years ago

As asked in http://programmers.stackexchange.com/questions/234900/go-lik...

I wonder if make sense to merge the julia multi-methods + GO interfaces.

MaysonL2• 12 years ago

The trouble with implicit interfaces is maintainability: suppose Awesome gets a func PerformAwesomeness() added to it: what then happens to the code that thinks myType is Awesome?

Not too bad when it's all in one source file, but get a few megabytes of code spread through a few thousand source files, and you've got a formula for chaos and heartburn.

thrownaway2424• 12 years ago

Not sure what you are getting at. If you add something to the interface of Awesome, then any call site which passes a thing not having a PerformAwesomeness() method will fail to compile.

microtonal6• 12 years ago

Unless you rely on introspection, which happens a lot when your language does not support generics.

MaysonL2• 12 years ago

Which means that every time you change an interface you have to compile everything (given that you don't explicitly call out the Awesome interface anywhere, so you don't know whether any particular source file will require the Awesome interface without compiling it).

mjburgess• 12 years ago

I dont know, these feel to me like very simple features ("functions are objects", "functions take a final-type") .. nothing really "wow". The wow-factor is, I guess, that Go can actually do it.. which seems to me a little masochistic. ("Wow my underpowered lang. isnt so bad!").

eridius4• 12 years ago

The first example is not emergent behavior. And it has nothing at all to do with the implicit nature of interfaces. It is in fact a pretty straightforward consequence of the fact that functions are first-class types. Because they're types, you can newtype them, and you can implement interfaces on the newtype. This should work in any language that has interfaces and first-class functions.

For reference, here's the exact same thing in Rust:

    // Define trait Foo, this is the equivalent of an interface
    pub trait Foo {
        fn foo(&self, x: int) -> int;
    }

    // Define a newtype called FooFunc
    pub struct FooFunc(fn(x:int) -> int);

    // Explicitly implement the trait on FooFunc
    impl Foo for FooFunc {
        fn foo(&self, x: int) -> int {
            let &FooFunc(f) = self; // unwrap the newtype
            f(x)
        }
    }

    // This is a method that uses a Foo, analogous to http.Handle()
    pub fn doFoo<T: Foo>(f: T) {
        println!("foo(42) = {}", f.foo(42));
    }

    // Here's our function that we want to wrap
    fn MyFoo(x: int) -> int {
        x+1
    }

    fn main() {
        // Call the function, just like http.Handle using http.HandlerFunc
        doFoo(FooFunc(MyFoo));
    }

And just like Go, this ends up being free at runtime.

bslatkin3• 12 years ago

But you had to explicitly define the relationship: "impl Foo for FooFunc"

dignati• 12 years ago

Which many would prefer because it's implicit documentation. Never knowing what interfaces a go type implements sometimes bugged me. Functions, defined in interfaces often look different in style and paradigm so sometimes they stand out and made me wonder why they would implement such functionallity in this way, not knowing that the definition comes from a hidden interface. It also takes away the ability to quickly look up all die functionality because I would have to know the functions names and what interface they are defined in. Imagine a interface changes and you would have to find all the types you have to change.

md224• 12 years ago

Yeah, it looks like their solution is just creating a function object whose interface-required method calls the object itself (or at least that's my OOP-centric interpretation). You could do something like that in Javascript (minus the interface, obviously):

    var f = function(){ console.log('called'); };
    f.callMe = function(){ this(); };

I wonder if this is a useful pattern in JS. Can't recall ever seeing it in practice.

NateDad10• 12 years ago

It's useful in go, because it means you can trivially turn a bare function into a type that fulfills an interface, which makes your function more flexible - it can easily take either a type with a method or just a method.

twic3• 12 years ago

That's it: a tiny adaptor that makes a bare function satisfy an interface. How is that possible?

No, why is it necessary?

In Java 8:

  public interface Handler {
      void serveHTTP(ResponseWriter w, Request r);
  }
  
  public class MyApplication {
      public static void myHandler(ResponseWriter w, Request r) {
          w.write("An even easier webserver?");
      }
  
      public static void main(String... args) {
          Http.handle("/", MyApplication::myHandler);
      }
  }

No adaptor needed!

Or even:

  public class MyApplication {
      public static void main(String... args) {
          Http.handle("/", (w, r) -> w.write("An even easier webserver?"));
      }
  }

And please bear in mind that this is Java, the slowest-moving and least sophisticated of contemporary languages! Why does Go require so much more boilerplate than Java?

dvirsky4• 12 years ago

It's just a design decision of the http library, that its router by default accepts an interface and not a function - so internally you need this little adapter to make the function comply with that interface. I also loved this little trick when I noticed it.

But if the HTTP library would directly accept a function, you could then just pass a function without that trick. Go doesn't mandate you use interfaces for callbacks (like Java did until 1.8), you can also define an API that accepts functions directly, provided they have a certain signature.

Also, the standard library hides this little trick from you, and just exposes the HandleFunc method, that lets you register plain functions as handlers. as in this example: http://golang.org/pkg/net/http/#ListenAndServe

tomp• 12 years ago

If I understand the Java 8 example correctly, the Http.handle accepts an interface as well (the Handler interface with a single method). Java compiler is smart enough to automatically convert functions into objects satisfying interfaces with exactly one method, while Go compiler isn't.

NateDad10• 12 years ago

What's nice about the go implementation is that it can trivially accept a function or a type with the right function.

nnutter• 12 years ago

Do you get to use Java 8 in production?

Edit: To clarify I mean if you had a Java X app deployed how hard is it to upgrade to Java 8? One thing I like, maybe naively, about Go is static binaries.

lmm3• 12 years ago

It's pretty trivial. 1. Upgrade JVM on server (i.e. change 1 line of puppet config) 2. Build with Java 8 (i.e. change target option in your build file) 3. Deploy.

I mean sure, you have to install the new JVM, but if you don't have a system in place for making changes to your servers then you've got bigger problems than which language you're using. And honestly I think the JVM with its classpath approach solves the library problem better than most platforms; upgrading the JVM binary occasionally is no great hardship, and aside from that everything is just jars, dynamic but not getting in the way of each other unless you want them to. What happens when security holes are found in Go libraries, do you have to recompile anything that depends on them?

twic3• 12 years ago

Personally, no, not yet. My current employer has been dragging its feet over Java upgrades, so we're only just transitioning onto Java 7.

However, most of the work involved in doing that is paving the road so that version upgrades are faster. I would anticipate that moving from 7 to 8 will be much faster than moving from 6 to 7.

NateDad10• 12 years ago

One of the nicest things about Go interfaces is that they encourage SMALL interfaces. In explicit interface implementations, you end up throwing everything including the kitchen sink in the interface, because you might need it somewhere the type is used.

In Go, interfaces are small, and focused more on the function rather than the type. The function defines the methods it needs. So you can have a hundred small interfaces with one or two methods each, and a small handful of types that implement some percentage of those interfaces. Small interfaces makes your functions a lot more flexible. If all you need is a Read() method and a Close() method, instead of a huge number of potentially type-specific methods, then your method that takes a ReadCloser can be used by a lot more types.

You can do that in languages with explicitly implemented interfaces, but it means you need to write down that your type implements these 40 interfaces... and that's a hassle that isn't really needed or useful.

lnanek2• 12 years ago

Honestly, I feel the version where he defines the struct as usual is clearer, simpler, and more maintainable. Other programmers will understand it immediately without thought. Allowing weird fancy ways like he raves about to save a couple lines of code is really a drawback in enterprise software development.

Go's power is in emergent behavior

Related Stories