Real Go Projects: SmartTwitter and web.go

http://blog.golang.org/2010/10/real-go-projects-smarttwitter-and-webgo.html

by pufuwozu • 16 years ago

80 25 16 years ago

blog.golang.org

enneff2• 16 years ago

Gotta thank Hoisie for writing this. Nice shout out to HN. :-)

marketer• 16 years ago

Thanks for the post :)

ed• 16 years ago

I've been thoroughly impressed watching web.go come together. It's now the top Go project on Github in every category they track: http://github.com/languages/Go

Personally I'm of the opinion that language expressiveness matters more than real world performance, but only to a limit. Go is definitely carving out a nice niche for performant systems-related apps.

And the fact that Smart Twitter serves 90k concurrent users from a VPS without breaking a sweat is freaking amazing. I suspect this is the reason the service can remain free.

(edit: err, for the "watched" categories)

SkyMarshal• 16 years ago

I imagine at some point Google will make Go their fourth official language, when they perceive it is mature enough. I do hope they'll add a Go version of App Engine, as well.

syllogism• 16 years ago

It's Go on Android that I want...

stcredzero• 16 years ago

If they would retarget Go to a VM with advanced debugging, then it could subsume a lot of the advantages of languages like Python/Ruby/Smalltalk. Not only do those languages have fast turnaround times due to their dynamic nature, they also have "debugger on steroids" facilities like iPython, irb, and the Smalltalk image. If Go could replicate that facet as well, then dynamic environments wouldn't do anything significantly better than Go.

binomial• 16 years ago

I just wanted to point out that it's not 90k concurrent users, but 90k total users (for some unknown definition of "user"). But it does process a couple dozen tweets per second according to him, with low load. That's quite a far ways off from 90k concurrent users. That's not to say that Go wouldn't help in that case, it really depends on what the application is doing.

kuber• 16 years ago

awesome!

qhoxie• 16 years ago

Well done marketer. Glad to see this getting some more publicity.

swannodette• 16 years ago

What do people find interesting about Go? It's barely faster than the best dynamic languages - http://shootout.alioth.debian.org/u32/benchmark.php?test=all..., and it offers few conceptual innovations (in comparison to say Racket, Scala, Haskell, hell, even JavaScript).

So it seems too slow for systems programming and too "blind-folders on" about where CS is going. I'm honestly curious, what are people actually excited about?

The only scenario I could see people using it is in the situation where speed is not of the essence but memory consumption is. Perhaps I'm naive and this is a bigger category of software than I'm aware of.

Raphael_Amiard• 16 years ago

The thing is you are actually comparing mature dynamic languages implementation to a very new static language implementation. In the long term, if any of the previous events in the matter are of any importance, Go is very likely to climb that list without much effort. That's just the reality of compiler optimization : It's much more easier to do for a static language.

On a related note, you sound very "dynamic language" centric. Some people actually like static typing, and i won't relaunch the debate here but both sides have pretty good argument, especially when you throw type inference into the mix.

In the static camp, here is a few advantages i can see to go (i'm not actually an user of the language) :

- Easier to get your head around than Ocaml/Haskell - Arguably better than java (pros: actually has proper closures, and a sane system for polymorphism. cons: no generics) - Compiles to machine code rather than byte-code, so doesn't depend on a VM. Can be important for some tasks.

I actually think Go has quite a lot going on for it. It just doesn't have any 'wow' factor. But that's not necessarily a bad thing

acqq• 16 years ago

> so doesn't depend on a VM

But "dependence" on VM is nothing tragic if the GC is not in the game. If you'd imagine a VM without GC then what remains is just the potential for run-time JIT and optimization which can actually be a good thing! There's a long history of p-code interpreters which provided the more compact code, see http://en.wikipedia.org/wiki/UCSD_Pascal and http://en.wikipedia.org/wiki/Microsoft_P-Code At that time tracing and JIT would have been too heavy thing to do but today it could maybe be interesting to have something like that.

And as far as I know, Go doesn't "depend" on VM but does on GC, but D also generates the native code but doesn't have to use GC and I think that is an important advantage for such a kind of the language.

kaib2• 16 years ago

> The only scenario I could see people using it is in the situation where speed is not of the essence but memory consumption is.

In the type of code we do (distributed systems programming) speed is almost always directly dependent on memory consumption and layout. For the parts of our codebase that are purely CPU bound Go is about 2x slower than C, fast enough that we haven't yet felt the need to rewrite in C and use GCC. For the rest of the system we get a nice flexible statically typed language that has good support for concurrency. Previously a similar project would have used Erlang/Python/C on the server side, now we can just use a single language.

YMMV, we (tinkercad) do data and computation intensive problems which fits the original design goal of Go pretty well. Plus we find that C/C++ programmers can transition to the language quite easily, which is a plus given the domain of graphics and computational geometry is still filled with a lot of people who have mainly used C++ for performance reasons.

fauigerzigerk2• 16 years ago

Go code itself is much faster than any dynamic language. But many Go libraries (like regex or http) are very immature and horribly slow, as is the garbage collector, which is the reason for the bad averages in the alioth shootout. Also, there are many well known compiler optimizations that haven't been implemented yet (e.g. optimizing away bounds checking where possible). The stated goal of Go's creators is that Go should be around 20% slower than C.

It's probably a stretch but considering the speed difference between RAM and the harddisk, real life speed of an app has a very close inverse relationship to memory usage. Go uses very little memory, and I don't just mean the runtime itself. Contrary to Java it doesn't treat everything (other than primitive types) as reference types, which hugely reduces the number of pointers.

I think Go is very promising. It's going to be very fast, it's very memory efficient, and it makes massive parallelism incredibly simple. It greately reduces the pain and inflexibility of statically typed languages through type inference and after the fact interface conformance (or whatever the proper name for that feature is)

igouy3• 16 years ago

> But many Go libraries (like regex or http) are very immature and horribly slow, as is the garbage collector, which is the reason for the bad averages in the alioth shootout.

If that was true, taking regex-dna and pi-digits and binary-trees out of the data would shift the median significantly - (last time I checked) it doesn't.

However it is an explanation for that 13x slower data point.

singular• 16 years ago

It is very tastefully designed, most notably 'true' interfaces, e.g.:-

  type Foo interface {
  	Bar()
  	Baz() string
  }
  
  type Something struct {
  	// ...
  }
  
  func (s *Something) Bar() {
  	// ...
  }
  
  func (s *Something) Baz() string {
  	// ...
        return "a string"
  }
  
  func DoFoo(foo Foo) {
 	// ...
  }
  
  func main() {
        f := new(Something)
  
        DoFoo(f)
  }

You don't have to explicitly implement interfaces, if your data structure provide interface functionality then you can use it as that interface.

This is literally the only means of inheritance the language provides, it is extremely simple, yet really rather powerful; it permits generalisation without having to modify the data structures you want to generalise over.

There are many other instances of this kind of restrained taste throughout the language, which I think places it well into 'very nice language' territory, regardless of perf comparison.

In any case, perf can only improve; what is important, i.e. the design, is solid, so I expect we'll see perf improve nicely over time.

And that's not to mention that it is specifically designed to compile very fast. Something that becomes important when you are comparing it to the likes of C++ which can take, ahem a /little bit longer/ to compile...

edsrzf• 16 years ago

I think it's worth pointing out that underneath, most of the libraries those dynamic languages are using are written in C or even assembly in some cases.

I'm not saying that those benchmarks don't have value, but saying that Go's barely faster than the best dynamic languages doesn't ring true to me.

igouy3• 16 years ago

> It's barely faster than the best dynamic languages

Go on x86 doesn't seem to perform as well as Go on x64

http://shootout.alioth.debian.org/u64/benchmark.php?test=all...

jbarham2• 16 years ago

Yes, the AMD64 Go compiler (6g) is the most mature.

aditya• 16 years ago

Nice, this made it to TechCrunch via HN: http://news.ycombinator.com/item?id=1816945

Real Go Projects: SmartTwitter and web.go

Related Stories