"100% of our production system is now running Go"

https://groups.google.com/d/topic/golang-nuts/hC0aJ8IASi4/discussion

by enneff • 14 years ago

297 182 14 years ago

groups.google.com

Creator of SASS and Haml now using Go for whole production system.

When asked about surprising things about Go, its creators have said that they expected to recruit people from c++ communities, instead it seems to be more ruby/python people.

cageface4• 14 years ago

In hindsight it doesn't seem so surprising. Go really isn't a good alternative to C++ for the kinds of things that really need the low-level control and direct memory access C++ provides. And if you don't really need those things you probably should have moved on to Java or Ruby/Python/etc already.

And since a lot of Java shops are fantastically risk averse I'm not at all surprised that the same kind of people that were willing to take a risk on Python or Ruby are now into Go. It offers a lot of the expressiveness of either of the former with much better performance, concurrency as a fundamental, and the extra safety of static type checking.

thebigshane• 14 years ago

Straight from Rob Pike -- http://commandcenter.blogspot.com/2012/06/less-is-exponentia...

  Python and Ruby programmers come to Go because they don't 
  have to surrender much expressiveness, but gain 
  performance and get to play with concurrency.

  C++ programmers don't come to Go because they have fought 
  hard to gain exquisite control of their programming 
  domain, and don't want to surrender any of it. To them, 
  software isn't just about getting the job done, it's 
  about doing it a certain way.

  The issue, then, is that Go's success would contradict 
  their world view.

  And we should have realized that from the beginning. 
  People who are excited about C++11's new features are not 
  going to care about a language that has so much less.  
  Even if, in the end, it offers so much more.

oofabz• 14 years ago

I write Go and I have been satisfied with the low-level control and direct memory access. The language has pointers and you can cast between ints and pointers, so you can do anything you can do in C.

If you're writing realtime code, you can avoid triggering the garbage collector simply by not allocating memory on the heap. It is harder to get anything done this way, but writing realtime code in any language is difficult.

StavrosK2• 14 years ago

> In hindsight it doesn't seem so surprising.

Most things don't seem surprising in hindsight. That's why you don't judge surprises by it.

If C++ programmers did flock to Go, would you then say that that was surprising? If not, you've just fallen victim to hindsight bias (because a model that predicts everything is useless).

Lewisham• 14 years ago

I think a lot of it is that people sort of did want some static type checking, but were not willing to give up on things like garbage collection or some of the other niceties that the dynamic languages provided.

As someone who has been clamoring for a statically-checked Python-esque language, Go looks awesome.

jonathansizz2• 14 years ago

I think that Mozilla's Rust is aiming squarely at C++ developers, whereas Go is more of a Java replacement.

And it's interesting to see Python and Ruby being squeezed lately, from various directions; Clojure, Go, other languages. Since Python went mainstream and started being picked up for serious projects, it naturally came under the spotlight a lot more than it had previously, and I think many people have started to see some of its warts. The transition from version 2 -> 3 wasn't exactly a smooth one, either. Furthermore, although Python is a pretty good all-round language, it doesn't really excel at anything in particular, unlike for example Perl which continues to maintain a strong niche in the areas of text processing and system administration, despite its popularity having fallen away in some of the other areas it was formerly strong in.

In my case, although I have recently moved from Perl to Racket for larger and/or more complex work, I still use Perl a lot for small, simple scripts (which constitute most of my programming). A couple of the Java guys I know switched from Python to Clojure for scripting tasks (I could never convince them to like Perl..), and have started to take an interest in languages like Scala and now Go for more substantial projects.

For many years now, Python has been the fresh, fun challenger going up against older (and perhaps clunkier) languages, whereas now things seem to be shifting and Python may soon find its role has reversed and it has become the old, crufty language.

pcwalton5• 14 years ago

"I think that Mozilla's Rust is aiming squarely at C++ developers, whereas Go is more of a Java replacement."

I basically agree with this as a Rust developer (although I'm not sure about Go being in Java's niche; I think of it more as in node.js's niche -- highly scalable web apps). Early on, I think both Rust and Go were thought to be targeting the same segment, but it turned out that we really weren't. Personally, I'm completely fine with that; different language designs are appropriate for different roles. The huge amount of work that we've done to make non-allocating and manually-memory-managed code easy, predictable, and type-safe is totally overkill for Go's niche, for example, while for games and web browsers this sort of thing is essential.

batista6• 14 years ago

>Furthermore, although Python is a pretty good all-round language, it doesn't really excel at anything in particular, unlike for example Perl which continues to maintain a strong niche in the areas of text processing and system administration, despite its popularity having fallen away in some of the other areas it was formerly strong in.

Well, it does excel in scientific computing. Biologists, astronomers and such use it a lot, and it has lots of advanced math/stats/physics etc libs, with fast C implementations underneath.

raphinou• 14 years ago

I'm very interested to know more about your use of racket in production. Could you share your experience?

drbawb• 14 years ago

If that's the case, I hope that Haml [and Sass] get properly ported to Go!

I'm aware of gohaml; but last I checked it hasn't been updated to work with Go1 and friends. Namely it's using `gomake` and the packages aren't organized for use with `go build`; those issues alone have stopped me from considering it.

I went for a version of mustache.go instead, and then bolted Lambda support onto it.

siculars3• 14 years ago

It's really annoying to have to log in to read something on google+, or groups for that matter. I just won't do it.

spicyj2• 14 years ago

This happens only if you have some Google login cookies already -- if you delete all your Google cookies then you won't be asked to log in.

siculars3• 14 years ago

Um. What? Did you just tell me to fuck myself?

lambda• 14 years ago

Rather than suggesting that people delete their cookies, you could suggest checking the article in private browsing mode (or whatever their browser of choice calls the feature).

EdiX• 14 years ago

I can't believe this bug still exists after two years.

Figs• 14 years ago

You have to login? It doesn't do that for me, and I'm not logged into any Google products...

jlarocco• 14 years ago

Same here.

tedunangst2• 14 years ago

Google has invented a new login state between "logged in" and "not logged in" called "not logged in enough". It's kind of like being logged in, except nothing works and you get asked to login again. Also, you can't logout.

batgaijin• 14 years ago

No it's the usenet remnants, not google+. But yeah, google made usenet worse too.

rhizome• 14 years ago

Delete your cookie or sign out. It's just a Google Groups thing for you to refresh your session and I dunno, parallel login or something (they may never have completed the dejanews cutover). Unknown users don't get a login prompt.

ck22• 14 years ago

To continue shopping on m.macys.com, please enable cookies in your settings.

Really? in 2012?

kyrra• 14 years ago

It's a shopping website. It's sorta hard to do cart management without cookies.

dkroy3• 14 years ago

They are a good fallback, I think he is talking about using things such as Local storage, session storage, or global storage. Since the site is their mobile site I see no reason why they aren't using the new technology available to us since most mobile devices would be able to use it. DOM storage seems to be a better fit for shopping carts than cookies anyways.

ck22• 14 years ago

You cannot even see the front page without cookies.

I suppose it's better than 1990s websites that would go into an endless loop but what about people who are just browsing?

Evbn7• 14 years ago

Just put a session id in the URLs.

grandpoobah• 14 years ago

Yes, in 2012 there are still people who disable cookies. I know, it's weird.

jemfinch• 14 years ago

Yeah, seriously. It's amazing that people are still afraid of cookies in 2012.

batista6• 14 years ago

Yes. It's 2012, cookies are non tin-foil material, get on with the program and enable them.

Or, if you insist, at least do not expect to use any web app / sales page, anything without them.

dbecker2• 14 years ago

This is a really awkward sentence: "100% of our production system is now running Go as 95% of our application."

I'm curious what the 5% is that they didn't convert to go.

jmaygarden• 14 years ago

Read the whole paragraph. It's C.

"100% of our production system is now running Go as 95% of our application (C being the other bits). Not side bits, but our critical path."

dbecker2• 14 years ago

I meant "what functionality is the remaining 5%." Not "what language is it."

Basically, once you are 95% Go, what parts are hard to convert to Go to make it 100%

zeeg2• 14 years ago

I'd be really curious to know what their "800 million requests a month" means. When we actually added up numbers a while back, we were doing around 250 million requests to our Python machines/day, and we only had 200 servers (which included dbs/etc), that was around 3-3.5 billion pageviews a month IIRC. We now have 300 machines and do about 5 billion pageviews, but that completely ignores the huge amounts of requests to our realtime stream servers and our API (which get shit on every "pageview" at least once).

jacques_chester• 14 years ago

It's really hard to compare different systems without knowing what kind of complexity each pageview engenders in terms of backend processing.

At one end there's flat HTML files, where you could serve 800 million pages/month from maybe a single particularly beefy server; at the other fully-dressed SOAs and CEPs and your uncle's pet dog rover performing multivariate regressions, meaning you need 9,000 servers.

zeeg2• 14 years ago

I'm hoping they'll provide some more insight into what the breakdown of tasks are between the machines, and if that 800 million number is actually correct.

It seems extremely high for what my absolutely-no-idea-what-these-sites-could-possible-need-that-many-servers-for thought process can fathom.

moe• 14 years ago

200 servers for 800mio requests seems a little nuts.

Two of my customers are doing 500mio and 400mio respectively on ~20 each, on Rails (i.e. as slow as it gets).

gizzlon• 14 years ago

Do you count cache hits as a request?

shawndrost• 14 years ago

I interviewed with moovweb a while back... they basically transcode non-mobile-optimized sites into mobile-optimized sites on the fly by transforming html/css/etc.

heretohelp5• 14 years ago

The results...aren't great.

joejohnson• 14 years ago

"100% of our production system is now running Go as 95% of our application (C being the other bits)"

So... 95% of their production system is running Go.

ww520• 14 years ago

I guess it's playing word semantic. 100% of their MACHINES are running GO in SOME capacity, while other languages play some roles in other capacity.

enneff• 14 years ago

I interpreted it as meaning that 100% of the code in the critical path is Go, while their greater application also includes some C code.

f2f• 14 years ago

the C code is also being run under Go's control via Cgo.

Evbn7• 14 years ago

My production system is written in 100% bash and some other code run under Bash's control. :-)

jessemc3• 14 years ago

Like all those people that say they are running 100% python/php/ruby etc.

quarterto• 14 years ago

They've done studies. 65% of the time, it works every time.

pkulak• 14 years ago

How's the garbage collector these days for Go? Is it close to the performance of the Java GC?

Evbn7• 14 years ago

It's not as sophisticated, but Go structs are smaller than similar Java objects, so that helps beat the tradeoff

microtonal• 14 years ago

I like Go, but this is total nonsense. It does help that you can embed values in a struct, rather than using pointers. But marking everything that looks like a pointer is not efficient. Not having generations does not help.

Go is still very young, a better GC will come, just give it time. In the meanwhile, let's not pretend that the current sub-par generator is as performant as that of modern JVMs.

batista6• 14 years ago

No, still the same old one. Improvements are supposedly coming though (though, they were coming 2 years ago also).

luriel8• 14 years ago

There have been many improvements to the GC over the last two years, and many more are coming in Go 1.1.

GC is an area where always there is room for improvement, no matter how much you work at it.

jebblue• 14 years ago

Edit: I just posted 3 minutes ago and I'm already in gray text. Wow.

We search with Google, run our phones with Google, run Google's browser, now they want us to write all our code in Google's language. Why Google? What's wrong with Java? Other than the fact that Oracle owns it, I miss Sun. Scott McNealy tried to make the technology world better. Scott, Gosling. Google...why should everything we do all day and night be about Google?

http://en.wikipedia.org/wiki/Go_(programming_language)

>> The syntax of Go is broadly similar to that of C

C, seriously? C??!? Why? Why would I want to go back to ... C?

>> Of features found in C++ or Java, Go does not include type inheritance, generic programming, assertions, method overloading, or pointer arithmetic.

And that is a good thing or a bad thing or what?

>> Go allows a programmer to write functions that can operate on inputs of arbitrary type, provided that the type implements the functions defined by a given interface.

Huh? Oh I can pass interface references, ok.

>> A Go interface is best described as a set of methods, each identified by a name and signature.

That sounds like Microsoft's crappy COM from the 1990's.

The code sample on that page shows there are no braces in the if evaluation statements. A wise man, Peter Van Der Linden formerly of Sun's Compiler Group and author of Deep C Secrets wrote about using more braces if needed as explicitness is always better. I agree with that.

At this point, I think I'd rather program in Groovy or even go to JavaScript if I had to, than use Go.

Oh...the language name sux.

Tip, buy Oracle and fix Java or by Java rights from them and fix it, then programming in a language owned by Google when you own so much of our lives now wouldn't be so bad.

PS How does Guido Van Rossom feel about Go?

Evbn7• 14 years ago

Do you hate C syntax or love C syntax? Make up your mind.

batista6• 14 years ago

>We search with Google, run our phones with Google, run Google's browser, now they want us to write all our code in Google's language.

Which is also open source and has a community behind it.

(Plus, you can search with Bing, run an iPhone, and run Safari/Firefox/IE)

>Why Google? What's wrong with Java?

You mean besides the stagnation (no closures? no type inference? no functional goodness? It's 2012 already), the enterprisey vibe and the lawsuits?

>>>The syntax of Go is broadly similar to that of C C, seriously? C??!? Why? Why would I want to go back to ... C?

They don't ask you to go back to C. The SYNTAX is similar. Which is also true for Java.

That said, from reading the rant, you don't seem like the type that ever was in C in the first place, so it wont be "going back" for you.

>>> Of features found in C++ or Java, Go does not include type inheritance, generic programming, assertions, method overloading, or pointer arithmetic.

And that is a good thing or a bad thing or what?

That is what it is. Languages have different features, and it's not a spec war.

Java doesn't have tons of features C++ has (e.g operator overloading, pointer arithmetic, hey, even closures, as of the latest standard). But you don't seem to be the type that wants to go to C++, right? So why the double standard?

>>> A Go interface is best described as a set of methods, each identified by a name and signature.

That sounds like Microsoft's crappy COM from the 1990's.

Yes, I can see how you are qualified to make this kind of comparisons: "Interface of methods, with name and signature. Ah, just like COM!"

>The code sample on that page shows there are no braces in the if evaluation statements. A wise man, Peter Van Der Linden formerly of Sun's Compiler Group and author of Deep C Secrets wrote about using more braces if needed as explicitness is always better. I agree with that.

Good for you. You missed the "if needed" part.

>Oh...the language name sux.

And Rob Pike pissed on your cornflakes.

>PS How does Guido Van Rossom feel about Go?

Why should we care?

brianobush• 14 years ago

>> C, seriously? C??!?

For those, like myself, that like C and do systems programming. Really never understood why people dislike or fear C so much. It is the foundation for so many systems and libraries.

jessemc3• 14 years ago

The main problem people have with C syntax is that it's 'old' and 'old' things are bad because new things are better. We're an industry built on fashions. Largely the outcome of the massive success programmers can have without any formal education and so just get by knowing the "best" way to do things instead of understanding why.

vorg• 14 years ago

> At this point, I think I'd rather program in Groovy or even go to JavaScript if I had to, than use Go. Oh...the language name sux.

Which language name were you talking about?

billwashere2• 14 years ago

>>What's wrong with Java? Everything is wrong with Java yet the JVM is an amazing feat. Java makes doing the simplest things ten times more convoluted. If you like the JVM then http://code.google.com/p/jgo/

>> The code sample on that page shows there are no braces in the if evaluation statements.

This is just Go's style but look you can still use braces. http://play.golang.org/p/_j7r6GB-ZN

bitcartel• 14 years ago

@enneff Do you know if they are using the default gc compiler, or gccgo which is supposed to have better optimisation?

luriel8• 14 years ago

I'm fairly sure they use the main gc compilers.

gccgo does better at some things, but the default compilers do quite well already.

bad_user2• 14 years ago

I want to try out Go, but the problems exposed by the garbage collector on 32bit systems scared me away.

Is that still the case with the latest version 1.0.2 ?

Also, is there any chance for a precise generational garbage collector to get implemented for Go? As far as I know, pointers make the implementation of precise garbage collectors difficult.

jessemc3• 14 years ago

There is code to make the garbage collector more precise currently in review. A few packages from the stdlib were changed to reduce the chance that package level variables might pin used sections of memory. Most people have no problems with the garbage collector on 32bit systems.

akavel2• 14 years ago

AFAIK, yes it is still the case (although somewhat slightly improved), but precise collector is officially expected (and being worked on) for version 1.1 . The latter was explicitly said on a Q&A session on Google I/O conference, the video is available somewhere on teh interwebs.

edit: I think it's this one: http://youtu.be/sln-gJaURzk ("Meet the Go team"), but maybe some other from: http://blog.golang.org/2012/07/go-videos-from-google-io-2012... (sorry, don't have time to watch them all again now to make sure ;)

bad_user2• 14 years ago

That's good to know, thanks.

jussij2• 14 years ago

> I want to try out Go, but the problems exposed by the garbage collector on 32bit systems scared me away.

If you run a 64 bit server then these problems just goes away.

And from all accounts these issues will also be fixed for 32 bit systems in then next Go release or two.

jff2• 14 years ago

Here's a dime, go buy yourself an AMD64.

zedzedzed4• 14 years ago

There was this dart. It somewhat failed, so what they did was, screw up a good systems language so that it sarted looking like dart..

tocomment3• 14 years ago

How do you run golang as a server? Can anyone point me to a basic tutorial? Is it still running under something like Apache is golang IS the server somehow? If so Wouldn't each http request have to fire up the compiled executable? Isn't that slow?

billwashere2• 14 years ago

>>How do you run golang as a server?

Using the net/http package that comes with Go.

>>Can anyone point me to a basic tutorial?

To learn Go - http://tour.golang.org/

To learn about http http://golang.org/doc/effective_go.html (A list of available functions are found here http://golang.org/pkg/net/http/)

Gorilla is a great web framework that builds upon the net/http package - http://gorilla-web.appspot.com/

>>Is it still running under something like Apache is golang IS the server somehow? If so Wouldn't each http request have to fire up the compiled executable?

What you are describing is CGI (http://en.wikipedia.org/wiki/Common_Gateway_Interface) whilst it could be possible but don't for real severs its really old and slow.

eckyptang6• 14 years ago

The compiled output will contain your network server. you run the process and it listens on a socket. It is pretty straightforward. Using HTTP as an example, each request will run in a goroutine which is somewhere between a thread and a coroutine.

It is extremely fast.

The main risk is process failure which is very rare but can be mitigated via putting a front end balancer such as nginx or apache mod_proxy. Process recovery and spawning can be managed by another tool such as supervisord.

However, you can run a process happily from init if you desire.

tocomment3• 14 years ago

Thanks! That's exactly what I was trying to figure out. You're a gem.

So is there anywhere I can see an example of code that runs a server like this?

tocomment3• 14 years ago

I was always taught never to write my own web server since something like Apache has been tested on millions of sites, and you have security experts designing it.

Does writing your own server in Go kind of violate this idea? Are you opening yourself up to re-inventing a web server and repeating decades of mistakes something like Apache has already figured out and fixed?

afc• 14 years ago

Whoah, 500 servers to handle an average of ~300 queries per second? Even if you generously assume they have traffic patterns that sometimes go all the way up to sustained say 5K queries per second... 500 servers? Isn't that like ... terribly wasteful?

"100% of our production system is now running Go"

Related Stories