Google's ‘Gopher Team’

http://www.wired.com/wiredenterprise/2013/07/gopher/

by suchabag • 13 years ago

98 38 13 years ago

No they don't. One guy started a major rewrite of a broken piece of infrastructure, and "the Go team [which he is part of] now regularly volunteers to help other teams with small projects", so as to learn more about how Go can replace and augment existing systems.

This headline is so far beyond journalism as to be a joke. It's just making shit up for the sake of having a "story".

I can imagine it now. They heard "gopher", thought of another animal, then thought about how awesome some of the coders on the Go team are (look at the blinking lights!), and BAM! invented an elite squadron of coders who fix whatever problem you have based on a combination of the navy Seals and the A-team.

eliben2• 13 years ago

Yep. I wouldn't surprised to see this article come from some other outlet; I did not think it's already time to add Wired to the "yellow media" bucket (though last week's article about LLVM being the last thread holding Google and Apple together just about did it)

girvo• 13 years ago

Wired really depends on the author, IMO. I still enjoy Threat Level

atonse• 13 years ago

Being married to a Journalist and also hanging out with her journalist friends has made me realize that, VERY OFTEN, the person writing the article isn't the one that writes the headline. And they sometimes hate the headline as much as the readers do, but that's how it goes in that industry, I suppose.

will118• 13 years ago

Discussion from 2 days ago: https://news.ycombinator.com/item?id=6110398

stuartcoope• 13 years ago

These slides from Brad Fitzpatrick himself on the subject were a lot more informative:

http://talks.golang.org/2013/oscon-dl.slide

inaudible• 13 years ago

Thanks. This form of internet journalism really irritates me, trying desperately to create a compelling narrative out of fairly dry technical parts, only to end abruptly to some upbeat testimonial - it reads more like a poorly written press release.

edw519• 13 years ago

If code doesn’t receive constant love it turns to shit,

I would prefer "If bad code doesn't receive constant love it turns to shit"

I am aware of thousands of examples of heavily used commercial software that hasn't been touched in 5, 10, 15, even 20 years because it just works and always has. Properly designed and written scalable software can last indefinitely with little or no modification, even through geometric changes.

But I like OP's quote. I think it should become part of every code reviewer's checklist:

  Will this turn to shit without constant love?
  No:  Pass.
  Yes: Then fix it now.

kawera• 13 years ago

Properly designed and written scalable software can last indefinitely with little or no modification, even through geometric changes.

Daniel Bricklin wrote an interesting article[1] related to long lasting software (PDF)

[1] http://changethis.com/manifesto/download/6.200YearSoftware

JulianMorrison2• 13 years ago

The analogy between bridge building and software is very old and very wrong.

Software can indeed be designed like a bridge, proven and then built for one unchanging use case and set in stone. And unless you are writing for NASA, soon enough that code will be junked as an obsolete millstone because use cases change. No engineer could design a bridge that might have to be rebuilt into an aircraft runway, or a 20-floor apartment complex, at a whim.

eliben2• 13 years ago

I would argue that bad code is already shit. There's no doubt that good code can be turned to shit too if it's being modified all the time with bug fixes, etc, without paying the required technical debts.

Not sure what your example of untouched code represents. The OP says that what makes code turn bad is accumulated local changes and bug fixes without thinking about the big picture. Of course if no one ever touches the code, its quality doesn't change.

xfax• 13 years ago

Properly designed and written scalable software can last indefinitely with little or no modification

The problem is that what may be "proper" today (thinking, say, 5-10 years out) may fail the test in 10-15 years. One has to design software under some constraints, usually defined by the problem boundary and thus it by definition cannot be infinitely scalable and flexible.

Also, the problem is the code ends up being maintained by programmers of varying levels of competency. Even if competency is not a problem, the varying styles inadvertently lead to spaghetti and inconsistent code.

acqq2• 13 years ago

It doesn't work that way. The underlying assumptions change. The program can be perfect but still not matching what becomes necessary in a few years. The example they write about is exactly such a kind. Sometime in 2007 it was apparently slow to access files from the distributed storage on Google. Therefore, a Python script was written to fetch these files to some local disks. Python was obviously enough since that process was slow. Then, another program, written in C++ was to deliver the files from the local disks to the clients. The C++ program was then "sub-optimally" maintained, simply because it wasn't on the critical path for the company. There weren't even the good test cases to verify correctness of the changes.

Still, until 2013 a lot of assumptions based on which the program was written changed completely: Accessing files from the distributed storage became faster. The typical computer which is to deliver the data seems to have much more free RAM, allowing using RAM for caching purposes instead of the local hard disk. The nature of the load changed so much that accessing the local hard disk became the bottleneck anyway. Which is not so surprising: one disk seek is still around 10 ms, which means as soon as you can't deliver something from RAM you can deliver only up to 100 items from the disk per second, when they are not in the same block of data.

Don't tell me you have to write the program for "all possible assumptions." Unless you have unlimited time, you write it to solve the problem under existing assumptions. Once they change, you change the implementation.

Moreover, to achieve the changes related to the underlying assumptions, Go wasn't needed in the current example, but it was the most convenient approach for Brad, who is by the way obviously an extraordinary programmer.

What I agree with you is that if the program is bad even under the initial assumptions, it's certainly something that should be observed and tracked by the good manager: he must know that he can't rely on such program and that whenever the program gets any heavier load, the problems will happen.

dman• 13 years ago

Not all software operates at scale. The backwards compatibility fostered by posix and wintel has meant many such applications have been silently doing their jobs for many years without (m)any changes.

raldi2• 13 years ago

Or just deploy the half-assed solution, which will get the job done and allow your limited engineering staff to focus on higher priorities. Then, a decade later, when the company is worth hundreds of billions, you can rewrite it from scratch the proper way.

See also: A Tale of Two Bridges http://hintjens.com/blog:16

InclinedPlane2• 13 years ago

It's a good sentiment, but it doesn't usually work that way.

A decade later you find that you don't just have one half-assed solution to rewrite, you have a decade's worth. And because you're spending every second of your working life maintaining that teetering pile of kludges the idea of setting aside enough time to do a proper rewrite is sheer fantasy.

Certainly there is a good case to be made for accepting technical debt as a necessity in order to get products into the market and revenue coming out of the market, but on the other hand the typical software development organization tends to be drowning in technical debt. Pay down your technical debt regularly and quickly, not after a decade. For every example of the half-assed temporary solution that made the company money there is a story of development hampered by the burden of maintaining more technical debt than is wise.

JulianMorrison2• 13 years ago

What causes code to turn to shit is taking a well designed, well-factored solution, and hanging a patch off the side to make it do something new, or to fix a use case it wasn't designed to handle.

The "constant love" needed is that after a change you must redesign or at least refactor. Preferably every time, or you build up a debt of work.

16s• 13 years ago

Have any heavy C++ devs switched to go? What's the major advantage/compelling reason to switch? It seems to me that Go is more comparable to Python/Ruby and Java on the Web, but maybe I'm wrong.

yareally• 13 years ago

I'm not a heavy C++ developer and only use it to augment other languages when performance demands it, but not having to deal with memory management and the problems it can create when binding to another language is a nice benefit. The only drawback with Go though for my usage is when using on Android, since it has to be compiled statically when used with the NDK.

ihsw2• 13 years ago

* No circular dependencies (big win here)

* Package importing is easy and a first-class language structure

* Fast compiles (related to no circular dependencies)

One of the more controversial features is no inheritance -- only composition. It's an interesting break from vertical/horizontal inheritance schemes.

ancarda• 13 years ago

I really don't understand why dl.google.com isn't just running Nginx or some web server. It's just serving files. Why does it need software written in house?

GeneralMayhem• 13 years ago

Google is the only company on the planet with anything approaching its scale requirements. It's entirely possible that even nginx breaks down at their level of speed/concurrency/distribution.

x10242• 13 years ago

True, but it isn't the only company serving apt packages, and it isn't even the biggest one. Considering that their apt server was as slow as it was(http://talks.golang.org/2013/oscon-dl.slide#9) - they obviously did something wrong that everyone else was doing right.

Perhaps the problem was that dl.google.com was being used for too many things at once, but this doesn't excuse building custom software that offers worse performance than free off-the-shelf products that literally everyone else has been using for years.

skriticos2• 13 years ago

Nginx was built with 'single server' architecture in mind while Google generally operates on a 'distributed everything' architecture. This involves stuff like distributed file systems/data stores. My guess is it's not compatible with the single file system interfaces of Nginx. Also note that dl.google.com serves stuff on a massively bigger scale than the majority of the rest of the internet. On that scale I'd guess they face some rather uncommon challenges and bottlenecks that can only be addressed by custom software.

x10242• 13 years ago

True, but a static-file server should be trivially scalable. That is, scalable by placing it behind a load balancer(or using any number of alternative methods that don't acutally touch the server itself). Even if the server implements Access Control, it should still be trivially scalable.

Also, nginx doesn't have such...interesting ideas as the custom server: http://talks.golang.org/2013/oscon-dl.slide#19

I mean, if by "rather uncommon challenges" you're referring to the "what are threads?" design philosophy of the original... then yeah, it does require "custom software".

ihsw2• 13 years ago

dl.google.com isn't just a place to dump files to the public web, it's also got complicated ACL schemes that define restrictions on data access. dl.google.com is used internally.

coldcode• 13 years ago

Maybe they should take a look at feedburner.

general_failure• 13 years ago

I hope all journalism majors are sitting up and reading this carefully. This is how you make a nice imaginative story out of a small interview. A good story/link bait needs something in desperate need ('lousy code'). Then it needs a hero ('gopher team'). Then it needs a title which will catch people's attention (people can't help reading about 'google'). Creative journalism at it's best.

In case it isn't clear, the only store here is that one guy wrote Google's download server in Go because it was not working well. That's it, that's the only story. Google doesn't send some task force or anything like that.

contol-m• 13 years ago

They should really dispatched to fix the Google Drive Client program for Mac. It consistently maxes out a CPU even when there is no sync going on. I can consistently reproduce this issue on all Macs I own.

VikingCoder• 13 years ago

"The team was able to make many improvements to the way the language handles clustering and file transfers."

Either the journalist had no business writing this sentence, or the people who designed Go put things in the language that should have been in the libraries. Or both.

outside1234• 13 years ago

Its totally untrue that Google uses open source first before writing something themselves. Literally, the whole infrastructure is custom google goop and it can take a noogler months before they can do the most basic task.

You can see this in the response to "we can't serve files correctly" in that there was to rush in write new code in a Google language (Go), as if there weren't thousands of existing ways to solve this with an open source component already.

WestCoastJustin• 13 years ago

I think this was initially reported in 2012 [1], because I remember reading about it a while ago. While I was Googling about for the link, I came across the "dl.google.com: Powered by Go" slide deck [2].

[1] http://grokbase.com/t/gg/golang-nuts/12asyfnbea/go-nuts-dl-g...

[2] http://talks.golang.org/2013/oscon-dl.slide#1

osth• 13 years ago

"If code doesn't receive constant love, it turns to shit."

But it sounds like this code was receiving "love", only the "love" was coming from run-of-the-mill "just get it to work" C++ programmers.

I guess we need context to understand Fitzpatrick's statement. Perhaps he just means code at Google.

Are there any examples of code that has survived for many years without "constant love"? Netcat has not received "constant love" over the years. It hasn't turned to shit. Neither has the original awk. I can think of many other examples. These programs have proven to need very little maintenance.

I posit that simple programs that are well written do not need "constant love". They only need love when there's a bug. And there are plenty of programs that are in constant use where no bug has been discovered for many years. The bugs were vetted and fixed early on, decades ago.

Hence I disagree with Fitzpatrick.

MaysonL• 13 years ago

Ah, the joys of language PR. Alan Kay was right when he bemoaned the eternal pop culture of programming.

Google's ‘Gopher Team’

Related Stories