Using Go at The New York Times [video]

https://www.youtube.com/watch?v=bAQ9ShmXYLY

by somerandomness • 11 years ago

140 55 11 years ago

ScotterC• 11 years ago

Slides: https://speakerdeck.com/nytdevs/using-go-at-the-new-york-tim...

hueving3• 11 years ago

Sometimes I really wonder if the choice of the language is just a way to filter out 'boring' engineers. It doesn't seem like they particularly needed to use Go, but look at the attention it gets them for being 'cutting edge'.

spoutn1k• 11 years ago

Could you elaborate on what you mean by "boring engineers"?

ARCarr• 11 years ago

Engineers who don't want to try new things.

hueving3• 11 years ago

Sadly it's people who don't keep up with what's cool on hackernews.

mkawia• 11 years ago

I read/heard yahoo suffers from this problem ,one of the reason Marissa Mayer is struggling.

thrwwy92• 11 years ago

The ones who actually know their history and consequently, when they hear "NoSQL," they think of IBM's IMS and run as fast as they can in the opposite direction.

meowface• 11 years ago

pg touches on this in his Blub essay: http://www.paulgraham.com/avg.html

The first thing I would do, after checking to see if they had a live online demo, was look at their job listings. After a couple years of this I could tell which companies to worry about and which not to. The more of an IT flavor the job descriptions had, the less dangerous the company was. The safest kind were the ones that wanted Oracle experience. You never had to worry about those. You were also safe if they said they wanted C++ or Java developers. If they wanted Perl or Python programmers, that would be a bit frightening-- that's starting to sound like a company where the technical side, at least, is run by real hackers. If I had ever seen a job posting looking for Lisp hackers, I would have been really worried.

I think it's a valid recruitment strategy, though it does sometimes cause languages' hype to outweigh their actual value, which many would argue is the case for Go (even though Go is generally a nice language).

illicium• 11 years ago

pg, being a Lisp hacker, is really stroking his ego there.

bootload• 11 years ago

"I think it's a valid recruitment strategy, though it does sometimes cause languages' hype to outweigh their actual value"

Not always, found this example yesterday: Farmlogs, Clojure and image processing ~ https://news.ycombinator.com/item?id=9522397

mahyarm• 11 years ago

TBH, I feel like go is the next java almost. But I'm an objective-c programmer, so rust's non-gc memory management is ok with me.

emehrkay• 11 years ago

If I'm not mistaken, he did mention that one server's memory requirements shrank from 4gb to roughly 1.

It's kinda amazing that they use so many different tools

nkozyra2• 11 years ago

That's true of any very big organization and news companies are very compartmentalized and segregated. I recall quite a few different dev groups not including R&D at the Times that used dissonant tools - I didn't even see RoR listed, which is used extensively in news teams within the Times.

nkozyra2• 11 years ago

As a big fan and daily user of Go, I agree with this. I've used it to solve concurrency problems or problems wherein I could quickly replace a slower long-running process in Python with something just as simple but more performant.

I see a lot of codebase refactoring in Go that doesn't really need to be written in Go. I'm not sure what the impetus is if not to just do it or to reposition a staff's codebase. The argument for these sorts of things should be better expressed, in my opinion. "We moved to language X because ..." should be a simple 1-line explanation, even if it is indeed "we want to work in the language or attract a certain type of developer."

brnstz2• 11 years ago

(Disclaimer: I'm a former engineer at NYT and was a Go proponent there.)

The reality of the situation at NYT is that every dollar counts. Go is an effective language that enables a programmer to do both "scripting" and "real" tasks without switching between different environments. The email system described sends more email than many companies that only send email send, and have many more employees working on said system. It is one of the few systems at NYT that pays for itself with zero doubt.

The speaker did not know Go before being hired by NYT, and famously solved a Java programming task in his interview without an IDE and barely a typo (the only time I'd ever seen it done successfully, some people can't even run javac or vim).

It certainly doesn't hurt in hiring to say that you get to program Go, but filtering out "boring" engineers is not a chase of "coolness". It is seeking out engineers who want to take a risk and want to be productive.

hueving3• 11 years ago

No, I get that you want to filter out boring engineers. It's just unfortunate that it seems the only way our industry can do this is by changing up the tools and seeing who makes it through.

Some of the smartest engineers I know are not interested in go because they think it solved the wrong problems and prefer rust for something new.

nu2ycombinator• 11 years ago

And now I am curious about what is the "famous java programming task". And want to see if I can do it without using IDE.

nemothekid• 11 years ago

I recall reading something similar wrt Scala & Twitter. At the time Twitter really needed to scale up they chose Scala and had the effect of attracting people who wanted to work at a company because they used Scala. I can't find the link however

thrwwy92• 11 years ago

Granted, you did put "cutting edge" in quotes, but it's still worth stating explicitly, Go isn't cutting edge, and neither are Ruby, Node.js, NoSQL, or most of the other technological rehashes ignorant developers get excited over.

These people would never pay any attention to old, boring technology like Lisp or Smalltalk or even (given the success of NoSQL) ACID-compliant relational databases like PostgreSQL. But repackage some of the good ideas from them and pass them off as your own, fire up your hype machine, and these clueless devs will eat it up.

Why, look at the cult-like following Matz enjoys for creating a half-baked Smalltalk and Perl knock-off.

brightball• 11 years ago

90% of the value created by using Ruby is because of the exemplary tools put out there by the people IN that cult-like following.

There's a lot of programming languages and programmers totally focussed on application performance as well as squeezing the limits out of their hardware. The ruby community is focussed on developer performance and squeezing the limits out of their people.

If more languages and communities spent as much time on streamlining everything around time to market, Ruby would not be nearly as big as it is. Because most are not focussed on that, Ruby stands out significantly when it comes to getting big things done with fewer people.

That's why so many start ups use it. That's also why so many start ups eventually come back and refactor the slower parts of their system with something like Go...but the important thing is that they got to market, built enough of a customer base and revenue stream that they can both a) afford to refactor and b) need to refactor because so many people are using it.

Building a language and tools around people rather than process is a unique undertaking. Because of that Ruby and Matz deserve every ounce of credit that they get. Doesn't have to be right for everything, but what they try to do well they do exceptionally well.

taf2• 11 years ago

it actually makes good business sense for a new organization to capitalize on these decisions. It is similar to a consulting agency - show off the use of a cool/upandcoming programming language or approach and earn more eyes or potential customers. Side benefit- the team also gets to improve their skills and enjoy the job more.

vinceyuan3• 11 years ago

Are Go servers at New York Times or other companies impacted by the Golang GC pause? I am learning Golang and want to know if Golang GC pause is a significant issue in production.

_ak2• 11 years ago

We're doing RTB with Google DoubleClick AdX, and our bidder (written in Go) can easily handle 4000 QPS per server with only a 0.3 % error rate.

vinceyuan3• 11 years ago

Is 0.3% error rate caused by GC pause?

mrfusion• 11 years ago

Can we change the title to "Golang"?

agumonkey• 11 years ago

It's a relief to see post-redesign diagrams. But at the same time I have tiny 'bus factor' fear by relying on a single language, is it (that) absurd ?

Using Go at The New York Times [video]

Related Stories