Go 1.6 GC improvements

https://docs.google.com/document/d/1kBx98ulj5V5M9Zdeamy7v6ofZXX3yPziAf0V27A64Mo

by artursapek • 11 years ago

115 30 11 years ago

docs.google.com

ascii_lover• 11 years ago

Is there a way to have docs.google.com serve up plain text?

NightMKoder• 11 years ago

Looks like there is. Given

https://docs.google.com/document/d/1kBx98ulj5V5M9Zdeamy7v6of...

change "preview" to "export?format=txt":

https://docs.google.com/document/d/1kBx98ulj5V5M9Zdeamy7v6of...

tedunangst3• 11 years ago

A+++ for the second link. Would read again.

sitharus2• 11 years ago

I feel like sharing google docs links is the new emailing a Word document. I'm sure loading a static HTML page faster for the features needed.

tedunangst3• 11 years ago

The best part is that if you read it on iPhone, every two minutes it switches tasks to the App Store trying to get you to download the app. Not once, not twice, repeatedly.

MoOmer• 11 years ago

Hasn't happened to me yet (on Safari)

rattray• 11 years ago

Stuff like that is why I switched to Android. Had to go back to iPhone for a couple weeks recently and couldn't stand the poor inter-app coordination.

eliben• 11 years ago

Except that it lets you export to TXT if you want. And open it from almost any device. And is much less likely to get your computer infected with something nasty

sitharus2• 11 years ago

oh it's much better than Word, but still a paged document format on an infinite canvas.

TheHippo• 11 years ago

I think they use this for internal planning. I saw this a few hours ago with not all issues from the bottom of the page crossed out.

This is more a "let's open our internal communication" thing than a final documentation for end users.

enahs-sf• 11 years ago

I feel guilty as I haven't even upgraded to go 1.5 yet.

zac1944• 11 years ago

i will wait 1.6 too.

nindalf• 11 years ago

You're missing out, to be honest. Moving to Go 1.5 is painless and the latency wins are huge.

landr0id• 11 years ago

You shouldn't. 1.5 stable hasn't been released yet.

twotwotwo2• 11 years ago

It looks like a few things in there could further reduce typical and peak pause times, by backgrounding more phases of the work and rescanning stacks late so it's less likely much marking has to happen in the pause. I'm less clear on this, but "embrace write barrier during scan" and the other pacing changes might also better spread out work during the concurrent phases.

The pause improvements in 1.5 are no joke -- here's a benchmark from the GopherCon talk on it: http://i.stack.imgur.com/4TOux.png (1.5 is the yellow dots along the bottom)

(That talk is https://www.youtube.com/watch?v=aiv1JOfMjm0 and the slides are https://talks.golang.org/2015/go-gc.pdf )

And here's a graph from a real-world memory hungry app on a beta--300ms pause down to 4ms: https://twitter.com/mkevac/status/620872308446625792/photo/1

(Presentation on that app here: http://www.slideshare.net/MarkoKevac/markogoinbadoo )

Another app's pauses went from ~12ms to ~2ms: https://twitter.com/inconshreveable/status/62065078666255564...

And this doc means Go isn't moving to a generational GC now, which the pre-1.4 proposal speculated they might do in 1.6. That would mostly be a throughput win (discard short-lived garbage cheaply and early, so full GCs can be rarer), not a pause win (it could add new short pauses for young-gen GCs). Seems sane for now, since the polish they're talking about seems difficult, directly useful, and important as a basis for future work, and the pauses seem like the larger headache for the working Gopher anyway.

tedunangst3• 11 years ago

I've wondered for a while how much go could benefit from better escape analysis. Especially if they don't do generational.

Intuitively, it seems quite some escapes are only returned to the caller, then lost. If the compiler rewrote those functions as operating on references to caller provided memory (on the stack), it could reduce transient garbage.

twotwotwo2• 11 years ago

So, watching https://twitter.com/golang_cls, some escape-analysis and general garbage-prevention stuff landed (much from, or with the involvement of, the always-amazing Dmitry Vyukov).

Maps don't always escape now, for example: https://go-review.googlesource.com/#/c/3508/

Code was added to recognize that some ops don't cause escaping: https://go-review.googlesource.com/#/c/3031/ https://go-review.googlesource.com/#/c/8201/ https://go-review.googlesource.com/#/c/3162/

dvyukov wrote about some potential improvements and linked to issues in Feb: https://docs.google.com/document/d/1CxgUBPlx9iJzkz9JWkb6tIpT...

I think I saw even string bytes can be stack-allocated sometimes now, but couldn't immediately find the CL.

It looks like this is one of those things where one just has to chip away at cases. Movement happening; just a ton of round tuits required.

namelezz• 11 years ago

Anyone knows when Go is going to have generics.

recentdarkness• 11 years ago

This SO question covers this question quite well: http://stackoverflow.com/questions/3912089/why-no-generics-i...

Beside the accepted answer, there are also some other useful answers included.

TheMagicHorsey2• 11 years ago

I have a question that maybe doesn't belong here. I know people have asked a lot of questions about Go in a skeptical way, and I promise I'm not coming at it from that angle. I actually use Go quite a bit (mostly on App Engine) so I do like it.

My question is, given all the things being built around Go, like Kubernetes and various other job and cluster management tools, why weren't those things absorbed into the language or standard library. The way, say it is with Erlang?

In Erlang a lot of the multi-machine coordination work seems to be really well thought out, and its part of their standard OTP library ... which seems pretty mature (at least to someone who is a novice like me).

Why doesn't Google use Erlang? And if they don't use Erlang, why isn't there some sort of cluster management stuff built into Go?

I mean things like cross machine channels, automatic go-routine (or process) restart on failure, process monitoring, etc.

hariharan_uno• 11 years ago

I think in the beginnning, there was some effort to have channels communicate over a network. But it didn't turn out easy / performant. Also, if they are absorbed into the standard library, I think their iteration speed is restricted because of compatibility requirements.

pjmlp• 11 years ago

Why doesn't Google use Erlang? And if they don't use Erlang, why isn't there some sort of cluster management stuff built into Go?

Because it requires a higher skill level than Go, which isn't what Google is after. Check Rob Pike's talk about it.

Gepsens• 11 years ago

Higher skill needed ? I don't think so, I think it's more about the fact that Erlang code is verbose and unreadable.

jlouis• 11 years ago

Google needs a platform they control. They don't control the Erlang ecosystem, and they have learned the hard way how that fares (Java+Oracle+Android).

The other problem is that Google needs to build tooling that fits into their existing code base. In this case Java, C++, Python, Javascript and so on. They had to build everything from scratch anyway for Go or Erlang had they chosen either.

A lot of the nice things Erlang provide is already in place in the Google infrastructure. So you are aiming at small services which are isolated. This is a place Go has an easy way of getting going.

Of course, language familiarity also plays a large role. Go is pretty easy to onboard if you have decent Java or C++ knowledge. Erlang requires you to rethink the way you handle computation. Granted, the long-term benefits in productivity you gain in Erlang is significant[0], but adding more developers to a project is easy for Google.

[0] Whenever I write Go, I get frustrated at how much I have to write to do even the basic data munging. In Erlang, this has far better flow.

zurn• 11 years ago

Erlang co-evolved with OTP so they have a naturally close relationship. The Go situation (and philosophy) is much different. Also the Go authors sound like they don't want the language to be too closely associated with Google.

jerf2• 11 years ago

"Why doesn't Google use Erlang? And if they don't use Erlang, why isn't there some sort of cluster management stuff built into Go?"

Cluster management is a hard problem. Erlang's cluster support really only hits one use case, which is that you've got a relatively small cluster... at least, by Google standards!... of several machines physically co-located, or at least, very close on the network together. That's a great use case to cover, but it is important to understand that's "all" it is. You can't build a 1000-machine, multiple-data center located around the world deployment on the default Erlang stack as if all those machines are on one cluster. What Erlang gives you is still a step up from the normal default; you can build that as a cluster in each DC and it's not that hard to start up "proxy processes" that allow you to talk to other machines in the data cluster via Erlang messages sent to processes... but now you're talking about doing work, not just using built-in support.

"cross-machine channels" are, in my considered opinion, impossible. The channel semantics include a synchronous rendevous that is impossible to guarantee over the network. (That is, when you advance past a channels send statement to a non-buffered channel, you are guaranteed that some other goroutine has received the value.) You can set up other objects that let you communicate remotely, but they can't be "channels"... they can only be other things.

(Note that you have to understand "Go channel" not as a vague "something that lets you send messages", but the specific and exact semantics it implements, which is important because without those very specific semantics, "select" doesn't work! And if you can't put it in a "select", it isn't a Go channel. Go channels have a very, very rich semantics full of strong guarantees, which can't be relayed across a network.)

Some of the rest of your last paragraph is fairly deeply addressed in this post of mine: http://www.jerf.org/iri/post/2930

TheMagicHorsey2• 11 years ago

Thank you for the article. That was quite illuminating.

azth• 11 years ago

Go channels have a very, very rich semantics full of strong guarantees, which can't be relayed across a network.)

Any place to read up on those guarantees?

Go 1.6 GC improvements

Related Stories