Goroutines are not significantly lighter than threads

https://matklad.github.io//2021/03/12/goroutines-are-not-significantly-lighter-than-threads.html

by todsacerdoti • 5 years ago

120 111 5 years ago

matklad.github.io

gbrown_• 5 years ago

The most commonly cited drawback of OS-level threads is that they use a lot of RAM. This is not true on Linux.

I was under the impression this was context switching rather than memory usage, which I haven't seen much about when these models are compared.

zlynx• 5 years ago

Goroutines had excellent context switching back when Go preferred to run in a single thread. Version 1.3 or so?

Now that Go defaults to multiple threads it has lost all of that advantage. Goroutine switching and channel send/receive has to apply all of the locking that any multithreaded program has to use.

rurban• 5 years ago

You have to use locking only with inferior frameworks, who never heard of lockless threading or non-blocking IO. Locking also makes concurrency not only slow but also unsafe.

tapirl• 5 years ago

A thread is only 3 times (memory consuming) as large as a goroutine.

Now, the official implementation allocates a 2k-byte stack for each new goroutine. The initial stack could be much smaller in theory.

labawi4• 5 years ago

Asides from significant memory use from uncounted overhead for native threads, fun starts when your threads start actually using more than a single page of stack, calling into who-knows-what, instead of just sleeping.

Typical programs will transparently grow stack on demand, both with native threads (memory not mapped until used) and goroutines / green threads.

Unlike green threads of typical runtimes, I don't know of native threading that releases no longer used stack space, until .. I guess the thread is destroyed, so sporadic large stack use on can blow up your memory.

masklinn4• 5 years ago

Unlike green threads of typical runtimes, I don't know of native threading that releases no longer used stack space, until .. I guess the thread is destroyed, so sporadic large stack use on can blow up your memory.

They don't. Although if you know that may be a concern for your application you can easily enough `madvise` after calls which may map significant amounts of stack, in order to release it.

otabdeveloper44• 5 years ago

Under Linux memory is allocated on first access to a mapped page, so actually no memory is being allocated in both cases.

nmca• 5 years ago

3x seems significant?

coder5432• 5 years ago

Not only that, but the author takes a single data point and attempts to draw a line.

Without measuring 10k, 20k, 30k of each and seeing how the memory usage changes, we are seriously comparing apples to oranges here.

Go's allocator will hold onto a decent chunk of memory just to amortize the cost of reaching out to the operating system. Just asking the operating system how much memory the Go application is using doesn't tell the whole story in this benchmark.

Is it 3x or 30x? The author certainly doesn't know based on taking a single data point from each application.

Goroutines are more memory efficient than full OS threads, but they're also cheaper to spawn and the Go runtime can optimize for certain common use cases very effectively.

I've also personally had serious difficulty in the past with getting Linux distros to let me launch tens of thousands of OS threads. Long before I run out of memory, I hit various limits that block the program from spawning additional threads. Some of them can be adjusted, but I never managed to spawn threads arbitrarily up to the amount of memory that was available... probably just my own failing, but that alone is reason enough not to spawn an unbounded number of OS threads.

labawi4• 5 years ago

There was a related post about a month ago[1].

I believe the article fails to account for, among other things, the virtual memory overhead that comes with sparse allocations for large maximum stack sizes, that would about double the numbers. With 2MB or so stack spacing, if you only use a single page of RAM per thread, you still need another whole page of RAM in the "page table" (actually a trie) and on linux those are not counted in the process RES memory usage.

That being said, creating thousands of native threads has lots of other pitfalls, including stack ones - would not recommend as a general strategy.

If haven't tried creating actual threads, but for test mapping stack-like staggered memory, I've had success with:

  sysctl vm.overcommit_memory=1
  sysctl vm.max_map_count=10000000
  swapoff -a

Remember to keep an eye on free memory, not process RES or similar, that doesn't count page tables and other overhead.

[1] https://news.ycombinator.com/item?id=25997506

gnabgib• 5 years ago

It does, the article and results contradict the title. I'm not sure that "lightweight" purely can be capture by memory usage either.

fulafel• 5 years ago

To argue in favour of the article: if you do real work in the coroutine / thread, there is going to be much more data in the work context than the stack and the memory usage difference is likely to be negligible.

eeZah7Ux2• 5 years ago

No, the article correctly points out:

A thread is only 3 times as large as a goroutine. Absolute numbers are also significant: 10k threads require only 100 megabytes of overhead. If the application does 10k concurrent things, 100mb might be negligible.

If it's still not clear: if your application has a good reason to run 10k parallel tasks it's most likely doing something complex that requires plenty of RAM.

It's very unlikely that you really have to save those 100MB of RAM and at the same time you cannot rethink the architecture to stop using this level of parallelism.

And even so, that would justify using a coroutine library, not a whole programming language.

hedora• 5 years ago

I’d be curious to know how C++20’s stackless coroutines compare on memory usage.

jbandela12• 5 years ago

C++ stackless co-routines are pretty lightweight.

A few years back, when I was writing something for a presentation, I spun up 50 million coroutines, and arranged them in a ring with channels and passed a value all the way from each co-routine to the other, on Windows Laptop with 8GB of RAM.

I don't recall the exact amount of memory used, but my laptop handled it really well.

Koiwai2• 5 years ago

That's stunning, if (honest if, I really don't know the answer) they consume 4K memory each, that would be 50M * 4K = 200G.

The_rationalist• 5 years ago

How does Loom threads overhead compare to goroutines and Kotlin coroutines?

aardvark179• 5 years ago

Well the costs are subtly different. The size of the virtual thread object is fairly small, but is not the only cost. Rather than allocating a stack for a virtual thread we freeze chunks of stack into special objects and thaw the back onto the stack of the OS thread (we can do this because we know there are no pointers into Java stack frames). This means that the cost can be very small, but you may place more of a load on the GC if a collection occurs between the freeze and the thaw which causes this stack object to be moved.

[edit] I should mention that there are other potential overheads if you use things like thread local variables as these may require more work from the GC to be collected. We are working on a new mechanism which will should be better in this regard and be a better API for many of he uses of thread locals.

anonuser123456• 5 years ago

If the application does 10k concurrent things, 100mb might be negligible.

When you think about memory in quantities of 1,10,100,1000 million units, a few hundred MB makes a big difference (e.g. Android devices).

hermanradtke• 5 years ago

Hence the word “might”. If it is not negligible then one needs to opt for something async IO at the cost of more complexity.

a2g4rAVy• 5 years ago

Developers don't care about memory on consumer devices. Look at any modern website.

eeZah7Ux2• 5 years ago

On an android device you don't need 10k coroutines.

And on severs - people implemented webservers like apache 25 years ago without coroutines and it ran with with very limited RAM.

smasher164• 5 years ago

I thought this article would talk about overcommit and demand paging, but it just counted the RSS between two processes and called it a day.

They don't even attempt to distinguish the size of the stack from the rest of the runtime.

echlebek• 5 years ago

With MADV_FREE behaviour, RSS isn't even a useful measurement.

mplewis• 5 years ago

Even if Goroutines were significantly heavier than threads, I would prefer them for many reasons.

twic4• 5 years ago

Such as?

adonovan• 5 years ago

Faster context switches. Vastly larger recursion depth before stack overflow.

b0sk• 5 years ago

Easier to reason about. easier to communicate. Threads are a low level primitive. the goroutines seemed to be designed with concurrency in mind.

pantulis• 5 years ago

I'd say that they are easier to write code and to reason about when compared with, say, POSIX threads.

tmpz222• 5 years ago

You're being a massive contrarian right now - asking others for supporting evidence while providing none to support your own position.

bww• 5 years ago

I guess this one contrived benchmark proves it. Case closed.

networkimprov2• 5 years ago

I think there's a bug, tho it might not make a difference to the results:

    for i := 0; i < 10; i++ {
       go func() {
          f(i) // sees whatever value i has when f() is called, usually 10
       }()
    }

thewakalix• 5 years ago

For the curious reader: this can be solved by simply passing i as a parameter.

    for i := 0; i < 10; i++ {
        go func(i int) {
            f(i)
        }(i)
    }

networkimprov2• 5 years ago

Or commonly, tho perhaps confusingly:

    for i := 0; i < 10 i++ {
       i := i // new variable i for every iteration
       go func() { f(i) }()
    }

mykowebhn• 5 years ago

This is a very common newbie mistake. Fix by passing in i.

    for i := 0; i < 10; i++ {
       go func(i int) {
          f(i) // sees whatever value i has when f() is called, usually 10
       }(i)
    }

thereare5lights• 5 years ago

This looks very much like the mistake in javascript with callbacks in for loops.

rmrfrmrf• 5 years ago

absolutely wild to me that Real Engineers (TM) complained about this in JavaScript for years only for it to be called a "newbie mistake" in Go.

matklad5• 5 years ago

Thanks for the correction, that is indeed a bug!

jayd16• 5 years ago

Interesting. Integers are captured by reference in Go?

shawnz2• 5 years ago

Closures capture all in-scope variables by reference in most languages.

Same situation in Python for example: https://stackoverflow.com/questions/54288926/python-loops-an...

wmf• 5 years ago

This has been known since the days of NPTL and the O(1) scheduler. I guess it has to be relearned every decade or so.

jeffbee5• 5 years ago

Known but not well-known. There are many false beliefs out there. One of the most durable is the incorrect belief that a thread incurs the maximum stack size on creation, which isn't true on Linux.

otabdeveloper44• 5 years ago

Too much ego is at stake on wanting to write cool shiny new async language runtimes and not wanting to learn how your existing kernel API's work.

minaguib• 5 years ago

Good start but very simplistic

What happens when these threads or goroutines start doing real work (CPU, IO) concurrently ?

platinumrad4• 5 years ago

It's curious how people associated with the Rust project will vocally complain about unfair/shallow criticism then turn around and post something like this.

tln• 5 years ago

Out of curiosity, how is this unfair/shallow? It's not obvious to me

platinumrad4• 5 years ago

Given the domains where Go is most likely to be used, any comparison of goroutines vs threads should include the netpoller vs blocking socket i/o.

dang• 5 years ago

Please don't post in the flamewar style to HN. It leads to flamewars, which are predictable, nasty, and dumb.

https://news.ycombinator.com/newsguidelines.html

eptcyka• 5 years ago

Yeah, but its significantly cheaper to switch between goroutines than is threads.

Ameo• 5 years ago

Yeah that's what I was thinking as well. The article leads with "The most commonly cited drawback of OS-level threads is that they use a lot of RAM." and that's not the impression I had at all.

I've always felt that the biggest reasons that threadless concurrency primitives were adopted was to avoid the non-negligible cost of spawning threads and context switching between them.

jchw• 5 years ago

Not only does 3x seem quite significant, but thread context switches are also a large overhead and that goes untested here. If thread context switch overhead was as low as usermode context switching, there would be no use for coroutines since you could just use threads instead; I doubt it’s non-trivial.

(Of course, in Go, the scheduler also weaves in the GC IIRC, so an apples-apples comparison may be difficult. Micro benchmarks are just not that useful.)

P.S.: this article seems to work under the assumption that 10,000 Goroutines is a reasonable upper limit, or at least it feels as though it implies that. However, you can definitely run apps with 100,000 or even 1,000,000.

refenestrator3• 5 years ago

I thought the major cost was cold cache after switch in both cases?

Kernel context switches are pretty light compared to the following slowdowns due to cache.

jeffbee5• 5 years ago

The performance of the Go runtime with a million blocked goroutines is pretty OK, but its performance with even 1000 runnable goroutines is not great at all. You really need to think about which you are going to have.

gnfargbl• 5 years ago

True, but if you regularly have 1000 runnable goroutines then could you not reconfigure your app to have, say, 64 runnable goroutines, and get better throughput? Large numbers of goroutines do seem to be a good fit for problems which are mostly waiting on the network.

EricBurnett• 5 years ago

Is 1k runnable goroutines materially worse than 1k runnable threads? (Honest question, I don't know myself).

otabdeveloper44• 5 years ago

If thread context switch overhead was as low as usermode context switching, there would be no use for coroutines since you could just use threads instead

Two points: a) native threads switching overhead is extremely low, probably lower than usermode Go thread overhead; b) the utility of usermode threads isn't to lower overhead.

It's a given that usermode threads introduce lots of overhead, if only because you're reimplementing the scheduling machinery that already exists in the kernel and that you're already paying the performance penalty for.

But theoretically, you could implement specially tailored scheduling algorithms specific for your app and gain an advantage there - kernel schedulers are general-purpose and can introduce latencies, starvation and other undesired special effects.

P.S. The Linux kernel also has interfaces for changing the thread scheduler algorithms. This gets you 99% of the way there - native threads + round-robin realtime thread scheduler gets you the performance benefits you want without having to rewrite schedulers and context switchers in usermode for every language runtime.

Thaxll2• 5 years ago

"native threads switching overhead is extremely low, probably lower than usermode Go thread overhead"

This is most likely very wrong.

eliben• 5 years ago

Some measurements of context switching: https://eli.thegreenplace.net/2018/measuring-context-switchi...

rufius• 5 years ago

For giggles, would've liked to have seen Erlang/Elixir in the mix.

thesz• 5 years ago

And Haskell. Haskell green thread overhead is half of Erlang.

whateveracct• 5 years ago

Haskell's concurrency blows Go's out the water.

masklinn4• 5 years ago

Slightly heavier than goroutines: according to the efficiency guide[0] a process is 338 words (2704 bytes), when spawned.

This includes a 233 words (1864 bytes) private heap.

This likely wouldn't work for TFA (as it uses "active" threads) but it should go lower if you immediately hibernate the processes:

The heap is then shrunken to the exact same size as the live data that it holds (even if that size is less than the minimum heap size for the process).

While I've not tested it, I assume a just-started process has nothing on the heap, so it'd go down to just the "overhead" 105 words (840 bytes).

Though that's at the cost of immediately allocating a new min-sized heap when the process wakes up.

[0] http://erlang.org/doc/efficiency_guide/advanced.html

flazzarino• 5 years ago

Would a better comparison let the threads build up a few MBs of non-identical stacks?

halayli• 5 years ago

there are cases were coroutines and threads can be used interchangeably.

But often times, they co-exist together to solve the problem. That's the case in Go. goroutines run on threads after all.

Op's could have used a better example like a web/tcp server handling 100k connections and showed the difference between coroutines alone, threads alone, and both combined.

ynik• 5 years ago

The most commonly cited drawback of OS-level threads is that they use a lot of RAM.

"RAM" is inaccurate in that sentence.

It's important to realize that the common knowledge "you can't have 10k threads" comes from 2 decades ago when we were running 32-bit CPUs. A thread may only need a couple pages of RAM; but it needs a whopping 1 MB (or more) of virtual address space (at least with default stack sizes). You simply can't start 10k threads in a 32-bit process, as you'd need 10 GB of address space. OS-level threads use a lot of address space and thus were unsuitable for the C10k problem back in the day. But in 64-bit world, 10 GB is nothing, the address space usage can almost be ignored and the RAM usage is much less bad. (but: RAM usage of page table entries also needs to be considered. the article forgot about this)

Of course, just because it works now doesn't mean that 10k threads are a good idea.

Koiwai2• 5 years ago

Nice elaboration!

Goroutines are not significantly lighter than threads

Related Stories