Rob Pike – 'Concurrency Is Not Parallelism' [video] (2012)

concurrency _is_ parallelism, but for I/O. People often think of parallelism for the case of making something go faster - eg placing two computations in parallel (the definition posed in the video), OR placing two I/O operations in parallel - so this is the keyboard-vs-mouse in the OS, even when you're on one core only; this is multiple web requests in JavaScript, which does not support multi-threading, but 100% does support concurrency for I/O operations - that... badum-tiss! RUN IN PARALLEL.

I get the point of the talk, and it's well interesting, but I think it depends on how one views things.

pdpi• 18 days ago

concurrency _is_ parallelism, but for I/O.

Not really. They're just separate but related concepts.

E.g. coroutines are a form of concurrency that doesn't have to involve any sort of I/O, you're just taking two logical processes (e.g generating a sequence and consuming it) and abstracting away how they execute relative to each other.

Describing your tasks using the language of concurrency is a requirement for process-based parallelism (multiple CPUs/cores), but data-level parallelism (SIMD) is a form of parallelism that doesn't involve concurrency either.

threatofrain• 18 days ago

Concurrency is the property of a program or algorithm such that:

    - the program is decomposable into partially ordered or unordered units of execution
    - the program result remains determinant despite partial ordering

Your data-level parallelism is taking advantage of the concurrent properties of a problem.

lelandbatey• 18 days ago

No, and that's the point of the article. What you are calling parallel w/r/t IO should be called concurrency (conceptually happening at the same time by virtue of being able to interrupt and resume units of work). The reason IO APIs like you've described is concurrent but not necessqrily parallel is because there is no guarantee in the API that they both happen literally simultaneously; I could build a JS runtime that "works" for all the code written against XMLHTTPRequest (ignoring side-effects) but which under the hood only ever makes one HTTP request at a time. And because I can do that, that means JS code is living in a concurrency-only world, even though as an implementation detail most runtimes support parallel execution of those concurrent operations.

quietbritishjim• 18 days ago

there is no guarantee in the API that they both happen literally simultaneously

There's no actual guarantee in the API that if you spawn multiple threads and call blocking network I/O that those happen literally simultaneously. Maybe the OS has a big mutex on network I/O to serialise them.

Of course, that's not what happens in practice. But neither is it what happens, in practice, to async network APIs called concurrently in one thread. So I don't think that can be the difference between concurrent and parallel.

sulam• 18 days ago

To HN moderators: title needs to note that this talk is 13 years old.

folkrav• 18 days ago

The title must have been edited in the last 40min - it shows (2012) now

thebeardisred• 18 days ago

Slides - https://go.dev/talks/2012/waza.slide

bjoli4• 18 days ago

It seems like we forgot about lightweight fibers for about 30 years and then collectively rediscovered it in about 2010. sure, Java did green threads, but not like m:n stuff.

I sometimes wonder where we would be now if people would have gone "Wow, mr Reppy! concurrentML is so cool!" in 1993.

instead we got pthreads and collective amnesia and later we got go's girly times and channels which are only half way there.

kllrnohj2• 18 days ago

That's because they are very rarely useful. This was true then and it's still true now. There's just not many workloads where it makes sense to need to rapidly launch a thread that doesn't need to do much of anything but does need to exist for a while before terminating.

What is useful is the state machine aspects of things like coroutines or async/await, but those aren't quite fibers and very much aren't M:N threading. A major use of them is in UI where they have strict thread requirements even.

freedomben• 18 days ago

Exactly. When I started getting into go 15 years ago, and 10 years ago or so into elixir, I loved the easy concurrency and tried to use it to the maximum possible. What I discovered is that they're just aren't that many things that you can do concurrently. Reading many different files or servicing many different network sockets being the exception, but realistically that isn't done very often relative to single threaded stuff.

State machine/management is really what is most useful

bjoli4• 18 days ago

you can implent something like futures on top of cml and use it as async. then you get async for stuff where async is useful and a proper way to write parallel programs for when async would lead to all the bad stuff async leads to.

i rarely write gui stuff and I find async is rarely what I need. in f# I at least have the option to use hopac.

smallstepforman• 18 days ago

Yet the average software engineer hasn’t heard or Carl Hewitt, Joe Armstrong, has never programmed in an Actor Programming model, and dont even know what it is.

BobbyJo• 18 days ago

Back then you could spend 6 months making something twice as fast, or wait six months for computers to become twice as fast. People obviously did the latter.

It also became a cycle: People write single threaded code -> CPUs/OSs optimize performance/ergonomics for single threaded programs -> People write single threaded code ->...

It wasn't until scaling slowed down that interest/investment in concurrency/parallelism took off.

badc0ffee• 18 days ago

"girly times" = goroutines?

bjoli4• 18 days ago

yep. writing on my phone. notoriously bad at proof reading.

petilon• 18 days ago

After watching the video I can see how go lang makes it easier to write correct concurrent programs. But with AI writing the code these days, it is just as "easy" to write correct concurrent programs in Java (because AI is doing the work). Java's virtual threads are light weight, just like go's routines. Java's LinkedBlockingQueue offers roughly the same functionality as go's channels. I would like to hear from go experts as to why I am wrong. Does go have any inherent advantage if AI is writing the code?

DonHopkins• 18 days ago

Just set the source pile of C++ manuals on fire. No gophers needed!

abetusk2• 18 days ago

Concurrency is a programming abstraction that conceptually allows multiple independent processes to run at the same time using scheduling. For example, running ten programs at once with only a single CPU. Concurrency is effectively CPU scheduling with maybe some other concepts of communication between processes for coordination.

Parallelism is running processes at the same time.

The driving motivation behind concurrency, as far as I can tell, is that it's a concept that can help reason about complex tasks with the added benefit of being able to take advantage of parallelism.

My problem with Pike's advocacy for concurrency, and Golang in particular, is that it's an abstraction that imposes a world view on what's a good programming paradigm and, more practically, how to take advantage of parallelism, both of which have limited utility compared with considering parallelism directly.

My opinion rather than some abstraction that imposes a world view on how to take advantage of parallelism, it's better to use how people actually take advantage of parallelism as a basis for abstraction.

I think Golang is a fine language but we now have 14+ years of hindsight to see how well it's fared and how much it's influenced the compute landscape. In my opinion, the "concurrency not parallelism" misses the simple issue that we want things to go fast. Which has had more influence, Golang's idea of concurrency or taking advantage of GPU parallelism? I think the answer is clearly taking advantage of GPU parallelism.

CyberDildonics2• 18 days ago

Parallelism is like data structures. There are many different ways to do it and people should not be trying to do it themselves in a bespoke way on every program.

Having one way built into the language is like a language using a hash map for everything. It might be convenient when it works, but the idea that there is one best way is only a little better than people fumbling their way through it from scratch.

abetusk2• 18 days ago

I agree with a lot of this sentiment but it assumes a level of stability so that standardization can be effective. Standardization before stabilization has the potential to cripple efficiency with inefficient abstractions.

One benefit of a programming paradigm is that it can allow for an increase in speed of software development at a marginal cost of run-time efficiency, but if the paradigm is so out of step with the underlying hardware, the run-time efficiencies can be orders of magnitude slower than a bespoke way.

Standardization and abstraction are most effective when there is data about which bespoke ways work the best. Choosing standardization and abstraction before experimenting on the solution space rarely works well.

zamalek• 18 days ago

For example, running ten programs at once with only a single CPU. Concurrency is effectively CPU scheduling with maybe some other concepts of communication between processes for coordination.

This is called preemptive multitasking. This is like declaring that "sorting a list is quicksort." This can realize concurrency.

The reason that this is important is that cooperative multitasking (async/cps/fibers/safe points) is also concurrency. And I'm sure there are other exotic forms of concurrency that I'm not aware of beyond multitasking.

Concurrency is a property of a program/system, not a mechanism.

nwhnwh• 18 days ago

What languages do you like?

neerajdotname2• 18 days ago

Concurrency is like "con man". That's how I remember it. https://www.bigbinary.com/blog/gvl-in-ruby-and-its-impact-in...

mattrighetti• 18 days ago

I won’t subscribe to watch a video, sorry. I think this is the YouTube version though: https://youtu.be/oV9rvDllKEg

em-bee2• 18 days ago

it works for me without login or subscription. no idea. but at least i an confirm that the youtube video is the same one.

you also want to open the slides and run them in parallel because the video doesn't show them very well. the slides are linked in the youtube description, but i don't see them linked on vimeo: https://talks.golang.org/2012/waza.slide

Rob Pike – 'Concurrency Is Not Parallelism' [video] (2012)

Related Stories