Thread pools: How do I use them?

http://jvns.ca/blog/2016/03/27/thread-pools-how-do-i-use-them/

by ingve • 10 years ago

188 45 10 years ago

jvns.ca

Pirate-of-SV• 10 years ago

Or maybe there is a totally simple way and this could take me 5 minutes!

At that paragraph I immediately thought of http://aadrake.com/command-line-tools-can-be-235x-faster-tha...

ambicapter• 10 years ago

Holy shit.

I have been in technology roles for over 17 years

I guess this is what experience gets you, huh.

bronxbomber92• 10 years ago

Or if you know some basic performance characteristics of your machine (i.e. memory bandwidth + processor speed + number of cores) and roughly how much work you need to do (i.e. number cycles to do X), some back of the envelope calculations can often give you the same insight (i.e. "is X feasible with compute resource Y?").

throweway• 10 years ago

I like the style of writing in this post. Its fresh in the programming world where everyone seems to know it all for someone to admit something is hard or confusing.

Smircio-• 10 years ago

Heck yea. Im kind of new here and it can be so intimidating. I feel like everyone is an expert at everything.

radicalbyte• 10 years ago

It's not that, it's just that experts tend to contribute to threads where they can share their expertise.

So while it appears that everyone here is amazing; that isn't the case: we just tend to have something that we're very good at and comment on it.

The good thing is that you can learn a lot here as long as you don't get sidelined by some of the inane arguments and fighting :)

tatterdemalion• 10 years ago

A lot of people have discovered that speaking confidently can be a solid substitute for actual expertise.

chongli• 10 years ago

This is how everybody feels at one point or another. It's called imposter syndrome[0] and it's extremely common!

[0] https://en.wikipedia.org/wiki/Impostor_syndrome

teh_klev• 10 years ago

Don't be intimidated. I've got 30 years of being in the business of software development, telecoms, networking etc under my belt and every day there's still something new to learn, sometimes too many new things :) Also don't be frightened to ask about things you don't understand, I find the folks around here are generally quite an affable bunch.

lambda• 10 years ago

Heh. Keep in mind that people are going to be more likely to comment on something which they have knowledge of or an opinion on, so you will see a certain amount of selection bias in that most of the people who comment on something know something about it, even if there are large numbers of people reading and interested who are just keeping quiet (though of course there are people who also make comments on things they know little about).

Also, I know that for some things that I write about, I'm not actually an expert on it beforehand. Instead, I see an interesting question, say to myself "hmm, that's interesting, and I don't actually know the answer", go out to Google, books I own, or just try some experimentation out myself, and then comment on what I found. I usually do try to provide sources and some details about what I did when doing this, but I realize that it may sometimes make me look like more of an expert on topics than I actually am. For example, I have a silver badge in JavaScript on StackOverflow, even though I've only ever written a few dozen lines of JavaSript in my life (in fact, I've probably written more lines of it on StackOverflow than off), since I tend to be good at quickly researching and experimenting with a question and providing a good explanation, not because I'm an expert on JavaScript.

So yeah, it's great the Julia writes down her thought and search process while doing this, as it helps people realize that they too can go through the same process and learn new things.

steveklabnik• 10 years ago

All of the posts on this blog are like this, so you should read the archives, for sure. I agree completely.

barrkel• 10 years ago

If you, like, don't mind the, like, valley/bro/speak lingo. AWESOME, right?

I like structured arguments against a status quo - that's what's really required here. This article is one notch earlier in the mental digestion chain; a kind of experiential report, an extended anecdote. It's harder to tell if the problem is the library or the user. It asks the reader to work more, because they need to go on a journey, rather than just parse an argument.

I enjoy reported experience too, but I have less time for them unless they're specific to something I have a strong interest in. Otherwise, give me the executive summary.

Yes, I read less of the internet than I used to.

charlesism4• 10 years ago

I've read a lot worse on that score. At least the author didn't parrot a bunch of memes, or throw animated gifs everywhere.

abalashov• 10 years ago

Yes, the narrative form gives the impression that the author is a very! excited! person! full! of! energy!

But that's kind of refreshing sometimes. It's easy to get cynical about software engineering like I am. And I followed her stream of consciousness a lot better than I follow a lot of recondite technical prose, particularly when it concerns things mired in proprietary abstractions (Java).

hibikir• 10 years ago

And if you had actually met Julia (I am lucky enough to work with her) you would realize that this impression is 100% correct. It is not just an impression, but her whole approach to technology.

While it is good to have more cynical people too (especially for things like software stability), a company that forgets enthusiasm ends up losing its soul.

allcentury• 10 years ago

According to her bio, she works in Stripe's machine learning department so clearly a smart gal and appears to have a great amount of humility - enjoyed her post, was easy to digest and a light read.

xerophyte12932• 10 years ago

I have been following her for well over an year now for exactly this reason. She never preaches. It's more like taking her readers along on this journey of discovery. Very very humble and yet insightful

jonnycache• 10 years ago

While i admit it was a thrilling take on a concept as abstract as thread-pools, i think the issue lies in the nature in which we programmers talk about computing theory.90% of the time we are either telling (stack-overflow help, or receiving (asking for help on stack overflow) which requires us either to be the teacher or the learner, never in between like this author is.

smegel• 10 years ago

Maybe "Scala concurrency surprises" would be a better title.

masklinn• 10 years ago

It uses the JVM's threadpools directly so it applies to more or less any JVM-based language, and the problems it talks about (expensive sequential operation before threadpool submission and unbounded threadpool queue) are pretty common issues.

Witness the unbounded and unconfigurable (though replaceable — at your own risk since you're setting a "private" variable of the pool) queue of Python 3's threadpool: https://hg.python.org/cpython/file/default/Lib/concurrent/fu...

bpicolo• 10 years ago

Here's what that looks like in Python.

Although python multithreading outside of IO tasks or dropping into C code that drops the GIL isn't going to parallelize, the latter of which is the only way you're going to burn up cores. The API does make it seem simple though.

delroth• 10 years ago

The example is using multiprocessing.pool.ThreadPool which does not have the GIL problem you mention (at the cost of making it harder to share data).

someguy1234• 10 years ago

ThreadPool would lock the GIL, wouldn't it? I think multiprocessing.Pool uses processes and that's where you can avoid the GIL (each process gets its own) at the cost of making data sharing harder.

Even with the GIL, ThreadPools are still fine performance-wise for tasks that release the GIL such as I/O or a number of functions in numpy and scipy.

aftbit• 10 years ago

You're thinking multiprocessing.Pool. I think ThreadPool actually uses threads and does hit the GIL.

metachris• 10 years ago

I recently ran into the issue that the multiprocessing.pool.ThreadPool class is not cancellable in Python 2. It seems that only in Python 3 can you exit a ThreadPool with eg. a KeyboardInterrupt before all tasks are finished.

Since then I've been using this code, which works great across all platforms and Python versions: https://github.com/metachris/pdfx/blob/master/pdfx/threadpoo...

megamouse• 10 years ago

I've experienced this bug in Python 2 using "pool.map" and "pool.apply", but KeyboardInterrupt does work if you're waiting for a finite amount of time. So as a workaround you can use "result = pool.map_async(...); result.get(sys.maxint)". A little hacky but functional.

emmelaich• 10 years ago

Great writing.

I wonder if a ForkJoinPool [1] would be helpful here as it does work-stealing giving better utilisation.

Although I've read admonitions that they should not be used when doing I/O. [2] That article seems a little ranty and overblown though, so I'd like opinions on that.

1. https://docs.oracle.com/javase/8/docs/api/java/util/concurre... 2. http://coopsoft.com/ar/CalamityArticle.html

cmsd2• 10 years ago

A coworker showed me Akka Streams recently. It allows you to overcome the impedence mismatch between threadpools processing jobs at different speeds where one task feeds into another. Could be a neat way to join up the i/o bound tasks to the cpu bound tasks.

https://opencredo.com/introduction-to-akka-streams-getting-s...

kevinavery• 10 years ago

Yep, it's a really cool library. I'm pretty new to it myself, but I made a little gist of how one might approach this problem: https://gist.github.com/kevinavery/941e7d67c4f8b104f610

I was kinda surprised by how tricky it was to make a Flow that turned ByteStrings into separate String lines, but it seems like the custom GraphStage approach generalizes really well to more complicated stages.

wsargent2• 10 years ago

https://rnduja.github.io/2016/03/25/a_first_look_to_akka_str... is also pretty good.

maherbeg• 10 years ago

The article is definitely super ranty and a bit overblown. The Fork/Join framework is actually pretty solid and works for a wide variety of use cases and has a relatively simple API. It's an excellent article though and I highly recommend anyone implementing the use of Fork/Join to read it and get more in depth knowledge.

Java 8 made some improvements in code quality and safety and the framework underlies all of the parallel streaming abstractions used going forward.

It's definitely worth learning all of the concurrency abstractions available in the JDK too though!

whalesalad• 10 years ago

My favorite blogger of 2016 thus far. Every post is great.

Also the bit that rings home the strongest is just how difficult it is to do trivial things. Even a senior level programmer with ten years of experience has to sit and say, why is this such a pain in the ass?

I guess that's one reason I really like clojure. Core.async has a really hideous API at first glance but using channels and buffers makes doing this sort of thing a lot more enjoyable. Anyway the intent of the article was to dive into abstractions not to suggest more of them so I'll shush.

bogomipz• 10 years ago

Always enjoy reading her posts, I particularly like her enthusiastic style too.

wsargent2• 10 years ago

The easiest way to use thread pools in Scala is to use an ExecutionContext. Akka will let you define an ExecutionContext from a configuration file using a wrapper called a Dispatcher -- you can ramp up the parallelism-max setting for more cores. Then you do actorSystem.dispatchers.lookup("my-dispatcher") and pass it to all the actor based code that you are using.

You can use a work pulling set of actors to handle computation after that, or use Akka Streams to provide backpressure, to avoid the OOM problem.

known• 10 years ago

Good write up.

BogusIKnow2• 10 years ago

As is sometimes the case, people start with the wrong abstraction (thread pool) and then optimize it.

Java/Scala streams look like they would solve most of the problems in the post on their own.

e.g. Java streams

http://radar.oreilly.com/2015/02/java-8-streams-api-and-para...

    private void runParallel() {
      trades
      .stream()
      .parallel().forEach(t->doSomething(t) );
    }

Java/Scala has a hierarchy of concurrency abstractions, from high level to low level:

Streams, Parallel collections, Futures, Actors, Fork/Join, ExecutionContext/ExecutorService, Threadpools, Threads, synchronized, Barriers, Atomic values and many more.

BogusIKnow2• 10 years ago

Something not so obvious for beginners: Java/Scala does not use all the available memory. If you get out of memory problems with the 10gb data set and you have much more RAM available, you need to tell Java/Scala to increase the Heap.

Terr_• 10 years ago

In my case, I was reading a bunch of data off disk. maybe 10GB of data. And I was submitting all of that data into the ExecutorService work queue. Unsurprisingly, the queue exploded and crashed my program.

In the spirit of brainstorming:

1) Abstract out the I/O step so that you can pass around a bunch of lightweight and lazy "I'll get the lines for you when you ask" objects. Then you can safely queue up a large number of them.

2) Choose to break the "job" apart into two parts, and submit the "get lines from filename" jobs into a different, smaller pool, with the results flowing into the larger (CPU-core-limited) pool.

Thread pools: How do I use them?

Related Stories