Concurrency with Python: Threads and Locks

https://bytes.yingw787.com/posts/2019/01/12/concurrency_with_python_threads_and_locks/

by yingw787 • 7 years ago

81 21 7 years ago

bytes.yingw787.com

stendinator• 7 years ago

Global lock much?

nzjrs• 7 years ago

Shallow drive by meme posts are frowned upon on HN

chrisseaton3• 7 years ago

You can use threads, locks, and concurrency usefully even if you have a GIL.

amelius2• 7 years ago

Sadly, that's not practically possible.

Yes, you can use multiple processes to mitigate some of the problems, but you will create other problems (for example: you now have to serialize all the objects through a message pipeline, which is expensive, or you have to use shared memory, which also requires copy operations, unless you want to write your entire code in a low-level kind of way but in that case, you might as well use a different language like C). In an existing system, moving everything to a different process can be a cumbersome experience.

What you really want is multicore support with a garbage collector that is concurrent and a runtime that doesn't have a global lock. Sadly, few environments support this. Erlang comes to mind, but also doesn't support structural sharing. Ocaml is working on multicore support [1]. GoLang, Haskell and Java seem your best choice for GC+multicore.

[1] https://github.com/ocamllabs/ocaml-multicore

securitybear• 7 years ago

What's your point? Please elaborate!

19962• 7 years ago

Having multiple asyncio loops and multiple threads do not mix well.

Too bad, because you want to isolate things sometime.

yingw7872• 7 years ago

Do people do that in practice? I would think since async/await does not reference any particular thread that it is always off of main thread.

sametmax• 7 years ago

It does after 3.5.3 and as long as you pop your loop after the thread has started.

19962• 7 years ago

In theory, yes it works in 3.7+

In practice, no

Can I ask you one example of what you mean? I want 2 separate event loop for 2 separate threads

aaron_m042• 7 years ago

The Counter example has a bug -- it doesn't join the threads before printing Counter.count. this could partially explain the low answer.

yingw7872• 7 years ago

Hi aaron_m04, I have posted a correction to the post and given you credit. Please let me know if I have made any more mistakes, I don't want to mislead any more readers. Thanks very much for your time!

aaron_m042• 7 years ago

Great. Thanks for giving me credit and for updating this. Good work, by the way.

Concurrency with Python: Threads and Locks

Related Stories