gh-116167: Allow disabling the GIL

https://github.com/python/cpython/pull/116338

by freediver • 2 years ago

470 244 2 years ago

helsinki• 2 years ago

While the title is correct, it is a bit misleading, because disabling the GIL breaks the asyncio tests. It's like saying the engine can be removed from my car. Sure, it can, but the car won't work.

TylerE4• 2 years ago

This comment feels very disingenuous because non-threaded programs do in fact work.

ollien2• 2 years ago

I mean, you're not wrong, but also it's a huge feat to provide a toggle for a major feature like the GIL. Though, if it's just asyncio that's broken, perhaps it's not like removing your engine, but rather your antilock brakes :)

EDIT:

[the test synchronous programs] all seem to run fine, and very basic threaded programs work, sometimes

Perhaps this is closer to removing the oil pan

Kranar6• 2 years ago

Well this release will break any code that uses threads. The goal of this particular release is to work for thread-free programs.

ollien2• 2 years ago

How do single-threaded programs benefit from a lack of GIL?

OskarS• 2 years ago

Really, any code? I thought they were adding fine-grained locks to the python objects themselves? Are you saying that if I share a python list between two threads and modify it on one and read it on the other, I can segfault python?

thibaut_barrere• 2 years ago

Couldn’t it work if each threads only touch thread-specific data structures?

neilkk2• 2 years ago

This isn't correct. TFA said that small threaded programs had been run successfully, but that the test suite broke in asyncio.

Async I/O and threads are two different things, and either can be present in real code without the other.

znpy3• 2 years ago

While the title is correct, it is a bit misleading, because disabling the GIL breaks the asyncio tests. It's like saying the engine can be removed from my car. Sure, it can, but the car won't work.

You're not supposed to drive a car that hasn't got out of the research and development laboratory either, so there's that.

petters• 2 years ago

You also need to compile Python with a special flag activated. It’s not only an environment variable or a command line option.

pharrington• 2 years ago

Being able to remove the engine from my car with the push of a button would be a pretty amazing feature!

jagged-chisel• 2 years ago

Analogy breaking down and all, but …

Only as long at it’s as easy to put back in

Retr0id2• 2 years ago

Is there a good overview of the bigger picture here?

gtirloni2• 2 years ago

https://peps.python.org/pep-0703/

dang• 2 years ago

Intent to approve PEP 703: making the GIL optional - https://news.ycombinator.com/item?id=36913328 - July 2023 (499 comments)

rkagerer• 2 years ago

https://realpython.com/python-gil/

protomikron2• 2 years ago

Although this is nice, the problems with the GIL are often blown out of proportion: people stating that you couldn't do efficient (compute-bounded) multi-processing, which was never the case as the `multiprocessing` module works just fine.

liuliu• 2 years ago

`multiprocessing` works fine for serving HTTP requests or do some other subset of embarrassingly-parallel problems.

skrause2• 2 years ago

`multiprocessing` works fine for serving HTTP requests

Not if you use Windows, then it's a mess. I have a suspicion that people who say that the multiprocessing works just fine never had to seriously use Python on Windows.

kroolik2• 2 years ago

Managing processes is more annoying than threads, though. Incl. data passing and so forth.

pillusmany4• 2 years ago

The "ray" library makes running python code on multi core and clusters very easy.

vita7777777• 2 years ago

On the other hand, this particular argument also gets overused. Not all compute-bounded parallel workloads are easily solved by dropping into multiprocessing. When you need to share non-trivial data structures between the processes you may quickly run into un/marshalling issues and inefficiency.

ynik• 2 years ago

multiprocessing only works fine when you're working on problems that don't require 10+ GB of memory per process. Once you have significant memory usage, you really need to find a way to share that memory across multiple CPU cores. For non-trivial data structures partly implemented in C++ (as optimization, because pure python would be too slow), that means messing with allocators and shared memory. Such GIL-workarounds have easily cost our company several man-years of engineer time, and we still have a bunch of embarrassingly parallel stuff that we still cannot parallelize due to GIL and not yet supporting shared memory allocation for that stuff.

Once the Python ecosystem supports either subinterpreters or nogil, we'll happily migrate to those and get rid of our hacky interprocess code.

Subinterpreters with independent GILs, released with 3.12, theoretically solve our problems but practically are not yet usable, as none of Cython/pybind11/nanobind support them yet. In comparison, nogil feels like it'll be easier to support.

pillusmany4• 2 years ago

"Ray" can share python objects memory between processes. It's also much easier to use than multi processing.

ebiester• 2 years ago

And I guess what I don't understand is why people choose Python for these use cases. I am not in the "Rustify" everything camp, but Go + C, Java + JNI, Rust, and C++ all seem like more suitable solutions.

jononor3• 2 years ago

I think that 90 or maybe even 99% of cases has under 1GB of memory per process? At least it has been the case for me the last 15 years.

Of course, getting threads to be actually useful for concurrency (GIL removed) adds another very useful tool to the performance toolkit, so that is great.

jcranmer2• 2 years ago

as the `multiprocessing` module works just fine.

Something that tripped me up when I last did `multiprocessing` was that communication between the processes requires marshaling all the data into a binary format to be unmarshaled on the other side; if you're dealing with 100s of MB of data or more, that can be quite some significant expense.

dvas• 2 years ago

Extra links for the no gil work for anyone else curious about this [0], [1].

[0] Multithreaded Python without the GIL https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsD...

[1] Github repo https://github.com/colesbury/nogil

tentacleuno• 2 years ago

Further context on noGIL in general: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

swtaarrs• 2 years ago

Those links are both fairly old. See PEP 703 [0] and Sam’s nogil-3.12 repo [1] for more current versions.

[0] https://peps.python.org/pep-0703/

[1] https://github.com/colesbury/nogil-3.12

Dowwie• 2 years ago

We are now one step closer to PythonOTP

behnamoh3• 2 years ago

I've been programming in Python for over 6 years now and every week I learn something new. But recently I've been thinking about moving to a more capable language with proper concurrency for backend API requests (FastAPI sucked).

I also want types, so Elixir is not in the picture. I dabbled in Rust a bit. Although I was able to get the hang of things and build a CLI tool pretty quickly, I'm worried I'll have to deal with numerous quirks later if I keep using Rust (like numerous string types). Is that something to be worried about if all I want from Rust is Python+Types+Concurrency?

jondwillis• 2 years ago

Swift or Kotlin might be what you’re looking for but nobody uses Swift for backend really, and I’m not sure about Kotlin.

kuschku• 2 years ago

Kotlin has basically replaced java for many spring shops. It's really common in backend nowadays.

imbusy111• 2 years ago

Sounds like you want Julia. It looks like Python, but also has, what you ask for.

You can even run Python from Julia, so that alleviates the problem with a lack of libraries somewhat.

sparks1970• 2 years ago

Golang? Pretty easy to pick up coming from Python and proper concurrency.

scubbo6• 2 years ago

Python supports types! https://www.mypy-lang.org/

cmeacham98• 2 years ago

This seems a bit like saying "JavaScript supports types!" because of typescript.

rightbyte• 2 years ago

Python+Types+Concurrency

Sounds like Groovy. But I wouldn't recommend it. Also the career padding hype is gone.

willcipriano• 2 years ago

What sucked about FastAPI?

behnamoh3• 2 years ago

Not fast enough! I used it to call llama.cpp server but it would crash if requests were "too fast". Calling the llama.cpp server directly solved the issue.

docc• 2 years ago

i moved from python timo Rust and i really like it. moving to a staticly typed language is bit of a mind fuck but cargo is amazing and so is the speed and portability of rust.

spprashant2• 2 years ago

As someone in a similar boat, just pick Golang. People often dislike the basic syntax, explicit error checking and the lack of algebraic data types. I did. Rust just seems like it offers so much more and you fear missing out on something really cool.

But once you get over it, you realize Golang has a good type system, concurrency model, package manager that's not pip, fast compile times, and static binaries. For most cases it will also offer great performance.

It has everything you need to build APIs, CLI tools, web servers, microservices - pieces which will form the building blocks of your software infrastructure. I have heard numerous stories of people being productive in Go in a few days, sometimes even hours.

If Python is 0 quality of life and Rust is a 100, Golang gets you all they way up to 80-90. That last bit is something you might never need.

Rust is a great language and something I hope to be in proficient someday, but I ll save it for where I actually need that last microsecond of performance.

seabrookmx• 2 years ago

GoLang as others suggested is a good pick. If you're in the "I don't like coloured functions" camp it's likely your best bet for web workloads.

If you're not a fan of GoLang's spartan syntax and are cool with async/await, C#/dotnet core is a great experience on all platforms. IMO it has the best async/await implementation (it originated there) on top of a multi-threaded event loop. ASP.NET is a great web framework and it has great library support for everything else. As someone who avoids "traditional" ORMs (Hybernate, Django) I really like Dapper.

neonsunset2• 2 years ago

Strong recommendation to look into ASP.NET Core with C#. It gives static typing, easy first-class concurrency, it runs on all platforms and now can be AOT compiled if that's your thing, if not - can still produce just a single executable if you choose to. Also its CLI is nice and similar to cargo. Naturally, you won't be having performance issues.

archargelod• 2 years ago

You might wanna try Nim [0].

Nim is a statically-typed compiled language with very pythonic syntax. It's easy to learn, especially if you already know Python, because Nim's stdlib is heavily inspired by it.

For multithreading in Nim see:

Weave - https://github.com/mratsim/weave

Malebolgia - https://github.com/Araq/malebolgia

[0] https://nim-lang.org

agacera• 2 years ago

DevX in Rust for backend development is quite good nowadays. Things are WAY easier than it was a couple of years ago.

I migrated a couple of projects from Java, TS and Go to Rust and honesly, I couldn't be happier.

Hugsun• 2 years ago

It will be very exciting to see how much faster they'll be able to make vanilla python. The value proposition is being challenged by the plethora of tools aiming to alleviate those issues. Speed improvers like Mojo, pytorch, triton, numba, taichi come to mind.

There are so many different attempts at solving this problem that that the last time I wanted to try one of them, I found myself overwhelmed with options. I chose taichi which is pretty fun and easy to use, although somewhat limited in scope.

nerpderp823• 2 years ago

Mojo should be viewed as an attack on the Python ecosystem due to it being a superset. It can consume Python, but it itself is not Python.

Taichi is really underrated, it works across all platforms (including Metal), has tons of examples and the code is easy to write. And lastly, it integrates with the ecosystem and doesn't displace it.

https://github.com/taichi-dev

great demo reel of what Taichi can do, https://www.youtube.com/watch?v=oXRJoQGCYFg

https://www.youtube.com/watch?v=WNh4Q7-OSJs

https://www.taichi-lang.org/

objektif• 2 years ago

Never heard of taichi before looks promising. Do you know any shop that uses it for prod code?

KerrAvon• 2 years ago

I think "attack" is a bit much; C++ isn't an attack on C.

its-summertime2• 2 years ago

Cython is also a superset, is Cython also guilty of such crimes?

boxed• 2 years ago

Mojo is not a superset. Not even close. They say they are AIMING for it to be a superset OF THE SYNTAX. This is a subtle yet enormous difference to being a superset of the language itself.

est• 2 years ago

Mojo should be viewed as an attack on the Python ecosystem due to it being a superset

A superset of pure .py code, not the numpy, cython, ctypes and stuff.

But once you get "superset" of CPython's C bindings, congratulations, you get GIL.

misoukrane• 2 years ago

Finally, looking forward to the benchmarks of many tools!

xuhu• 2 years ago

PEP-703 predicted in June 2023 an overhead of 15% when running with NoGIL: https://discuss.python.org/t/pep-703-making-the-global-inter...

rnmmrnm• 2 years ago

More than seeing it in main, I'm happy for the "python thread slow" meme officially going away now.

scubbo6• 2 years ago

I wish I had your optimism. Thoughtless bandwagon-y "criticism" is extraordinarily persistent.

samatman2• 2 years ago

There's no need to pretend Python has virtues which it lacks. It's not a fast language. It's fast enough for many purposes, sure, but it isn't fast, and this work is unlikely to change that. Faster, sure, and that's great.

bmitc6• 2 years ago

It isn't thoughtless. I'm working in Python after having come from more designed languages, and concurrency in Python is an absolute nightmare. It feels like using a language from the 60s. An effectively single threaded language in 2024! That's really astonishing.

znpy3• 2 years ago

I still hear the "java slow" meme from time to time... Memes are slow to die, sadly. Some people just won't catch on with the fact that java has had just-in-time compilation for like 15 years now (it was one of the first major platforms to get that), has had a fully concurrent garbage collector for a number of releases (zgc since java 11) and can be slimmed down a lot (jlink).

I work on low-latency stuff and we routinely get server-side latencies in the order of single to low double-digit microseconds of latency.

If python ever becomes fully concurrent (python threads being free of any kind of GIL) we'll see the "python slow" meme for a number of years... Also doesn't help that python gets updated very very slowly in the industry (although things are getting better).

RyEgswuCsn• 2 years ago

I feel Java deserves better. When Python finally gets true thread concurrency, JIT (mamba and the like), comprehensive static analysis (type hints), and some sophisticated GC, and better performance, people will realise Java have had them all this time.

PhilipRoman• 2 years ago

I think java being slow has less to do with the implementation (which is pretty good) and more to do with the culture of overengineering (including in the standard library). Everything creates objects (which the JIT cannot fully eliminate, escape analysis is not magic), cache usage is abysmal. Framework writers do their best to defeat the compiler by abusing reflection. And all these abstractions are far from zero cost, which is why even the JDK has to have hardcoded special cases for Streams of primitives and ByteBuffers.

Of course, if you have a simple fastpath you can make it fast in any language with a JIT, latency is also generally not an issue anymore, credit where credit is due - java GCs are light years ahead of everything else.

Regarding jlink - my main complaint is that everything requires java.base which already is 175M. And thats not counting the VM, etc. But I don't actively work with java anymore so please correct me if there is a way to get smaller images.

IshKebab2• 2 years ago

Well, technically it still won't be able to use the full power of threads in many situations because (I assume) it doesn't have shared memory. It'll presumably be like Web Workers / isolates, so Go, C++, Rust, Zig, etc. will still have a fundamental advantage for most applications even ignoring Python's inherent slowness.

Probably the right design though.

Difwif• 2 years ago

Why would you think it's not shared memory? Maybe I'm wrong here but by default Python's existing threading implementation uses shared memory.

AFAIK we're just talking about removing the global interpreter lock. I'm pretty sure the threading library uses system threads. So running without the GIL means actual parallelism across system threads with shared memory access.

agent2812• 2 years ago

I doubt that Python will ditch the meme. The fundamental model of dynamic dispatch using dictionaries on top of a byte code interpreter is pretty slow. I wouldn't expect it to get within 2x of JavaScript.

DinaCoder98• 2 years ago

Javascript may not have an official bytecode, but is it not also based on the same concept of using dictionaries to dispatch code and slow as a result? I certainly had always filed it away as "about as fast as python" in my head. Why else would it rely on evented i/o?

imtringued3• 2 years ago

Ok but given enough cores even python code will run into memory bandwidth problems rather than be bottlenecked by memory latency.

vlovich1233• 2 years ago

Does anyone know why the biased reference counting approach described in https://peps.python.org/pep-0703/ just has a single thread affinity requiring atomic increments/decrements when accessed from a different thread? What I’ve seen other implementations do (e.g. various Rust crates implementing biased reference counting) is that you only increment atomically when moving to a new thread & then that thread does non-atomic increments/decrements until 0 is hit again and then an atomic decrement is done. Is it because it’s being retrofitted into an existing system where you have a single PyObject & can’t exchange to point to a new thread-local object?

colesbury• 2 years ago

We could implement ownership transfer in CPython in the future, but it's a bit trickier. In Rust, "move" to transfer ownership is part of the language, but there isn't an equivalent in C or Python, so it's difficult to determine when to transfer ownership and which thread should be the new owner. We could use heuristics: we might give up or transfer ownership when putting an object in a queue.SimpleQueue, but even there it's hard to know ahead of time which thread will "get" the enqueued object.

I think the performance benefit would also be small. Many objects are only accessed by a single thread, some objects are accessed by many threads, but few objects are exclusively accessed by one thread and then exclusively accessed by a different thread.

vlovich1233• 2 years ago

I think you would do it on first access - “if new thread, increment atomic & exchange for a new object reference that has the local thread id affinity”. That way you don’t care about whether an object actually has thread affinity or not and you solve the “accessed by many threads” piece. But thanks for answering - I figured complexity was the reason a simpler choice was made to start with.

karmasimida• 2 years ago

This is exciting, can't wait

tommiegannert• 2 years ago

First I read the news of tranched bread, and now this?! What a time!

I was a bit disheartened when the Unladen Swallow project [1] fizzled out. Great to see Python back on the core optimization track.

[1] https://en.wikipedia.org/wiki/CPython#Unladen_Swallow

fragmede5• 2 years ago

tranched bread?

KMnO4• 2 years ago

I could be wrong, but I think it’s a clever alternative to the expression “best thing since sliced bread”.

maest• 2 years ago

GP is joking that this is the best thing since sliced (tranched) bread.

oblio• 2 years ago

OP is a subprime mortgage seller.

whalesalad• 2 years ago

A great video tour of the GIL - https://www.youtube.com/watch?v=Obt-vMVdM8s

tiffanyh• 2 years ago

ELI5

I get in concept what the GIL is.

But what's the impact of this change?

Packages will now break, for the hope of better overall performance?

nextaccountic• 2 years ago

If any package depends on the GIL, it will be enabled. Packages won't break

hsbauauvhabzb2• 2 years ago

What packages might depend on the GIL and why would they need it?

dathery• 2 years ago

Previously people basically didn't bother to write multithreaded Python at all due to the GIL. Threads were primarily used when you had multiple pieces of work to do which could end up blocked on independent I/O. Which is common and useful of course, but doesn't help with the performance of CPU-bound Python code.

Even outside of high-intensity CPU work, this can be useful. A problem lately is that a lot of code is written using Python's native asyncio language features. These run single-threaded with async/await to yield execution, much like in NodeJS, and can achieve pretty good throughput even with a single thread (thousands of reqs/second).

However, a big problem is that any time you do _any_ CPU work, you block all other coroutines, which causes all kinds of obscure issues and ruins your reqs/second. For example, you might see random IO timeouts in one coroutine which are actually caused by a totally different coroutine hogging the CPU for a bit. It can be very hard to get observability into why this is happening. asyncio provides a `asyncio.to_thread()` function [1] which can help to take blocking work off the main thread, but because of the GIL it doesn't truly allow the CPU-bound to avoid interfering with other coroutines.

[1] https://docs.python.org/3/library/asyncio-task.html#asyncio....

hsbauauvhabzb2• 2 years ago

Will this also support threads spanning multiple cores, or is that unrelated?

Edit: https://peps.python.org/pep-0703/ suggests it will support multiple cores, unless the current work does not yet achieve that.

fireattack2• 2 years ago

Would GIT affect how Python runs across multiple Python processes?

I'm asking because I encountered a weird phenomenon before.

I use a simple Python lib called "schedule" which is to run some tasks periodically (not precise). And I often run a script multiple times (with different arguments) to monitor something say, every 30 seconds. So they're in three separate Python Interpreter processes.

What I've noticed is that while when I initiated them, they were something like 5 seconds apart, they eventually will end up running in sync. Probably not related to GIL at all, but I guess do no harm to ask.

pgraf• 2 years ago

If anyone wondered GIL = Global Interpreter Lock

purpleidea• 2 years ago

It's neat that this will eventually improve some python code, but at the end of the day, it's still a badly typed language, which will still be slower and less safe than more modern languages.

Learn golang or rust instead. Impressive that they are managing this though!

Cacti• 2 years ago

but Python has my libraries, and go and rust do not.

> 99% of my compute is offloaded to compiled BLAS or CUDA.

types are enough.

and memory and type safety are not terribly relevant for my use cases.

alfalfasprout• 2 years ago

Bad take. One doesn't just migrate massive ecosystems to "go or rust". Python, like it or not, is the lingua franca of ML/AI and science. Worse, as a user of Go/Rust it's very disingenuous to expect the same kind of iteration ability in those languages as you get in Python.

With tools like pyright now + the work on nogil everyone benefits from this using Python.

takeda• 2 years ago

Rust, sure, but typing in golang isn't superior to python's type annotations.

Also Rust became a quite popular tool for python extensions, where you can offload performance to rust and business logic to python.

jen202• 2 years ago

typing in golang isn't superior to python's type annotations.

Yes it is, in every way. When the first job of choosing to use types is choosing which of the five or so type checkers, each deficient in their own way and incapable of dealing with the poor idioms the language encourages to proliferate, you know you're on the wrong path.

AdamJacobMuller3• 2 years ago

Learn golang or rust instead.

Bad take. Learn golang and rust and python.

You should use the language which is suited to the task, sometimes that's golang sometimes that's python and sometimes it's rust.

It's impressive that the python team as a whole continues to improve in such big ways after more than 30 years of development. It's more impressive that the python team managed to navigate 2to3 and come out stronger.

thisismyswamp2• 2 years ago

Learn golang

Having to "if err != nil" every single function call is a big put off - imagine having to "try catch" everything in a language like C#!

neonsunset2• 2 years ago

Python is a bad programming language, but a great scripting one. You use it when you either have to do scripting or need a framework that is only offered in Python (the entirety of ML domain). Otherwise, the only reason to pick it is if you have no choice, you don't want to do a rewrite or you are being held at a gun point.

Instead, pick C#/F#, Kotlin/Clojure or Rust depending on the use case.

gh-116167: Allow disabling the GIL

Related Stories