Intent to approve PEP 703: making the GIL optional

https://discuss.python.org/t/a-steering-council-notice-about-pep-703-making-the-global-interpreter-lock-optional-in-cpython/30474

by pablogsal • 3 years ago

478 497 3 years ago

discuss.python.org

willm• 3 years ago

I'm looking forward to a GIL-less Python. I think the SC's pragmatic approach is the right one.

frfl• 3 years ago

If I remember Guido van Rossum did mention the status of the GIL in one of the Lex Friedman episodes [1] he was on (it's been a while, so I may be misremembering). Surprised to see a big decision like this happen so quickly. Did Meta's announcement play a big role in this [2]?

[1] https://www.youtube.com/watch?v=-DVyjdw4t9I

[2] https://news.ycombinator.com/item?id=36643670

KRAKRISMOTT2• 3 years ago

Removing him as BDFL was probably the best thing to have happened to Python. He never prioritized performance as a top priority, at least not the same way Lua, JavaScript and Java did. Even Ruby has a JIT now.

ambivalence• 3 years ago

You know that people change their minds and now Guido is working as part of a team at Microsoft that is literally called "Faster CPython"?

v3ss0n• 3 years ago

Python had JIT for long, it's called PyPy, One of the most ambitious project ever happened to opensource

yjftsjthsd-h2• 3 years ago

Removing him as BDFL was probably the best thing to have happened to Python. He never prioritized performance as a top priority

That doesn't follow; you're assuming that Python should prioritize performance as a top priority, which is very much not a given. Python has always excelled at being easy to use, being flexible, being a great glue language - but performant? An interpreted, dynamically typed language? That's like making a C interpreter - you can do it, but that doesn't make it a good idea.

miraculixx53• 3 years ago

His leadership & decision arbitration is dearly missing. Now we have decide by committee and as a result we see Python being pushed and pulled in all directions.

shosca• 3 years ago

Anaconda also has committed engineering resources to assist with the no-GIL transition.

miraculixx53• 3 years ago

It sure did. Also the decision was pretty much rushed through, if you follow the latest discussion (which got shut down on the notion that everything has been said and trust the SC and coredevs).

https://discuss.python.org/t/a-fast-free-threading-python/27...

tremon• 3 years ago

I know they say specifically that they don't want a repeat of the Python3 transition scenario, but the approach they're taking now still veers eerily close to that path, at least it looks that way to me.

A lot will depend on the Python community and the distribution channels. I could see the community struggling to adopt it in a timely fashion, or distributions jumping the gun (Ubuntu, Fedora, Anaconda). Maybe it's too early to make hard decisions, but how much control does the SC really have to avoid such a scenario?

trwsxcn• 3 years ago

Yes, it will resemble the 2to3 scenario. Corporations that pledge support will mechanically convert some projects (pestering the actual developers or threaten with forks?), bugs will be ironed out by the actual, unpaid developers over years.

But apparently Python needs some "success" and this makes a good bullet point. Correctness does not really matter in the Python world.

miraculixx53• 3 years ago

Is it success though?

a_nop2• 3 years ago

They kind of burned a breaking major version transition for no good reason with 2-to-3, now they are prefacing a major change with "it won't be like 2-to-3". It sounds like they may be maintaining two operating modes in CPython 3 instead of going forward with another major transition, just because of that history.

jborean93• 3 years ago

They kind of burned a breaking major version transition for no good reason with 2-to-3

The unicode/text changes alone were a pretty good reason. Division producing floats are also a nice change IMO. I don’t want to discount the challenges with the transition but saying there was no good reason isn’t right to me.

gary_0• 3 years ago

They say they want no-gil to be the only build mode 5 years from when it becomes available. That is both too long and too short.

That is too long for Python to have 2 modes. Half a decade is more than enough time for 2 modes to become the status quo. For one thing, think of all the outdated Stackoverflow threads that will be hanging around after that 5 years. I am not optimistic that 5 years won't turn into 10 years of uncertainty and breakage.

But 5 years might be too short for everyone to dredge up all that C code, update it, test it, and call it mature.

I guess we'll see.

toyg2• 3 years ago

5 years might be too short for everyone to dredge up all that C code, update it, test it, and call it mature.

Developers are lazy. Most people will just do nothing until, 5 years from now, some blog post will go "oh, next month we switch noGil as default, good luck!". At that point everyone will scramble, rush out buggy releases, and spend 5 years finding all the problems.

mappu2• 3 years ago

With PEP703 you would compile Python either for multi or single-threading mode. The mode affects the ABI and therefore which C extensions are available. Eventually all C extensions would have an available port to the new ABI.

The chosen solution is similar to how PHP used TSRMLS_ macros in the Zend engine - if threadsafety (ZTS) was #defined, all functions took an extra thread context parameter, breaking ABI.

miraculixx53• 3 years ago

How did it work out for PHP?

no_wizard• 3 years ago

Honestly it didn't. PHP didn't get threads. It did eventually get Fibers though, which I suppose may have been in part influenced by this work

mappu2• 3 years ago

The main benefit of the threadsafe builds was reentrancy support for multithreaded web servers (e.g. IIS / some apache MPMs). They were only slightly slower for single-threaded code.

The new PHP 8 fibers are only coroutines on a single thread, but PHP has had fork/join since 2001 (!), which works pretty well with Linux CoW.

There has been a pthreads extension since about 2012. However keeping a 1:1 pthreads API prevents some optimization possibilities [1], so the new hotness is php-parallel [2], which will transparently copy closed-over variables to a subinterpreter.

1. https://github.com/krakjoe/pthreads/issues/929#issue-4106367...

2. https://www.php.net/manual/en/philosophy.parallel.php

samus10• 3 years ago

All C extensions are available when running without GIL. A challenge for distribution is that all extensions will have to built twice to be compatible with the two Python builds. However, few source code changes are required where the developers don't want to make it compatible with running without GIL. Executing such extensions forces using the GIL for the whole interpreter, which is slower than the GIL-only build.

miraculixx53• 3 years ago

Unpopular opinion: This is a missed opportunity.

What?

Python could have been the one language with a sane multithreading model. Now it risks becoming a second version of Java. I fear this will make it a less attractive programming language, not least because it might lose its beginner friendlyness. For example, without the GIL a lot more care must be put into designing your programs. This can be true even though your own code is single threaded, for example when you use a library that is multithreaded and has callbacks to your code

Why?

Free threading as introduced by PEP 703 is well known to be errorprone, hard to get right and generally advised against, unless you know exactly what you are doing. In other words free threading is for expert (as in very experienced) use only. And Python already has an expert mode - called Cython oer Numba to name just two.

Personally I can see no good will come from bringing free threading to the masses. Yes it addresses a common critique (by many) and need (by very few), but it addresses it in a very risky way (for the vast majority of Python users).

A better alternative

IMHO the far better and still my preferred approach would have been to favor a per-thread GIL with an explicit mode to share particular objects. This would benefit everyone without the risks. It would be consistenly beginner friendly, and above all, offer a safe path to concurrent programming without impacting the whole ecosystem. Heck we could even call it the "Pythonic Threading Model", and it would be seen as a differentiator.

ajkjk2• 3 years ago

Why convince you otherwise? You're the one with the weird opinion, you should be convincing us.

n2d410• 3 years ago

Heads-up, the parent edited their comment to remove that sentence and added some "convincing" instead, so that's why your comment looks out of place.

banthar• 3 years ago

You can pin your JVM process to a single core and will effectively get Python multithreading model.

superjared• 3 years ago

Personally I can see no good will come from bringing free threading to the masses

I’ve often thought that proper threading should be learned by the masses, not to be scared of it. This sort of statement does nothing but spread FUD.

miraculixx53• 3 years ago

How do you suggest we could teach proper threading (what is that?) to the masses?

jiggawatts• 3 years ago

It’s virtually impossible to do free threading safely, especially with large codebases developed by multiple people. This includes tiny Python scripts that pull in a bunch of dependencies.

It’s like saying that C is a safe language, just “get good” at it.

There are safe alternatives such as structured concurrency.

nottorp3• 3 years ago

I’ve often thought that proper threading should be learned by the masses, not to be scared of it.

... or at least it should be public knowledge that anything prefixed with "async" is windows 3.1 multitasking from the 90s...

Why are we getting 64+ core servers to run single threaded cooperative multitasking on them? :)

n2d410• 3 years ago

The GIL does very little to protect unexperienced users. It's still really easy to run into race conditions, for example, if your thread gets scheduled out in any multi-instruction operation (this is more common than you think [1]). In general, Python code still has to be thread-safe; you get the risks without the benefits.

If you don't care about CPU performance, instead of threads you should go for an event-loop approach (see asyncio in Python). As soon as you have threads (on a language-level, not implementation-level), there is some notion of implicit switching, and you run into issues. So, the language you're looking for is JavaScript, which is single-threaded and every context switch is explicit (in form of `await` or `yield`).

[1] https://verdagon.dev/blog/python-data-races

paulddraper8• 3 years ago

this is more common than you think

Like, for example,

  count += 1

---

The GIL primarily protects C code, not Python code.

asvitkine• 3 years ago

I don't think the blog you cite backs up your claim of "it's still really easy to run into race conditions".

The author literally says: "This was actually pretty hard to discover. The first few experiments failed, because Python is pretty smart about when it runs each thread."

But I think the main way the GIL protects inexperienced users is that its presence has the effect that Python code using the threading APIs is very uncommon, since those APIs don't currently provide parallel computation (only parallel IO). So inexperienced users are protected due to 99% of Python programs not using multithreading as a result (and so likely the one they're developing also doesn't), whereas this will presumably change when the limitations imposed by the GIL go away.

paulddraper8• 3 years ago

Python is already Java but worse.

All of the multithreading bugs without any of the multithreading performance.

dpedu2• 3 years ago

That's not quite true. For example, most operations on lists and dicts are thread-safe in the current version of python. You can't say that about languages with true multithreading.

veave• 3 years ago

Free threading as introduced by PEP 703 is well known to be errorprone, hard to get right and generally advised against, unless you know exactly what you are doing.

You should hear about async...

miraculixx53• 3 years ago

Indeed. You should hear my opinions on async (in Python).

;)

Phil_Latio2• 3 years ago

sane multithreading

It's simply not "sane multithreading" not being able to run Python (byte)code concurrently. It's just a huge annoyance.

because it might lose its beginner friendlyness

Python is not beginner friendly. It has one of the worst documentations out there. Also it does things very different compared to other languages (C#, PHP, Java, JS, ...) - I would advise anyone against learning Python as their first language, while Python is my favorite language.

And Python already has an expert mode - called Cython oer Numba to name just two.

CPython is the "official" Python. How is it beneficial to depend on some 3rd party projects? I don't want to use such projects.

As to the PEP itself: It's optional, with the GIL being enabled by the default. So what's the problem...

miraculixx53• 3 years ago

It's simply not "sane multithreading" not being able to run Python (byte)code concurrentltly

Please read my post again. I am advocating a GIL per-thread model with an explicit feature for sharing selected objects. This allows for all cores concurrency, essentially like free threading yet with safeguards. I like to call that a sane way because it builds on decades of research and industry experience of the software engineering community.

As to the PEP itself: It's optional, with the GIL being enabled by the default.

It's not optional if you build tools and libraries that need be able to run with both gil and no-gil.

Galanwe5• 3 years ago

I don't quite get your "unpopular opinion".

First, I don't see how the GIL would have much to do with free multithreading. The GIL should not have much observable logical impact on multithreaded _Python_ code. It should not make it more or less susceptible to race conditions. It's only practical impact should be slowness.

Second, your proposed "one GIL per thread" is pretty much the equivalent of the current state of multiprocessing. In that you fork your current interpreter state in an other thread with it's own GIL and start from there. This has been used for decades already, nothing new there. Sharing can be done through queues or shared memory.

miraculixx53• 3 years ago

Yes, used for decades - and for good reason and benefit.

No, no the same thing. Sharing objects between processes is not easily achieved for various reasons (at least in Python). It would be easier to get it in multithreading with an arena based allocation model where objects live in a shared or non-shared area of memory.

Also it's not my idea. I am just advocating it as the better model for Python to advance to. It would also not take years to implement and the risks are minimized as there is full compatibility with existing code.

nine_k• 3 years ago

Indeed, multiprocessing with pipes / queues and sometimes areas of shared memory is a pretty sane way to do parallel processing, when more than one CPU core works for you at once. It's pretty ergonomic, and it's largely equivalent to Node's workers.

It has nothing to do with multithreading or GIL, though, and happily works without threads and with GIL in place.

TX81Z3• 3 years ago

Queues are a HUGE pain in the arse though.

xerxes901• 3 years ago

a per-thread GIL with an explicit mode to share particular objects

This is like Ruby's Ractors and I haven't really seen that be super successful so far. The "Objects" that need to be shared are things like class definitions, etc.... there are a ton of subtle issues with objects that are being marked as sharable when they should really not be or vice versa.

schneems• 3 years ago

IMHO the far better and still my preferred approach would have been to favor a per-thread GIL with an explicit mode to share particular objects.

You just described Ractors in Ruby, which didn’t turn out great. The setup cost for either freezing or copying memory to the target ractor to guarantee memory safety is often higher than the perf gains of the parallelism.

Not that it can’t work or won’t be improved. But there is a real world case study of what you’re recommending that we can reference without having to guess.

miraculixx53• 3 years ago

Thanks for this. I'll read up on that. Any good links pointers to start?

opportune3• 3 years ago

Wouldn’t that require running an interpreter in each thread? How on earth could that be a “sane multi threading model”?

offer a safe path to concurrent programming

Uh, doesn’t Python support this already? Python has “concurrency” from async and parallelism from multiprocessing. What it doesn’t support is thread-based parallelism. What you’re suggesting (if I understand it) is an implementation of parallelism that is barely different at all from the typical Python parallelism approach of using multiprocessing to achieve parallelism.

dingi• 3 years ago

I’m genuinely curious, What's wrong with Java's model?

mrkeen2• 3 years ago

* It's hard to program correctly using the thread model. And saying that locks are the solution to thread safety issues in Java, is like saying that malloc/free is the solution to memory safety issues in C.

* Threads are also relatively fat. You'd probably think twice about spinning up 1000 of them.

* Futures came in with Java 8 and are lighter weight, but have certain implementation defects - can't cancel them for example. Although there are some instances in which regular threads aren't cancellable. Also, the API isn't terribly popular. I happen to like flatmapping my way around a codebase, but plenty of devs' eyes glaze over when they read that code.

* Fibers are released I think - at least in beta. More like JS's model I believe. They're even lighter weight and get rid of the flatmappy stuff again. I assume it's still up to the programmer to get the locking right.

thayne2• 3 years ago

my preferred approach would have been to favor a per-thread GIL with an explicit mode to share particular objects.

From what I understand, this is how threading worked in perl. But that functionality is now "discouraged".

I do think that would have been a good way to do it. Especially with an emphasis on message passing.

kzrdude8• 3 years ago

I want to agree with you. My hope is that PEP 703 can lead to better implementations of even your safe / structured concurrency model. It will after all still be possible to use, it just won't unfortunately be the only option.

Just like you can today spawn threading.Threads one by one in python, or just map a bunch of function calls over a threadpool using concurrent.futures thread pool.

mepian• 3 years ago

I wonder if there will ever be Python 4, it seems that the core developers want to avoid bumping the major version number ever again after 3 under any circumstances.

miraculixx53• 3 years ago

Yes bc they fear a Py3 to 4 transition would be perceived as a major burden.

I'm afraid we'll soon learn its not the version # that's burdensome, but the real or perceived(!) incompatibility between versions.

I wonder if introducing such a monumental change in a build flag of the minor version is really wise. Certainly its not in line with any interpretation of semantic versioning (to be fair I think the PSF does not claim to use that).

Alifatisk• 3 years ago

No-GIL means better performance at the cost of being less beginner-friendly, right?

Waterluvian2• 3 years ago

Not really. The GIL doesn’t actually make threading easier for a typical developer as they still have to worry about thread safety. You can ignore locks if you know what Python operations are atomic. But that’s incredibly perilous and you really shouldn’t try given that relies on implementation details. Eg. What if you didn’t realize a setter was overridden and setting to a dict-like isn’t atomic anymore?

It’ll make the Python source code much more complex and complicated, which is probably not a big deal, though I’ll say the CPython source is quite brilliant.

It’ll also mean for C library developers that they can’t assume Python opcodes are atomic. But I’m not sure C library developers will really mind too much because they already worry about this kind of stuff.

JamesSwift• 3 years ago

I think multi-threadedness is already an intermediate level concept so maybe its not a big downside. In turn, the ones that understand and need the performance get it.

tedivm• 3 years ago

Only if the user explicitly uses threads. By default people can still approach their code in the exact same way they do. I imagine most users won't think about threads at all, but may relay on frameworks and libraries that take advantage of them under the hood.

qbasic_forever15• 3 years ago

It's only more performance if you're using the threading primitives and spawning new threads, moving work to them to do in parallel, etc.--this isn't something any beginner will consciously be doing. It might actually be slower in regular single process use that 99% of python users use (since the GIL is there for a very good reason and synchronizing access to python's internal state doesn't just happen for free or without some cost somewhere).

Waterluvian2• 3 years ago

I’m glad they’re very conscious about how easily this could turn into a Python 4 debacle.

They’ll have to be intensely careful not to accidentally affect yes-GIL behaviour. All kinds of weird cases are possible if any sort of emulated GIL isn’t exactly like with a GIL.

miraculixx53• 3 years ago

I am sure the intent is good. I am not so sure it is possible to avoid. They already say it could take 5+ years of having gil + nogil exist in parallel.

For any tool builder that means their cost has just doubled for the next five years, at least. Why? Because people will want to use tools in either mode, no matter if it is deemed productive or experimental.

slt20219• 3 years ago

essentially the adoption of No-GIL Python will depend on:

  1. In which version No-GIL will become default option in CPython
  2. When that CPython version will come standard in LTS Linux distro
  3. When all earlier LTS distros will go out of support
  4. When companies switch from outdated to target LTS version of distro

currently quite a lot of companies use python3.6 only because it comes standard with the Ubuntu 14.04.6 which happens to be the oldest LTS version - and companies have habit of migrating from out-of-support LTS version to currently-supported-oldest-LTS

travisjungroth2• 3 years ago

I’ve seen no description of how this won’t be like 2 -> 3 except:

1. We don’t want it to be.

2. We’ll give up quickly if it is.

Those are both important points. But there seems to be an important missing third piece of “and we’ll achieve this by…”.

miraculixx53• 3 years ago

Indeed. Also once that Genie is out putting it back will be next to impossible.

n2d410• 3 years ago

Quoting my comment from elsewhere:

I didn't dive deep on this, but I assume that GIL mode can still run anything, including no-GIL code (it is one of their promises, at least). So, unlike 2->3, there is forwards-compatibility.
It also seems like the latter isn't meant as a replacement (for the moment), but rather as an option.

As long as GIL mode remains compatible with both old and new code, I see very little danger in having a no-GIL mode (besides hogging CPython maintainers' time).

qbasic_forever15• 3 years ago

Agreed--there needs to be discussion and thought about how this impacts library maintainers. How do they tell users their library supports or doesn't support non-GIL mode? Will pypi have new metadata to specify and enforce projectsdependencies support non-GIL mode, or is it just a chaotic free for all where users have to figure that out themselves? How will a library author have one codebase that supports both GIL and non-GIL mode--will they effectively fork the code and maintain two codebases (yuck!) or will there be support for detecting GIL mode? How does this work for C extensions too? There's a ton of work to make this smooth for libraries and I really hope it is being thought through better than the python 2 to 3 story for library authors (which was no story and chaos).

slt20219• 3 years ago

quite a large set of users are using default python installation. There is large number of companies and non-trivial codebases that run on default python that comes with the oldest LTS version of Linux distro.

this can repeat 2->3 because there will be users stuck with older versions of python and library maintainers will have to maintain both versions: with GIL and without GIL (just like PHP extension developers did with thread safe methods)

vasili111• 3 years ago

What are advantages of non-GIL Python?

KMnO4• 3 years ago

GIL is a promise from the internal implementation (eg CPython) of Python that things will happen atomically within the Python interpreter. This means that when multiple threads try to access and modify Python objects at the same time, the GIL ensures that only one thread can execute Python bytecode at any given moment, preventing potential conflicts and ensuring data integrity. However, this comes at the cost of limiting the full utilization of multiple CPU cores for certain CPU-bound tasks.

Non-GIL adds some complexity to the implementation and some risk when writing multithreaded code at the benefit of improving performance.

fbdab103• 3 years ago

At the current implementation, GILless is a performance regression[0], of 5-7%

[0] https://peps.python.org/pep-0703/#performance

slt20219• 3 years ago

But people are not doing cpu bound tasks in native python (performance is a joke), it all comes down to calling a C library that is optimized for compute - what will be the change that GIL brings here?

lostdog• 3 years ago

In GIL Python, you might think you could speed something up by multithreading, but it turns out you can't. The GIL will just run it serially anyways.

No-GIL means it is possible to run things in parallel (without resorting to fancy C extensions).

miraculixx53• 3 years ago

In theory it's easy. In practice you need to design programs very carefully to leverage multiple threads.

The same is true for multiple processes, yet that is much easier to reason about and a lot more likely to get it right.

valyagolev3• 3 years ago

there's a lot of code I wrote (and saw people write) in Python over the years, conscious that noone will ever run it in threads (ofc it's possible, but typically pointless), thus going quite easy on things that wouldn't be thread-safe. this used to quite a comfortable stance. community ended up inventing other ways to share state, other ways to vectorize, other ways to avoid blocking on I/O, that might sometimes be annoying, but evolved to be quite reasonable for Python.

giving up this stance? a lot of code is instantly a legacy, and a lot of it is a legacy people won't even know about before they notice the problems. and for what?

i must say that i have no experience running Python without GIL so my idea of the ways things can be not thread-safe is purely speculative/borrowed from very different languages (that I finally moved on to long ago, thank god). so maybe i'm wrong, i misunderstand the impact, and all this code is just fine

miraculixx53• 3 years ago

Well said. Thanks

valyagolev3• 3 years ago

people in this thread mention that, for some reason, "even with GIL you still have to write thread-safe code", which is an admirable stance, but I don't think many people do it, because their webserver or whatever uses the many single-threaded processes model and they don't want to waste time on that

miraculixx53• 3 years ago

Indeed. And the reason they use multiprocessing is bc they have learned that Python's multithreading is not a good option in cpu bound tasks. The blessing in disguise of course being that multiprocessing is also a shared nothing model, so (mostly) lock free programming is the default. Oth if you have a need for concurrently accessed shared memory/resources and need locks, it comes with an explicit cost. I think that's a good thing.

In the future the default concurrency model will be shared everything free threading, and all hell might break loose. Hopefully not.

ggm2• 3 years ago

Lots of C library code for decades carried man page warnings it was unstable for use in async, re-entrant and recursive contexts. We learned how to cope and incrementally re-entrant safe versions deployed without too much API instability. Maybe time has healed wounds and caused me memory loss of the pain of discovery you'd tripped over them.

String parsing which tokenised in-place. DNS calls which used static buffers. Things which exploited Vax specific stack behaviour.

I think the GIL has been a blessing and a curse.

neilv• 3 years ago

I remember scouring those C runtime docs, for every non-reentrant function. It might be what got me in the habit of checking docs when using some API that I know moderately well, just in case there's some important detail I missed before, or something had changed.

Around that time, doing cross-platform C++, I got an early look at Java, with concurrency built in from the start, along with GC and various other nice features that were easier to use than C++, and I "knew" it was going to be huge. (But who knew that the MIS people would take over Java, when it seemed clearly targeted at non-MIS programmers, and now MIS people are stuck with the C++ syntax and verbosity, after coming from 4GLs, etc.)

Then mainstream programmers picked up Python, which, IIRC, originally was an embeddable extension language, which was why it was simple. And for which the GIL made more sense.

eru2• 3 years ago

I got an early look at Java, with concurrency built in from the start, [...] and I "knew" it was going to be huge.

And Java doesn't even have good (conceptual) support for concurrency. Compared with eg Erlang, Rust or even Haskell.

But Java was still better at it than C or C++ at the time.

PaulDavisThe1st• 3 years ago

I remember scouring those C runtime docs, for every non-reentrant function.

hint:

  % cd /usr/include
  % ag --no-color '[^A-Z]_REENTRANT'

cjfd• 3 years ago

What are 'MIS people'?

otabdeveloper4• 3 years ago

originally was an embeddable extension language

Python was originally a teaching language, to replace BASIC and Pascal for kids and beginners.

(Yes, there once was a time when people took teaching programming seriously.)

weinzierl4• 3 years ago

That Java had concurrency built in from the start is a blessing mostly, but also a bit of a curse. Most of the Java ecosystem is still in the mindset that threads are cheap and firing up a couple more cannot hurt. So we end up with apps that run thousands of threads and this disease is hard to contain.

formerly_proven2• 3 years ago

Python has many similar restrictions even in the language itself, they just tend to be exceptionally poorly documented.

For example, you can't safely use print in a Python signal handler, because the entire I/O system is not reentrant [1], yet the documentation shows doing this in a "how to do it properly" example [2] while explaining that it's horribly unsafe to throw from a signal handler, which is the default for SIGINT. (And also the reason why the threading module doesn't expose PyThreadState_SetAsyncExc). So despite appearances to the contrary (since the Python signal handler is invoked by the VM, it doesn't actually run in a signal handler context; the signal module registers a C handler which simply sets a flag that is checked by the VM every instruction), you should probably only do in a Python signal handler what you would do in C, which is very little. Set a flag or write to a pipe. Don't go around and call library functions.

[1] E.g. https://bugs.python.org/issue24283 [2] https://docs.python.org/3/library/signal.html#note-on-signal...

paulddraper8• 3 years ago

No-GIL mode is optional, and libraries will be marked "no-GIL compatible" and the ecosystem will gradually support more and more of these.

No one's flipping a switch and breaking mountains of sketchy C.

Steltek• 3 years ago

Well someone has to say it so I might as well: Is this going to be another Python 2 -> 3 cat herding exercise again?

I suppose the need to run your app in "no-GIL" mode is less than needing to jump from 2.7 to 3.

miraculixx53• 3 years ago

I hope you are right. I fear it will turn out rather painful (I have lived through 10 years of Java multithreading, I don't miss it)

* update: I have also lived through 10 years of Python 2-3. It was not pretty.

qbasic_forever15• 3 years ago

That's the exact attitude that lead to decades of pain with the 2 to 3 transition and libraries though. There was zero plan for how to help libraries migrate from python 2 to 3, or more importantly how does one library support _both_ python 2 and 3 from one codebase as their users take time to switch their python interpretor. The attitude of "just turn off GIL if you don't need it, just use libraries that support turning it off" means library authors will be asked to provide versions of their library that do and don't support non-GIL mode. That's a big burden to dump on library maintainers.

raverbashing• 3 years ago

marked "no-GIL compatible"

Ugh, Nomenclature is hard

Because non-GIL compatible doesn't mean GIL incompatible

mikepurvis• 3 years ago

Coming from a place of total ignorance, it would be nice if you could do this more incrementally, like have it be in no-gil mode by default, but then have a context manager you can use for gil sections, and have the interpreter bomb out if you try to enter gil-required code while still in no-gil mode.

collinmanderson• 3 years ago

No-GIL mode is optional

Long term the plan is to 100% remove the GIL from python.

From the article:

Long-term, we want no-GIL to be the default, and to remove any vestiges of the GIL (without unnecessarily breaking backward compatibility). We don’t want to wait too long with this, because having two common build modes may be a heavy burden on the community (as, for example, it can double test resources and debugging scenarios), but we can’t rush it either. We think it may take as much as five years to get to this stage.

Under base assumptions it also says:

We want to be very careful with backward compatibility. We do not want another Python 3 situation, so any changes in third-party code needed to accommodate no-GIL builds should just work in with-GIL builds (although backward compatibility with older Python versions will still need to be addressed).

winter_blue• 3 years ago

I wonder why so many library developers even chose to build native libraries on the shaky and poorly-architected foundation that Python is.

Even writing a JVM native JNI library would have allowed to avoid a lot of that pain (and the library would have been useable from Clojure, Kotlin , Scala, JRuby[1], Jython[2], Java, etc) without any painful threading issues.

[1] which I’m aware of having been used in production by companies in the past

[2] which I’m aware has been quite a bit under maintained for the last several years

lanstin• 3 years ago

Because with the GIL it was dead simple to glue a C library into Python, and also was the canonical way to address hot inner loops in Python: rewrite into C. Nothing fancy but a little trial and error with module loading and you get 30% speed ups without being a great C programmer.

I don’t think it’s false to say that the ease of moving hot spots into C is part of the reason Python has been so successful for thirty years.

I hope this works, but I am very sceptical about being able to port code that worked with a locking solution provided for the enthusiast trying out C working without running into concurrency bugs.

gjulianm2• 3 years ago

I think there are several reasons:

- The history is backwards: it isn’t that devs wanted to make a native library and then chose Python, it’s that they chose Python and then they needed a native library.

- Python works very well for scripts and small programs, and decently well for medium-size programs. This makes libraries with concrete purposes very useful and productive. If your library, say, helps devs do some basic calculation with time series, the ability to be used in quick scripts is a big plus.

- The C API is fairly good, and libraries such as pybind11 make it even better. You don’t need a lot of code or boilerplate for an extension.

imtringued7• 3 years ago

Actually, Java virtual threads have a similar problem with JNI. Your virtual thread gets pinned to a core if you call native code. This isn't as bad as the python GIL but it still limits the usefulness of virtual threads if they can't migrate to other cores.

weinzierl4• 3 years ago

I remember these warnings as well from when I started programming C seriously in the 90s. When I first encountered them I was convinced these issues will be resolved in a matter of weeks, maybe a couple of month at worst. Ohh, so little did I know.

zrball10• 3 years ago

I have a hard time seeing the equivalence here. In C the interaction with non thread safe functions is much more direct. Most people are also more cautious when writing in C.

In Python you have whole C modules with global state. Load 10 of them, add the interpreter complexity and soon enough no one knows what is going on any more.

As it is, most developers (including core devs!) don't even bother to check for memory leaks. I don't think they'll run tsan, and if they do, it will be on a small test suite that only covers 10% of the code.

Given the software development practices in Python and especially in the AI space, I'm very pessimistic about this feature.

dale_glass• 3 years ago

That can't stay forever.

We're now up to 128 core CPUs, and even cheap CPUs have 6 cores on them. Restricting things to a single core's performance gets more and more limiting as time passes.

westurner• 3 years ago

While I'm happy to see optional GIL approved and happening,

I also suspect that the GIL has saved us from debugging reentrant and/or dangerously concurrent code for years, and I salute the GIL for forcing us to build Arrow for IPC in Python, in particular.

Someday, URI attributes in CPython docstrings might specify which functions are constant time, non re-entrant, functions.

Reentrancy (computing): https://en.wikipedia.org/wiki/Reentrancy_(computing)

Global Interpreter Lock: https://en.wikipedia.org/wiki/Global_interpreter_lock

pwdisswordfishc• 3 years ago

What is Vax specific stack behaviour?

Systemmanic• 3 years ago

GIL: Global Interpreter Lock.

Good explanation here: https://realpython.com/python-gil/

jmount• 3 years ago

Why would you even want a no-GIL Python? Java and C showed how much more effort it takes to maintain slower thread safe code for no real benefit. Parallelize at the fork level or at the isolated numeric library level.

commonlisp948• 3 years ago

Exactly. I think a lot of the negativity about GIL comes from a misunderstanding about forking processes. If python is being used as a scripting language, and spawning other tools, you're already getting free multi-core.

A similar misunderstanding exists about SQLite and concurrency.. but that's a topic for another time.

AlphaSite5• 3 years ago

Forking had a ton of its own downsides, it’s not a free lunch either, from poor ergonomics to communications overhead it works well for somethings and very poorly for others.

kzrdude8• 3 years ago

Another new feature for 3.12 is per-subinterpreter GIL which is a middle ground. It would offer isolated interpreter threads without gil.

usrbinbash27• 3 years ago

you're already getting free multi-core.

Please explain: In what sense is the overhead of starting actual OS processes, and relying on IPC "free", compared to running threads or even greenlets, and using shared process memory?

soulbadguy5• 3 years ago

If python is being used as a scripting language

And when it's not ?

qbasic_forever15• 3 years ago

Yep I think over the next year a lot of python devs are going to learn threading isn't magic pixie dust that makes your code fast, and in reality is starts by making your code very unstable.

miraculixx53• 3 years ago

I'm afraid that's exactly what will happen. Unfortunately the overarching sentiment will not be "multithreading is hard" but "Python has become really hard to work with"

IshKebab• 3 years ago

Well, multithreading in C or Java makes sense because it's the only way to increase performance if you've already optimised the single threaded case.

In Python there's a much better option if you care about performance - use a different language!

r0l13• 3 years ago

We are working with a huge Go and Python codebase and Python is just a pain in terms of using all system resources. We moved many parts to C++ which are called and handled by goroutines. The outcome was a big success. This proposal/change is a big step forward, especially for the deep learning community.

csmpltn7• 3 years ago

"Python is just a pain in terms of using all system resources"

Whatever this comment means (I honestly can't properly tell) - removing the GIL will have absolutely no impact on Python's resource utilization.

JonChesterfield5• 3 years ago

Python gets an absolute kicking for being too slow relative to basically everything else and that performance characteristic is partially attributed to the interpreter lock. Misattributed in my opinion, but there we are.

usrbinbash27• 3 years ago

Parallelize at the fork level

IPC is a PITA, and orchestrating processes is even worse.

or at the isolated numeric library level

Not everything I want to parallelise in python runs in numpy. Simple example: WebService Backends. I have a 64 core server running a Werkzeug/Gunicorn application. The Service is mostly doing CPU bound tasks (data aggregation and analysis), so asyncio is pointless.

What happens is, it runs 60 worker processes. Which puts hefty limitations on any crosstalk and data sharing, because these either require IPC, or using redis/sql. Which are nowhere near as performant as actually shared memory would be.

seanthemon• 3 years ago

Exactly my thinking, using more cores is exactly your use case and will hopefully make Python superb at large scale data processing with cross-communication.

I think, rightfully, the concern is people who will try to use this incorrectly causing major bloat to CPython.

miraculixx53• 3 years ago

You would be well served with arena based (shared) object allocation and a gil per thread model. No need to draw everyone else into this.

kerkeslager6• 3 years ago

What an ignorant comment. No real benefit? Really?

The website you're using likely benefits immensely from thread-safe code. The browser you're using benefits immensely from thread-safe code. Your entire user experience using the modern internet is in an ecosystem of thread-safe code which you apparently are completely oblivious to.

There are a lot of threading models besides Java and C's as well, which you're apparently also unaware of.

Why would you assume you know all the possible use cases of a multi-purpose programming language?

andrewstuart• 3 years ago

As a Python developer, what would be the benefit of no GIL?

qbasic_forever15• 3 years ago

Slightly faster performance when you write multithreading code correctly (much easier said than done). Very few python devs are actually running into this as a bottleneck day to day.

slt20219• 3 years ago

Naiive question: Who needs No-GIL when we have asyncio and multiprocessing packages ?

never ever had a problem with GIL in python, always found a workaround just by spinning up ThreadPool or ProcessPool, and used async libraries when needed.

is there any use case of No-GIL which is not solved by multiprocessing ?

I thought Single threaded execution without overhead for concurrency primitives is the best way to high performance computing (as demonstrated by LMAX Disruptor)

dragonwriter3• 3 years ago

is there any use case of No-GIL which is not solved by multiprocessing ?

Anything that benefits from both parallelism and replacing IPC overhead with shared data between parallel tasks.

slt20219• 3 years ago

but it would incur overhead of concurrency control: mutex, locks, semaphores.

I dont believe python will ever have atomic operations, even if it had - they still incur significant overhead for concurrency control.

sharing state between threads is such a narow niche use case, this pattern is practically solved by memcached/redis for larger scale python based systems

Galanwe5• 3 years ago

Not sure why this is downvoted, I never had much issues with the GIL as well.

Multiprocessing does the parallel computation pretty well as long as the granularity is not too small. When smaller chunks are needed most of the time that's something better done from an extension.

miraculixx53• 3 years ago

Some of the use cases advocating for nogil come from the AI/ML group of library builders, stating a need for free threading concurrency.

pama• 3 years ago

Agreed. Feeding the GPUs with multiple forked memory-hogging processes is no fun and leads to annoying hacks. And, yes, as per your other post, there could have been other solutions to this problem, some of which might have been better.

n2d410• 3 years ago

It's only about performance. asyncio is still inherently single-threaded, and hence also single core. multiprocessing is multi-core and hence better for performance, but each process is relatively heavy and there's additional overhead to shared memory. GIL multi-threading is both single-core and difficult to use correctly.

No-GIL multi-threading is multi-core, though difficult to use. I don't know the Python implementation but shared memory should be faster than using multiprocessing.

That said, when designing a system from scratch, I completely agree with you that for almost almost almost all Python use cases, threads should never be touched and asyncio/multiprocessing is the way to go instead. Most Python programs that need fast multi-threading instead should not have been written in Python. Still, we're here now and people did write CPU-intensive code in Python for one reason or another, so no-GIL is practical.

In these threads, I also always see a lot of people who simply aren't aware of asyncio/multiprocessing. I assume these are also a significant share of people asking for no-GIL, though probably not the ones pushing the change in the committee.

slt20219• 3 years ago

I would argue that if you have large concurrency and shared complex state - you better off use kafka and redis/memcached as a shared state - and design proper fan-out.

This design scales much better for systems that will eventually overgrow one big machine. the No-GIL pytohn will be of no use, when you need to deploy your app across 100s machines.

I understand people want to take advantage of all cores etc, but at large scale - you will eventually need to split computation across machines and will resort to in-memory cache/queue anyways - so better just architect your system since day0

KeplerBoy2• 3 years ago

Even with multithreading Python will still be slow. It's not like singlethreaded Python can keep up with singlethreaded C or even JS.

This will just allow people to waste even more compute resources to get somewhat quick results from Python instead of doing the right thing.

From an ecological viewpoint this PEP will be disastrous and keep entire power plants online.

geekraver2• 3 years ago

A free-threaded Python will be harder to make faster for single-threaded cases. So this could be a win for those who want to write multi-threaded code at the expense of everyone else.

chippiewill• 3 years ago

asyncio is still inherently single-threaded, and hence also single core.

IIRC some of the proposals around removing the GIL in the past have actually suggested that the asyncio paradigm could become multithreaded for parallelism.

spamizbad• 3 years ago

This probably isn't going to be that groundbreaking for your average web application. But for several of the niches where Python has a large footprint (AI, Data Science), being able to spin up a pile of cpu/gpu-bound threads and let them rip is a huge boon.

d0mine2• 3 years ago

how likely, the corresponding code doesn't release GIL already? Pure Python is 100x slower than native code therefore the number crunching itself happens in C extensions where GIL can be released.

TX81Z3• 3 years ago

Multiprocessing has a lot of issues, one of which is handling processes that never complete, subprocesses that crash and don’t return, a subprocesses that needs to spawn another subprocesses, etc.

miraculixx53• 3 years ago

Threads can end in limbo too. At least with multiprocessing you get to kill those that hang. Threads, not so much.

opportune3• 3 years ago

The problem with only relying on asyncio and multiprocessing is that they only implement per-process concurrency and parallelization per-process.

Threads let you use the same unified abstraction for parallelization and concurrency. They also make it easier to share state with parallelization (no need to go out of your way to do it) at the cost of requiring you to think about and implement thread safety when you do so.

Also, with no-GIL + threads the computational costs of creating and maintaining a parallel execution is much less vs multiprocessing. And data sharing and synchronization are less expensive.

What LMAX is doing is really just an overhyped way to speed up producer-consumer models. It might apply to your use case but it’s not the only reason you’d use parallelism or concurrency. I don’t even understand why they are claiming it to be an innovation when it’s just using a LockFreeQueue implementation within a pre-allocated arena? You also can’t synchronize with their implementation, which sometimes you really need to do. Not a silver bullet

slt20219• 3 years ago

multithreading with shared state introduces several limitations:

  1. random jumps in memory and branch misses
  2. L1/L2 cache flush
  3. context switch cost
  4. concurrency locks cost

my understanding is that LMAX eliminated these costs:

  1. pre-allocated arena ensures cache locality of operations
  2. we dont jump form one sector of memory into another. Algorithm more resembles linear scanning of working memory set, and mostly within L1/L2 cache
  3. no context switches, no cache flushes
  4. no concurrency control costs

influx• 3 years ago

AsyncIO is great for IO bound applications, not so much for CPU bound...

usrbinbash27• 3 years ago

Naiive question: Who needs No-GIL when we have asyncio and multiprocessing packages ?

1. Because asyncio is completely useless when the problem is CPU bound, as the event loop still runs only on a single core. As the name implies, it is really only helpful when problems are IO bound.

2. Because sharing data between multiple processes is a giant PITA. Controlling data AND orchestrating processes is an even bigger pain.

3. Processes are expensive, and due to the aforementioned pain of sharing data, greenlets are not really a viable solutions.

samsquire3• 3 years ago

LMAX Disruptor is multithreaded.

Multithreading is more efficient but more difficult to work with.

You share the same address space in threads, so you can communicate any amount of data between threads instantly within a lock. The same cannot be said for network traffic or OS pipes or multiprocessing.

Multiprocessing uses pickle to serialize your data and deserialize it in the other python interpreter.

If you start a Python Thread, you're still single threaded due to the GIL.

kzrdude8• 3 years ago

GIL blocks parallelism for ThreadPool.

But I have the same question as you have if we add another coming concurrency model: SubinterpreterThreadPool, which will be possible with the per-interpreter GIL in python 3.12 and later.

That's another new model that is already confirmed to be coming: interpreters (isolated from each other) in the same process, that can run with each their own GIL.

lucidyan• 3 years ago

Are we talking about "PEP 554 – Multiple Interpreters in the Stdlib"[1] proposal?

[1] https://peps.python.org/pep-0554/

_flux• 3 years ago

I haven't use Python multiprocessing packages, so I need to ask, how does one do flexible work queues with them?

I mean a situation like where in threaded context there would be code like:

    def determine_quest_latency(quest_name: str) -> int:
        return other_thread.wait_sync_job(lambda context: context.ping_quest(quest_name))

..without needing to provide a protocol that covers each possible scenario the client might wish to execute in the process?

I believe the answer is "you don't", but passing functions in messages is a highly convenient way to structure code in a way that local decisions can stay local, instead of being interspersed around the codebase.

xmcqdpt26• 3 years ago

I believe you just pass objects instead, like you would in OOP, and take the hit of pickling and unpickling them every time.

If you really want to pass lambdas, you can use a third party library to pickle them

https://github.com/cloudpipe/cloudpickle

Yes, this is not great.

gjulianm2• 3 years ago

is there any use case of No-GIL which is not solved by multiprocessing ?

Tons:

- A web server that responds using shared state to multiple clients at the same time.

- multiprocessing uses pickle to send and receive data, which is a massive overhead in terms of performance. Let’s say you want to do some parallel computations on a data structure, and said structure is 1GB in memory. Multiprocessing won’t be able to deal with it with good performance.

- Another consequence of using pickle is that you can’t share all types of objects. To make matters worse, errors due to unpickable objects are really hard to debug in any non-trivial data structure. This means that sharing certain objects (specially those created by native libraries) can be impossible.

- Any processing where state needs to be shared during process execution is really hard to do with the multiprocessing module. For example, the Prometheus exporter for Flask, which only outputs some basic stats for response times/paths/etc, needs a weird hack with a temporary directory if you want to collect stats for all processes.

I could go on, honestly. But the GIL is a massive problem when trying to do parallelism in Python.

samus10• 3 years ago

PEP-703 contains a whore Motivation section. Long enough to require a summary:

Python’s global interpreter lock makes it difficult to use modern multi-core CPUs efficiently for many scientific and numeric computing applications. Heinrich Kuttler, Manuel Kroiss, and Paweł Jurgielewicz found that multi-threaded implementations in Python did not scale well for their tasks and that using multiple processes was not a suitable alternative.
The scaling bottlenecks are not solely in core numeric tasks. Both Zachary DeVito and Paweł Jurgielewicz described challenges with coordination and communication in Python.
Olivier Grisel, Ralf Gommers, and Zachary DeVito described how current workarounds for the GIL are “complex to maintain” and cause “lower developer productivity.” The GIL makes it more difficult to develop and maintain scientific and numeric computing libraries as well leading to library designs that are more difficult to use.

chx• 3 years ago

Your comment has been nominated for the best typo in 2023.

Let's hope the change is not this badly mercenary.

oars• 3 years ago

From PEP 703:

Manuel Kroiss, software engineer at DeepMind on the reinforcement learning team, describes how the bottlenecks posed by the GIL lead to rewriting Python codebases in C++, making the code less accessible:
"We frequently battle issues with the Python GIL at DeepMind. In many of our applications, we would like to run on the order of 50-100 threads per process. However, we often see that even with fewer than 10 threads the GIL becomes the bottleneck. To work around this problem, we sometimes use subprocesses, but in many cases the inter-process communication becomes too big of an overhead. To deal with the GIL, we usually end up translating large parts of our Python codebase into C++. This is undesirable because it makes the code less accessible to researchers."

For average usage like web apps, no-GIL can be solved by multiprocessing. But for AI workloads at huge scale like Google and DeepMind, the GIL really does limit their usage of Python (hence the need to translate to C++). This is also why Meta are willing to commit three engineer-years to making this happen: https://news.ycombinator.com/item?id=36643670

umanwizard4• 3 years ago

So Python is being fundamentally changed for everyone because of the needs of a niche subset of Python programmers (AI researchers), because that niche subset refuses to learn a language more suited to their task?

cutler3• 3 years ago

I never really understood Meta/Facebook's practice of relying on scripting languages. Ok, replacing PHP might not have been an option given the accelerated growth of Facebook but Python was only used for tooling originally, as I understand. If they needed threading and performance so badly why didn't they go for a compilted, statically-typed language?

the84722• 3 years ago

I thought Single threaded execution without overhead for concurrency primitives is the best way to high performance computing

You can have shared-memory parallelism with near-zero synchronization overhead. Rust's rayon is an example. Take a big vector, chunk it into a few blocks, distribute the blocks across threads, let them work on it and then merge the results. Since the chunks are independent you don't need to lock accesses. The only cost you're paying is sending tasks across work queues. But that's still much cheaper than spawning a new process.

aflag• 3 years ago

When you create a new process you can't share things like network connections. Also, IPC tends to be very slow. It is abstracted away nicely in python, but it's still very slow, making some parallelism opportunities impossible to exploit.

For creating stateless, mostly IO bound, servers, it's great. Try to squeeze in performance and it all starts to fall apart.

nnx• 3 years ago

I hope this won't make Python's dependency hell even worse, but I'm not hopeful.

phkahler• 3 years ago

It's optional. Just keep using the GIL.

qbasic_forever15• 3 years ago

Yeah it's going to be weird for some years where some libraries support no-GIL and others don't, while folks cry about the ones that don't support it holding them back.

Like asyncio's introduction we'll probably see core stuff like http requests, file IO etc. all now have an entirely new permutation of libraries made to support non-GIL mode. This is going to get pretty spicy as stuff like http already has regular (blocking IO) and asyncio (non blocking IO) versions, so now do they need regular non-GIL and asyncio non-GIL versions too? Is the default for a library author going forward to be creating four permutations of your library with vastly different behavior in each of them? Yuck.

miraculixx53• 3 years ago

import this

;)

ies7• 3 years ago

while folks cry about the ones that don't support it holding them back.

And someone may prefer to make NON GIL wall of shame/fame instead of directly contributing to those libraries

sheepscreek• 3 years ago

I do most of my performance coding in Numba which comes with a nogil mode. Still, I have been looking forward to this. The fewer layers we can have in our libraries, the better!

bjourne• 3 years ago

This can (and I think will) cause issues for C extensions because many are written without multi-threading in mind. Here is a small example which is unsafe if lst can be accessed from another thread: https://news.ycombinator.com/item?id=36649769 Note that the code may cause a context switch even today if the C code callbacks into Python bytecode (via a __del__ method) and the bytecode is long enough (100 instructions I think). However, that is extremely unlikely and much C extension code is not written with such situations in mind.

People using C extensions may also rely on them executing atomically. For example, you could have a thread pool that posts and receives from a numpy array. Would work fine today but break without the GIL.

qbasic_forever15• 3 years ago

Yep there are a ton of issues like that to be found, and unfortunately they will manifest as difficult to find and debug race conditions. This is why the proposal and work is to make non-GIL mode entirely optional and not the default.

It just means for the brave few that flip it on and use it, be prepared to spend a huge amount of time finding and fixing subtle race conditions in decades of old python library code. The early adopters are going to be in for a lot of pain, or more likely they'll restrict their use of non-GIL processes to very specialized and dedicated processes that have as few dependencies as possible.

miraculixx53• 3 years ago

The intent is to make no GIL the default eventually.

hinkley2• 3 years ago

Does Python have a lot of secondary dependencies? I could see someone pulling in two dependencies, not realize they both use the same unsafe library, and end up having them step all over each other.

smcl2• 3 years ago

I think the CPython core devs are very keenly aware of these issues. Otherwise they'd have announced a plan to suddenly rip out the GIL altogether, rather than a phased approach that allows people to opt-in to the no-GIL mode.

jillesvangurp• 3 years ago

That's fine. We go from suspecting there are issues to knowing exactly where the issues are. The rest is just chipping away at that list and making the issues go away. Either by adding some kind of mutex around the code or by replacing the native code with something less likely to have issues.

The argument against this seems mainly that it's a lot of work; not that it's impossible work. It probably is a lot of work but if there are enough people doing the work, we should get some results.

mkoubaa• 3 years ago

Might be a good time to write concurrency first C extension to compete with the legacy ones

xmcqdpt26• 3 years ago

Note that the code may cause a context switch even today if the C code callbacks into Python bytecode (via a __del__ method) and the bytecode is long enough (100 instructions I think). However, that is extremely unlikely and much C extension code is not written with such situations in mind.

As someone who works professionally on a parallel / async runtime that supports thousands of continuously running servers, "extremely unlikely" means that actually it breaks all the time, but it's also impossible to debug.

amluto• 3 years ago

I’m not in love with some of the details.

PYTHONGIL is an awkward tri-state. 0, 1, and unset all do different things. Wouldn’t some self-explanatory strings be better? PYTHONGIL=auto for the default, force-gil and force-nogil for the forced modes.

catnibbler2• 3 years ago

Is it really too late to not do this ? The only reason to get rid of the GIL is to help threading, but that's not a thing we should be doing. Threads need to just die, and be replaced by something less idiotic. Seriously, having the CPU run fragments of your program at random, so that all the previously ordered pieces are now contending with each other and even themselves ? How can anyone not see that this is the stupidest idea in the world ?

Jabrov• 3 years ago

As opposed to?

n2d410• 3 years ago

In Python, asyncio and multiprocessing packages can get nearly the same or better performance for IO- and CPU-intensive tasks respectively as no-GIL multithreading (and are more performant than GIL multithreading), with only a tiny fraction of the pitfalls. For any use case where the last few percent matter, consider not using Python (which will be much much more significant).

Regardless, we did somehow end up here, and there's plenty of multi-threaded Python code that would benefit from no-GIL, so I support the proposal just from a practical perspective. But when designing a new codebase, you'll almost almost almost always want to avoid Python threads, even with no-GIL.

catnibbler2• 3 years ago

Keep the GIL and avoid the problems its removal will cause. Allow parts of your program to run in a separate namespace with explicit passing of objects. No sharing means no contention, so no overhead.

miraculixx53• 3 years ago

I was called inconsiderate for essentially asking this very question. I still think it's the right question to ask:

Why should Python even have a free threading model?

AlphaSite5• 3 years ago

Python already has threads, they just have huge downsides in their current form, so this ship has long since sailed.

nottorp3• 3 years ago

It's a workaround for not having 1200 Ghz CPUs but instead 128 cores at 3 Ghz...

usrbinbash27• 3 years ago

Threads need to just die, and be replaced by something less idiotic.

Please tell us: What other solution do you propose for running CPU bound workloads in parallel?

There are exactly 2: Multiprocessing and using another language.

TX81Z3• 3 years ago

But how will I waste thousands of person hours battling subprocesses now????

carabiner• 3 years ago

We are so back.

kalb_almas• 3 years ago

Even with improved support for parallelism, what role will Python have in the future if Mojo makes good on even half of its promises?

nologic01• 3 years ago

Mojo is not Python.

The underlying pressure on the Python ecosystem is to transition to a post-Moore's law era and effectively become a HPC platform where the "same" code runs on a CPU, a GPU, multicore, clusters etc.

Python may feel the pressure more than others because of the GIL and the fact it is used in compute intensive tasks more than others.

But this major need to transition to easy and seamless HPC/heterogeneous computing is the same for all languages. The question is who will get there first.

Too2• 3 years ago

Mojo is a superset of python, with goal to be able to run any python code and to import any python module under its own execution model, that runs magnitudes faster. Now, if that actually works as advertised, it would render python obsolete and become mojo instead, even if the authors don’t put it that way.

Lex Fridmans podcast had an interview with one of the creators recently. Chris Lattner, who is also the creator of LLVM and Swift. Recommended listen for those who haven’t heard of mojo yet, if you have 3h to spare.

ahgamut• 3 years ago

"Python 4, but not really", because we want to squeeze out more multithreading performance and be cool again. Some questions from reading the OP:

- How much does performance improve due to this No-GIL thing? Is it greater than 2x? For what workloads? - Do I have to compile two versions of every extension (gil/nogil)? I would prefer building extensions does not get any more complicated. - Can I automatically update my code to handle nogil? (a tool like lib2to3 or six)

donio• 3 years ago

The estimate in the PEP is that it will be 5-8% slower. Having to use more granular locks and atomic operations has a cost.

https://peps.python.org/pep-0703/#performance

ram_rar• 3 years ago

This seems somewhat delayed, and it may be considered too little, too late. Python community had the chance to leapfrog and embrace alternative concurrency abstractions, such as go routines etc, but it appears that this opportunity was not fully utilized.

After enduring the arduous process of migrating from Python 2 -> 3 and navigating the complex world of dependencies, my hope is that we won't encounter another nightmare of dependency management, forcing users to choose between GIL and no-GIL builds.

tgv5• 3 years ago

Something akin to go routines won't solve the C-library problem.

raminf• 3 years ago

Remember the transition of text to Unicode? 32 to 64-bit? Intel to ARM? Y2K?

No-GIL is a much smaller shift. It can follow the same transition path without radically breaking things. And if some things do break, there would be a well-defined way to handle those cases.

We all somehow survived those. Glad to see forward motion on this. It will open up a lot more terrain that has been marked off as untenable.

One of the things about early Swift that they got right was building breaking changes into the promise. Everyone knew where they stood and adjusted just fine. Sometimes I wish Python would take the same path.

viraptor• 3 years ago

I think that's a bit different. 32 to 64 - you could test whether it works. Same for arm. Same for y2k. Sure, maybe the testing wouldn't cover the failing case, but the testing you did would be deterministic. But here? Test all you want and the answer is: it's either correct or you haven't triggered the right race yet.

samus10• 3 years ago

Yes, it is different because nobody would be forced to run the interpreter without the GIL. Applications where race conditions are unacceptable can keep using the GIL build. Even the `--disable-gil` build can be forced to keep using the GIL.

geewee• 3 years ago

I mean going from text to unicode did pose a huge problem for python specifically.

asah• 3 years ago

I have PTSD from that transition.

umanwizard4• 3 years ago

I’m not worried about the difficulty of migration; I’m worried that the end state might actually be worse than what we have now.

pyeri• 3 years ago

Let us wait and watch but I somehow feel that this no-GIL mode is just a band-aid solution to Python's performance problem. The cause goes deep inside the core of Python, it gradually came to this stage as more and more features got added to the language since the 3.x transition.

I think new language features shouldn't just be added to provide syntactic sugars or coding shortcuts to programmers or just because a certain feature has become very cool (like lambda functions, for eg).

I'm glad that the Python community has realized that performance is an issue and started working on things like no-GIL mode.

People often say that Python's biggest strength is its readability and easy syntax but I disagree. Python's real strength is the enormous third party library ecosystem, popular packages like numpy, pandas, scikit, etc. which have almost become addictive in most data science projects. But now, people are thinking of other alternatives to these due to Python's performance issues. Other ecosystems like golang and rust are getting built at rapid pace and at some point, they will also have (more performant) equivalents of these packages if public shows enough interest.

ehsankia• 3 years ago

Hasn't python been getting much faster since 3.x? Where is your evidence that new py3 features is making python slower?

MrYellowP• 3 years ago

I'm going to miss the thread-safety non-GIL python offered.

That being said, this is interesting.

Can we get an in-python fork() mechanic ... please?

pritambaral4• 3 years ago

I'm going to miss the thread-safety non-GIL python offered.

The old method — which is the GIL, or non-non-GIL — provides no thread safety to Python code. It only protects C code

Can we get an in-python fork() mechanic ... please?

You probably want Multiple Subinterpreters: https://github.com/python/cpython/issues/84692

m3kw9• 3 years ago

What does this mean?

hgs3• 3 years ago

Python uses a GIL (global interpreter lock) which prevents both Python code and native C modules from executing in parallel in the interpreter. Removing the GIL means Python could provide in-process parallelism.

dwaite• 3 years ago

I sure hope they decide to call execution of mixed GIL and no-GIL codebases "amphibious mode"

worik• 3 years ago

I am not a python programmer.

What is the use case for this?

Who needs it?

12_throw_away• 3 years ago

I am not a python programmer

I _am_ a python programmer.

What is the use case for this?
Who needs it?

This can be answered with a simple user story:

As "THE PYTHON STEERING COUNCIL",

I want to "GET RID OF THE GIL",

so that "PEOPLE WILL STOP WHINGING ABOUT THE GIL."

To be fair, there's another group of people who stand to benefit. Namely, any python programmers who currently believe that "threads are an easy way to make my program go faster" will soon be the recipients of a valuable learning experience.

miraculixx53• 3 years ago

Nice!

;)

kgeist• 3 years ago

Is the following possible?

- library author marks their library "no-GIL" after making sure it's thread-safe without GIL

- if the interpreter sees this metainformation, it temporarily disables GIL for the current OS thread while running the library's code

- result: old versions of Python can still run no-GIL libraries under GIL, while new versions of Python allow to gradually remove GIL

Or it's not how CPython works?

BGINBarbarian• 3 years ago

Afaik, nogil will be a compile flag, which means that when there are two builds, you separately compile Gil and nogil builds. They will be two separate programs/binaries/packages. It could be possible for something like conda to install both binaries, then run your program with the one that matches the library flags, but python itself could not do this (afaik).

dotnet00• 3 years ago

I'm looking forward to this, Python plays a fairly significant role in our scientific computing code and not having to have entirely separate processes will be very convenient for cutting down data duplication.

akdor1154• 3 years ago

Get exc's happening!ited, it

lraxny• 3 years ago

The ruling class in python-dev are populists who are not threading experts. Python is run by the wrong people.

They will approve something if it serves a corporation. The submission here is likely CYA, so they can say that "they asked the community".

There is no appreciation for people doing grassroots open source software. If Instagram can add another hack instead of switching to Java, it will be approved.

It is important to remember that paid corporate developers will have job security every time new pain is introduced in the Python ecosystem.

miraculixx53• 3 years ago

I have the same impression though I think its not intentional or otherwise ill intended.

hanselot2• 3 years ago

for self in selfs:

JonChesterfield5• 3 years ago

Exciting. Python is mostly written as C shared libraries that knew they had a global lock to rely on.

Some of those do sufficiently simple things that they can run without any locking and all will be fine.

Others will still need locking, but are now under pressure to run without the gil. Some of those are going to do DIY locking within their own bounds.

Maybe what python has really been missing all these years is loads of ad hoc mutex calls scattered across the ecosystem. Data races and deadlocks introduced in the name of performance is not how I expected python to go out.

edit: expanding on this pessimism a bit.

Making C libraries written assuming a global lock thread safe is the sort of thing I'd expect concurrency experts to advise against and then make mistakes while implementing it. My working theory is that most people who wrote C extensions for python are not concurrency experts and are great programmers who won't back down from a challenge.

The data-race/hang/segfault consequences of this combination look totally inevitable to me. Python application developers are not going to love the new experience and I'm thankful my products are not built on top of the python ecosystem.

tgv5• 3 years ago

I think you're right. Making it an explicit opt-out, as is planned for the first stage, should be fine. Expecting to make it opt-in in 5 years seems too optimistic to me. It relies on all the library developers to fix their libraries (also the Python ones). That's tough work, and importantly, if done well, it will even go unappreciated: nobody will notice it.

Many libraries have never had a multi-processing use case, others are so big that bugs are bound to happen, many of them subtle, so one guaranteed outcome will be unreproducible complaints and devs throwing in the towel. Opt-in will make people unhappy.

Arbortheus• 3 years ago

Surely if it goes well people will see their existing python codebases become more performant with no development required aside from updating some dependencies? Not nobody will notice it.

It seems like it could be a great outcome for developers.

xmcqdpt26• 3 years ago

Most C libraries that are often called from python also have bindings for other languages and so at this point should be threadsafe, right? Also even GIL python supports threads so all these libraries will at least be reentrant safe already.

For libraries that are reentrant but not thread safe, it should be sufficient to just add a global lock wrapping every call, which is pretty close to what the GIL was doing anyway.

It seems to me that in many (most?) cases it should be relatively straightforward to make existing libraries work without the GIL (at the cost of parallelism). I guess the main issue will be for libraries that call back into the python runtime from the C side.

elisbce• 3 years ago

Seems these guys never learn from the Python 2 -> 3 mess. GIL to NoGIL is even worse. Most changes from Python 2 to 3 are syntactic, but GIL to NoGIL could require a complete redesign and rewrite. Either it takes another 10+ years migrating OR nobody with a realistic production codebase with 10k dependencies can turn this No-GIL thing on.

ksec• 3 years ago

It will be interesting to see how this will be executed. I think many in Ruby land wanted something similar but couldn't get some general agreement. Ractors tried and arguably failed. We have Samuel Williams basically the one person pushing very hard for Async changes.

Ruby could learn a lot once this is done, but at the moment GIL optional in python seems to be a 2030 goal.

birdyrooster• 3 years ago

Soon I’ll be able to ditch golang

miraculixx53• 3 years ago

Right about ~2033

samsquire3• 3 years ago

Thank you so much Python core developers and steering council. Python is one of my favourite languages along with Java and C.

I greatly welcome true multithreading in Python.

I use both multiprocessing and multithreading in Python for different projects. See [0] for my multiprocessing example and python Threads for IO heavy tasks in [1]. But it would be far more efficient to use true threads.

Threads can communicate any amount of data in a single atomic almost instant operation. Using the local loopback interface or multiprocessing or pipes, this is not possible.

I am working on a multithreading architecture I call three tier multithreading architecture

https://github.com/samsquire/three-tier-multithreaded-archit...

My goal is extremely scalable and performant servers but Python is probably the wrong job for that.

[0] https://news.ycombinator.com/item?id=36897054 (my description of my use of multiprocessing) [1]: https://devops-pipeline.com/ (my use of multithreading)

fulafel• 3 years ago

How will this impact single thread performance?

miraculixx53• 3 years ago

From the POC implementation the reports state ~5-7% if memory serves right.

kerkeslager6• 3 years ago

Two major problems here:

1. There are some improvements worth breaking reverse-compatibility for, and removing the GIL is such an improvement. Whether the changes in Python 3 were worth making breaking changes for is debatable: certainly I don't see "print" being a function as particularly valuable. But the flipside is that the 2-to-3 transition was overblown by a vocal minority. I've transitioned more than 5 codebases from 2 to 3, and in most cases, there were few problems. Most problems were with codebases where previous developers had pulled in libraries for everything, resulting in an amalgamation of abandoned libraries, but these codebases run into problems even without the core language breaking compatibility. The answer isn't to flame your language into never breaking compatibility, it's to not import all of pip and expect that to be a sustainable strategy.

The situation we have now is that the steering committee has received so much heat from the vocal minority that they're terrified to make breaking changes. But removing the GIL should be a breaking change. It's too fundamental to how Python works to not be. So they're trying to remove the GIL and make it not a breaking change, which is a bad idea, because it is ultimately going to be a breaking change. It would be much better to admit this is a breaking change and start working on the transition plan, than to try the impossible task of making it not breaking because you're too terrified of your users to admit the truth.

We've already seen this in Python 3.11 which broke code in my codebase. The changes to fix the breakage weren't hard, but I would have liked better communication that this might happen. But I also understand why this was hidden in a deprecation warning in a minor release rather than publicized, because the Python team is probably tired of being flamed for making breaking changes.

2. The more fundamental problem here is that a lot of other features of Python were built around the GIL. Most obviously, the async paradigms makes sense largely because of the GIL. Sans-GIL, it looks like in retrospect a send/recv actor model a la Erlang would have been a much better way forward. It's not really possible to reverse this, and this might be pushing Python toward a less cohesive set of features that don't really make sense together. This makes it feel like this is too little too late.

miraculixx53• 3 years ago

The creator and lead maintainer of SQLAlchemy, one of the most popular and most used Python libraries for accessing databases (who doesn't?) gave a rather interesting response to PEP703.

Basically for the moment the GIL-less idea would likely be burdensome for us and the fact that it's only an "option" seems to strongly imply major compatibility issues that we would not prefer.

(...)

Adding an entirely new mode of operation to cPython that's optional would be an enormous burden for us as far as ensuring we use APIs appropriately, adding support, testing, we would have to spin up new test workers to test SQLAlchemy in both modes of operation, we would be getting strange new race condition related issues reported

https://github.com/sqlalchemy/sqlalchemy/discussions/10002#d...

Intent to approve PEP 703: making the GIL optional

Related Stories