The first year of free-threaded Python

https://labs.quansight.org/blog/free-threaded-one-year-recap

by rbanffy • 1 year ago

291 287 1 year ago

labs.quansight.org

Am I the only one who sort of fears the day when Python loses the GIL? I don't think Python developers know what they’re asking for. I don't really trust complex multithreaded code in any language. Python, with its dynamic nature, I trust least of all.

DHolzer• 1 year ago

I was thinking that too. I am really not a professional developer though.

OFC it would be nice to just write python and everything would be 12x accelerated, but i don't see how there would not be any draw-backs that would interfere with what makes python so approachable.

NortySpock• 1 year ago

I hope at least the option remains to enable the GIL, because I don't trust me to write thread-safe code on the first few attempts.

txdv• 1 year ago

how does the the language being dynamic negatively affect the complexity of multithreading?

nottorp• 1 year ago

Is there so much legacy python multithreaded code anyway?

Considering everyone knew about the GIL, I'm thinking most people just wouldn't bother.

breadwinner2• 1 year ago

When the language is dynamic there is less rigor. Statically checked code is more likely to be correct. When you add threads to "fast and loose" code things get really bad.

jerf• 1 year ago

I have a hypothesis that being dynamic has no particular effect on the complexity of multithreading. I think the apparent effect is a combination of two things: 1. All our dynamic scripting languages in modern use date from the 1990s before this degree of threading was a concern for the languages and 2. It is really hard to retrofit code written for not being threaded to work in a threaded context, and the "deeper" the code in the system the harder it is. Something like CPython is about as "deep" as you can go, so it's really, really hard.

I think if someone set out to write a new dynamic scripting language today, from scratch, that multithreading it would not pose any particular challenge. Beyond that fact that it's naturally a difficult problem, I mean, but nothing special compared to the many other languages that have implemented threading. It's all about all that code from before the threading era that's the problem, not the threading itself. And Python has a loooot of that code.

rocqua4• 1 year ago

Dynamic(ally typed) languages, by virtue of not requiring strict typing, often lead to more complicated function signatures. Such functions are generally harder to reason about. Because they tend to require inspection of the function to see what is really going on.

Multithreaded code is incredibly hard to reason about. And reasoning about it becomes a lot easier if you have certain guarantees (e.g. this argument / return value always has this type, so I can always do this to it). Code written in dynamic languages will more often lack such guarantees, because of the complicated signatures. This makes it even harder to reason about Multithreaded code, increasing the risk posed by multithreaded code.

miohtama• 1 year ago

GIL or no-GIL concerns only people who want to run multicore workloads. If you are not already spending time threading or multiprocessing your code there is practically no change. Most race condition issues which you need to think are there regardless of GIL.

immibis2• 1 year ago

With the GIL, multithreaded Python gives concurrent I/O without worrying about data structure concurrency (unless you do I/O in the middle of it) - it's a lot like async in this way - data structure manipulation is atomic between "await" expressions (except in the "await" is implicit and you might have written one without realizing in which case you have a bug). Meanwhile you still get to use threads to handle several concurrent I/O operations. I bet a lot of Python code is written this way and will start randomly crashing if the data manipulation becomes non-atomic.

fulafel• 1 year ago

A lot of Python usage is leveraging libraries with parallel kernels inside written in other languages. A subset of those is bottlenecked on Python side speed. A sub-subset of those are people who want to try no-GIL to address the bottleneck. But if non-GIL becomes pervasive, it could mean Python becomes less safe for the "just parallel kernels" users.

monkeyelite8• 1 year ago

When you launch processes to do work you get multi-core workload balancing for free.

quectophoton• 1 year ago

I don't want to add more to your fears, but also remember that LLMs have been trained on decades worth of Python code that assumes the presence of the GIL.

rocqua4• 1 year ago

This could, indeed, be quite catastrophic.

I wonder if companies will start adding this to their system prompts.

dotancohen6• 1 year ago

As a Python dabbler, what should I be reading to ensure my multi-threaded code in Python is in fact safe.

cess11• 1 year ago

The literature on distributed systems is huge. It depends a lot on your use case what you ought to do. If you're lucky you can avoid shared state, as in no race conditions in either end of your executions.

https://www.youtube.com/watch?v=_9B__0S21y8 is fairly concise and gives some recommendations for literature and techniques, obviously making an effort in promoting PlusCal/TLA+ along the way but showcases how even apparently simple algorithms can be problematic as well as how deep analysis has to go to get you a guarantee that the execution will be bug free.

HDThoreaun2• 1 year ago

Honestly unless youre willing to devote a solid 4+ hours to learning about multi threading stick with ayncio

bayindirh3• 1 year ago

More realistically, as it happened in ML/AI scene, the knowledgeable people will write the complex libraries and will hand these down to scientists and other less experienced, or risk-averse developers (which is not a bad thing).

With the critical mass Python acquired over the years, GIL becomes a very sore bottleneck in some cases. This is why I decided to learn Go, for example. Properly threaded (and green threaded) programming language which is higher level than C/C++, but lower than Python which allows me to do things which I can't do with Python. Compilation is another reason, but it was secondary with respect to threading.

bgwalter5• 1 year ago

Knowledgeable people? Pytorch has memory leaks by design, it uses std::shared_ptr for a graph with cycles. It also has threading issues.

jillesvangurp2• 1 year ago

You are not the only one who is afraid of changes and a bit change resistant. I think the issue here is that the reasons for this fear are not very rational. And also the interest of the wider community is to deal with technical debt. And the GIL is pure technical debt. Defensible 30 years ago, a bit awkward 20 years ago, and downright annoying and embarrassing now that world + dog does all their AI data processing with python at scale for the last 10. It had to go in the interest of future proofing the platform.

What changes for you? Nothing unless you start using threads. You probably weren't using threads anyway because there is little to no point in python to using them. Most python code bases completely ignore the threading module and instead use non blocking IO, async, or similar things. The GIL thing only kicks in if you actually use threads.

If you don't use threads, removing the GIL changes nothing. There's no code that will break. All those C libraries that aren't thread safe are still single threaded, etc. Only if you now start using threads do you need to pay attention.

There's some threaded python code of course that people may have written in python somewhat naively in the hope that it would make things faster that is constantly hitting the GIL and is effectively single threaded. That code now might run a little faster. And probably with more bugs because naive threaded code tends to have those.

But a simple solution to address your fears: simply don't use threads. You'll be fine.

Or learn how to use threads. Because now you finally can and it isn't that hard if you have the right abstractions. I'm sure those will follow in future releases. Structured concurrency is probably high on the agenda of some people in the community.

HDThoreaun2• 1 year ago

But a simple solution to address your fears: simply don't use threads. You'll be fine.

Im not worried about new code. Im worried about stuff written 15 years ago by a monkey who had no idea how threads work and just read something on stack overflow that said to use threading. This code will likely break when run post-GIL. I suspect there is actually quite a bit of it.

dkarl• 1 year ago

What changes for you? Nothing unless you start using threads

Coming from the Java world, you don't know what you're missing. Looking inside an application and seeing a bunch of threadpools managed by competing frameworks, debugging timeouts and discovering that tasks are waiting more than a second to get scheduled on the wrong threadpool, tearing your hair out because someone split a tiny sub-10μs bit of computation into two tasks and scheduling the second takes a hundred times longer than the actual work done, adding a library for a trivial bit of functionality and discovering that it spins up yet another threadpool when you initialize it.

(I'm mostly being tongue in cheek here because I know it's nice to have threading when you need it.)

rbanffy4• 1 year ago

There's some threaded python code of course

A fairly common pattern for me is to start a terminal UI updating thread that redraws the UI every second or so while one or more background threads do their thing. Sometimes, it’s easier to express something with threads and we do it not to make the process faster (we kind of accept it will be a bit slower).

The real enemy is state that can me mutated from more than one place. As long as you know who can change what, threads are not that scary.

tgv• 1 year ago

Nothing unless you start using threads.

Isn't it also promises/futures? They might start threads implicitly.

zem• 1 year ago

this looks extremely promising https://microsoft.github.io/verona/pyrona.html

freeone3000• 1 year ago

I'm sure you'll be happy using the last language that has to fork() in order to thread. We've only had consumer-level multicore processors for 20 years, after all.

im3w1l• 1 year ago

You have to understand that people come from very different angles with python. Some people write web servers where in python, where speed equals money saved. Other people write little UI apps that where speed is a complete non-issue. Yet others write aiml code that spends most of its time in gpu code. But then they want to do just a little data massaging in python which can easily bottleneck the whole thing. And some people people write scripts that don't use a .env but rather os-libraries.

monkeyelite8• 1 year ago

I don’t understand this argument. My python program isn’t the only program on the system - I have a database, web server, etc. It’s already multi-core.

bratao• 1 year ago

This is a common mistake and very badly communicated. The GIL do not make the Python code thread-safe. It only protect the internal CPython state. Multi-threaded Python code is not thread-safe today.

amelius3• 1 year ago

Well, I think you can manipulate a dict from two different threads in Python, today, without any risk of segfaults.

porridgeraisin• 1 year ago

Internal cpython state also includes say, a dictionary's internal state. So for practical purposes it is safe. Of course, TOCTOU, stale reads and various race conditions are not (and can never be) protected by the GIL.

kevingadd• 1 year ago

This should not have been downvoted. It's true that the GIL does not make python code thread-safe implicitly, you have to either construct your code carefully to be atomic (based on knowledge of how the GIL works) or make use of mutexes, semaphores, etc. It's just memory-safe and can still have races etc.

tialaramex• 1 year ago

You're not the only one. David Baron's note certainly applies: https://bholley.net/blog/2015/must-be-this-tall-to-write-mul...

In a language conceived for this kind of work it's not as easy as you'd like. In most languages you're going to write nonsense which has no coherent meaning whatsoever. Experiments show that humans can't successfully understand non-trivial programs unless they exhibit Sequential Consistency - that is, they can be understood as if (which is not reality) all the things which happen do happen in some particular order. This is not the reality of how the machine works, for subtle reasons, but without it merely human programmers are like "Eh, no idea, I guess everything is computer?". It's really easy to write concurrent programs which do not satisfy this requirement in most of these languages, you just can't debug them or reason about what they do - a disaster.

As I understand it Python without the GIL will enable more programs that lose SC.

qznc• 1 year ago

Worst case is probably that it is like a "Python4": Things break when people try to update to non-GIL, so they rather stay with the old version for decades.

odiroot• 1 year ago

It's called job security. We'll be rewriting decades of code that's broken by that transition.

almostgotcaught• 1 year ago

Do you understand what you're implying?

"Python programmers are so incompetent that Python succeeds as a language only because it lacks features they wouldn't know to use"

Even if it's circumstantially true, doesn't mean it's the right guiding principle for the design of the language.

frollogaston6• 1 year ago

What reliance did you have in mind? All sorts of calls in Python can release the GIL, so you already need locking, and there are race conditions just like in most languages. It's not like JS where your code is guaranteed to run in order until you "await" something.

I don't fully understand the challenge with removing it, but thought it was something about C extensions, not something most users have to directly worry about.

seabrookmx2• 1 year ago

While it certainly has its rough edges, I'm a big asyncio user. So I'll be over here happily writing concurrent python that's single threaded, ie. pretending my Python is nodejs.

For the web/network workloads most of us write, I'd highly recommend this.

dontlaugh2• 1 year ago

Asyncio being able to use thread pools would reduce memory usage at the very least.

monkeyelite8• 1 year ago

Good engineering design is about making unbalanced tradeoffs where you get huge wins for low costs. These kinds of decisions are opinionated and require you to say no to some edge cases to get a lot back on the important cases.

One lesson I have learned is that good design cannot survive popularity and bureaucracy that comes with it. Over time people just beat down your door with requests to do cases you explicitly avoided. You’re blocking their work and not being pragmatic! Eventually nobody is left to advocate for them.

And part of that is the community has more resources and can absorb some more complexity. But this is also why I prefer tools with smaller communities.

pawanjswal• 1 year ago

This is some serious groundwork for the next era of performance!

pansa22• 1 year ago

Does removal of the GIL have any other effects on multi-threaded Python code (other than allowing it to run in parallel)?

My understanding is that the GIL has lasted this long not because multi-threaded Python depends on it, but because removing it:

- Complicates the implementation of the interpreter

- Complicates C extensions, and

- Causes single-threaded code to run slower

Multi-threaded Python code already has to assume that it can be pre-empted on the boundary between any two bytecode instructions. Does free-threaded Python provide the same guarantees, or does it require multi-threaded Python to be written differently, e.g. to use additional locks?

rfoo2• 1 year ago

Does free-threaded Python provide the same guarantees

Mostly. Some of the "can be pre-empted on the boundary between any two bytecode instructions" bugs are really hard to hit without free-threading, though. And without free-threading people don't use as much threading stuff. So by nature it exposes more bugs.

Now, my rants:

have any other effects on multi-threaded Python code

It stops people from using multi-process workarounds. Hence, it simplifies user-code. IMO totally worth it to make the interpreter more complex.

Complicates C extensions

The alternative (sub-interpreters) complicates C extensions more than free-threading and the top one most important C extension in the entire ecosystem, numpy, stated that they can't and they don't want to support sub-interpreters. On contrary, they already support free-threading today and are actively sorting out remaining bugs.

Causes single-threaded code to run slower

That's the trade-off. Personally I think a single digit percentage slow-down of single-threaded code worth it.

celeritascelery2• 1 year ago

That's the trade-off. Personally I think a single digit percentage slow-down of single-threaded code worth it.

Maybe. I would expect that 99% of python code going forward will still be single threaded. You just don’t need that extra complexity for most code. So I would expect that python code as a whole will have worse performance, even though a handful of applications will get faster.

rocqua4• 1 year ago

Note that there is an entire order of magnitude range for a 'single digit'.

A 1% slowdown seems totally fine. A 9% slowdown is pretty bad.

jacob019• 1 year ago

Your understanding is correct. You can use all the cores but it's much slower per thread and existing libraries may need to be reworked. I tried it with PyTorch, it used 10x more CPU to do half the work. I expect these issues to improve, still great to see after 20 years wishing for it.

btilly2• 1 year ago

It makes race conditions easier to hit, and that will require multi-threaded Python to be written with more care to achieve the same level of reliability.

monkeyelite8• 1 year ago

Yes it makes every part of the ecosystem more complex and prone to bugs in hopes of getting more performance in a scripting language.

heybrendan• 1 year ago

I am a Python user, but far from an expert. Occasionally, I've used 'concurrent.futures' to kick off running some very simple functions, at the same time.

How are 'concurrent.futures' users impacted? What will I need to change moving forward?

rednafi2• 1 year ago

It’s going to get faster since threads won’t be locked on GIL. If you’re locking shared objects correctly or not using them all, then you should be good.

0x000xca0xfe• 1 year ago

I know it's just an AI image... but a snake with two tails? C'mon!

brookst• 1 year ago

Confusoborus

vpribish• 1 year ago

shh. don't complain too loudly or we'll lose an important tell. python articles using snake illustrations can usually be ignored because they are not clueful.

-- python, monty

bgwalter5• 1 year ago

This is just an advertisement for the company. Fact is, free-threading is still up to 50% slower, the tail call interpreter isn't much faster at all, and free-threading is still flaky.

Things they won't tell you at PyCon.

tomrod5• 1 year ago

QuantSight isn't a formal company though, it's a skunkworks/OSS research group run by the Travis Oliphant.

lenerdenator• 1 year ago

I don't see how any of that's a problem given that it's not the default for how people run Python.

It's a big project that's going to take lots of time by lots of people to finish. Keep it behind opt-in, keep accepting pull requests after rigorous testing, and it's fine.

pjmlp3• 1 year ago

On the other news, Microsoft dumped the whole faster Python team, apparently the 2025 earnings weren't enough to keep the team around.

https://www.linkedin.com/posts/mdboom_its-been-a-tough-coupl...

Lets see whatever performance improvements still land on CPython, unless other company sponsors the work.

I guess Facebook (no need to correct me on the name) is still sponsoring part of it.

falcor843• 1 year ago

It wouldn't have bothered me if you just said "Facebook" - I probably wouldn't have even noticed it. But I'm really curious why you chose to write "Facebook", then apparently noticed the issue, and instead of replacing it with "Meta" decided to add the much longer "(no need to correct me on the name)". What axe are you grinding?

pjmlp3• 1 year ago

Yes, because I am quite certain someone without anything better to do would correct me on that.

For me Facebook will always be Facebook, and Twitter will always be Twitter.

rich_sasha• 1 year ago

Ah that's very, very sad. I guess they have embraced and extended, there's only one thing left to do.

biorach8• 1 year ago

At this stage the cliched and clueless comments about embrace/extend/extinguish are tiresome and inevitable whenever Microsoft is mentioned.

A few decades ago MS did indeed have a playbook which they used to undermine open standards. Laying off some members of the Python team bears no resemblence whatsoever to that. At worst it will delay the improvement of free-threaded Python. That's all.

Your comment is lazy and unfounded.

stusmall• 1 year ago

That shows a misunderstanding of what EEE was. This team was sending changes upstream which is the exact opposite of "extend" step of the strategy. The idea of "extend" was to add propriety extensions on top of an open standard/project locking customers into the MSFT implementation.

bgwalter5• 1 year ago

They were quite a bit behind the schedule that was promised five years ago.

Additionally, at this stage the severe political and governance problems cannot have escaped Microsoft. I imagine that no competent Microsoft employee wants to give his expertise to CPython, only later to suffer group defamation from a couple of elected mediocre people.

CPython is an organization that overpromises, allocates jobs to the obedient and faithful while weeding out competent dissenters.

It wasn't always like that. The issues are entirely self-inflicted.

biorach8• 1 year ago

CPython is an organization that overpromises, allocates jobs to the obedient and faithful while weeding out competent dissenters.

This stinks of BS

make33• 1 year ago

Microsoft also fired a whole lot of other open source people unrelated to Python in this current layoff

vlovich1232• 1 year ago

That’s unfortunate but I called it when people were claiming that Microsoft had committed to this effort for the long term.

mtzaldo• 1 year ago

Could we do a crowdfunding campaign so we can keep paying them? The whole world is/will benefit from their work.

morkalork• 1 year ago

Didn't Google lay off their entire Python development team in the last year as well? I wonder if there is some impetus behind both.

make33• 1 year ago

doesn't print money right away = cut by executive #3442

monkeyelite8• 1 year ago

This is a story from a few years ago, and they obviously have lots of teams that use python. This team was an internal support for python and tools - which honestly sounds exactly like the kind of thing that would get cut in a pinch (no value judgement).

amelius3• 1 year ago

The snake in the header image appears to have two tail-ends ...

cestith2• 1 year ago

I guess it’s spawned a second thread in the same process.

sgarland4• 1 year ago

Instead, many reach for multiprocessing, but spawning processes is expensive

Agreed.

and communicating across processes often requires making expensive copies of data

SharedMemory [0] exists. Never understood why this isn’t used more frequently. There’s even a ShareableList which does exactly what it sounds like, and is awesome.

[0] https://docs.python.org/3/library/multiprocessing.shared_mem...

ogrisel• 1 year ago

You cannot share arbitrarily structured objects in the `ShareableList`, only atomic scalars and bytes / strings.

If you want to share structured Python objects between instances, you have to pay the cost of `pickle.dump/pickle.dump` (CPU overhead for interprocess communication) + the memory cost of replicated objects in the processes.

tomrod5• 1 year ago

I can fit a lot of json into bytes/strings though?

notpushkin4• 1 year ago

We need a dataclass-like interface on top of a ShareableList.

sgarland4• 1 year ago

So don’t do that? Send data to workers as primitives, and have a separate process that reads the results and serializes it into whatever form you want.

modeless• 1 year ago

Yeah I've had great success sharing numpy arrays this way. Explicit sharing is not a huge burden, especially when compared with the difficulty of debugging problems that occur when you accidentally share things between threads. People vastly overstate the benefit of threads over multiprocessing and I don't look forward to all the random segfaults I'm going to have to debug after people start routinely disabling the GIL in a library ecosystem that isn't ready.

I wonder why people never complained so much about JavaScript not having shared-everything threading. Maybe because JavaScript is so much faster that you don't have to reach for it as much. I wish more effort was put into baseline performance for Python.

dhruvrajvanshi3• 1 year ago

I wonder why people never complained so much about JavaScript not having shared-everything threading. Maybe because JavaScript is so much faster that you don't have to reach for it as much. I wish more effort was put into baseline performance for Python.

This is a fair observation.

I think a part of the problem is that the things that make GIL less python hard are also the things that make faster baseline performance hard. I.e. an over reliance of the ecosystem on the shape of the CPython data structures.

What makes python different is that a large percentage of python code isn't python, but C code targeting the CPython api. This isn't true for a lot of other interpreted languages.

com2kid• 1 year ago

I wonder why people never complained so much about JavaScript not having shared-everything threading. Maybe because JavaScript is so much faster that you don't have to reach for it as much. I wish more effort was put into baseline performance for Python.

Nobody sane tries to do math in JS. Backend JS is recommended for situations where processing is minimal and it is mostly lots of tiny IO requests that need to be shunted around.

I'm a huge JS/Node proponent and if someone says they need to write a backend service that crunches a lot of numbers, I'll recommend choosing a different technology!

For some reason Python peeps keep trying to do actual computations in Python...

zahlman10• 1 year ago

I wish more effort was put into baseline performance for Python.

There has been. That's why the bytecode is incompatible between minor versions. It was a major selling(?) point for 3.11 and 3.12 in particular.

But the "Faster CPython" team at Microsoft was apparently just laid off (https://www.linkedin.com/posts/mdboom_its-been-a-tough-coupl...), and all of the optimization work has to my understanding been based around fairly traditional techniques. The C part of the codebase has decades of legacy to it, after all.

Alternative implementations like PyPy often post impressive results, and are worth checking out if you need to worry about native Python performance. Not to mention the benefits of shifting the work onto compiled code like NumPy, as you already do.

frollogaston6• 1 year ago

"I wonder why people never complained so much about JavaScript not having shared-everything threading"

Mainly cause Python is often used for data pipelines in ways that JS isn't, causing situations where you do want to use multiple CPU cores with some shared memory. If you want to use multiple CPU cores in NodeJS, usually it's just a load-balancing webserver without IPC and you just use throng, or maybe you've got microservices.

Also, JS parallelism simply excelled from the start at waiting on tons of IO, there was no confusion about it. Python later got asyncio for this, and by now regular threads have too much momentum. Threads are the worst of both worlds in Py, cause you get the overhead of an OS thread and the possibility of race conditions without the full parallelism it's supposed to buy you. And all this stuff is confusing to users.

monkeyelite8• 1 year ago

I wonder why people never complained so much about JavaScript not having shared-everything threading

Because it greatly simplifies the language and gives you all kinds of invariants.

chubot• 1 year ago

Spawning processes generally takes much less than 1 ms on Unix

Spawning a PYTHON interpreter process might take 30 ms to 300 ms before you get to main(), depending on the number of imports

It's 1 to 2 orders of magnitude difference, so it's worth being precise

This is a fallacy with say CGI. A CGI in C, Rust, or Go works perfectly well.

e.g. sqlite.org runs with a process PER REQUEST - https://news.ycombinator.com/item?id=3036124

Sharlin• 1 year ago

Unix is not the only platform though (and is process creation fast on all Unices or just Linux?) The point about interpreter init overhead is, of course, apt.

charleshn3• 1 year ago

Spawning processes generally takes much less than 1 ms on Unix

It depends on whether one uses clone, fork, posix_spawn etc.

Fork can take a while depending on the size of the address space, number of VMAs etc.

LPisGood• 1 year ago

My understanding is that spawning a thread takes just a few micro seconds, so whether you’re talking about a process or a Python interpreter process there are still orders of magnitude to be gained.

kragen7• 1 year ago

To be concrete about this, http://canonical.org/~kragen/sw/dev3/forkovh.c took 670μs to fork, exit, and wait on the first laptop I tried it on, but only 130μs compiled with dietlibc instead of glibc, and with glibc on a 2.3 GHz E5-2697 Xeon, it took 130μs compiled with glibc.

httpdito http://canonical.org/~kragen/sw/dev3/server.s (which launches a process per request) seems to take only about 50μs because it's not linked with any C library and therefore only maps 5 pages. Also, that doesn't include the time for exit() because it runs multiple concurrent child processes.

On this laptop, a Ryzen 5 3500U running at 2.9GHz, forkovh takes about 330μs built with glibc and about 130–140μs built with dietlibc, and `time python3 -c True` takes about 30000–50000μs. I wrote a Python version of forkovh http://canonical.org/~kragen/sw/dev3/forkovh.py and it takes about 1200μs to fork(), _exit(), and wait().

If anyone else wants to clone that repo and test their own machines, I'm interested to hear the results, especially if they aren't in Linux. `make forkovh` will compile the C version.

1200μs is pretty expensive in some contexts but not others. Certainly it's cheaper than spawning a new Python interpreter by more than an order of magnitude.

jaoane2• 1 year ago

Spawning a PYTHON interpreter process might take 30 ms to 300 ms before you get to main(), depending on the number of imports

That's lucky. On constrained systems launching a new interpreter can very well take 10 seconds. Python is ssssslllloooowwwww.

morningsam• 1 year ago

Spawning a PYTHON interpreter process might take 30 ms to 300 ms

Which is why, at least on Linux, Python's multiprocessing doesn't do that but fork()s the interpreter, which takes low-single-digit ms as well.

ori_b2• 1 year ago

As another example: I run https://shithub.us with shell scripts, serving a terabyte or so of data monthly (mostly due to AI crawlers that I can't be arsed to block).

I'm launching between 15 and 3000 processes per request. While Plan 9 is about 10x faster at spawning processes than Linux, it's telling that 3000 C processes launching in a shell is about as fast as one python interpreter.

Too3• 1 year ago

In Python, if you are spawning processes or even threads in a tight loop you have already lost. Use ThreadPoolExecutor or ProcessPoolExecutor from concurrent.futures instead. Then startup time becomes no factor.

seunosewa• 1 year ago

You can use a pool of interpreter processes. You don't have to spawn one for each request.

isignal• 1 year ago

Processes can die independently so the state of a concurrent shared memory data structure when a process dies while modifying this under a lock can be difficult to manage. Postgres which uses shared memory data structures can sometimes need to kill all its backend processes because it cannot fully recover from such a state.

In contrast, no one thinks about what happens if a thread dies independently because the failure mode is joint.

wongarsu• 1 year ago

In contrast, no one thinks about what happens if a thread dies independently because the failure mode is joint.

In Rust if a thread holding a mutex dies the mutex becomes poisoned, and trying to acquire it leads to an error that has to be handled. As a consequence every rust developer that touches a mutex has to think about that failure mode. Even if in 95% of cases the best answer is "let's exit when that happens".

The operating system tends to treat your whole process as one and shot down everything or nothing. But a thread can still crash in its own due to unhandled oom, assertion failures or any number of other issues

jcalvinowens3• 1 year ago

This is a solvable problem though, the literature is overflowing with lock-free implementations of common data structures. The real question is how much performance you have to sacrifice for the guarantee...

tinix4• 1 year ago

shared memory only works on dedicated hardware.

if you're running in something like AWS fargate, there is no shared memory. have to use the network and file system which adds a lot of latency, way more than spawning a process.

copying processes through fork is a whole different problem.

green threads and an actor model will get you much further in my experience.

bradleybuda• 1 year ago

Fargate is just a container runtime. You can fork processes and share memory like you can in any other Linux environment. You may not want to (because you are running many cheap / small containers) but if your Fargate containers are running 0.25 vCPUs then you probably don't want traditional multiprocessing or multithreading...

sgarland4• 1 year ago

Well don’t use Fargate, there’s your problem. Run programs on actual servers, not magical serverless bullshit.

perlgeek• 1 year ago

Never understood why this isn’t used more frequently.

Can you throw a JSON-serializable data structure (lists, dict, strings, numbers) into SharedMemory? What about regular instance of random Python classes? If the answer is "no", that explains why it's not done more often.

The examples in the docs seem to pass byte strings and byte arrays around, which is far less convenient than regular data structures.

dragonwriter2• 1 year ago

Can you throw a JSON-serializable data structure (lists, dict, strings, numbers) into SharedMemory?

You can throw a JSON-serialized data structure into SharedMemory, sure, since you can store strings.

The examples in the docs seem to pass byte strings and byte arrays around

The examples in the docs largely use ShareableList, which itself can contain any of int, float, bool, str, bytes, and None-type values.

YouWhy• 1 year ago

Hey, I've been developing professionally with Python for 20 years, so wanted to weigh in:

Decent threading is awesome news, but it only affects a small minority of use cases. Threads are only strictly necessary when it's prohibitive to message pass. The Python ecosystem these days includes a playbook solution for literally any such case. Considering the multiple major pitfalls of threads (i.e., locking), they are likely to become a thing useful only in specific libraries/domains and not as a general.

Additionally, with all my love to vanilla Python, anyone who needs to squeeze the juice out of their CPU (which is actually memory bandwidth) has a plenty of other tools -- off the shelf libraries written in native code. (Honorable mention to Pypy, numba and such).

Finally, the one dramatic performance innovation in Python has been async programming - I warmly encourage everyone not familiar with it to consider taking a look.

kstrauser18• 1 year ago

I haven’t been using it that much longer than you, and I agree with most of what you’re saying, but I’d characterize it differently.

Python has a lot of solid workarounds for avoid threading because until now Python threading has absolutely sucked. I had naively tried to use it to make a CPU-bound workload twice as fast and soon realized the implications of the GIL, so I threw all that code away and made it multiprocessing instead. That sucked in its own way because I had to serialize lots of large data structures to pass around, so 2x the cores got me about 1.5x the speed and a warmer server room.

I would love to have good threading support in Python. It’s not always the right solution, but there are a lot of circumstances where it’d be absolutely peachy, and today we’re faking our way around its absence with whole playbooks of alternative approaches to avoid the elephant in the room.

But yes, use async when it makes sense. It’s a thing of beauty. (Yes, Glyph, we hear the “I told you so!” You were right.)

zahlman10• 1 year ago

That sucked in its own way because I had to serialize lots of large data structures to pass around, so 2x the cores got me about 1.5x the speed and a warmer server room.

In many cases you can't reasonably expect better than that (https://en.wikipedia.org/wiki/Amdahl's_law). If your algorithm involves sharing "large data structures" in the first place, that's a bad sign.

BrokrnAlgorithm• 1 year ago

I find python's async to be lacking in fine grained control. It may be fine for 95% of simple use cases, but lacks advanced features such as sequential constraining, task queue memory management, task pre-emption etc. The async keword also tends to bubble up through codebases in aweful ways, making it almost impossible to create reasonably decoupled code.

sylware3• 1 year ago

Got myself a shiny python 3.13.3 (ssl module still unable to compile with libressl) replacing a 3.12.2, feels clearly slower.

What's wrong?

ipsum22• 1 year ago

python 3.13 doesn't ship with free-threaded Python compiled AFAIK.

sylware3• 1 year ago

You mean it is not default anymore?

jdsleppy• 1 year ago

Did you compile the Python yourself? If so, you may need to add optimization flags https://devguide.python.org/getting-started/setup-building/i...

aitchnyu• 1 year ago

Whats currently stopping me (apart from library support) from running a single command that starts up WSGI workers and Celery workers in a single process?

gchamonlive• 1 year ago

Nothing, it's just that these aren't first class features of the language. Also someone already explained that the GIL is mostly about technical debt in the CPython interpreter, so there are reasons other than full parallelism to get rid of the GIL.

hello_computer• 1 year ago

Opting to enable low-level parallelism for user code in an imperative, dynamically typed scripting language seems like regression. It’s less bad for LISP because of the pure-functional nature. It’s less bad for BEAM languages & Clojure due to immutability. It is less bad for C/C++/Rust because you have a stronger type system—allowing for deeper static analysis. For Python, this is “high priests of a low cult” shitting things up for corporate agendas and/or street cred.

p0w3n3d• 1 year ago

Look behind! A free-threaded Python!

EGreg• 1 year ago

I thought this was mostly a solved problem.

  Fibers
  Green threads
  Coroutines
  Actors
  Queues (eg GCD)
  …

Basically you need to reason about what your thing will do.

Separate concerns. Each thing is a server (microservice?) with its own backpressure.

They schedule jobs on a queue.

The jobs come with some context, I don’t care if it’s a closure on the heap or a fiber with a stack or whatever. Javascript being single threaded with promises wastefully unwinds the entire stack for each tick instead of saving context. With callbacks you can save context in closures. But even that is pretty fast.

Anyway then you can just load-balance the context across machines. Easiest approach is just to have server affinity for each job. The servers just contain a cache of the data so if the servers fail then their replacements can grab the job from an indexed database. The insertion and the lookup is O(log n) each. And jobs are deleted when done (maybe leaving behind a small log that is compacted) so there are no memory leaks.

Oh yeah and whatever you store durably should be sharded and indexed properly, so practicalkt unlimited amounts can be stored. Availability in a given share is a function of replicating the data, and the economics of it is that the client should pay with credits for every time they access. You can even replicate on demand (like bittorrent re-seeding) to handle spikes.

This is the general framework whether you use Erlang, Go, Python or PHP or whatever. It scales within a company and even across companies (as long as you sign/encrypt payloads cryptographically).

It doesn’t matter so much whether you use php-fpm with threads, or swoole, or the new kid on the block, FrankenPHP. Well, I should say I prefer the shared-nothing architecture of PHP and APC. But in Python, it is the same thing with eg Twisted vs just some SAPI.

You’re welcome.

kccqzy2• 1 year ago

It's only a mostly solved problem for concurrent I/O heavy workloads. It's not solved in the Python world for parallel CPU-bound workloads.

henry700• 1 year ago

I find it peculiar how, in a language so riddled with simple concurrency architectural issues, the approach is to painstankingly fix every library after fixing the runtime, instead of just using some better language. Why does the community insist on such a bad language when literally even fucking Javascript has a saner execution model?

mylons• 1 year ago

i find it peculiar how tribal people are about languages. python is fantastic. you're not winning anyone over with comments like this. just go write your javascript and be happy, bud.

forrestthewoods• 1 year ago

instead of just using some better language

Python the language is pretty bad. Python the ecosystem of libraries and tools has no equal, unfortunately.

Switching a language is easy. Switching a billion lines of library less so.

And the tragic part is that many of the top “python libraries” are just Python interfaces to a C library! But if you want to switch to a “better language” that fact isn’t helpful.

kubb• 1 year ago

I wonder if we get automatic LLM translation of codebases from language to language soon - this could close the library gap and diminish the language lock in factor.

dash2• 1 year ago

I think the opposite. Every language has flaws. What's impressive about Python is their ongoing commitment to work on theirs, even the deepest-rooted. It makes me optimistic that this is a language to stick with for the long run.

rednafi2• 1 year ago

I agree about using other languages that have better concurrency support if concurrency is your bottleneck.

But changing the language in a brownfield project is hard. I love Go, and these days I don’t bother with Python if I know the backend needs to scale.

But Python’s ecosystem is huge, and for data work, there’s little alternative to it.

With all that said, JavaScript ain’t got shit on any language. The only good thing about it is Google’s runtime, and that has nothing to do with the language. JS doesn’t have true concurrency and is a mess of a language in general. Python is slow, riddled with concurrency problems, but at least it’s a real language created by a guy who knew what he was doing.

make33• 1 year ago

I hate how these threads always devolve in insane discussions about why not using threads is better, while most real world people who have tried to do real world speeding up of Python code realize how amazing it would be to have proper threads with shared memory instead of the processes that have so many limitations, like forcing to pickle objects back and forth, & fork so often just not working in the cloud setting, & spawn being so slow in a lot of applications. The usage of processes is just much heavier and less straightforward.

yunnpp• 1 year ago

In 2025, 20 years since multi-core is a thing on consumer devices. Great progress, guys. Can't wait for what the Python community has up its sleeves next.

MichaelMoser123• 1 year ago

cpython doesn't have a JIT, why is free-threaded python a higher priority than developing a just in time compiler? The later would be more resonant with the typical use case for python and benefit a larger portion of users, wouldn't it? (Wouldn't a backend server project use golang or java to begin with?)

diziet_sma• 1 year ago

There have been many attempts to make a Python JIT that is compatible with CPython to various levels of success. However the larger reason is that the gains from removing the GIL far exceeds the gains from a JIT.

If you're writing performance sensitive python code, the "hot" code is likely already in a C-extension, such as Numpy. So there is negligible benefit to running the code with a JIT.

The first year of free-threaded Python

Related Stories