Instrumenting Python GIL with eBPF

https://coroot.com/blog/instrumenting-python-gil-with-ebpf

by lukastyrychtr • 2 years ago

118 30 2 years ago

polotics2• 2 years ago

Wow, 36ms per second is only 3.6% of the time. Python waiting on the GIL is then a pretty overblown problem, as this is not very significant. I wonder if this measure could be run on apps built with various frameworks. I expect that with uvloop and all, the percentage would be even less.

black_puppydog• 2 years ago

It may not be very significant in this context.

But the author literally starts by explaining that that's a typical argument for webservers because they're mostly I/O bound. Anyone working with code that's more CPU bound will have very different numbers, and interpret them differently.

polotics2• 2 years ago

are you sure though that being more CPU-bound will imply more waiting on the GIL? CPU-bound python in my experience means libraries, like eg. numpy, that are well-designed and release the GIL.

godelski3• 2 years ago

for webservers because they're mostly I/O bound

I'm not a web person, but can't you also gain I/O from parallelization? The I/O bound is waiting on responses right? So parallelization should increase I/O because you can make multiple asynchronous requests and even if there is dependence you can often stage or do partial computation in the mean time (at least this is common in scientific computing). (And if disk, well reading/writing to disk in parallel is far faster than serial but idk why you'd read/write to disk with pure python. Though it seems people do). So wouldn't this have significant effects that are more than the 36ms that we see in TFA? Or am I missing something and can someone explain why my guess is wrong?

memco• 2 years ago

That’s over 40 minutes of a full day or 5 hours a week spent locking! As the author states it’s highly context dependent but in some cases this is a lot of time to do other work if it can be reduced.

kevingadd• 2 years ago

If you're running your application on 100 servers, that's potentially 3-4 entire machines you no longer have to pay EC2/S3/etc fees for, or it's more scaling headroom because you've decreased load across your whole fleet by 3.6%.

It's "small", sure, but in production performance issues are often "death by a thousand cuts" situations, so a 3.6% reduction is a big win compared to optimizations that are often in the 1% range.

in_a_society• 2 years ago

I'm not sure your conclusion is a fair take. In the app I work on, GIL acquisition would easily take 2-3x longer than the postgres queries which would subsequently be issued.

godelski3• 2 years ago

  > only 3.6%
  > GIL is then a pretty overblown problem, as this is not very significant

How do you conclude this? 3.6% of the time spent being locked seems pretty significant. As another user notes, this is 3-4 fewer machines you need to pay for out of every 100 you currently use.

But the problem is worse than that. That's 3.6% of the time you're locked, not 3.6% of the time your program takes vs the time it would take without GIL. During that same time other processes could be run. I've seen plenty of mundane and ordinary tasks be 10x to 1000x faster when comparing serial to even unoptimized parallel. I'd be really careful to take this conclusion unless you actually have experience in writing parallel and/or optimized code.

almostgotcaught• 2 years ago

people who think 3.6% doesn't matter don't have any imagination. ballpark whatever you imagine is FB's compute cost vis-a-vis pytorch and take 3.6% of that and ask yourself whether you'd be happy passing up that kind of bonus/pay-bump/....lottery winnings. and of course for many pytorch workloads (smallish kernels in a highly multithreaded environment) i'm sure that number can be higher than 3.6%.

galdosdi• 2 years ago

Thoroughput vs Latency

klysm• 2 years ago

I don’t really agree with the characterization. That can matter at scale and the other thing to consider is latency

PeterZaitsev• 2 years ago

I think what would also be very interesting in this case is to get latency histogram, This 36ms per second can be 1000 stalls of 36microseconds which might not be significant for your app or it could be one giant 36ms stall... which can be an issue.

noident2• 2 years ago

Every Python developer has heard about the GIL

Sadly, that is not the world we live in.

I've cleaned up dozens of applications written by people with flawed understandings of threads, multiprocessing, and asyncio. I don't even blame the developers for this; it's a glaring language design problem.

If you need parallelism, Python is not the language you should reach for. Nobody ever takes my word for it until it's release day and the product is a broken pile of spaghetti code.

simonw• 2 years ago

How do you feel about the GIL free option coming in Python 3.13?

noident2• 2 years ago

It looks like it will be some time before free-threading can be safely used in production. I don't want to have to worry about whether the underlying C code supports no-GIL mode, or trade off single-threaded performance. [0]

Maybe someday this will make parallelism in Python at least as sane as other languages, but in the meantime, I still want to use a compiled language any time performance matters and wait for the kinks in no-GIL to be ironed out.

[0] https://docs.python.org/3.13/whatsnew/3.13.html#free-threade...

icedchai• 2 years ago

Same. I "inherited" an app doing crazy slow sync IO from their "async" functions... Sure, there was often some async IO mixed in, too, but what's the point?

ablekh• 2 years ago

Here is another post describing an alternative approach to instrumenting Python GIL: https://www.maartenbreddels.com/perf/jupyter/python/tracing/....

Instrumenting Python GIL with eBPF

Related Stories