Faster CPython at PyCon, part one

https://lwn.net/SubscriberLink/930705/4bbe9c26d7884277/

by jwilk • 3 years ago

265 162 3 years ago

cfiggers• 3 years ago

Nooooo if Python gets faster then all the Mojo demos are going to sound so much less impressive nooo

npalli• 3 years ago

I don't know, looks like ~15% improvement [1]. Doubt the Mojo guys are going to lose much sleep (they claimed 35,000x improvement).

[1] https://speed.python.org/comparison/?exe=12%2BL%2B3.11%2C12%...

TwentyPosts2• 3 years ago

Somehow I'm still skeptical! Feels like "strict superset of Python, all Python code works" and "several orders of magnitude faster" just sounds like you're trying to have your cake and eat it, too.

I doubt that the Mojo developers have some sort of 'secret sauce' or 'special trick' that will get them there. And even if they have something, I don't see why the Python devs wouldn't just implement the same approach, considering they're currently trying to make Python faster.

I assume that (as long as Mojo wants to stick to its goal of being a strict superset of Python), there will be a lot of things that just cannot be 'fixed'. For example, I'd be surprised if Python's Global Interpreter Lock isn't entangled with the language in a few nasty ways that'd make it really difficult to replace or improve upon (while retaining compatibility).

Then again, the developer of Swift is working on it, right? I guess he's got the experience, at least.

tgma4• 3 years ago

That comparison seems quite cherry picked. Unlikely that it is generalizable. In my tests with Mojo, it doesn’t seem to behave like Python at all so far. Once they add the dynamism necessary we can see where they land. I’m still optimistic (mostly since Python has absolute garage performance and there’s low hanging fruit) but it’s no panacea. I feel their bet is not to have to run Python as is but have Python developers travel some distance and adapt to some constraints. They just need enough momentum to make that transition happen but it’s a transition to a distinct language, not a Python implementation. Sort of what Hack is to PHP.

KeplerBoy3• 3 years ago

I don't buy the mojo hype.

The concept of compiling python has been tried again and again. It has its moments, but anything remotely important is glued in from compiled code anyways.

klyrs5• 3 years ago

I've been rewriting Python->C for nearly 20 years now. The expected speedup is around 100x, or 1000x for numerical stuff or allocation-heavy work that can be done statically. Whenever you get 10,000x or above, it's because you've written a better algorithm. You can't generalize that. A 35k speedup is a cool demo but should be regarded as hype.

ehsankia• 3 years ago

35,000x suddenly becomes 30,000x, not as impressive!

Alex3917• 3 years ago

How does that reconcile with the benchmarks here, which say that Python 2.12 is currently somewhere between 5% slower to 5% faster?

https://github.com/faster-cpython/benchmarking-public

make32• 3 years ago

if it's an improvement that big it needs to be with GPUs, & gpus can be used normally with torch etc

traverseda2• 3 years ago

Meh, mypyc and codon both do the same thing as mojo. Codon seems to have similar speed improvements and the source code is available.

jjtheblunt3• 3 years ago

Do they compile to MLIR so when Alteryx writes a code generator you can target FPGAs, for example?

albertzeyer3• 3 years ago

JAX or torch.compile also do sth similar as Mojo, right?

brrrrrm2• 3 years ago

Once fused into super-instructions/adaptive instructions, does `disas` still allow users to manipulate the bytecode as if it were not optimized?

That'd be a very nice feature and I'm not sure it exists in other languages.

kmod2• 3 years ago

I haven't followed things recently but I know that at least initially they purposefully kept the bytecode intact so that existing debuggers and compilers and what not would still work. I believe they put the optimized bytecode in a different field, and accept this duplication as the cost of supporting the feature you requested.

Spivak• 3 years ago

Speaking of Python speed you gotta hand it to the core Python devs having some really dedicated numerical analysis folks. A really good gentle introduction talk https://www.youtube.com/watch?v=wiGkV37Kbxk

Python really has earned its place as the language of choice for scientific, research, and ML. I've stolen quite a few algos from CPython.

wheelerof4te• 3 years ago

"Speaking of Python speed you gotta hand it to the core Python devs having some really dedicated numerical analysis folks."

Well, they are Python devs after all.

albertzeyer3• 3 years ago

LOAD_ATTR_ADAPTIVE operation can recognize that it is in the simple case

How can it do that? The `self` variable could be anything, or not?

Sohcahtoa82• 3 years ago

If you define a .__getattr__ method, then "self.y" is no longer simple.

Jasper_2• 3 years ago

If it finds itself not in the simple case it specialized for, it reverts back to the unspecialized case. In JIT compiler terms, this is called a "guard"

albertzeyer3• 3 years ago

But isn't this the same as what LOAD_ATTR also does? It first checks the simple case, and if that fails, it goes the complex fallback route. Why does this need a new op? Or why would the standard LOAD_ATTR not do this?

Or are there multiple versions of the compiled function, and it modifies the bytecode per each call of the function?

JaDogg4• 3 years ago

Shameless plug: https://yakshalang.github.io/

This is my passion project. (Less activity these days as I'm sick)

mpenick• 3 years ago

I hope you feel better. This is a neat project and I’m looking forward to trying it out.

tpoacher• 3 years ago

+1 same here

JaDogg4• 3 years ago

thank you

abnercoimbre• 3 years ago

I'm impressed. Shameless plug to your plug: you may want to join the Handmade [0] community or attend our conferences [1]. You'd fit right in with us!

[0] https://handmade.network

[1] https://handmadecities.com

JaDogg4• 3 years ago

Nice. I'm already aware of handmade.network. I was plan on joining but lost track of it. handmadecities also look interesting. Will take a look. Might take until end of month.

TheMagicHorsey2• 3 years ago

Python's real strength at this point is the ecosystem. If you want speed you wouldn't use Python. Having said that, its better for the planet if default Python is more efficient. Everyone has seen Python code doing stuff Python code should not do at scale.

melenaboija2• 3 years ago

Thanks for the information, it has never been said before in HN when an improvement to Python happens and I am sure there is still someone out there thinking that Python is fast.

BTW from the PEP:

“Motivation

Python is widely acknowledged as slow”

jerf2• 3 years ago

There are a lot of people out there who interpret "Python is a slow language" as an attack, rather than just something that engineers need to know and account for at solution time.

There are also a non-trivial number of people out there who read about the constant stream of performance improvements and end up confusing that with absolute high performance; see also the people who think Javascript JITs mean Javascript is generally at C performance, because they've consumed too many microbenchmarks where it was competitive with C on this or that task, and got 500% faster on this and 800% faster on that, and don't know that in totality it's still on the order of 10x slower than C.

It is, unfortunately, something that still needs to be pointed out.

morelisp5• 3 years ago

Last year we had a team switch from Java to Python for what should be a high-performance API, because "they're really speeding up Python 3 these days."

It's now at least 2x off its original perf target and only the trivial case is being handled correctly.

I think reminding people is good.

TheMagicHorsey2• 3 years ago

No need to get testy. I know its widely known. The problem is, we still see Python in performance sensitive areas at scale anyway. So I said its a good thing.

I'm not trying to give anyone a wedgy by saying that. I thought I was just giving some justification for the optimization that OP describes.

mardifoufs2• 3 years ago

Yes we all know python is slow. But that's not a virtue of the language, it's not a feature that anyone really wants. We live with it but increasing speed is always a good thing. Which is why even Python's creator is making it his priority right now.

pjmlp8• 3 years ago

I want productivity and speed, it turns out that most dynamic languages learned to embrace JIT compilers, only the Python community so far has been against them, and appears to rather write C code instead.

Most attempts thus far haven't failed because it isn't possible, rather because the community hasn't railed around them.

Smalltalk, SELF, CL, Dylan, Lua have shown how it is done, Julia, JS and Ruby are following along, only Python keeps resisting it.

d0mine• 3 years ago

In practice, Python is very often fast enough. Mostly because hotspots are implemented in another language e.g., if you use Python to multiply matrices, then something like numpy would use blas (C, Fortran) or similar under the hood. Your handwritten code in any language will have a hard time to beat blas in performance.

titzer2• 3 years ago

I like tinkering with interpreters, and I do it a lot, but this is chump change compared to even a baseline compiler, especially one that makes use of type profiles. There is a pretty big investment in introducing a compiler tier, but I'm not convinced that it's less engineering effort than squeezing more out of an interpreter that fundamentally has dispatch and dynamic typing overhead. There aren't that many bytecode kinds in CPython, so it seems worth doing.

tekknolagi3• 3 years ago

That's what we did in Skybison before it got wound down: https://github.com/facebookexperimental/skybison

titzer2• 3 years ago

Are there publicly-available performance results?

mardifoufs2• 3 years ago

What's the difference between that and cynder? I know they both come from Meta/IG but I wonder if cynder is an evolution from skybison or is it more of a different approach?

Seattle3503• 3 years ago

Small improvements to cpython are multiplied out to the millions of machines cpython is installed on by default.

AbsoluteCabbage2• 3 years ago

I prefer C without braces

low_tech_punk• 3 years ago

When will such goodness come to JavaScript engines? I’m curious if there is some fundamental design flaws that made JavaScript difficult to optimize.

Jasper_2• 3 years ago

JavaScript engines have been using inline caches and specialization for over 10 years now. It's Python that's catching up to JavaScriptCore from 2007.

brrrrrm2• 3 years ago

I wonder if the parent comment was made tongue-in-cheek :)

pjmlp8• 3 years ago

Or catching up to Smalltalk/SELF/CL from 1980's, actually.

formerly_proven• 3 years ago

I mean JavaScript is definitely gross to optimize, if you'd design a language for easy optimization and jitting it would look different. Python is just way worse, especially because of the large and rich CPython C API, which used invasive structures almost everywhere (it has been slowly moving away from that). The CPython people, GvR in particular, always considered it very important for the codebase to remain simple as well, and it is. CPython is a really simple compiler and straightforward bytecode interpreter.

esaym• 3 years ago

Node.js has been on par with Java for many years now due to the Jit and tons of money google has pumped into that ecosystem: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

shepardrtc• 3 years ago

Java: 2.52s Node.js: 6.37s

I don't know if I'd call that on par.

kmod2• 3 years ago

Funny that you say this, the more common view I've heard is "we made JS way faster why can't we do that for python". I believe I heard Mark Shannon (one of the lead devs on Faster CPython) say this at one point.

AbsoluteCabbage2• 3 years ago

I love me some Python, trust me. But this is just putting lipstick on a pig. The language needs to be rewritten from the ground up. If you’ve ever analyzed the machine code a simple python script spits out you’ll see how helplessly bloated it is.

leetrout• 3 years ago

Do you have thoughts around the concept of Python becoming a Rust macro based language? I read another comment about that being a potential future for Python but I don't know how feasible it is.

I've moved on to Go for things where I need to care about performance but like thinking about things in a similar way Python lets me think about them.

tredre3• 3 years ago

If you’ve ever analyzed the machine code a simple python script spits out you’ll see how helplessly bloated it is.

How would one do that? I thought python generated bytecode and interpreted that code? Where's the machine code? Do you mean dissembling the python interpreter itself? In which case it would be gcc/llvm spitting out machine code, no?

__d3• 3 years ago

You can make a Python-ish language that can run fast (or at least, a lot faster) but it cannot be Python: the dynamic nature of the language means many optimizing techniques just aren't available.

The unfortunate thing is that the vast majority of Python code in use today doesn't need all those super-dynamic features. So it would run just fine on a cut-down JITed interpreter. But there's always the corner cases.

In retrospect, Python 3 was a lost opportunity here: it could have broken more compatibility, enabled multi-core and JITs, and then the 10 year transition pain would actually have been worth it. But that's hindsight, of course.

chlorion4• 3 years ago

the dynamic nature of the language means many optimizing techniques just aren't available.

This is something that seems like it should be true, but counter evidence exists that proves that it's not the case.

The first example would be V8, the JavaScript JIT compiler used in Chrome and NodeJs (and probably other things). V8 is many times faster than CPython in pretty much every situation.

The second, and even better example is SBCL, a Common Lisp compiler. SBCL is quite a bit faster than CPython and V8, it's closer to the JVM in terms of performance in benchmarks that I have saw.

The third example would be some of the Scheme compilers, like Chez and Gambit, which are not far off from SBCL.

Maybe you could argue that JavaScript is not as dynamic as Python. I don't know JavaScript at all so maybe that is the case.

I'm pretty sure that Common Lisp and Scheme are not less dynamic though. I think Common Lisp is actually more dynamic but I don't have any way to measure this, so it's just my opinion.

So assuming these languages are as or more dynamic than Python, this seems to be proof that Pythons dynamic-ness is not the reason for it's poor performance!

The Lisp compilers are also much less widely used and have much less engineering power available!

I think these counter examples are pretty interesting and don't know exactly what to make of it. Python has more funding and more users to contribute to it (except in the case of V8), I guess until now they just haven't put any of that into performance.

pjmlp8• 3 years ago

Besides the sibling comment, Smalltalk and SELF JITs prove otherwise.

Ongoing Ruby efforts as well.

robertlagrant2• 3 years ago

It would be cool if they'd put some resource into Pypy as well. Feels like that project is such a great achievement, and with just a bit more resource it would've been an amazing alternative. As it is, it takes a long time to get the new language features out, which always hamstrings it a little.

adamckay• 3 years ago

There's a limited number of volunteers with the experience to improve (C)Python, though, so the resources are better spent on improving the CPython implementation to such a point where PyPy is no longer necessary.

girvo• 3 years ago

See: HHVM and PHP. HHVM is what lit the fire under the PHP team's collective butts to improve perf, and now most just run the mainline PHP interpreter (well, they did years ago when all this happened and I was still doing PHP)

qaq• 3 years ago

Now there is a number of payed MS employees lead by Guido van Rossum working on improving (C)Python

make32• 3 years ago

the userbase of pypy is a tiny fraction of cpython, likely not worth it

v3ss0n5• 3 years ago

PyPy is very much compatible with cpython, you can just change interpreter and see it work. ( I. Many cases)

robertlagrant2• 3 years ago

I think that's because when new codebases are written, pypy isn't on the latest version of the Python spec, so people choose CPython. If it got a little more support it could get to parity or n-1, and far more projects would start with it and stick with it.

I don't use it myself, I just see the relatively small difference in investment it would be between a great project that's slightly languishing in obscurity despite incredible talent and effort, and a genuine full alternative to CPython.

mkoubaa• 3 years ago

The future of pypy is with hpy, and that is direction cpython is going in, too

freecodyx• 3 years ago

Always keep in mind that you can’t go faster than what the harware can allow. Sometimes even by what the OS can allow. We can do way better on some specific tasks. But for for that you need to rewrite some algo implementation to better take advantage of all the hardware can allow. ML is a specific task, if you can take advantage of the harware to do better matrix mul. Does not mean you can have the same speed ups overall. Mojo for example

pipeline_peak2• 3 years ago

Porting to WASM and getting C bindings to be compatible on steady well developed VMs will make Python faster.

jjtheblunt3• 3 years ago

How, specifically?

pipeline_peak2• 3 years ago

Because C code in web assembly can translate to native machine code and still interop with Python.

All this PyObject nonsense does nothing but slowdown the runtime. V8 was built for performance from day 1. CPython is old, slow, and will always be outperformed by V8 regardless of the tweaks this team does.

I am not a compiler engineer, I can’t give you a specific, technical answer.

pjmlp8• 3 years ago

So basically following JVM footsteps and CLR in regarding to specialization bytecodes, e.g. invokedynamic would be one of them.

jwilk• 3 years ago

Part two: https://news.ycombinator.com/edit?id=35919949

Faster CPython at PyCon, part one

Related Stories