How fast can we make interpreted Python?

by heydenberk • 13 years ago

54 25 13 years ago

kghose• 13 years ago

From the paper: http://github.com/rjpower/falcon/

Does Falcon support all of Python?
Not yet! Lots of constructs (like catching exceptions, constructing objects, etc...) aren't implemented in the Falcon virtual machine.

phonon• 13 years ago

From the project page, https://github.com/rjpower/falcon/

"... However, this doesn't mean that programs which use these constructs won't run. Any missing functionality is routed through the Python C API, foregoing any potential performance benefit you might have gotten from Falcon. So, though Falcon isn't a complete Python implementation, it should still run all of your code."

bayesianhorse• 13 years ago

The GIL must go!

DanWaterworth• 13 years ago

I'll get rid of the GIL for you. You're not too attached to mutating state, are you?

willvarfar• 13 years ago

I wish pyscho was ported to 2.7 64 bit :(

erdewit2• 13 years ago

I totally long for Psyco too. PyPy turned out to be not really a replacement for it.

chubot• 13 years ago

Oh really, why? Because it's not as compatible?

wyuenho• 13 years ago

While this paper is quite easy and occasionally funny to read, being a total math idiot, I need somebody to help me out how to read the benchmarks at the end. The author claims that on average converting stack-based bytecode to register-based bytecode results in an average of 25% performance improvement, I have trouble finding where that number comes from. The charts are said to be a comparison of optimized code relative to unoptimized code, my question is, how come the unit on the y-axis is a percentage, and the unoptimized code isn't used as a base-line all labeled as 100% or given an absolute average time-took? The tiny gaps between unoptimized and optimized code are confusing me.

maxerickson2• 13 years ago

The baseline in the charts is standard CPython. The unoptimized is their register machine without optimizing the bytecode.

Demiurge• 13 years ago

Since it's compatible with PyObject, sounds like it can be folded into CPython? Are there any arguments against that?

maxerickson2• 13 years ago

The maintenance effort.

The big performance win is nice (this benchmark:

https://github.com/rjpower/falcon/blob/master/benchmarks/mat...

), but it is still going to be a lot slower than doing it the horrible way (calling out to Numpy).

The benchmark is more aimed at demonstrating the reduction in loop overhead than it is at numerical speed, but numerical stuff is usually where you end up with a lot of loop overhead...

freyrs3• 13 years ago

It'll never happen, CPython is the reference implementation and a change as drastic as rewriting all of ceval.c is too drastic for any point release.

albertzeyer• 13 years ago

Previous discussion:

http://www.phi-node.com/2013/06/how-fast-can-we-make-interpr...

https://news.ycombinator.com/item?id=5943258

jballanc• 13 years ago

It seems like the biggest point here is that compatibility with the extension API in Python is the anchor dragging down performance. Similarly, Ruby suffers from continued compatibility with its extension API. In the paper they compare with Lua and JavaScript, but it's worth noting that JavaScript doesn't have an extension API (well, JavaScript proper...not saying anything about node.js), and the team behind Lua are notorious for not caring about breaking API compatibility between versions.

I guess what I'm trying to say is, while this is a laudable effort, it seems what Python (and Ruby) really needs is a way to free itself from the chains of extension API compatibility.

justincormack• 13 years ago

Lua versions that break API compatibility are only every few years, but not keeping compatibility forever is sensible.

ProblemFactory3• 13 years ago

On the other hand, having a stable C extension API is what allows Numpy and Cython (http://docs.cython.org/src/quickstart/cythonize.html) to be as good and fast as they are.

My experience with numerical algorithms in Python has been a 10000x speedup and 10x memory reduction by using Numpy arrays and C/fortran functions. And when Numpy doesn't have a necessary function, a few extra lines of static C types and on-import compilation with Cython has still resulted in a 1000x speedup.

"Optimizing Python" needs to look at the whole existing ecosystem of C libraries - to be useful the end result has to be faster than what's possible now, not just faster than pure Python.

tomrod2• 13 years ago

What exactly is Cython? I've not had a reason to use it yet; are the speed improvements worth it?

lettucecrisper• 13 years ago

It would be good to compare Falcon with Numba: "Numba’s job is to make Python + NumPy code as fast as its C and Fortran equivalents without sacrificing any of the power and flexibility of Python." Like Falcon, Numba is compatible with CPython and whatever extensions you want to use with CPython.

https://github.com/numba/numba

Intro to Numba, parts 1 and 2:

http://continuum.io/blog/numba_growcut

http://continuum.io/blog/numba_performance

mlubin• 13 years ago

If the goal is to get performance without giving up on Python's existing libraries, why not use Steven Johnson's PyCall (https://github.com/stevengj/PyCall.jl) package for Julia?

ngoldbaum• 13 years ago

No need to learn a new language, perhaps.

How fast can we make interpreted Python?

Related Stories