PyPy warmup improvements

http://morepypy.blogspot.com/2015/09/pypy-warmup-improvements.html

by hencq • 11 years ago

166 17 11 years ago

morepypy.blogspot.com

ngoldbaum• 11 years ago

Is it possible to get cython to build C extensions that are compatible with pypy? If not, is that something the pypy developers are interested in? A lot of the code I use depends heavily on C extensions, but it would be nice if I could the power of pypy's JIT compiler on the pure-python portions of my codebase.

dman• 11 years ago

You might want to edit that comment. I suspect "depends heavily on pypy" should be "depends heavily on C extensions".

msellout2• 11 years ago

AFAIK it's partially supported and under development. http://pypy.readthedocs.org/en/latest/faq.html#do-cpython-ex...

CuttlefishXXX• 11 years ago

Your link mentions CPython (the reference/official Python interpreter), not Cython (optimizing static compiler and extension of the Python language) so I'm not sure it answers the question raised by parent.

wirrbel2• 11 years ago

wouldn't it be feasible to write a Cython compiler that doesn't compile to CPython API but to the ffi that PyPy likes more?

srean• 11 years ago

...or to RPython to expose more surface area to the JIT optimizer.

BTW very curious of our login name. There used to be this type inferred language called Wirbel that compiled a subset of Python to native binary. Are you anyway associated ?

fijal2• 11 years ago

see my comment above - in short, very feasible, but someone has to care enough to do (I'm not volunteering)

fijal2• 11 years ago

Currently it should work via cpyext (CPython C API legacy infrastructure), but that's gonna be very slow

It's possible to write a Cython backend that would generate cffi bindings instead of C, but the work has not been done yet, so it's a bit unclear when it'll happen. Would be a good summer of code project for example.

empyrical• 11 years ago

Some more info/caveats of Cython's cpyext support can be found here:

http://docs.cython.org/src/userguide/pypy.html

https://github.com/cython/cython/wiki/enhancements-pypy

TheCondor• 11 years ago

Off topic, but you might consider flagging some bugs as good starter bugs for joining the project. You have great technology but it's a challenge diving in to it, even with the docs.

Animats• 11 years ago

If PyPy gets good enough, some of the need for C extensions should go away. Things such as "numpy" are written in C because numbers in Python are always boxed, that is, contained in a object. PyPy tries to figure out when that's not necessary, so it can use hardware arithmetic.

PyPy has to work very hard to do this, because you can muck with almost anything from anywhere with "setattr()" Even from other threads. So any guess PyPy makes has to be un-doable at any time. Python, the language, is what a naive interpreter such as CPython can do easily. In CPython, almost everything is a dictionary, and there are a lot of lookups. PyPy gets rid of much of that overhead. I'm enormously impressed that the PyPy crowd got it to work as well as they did.

DasIch• 11 years ago

These libraries are not just used because of performance. Some of them are quite old, large and partly written in Fortran. Unless you can do better in other ways than performance, you have to be quite a bit faster to make the effort of porting all these libraries worthwhile.

JulianWasTaken• 11 years ago

PyPy already is good enough for the need for C extensions to go away in a very large majority of cases.

chrisseaton• 11 years ago

Another option is to write an interpreter for the C extension source code that works like the Python interpreter in PyPy. This is what we do to support C extensions in JRuby+Truffle http://chrisseaton.com/rubytruffle/cext/.

PyPy warmup improvements

Related Stories