PyPy funded to begin support for Python 3 and Numpy

http://morepypy.blogspot.com/2012/01/py3k-and-numpy-first-stage-thanks-to.html

by synparb • 14 years ago

178 30 14 years ago

morepypy.blogspot.com

ColdAsIce2• 14 years ago

I like pypy and its ambitions, last time I tested it, about a month ago it was very speedy and startup time considerable faster than Cpython. However the regex exercises I wanted to do couldnt be done. Pypy seemingly didnt have a good regexp engine. If I remember correctly, something with groups and backwards-reference...

Has that changed?

bitcracker• 14 years ago

The fundamental question is: How compatible will PyPy ever be? Which kind of applications can be run with PyPy?

http://pypy.org/compat.html mentions compatiblity according to the standard library. This is fine for (web) servers and command line applications.

But what about desktop applications? Can I (someday) take a PyQt or PyGTK code and compile it with PyPy without modifications?

dripton• 14 years ago

Maybe.

cpyext is a hack to let PyPy run Python/C API modules. It works for some, but not all, and it's slow. I'm sure it'll keep getting better, but not sure if it'll ever be good enough to run PyGTK or PyQt.

PyPy has good support for ctypes, so ctypes bindings are a good option. There are projects out there like pygir-ctypes and ctypes-gtk. One of them just needs to become complete enough to be a good choice for GTK programming. Compatibility with new PyGObject is more likely than compatibility with legacy PyGTK, though.

PyQt is harder because it's C++.

DasIch• 14 years ago

I believe that C extensions, which have always been and still are very CPython specific, will have to be replaced with ctypes and possibly Cython versions eventually.

I expect to see quite a lot of improvement in that area as a side effect of the work on numpy.

kingkilr3• 14 years ago

No, not much has changed about our regex engine in the last month. However, our `re` module is fully compatible with CPython's so I'm bit confused, are you saying it didn't run something, or was slow?

ak217• 14 years ago

Hi, could you give some details about those regexes that you were trying to run? Are you sure these are problems with the PyPy implementation, not with the Python regexp spec itself? I've used grouping and back-references with pypy with no problems so far, so I wonder which case you're talking about.

ColdAsIce2• 14 years ago

Yes I am sure it was the pypy implementation since the same regex would do fine on python2.7.

Unfortunately I dont have the regex at hand, just remembering it was something with groups and backtracking.

andrewcooke• 14 years ago

i have a pure python regexp implementation that you can use in pypy (and it passes almost every test used by the cpython re package - only the LOCALE flag is not supported). however, it's around 100x[1] slower that the re package (on simple matches; it's not backtracking so it can be similar in speed for pathological cases).

also, when i looked at pypy the re implementation was in c and looked so similar to cpython's i assumed it was the same or closely related. but i didn't pay much attention, so i may be mistaken.

[1] roughly. it actually runs 6x faster on recent pypy that cpython.

EdM2• 14 years ago

This post made me think of this article: http://technicaldiscovery.blogspot.com/2011/10/thoughts-on-p...

For those who don't know, Travis Oliphant is the creator of NumPy and(?) SciPy.

It's a good read and it puts some of the issues with a port into perspective.

fijal6• 14 years ago

Hey.

While I generally grossly disagree with Travis in regard to how far PyPy can go I wonder what kind of perspective are you talking about?

EdM2• 14 years ago

I think this statement sums up the perspective I'm talking about:

"NumPy is just the beginning (SciPy, matplotlib, scikits, and 100s of other packages and legacy C/C++ and Fortran code are all very important)"

I'm not that familiar with matplotlib and not familiar at all with scikits. But, the point is that there is a lot of other C/Fortran code that users of NumPy rely on. How much do you gain by porting NumPy to PyPy? (Not a rhetorical question... I'm genuinely curious why the PyPy folks have chosen this as a goal?)

PyPy team, if you're out there, please don't take my question as criticism -- it's not. I'm just genuinely curious. Congrats on getting the funding and keep doing what you love!

teoliphant2• 14 years ago

Fijal, I'm not sure you even understand my perspective. None of your comments in response have given me confidence that you do. I have no disagreement with you about how far "PyPy could go". Obviously, we could re-create the entire Python ecosystem including the scientific stack under its run-time. I just question your understanding of how expensive and time-consuming that would be. My official position is that I think it's possible to get the benefits of PyPy (i.e. fast Python loops) in different ways that don't also toss out years of extension modules in the process and whose answer to current users about the features they rely on in CPython not working under PyPy is "just port it" or "just use ctypes".

PyPy funded to begin support for Python 3 and Numpy

Related Stories