A brief experiment with PyPy

http://lwn.net/SubscriberLink/442268/22f66371348bd7c5/

by jnoller • 15 years ago

83 23 15 years ago

lwn.net

levesque• 15 years ago

Can't wait to have a version of PyPy that supports numpy!

Since the benefits have been proven, I can't help but wonder why there aren't more people working on this...

bobbyi• 15 years ago

People are working on it. For example, Quora is having Alex Gaynor help move their site to PyPy this summer: http://alexgaynor.net/2011/may/06/this-summer/

fijall• 15 years ago

The main reason why more people are not working on this is that:

* volunteers are interested somewhere else

* nobody is willing to put money into numpy on pypy happening

true_religion5• 15 years ago

I think part of the issue is that even PyPy + NumPy wouldn't give you compelling gains versus NumPy + Cython in CPython.

The big win of PyPy vs. Cython is that you get a 10x or so speed improvement without having to specify types, but if you're using NumPy already, then its pretty standard to specify types already so NumPy can optimize. At that point, you really have to ask yourself if it isn't worth it to just build a Cython module and get next to C speed for that hotspot.

The great thing about Cython is that it lets you call inbetween C and Python at little overhead without having to write any code for the Python C-API like you would if you were using CTypes or even Boost::Python.

treo2• 15 years ago

That 3x speed up is about the same that I have seen with my code. I'm currently writing a database cache simulator to try different algorithms with it, and if I want to have anywhere near realistic results I have to use realistic access traces.

Tried it today with a tpc-c trace which has about 500 million accesses. The result: CPython would have run for about 90 minutes (I stopped it after 30 minutes, and began to look for a speedier possibility), PyPy only took 22 minutes.

true_religion5• 15 years ago

I've gotten about a 10x speed up on numerics code where there's so much branching involved in the calculations that I can't afford to use NumPy.

As for me, the main reason I haven't moved to PyPy yet is the lack of database and messaging support.

kingkilr5• 15 years ago

Which databases? At the moment we have SQLite, Oracle (haven't tested it myself), and Postgresql. Plus whatever you can find a pure python driver for. Also, what do you mean by messaging?

true_religion5• 15 years ago

Do you use TPC-UVA? http://www.infor.uva.es/~diego/tpcc-uva.html

treo2• 15 years ago

No, tpc-uva is a bit more than what I need right now. I might use it later on when I have decided on any single algorithm that I want to test in a more realistic environment. Because changing the caching algorithm that postgres uses isn't as easy as doing so in a standalone python simulator, I will have to be sure that I want to do that. I have already tried that before and it is a lot harder and takes a lot more time.

TheBoff• 15 years ago

This is really really great to see: pypy is such an interesting project, and it's really encouraging to see it make so much ground.

It's interesting that Guido deliberately didn't go for a full re write for python 3, but this project which is a full rewrite in a whole different language has provided a faster implementation with less developers!

sho_hn3• 15 years ago

To be fair, it also needed eight years to get here, PyPy inherits much standard lib code, and the py3k effort wasn't limited to the interpreter but also involved a lot of standard lib development :-). So, apples and oranges. Still, amazing work by the PyPy folks any way you look a it.

sho_hn3• 15 years ago

Can anyone comment on startup latency and performance early in a run?

One of my use cases for Python is relatively small, short-running scripts, and JIT engines often take a fair amount of runtime before all the optimizations kick in. So I wonder how PyPy does at startup time and whether it's able to leverage its execution speed prowess over brief runtimes - is it still a net performance win in the end, or at least not-worse-than-CPython?

kingkilr5• 15 years ago

Last I checked we startup faster than CPython. As for early-run-performance. Startup really depends on the total amount of code you have, if you've got a few hundred lines of code the JIT's often warmed up and fast in under half a second. On the other hand if you have a few hundred thousand lines it might take a minute to warm up.

sho_hn3• 15 years ago

Faster startup sounds nifty :-).

How's performance while the JIT is not yet warmed up, compared to CPython?

A brief experiment with PyPy

Related Stories