Python performance is a fool's errand

http://whoosh.ca/blog/fools_errand?repost

by ntoshev • 17 years ago

18 15 17 years ago

timwiseman2• 17 years ago

I find this an interesting post, but I must partially disagree.

Performance is not always the the primary issue in programming. Using python I am much more worried about code readability, maintainability, and programmer speed than I am code speed. And Python certainly is designed in a way that makes true high performance somewhat challenge. If I were ever truly concerned about maximizing performance, I would use C++ or similar.

On the other hand, performance is always a consideration. It should never be totally forgotten about. For instance, if you will be using a single regex pattern multiple times in a program (or putting it in a loop) the performance increase from compiling it once is noticeable and does not subtract from any of the other attributes. Similarly, in many cases a list comprehension will be faster than an equivalent for loop.

Python performance is not a fool's errand, but I do think that people programming in python should realize they are making something of a trade off (unladen swallow and future research may partially change this in the future.)

(Edited to fix spelling typo.)

krakensden• 17 years ago

Performance is not always the the primary issue in programming

Except when it is, python rapidly becomes depressing. Try writing a game in python/pygame, for instance, and watch how quickly things get out of hand.

l0nwlf• 17 years ago

Well, I code python for ease of use and readability. Every language has its pros and cons. To be frank speed was never a forte of python but it was never meant to be as such. If you need speed there are options available, use them. However C have a faster run-time and slow development time. There are issues in all languages. However there is always a favourite language. Mine is python :)

timwiseman2• 17 years ago

True, some times performance is the primary issue, but not always.

When it is, Python is probably simply not the right langauge. Unladen swallow and future developments may change this, but for now if performance is a major issue than Python is not for you. C++ is generally a much better option for that, and there are a few that are even faster if you are willing and knowledgeable enough to do the optimizations.

mkramlich• 17 years ago

I wrote a 2D real-time spaceship shooter in Python with PyGame (called Ganymede, maybe a link from my site) and found the performance excellent. Most of the hard bits are done by native code or hardware anyway. Plus your application architecture has a big impact on performance. And number of polys or entities, etc. Perhaps trying to write a state-of-the-art 3D FPS would yield poor performance, is that more what you were talking about?

ambulatorybird• 17 years ago

I haven't kept up with Python developments, but the fact that almost everything is a hash table (even stack frames for interpreted function calls, I believe) would seem to place a relatively low upper-bound on performance. And this is in addition to the usual overhead of a straightforward interpreter.

stcredzero• 17 years ago

Stack frames in modern JIT VMs are just normal machine stack frames. When running in debugger mode, VisualWorks reifies the Object stack frames and runs the interpreter for that process as a program in the image. This way, you get speed for normal runtime, and access to everything as an object. This might seem like a bit of a stunt, but remember, the basic assumption for a JIT is that the JIT is a perfect simulation of bytecode interpretation. So if you have a valid JIT, you have a valid Debugger. This also means that anyone can extend the debugger, write tools based on it, or even write their own, as it's mostly an ordinary Smalltalk program.

Then again, there's a few bad side effects from this. You have to copy the Debugger and its infrastructure to debug anything used by the Debugger. So, for example, you can't debug ListViews in the debugger, because the debugger application uses them! (But copying that stuff, and renaming is not a big deal in Smalltalk. Plus, it makes you feel like an uber-hacker.)

viraptor• 17 years ago

I'm not sure if that's a low upper-bound. He's complaining about an attribute access speed - but he's not trying to fix it... missed a great opportunity for __dict__ usage.

From a quick py2.5 benchmark:

    noop 0.3s
    class_create 0.76s
    with_dict 1.68s
    without_dict 2.06s

Functions tested 1,000,000 times - noop is an empty function for comparison; class_create is just x=A(); with_dict and without_dict create a class each time, assign 3 attributes, then read the same 3 attributes. with_... does it on an object that lists the attributes in __dict__, the other one doesn't use __dict__ and assigns attributes in the constructor.

So... 1s (without the constructor and loop time) for 3,000,000 attribute reads and 3,000,000 attribute writes. Slow? Is he sure that it's the attribute read and write that's causing the problems? I'm not sure if I believe that without some more details - especially if he works with some documents that look like they're on some external storage (disk / db / whatever).

icey• 17 years ago

What's changed since you submitted this a few weeks ago?

http://news.ycombinator.com/item?id=684017

lacker• 17 years ago

What's changed is this time the url has "?repost" added to the end of it, presumably to get around the HN dupe filter.

ntoshev• 17 years ago

I felt it received undeservedly little attention due to the time of the day it was submitted. Sorry for the redundancy.

algorias• 17 years ago

Optimizing like this in python is clearly the wrong approach. Python will never be fast in the traditional sense.

The only optimization you should be worrying about is avoiding quadratic behaviour and such, which usually accounts for 99% of all inefficiencies.

If you really need speed, move the code to C and call it from python.

billjings• 17 years ago

First, I second the notion that switching to another tool is a good idea when performance becomes important. I was doing this by diving down into C years ago, and I have to imagine the toolset has only gotten better since then.

Second, you can make your generator method a coroutine and send it data as it iterates.

zokier• 17 years ago

Python is designed to be easily extended with C. If you need performance then write those performance-critical parts in C and write lightweight wrapper to python.

ivan_i• 17 years ago

Or write speed-critical Python iterator directly in C. Python source is the best reference: http://svn.python.org/view/python/trunk/Objects/dictobject.c... (look at the end).

Python/C API rules.

Python performance is a fool's errand

Related Stories