Unladen Swallow 2009Q3 Released with LLVM optimized Python

http://code.google.com/p/unladen-swallow/wiki/Release2009Q3

by yarapavan • 17 years ago

127 35 17 years ago

code.google.com

mbrubeck2• 17 years ago

"2009Q3 uses up to 930% less memory than the 2009Q2 release."

Does that math make sense in some way I'm not seeing?

sparky3• 17 years ago

It's unconventional, but 9.3x less memory can be seen in some sense as 1/9.3=10.7 percent as much memory. Not defending it, just trying to interpret :P

sp332• 17 years ago

You can't have 9.3x less memory, either. You can have (1/9.3)x as much memory, or they could just say 10% as much or 90% less.

iclelland• 17 years ago

Actually, it's 10.3x less memory (they've subtracted 1), so it's using 9.7% as much memory as the previous release. I would have said that 90.3% less than before, but maybe that's just me

wisesage5001• 17 years ago

From their benchmarks rietveld: $ ./perf.py -r -b rietveld --args "-j always," --track_memory ../q2/python ../q3/python Mem max: 575116.000 -> 55952.000: 927.87% smaller

This 930% basically means that 2009Q3 can use as little as ~1/10 of the memory of 2009Q2.

jimbokun• 17 years ago

"Uses 1/10 the memory" would have been much easier to parse.

ErrantX• 17 years ago

I never checked but I assumed it meant that the memory savings of Q3 compared to the savings of Q2 increased 930%

mbrubeck2• 17 years ago

There's no "savings" vs. CPython: Memory usage is still 2-3x that of Python 2.6.1.

(Edit: Or did you mean that the 2009Q2->Q3 savings was 930% greater than the Q1->Q2 savings?)

simonw• 17 years ago

Is it noticably faster than regular Python yet?

sparky3• 17 years ago

Their first release (2009Q1) consisted mostly of patches to mainlines CPython, which improved performance for everybody by roughly 10%. They've added about 10-15% performance on each of the last 2 releases in Unladen Swallow-specific code, but it's unclear (to me) how CPython performance has evolved over the last 6 months. They haven't compared explicitly since 2009Q1, which may or may not be revealing.

From the release notes:

Lowlights:

    * LLVM's JIT and other infrastructure needed more work than was expected. As a result, we did not have time to improve performance as much as we would have liked.
    * Memory usage is still 2-3x that of Python 2.6.1. However, there is more overhead that can be eliminated for the 2009Q4 release.

The last 2 releases have been mostly building up infrastructure; hopefully most of that is behind them and they can work on getting the 5x performance they're after.

bobbyi2• 17 years ago

Do you know whether those Q1 changes have actually been merged into CPython? Will they be coming in 2.7?

cool-RR2• 17 years ago

Firstly let me say I think this is a wonderful project and one of the most important Python projects I know.

Something I noticed though, in the Project Plan page:

http://code.google.com/p/unladen-swallow/wiki/ProjectPlan#20...

They state that one of the goals for "2009Q3 and beyond" is "to remove the GIL and fix the state of multithreading in Python." Which I think is an awesome and ambitious goal that I would find very useful as will many others.

So I hoped the GIL thing would fall into the "2009Q3", but apparently it's in the "beyond" category. Anyway, they need to update that project plan.

kingkilr3• 17 years ago

If you head over to python-ideas there's actually some really good discussion about removing the GIL right now. Someone has proposed using CAS operations (which I know nothing about) to remove the fine grained locking that is needed under the refcounting GC without the GIL. In addition Antoine Pitrou has been working on a patch to make the GIL more performant on POSIX systems.

sb2• 17 years ago

jfyi: CAS is the abbreviation of Compare-And-Swap, which are dedicated native machine instructions that offer atomicity and therefore simplify locking.

tlb• 17 years ago

My big python app (the robot end of a telepresence connection) suffers from GIL problems. Mostly, threads are used for C-only stuff like video compression so they can easily release the GIL while running. But I gotta say, the complexities of GIL correctness, thread performance, gtk/gobject, callbacks, boost.python, GC-correctness of both Python and C++ objects, and polling make my head hurt. Simplicity would be a find goal for the new system.

cheriot• 17 years ago

Is there a good python message passing implementation? Some problems fit message passing more naturally than threading.

beambot• 17 years ago

If you're in the robotics space, you should seriously consider the Robot Operating System: http://www.ROS.org

You'll find their publish-subscribe message passing is quite simplistic, and the support is great.

bdr• 17 years ago

They mentioned this at the recent SF Python Meetup. They had found an academic paper that they thought they could apply to Python to get rid of the GIL, but it turned out someone had already tried it unsuccessfully. I think that goal of unladen-swallow is definitely "beyond" for now.

clopey2• 17 years ago

Has anyone got this working on Windows?

rbanffy• 17 years ago

I am considering giving it a try under Cygwin. If you get there first, please, tell us.

clopey2• 17 years ago

I did try under cygwin, and failed with errors in llvm. Will let you know if there's any more progress though.

maksimka• 17 years ago

What's the plan for Unladen Swallow to support py3k?

Unladen Swallow 2009Q3 Released with LLVM optimized Python

Related Stories