Numba: NumPy-aware optimizing compiler for Python

by pash • 14 years ago

64 23 14 years ago

textminer• 14 years ago

Big user of Scipy Sparse. Anyone know if this is compatible?

alexenko• 14 years ago

... It uses the remarkable LLVM compiler infrastructure to compile Python byte-code to machine code especially for use in the NumPy run-time and SciPy modules... from the README on github

montecarl• 14 years ago

Here is a mailing list thread with the results of a simple benchmark: https://groups.google.com/a/continuum.io/forum/m/#!msg/numba...

Looks promising!

bravura• 14 years ago

The github README (https://github.com/numba/numba) explains how to install Numba, but not how to use it.

Or is it used automatically for all Python byte code? If so, can it be disabled?

shoo• 14 years ago

the tests suggest that it is only used where you tell it to be used: https://github.com/numba/numba/tree/master/tests

pash4• 14 years ago

The main way to use it right now is by decorating the functions you want it to compile. See the examples directory [0] in the Github repository.

0. https://github.com/numba/numba/tree/master/examples

fosap• 14 years ago

And the PyPy authors told us that Python would be to dynamic for ahead of time compilation. This is awesome.

fijal6• 14 years ago

this is not a python compiler. it's a subset of python compiler (this exercise has been done already quite a few times)

buster• 14 years ago

How good does this work with other python modules? Can i use this to compile _every_ python program to native executables?

andreasvc2• 14 years ago

No.

In any case, doing that wouldn't be difficult, you just wrap a Python interpreter and the Python code in a single executable and you're done. What would make it worthwhile is when it would be an optmizing compiler, which is of course difficult for such a dynamic language.

fijal6• 14 years ago

So maybe to clarify few things. This is just sensationalism - numba is not a python compiler. It's a compiler of some restricted subset of python into LLVM. I don't think the subset is very well defined by now, but I would expect it to be at some point in the future. It's just that you can choose what to compile and what not, which is very convinient for a lot of numerics.

The whole approach might be viable and it definitely has use cases, however the sensational headline makes it so bad :/ Hackernews - you let me down.

pash4• 14 years ago

You mean sort of like how PyPy is a JIT for only a restricted subset of Python?

fijal6• 14 years ago

No, PyPy JITs all the Python by design. It also runs all the Python and JITs all the constructs. It does not make all the possible python magically faster however, but that does not change my point. A JIT for only a restricted subset of Python would mean that it segfaults/crashes/gives a wrong answer if you don't provide a correct subset. This is not true.

andreasvc2• 14 years ago

That restricted subset, RPython, is what PyPy itself is implemented in; but PyPy can run arbitrary Python code.

kkuduk• 14 years ago

here is a talk form SciPy 2012 about Numba http://www.youtube.com/watch?v=WYi1cymszqY

juanlu001• 14 years ago

Here is the post at Travis Oliphant's blog about numba

http://technicaldiscovery.blogspot.com.es/2012/08/numba-and-...

As it is said there, it's still early software and its "road-map is being defined right now by the people involved in the project". Sure the subset is not well defined now and there are no docs, but hey - let's give the thing a few months.

Haven't tested thoroughly yet, but I think no NumPy calls can be made from inside a numba-compiled function - so this might be the case for other Python modules as well.

travisoliphant• 14 years ago

Actually some NumPy calls can be made inside the numba-compiled function as there is nascent support for the NumPy C-API --- it translates the call to a C-call.

In the future, we want to support all Python, but not necessarily optimized --- just using the Python Object C-API.

Numba: NumPy-aware optimizing compiler for Python

Related Stories