DSP Performance Comparison Numpy vs. Cython vs. Numba vs. Pythran vs. Julia

https://jochenschroeder.com/blog/articles/DSP_with_Python2/

by cycomanic • 6 years ago

3 18 6 years ago

DNF27• 6 years ago

That Julia benchmark is really quite unfortunate:

1. It treats all arrays as row-major, while Julia arrays are column major. That is pretty unfair. 2. It creates lots of unnecessary copies, as slices in Julia does not return views (but you can easily make them do that.) 3. It creates an output array with an abstract element type, `zeros(Complex, pols, L)`, which can seriously hurt performance. 4. Benchmarking should preferably be done with BenchmarkTools.jl, though I'm not sure it that makes a big difference here.

It's ok to be a novice user of a new language, but when comparing and publishing benchmarks like this, I really think the author should solicit some feedback to avoid obvious rookie mistakes. I found no way of offering any feedback on that blog, however.

cycomanic6• 6 years ago

Fair enough, as I said my experience with julia is limited.

I did ask for feedback (there is a link to the repository under code, but I saw that I accidentally removed it in one push to the repository, so it might have not been there when you looked).

I've just checked with your suggestions and I get a 2x improvement by going to zeros(Complex{Float64}, pols, L) and a bit more improvement by changing the calculations to column major to result in an overall improvement of about 60x compared to numpy (using @views did actually result in a slow down). I will update the post, thanks.

DNF27• 6 years ago

That's good. I still cannot see the repository, though. I may be looking in the wrong place.

It's no problem being new to a language, I just found the conclusions as bit too 'conclusive', so to speak, in that case.

Views should definitely not slow this down, so there may be something off. Could you perhaps share some dummy input arrays? Can I generate them with rand, perhaps, if you can tell me the sizes?

dklend1222• 6 years ago

Which version of Julia did you use? @view didn't actually speed up code until 1.5, where they got stack allocated.

dklend1222• 6 years ago

Also, the first run of "@time" includes compilation. So for an accurate runtime, you'd want to run it twice and use the second timing.

cycomanic6• 6 years ago

Doesn't the run before using @time take care of that?

DNF27• 6 years ago

Yes it does. There are still issues with benchmarking in global scope, though, which is why it is better to use BenchmarkTools.jl

amkkma2• 6 years ago

Sorry- missed that

amkkma2• 6 years ago

Are the Cython and Pythran codes running in parallel? To do that with Julia: https://docs.julialang.org/en/v1/manual/multi-threading/

Or https://github.com/JuliaFolds/FLoops.jl

DSP Performance Comparison Numpy vs. Cython vs. Numba vs. Pythran vs. Julia

Related Stories