The Python scientific stack, compiled to WebAssembly

https://github.com/iodide-project/pyodide

by tosh • 8 years ago

173 52 8 years ago

github.com

mr_toad• 8 years ago

Part of an effort to make a Jupytr notebook-like environment without relying on server side code.

https://github.com/iodide-project/iodide

xtrapolate2• 8 years ago

"Part of an effort to make a Jupytr notebook-like environment without relying on server side code."

So there won't be any server side code, you'd just be downloading the entire Python stack and dependencies every time you want to use this through your browser. This time it will be WASM though, instead of x86/x64/pyc.

That we can, doesn't mean that we should.

m_mueller2• 8 years ago

I guess we need an in-browser package management like npm?

tomtimtall• 8 years ago

That would be dumb, so why assume that would happen? You don’t download every picture on every refresh.

zitterbewegung• 8 years ago

It would work on a Chromebook and a iPad also.

And yes I could pay for a Jupiter notebook hosted on some instance but I dont want to.

Vinnl2• 8 years ago

Well, you'd download the entire stack indeed - but not every time. Your browser does caching too.

andybak• 8 years ago

I can think of several scenarios where this could be useful.

sp332• 8 years ago

There's an advantage even if you're running it locally. Now you can just load up some static HTML + JS files in your browser instead of spinning up a server and then pointing your browser at localhost.

Waterluvian• 8 years ago

As long as it's framed properly it can be acceptable. Ie. You get to a page with details and a "load environment" button with progress bars and all.

Of course when it's like 50-100MB you wouldn't use it in some cases. But once cached you're golden for your repeat visitor audience.

krona• 8 years ago

Coming soon [...] Plotting using D3 from Python

Wouldn't Bokeh make more sense? I'd be very interested in a serverless Bokeh that didn't force me to ditch Python, for example.

mdboom6• 8 years ago

Bokeh could probably be done, but most of Bokeh's client/server complexity becomes unnecessary in this execution model.

More recently, pyodide has grown full matplotlib support. See, for example, https://iodide.io/pyodide-demo/matplotlib-sideload.html?side...

amelius2• 8 years ago

Is there a performance hit compared to "native" Python?

drchickensalad• 8 years ago

Without a doubt.

lkbm• 8 years ago

There's a chart and some discussion here: http://droettboom.com/blog/2018/04/04/python-in-the-browser/ (and some followup at http://droettboom.com/blog/2018/04/11/profiling-webassembly/ )

TL;DR: pyodide in Firefox was slower than cpython.

amelius2• 8 years ago

Thanks. Note that the author of that post said:

UPDATE 2018-04-11: My hunch was wrong, and I was able to get to the bottom of the root cause and significantly speed up these benchmarks.

And there's an updated graph here:

http://droettboom.com/blog/2018/04/11/profiling-webassembly/

goatlover• 8 years ago

Certainly for anything using Numpy.

mdboom6• 8 years ago

Actually, Numpy-using-code is closer to native speeds than pure Python is.

The thing that seems to cause a greater gap between wasm and native speeds is lots of Python-level function calls, not the tight C loops that make up much of Numpy.

nerdponx• 8 years ago

What happens to all of the C code in Numpy, and the calls to external libraries like BLAS?

Scarblac• 8 years ago

In particular, does this mean Javascript has access to Numpy? Sounds unbelievable, but would be extremely cool.

mdboom6• 8 years ago

Yep. You can "import" python objects over to the Javascript side and start using them from there. It's not as pleasant as working with Numpy in Python, of course, due to lack of operator overloading, of course.

mjs2600• 8 years ago

C can compile to WASM, but I have no clue about external tools like BLAS. I'd assume that things like Numpy are able to work without tools like BLAS, but you might also be able to compile the essential ones to WASM as well.

gh02t• 8 years ago

NumPy can work without libraries like BLAS or LAPACK, but it will be sloooooooooow. There are C versions of those libraries however, maybe they are using those?

azakai• 8 years ago

Python C extensions like NumPy are compiled into shared libraries, and loaded using dlopen() etc. normally by the main compiled Python runtime. (The only special thing here is that all those components are compiled to WebAssembly so it can run on the Web.)

alexeiz• 8 years ago

This is great. Many times I wanted to showcase to colleagues who don't use Python how things would work in Python, but they don't have either Python or Jupyter installed. So I had to run Jupyter server in my own environment and give them access to it. This is quite an inconvenience as well as a security risk. Having a completely server-less Python/Jupyter environment should make this kind of showcasing much easier.

nsriv• 8 years ago

You could also use a Jupyter notebook uploaded to colab.research.google.com

existencebox• 8 years ago

I'll chime in to mention notebooks.azure.com as well.

(Typical disclaimer, I'm a dev on the azure notebooks team; we just try to solve exactly the scenario GP was asking so I would be remiss to not throw our hat in the ring)

kowdermeister• 8 years ago

Why not show them the basics in https://repl.it/ or something similar?

quiq• 8 years ago

Do they have Docker? Jupyter maintains a few docker images with various parts of the scientific stack.

nategri• 8 years ago

How and why would they have docker and not python. Docker's like 10 rungs up the skill ladder.

acbart• 8 years ago

I do a lot of work with Skulpt, which included adding (somewhat superficial) support for MatPlotLib. You can make line plots, scatter plots, and histograms. It's a pretty cool system, though I will say developing in it isn't always very fun :)

lx34596832• 8 years ago

I benchmarked this recently. It comes in at 4-10x slower than cpython for random typical use cases on my machine, which is not too shabby at all. The author also states that numpy is slower than it should be due to lack of support for BLAS at this time.

The real impressive feat to me isn't hosting jupyter in the browser. It's access to a reasonably fast implementation of numpy in the browser which smokes native JS code for homogeneous array operations. I would love to see a minimalistic WASM implementation of numpy that can seamlessly interop with normal JS. Such a library would open up all sorts of possibilities that aren't currently feasible due to perf reasons.

makmanalp• 8 years ago

I can't believe I'm saying this but it would be neat to have a browser native BLAS, or absent that perhaps cross-compiled to WASM from a RUST implementation or something! Makes me think of all the demos like the tensorflow playground (dunno if all that happens in the browser or not).

TAForObvReasons• 8 years ago

People are working on the pure JS version of BLAS: https://github.com/jacobbogers/blasjs . Hopefully that work leads to best practices and a standard

singularity2001• 8 years ago

Interesting, how do you derive at "4-10x slower"?

Running the exported notebook here takes

time py3 python.py real 0m0.145s

chrome: load html: 2sec run html: 13sec!

That's not 10 times slower, but a 100 times slower!

Still good start.

lootsauce• 8 years ago

I have used Weblas "GPU Powered BLAS for Browsers" for matrix multiply and it smokes! https://github.com/waylonflinn/weblas

travisoneill1• 8 years ago

Did you use js typed arrays (e.g. Uint32Array) to benchmark large array operations or just Array?

xtrapolate2• 8 years ago

Numpy is fast thanks to having much of its code compiled as a native library.

They were purposefully working to reduce abstraction, since performance mattered from day one.

WASM is a huge step backwards in that regard. If you care about performance, just install Python on your box and "pip install numpy". Do you absolutely have to have it come in a browser now?

X6S1x6Okd1st• 8 years ago

Huh I am getting

``` TypeError: window[s] is undefined ```

with firefox.

mdboom6• 8 years ago

File a bug over at https://github.com/iodide-project/pyodide/

I'd love to get to the bottom of that.

sitkack• 8 years ago

The goal of http://www.graalvm.org/docs/why-graal/#for-ruby-r-or-python is to run scipy.

The Python scientific stack, compiled to WebAssembly

Related Stories