Introduction to Parallel Python with IPCluster and Wakari

https://www.wakari.io/sharing/bundle/ijstokes/ipcluster-wakari-intro

by corinna • 13 years ago

41 17 13 years ago

wakari.io

paddy_m4• 13 years ago

How many people on news.yc are familiar with IPcluster?

Note: I work for Continuum Analytics.

lost-theory• 13 years ago

I am not familiar with IPcluster. But I have used pyro and execnet. I think I will continue to use execnet, since it seems more lightweight than IPcluster. But the web interface provided by ipython notebook is really attractive..

fperez_org3• 13 years ago

Hey guys,

would you be OK if we use the image at the top of that notebook in the official IPython docs/tutorials/talks? We haven't made one that nice yet :)

If you have the svg sources, that would be awesome...

paddy_m4• 13 years ago

That should be fine. Thanks for building IPython, it's great.

jtmcmc• 13 years ago

I use it all the time!

paddy_m4• 13 years ago

Note, there is sometimes an internal server error when loading this page. If you reload it should work. There are some bugs on our end that we are working out

spenczar5• 13 years ago

IPython is pretty unusual. When I was first introduced to it, I thought of it as mostly an upgrade for the default python shell. But as I find more and more features like IPy Notebooks and IPCluster, I'm realizing that it's a completely new sort of thing. It's not just better command history in your shell or whatever - it's a whole new approach to computing emphasizing reproducibility and distributed workflows.

paddy_m4• 13 years ago

I honestly thought the same thing until I was introduced to the scientific computing world last summer. I will have a blog post soon about how the differences of developing in a datacentric / ipython enabled python environment. The biggest difference is that the IPython notebook makes it reasonable to edit multi line functions at a repl.

fperez_org3• 13 years ago

Indeed, and communicating that clearly and effectively is something that we don't always succeed at. This is the reason why, on many occasions I had to resist fairly vociferous requests that we split up the parallel computing code from the shell, and also all the clients.

In the future that may make sense, once our internal protocols and APIs are so stable that we can do so without breaking functionality just as a way to help decouple development. But as a way to think about computational workflow, these pieces are all part of a coherent vision. It's just one that we don't always communicate clearly :)

ceph_• 13 years ago

Scrolling is broken on the site for iOS.

onalark• 13 years ago

Thanks for the report. Out of curiosity, are you accessing from an iPad or an iPhone, and are you using Safari or Chrome?

wooster• 13 years ago

Broken on an iPhone running 6.1.4 on Safari and embedded webviews, for me. There also appear to be some resources which never finish loading.

mmcclellan2• 13 years ago

Quick sincere question. I build HPC in the cloud tooling with AWS, MIT Starcluster, IPython, etc for a living. I'd like to give Wakari a try, but could the Terms compromise my ability to offer a product in the future that could be construed as a competitor?

pwang• 13 years ago

Hmm, our goal is not to restrict what users can do. Can you email support@continuum.io with the specific clauses that are troubling you?

mmcclellan2• 13 years ago

Thanks Peter, I'll do that. I'm a big fan, I was mainly thinking appearances on my side, like "hey you integrated Gate One shortly after signing up for Wakari." For reference this prompted my inquiry:

"Customer shall not ... access the Service, in order to build any competitive Service or product, or copy any ideas, features, or functions of the Service."

Introduction to Parallel Python with IPCluster and Wakari

Related Stories