Computational reproducibility: IPython in the age of data-driven journalism

http://blog.fperez.org/2013/04/literate-computing-and-computational.html

by phreeza • 13 years ago

46 16 13 years ago

blog.fperez.org

minopret2• 13 years ago

Sage (sagemath.org) provides symbolic computing as well as numerical computing. Symbolic computing makes it easier to explain the equations behind the bottom-line results or diagrams and make that mathematical procedure open to computer-assisted experimentation. Sage is built on top of IPython. It is freely available both for download and as a web application (sagenb.org).

omaranto• 13 years ago

It might be misleading in this context to say Sage is built on IPython: while the Sage terminal repl is built on the IPython terminal repl, the Sage notebook is not built on the IPythn notebook.

dschep• 13 years ago

Is there any real benefit over IPython + SymPy?

minopret2• 13 years ago

If you're happy with SymPy, then Sage doesn't really give you anything. That is, it gives you SymPy, which you already had.

Sage is many things intelligently integrated, of which SymPy is just one example. Superficially Sage is a set of command-line and web user interfaces with a preprocessor on top of the Python programming language. The deeper part of Sage is a set of libraries that tie together many high-quality tried-and-true open-source packages by providing a unified API and object model (theoretically based in category theory, I think), plus a set of package-specific APIs for when those are more desirable. The packages include Maxima as the main tool for symbolic computation, SymPy as another, NetworkX for graph-theoretic operations, PARI/GP for number theory, and many more: http://www.sagemath.org/links-components.html

rogerallen• 13 years ago

This helped me understand the salient differences http://blog.fperez.org/2012/01/ipython-notebook-historical.h...

mistercow3• 13 years ago

Since CoffeeScript started supporting literate programming, I've enjoyed dabbling with it quite a lot. I wrote a program to generate a recipe for a variant on the "soylent" concept, using numeric.js to solve and optimize for various constraints based on a target nutritional profile. The goal of that was rather close to what TFA is discussing; I had previously computed the recipe by hand, realized later that my calculations were wrong, and decided to build the recipe as a well-documented reproducible computation instead of an opaque final result.

But what I have not found yet (or discovered for myself) is good guidance on how to document larger blocks of code. If a function is more than a few lines long, I haven't found a satisfactory way to document it.

The first approach I tried was not to respect the atomicity of the function in terms of formatting. That is, I would just write along literately about what the function was doing, right inside the function. The problem is that in both the code and the generated documentation, it becomes very difficult to discern scope visually.

The second approach I tried was to give a brief overview description above the function, then include ordinary comments inside it. That's fine and all, but at that point, you're not gaining much over simple block comments above the function.

The third approach I've tried is to either use an ordered list of steps preceding the function, or to describe the function with numbered footnotes, and then to add corresponding numeric comments in the code. This is the most satisfactory solution I've found yet, and the documentation ends up looking like a programming book. But the locality of reference is just terrible. It's also just terrible for control flow statements. Using an outline format instead of an ordered list can allow for a bit better control flow, but it's still not great.

jasongrout• 13 years ago

Would it make a difference if your editor had better support for these sorts of things? For example, collapsed comments, where its shows only the first line, might help a lot. Then you could put a summary in the first line and elaborate in a paragraph that would be hidden by default.

Or what if the editor supported such comments in the margin, like the LibreOffice or Word review comment features?

mistercow3• 13 years ago

The problem really isn't that the description is too long, though. The problem is that you have to jump back and forth to see what part of the explanation applies to what part of the code. I'm also wary of making code readability too dependent on a specific editor

The other problem with using numeric footnotes is that any time you change the code, you have to laboriously renumber the comments.

That part really could be helped with better editor support. And if the generated documentation were, say, a rich HTML page, you could even have it so that mousing over a line of code would highlight the relevant point in the documentation (and vice versa).

But LP has been around a lot longer than I have been programming at all, and I find it hard to believe that after a month of dabbling, I have better insights into improving it than those who have been using it for decades.

That said, the more I read into LP, the more I feel like part of my problem may simply be that CoffeeScript's implementation of it is rather anæmic. Traditional LP is apparently supposed to have macros that are essentially section headers that double as language constructs. That's a lot more powerful than just stripping everything but the code blocks from a markdown file and executing the result.

datr• 13 years ago

Documentation generator templates which split the comments and the code help with the scope problem of solution 1. E.g.:

http://reload.github.io/phing-drupal-template/

The trade off being that the vertical spacing of the code is now inconsistent.

mistercow3• 13 years ago

That does certainly help in terms of making the documentation readable, but it doesn't help the readability of the source code.

arpineh2• 13 years ago

The stuff you can do with current IPython notebook is already amazing and immensely useful, as the examples mentioned in the posts demonstrate. Browser is not a good place to manipulate gigabytes of data (like Python), but it is the easieast to use, most versatile rendering engine for visualizations.

I keep envisioning a marriage of Light Table with IPython. If this post is any indication of Light Table's prowess, it could very well work with IPython backend: www.chris-granger.com/2012/05/21/the-future-is-specific/

I mourn the lack of language to build these editors. Schema languages only validate document's structure and content. Browser based editors either abstract editing away to wiki like syntax or hide it inside Word processor emulating widgets. Both come with a set of assumptions that makes them hard to customize to suit the needs of the data.

It should be possible to create a declarative editing and visualization language (a Domain Specific Language) that would drive this browser/data structure server combination. You could describe the editing environment for one particular domain (like exploring and visualizing a statistical dataset). This should be data driven, ie. it builds the environment according to chosen dataset. Language should allow macros or modules in suitable languages for additional functionality, like complex validation, animations and transitions. You can export your view of the data, complete with your code and command history.

Tools for data visualization and editing need to look like they are build for this data, for these particular visualizations. And they should be customizable, right there in the same environment so the link between the data and your knowledge of its domain stays in view. I think these tools hold the promise of becoming toolmakers tools, too.

gajomi• 13 years ago

I also would be very interested in seeing what could be done with IPython + Light Table.

I currently do most of my exploratory work (which is in turn most of my total work) in the IPython Notebook and it will be interesting to see what Light Table can do beyond what the notebook does (for starters, I imagine it will have a more refined approach to Javascript hooks for visualization and the like).

I will point out though that the only thing that gets posted to the browser in the IPython Notebook is an objects __repr__. So, unless you are looking at raw dumps of tens of gigabytes of data, you shouldn't have to worry about browser memory limitations. The IPython backend of the Notebook can handle just about anything that your standard python interpreter can handle. It could even be an EC2 instance or cluster resource (here is one example: http://www.nature.com/ismej/journal/v7/n3/full/ismej2012123a...).

dr_doom• 13 years ago

Have you checked out Continuum's Wakari project.[1] It is essentially IPython with an online ide.

http://continuum.io/wakari.html

arpineh2• 13 years ago

I first mistook this for their other in browser data analysis environment: https://github.com/ContinuumIO/bokehjs

I have yet to test this, since it looks like early days yet.

Wakari is a cloud service and somewhat expensive for my needs. Though its hard to say how big an instance my data would actually require. If my Pandas experimentations don't pan out, I might have to give it a whirl.

ianstallings• 13 years ago

Inline twitter comments confuse the bejesus out of me.

Computational reproducibility: IPython in the age of data-driven journalism

Related Stories