GitHub and Jupyter IPython Notebooks

https://github.com/blog/1995-github-jupyter-notebooks-3

by benn_88 • 11 years ago

199 51 11 years ago

shepardrtc• 11 years ago

So they're only displaying the notebooks, but not actually allowing you to work in them, right? Because for a minute there I was pretty excited!

carreau2• 11 years ago

you cannot work on it yet.

rspeer• 11 years ago

It sounds like you expect this to change. If you could actually run the commands, where would the computing power come from?

munro2• 11 years ago

There are a lot of tools for editing notebooks, and they're complex. I also don't use their online code editor, because mine is better. Through my personal experiences, my assumption is they'll never take on that project.

andmalc• 11 years ago

Excited about IPython notebooks generally or possibility of editing them on GitHub?

If you want one to edit, try here: https://cloud.sagemath.com

yes_or_gnome• 11 years ago

Screen has both 'c' and 'C-c' set to new window by default. tmux only has 'c' by default.

From https://www.gnu.org/software/screen/manual/html_node/Default...:

  C-a c
  C-a C-c
  (screen)
  Create a new window with a shell and switch to that window. See Screen Command.

Edit: d'oh. I had to reread the original comment. You already know this, but by adding 'C-c' I don't have to release the Ctrl button.

You can add this to tmux.conf to get screen-like behavior:

  # New Window
  bind C-c new-window

wyldfire• 11 years ago

The wires are crossed on HN or your tabs. Your tmux response is in the github-ipython notebook thread.

jlrk• 11 years ago

Yes! A million times, yes! This lowers the barrier to sharing research results stored in a private repo with collaborators. Also, for those who have been looking for ways to display equations in markdown[1], you can embed the equations in markdown (Ipython uses Mathjax) within the notebook.

[1] https://github.com/github/markup/issues/274

saboot• 11 years ago

Okay, what is the best format to store images in? My notebooks contain lots of graphs and output but I'd rather avoid saving the raw pixels. So far I've been using SVG with mixed success. Does anyone use anything else to save space?

rcpt2• 11 years ago

I just tried out https://pypi.python.org/pypi/ipynbcompress/0.3.0 and it worked pretty well. Need to use 'png' to get nice rendering on github.

The last cell here has my use https://github.com/rcompton/ml_cheat_sheet/blob/master/super...

rcpt2• 11 years ago

Hey this is great! I just took a look and saw that a notebook [1] I was tracking in github is now rendering beautifully - and I didn't have to do anything!

https://github.com/rcompton/ml_cheat_sheet/blob/master/super...

voltagex_• 11 years ago

They've really relaxed the filesize restrictions for these - I don't think a 7mb PDF will render in GitHub but this notebook will.

calebm• 11 years ago

This is awesome! Similar to nbviewer.ipython.org functionality.

batbomb• 11 years ago

There's a new problem which still hasn't been resolved w.r.t. iPython notebooks and reproducible results, and that's the problem of private, ephemeral, or mutable data. At the very least, iPython theoretically helps with the audit trail for private and mutable data, but it doesn't necessarily foster (re)producible results or alternative analysis.

This is a more general problem overall, but potentially something like magnet URIs and bittorrent could really help with part of the problem. (I don't really believe git as a system nor GitHub as a platform to be the appropriate place to solve this either).

tmarthal2• 11 years ago

Are you saying that people aren't storing data on github? How large of datasets are you talking about?

It's a shame that Github's large file storage (LFS) mechanism isn't publicly released yet, because I am sure that will appease your problems. (This is assuming your datasets don't fit into a csv (note that the current hard limit for github files is 100MB, and a 100MB CSV file is a lot of data)).

Fomite4• 11 years ago

It should be noted that there are reasons beyond file size not to store data on GitHub, though I do regularly generate simulation data that's way larger than 100 MB CSV files.

qznc• 11 years ago

It helps with ephemeral data, unless I misunderstand you. Notebook document how to generate the ephemeral data.

Surely, it does not help with private or mutable data, but how would magnet URIs and bittorrent help? If it is private, it cannot be shared with bittorrent either and sharing mutable data via bittorrent is not its primary use case.

Bittorrent might help with big data, where big means 100MB+ in the case of Github. There are other approaches for big files in git (git annex, Github's LFS, git-bigfiles, etc).

bigredhdl• 11 years ago

Just a general comment, but in my opinion IPython/Jupyter is worth learning Python for. It is such a great platform for tinkering with data or really any idea you might play with in code.

RogerL2• 11 years ago

The notebooks support a very large number of kernels, including Julia, R, bash, Lua, Erlang, Perl, OCaml, and so many more[1].

[1] https://github.com/ipython/ipython/wiki/IPython-kernels-for-...

pash• 11 years ago

And in case you were wondering, GitHub does render notebooks using other language kernels. For example, here's a notebook in Haskell: https://github.com/gibiansky/IHaskell/blob/master/notebooks/...

baldfat• 11 years ago

I would like to also make note you can use any of the other languages that Juypter supports. I am using R right now and it works, but it isn't as useful as RStudio.

I am guessing that the Juypter notebook format might get traction with other projects besides Jetbeans.

mvermaat• 11 years ago

This is great! We use IPython Notebook a lot in our group, for exploratory data analysis, teaching, sharing experimental results, and as an electronic lab notebook. Inline GitHub rendering is very nice for all our public work.

If you run a private GitLab server, you could have a look at my patch which adds similar functionality to GitLab (in a rather ad-hoc way): https://gist.github.com/martijnvermaat/6926070

sytse• 11 years ago

GitLab CEO here, thanks for maintaining your patch for GitLab. If you want to discuss contributing jupyter support to GitLab itself I'd be happy to discuss.

munro2• 11 years ago

So weird, I was playing with IPython Notebook with PyCharm yesterday. Then I went on to think about how it could be integrated into Github without building the compiled files into the repo. :)

Really there's no solution for extending Github except wait for them to integrate hooks. It would be nice if they made this more generic, and created a file render hook, that would allowed devs to render whatever types of files they want into Github Markdown.

hcrisp• 11 years ago

How does iPython integrate with PyCharm?

usef-• 11 years ago

PyCharm allows you to edit and run ipynb files directly – see here: http://blog.jetbrains.com/pycharm/2014/12/feature-spotlight-...

jszymborski• 11 years ago

For the first time today I put an IPython notebook into a github repo, and thought "cool, they render on GitHub", only to read that they rolled it out today on HN.

Super neat.

cabhishek• 11 years ago

Very thoughtful and awesome. Now is there an easy way to save notebooks directly on GitHub?(git as storage system?). That would make the whole flow seamless and make sharing and versioning easy :)

Fomite4• 11 years ago

This also means, at GitHub repos can get DOIs via FigShare, you can have DOIs for an iPython Notebook, which is awesome.

huac• 11 years ago

Hope support for R-markdown is next!

haddr• 11 years ago

I was also a bit dissapointed for no mention about R :( Anyway, i think it's possible to get notebook for R using this: https://github.com/IRkernel/IRkernel

look at this demo: https://github.com/IRkernel/IRkernel/blob/master/Demo.ipynb

RogerL2• 11 years ago

I am utterly giddy about this. I've spent a lot of time in the book I am writing to generate PDFs, to create TOCs for links to nbviewer, managing all these links and production every time I made a change. Now people can just view it directly in github. Life is as it should be.

There is just so much innovation and hard work coming out of this group, which is quite tiny. Any time I've raised an issue, even if it turned out to be my own dumb fault, gets the immediate attention and support of a core developer.

If you want to communicate with programmers there really isn't a better platform out there. The old 'write code, run code, save results, write latex, gen document, find error, repeat, opps code is out of sync with results....' is pining for the fjords.

Myrmornis• 11 years ago

This is amazing. I think ipython notebook is fantastic in all regards. However, I feel there's a bit of an elephant in the room. The main tools I use as a programmer are shell and a text editor. I'm not about to start writing code in a web browser[1]. Is there any hope that there will be a good workflow for somehow editing .ipynb from a text editor or is that antithetical to the design? Could markdown perhaps be used as a primary editing format to target ipynb?

Looks like there's at least one project doing that. https://github.com/aaren/notedown

And there's the emacs-specific project https://github.com/tkf/emacs-ipython-notebook

[1] EDIT: I mean, not right now anyway. I've got a feeling that statement could seem a bit dated later in my life...

idunning• 11 years ago

It is an interesting point. In my usage, I don't view it as a one-or-the-other proposition. I use Julia, so if I'm doing some exploratory work and experimentation, or making something to present results, then the IJulia notebook is great. If I'm writing some serious longer-running stuff, or a package, I'm in an editor. Sometimes I'll write code in a separate file and call it from a notebook just to keep the notebook focussed on communicating something, and "hiding" the details.

ngoldbaum• 11 years ago

Yeah, notebooks are great for experimentation but not so much for anything that needs to be reused.

I tend to write prototype code in the notebook then copy/paste it into a package as I become satisfied that the code is correct.

wodenokoto• 11 years ago

Jetbrains also have support https://www.jetbrains.com/pycharm/help/ipython-notebook-supp...

julienchastang• 11 years ago

I've been a big fan of TFK's EIN. Unfortunately, TFK no longer maintains the project and I have not had good luck with the various EIN forks. (Anyone else having better experiences?) I still get by with the original EIN by using older versions of ipython/IPyNB. Moreover, the IPyNB technology is getting somewhat bloated with widgets and whatnot and I don't know how that will integrate with an emacs environment. Nevertheless, I still hope emacs modes similar to EIN will be forthcoming in this area. I was hoping to propose a code sprint on this topic at SciPy this year, but I won't be going to Austin, alas.

aaren• 11 years ago

I wrote notedown precisely so that I could edit notebooks in my text editor and version control them as plain text.

It would be good to be able to transparently edit a .ipynb in a text editor - I've been meaning to wrap notedown in a vim plugin but haven't had time. This still leaves the version control problem though.

Note that I recently enabled the reverse (editing markdown in the browser as if it was a notebook), enabled by setting the following in ~/.ipython/profile_default/ipython_notebook_config.py (or similar):

    c.NotebookApp.contents_manager_class = 'notedown.NotedownContentsManager'

This is useful for more interactive work, e.g. iterating on plots, whilst still having everything stored in markdown. See [1] for more info.

[1] https://github.com/aaren/notedown/issues/22

agrothberg• 11 years ago

It would still be nice if IPython Notebook had an option to save the code separate from the output, for example as two files in the same directory. This would make source controlling easier.

c0lin• 11 years ago

Running `ipython --notebook` with the additional option `--script` may be what you are looking for. This saves a `.py` file with the same name next to your `.ipynb` file (the non-code sections are present, but they are commented out).

vhost-• 11 years ago

I just started using these for my data science work and it has been absolutely awesome. Even better now that they display on Github and apparently (based on a comment below) Gitlab too.

stevetjoa• 11 years ago

Great idea! Although it appears that the Audio widget does not display correctly. Compare:

http://nbviewer.ipython.org/github/stevetjoa/stanford-mir/bl...

https://github.com/stevetjoa/stanford-mir/blob/master/notebo...

bruce487• 11 years ago

Awesome, thanks for sharing.

andersonfreitas• 11 years ago

Would be nice to have .ipynb preview on Gist also!

GitHub and Jupyter IPython Notebooks

Related Stories