LIGO Gravitational Wave Data in iPython Jupyter Notebooks

https://losc.ligo.org/s/events/GW150914/GW150914_tutorial.html

by Mizza • 10 years ago

209 32 10 years ago

losc.ligo.org

jeffjose• 10 years ago

As a python fanboy this makes me really happy. Great job, folks at PSF and Jupyter

ck2• 10 years ago

For a moment I thought "notebook" meant like laptop.

And I was like, wow that is amazing, they crowdsourced data from all over the world.

Then I remembered the sensitivity of the instruments used and how dumb I was.

But it really would be cool if oneday people had smartphones so advanced they could contribute to worldwide collection of data that needed a huge area to sample. I think they are talking about doing that for earthquake alerts.

madvoid• 10 years ago

WeatherSignal is doing something along those lines for weather data: http://weathersignal.com/about/ I'm excited to see where this kind of tech goes.

saboot• 10 years ago

http://crayfis.io/

Use your smartphone camera to create a global network for detecting cosmic ray showers!

Xcelerate• 10 years ago

As someone who keeps getting comments about my "weird" toolkit for doing scientific research (Jupyter/Julia), this made my day.

gaur3• 10 years ago

What would people rather have you use? Excel?

joshvm3• 10 years ago

You would be surprised at the number of Excel plots in the literature.

Most astrophysicists use IDL or IRAF, I've never seen anyone use Matlab. The benefits are tons of functions specifically tailored for astro data analysis. Python is gathering momentum though, there are libraries like sunpy and astropy. Plenty of IDL fanatics are floored when you can show them just how easy it is to process data with the Numpy stack.

I'm _not_ an astrophysicist, but I work with a lot of them. I use Jupyter notebook daily. It's the perfect balance between REPL python and standalone scripts. My typical workflow is to hack something together in a notebook, which lets you iterate very quickly, then once I'm happy I freeze the code into a module.

jjoonathan2• 10 years ago

The two that I'm familiar with are mathematica and matlab.

Mathematica is a lisp for representing and manipulating mathematical expressions combined with an IDE that knows 2D layout (so you can write expressions like you would on paper) and a massive integrated library of mathematical routines. The "gateway drug" is its ability to symbolically integrate, differentiate, factor, simplify expressions, solve equations, interactively plot without explicitly sampling, etc. Then you discover that all it's "heavy lifting" capabilities are integrated with each other -- i.e. you could use a piece-wise implicit surface to define boundary conditions for a differential equation, solve it with finite element on 20 different tessellation levels, and compare the results using a norm built out of an interpolator and integrator to check for convergence. All in a handful of lines of code where you only have to worry about high-level details rather than dozens of for loops and hundreds of lines of glue. I really don't think there's anything comparable in the open source ecosystem yet, but I'd love to be wrong (yes, I know about SAGE).

Matlab is relatively unremarkable as a language -- it's not a lisp, it deals with matrices of floats not expressions, and its only competitive language feature is the eponymous set of linear algebra primitives that CS-trained language lawyers tend to roll their eyes at but that really do make a difference for the scientists and engineers who use it day-to-day. The killer value proposition, though, is its collection of libraries. They're not symbolic like what you would find in Mathematica but they're usually more extensive and relentlessly practical. Sometimes that means speed, sometimes that means features which cater to your particular obscure workflow, sometimes it means integration, but it always seems to result in a decision along the lines of "I could spend a day munging python libraries A, B, C, and D together, or I could open matlab which already has a package and a GUI for it."

Python, Julia, and R do many things very well. They can beat mathematica/matlab in a number of areas but there are still huge swaths of math/science/engineering where they're just not competitive. That goes double when you take into account legacy code. It's changing slowly, but science is a highly competitive environment which is not keen on rewarding contributions of this sort, so it could be quite a while before they catch up.

make32• 10 years ago

matlab, mathematica, jupyter/python or jupyter/R I expect

daveguy• 10 years ago

Note the sample code is in Python 2.7 not 3.x. In general you should avoid 3.x for scientific computing as it is slower and less supported in academia.

erispoe• 10 years ago

That's interesting, do you have a good source on that? I switched to 3.x for development but I do academic projects too.

joshvm3• 10 years ago

I don't know about speed, most of the modules where speed matters, like Numpy, rely on calling C routines somewhere down the line. The version of Python shouldn't make a significant difference.

Numpy, Scipy and Pandas work with Python 3, Jupyter certainly does. That covers probably 80% of scientific grunt work. For specialist applications, OpenCV 3.0 works, as does scikit-learn/image. I can't speak for other development like web though.

I think the main problem is that if you're using someone else's code in academia, it's likely to be written in 2.7 and you would have to go through and update everything that's not back-compatible.

gaur3• 10 years ago

This is a self-fulfilling prophecy.

Someone tries to write an analysis routine in 3.x, but then is quickly told "nobody in science uses 3.x, because of <vague reasons>". Result: everyone sticks to 2.7, so nobody ever even tries to build against 3.x when writing packages.

babo• 10 years ago

I switched to python3 after measured a large ETL process converted. As scientific packages are rely on FFI and the actual code is running outside of python I'm puzzled with your comment about speed.

fonnesbeck• 10 years ago

Neither of these assertions are true. I use Py3 to teach graduate-level scientific computing and machine learning with great success, and the scientific Python community is making a very strong push for adoption. All major packages in the scientific stack now support Python 3.

make32• 10 years ago

slower.. <citation needed>

wiz21• 10 years ago

Donald Knuth is probably swimming in joy right now. This so literate programming (or should I say literate research)...

Mizza• 10 years ago

If you liked this, you may like this surprisingly fascinating talk on the color scheme in Matplotlib used in this paper: https://www.youtube.com/watch?v=xAoljeRJ3lU&feature=youtu.be

cscheid• 10 years ago

This talk is great! And if you watched it all the way to the 6:30 mark, where he says "you can't have negative amounts of light", then you'll really enjoy chimerical colors: https://en.wikipedia.org/wiki/Impossible_color

The trick is you exploit retinal fatigue in order to perceive colors both "above 100%" and "below 0%". Wild stuff.

po84• 10 years ago

https://github.com/minrk/ligo-binder

Re-run the analysis yourself on Jupyter using Binder. Click the launch binder button.

qhoc• 10 years ago

Just curious on binder specific here. I could never got it to work out-of-the-box. In this example, it gave me "ImportError: No module named 'seaborn'" error. Now I could do something like !sudo pip install seaborn but still... Did you guys have same experience?

melted• 10 years ago

Makes for a nice demo of Python Notebook capabilities, especially when it comes to graphing.

anotheryou• 10 years ago

can anyone tell me how the figured out where the black holes are?

with just 2 "ears" I'd expect to be only able to determen a circle, but here is the picture they released: http://content.screencast.com/users/cougarten/folders/Jing/m...

Do i see a warped circle, or the bottom part of one? In the latter case I wonder how they found out.

Are the L-shape sensors capable of seeing some direction depending on which way the phases shift first/last?

If you'd build one of the sensors in reverse you'd see a reversed signal, no? Given that they will have optimized the orientations of both stations this is probably how it worked and I'm seeing just part of a circle, right? That one lonely blob might be an unlikely mirrored version.

LIGO Gravitational Wave Data in iPython Jupyter Notebooks

Related Stories