Observable’s Not JavaScript

https://observablehq.com/@observablehq/observables-not-javascript

by escot • 7 years ago

283 97 7 years ago

jacobolus13• 7 years ago

Observable is great, and I hope more people learn about it and try it out, but I am afraid that outside of the audience of people who already have worked with it a bit, both the title and content of this particular notebook are going to be a bit misleading and out of context. The intended audience isn’t really people new to the platform.

One of the most important things about Observable is that it is mostly just javascript.

jdlshore2• 7 years ago

This is the first I've heard of Observable and I loved this. It was a great introduction for me--practical and immediately obvious.

jacobolus13• 7 years ago

Here are a small handful of my favorite notebooks, chosen for variety of style/theme/author, and in no particular order:

https://observablehq.com/@freedmand/sounds https://observablehq.com/@freedmand/sounds-2

https://observablehq.com/@benmaier/a-visually-more-appealing...

https://observablehq.com/@veltman/centerline-labeling

https://observablehq.com/@pstuffa/making-maps-with-nyc-open-...

https://observablehq.com/@mootari/getting-artsy-with-lloyd

https://observablehq.com/@visnup/cheat

https://observablehq.com/@bps10/sensations-from-single-cones

https://observablehq.com/@gre/introduction-to-functional-ren...

https://observablehq.com/@redblobgames/delaunator

https://observablehq.com/@larsvers/hexgrid-maps-with-d3-hexg...

https://observablehq.com/@lemonnish/rotating-geometry

https://observablehq.com/@stwind/how-raymarching-works

https://observablehq.com/@fil/modified-stereographic-project...

https://observablehq.com/@mbostock/tissots-indicatrices-on-a...

https://observablehq.com/@mbostock/manipulating-flat-arrays

breck4• 7 years ago

I loved the title. Because indeed, I personally thought that Observable was mostly just Javascript. But now I understand it a lot more and more importantly have an idea now of where they are going with it. I love JupyterLab, but I think long-term it is boxed in by its design of being Python/R in the browser. Observable is breaking from Javascript more than I thought, which will allow it to do really cool things and hit an audience that doesn't necessarily care that it's not quite Javascript.

jadbox2• 7 years ago

I kinda want Observable to be it's own language that can compile to JS... not saying it would be a good general purpose lang by any means, but it would be interesting to experiment with a 'first-class observable' language.

sametmax• 7 years ago

Jupyter is above all a kernel. A kernel that works with several languages, and can communicate using a socket with any program, not just the browser.

colemannugent2• 7 years ago

I really wish tools like this or Jupyter Notebooks were around when I first started studying CS in school. Mixing notes and code is such an interesting use-case.

WaxProlix• 7 years ago

I see it as sort of an extension of Knuth's 'literate programming' concept [1], which has existed in some form or another in many languages. Haskell has .lhs ('literate haskell') files where comments are the default and to have the compiler actually pay attention to a line you need to specifically preface it. Nice in the modern age to have the addition of graphs, charts, etc, without having to resort to TeX or ps or something.

1 https://en.wikipedia.org/wiki/Literate_programming

jacobolus13• 7 years ago

Knuth has been doing literate programming for 35+ years https://en.wikipedia.org/wiki/Literate_programming

I have really enjoyed Observable as a literate programming platform though. Being able to mix text, diagrams, data tables, multimedia output, interactive inputs, easily modified subroutine implementations, imports of external data and subroutine libraries, ... makes for a very expressive and reader-friendly (albeit with a bit of reading learning curve) platform for writing interactive documents and for doing research.

For example, here’s a recent relatively literate-programming style notebook of mine (this one was mostly done in an afternoon for fun as a way of procrastinating from other work, not as a research project), https://observablehq.com/@jrus/munsell-spin

It’s very low-friction to set up a new notebook and just start writing, or to open an existing draft notebook and work on it. I have found this to be extremely helpful in the past year or two trying to work during my 2-year-old’s nap time.

Pen and paper and Observable notebooks are the two most important tools of my recent research efforts.

colemannugent2• 7 years ago

Knuth has been doing literate programming for 35 years

I shouldn't be surprised, that man had a lot going on. Just TeX and Big-O notation alone would be enough to permanently enshrine him in the CS heap of fame, but the hits don't stop there!

Kinda funny how much just one guy could affect research.

TeMPOraL3• 7 years ago

Org Mode in Emacs has been around for quite a while now, and it has full literate programming features. You can use it essentially as a supercharged Jupyter Notebook.

shereadsthenews• 7 years ago

All the old-line science tools are in the interactive notebook or logbook style. Think of Maple or Mathematica.

whymsicalburito2• 7 years ago

This product is very confusing. What is a "magic notebook" and why do I need one?

I wish companies like these would just explain their product in simple language like a normal person.

jacobolus13• 7 years ago

You might enjoy one of these as a starting place:

https://observablehq.com/@observablehq/five-minute-introduct...

https://observablehq.com/@observablehq/introduction-to-noteb...

https://observablehq.com/@observablehq/why-observable

whymsicalburito2• 7 years ago

Thanks, those are good articles :)

This is what should be on the homepage!

amirathi• 7 years ago

Just checked out live commenting feature on Observable [1], the ability to send code suggestions and preview + merge suggestion with 1-click looks amazing.

I built a tool [2] for Jupyter notebook code reviews which has commenting feature for GitHub pull requests (regular text comments). It would be great to have actual code edits/suggestions as review comments.

[1] https://observablehq.com/@observablehq/suggestions-and-comme...

[2] https://www.reviewnb.com/

lwb• 7 years ago

Am I right in thinking that this is mostly interesting for statistics-adjacent fields? I imagine someone using this mostly to embed charts and graphs alongside code a la Jupyter Notebooks.

What would be fascinating to me is if these tools made it significantly easier to make "Explorable Explanations" (nicky case, bret victor).

I love the Raft visualization (https://raft.github.io/) and would be ecstatic to see more stuff like that surfacing around, making it easier to grok sophisticated distributed systems. Next logical step would be to actually control said systems from the visualization itself.

jacobolus13• 7 years ago

(a) This tool does make it significantly easier to make explorable explanations, and in particular facilitates sharing / re-using code and ideas.

(b) This tool is useful in a wide variety of fields that are not statistics- or data-visualization-adjacent.

Personally I find it to be the best (most reader friendly) available platform for most types of code documentation and learning materials, as long as the code involved happens to be Javascript. I would also love to see more research papers published as notebooks.

catacombs4• 7 years ago

Am I right in thinking that this is mostly interesting for statistics-adjacent fields?

Eh. Not entirely right. Many journalists, specifically those who work with data analysis and visualization, use Observable notebooks in their reporting. Other groups also use the notebooks for documentation purposes.

mathisonian• 7 years ago

You might be interested in Idyll [1], a project that I started as part of my research at UW. One of the explicit goals is to make it easier to create explorable explanations.

[1] https://idyll-lang.org/

jashkenas5• 7 years ago

In that case, you may be interested in checking out some of the fancy explorable explanations folks have been publishing: https://observablehq.com/collection/@observablehq/explorable...

Here’s a sequence of sound wave primers by Dylan Freedman: https://observablehq.com/@freedmand/sounds

Mike wrote an exploration of the Lotka-Volterra equations for simulating the dynamics of populations of predators and prey: https://observablehq.com/@mbostock/predator-and-prey

Or something physical, like Jacob Rus figuring out how a chunk of metal from a machinist’s workshop can scribe out a perfect sine curve: https://observablehq.com/@jrus/sinebar

Or something fun, with Krist Wongsuphasawat’s tapioca pearl to tea level Boba Calculator: https://observablehq.com/@kristw/boba-science

Or my personal favorite (although I can't view it in Chrome), Toph Tucker’s poignant reverie on correlation, eugenics, and the morality of applying statistics to human affairs: https://observablehq.com/@tophtucker/inferring-chart-type-fr...

kreetx• 7 years ago

The title is a bit click-baity since I don't think anyone would think Observable would run javascript as if it were in editable <script> tags in a html document. But this is what most of the points try to explain: that there is a framework around the js it runs (so errors don't leak to other blocks (cells)), some types of imports are not supported since they are not needed, etc.

Not saying Observable isn't great, just pointing out that it is javascript - other languages have playgrounds as well.

mattigames• 7 years ago

JavaScript is the runtime executing it yes... but if you write C code and transpile it using Wasm you are confusing people if you say that what you wrote initially is not C code; so for the sake of comprehension (what language was created for) it is indeed "not JavaScript" but a new word we can agree to use to make it clear what plataform we are talking about: Observable.

catacombs4• 7 years ago

While this is great, I'm still waiting for Observable to include a Vim-like editing environment in the cells.

Working in Emacs and Vim all day, I can't tell you how immensely annoying it is to switch between mouse and keyboard when editing code blocks.

mbostock3• 7 years ago

We did just launch a big improvement to keyboard controls this week that mirrors vim’s “Normal Mode”:

https://observablehq.com/@observablehq/cell-shortcuts

catacombs4• 7 years ago

I saw that. What I'm talking about is editing code with Vim-like keys INSIDE the cells. An implementation of the Ace Editor[1] or something similar would be amazing.

[1] https://ace.c9.io/

stared• 7 years ago

As much I try to like this project, still, something feels clunky. I am an avid user of Jupyter Notebooks for prototyping and code sharing in data science / deep learning.

Fir JavaScript (and D3.js in particular), still - I don't have a default go-to solutig. Gist -> blocks? GitHub and and a standalone page? JSFiddle or CodePen? Observable? (I would love to be convinced to one or the other approach.)

My main concerns with observable are:

1. Unlike Jupyter notebook, the code is non-linear. Very often it refers to some pieces.

2. It takes some clicking to reveal the code. (Also, maybe a Jupyter habits, but I am used to results below code.)

3. I am still not sure what is the full syntax besides JavaScript and Markdown.

4. I am not sure if it is easy (or possible) to backup / self-host the code. (Pretty much a requirement to make sure a project won't disappear in a few years.)

BTW: Once I gave Obserbavle a try, https://observablehq.com/@stared/tree-of-reddit-sex-life (and it got really popular here: https://news.ycombinator.com/item?id=19640562).

tolmasky• 7 years ago

Have you had a chance to try our project, https://runkit.com? The two projects intersect a bit, but given your specific concerns it may be more along the lines of what you're looking for:

1. Specifically, we have gone through great pain-staking measures to make it just JavaScript and linear (while avoiding some of the pitfalls of Jupiter notebooks, like "losing state in cells"). You can read more about this specific issue here: https://blog.runkit.com/2015/09/10/time-traveling-in-node-js...

2. We are also huge believers in being able to own your code -- as such, every notebook has a download button that downloads a package with the entire dependency tree in a lock file so you can get exactly equivalent behavior. Again, this is only possible since we've started with the premise that RunKit notebooks just are Node modules.

3. Since it is just Node, every single module just works, including binary ones. In fact, every package on npm is built in -- no install, just require it.

shezi• 7 years ago

I haven't tried it myself, but there is a Javascript kernel for jupyter[1]. In fact, there is a huge number of kernels for jupyter now[2].

[1] https://github.com/n-riesco/ijavascript

[2] https://github.com/jupyter/jupyter/wiki/Jupyter-kernels

jacobolus13• 7 years ago

My own opinions:

(1) Non-linear cell order (and in particular re-running all dependent cells anytime anything gets changed) is a huge improvement vs. mathematica/matlab/jupyter notebooks. Once you get used to it you will feel crippled when you try to go back. Being non-linear allows the author to optimize for whatever flow best matches the narrative, rather than ordering things based on arbitrary implementation-specific requirements of the programming environment. It is also possible to e.g. put simplified or brute-force implementations inline in the prose, and then move tricky optimized versions to technical appendices at the end of the document.

Being non-linear also allows for showing some placeholder output right away and letting expensive-to-compute output (or slow-to-fetch data) execute after initial notebook load and progressively replace the placeholders. This lets readers get started reading the prose right away without needing to wait for slow operations.

(2a) Being able to pin the source of a cell to be shown / hidden at the author’s discretion makes for an improved reader experience in the typical case (assuming a conscientious author). Nobody wants to read e.g. big blobs of redundant LaTeX source in addition to their mathematical formula output. Needing to sometimes click to show code that an author left hidden in the initial view is easy after just a tiny bit of experience as a reader. As an author I tend to mostly leave prose markup pinned closed and code cells pinned open (unless they contain too much distracting or overly technical detail) when publishing notebooks.

(1 & 2a) These two features both change incentives for notebook authors: since code can be hidden in the default view authors are not afraid to use a better implementation which includes distracting technical details; in other notebook environments I have noticed authors intentionally use worse-performing but simpler implementations, and technical details often get moved to some external library, hiding the code much more deeply than just putting it in a cell with hidden source / moving the cell to the bottom of the document.

(2b) I also prefer output below code, but the reason for the order here is so that showing/hiding the source won’t make the output jump up and down the page excessively; with a bit of experience the other order is not really an obstacle in practice.

(3) It’s Javascript (with a few minor additions as shown in TFA), but you can use whatever javascript tagged template you want, including html, markdown, or anything else you can dream up. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

(4) It is fairly easy, but it’s a fair complaint that it takes some trust / involves some risk to build work on top of a closed-source platform, especially for folks with strong free software ideology; even if it is possible to mirror code outside observablehq.com, it is at least mildly inconveient, and necessarily loses access to some great platform features. I personally have a lot of faith in the Observable team’s commitment to not breaking documents published on their platform or letting them disappear. But when business is involved, there’s no absolute guarantee.

kragen4• 7 years ago

Oh, this is interesting — Observablehq isn't free software? It hadn't even occurred to me that it might be proprietary and not even runnable off the platform. You say, "when business is involved, there's no absolute guarantee", but to me, that sounds like an absolute guarantee that your documents will disappear — consider how many companies are still around from only 3000 years ago, right before the Bronze Age Collapse, and compare to how many documents survive from that time.

billconan• 7 years ago

I'm making an alternative called epiphany, focusing on tech blogging,

https://epiphany.pub/post?refId=2684bc94f9fcb9ffe637ebfbeba2...

for 1. epiphany is also non-linear, because I'd like to reveal the results and then explain pieces.

for 2. one click to the edit mode.

for 3. epiphany uses vanilla js with some apis to print results and create ui.

for 4. I try to use a plain text format, I also implemented pullrequest and versions like that of github. you could download all your data.

jashkenas5• 7 years ago

With the gentle caveat that this list of concerns feels fairly low effort — surely, with a little elbow grease, you can think of much more damning complaints! ;) — I'll try to respond in good faith:

1. The non-linearity of the code is really the whole point here. Reactivity means that you wan work on any little piece of the code at any time, and you don't have to reload the page, or re-evaluate the entire notebook. It closes the feedback loop, and speeds up the process of iterating on your ideas immensely.

2. We’ve added more keyboard shortcuts, so you can now "Shift-A" to select every cell, and then press "P" to pin every cell open at once (and then to close them again).

3. The full syntax is just the "Observable JavaScript" described in the OP. The Markdown tagged template literal is part of the standard library, and you can define your own that work just like it. In that sense, the syntax is extensible. For example, here’s a different "pl" tagged template literal that runs Prolog programs: https://observablehq.com/@mbostock/hello-tau-prolog

4. Every notebook is available for download and self-hosting. You can download the code as an ES Module, or as a tarball that you can install with npm, or load and render on a web page: https://observablehq.com/@observablehq/downloading-and-embed...

fourthark• 7 years ago

What's the rationale for results above code?

ww5202• 7 years ago

1. Non-linear is a plus. Why force an order. Spreadsheet is non-linear and it works pretty well.

2. Clicking to reveal can be simply addressed with a global Expand-All toggle.

akhilcacharya2• 7 years ago

This is insanely cool - I didn't know an implicit DAG notebook was on the market already.

VectorLock• 7 years ago

Is it acyclic? Can you make cycles of cell changes?

akhilcacharya2• 7 years ago

I would hope it is...

mbostock3• 7 years ago

By default it is acyclic: any circular definitions will thrown an error. (I added an example to the notebook.) But you can opt-in to circular relationships using mutables or side-effects.

SquareWheel• 7 years ago

This blog is very frustrating to read. When I try and select text to mark my place, it starts editing markdown instead.

jacobolus13• 7 years ago

Can you elaborate? If you click (without dragging) in the left margin next to a cell output, it will show the source for that cell. If you just drag over some text, it should select just fine.

xeromal• 7 years ago

Trello is the same way. I hate it.

AgentME• 7 years ago

I notice whenever I highlight a paragraph, clicking the paragraph again doesn't de-select it, unlike every other program and webpage. I have to click outside of the paragraph to unhighlight it. As a compulsive highlighter who often double-clicks and then clicks paragraphs to highlight them and then unhighlight them, this is so bothering to me. (Also, if I click on a code snippet, a highlight appears to the left of it, and then gets left behind and doesn't reset when I click outside of the code snippet!) It feels like I've picked up an object and it was sticky and got residue left behind on my hands each time I interact with it. Sure, I could just stop repeatedly touching the object so much, but I fidget with what I'm holding. An object that's not fidgetable like every other of its kind is unnecessarily hostile. I'm baffled that a webpage went through extra work to make highlighting act differently and infuriatingly.

Highly related xkcd: https://xkcd.com/1271/, and highly related alt-text: "And if clicking on any word pops up a site-search for articles about that word, I will close all windows in a panic and never come back."

trnglina• 7 years ago

Interestingly enough, from my quick tests, the highlighting behaviour works as normal in Firefox, but not in Chrome.

anderspitman• 7 years ago

I somehow didn't realize the article itself was a notebook until I got to the bottom. I can't decide whether that's a good thing or a bad thing, and whether it says more about me or observable. The whole thing is way impressive though.

kragen4• 7 years ago

I figured out it was a notebook when it just showed a blank page in the first two browsers I tried to read it in.

mattlondon• 7 years ago

I thought they were talking about Observables as in the Reactive Observables (1) from RxJS (2) et al.

Unfortunate naming! No doubt SEO will be super difficult for them considering how much "javascript" && "observable" keyword stuff there is out there already.

1 - https://en.wikipedia.org/wiki/Reactive_programming 2 - https://rxjs.dev/guide/observable

theoh• 7 years ago

I don't know the Observable folks personally, but I've been watching what they post on Twitter. It's not snobbery to say that the level of polish and sophistication present in what they put out (in terms of the reactive architecture and the graphic side of it) is second to none. Their product has got so much class that the idea that they would spend time on SEO is kind of incongruous, right?

asutekku• 7 years ago

The polish of the product sadly doesn’t matter if no one knows about it. SEO is the necessary evil.

jashkenas5• 7 years ago

Your comment made me curious about our current SEO rankings, so I used a VPN and fresh incognito windows in a browser I never use (Safari) to Google for "observable" from Silicon Valley, New York City and Madrid.

ObservableHQ.com is in the top spot in Silicon Valley and NYC, and a few spots down the first page in Madrid, after Angular Observables and RXJS Observables. So it looks like we’re doing pretty alright so far.

graphememes• 7 years ago

That's for users who have never searched for anything regarding observables and clicked links which will tailor their ranking algorithm. Not sure how much optimization for "pure" ranking we should do nowadays.

its_the_future• 7 years ago

IMHO call it something other Observable? Observables in JavaScript and development are something else, only reason I clicked the headline is due to the confusion, and I'm slighly upset and baseline hostile to the product. This hostility is probably because I suspect the confusing name is intentionally confusing.

daotoad• 7 years ago

No need to assume malice, other human foibles like attribution bias and hubris nicely explain the horrific idea of calling your language/tool/project 'Observable'.

I, for one, am irritated by the name and hope they either change it or that the project fades. Things are confusing enough without this kind of nonsense. Yes, I get that each cell in a spreadsheet is observing its neigbors. Yes, it's nice that you can "observe" your code auto updating. Yes it's nice that people are trying to improve upon spreadsheets as a programming model. Don't be purposely obtuse right out of the gate by naming your project after a widely used pattern, tool or technique. Call it Cells or Panopticon or something. Please.

In other news, check out my new web framework, called MVC.

jacobolus13• 7 years ago

Is this really that common a term in computer programming? I can’t say I see it used with any regularity. Maybe it’s a common term in some specific programming subcultures? Apparently it’s a deprecated Java class, https://docs.oracle.com/en/java/javase/11/docs/api/java.base... and something in RxJS (which I had never heard of before doing a web search just now) https://www.learnrxjs.io/concepts/rxjs-primer.html

The term “observable” used as a noun in a programming context seems to me very clunky and vague. YMMV.

I am much more familiar with the term “observable” in physics. https://en.wikipedia.org/wiki/Observable

TeMPOraL3• 7 years ago

Hah, I feel the opposite. What irritates me is that random JS libraries took over search results for terms like "observable" or "reactive programming". I've wasted a lot of time recently because of that, and ended up inventing some of the reactive stuff for a project from scratch, because it was less effort than trawling through pages of tutorials and advertisements for hot NPM packages.

dang2• 7 years ago

I suspect the confusing name is intentionally confusing

"Assume good faith."

https://news.ycombinator.com/newsguidelines.html

IggleSniggle2• 7 years ago

I thought this would be referring to the Observable ie Observer/Subscriber pattern as a primitive in JavaScript, of which RxJS (Reactive Extensions) is the best known implementation followed closely by MobX. I like this project quite a bit, but it is unfortunately named.

It feels kinda like naming your project "Array map for JavaScript."

TeMPOraL3• 7 years ago

Come on, RxJS stole that (and "reactive") terms from prior work, which was arguably more interesting for anyone except web developers. So we can keep passing the blame about who steals SEO from who...

IggleSniggle2• 7 years ago

To be clear, I was only quoting RxJS as the most popular implementation in js for the Observable primitive. This has nothing to do with the Observable primitive. My complaint isn’t that this steals SEO, it’s that this becomes less searchable because any search for “Observable JavaScript” is going to turn up RxJS/MobX or one of the other libraries that implement Observable primitive types.

Edit: with regard to ReactiveX...well yeah. But it’s nice that it’s implemented in so many languages! (Not just RxJS) http://reactivex.io/languages.html

sethf• 7 years ago

My exact reaction - and the reason I clicked through, though I must say I was pleasantly surprised it was this, and not JavaScript Observables. May have to make the switch from Notion https://www.notion.so/

ran3824692• 7 years ago

I get blank page without running javascript. Not observable.

forgotmypw• 7 years ago

Ditto.

gloflo• 7 years ago

It is a project about highly interactive content on the web. That does require JavaScript.

aerovistae• 7 years ago

You know what would be nice, in a scifi universe? Being able to hover over HN links and get a short context for the topic.

E.g. "Observable is a <something something> used for <something something>. Since the language used in its <something something> appears similar to JavaScript, they put together a blog post clarifying the differences."

It would be great if the HN submissions form had a field you could optionally fill out with such a text. Because if you can't tell from my example above, I have no idea what this is.

artursapek• 7 years ago

This is off-topic, but Observable's layout code seems to have some kind of rounding issue. The farther I scroll down the page, the more the sections overlap: https://i.imgur.com/zuYh3j4.png

It's a very gradual progression into that. Running Chrome Version 71.0.3578.98 (Official Build) (64-bit) on Ubuntu 18.04.

simonw• 7 years ago

Observable is SUCH a clever piece of software design. Every time I read more about how it works I learn something new and interesting.

ww5202• 7 years ago

JsLive or JsLiveDoc.

XCSme• 7 years ago

Click-bait title, it's not what you think, not about RxJS Observable, but some company/product.

emilfihlman• 7 years ago

The site CSS is broken: https://emil.fi/m/2019-06-15_13-17-36_siU3iwbN.png

nicwolff• 7 years ago

The demo notebook at https://observablehq.com/demo works on Safari 11.1.2 but at this link I just see a JSON object starting `{"@context": "http:\/\/schema.org", "@type": "Article",...`.

Observable’s Not JavaScript

Related Stories