Om Sweet Om: functional front-end engineering with ClojureScript and React

http://blog.getprismatic.com/om-sweet-om-high-functional-frontend-engineering-with-clojurescript-and-react/

by trevoragilbert • 12 years ago

262 68 12 years ago

blog.getprismatic.com

jlehman4• 12 years ago

I've had the same experience building http://fitsmeapp.com with Om/Clojure -- great to see real, production apps being built in CLJS!

presty4• 12 years ago

your scrolling seems to be broken when using chromium and arrows (up/down)

when going all the way down and then up, it gets stuck around the 2nd or 3rd panel

jlehman4• 12 years ago

Thanks for letting me know -- I'll look into it.

acconrad• 12 years ago

I think your sprite for your logo is all messed up (looks way too large). Also have you heard of http://www.eatthismuch.com/ ? How does yours differ?

jlehman4• 12 years ago

Just learned about that recently -- it's a retina display issue (fix going up tonight). Thanks for pointing it out!

I have heard of it and used it myself.

The primary difference is really the intended use case. Where ETM is focusing on highly-specific (usually on a nutrient/caloric basis) meal plans to hit a target nutritional intake, we are more centered around using general preferences about food (ingredients, cuisine, cooking methods, etc.) in a "browsing" capacity -- currently in the form of recipes aggregated from other sites and bloggers. The goal is eventually to also incorporate nearby restaurant dishes.

dustingetz5• 12 years ago

fwiw it's possible to achieve all of these benefits in vanilla react using javacsript with regular javascript data structures

    * single mutable ref to normalized app state
    * cursors for encapsulation and modularity
    * O(1) deep equality checks
    * fastest possible react performance

source: I have done it in two large react apps, but i have not documented it yet. Blog post coming soon.

Interestingly, as your app state gets bigger, you are actually forced to use something like cursors for all state updates, for performance reasons. Naive immutability in javascript means clone-before-modify, which is far too slow when your state values are large. It quickly bubbled up in our profiler. Funneling all immutable updates through a cursor means the cursor can use structure sharing on the state value, skipping the defensive clone and preserving reference equality for equal subtrees. Also afaik, implementing a general shouldComponentUpdate (edit: in js) without something like cursors is not possible, because you need === equality for onChange props.

beat• 12 years ago

If ClojureScript had (or has) something like Clojure refs, controlled high performance mutability should be possible.

swannodette4• 12 years ago

refs are not the highest performance reference type due to transactional semantics. ClojureScript already has efficient mutability in the form of atoms and transients. You can actually build a transient vector faster than you can build a mutable JavaScript array in recent builds of V8. I actually demo'ed this at JSConf. Fun stuff.

ericflo• 12 years ago

It's funny, I ran into the same problem a few months back and came up with something similar to cursors as well.

dustingetz5• 12 years ago

There are like six implementations of cursor-y things for React on the internet in varying degrees of quality. I feel pretty strongly that a quality cursor implementation should make its way into react addons before 1.0.

skrebbel4• 12 years ago

One question I have for you: In those large React apps, did you put all state changes in the big single central app state data structure?

I'm currently building an app where the core state is simply a tree of plain old mutable JS objects. We have all the expected disadvantages (mostly: no easy shouldComponentUpdate), but things are easy to reason about, otherwise. We found that stuff like "has the user expanded a dropdown" or "which auto-completeable search term has just been entered" works perfectly in a local component's setState. The idea is, basically, anything that the user doesn't care losing when navigating away from the current screen can go in a local component's state. This seems to go entirely against the functional immutability idea, but it significantly reduces the amount of updates we need to do to our big central data structure.

I'm going to build a new thing and I'm considering going the nice functional way. However, I can't see any downsides to using directly mutable state to simple, local, unimportant stuff.

Would you do it like that? Or would you avoid component state altogether and go with props only?

dustingetz5• 12 years ago

We also use component local state for things like that, and David Nolen (Om) likes component local state too. But the state goes up top by default. The biggest reason for this is because you might want to attach a diagnostic component to your state (e.g. a json editor [1]), and you don't want to have to change the shape of your state to do that.

[1] https://github.com/dustingetz/react-json-editor

jungziege• 12 years ago

Why not make the switch to ClojureScript then? Seems like you're reimplementing things that cljs gives us fro free like immutability and cursors.

skrebbel4• 12 years ago

Thanks for this post! I'm very inexperienced with Clojure, but I'm a big fan of React. I've always wanted to understand how Om encourages immutability and how it deals with "UIs are stateful by definition", but I never managed to wrap my head around it, probably due to my missing Clojure skills.

Your post explained it in general terms. I will probably apply the same in plain JS now. It sounds like an interesting challenge to implement the Cursors idea using React's Update addon [1] instead of depending on Mori. (I don't like Mori because I don't get the feeling that it's intended to be used without ClojureScript - its API is rather cumbersome from plain JS)

[1] http://facebook.github.io/react/docs/update.html

swannodette4• 12 years ago

I'm curious about the "cumbersome" claim. Is it because Mori forces you to write in a functional style? I've tried as much as possible to supply an API that doesn't stray too far from conventions JavaScript programmers are used to from underscore, lo-dash, etc.

skrebbel4• 12 years ago

Oh, it might have very well been me being to much of a newbie to even understand what was going on, so please don't take the claim too seriously.

I believe that it had something to do with it being relatively much work to update things deep in a tree.

Like, if I have Model -> Users -> User -> WallUpdates -> WallUpdate -> Comment, and I want to make an update to a comment, I'd have to do a lot of code writing. Is that true, or did I just miss the magic function that helps me do this?

But I'm now realizing that maybe, if I want to "manually" do that, I'm just doing it wrong - this should partly be what Om's cursors are meant to avoid, right? (this, and locality to components)

In general, if it helps you as feedback, I found Mori a little difficult to approach, having no Clojure experience. I suspect that some more examples of typical usage patterns at the top of the docs would do a lot. Like, which methods are supposed to be commonplace, and which ones are more for edge cases? What kind of things am I supposed to do all the time, what kind of things should I wish to avoid except in these and these cases?

FWIW, I feel like the same holds for Underscore/Lo-Dash's docs, except it's been less of a problem for me because I'm more familiar with typical usage patterns of mutable data structures.

Touche6• 12 years ago

I've tried as much as possible to supply an API that doesn't stray too far from conventions JavaScript programmers are used to from underscore, lo-dash, etc.

This is a nitpick but JavaScript convention is to use camelCase, not underscores, and it feels weird whenever you encounter a library that uses them, usually it's a library that is written by Ruby or Python programmers. It's irritating if it's a library you need to use a lot.

andrejewski• 12 years ago

For people who have wanted to try this type of programming out but have not jumped the boat from JavaScript to ClojureScript, I created a little library that mixes Backbone and React in a similar model described in the article: https://github.com/andrejewski/reactbone

It takes away a lot of the data-binding cruft that you find in Backbone.View and works well with React without making you switch languages, design paradigms, or add new libraries. The Backbone component makes it easier to build data hierarchies than with plain JS, but I assume other MVC frameworks could be retrofitted to work with React.

cnp2• 12 years ago

Examples would be lovely

Touche6• 12 years ago

It's going to be much nicer when ClojureScript runs in Node without needing a JVM. Now a ClojureScript project requires a lot of boilerplate and running cljsbuild auto. For big projects this is fine but I want to write smaller scripts in ClojureScript as well. Basically I want to do:

   cljsbuild app.cljs > app.js

And that's all. No project.clj, no waiting 5 seconds for clojure/jvm to load and Closure to compile it all.

lennel5• 12 years ago

clojurescript run in node? is this on the roadmap?

dkersten4• 12 years ago

ClojureScript already runs on node: https://github.com/clojure/clojurescript/wiki/Quick-Start#ru...

I think what GP is complaining about is the compilation step, which requires a JVM and isn't exactly the fastest.

Touche - You don't need project.clj and leiningen and can run a command basically like you're asking for. I'm not sure how much faster it is than using leiningen and cljsbuild, as I haven't tried it: https://github.com/clojure/clojurescript/wiki/Quick-Start#cl...

atarian• 12 years ago

Pretty much this. If I can get a ClojureScript interpreter (similar to CoffeeScript) that would be amazing and would lower the barrier for a lot of people.

w01fe4• 12 years ago

Engineer @ Prismatic here. I'm happy to answer any questions about the post or our experiences with the rewrite.

dkersten4• 12 years ago

An Om question: what do you store in the app state and what do you store in component local state?

In my own application, I started off storing everything in app state, using local state only for short-lived transient data, but over time I found that having a bunch of information like what tab is currently visible in the app state became difficult to manage and made the app state a nightmare. So now I like to keep a strict separation: app state is basically my model - the data being processed (if I were writing a text editor, its the document being edited) and local state is everything else: which dialog is open, which tab is selected, button toggle state, mouse state etc (in a text editor, whether bold is selected, whether I'm in print preview mode or edit mode, etc). Basically app state is what is being processed and local state is how to process or display it. This separation also means that the app state is what gets synced with the server. I like to think of it as "if its something I might want to persist in the database, then it lives in app state, if its something I would never want in the database, then it lives in local state"

I'm curious what other peoples experience with this is.

Another Om question: how do you sync state between client and server, or how do you handle communication?

I looked at how om-sync does this (using tx-listen to automatically sync changes), but ended up building a more traditional request/response model using sente[0] and having my app ask for sub-trees of my app state as needed. Again, just wondering about peoples experience with this and different approaches tried.

[0] https://github.com/ptaoussanis/sente

cnp2• 12 years ago

#1 is a great question that I've been curious about as well, particularly for larger UI's

w01fe4• 12 years ago

We're still figuring out exactly where the boundary lies, but our current approach is much like yours -- view state that's purely local to a component goes in local state, and everything else goes in app state. This feels more natural, but means you lose some of the nice benefits of things like replay/undo for this local state.

For client/server, we've started building up some nice abstractions around our API (like fetching more data in a generic paginated list stored in a cursor), but the write side is still mostly manual. We're trying to use the same API for iOS and web, which probably prevents us from doing some fancier things that continue the same architecture back into the server.

RoboTeddy• 12 years ago

It sounds like you're differentiating between model state and view model state.

I haven't used Om yet, but a potential approach could be to maintain two stateful objects: one for the model (e.g. stuff that you'd persist permanently), one for the view model (e.g. which tab is selected).

If you wanted your app to come back to exactly the same state after a page reload, you could persist the model in databases, and the view model in browser local storage.

dustingetz5• 12 years ago

I put viewmodel state with the app state - my app state basically is viewmodel state, domain state source of truth is the database really, so you need to do ajax to traverse the domain object graph, its not stored on the client.

Just like you want to control a react.dom.input, you also may want to control a tabstrip. Maybe you have some biz logic that says "When you click save, select the next record, and if record.type===2, select the third tab". You never know in advance what state needs to be controlled from the top, I found myself refactoring my viewmodel state higher and higher.

The mouse state is never going to be controlled from the top, so that is component local state.

lsb• 12 years ago

How do you deal with propagating events through the system? If I'm clicking Follow as people are following and unfollowing, how does that number change?

loganlinn• 12 years ago

Clicking follow mutates the global app data. Changes to the app data causes the entire application to re-render, and React pushes those changes out to the DOM.

So, to answer your question, we don't propagate events :)

Arcanum-XIII• 12 years ago

Maybe a stupid question, but since I didn't see anything about that : what kind of database do you use, and how is it with Clojure ? React + ClojureScript seem a given for me on the interface side, especially with core.async and the other possibility, but I'm a bit more weary for the server side, and that's one of the last showstopper I have.

And, how do you go for teaching new comer on the project ? I do suppose you don't only hire Clojure or functional dev, so the transition part seem very interesting, especially with Om getting more (deserved) attention !

presty4• 12 years ago

not talking for OP

most popular databases have native libraries developed by the community (postgres, redis, mongo, etc)

but since Clojure runs on top of the JVM, you can probably access any database through the Java World

ds_• 12 years ago

This is also something I'm curious about, particularly how migrations are managed within a team. Most of the solutions I've seen require writing plain SQL in migration files which you name yourself. It's simple but I miss the productivity of DjangoORM or ActiveRecord for example. Or do most people use a schema-less database?

stu_k• 12 years ago

What do your "templates" look like, and how easy they are to edit? In Montage we've kept our templates as close to plain HTML documents as possible, and the CSS experts/designers we've worked with have really appreciated how easy it is to understand.

I'm wondering if this is an issue for you (or perhaps everyone is comfortable enough with the ClojureScript/Om syntax that this isn't a problem)?

bguthrie• 12 years ago

Strategically, how did you approach the rewrite? It sounds like you had a significant amount of frontend code; replacing everything at once can be risky. Did you choose specific pages and work your way in, or scrap everything and work your way from the inside out?

fdsary• 12 years ago

I'm a Javascript/Python/Ruby professional, and have made a few really swell SPA's in JS.

But my dream is to be a really good Clojure(Script) lisp programmer. I'm not so good at any lisp, but have went thru a couple of books, try to do every tutorial and to play in the LightTable instarepl and implement things that I can easily do in JS.

How can I, who am far from a lisp pro, get a job in a lisp/clojure shop? I think by doing something 8h/day in a team is the best way to learn, but of course no one will hire you if you slow down the rest (at least in the beginning).

presty4• 12 years ago

I'm interested in knowing more about how people are architecturing their Om projects

e.g. are you using something like:

app component <-> page components <-> (widget) components

where the app creates the page and the page creates the widgets and communication is flowing through channels?

and how do you deal with url transitions?

do you capture and flow up the click from a widget back to the app so it knows it's time to render a different page? and have some (case ...) at the app level?

or using some kind of mix with secretary and changing the location.href/a hrefs?

etc

w01fe4• 12 years ago

Yep, that sounds about right. For us, API communication and some global events flow through channels into the app state, where it's distributed to components. For routing we've built up a library around secretary, where we write down a map of uris, handlers, and metadata (can be a modal, etc). Info about the current page and such goes into the app state and that drives the transitions between page components.

haberman2• 12 years ago

One thing that's unclear to me: does this require the entire application state to be loaded into memory all the time?

If your application state is large but you only care about a few parts of it at a time (a few "cursors") how does Om deal with that?

swannodette4• 12 years ago

Not sure what you mean? Om uses whatever state information has been delivered to the client browser same as other JavaScript approaches/frameworks.

haberman2• 12 years ago

I saw some stuff about cursors as a way of displaying/modifying only a part of the global state. But it wasn't clear to me whether this is a formalized way of only loading part of the global state, or if cursors still require that the entire underlying "global state" is loaded into memory.

To give more background, let me give more specifics over what I'm thinking about when I ask this.

I've been working on a JS accounting app. One thing about accounting is that the balance of an account is dependent on every prior transaction that the account has ever had. So modeled in functional terms, it seems like you'd have to make this a tree of depth "N" where N is the total number of transactions in the account. It seems like you have to model it this way to get the data dependencies right (a change in an early transaction affects every subsequent transaction).

A functional approach with such a deep tree seems to break down (at least from the relatively little I know about Om). For one, I don't want to have to load every transaction into memory. Even though the most recent transaction depends logically on the first one, I don't want to be forced to load the first one just to show the most recent one. One possible way to work around this is to collapse all transactions prior to the ones I want to display into a single "fake" transaction that stores the accumulated balance as of this point. But now my in-memory model has to differ from my real model, and as the two diverge things start to get more complicated.

A bigger problem: I can't think of a good way, with this design, to indicate to the data layer which transactions the UI actually cares about and wants to have loaded into memory. If the tree has only one logical root (the most recent transaction) and is logically N transactions deep, how does the UI indicate which of those N it actually cares about? Maybe Om cursors can do this somehow?

I've been using a more traditional mutation-based/observable design, which seems to be working well. The UI can "subscribe" to certain transactions/balances, and this serves as a signal to the data layer about what needs to be loaded and what derived values need to be kept up to date. The data layer can be smart about how many transactions it actually needs to load to compute what the UI cares about.

dkersten4• 12 years ago

I've been a very heavy user of Om for the past three months, so here's my take on it.

In my application, there is a lot of data that doesn't always need to be loaded - for example, various metrics can be viewed in charts, but since there can be a lot of data points, you wouldn't want to load metrics which aren't being viewed (since there could be many dozen different metrics, yet you may only care about 3 or 4 of them). So I think this is a good example for your question.

In itself, Om does not deal with this and assumes that the entire application state is loaded at all times. As far as Om is concerned, everything is always there in a big tree structure.

But since Om lets you update the state easily through cursors, it is easy to update sub-trees when needed. So there are two very similar approaches that I've tried out:

1. If you have data where you basically select one of a collection datasets (eg in my application, if I can only chart one metric at a time), then you can have an end-point in the tree for this data and you attach the currently selected data when it is needed, swapping the old for the new. You can easily do this with an om/update! command. It is also easy to then send the old data to the server while the new data is loaded into the state, if you have changes you need to keep in sync (om/transact! gives you the old value).

2. I have experimented with creating Om components that read the value of their cursor for some sentinel value (maybe nil or a special keyword or something) that tells it the data is not loaded, which then requests the data from the server before passing it to its child component for consumption (perhaps throwing up a loading indicator in the meantime). You can then also implement logic to unload data again (maybe a timeout after the child stops using it). This works quite well for me.

A possible improvement on #2 that I want to try out when I get a few hours is to make use of Clojures lazy sequences, rather than using a sentinel value to implement lazy loading. It seems like a natural fit, but I have yet to try it.

There are, of course, many other approaches one could take (eg, keeping track of what's loaded in a collection instead of using sentinels), but I have not tried any others.

So I guess, in summary, no, Om doesn't deal with this for you, but it gives you the tools to build it yourself to suit your needs. Personally, I like it this way because I know my access patterns better than Om does.

w01fe4• 12 years ago

No, you are still free to organize your application state however you wish. For example, our app dynamically pulls in content as you browse from feed to feed and use infinite scroll. It's up to you to decide what to fetch when, and what to cache/discard as the user navigates around.

kylebrown2• 12 years ago

I'm real curious about the potential interplay between core.async and om/react. Bruce Hauman has a great tutorial on using core.async channels (as opposed to event callbacks) to process user input and gestures.[1] David Nolen (creator of Om) also covered core.async.[2] But these tutorials pre-date Om; core.async came to clojurescript in summer 2013, Om wasn't released until 2014.

I guess the way React handles input is by attaching event handlers to child components, so that changes to the child properties bubble up to the root component state. Is this the optimal pattern for handling events in a React app? Or is there an advantage to be gained by using core.async channels to process event input for Om/react?

I haven't seen any examples which use both core.async and Om/react, but I'm very curious about the possibility.

1. http://rigsomelight.com/2013/07/18/clojurescript-core-async-...

2. http://swannodette.github.io/2013/07/12/communicating-sequen...

diego• 12 years ago

Check out https://github.com/sgrove/omchaya

presty4• 12 years ago

I haven't seen any examples which use both core.async and Om/react, but I'm very curious about the possibility.

that's strange, because Om's tutorials use core.async

https://github.com/swannodette/om/wiki/Basic-Tutorial

https://github.com/swannodette/om/wiki/Intermediate-Tutorial

kylebrown2• 12 years ago

Ah yup, there it is in the basic tutorial, "Intercomponent communication". It is strange that I don't remember noticing that when I skimmed it. I must've been too distracted counting parentheses and trying to wrap my head around clojure syntax.

I'm relieved to know it was there the whole time and I simply missed it, thank you.

jlehman4• 12 years ago

I rewrote the AngularJS tutorial in Om [1] and utilize core.async there.

The basic [2] tutorial on the Om wiki uses core.async heavily as well. Also, there are a ton of Om examples [3] within the repo itself that are a good reference.

1. https://github.com/jalehman/omtut-angular

2. https://github.com/swannodette/om/wiki/Basic-Tutorial

3. https://github.com/swannodette/om/tree/master/examples

Om Sweet Om: functional front-end engineering with ClojureScript and React

Related Stories