Introducing Incremental

https://blogs.janestreet.com/introducing-incremental/?utm_source=rss&utm_medium=rss&utm_campaign=introducing-incremental

by kilimchoi • 11 years ago

188 50 11 years ago

blogs.janestreet.com

tomjen3• 11 years ago

Am I the only who could can't find the RSS feed for that blog? Pretty strange for a tech blog to only allow subscriptions via email.

kilimchoi2• 11 years ago

https://blogs.janestreet.com/feed/

lifeisstillgood2• 11 years ago

I had no idea RSS was still expected... (Not trolling just surprised)

hvs• 11 years ago

How else would you follow blogs? Do you visit 20+ blogs to see if they've been updated recently?

coldcode• 11 years ago

In my blog I support RSS and twitter to give people a choice.

e12e• 11 years ago

From the source: <link rel="alternate" type="application/rss+xml" title="Jane Street Tech Blogs » Introducing Incremental Comments Feed" href="https://blogs.janestreet.com/introducing-incremental/feed/" />

mands2• 11 years ago

Looks like really cool tech - at a first glance reminds me of the UI.Next reactive rendering model in WebSharper/F# (http://www.websharper.com/docs/ui.next) which also has an OCaml/ML heritage.

Have been thinking about using OCaml again for a new project - does anyone know the state of libraries for common web tasks, like AWS, these days?

seliopou• 11 years ago

I've been using OCaml to build a language and runtime system for cloud orchestration[0]. As part of that, I've developed an AWS library whose code is generated from the service descriptions published by boto. It will be open-sourced at some point in the future, along with a few other things that should make developing web stacks using OCaml much easier.

If you're looking to try out OCaml, I'd say the time to do so is quickly approaching.

[0] http://www.inhabitedtype.com/

mands2• 11 years ago

Great to hear, sounds like an fascinating project and look forward to trying out the AWS library. Funnily enough the project I considered using OCaml for was to build a language & runtime for container orchestration.

Yes, OCaml dev seems to be picking up again - I used to use OCaml/JoCaml a few years ago but moved to Haskell and F# in the meantime. However am very excited by the work on Flamba, multi-core and hopefully modular implicits, as OCaml always felt most natural.

rubiquity• 11 years ago

I've been learning OCaml lately and I can't help but wonder (aside from the fact that it's an ML lang) why OCaml didn't capture the infrastructure tooling space and Go did. OCaml is so much fun to write, easy to reason about and has pretty good TLS/SSH libraries. Here's to hoping OCaml grows more!

pron• 11 years ago

We've made something very similar for Java/Clojure as part of Quasar: http://docs.paralleluniverse.co/quasar/#dataflow-reactive-pr...

Every Var keeps track of the inputs that affect its computed value, and has an associated continuation responsible for carrying out the computation whenever an input changes (which, in turns, triggers recomputations further down in the graph).

pvitz• 11 years ago

I would love to see an OCaml alternative to Prophet/MoSes using stuff like this...

an_ko• 11 years ago

Could we have a more informative title? "Incremental" means little without context.

Maybe just "Introducing Incremental: self-adjusting computations".

silverlake• 11 years ago

Does this have any similarity to Goldman Sach's internal language called Slang/SecDB?

achamayou2• 11 years ago

Yes, and to several other similar systems, it's another reactive graph.

taion• 11 years ago

One distinction is that Goldman's SecDb graph is really optimized around offline derivative pricing and risk calculations, which has always been one of the core strengths of Goldman's quant platform. There is a slight distinction between reactive graphs used for those approaches v. those that are more targeted toward real-time online applications.

gaius3• 11 years ago

Such as https://github.com/manahl/mdf

infinite8s• 11 years ago

How is a reactive graph different to a complex event processor? Is it the difference of pull vs push?

hellofunk• 11 years ago

What is the difference between a "self adjusting computation" and a "genetic algorithm," or is it merely a choice of terminology? Here is a scholarly article that uses both in the same title: http://link.springer.com/chapter/10.1007%2F978-3-540-87734-9...

AaronFriel• 11 years ago

It looks like a self-adjusting computation, in the sense that Jane Street and the paper they refer to uses, is similar to functional reactive programming. It looks like you can create observable values, and create observable computations on those values. The dependency graph is dynamic, which it appears is the improvement here over other FRP designs?

Genetic algorithms are a different sort of thing, and I can see how "self adjusting computation" could be used to describe either.

seanmcdirmid• 11 years ago

Functional reactive programming is not usually incremental, while self adjusting computations are not usually reactive, which is why there isn't much overlap between the concepts. Of course, you are free to mix concepts from both as you need to.

loosescrews2• 11 years ago

How is this different than lazy evaluation?

mercurial4• 11 years ago

Actually, that's an interesting question. Having built similar systems before, the main reason why you would not simply run a function every time your input is updated is that you often need for your computations to keep a certain amount of state, which may depend on the input. However, as Incremental does not seem to support much in the way of stateful computations, I'm a bit puzzled wrt to what problem it addresses.

hebdo• 11 years ago

Quite different. To achieve the effects of "Incremental" you would need lazy evaluation coupled with a caching layer, so that successive calls to pure functions with same arguments are not re-computed. This optimization technique is in a sense similar to Common Subexpression Elimination.

loosescrews2• 11 years ago

Doesn't Haskell already do this? What is the advantage of OCaml + Incremental over Haskell?

mercurial4• 11 years ago

But this is possible to do trivially using a hash keyed by the input variables. It seems to me that SAC offers an interest only in the case where:

- the input variables of the DAG are updated at a different rate

- and the result of complex subgraphs depend on only a few of these variables (and thus avoiding to recompute their result is a win)

I don't know if it's enough to get excited about.

makmanalp• 11 years ago

This reminds me a bit of Elasticsearch Percolate (https://www.elastic.co/guide/en/elasticsearch/reference/1.6/...) a little bit.

EvanYou2• 11 years ago

I'm not familiar with OCaml, but this looks very similar to the reactivity model used inside web front-end libraries like Knockout, Vue.js - in which each "binding" in the template is essentially a self-adjusting computation that updates the DOM. In addition, the Tracker library in Meteor enables similar dependency-tracking computations across the entire stack. It's interesting to see this type of reactivity model used in different contexts.

gaius3• 11 years ago

You've got it backwards. This concept long predates the existence of the web, let alone JS.

EvanYou2• 11 years ago

I never implied which predates which. Read again.

ScottBurson• 11 years ago

This sounds very similar to Kenny Tilton's Cells package for Common Lisp: https://github.com/kennytilton/cells

I haven't had occasion to use Cells, but I'm glad to be reminded of it, as I may have a use for it soon.

Introducing Incremental

Related Stories