Datomic, Datalog, GraphQL, APIs, etc.

http://tank.hyperfiddle.com/:dustingetz!timothy-baldridge-datomic-datalog/

by tosh • 7 years ago

177 11 7 years ago

tank.hyperfiddle.com

rmbryan• 7 years ago

If "Hyperfiddle is after more of a vision where like, what would it take for my mom or my future kids to be programmers and yet not be professional software engineers." then I suggest you focus on drastically simplifying the vocabulary you use to explain what Hyperfiddle is.

For example: "This is a pastebin like environment for learning Hyperfiddle. Please namespace :idents with your username Schema changes are allowed To make a fiddle interactive, change tank.hyperfiddle.net to demo.hyperfiddle.net"

My mom no grok: pastebin, :idents, namespace as a verb, schema, "change" without a where to change.

Granted, this is a conversation between to engineers, so there's lexicon, but even listening, I get the sense that words get used without definition, or at least without a definition that speaker and listener share.

dustingetz5• 7 years ago

For those out of the loop - At Hyperfiddle we are building a tool for Clojure/Datomic programmers to make back of house dashboards easily. http://www.hyperfiddle.net/ Because Hyperfiddle/Datomic has such a simple information model, we see a path to eventually making this easy enough for code-savvy amateurs. But today you need to know Clojure.

jives• 7 years ago

How did you uncover the pain point around making these internal dashboards? What kind of resistance have you encountered from developers who typically enjoy rolling things from scratch?

frankmcsherry2• 7 years ago

For some context on "what Niko presented at the Conj",

    https://2018.clojure-conj.org/nikolas-gobel/

which is a reactive Datalog implementation built over differential dataflow. The .. um reported anxiety that to add new queries requires a recompile loop, replaying all of history, etc, suggests maybe they aren't entirely on top of how the system works.

Edit: and again tomorrow at clojureD:

    https://clojured.de/schedule/#nikolasGoebel

dustingetz5• 7 years ago

hi Frank, we are definitely not entirely on top of this stuff, but it is very awesome and we will get there eventually :)

Can you help me understand where we went awry here:

"Now of course the problem there with a rules engine... I have to look at the incremental dataflow, but I imagine it's kind of the same sort of approach. The problem with that is that if you want to sort by a new thing or you have a new piece of information, you have to change the network and backflow any... [You have to know compile time...] Yeah. Yeah. Exactly. And there are systems where you can add a new rule at runtime, but a lot of those also involve then re-ingesting history into the network to repopulate anything they... 'Cause a lot of these systems to save memory, will just throw away any information you're not interested in, from compiled query."

If we are close can you tighten up the language and if we are far where did we go wrong? (In the context of differential dataflow, not rules networks)

frankmcsherry2• 7 years ago

No worries!

Niko's talk should probably be the reference material, but you definitely can deploy new queries interactively, against maintained indexed representations of the historical streams. No recompilation required. Streams are maintained in indexed form and don't need to be re-flowed (they may need to be re-indexed if your query calls for an index that doesn't exist).

Deploying a new query takes ~milliseconds if the correct indices exist, and if not it seems to be no slower than purpose-built Datalog systems (e.g. LogicBlox, but performance seems solid compared with Souffle). This is despite then running against live-updated data.

As I suspect you know, we cannot report performance relative to Datomic. ;)

dustingetz5• 7 years ago

also see the healthy reddit clojure discussion: https://www.reddit.com/r/Clojure/comments/as16f6/transcript_...

fiatjaf• 7 years ago

I love Hyperfiddle, but I'm not good enough to use it.

dustingetz5• 7 years ago

Hyperfiddle 2.0 landing soon!

bzg• 7 years ago

\o/ Can I squeeze a last minute request? Allow to login to http://demo.hyperfiddle.net for users who don't have a Google account.

In any case, I love the idea. Looking forward!

Datomic, Datalog, GraphQL, APIs, etc.

Related Stories