An Opinionated Clojure CQRS/ES Using Onyx, Datomic, DynamoDB, Kafka

http://yuppiechef.github.io/cqrs-server/

by CmdrDats • 11 years ago

111 33 11 years ago

yuppiechef.github.io

chadly• 11 years ago

The current relational database is too limiting. We're dropping all sorts of interesting data on the ground because we don't have suitable pigeonholes to put it into.

That is the best description of the problem event sourcing tries to solve that I have seen. Short and to the point.

mrits• 11 years ago

I disagree. I mean sure, I put humans and radiation in the same pingeonhole, but thats more of a devops problem to work out.

solipsism2• 11 years ago

Maybe to someone who's already familiar with other, longer descriptions of the problem. I don't know what problem this is solving and that description doesn't do much for me.

Can someone either explain, or point to a fuller explanation, of the purported problem?

Terr_3• 11 years ago

There are some overlapping architectural ideas here, I find they build upwards like this:

CQRS: You have one "write" data-model which is specialized for transactions and applying business rules... and other "read" models that take care of things like dashboards, detailed information pages, searching, generating reports, etc.

Chronological events: A really handy way to propagate changes from the write-model into the various read-models.

Event-sourcing: This is "dogfooding" your own events. Instead of just emitting optional messages to other systems, you make them essential, and use them as the authoritative record for your own system's state. This adds certain kinds of flexibility that you may (or may not) need.

jamii• 11 years ago

This is an excellent explanation - http://blog.confluent.io/2015/01/29/making-sense-of-stream-p...

Terr_3• 11 years ago

I'd add: "All data-models have strengths and weaknesses. The event-log is weak on its own, but its strength is that it nurtures an ecosystem of other models for specific use-cases."

dragonwriter• 11 years ago

That is the best description of the problem event sourcing tries to solve that I have seen.

Huh. I always thought that the problem event sourcing was trying to solve was more that modeling complex interactive systems through their static state rather than modeling the interactions directly as the principal domain objects is invariably futile. While "we're dropping all sorts of interesting data on the ground" may be one manifestation of that, I think the more common manifestation is "we bear too high of an expense when adapting our information system to changes in business process".

solipsism2• 11 years ago

modeling complex interactive systems through their static state rather than modeling the interactions directly as the principal domain objects is invariably futile

Now that actually makes sense to someone not already familiar with the context.

jared3145• 11 years ago

I was under the impression that Datomic alone could handle everything involved with an eventstore, just by adding the domain event as an attribute on the "transaction" (because transactions themselves are also entities). I don't understand what this adds.

CmdrDats5• 11 years ago

Technically it could - but I felt Datomic is particularly strong as an aggregate view. Adding transactions that just have events attached to them isn't a particularly useful aggregate to query.

Having the eventstore separate means that you can keep Datomic focussed at what it's really good at. When you find good aggregate views for your events, you can scan through old events and populate Datomic.

This separation also means that you can make the events as rich and plentiful as you want without burdening Datomic with stuff that you don't have a good way to query anyhow.

jared3145• 11 years ago

Adding transactions that just have events attached to them isn't a particularly useful aggregate to query.

I still don't understand.

The transaction log, in datomic, can be queried as easily as you would query the rest of the data. The view exists on the client. Changes are already queued by the transactor. And, changes are pushed to the clients from the transactor.

hueyp• 11 years ago

The event is the transaction.

For example of I have a 'make_user_preferred' event, I just transact the database and then add any metadata I need to the Datomic transaction.

Datomic is Event Sourcing if you create a realized version of your database on every event. It is just smart about it and keeps diffs.

mandeepj• 11 years ago

There is a guy by name Udi Dahan. He is a self proclaimed CQRS guru. He also sells his trainings for at least few grands. I have listened to his podcasts and read his blogs but I never got any clue what is CQRS and under what scenarios it is best to go with this pattern. Here, I just read the first page and I got it what it is and when I can use it.

_pmf_• 11 years ago

I like this trend of indicating articles as being opinionated. It acknowledges the fact that there's no be-all end-all solution and prevents having to include useless fuzzy apologies and hand waving in the article.

crumley• 11 years ago

I really like the ideas behind CQRS and Event Sourcing. So much so we are building a platform using these patterns.

Want to live in Austin, TX and build a data platform from the ground up? We're hiring:

https://careers.stackoverflow.com/jobs/80205/senior-develope...

An Opinionated Clojure CQRS/ES Using Onyx, Datomic, DynamoDB, Kafka

Related Stories