Uplevel database development with DataSQRL: A compiler for the data layer

https://www.datasqrl.com/blog/lets-uplevel-database-datasqrl/

by mbroecheler • 3 years ago

4 7 3 years ago

We'd love for you to join us in building a high-level data development language to simplify data-driven application development.

nerpderp823• 3 years ago

Is this similar in spirit to Noria?

https://github.com/mit-pdos/noria

mbroecheler4• 3 years ago

Yes, the idea to maintain materialized views based on standing queries to make the queries instantaneous is the same. In addition, DataSQRL handles the ingest (e.g. consuming events off a queue, pre-processing the data, and populating the database) and egress (i.e. serving the data through an API) so that all your data logic can be in one place.

Another key difference to Noria is that DataSQRL is an abstraction layer on top of existing technologies like Postgres, Flink, Kafka, etc and does not aim to be another datastore. That way, you can use the technologies you already trust without having to write the integration code.

nerpderp823• 3 years ago

This sounds wonderful! And it validates many of my own thoughts. :)

Your product would align nicely with these DAG recomputation engines like Fluvio and Temporal (Seattle).

Well Noria implements MySQL protocol, so if your system targets MySQL, you could run DataSQRL on Noria!

Uplevel database development with DataSQRL: A compiler for the data layer

Related Stories