HN user

AntonioL

7 karma
Posts2
Comments9
View on HN

Reminds me of the work from Bob Coecke at the University of Oxford. He came up with a pictorial language for quantum processes.

At my shop we do exactly this, we do ELT as opposed to ETL. (E=Extract, L=Load, T=Transform). We put our input files (think of JSON documents) in the database, and then the data processing is a materialised SQL query. This has a few benefits: - The SQL dumps are very light as they will consist only of the input files, the materialisation is just a query, no need to store the transformed data. - Some time there is an error in the business logic, how do we backfill? That is easy, we update the body of the materialised SQL query and then refresh the materialisation. - Transactions are very hard.

We find this great for batch use cases.

Exciting to see progress in this space in the last years: - materialize for incremental maintenance of materialised views - postgres has a patch to start support incremental maintenance of materialised views in the works .

An impressive startup in the UK in this space was Wrapidity.

It is my favourite university spinout story here on this side of the pond.

Hi David,

Congrats on the funding!

I am wondering whether retool can fit this use case? I have some bunch of data which I would like to sell, and would like to give a dataviz tool to a 3rd party customer to play with my data.

Are you aware of any customer of you with such an use case?