My other database is a compiler

https://blog.chiselstrike.com/my-other-database-is-a-compiler-10fd527a4d78

by alainchabat • 4 years ago

27 13 4 years ago

blog.chiselstrike.com

AzzieElbab• 4 years ago

Interesting. Aligns with something Martin Odersky(creator of Scala) said a while ago

Zababa2• 4 years ago

Isn't this what an ORM does? You write "User.findOne(user => user.name == "Glauber Costa" && user.age >= 40);" and it will be translated into SQL. Maybe I'm missing something here, but I don't see the innovation.

penberg• 4 years ago

Remember, that argument to `findOne()` is an arrow function. You need a compiler to turn it into a efficient, deferred query; otherwise you need to _evaluate_ the function at runtime.

Zababa2• 4 years ago

So the innovation here is instead of doing "User.findOne({name: "Glauber Costa", age: {superiorOrEqual: 40}});" you do what's in my previous comment?

waynesonfire2• 4 years ago

This is the disconnect I've found in my self-taught compiler education journey. Every time I try to tackle this topic, I get bogged down by parsers and lexers. I already have that structure.

Users.find(x => x.name == "Bob").

My programming language has done that for me. What's the next phase? What data structures do I need to maintain to construct these DSLs? How do I optimize them?

I can fight my way through this but I wish I just had the theory and making this kind of stuff easier. It just comes up so often in the problems that I solve and I'd like to reach for this tool more often.

freemint• 4 years ago

I find it said that query plans an SQL database generated were not discussed more. Drawing this comparison and highlighting differences would have been very helpful.

rurban• 4 years ago

newer postgresql also jit's some queries in its new query optimizer.

but for such use-cases (exact matches, no range queries) it's still better to compile a perfect hash at run-time into a .so/.dll and load this at run-time. way faster.

keyless_• 4 years ago

A long held view of mine is that our current architecture where we keep logic on the server and data in the database is rather unfortunate, because it means you either have to

* marshall data to your server to process (which can be slow), or

* marshall code to your db to run (which can be messy, in part because you're translating from one programming language to your db's language - this is the main reason ORMs exist)

I always thought that the solution is to have all computation live in the DB (similar to how kdb does it), but this is pretty nice too. If you squint your eyes this looks like a much cleverer ORM.

It would be even better if the language supported SQL as a first class citizen, but that's not really an option here, since you're stuck with js.

timeserious2• 4 years ago

fyi kdb+ now supports ANSI-SQL. https://code.kx.com/insights/1.1/core/sql.html

Including support for the PostgreSQL wire protocol https://code.kx.com/insights/1.1/core/pgwire.html

jnordwick• 4 years ago

KDB+ has always supported SQL92 I thought (minus the DDL stuff). I've seen and edited Arthur's files (and blind now). Much of '99 was there too when k had those alternate syntax libraries just translated down to k (may still be that way). How is this different?

When FD picked up Kx, I was kind of hoping for better admin tooling than support for poor query languages that you shouldn't even be writing in. I know it is there as a necessity to help move code to kdb, but the more sql the less kdb shines. Its strange to push something that makes the product inferior. sql is just a really bad fit for a column db.

My other database is a compiler

Related Stories