maxdemarzi

Wouldn’t it be nice if your database could help you fine tune your model for better faster and cheaper text to sql? That’s what we’re exploring on this post.

I'm happy enough to use LLMs for code, but I think it's nuts people are using it for their PERSONAL blogs. It's supposed to be me talking to my super niche audience. Never gonna happen.

I can't shake the feeling that something is off about “context graphs”. This is my attempt at trying to explain why I feel this way.

The current benchmark for real world text to sql is https://spider2-sql.github.io/ and the SOTA is at 64%

Anyone selling you 99% accuracy can prove it there first.

I’ve had tattoos removed and currently removing two more. They are all 20 years old, very faded, blurry they honestly just didn’t look good any more.

The first removal 5 years ago would leave me bleeding and require bandage changes for a week.

The new tattoo removal lasers leave you feeling like a sun burn for two days. No blood, no bandages.

It will become way more common.

Tell me more…

It’s $275 a month plus $99 every 6 months at SlimDownRX. The compounded pharmacies are helping.

The 14th way is “multi way joins” also called “worst case optimal joins” which is a terrible name.

It means instead of joining tables two at a time and dealing with the temporary results along the way (eating memory), you join 3 or more tables together without the temporary results.

There is a blog post and short video of this on https://relational.ai/blog/dovetail-join and the original paper is on https://dl.acm.org/doi/pdf/10.1145/3180143

I work for RelationalAI, we and about 4 other new database companies are bringing these new join algorithms to market after ten years in academia.

Quite the opposite. The idea is to move as much of the business logic into the database. “Rel” definitions are meant to be written once and reused everywhere. Instead of letting the client decide different business logic every time, you capture and control it in one place.

It works a little different in “Rel” (the query language Relational.ai uses). You would create multiple definitions of what a “full polite address” is for each “case” of valid arguments/empty columns and use that going forward. A bit like a UNION without the same column width requirements.

You can by using “narrow” tables (key value and key key). Download the slide notes from this presentation of how Relational.ai is doing it: https://www.slideshare.net/maxdemarzi/developer-intro-deckpo...

They don't have joins at all because of how expensive binary joins are to do. NoSQL pre-joins relations (graphs dbs), pre-joins foreign keys (document dbs) , pre-joins everything (queries) (wide column dbs). Saying "all" is a bit of a hyperbole but it gets to the point of the matter.

It took about 10 years, but worst case optimal joins and multi-way joins in general are finally fixing the Join problem in databases that led to the proliferation of NoSQL systems over the past decade.

Author here: I did not write that I was proud of not knowing Python. I just wrote that I don't know Python. The thought of trying to understand 2k lines of it looking to see where Memgraph 'cheated' to make their product look good and the other bad was beyond my current capabilities.

Author here to clear up a few questions: I did not run any benchmarks for Memgraph, just Neo4j on my machine and compared them to their numbers on their machine. My 8 faster cores to their 12 slower cores, so not apples to apples, but close enough to make the point that Memgraph is not 120x times faster than Neo4j. I used to work at Neo4j, then at AWS for Neptune, I work on my own graph database http://ragedb.com/, and work for another database company https://relational.ai/

If you want to be my hero, find a way to fix this problem: https://maxdemarzi.com/2023/01/09/death-star-queries-in-grap...

The lack of a Schema does hurt Neo4j performance. Properties are stored "willy nilly" on a linked list of bytes per node/relationship. No order, an "age" property can be: 45, 38.5, "adult", [18,19], false... and that makes a terrible mess when aggregating, sorting, filtering, searching, etc.

It will stop your hunger, but you will still fight your bad habits or social eating. If you eat like “normal” you will have a bad time. Your stomach will hurt, you will feel miserable. Gotta have your partner help and stop eating out or eat separately for a few months.

You are right. This code runs just fine on both of these online Lua websites. local tab = {} tab[938388893] = "hi" tab[987383332] = "Hello"

for k,v in pairs(tab) do print(k) print (v) end

https://www.lua.org/cgi-bin/demo https://luajit.me/

But when I try it using the Sol library (embedded in C++) it eats all my memory. Is it a Sol issue or a C Lua binding issue? Anybody know?

One Lua thing that is wacky is that you cannot have large integers as keys to a table. Lua converts {2938433:"hello", 983748323:"hi"} to positions in the array portion of a table and creates a giant 99.999% empty table eating all the RAM. Unless somebody knows something I don't on how to make it work.

How are you storing and querying your graph data? Are you using a graph database underneath all that or ?

What were the stats on each Shard? Was each shard running 4 cores? 8 cores? 32 cores? RAM? The blog post is light on details.

Get your Testosterone checked in the early morning two times (about a week apart). If you are under 300 both times, go see a Urologist for treatment. It could be your hormones.

Sales Engineering is the best. You get to build things to help customers and generate revenue. You get to code and present. Learn your customers business and teach your businesses product.

Does another party need to decode the url? What about using a dictionary for the top 10k seen starting combinations and then encode the rest?

What about run length encoding? 1-9 for positive sequences. a-i for negative sequences (max means pattern continues) and the rest for frequent patterns like alternating sequences, etc

9967b would be 24 yes, 1 no, 7 yes, 3 nos, 1 yes etc

Querying data in “Cypher” is so much easier. I have 20 years as a database developer, cypher is way better. I can live code 50 complex queries in Cypher before I get one done in SQL.

Yup. Same with Neo4j. You can write your stored procedures in Java ( or any JVM) language. Import any library you want, go crazy.

Neo4j is by far used more than any other graph database out there. Regardless, the standard is not Neo4j specific. It was a collaboration from many companies in the space. They all agree it's a good move.

Gremlin was written by a genius level developer to be used by other genius level developers. There are maybe a handful of Gremlin experts in the entire world and less than 100 that are any good at it.

It is extremely powerful, but after a few lines, the mental acrobatics needed to understand what the query does is beyond your average developer.

My first paid Neo4j gig 7 years ago was writing a rules engine in gremlin. It was about 25 lines of code. If you were to ask me today what each of those lines did, I would be at a loss. So would anyone who didn't live in those specific queries day in and day out.

Graph adoption was severely limited by its use. Cypher can be learned in a day, and "business people" can look at a cypher query and understand what is going on for the most part.

It takes about a week to "bolt on" Gremlin to any database. I've done it myself, that's why you see it so often. It takes months to be any good at it.

Sure. Amazon Neptune is garbage, but that's not the issue. They are competing fair and square here. The problem is when they fork <insert liberally licensed open source project> completely.

If you used Neo4j you would get an easy hierarchy. If you need multiple inheritance then no problem. If all of your classes have completely different properties then no problem. If some discovery reclassified the tree somehow then no problem. But it all depends on how important this use case is to the overall application.