HN user

zainhoda

441 karma
Posts11
Comments141
View on HN
Large Enough 2 years ago

I’m waiting for the same signal. There are essentially 2 vastly different states of the world depending on whether GPT-5 is an incremental change vs a step change compared to GPT-4.

[dead] 2 years ago

I think of all the options for “carrots” and “sticks” that companies are offering for RTO, I like this one the best.

That's a fair concern. In actual usage, however, the vast majority of hallucination (>90%) tends to be:

- phantom tables and columns, in which case the query will fail

- incorrect syntax for date functions (i.e. the wrong flavor of SQL)

And we tend to see less of this type of hallucination when there are lots of example SQL queries that have been "trained" into the RAG system.

Author of the package here. Apologies if this wasn't clear in the documentation, but what you're talking about is absolutely possible. Feel free to join our Discord -- we have other users who have this multi-tenant setup.

One added benefit of RAG is that it's more "pluggable." It's a lot easier to plug into newer LLMs that come out. If and when GPT-5 comes out, it'll be a one character change in your code to start using it and still maintain the same reference corpus.

What would be missing from that is the ability to look up what tables are in the database, how they join together, what terminology the business uses, preferences around whether to exclude nulls from certain columns, etc.

This allows you to ask more of a “business question” like “who are the top 10 customers by sales?” and the LLM will be able to construct a query that joins the customers table to an orders table and get the results that you’re looking for.

With a simple NL to SQL, you’d have to say “join the customer table with the sales table, aggregate on the sales column from the orders table and limit the results to 10 rows” or something along those lines.

Author of the package here. Thank you! That's exactly what we've seen. Businesses spent millions of dollars getting all their structured data into a data warehouse and then it just sits there because the people who need the information don't know how to query the database.

100% -- in fact originally the package used to parse out SELECT statements and only execute SELECT statements. After some feedback, we decided that the permissions on the user should handle that level of detail.

Author of the package here. Joining 5 tables is not a problem.

The training does not necessarily require you to write the queries by hand. A trick that we've seen people do is to just train with DDL statements and then ask "leading" questions if it can't answer on the first try.

I've been using the package myself for about 6 months and while I haven't forgotten SQL, what I have forgotten are the fully qualified table names and which tables live in which schemas etc since I never have to think about that.