A hyper-fast local vector database for use with LLM Agents

https://github.com/jdagdelen/hyperDB

by lopkeny12ko • 3 years ago

74 21 3 years ago

github.com

superkuh• 3 years ago

A "DB performance vs VC funding raised" plot is an odd choice for a github repo dealing with code.

theolivenbaum• 3 years ago

I think that was perfectly on point if you reviewed the code.

brigadier132• 3 years ago

Absolutely hilarious. You're ready to raise $18 million.

egeozcan• 3 years ago

https://github.com/jdagdelen/hyperDB/blob/main/hyperdb/galax...

This was a great laugh. Also, after the benchmark, it says:

Benchmark Credit: Benim Kıçım

"Benim Kıçım" means "my ass" in Turkish.

kristianp• 3 years ago

References "[Submitted on 31 Mar 2023] Programmed but Arbitrary Control Minimization of Amplitude and phase for speckle Nulling (PACMAN)" https://arxiv.org/abs/2303.18231

No idea if that's a joke paper.

0x008• 3 years ago

HyperSVMRanking (Such Vector, Much Ranking) algorithm proposed by Andrej Karpathy (2023) https://arxiv.org/abs/2303.18231

shreyansh26• 3 years ago

Haha, so all one has to do is to add some fancy terms like LLMs, vectors, hyper-fast and 300+ Github stars. What a a joke lol!

orasis• 3 years ago

It’s a prank.

jazzido• 3 years ago

`def derridaean_similarity`

Instant classic.

celestialcheese• 3 years ago

Damn this is good - well played.

For those who aren't neck deep in LLM hype - ChromaDB raised $18m at a $75m valuation a couple weeks ago with what's essentially just a thin wrapper around duckdb with a parquet import/export for persistence.

elforce002• 3 years ago

Hahaha. Really?! Good for them.

merpnderp• 3 years ago

I couldn't believe they were raising $35 million rounds with a brand new embedding database that was open source. Then I poked through the dozens of lines of code and realized they had implemented the same DB I had.

LOL, much funny.

alfor• 3 years ago

It’s a joke, but is it a functional one?

I found it crazy to run auto-GPT with an external DB over the internet, how can this be ever fast and efficient? (unless the dataset is humongous)

gavi2• 3 years ago

Yes, it used OpenAI embeddings and numpy to find cosine similarity

yawnxyz• 3 years ago

aside from making fun of Chroma, this seems to 100% work though? What would be the pros/cons of actually using this instead?

jdaggers2• 3 years ago

Author here. Yes, it actually does work. If I’m ever indexing fewer than ~100k documents/paragraphs I would use something like this approach rather than reaching for a vector database.

gavi2• 3 years ago

Great work!

ShamelessC2• 3 years ago

Damn - did this get flagged? Went from top-of-HN to missing from the front page _pretty_ quick.

jdaggers2• 3 years ago

Yeah, this was a little strange. I'm the author of the repo linked here.

I got alerted to it hitting #1 on HN and was able to see it there for only a little bit before it immediately got tanked to # 60. I'm a little bummed, but c'est la vie.

ShamelessC2• 3 years ago

Certainly suspicious - almost like VC's can't take a joke ;) I'm sure there's an innocent explanation though.

Great satire in any case! Strikes the perfect balance between highly plausible and glaringly obvious.

arthurcolle• 3 years ago

https://raw.githubusercontent.com/jdagdelen/hyperDB/main/_st...

Y-axis on this chart is actually hilarious

A hyper-fast local vector database for use with LLM Agents

Related Stories