You don't need Langchain, here's how to do RAG without it

https://vigneshwarar.substack.com/p/hackernews-support-page-using-retrieval

by vignesh_warar • 3 years ago

3 2 3 years ago

vigneshwarar.substack.com

irtefa• 3 years ago

You mentioned quantizing the vector index to reduce size. I am curious about how much compression were you able to achieve and how did that affect performance?

vignesh_warar• 3 years ago

I might be wrong here. I just know some product quantization techniques, but you can reduce the index by a lot! However, from my research, the more size you reduce, the more retrieval quality is also reduced.

Quoting from https://github.com/criteo/autofaiss

Using faiss efficient indices, binary search, and heuristics, Autofaiss makes it possible to automatically build in 3 hours a large (200 million vectors, 1TB) KNN index in a low amount of memory (15 GB) with latency in milliseconds (10ms).

I would highly recommend taking a look at AutoFaiss. You just have to set the maximum memory to build the index, and it will come up with the configuration.

You don't need Langchain, here's how to do RAG without it

Related Stories