HN user
Passionate about tech, AI, and developer products.
If you'd like to get in touch, you can email my username at my username dot com.
Post-co author here. This is actually something that we are considering implementing in future versions of pgai Vectorizer. You point the vectorizer at database A but tell it to create and store embeddings in database B. You can always do joins across the two databases with postgres FDWs and it would solve issues of load management if those are concerns. Neat idea and one on our radar!
Post co-author here. Really appreciate the feedback.
Your point about HNSW being resource intensive is one we've heard. Our team actually built another extension called pgvectorscale [1] which helps scale vector search on Postgres with a new index type (StreamingDiskANN). It has BQ out the box and can also store vectors on disk vs only in memory.
Another practice I've seen work well is for teams use to use a read replica to service application queries and reduce load on the primary database.
To answer your third question, if you combine Pgai Vectorizer with pgvectorscale, the limitations around filtered search in pgvector HNSW are actually no longer present. Pgvectorscale implements streaming filtering, ensuring more accurate filtered search with Postgres. See [2] for details.
[1] https://github.com/timescale/pgvectorscale [2]: https://www.timescale.com/blog/how-we-made-postgresql-as-fas...
Post co-author here. The point is a little nuanced, so let me explain:
You are correct in saying that that you can store embeddings and source data together in many vectordbs. We actually point this out in the post. The main point is that they are not linked but merely stored alongside each other. If one changes, the other one does not automatically change, making the relationship between the two stale.
The idea behind Pgai Vectorizer is that it actually links embeddings with underlying source data so that changes in source data are automatically reflected in embeddings. This is a better abstraction and it removes the burden of the engineer to ensure embeddings are in sync as their data changes.
Good q. For most standalone vector search use cases, FAISS or a library like it is good.
However, FAISS is not a database. It can store metadata alongside vectors, but it doesn't have things you'd want in your app db like ACID compliance, non-vector indexing, and proper backup/recovery mechanisms. You're basically giving up all the DBMS capabilities.
For new RAG and search apps, many teams prefer just using a single app db with vector search capabilities included (Postgres, Mongo, MySQL etc) vs managing an app db and a separate vector db.
Hey HN! Post co-author here, excited to share our new open-source PostgreSQL tool that re-imagines vector embeddings as database indexes. It's not literally an index but it functions like one to update embeddings as source data gets added, deleted or changed.
Right now the system only supports OpenAI as an embedding provider, but we plan to extend with local and OSS model support soon.
Eager to hear your feedback and reactions. If you'd like to leave an issue or better yet a PR, you can do so here [1]
While I value the depth in which the blog authors went into, some of these are not problems with pgvector but problems with similarity search itself and would be problems with other vector databases as well. There are also solutions to the problems describe that the authors do not mention that I think any reader should take into account.
1. Required and Negated Words
This is not a bug of pgvector but a bug in embedding models and simple similarity search itself. You'd run into this issue on when doing RAG with any vectordb or ANN search library. You could probably solve this with query expansion, running multiple similarity searches in parallel, and doing filtering of results containing the required theme vs trying to get this all in a single search.
2. Explainability With Highlights
This is a misunderstanding of the purpose of embeddings based semantic search. The point of semantic search is not to match on exact keywords but to match on the /meaning/. If you want exact keyword matching, use full text search. This is something that can also be solved by hybrid search, combining full text search and semantic search and using a re-ranker. PostgreSQL has built in FTS with tsvector.
3. Performant Filters and Order By’s
The authors do not disclose any details about what index they use here or what kind of filtering they are trying to do. As other commenters point out, the StreamingDiskANN in the pgvectorscale extension [0] (complement to pgvector, you can use them together) improves on performance and accuracy of filtering vs pgvector HNSW (see details in [1]).
4. Support for sparse vectors, BM25, and other inverse document frequency search modes
This is probably the most fair point in the post. But there exists projects like pg_search from ParadeDB which bring bm_25 to Postgres and help solve this [2]
Lastly, I respect companies trying to provide real-world examples of the trade-offs of different systems, and so thank the authors for sharing their experience and spurring discussion.
Disclaimer: I work at Timescale, where we offer pgvector, and also made other extensions for AI/ vector workloads on PostgreSQL, namely pgvectorscale, and pgai. I've tried to be as even in my analysis as possible but as with everything on the internet, you can make up your own mind and decide for yourself.
[0] https://github.com/timescale/pgvectorscale/ [1]: https://www.timescale.com/blog/how-we-made-postgresql-as-fas... [2] https://github.com/paradedb/paradedb
There's actually quite a few ways to do shared database multi-tenant setups with pgvector. You can have separate out tenants into separate tables, schema, and even logical databases within the same database service itself.
Here's an overview of the methods [1] (all except separate database of course)
See discussion for pgvectorscale in this HN post: https://news.ycombinator.com/item?id=40646276
Blog co-author here (PM at Timescale).
We're excited to release pgvectorscale. Our team built this extension to make PostgreSQL a better database for AI and to challenge the notion that PostgreSQL and pgvector are not performant for vector workloads.
pgvectorscale is open-source under the PostgreSQL license and free to use on any PostgreSQL database.
Here are two helpful companion reads to the post linked by OP: A benchmark of how PostgreSQL with pgvector and pgvectorscale performs against Pinecone [1], and a technical deep dive into pgvectorscale's StreamingDiskANN index and Statistical Binary Quantization implementations [2].
Questions and feedback welcome!
[1] https://www.timescale.com/blog/pgvector-vs-pinecone/ [2]: https://www.timescale.com/blog/how-we-made-postgresql-as-fas...
Timescale recently released Timescale Vector [0] a scalable search index (DiskANN) and efficient time-based vector search, in addition to all capabilities of pgvector and vanilla PostgreSQL. We plan to add the document processing and embedding creation capabilities you discuss into our Python client library [1] next, but Timescale Vector integrates with LangChain and LlamaIndex today [2], which both have document chunking and embedding creation capabilities. (I work on Timescale Vector)
[0] https://www.timescale.com/blog/how-we-made-postgresql-the-be... [1]: https://github.com/timescale/python-vector [2]: https://www.timescale.com/ai/#resources
Agreed on pgvector being simple and a great choice for POCs and low scale, especially if you're familiar with Postgres. Our team released something new last week built for folks looking to use PostgreSQL at scale as a vector store [0], featuring a DiskANN index type.
[0] https://www.timescale.com/blog/how-we-made-postgresql-the-be...
(Blog author here). Thanks for the question. In this case the index for both DiskANN and pgvector HNSW is small enough to fit in memory on the machine (8GB RAM), so there's no need to touch the SSD. We plan to test on a config where the index size is larger than memory (we couldn't this time due to limitations in ANN benchmarks [0], the tool we use).
To your question about RAM usage, we provide a graph of index size. When enabling PQ, our new index is 10x smaller than pgvector HNSW. We don't have numbers for HNSWPQ in FAISS yet.
Blog author here. This index is built in PostgreSQL. We make it very clear that this is cloud-only in the post and never mention OSS. While we offer it via the Timescale cloud platform, everything under the hood is still PostgreSQL and works with the entire postgresql ecosystem.
Yup 2GB is fine (especially if using quantization), depending on the dataset size of course.
Totally fair about cloud-only. Many developers prefer developing on cloud, but some prefer local dev. YMMV.
By the way timescale vector offers pgvector as well so it's easy to test and compare. (Note: I work for Timescale)
Timescaler here. Linking a few posts below [0][1] that answer the majority of your very good questions. The posts detail why and how TimescaleDB started and why the founders chose to build a time-series database on PostgreSQL.
[0] https://www.timescale.com/blog/when-boring-is-awesome-buildi... [1]: https://www.timescale.com/blog/40-million-to-help-developers...
Post author here. We made the NFT Starter Kit to help fellow developers make sense of the NFT space using data. We're looking to improve the project in the coming months, so I'd be curious to hear your take and suggestions for improvement!
Timescale and Citus are are open core, holding back features for customers.
One clarification. While TimescaleDB is open-core, our community version is source-available and 100% free to use. We do not "hold back features for customers". You do not need to pay to use any of TimescaleDB's best features, it's all free via the Timescale Community license.
You only pay if you'd like to use our hosted offerings (and save the hassle of self-managing your DB): Timescale Cloud or Managed Service for TimescaleDB.
For more see: https://www.timescale.com/products
(Disclaimer: I work at Timescale)
Timescaler here. I think you're referring to this comparison of InfluxDB vs TimescaleDB [0]?
There's also comparisons of TimescaleDB vs MongoDB[1] and AWS Timestream [2].
[0] https://blog.timescale.com/blog/timescaledb-vs-influxdb-for-...
[1] https://blog.timescale.com/blog/how-to-store-time-series-dat...
[2] https://blog.timescale.com/blog/timescaledb-vs-amazon-timest...
Timescale team member here. We take our responsibility to build a rock-solid platform very seriously. We have multiple "levels" of product within Timescale. At our core, we have the open-source database, TimescaleDB. This product releases on a more deliberate and careful cadence, always making sure that we are optimizing for reliability, security, and performance. This has been our approach since our initial launch [0], where we embraced the mantra "boring is awesome", recognizing that for our users stability and reliability is of paramount importance.
Within the core database, we offer features that are carefully marked as "experimental", which we discuss at length in this blog post [1].
Beyond TimescaleDB, we also offer other products that are more SaaS-y in nature. While they're all based on the rock-solid foundation of TimescaleDB, we are also able to ship new features more quickly because they are UI components that make using the database even easier.
Finally, some of our "launches" are more textual in nature, such as this benchmark, which we have spent months researching and compiling.
[0] https://blog.timescale.com/blog/when-boring-is-awesome-build...
[1] https://blog.timescale.com/blog/move-fast-but-dont-break-thi...
The point of the blog post is that Timescale cloud isn't totally "serverless", because for some 20% of use cases, you do want to think about nodes and other database internals.
From the blog post: "So today’s serverless data platforms are not familiar or flexible. But further, black boxes are never truly easy and worry free: you never know if there are any skeletons lurking in the proverbial closet, just waiting to cause your service to fall over."
(Timescale employee here)
Generally speaking, if it works with Postgres, it works with Timescale. Hasura even has documentation on how to use TimescaleDB with Hasura [1].
That said, I'd be curious to hear about other folks experiences.
Disclaimer: I work at Timescale.
[1] https://hasura.io/blog/using-timescaledb-with-hasura-graphql...
Timescale employee here. While we do offer plans starting at $24/month [1] on Timescale Cloud, it's worth noting that DO Postgres (and other hosted Postgres solutions) are providing an undifferentiated open-source solution, while on Timescale Cloud you get Postgres underneath (for relational workloads) and TimescaleDB (for time-series workloads), plus other features which provide an enhanced time-series experience (like the explorer dashboard [2], autoscaling, automated upgrades, auto-tuning of configs, just to name a few). So do keep that in mind when comparing prices.
Many users actually find Timescale Cloud to be cheaper because it replaces both a time-series and relational db with a single database that can store (and analyze) both kinds of data.
[1] https://www.timescale.com/cloud [2] https://blog.timescale.com/blog/announcing-explorer-a-better...
Thanks for the suggestion! I haven't read that one yet.
I read a bunch of books by Mr De Bono as a teenager, including "Six Thinking Hats" and "Lateral Thinking". As math and science nerd interested in creativity, it really gave me a different way to think.
Cool dataset! I wish there was a similar dataset for Premier League football (or even international soccer). Does anyone know of a good resource?
Wish we had classes like this back when I was doing my CS degree. Not saying that algos/ theoretical / systems stuff isn't valuable, but if you want to build anything that reaches real customers, then knowing how to build for the Apple Ecosystem is essential!
I'm curious about advice folks would give for remote YC final interviews vs in person ones?
Timescale employee here. We generally update our benchmark blogs every year to keep them fresh. No doubt you'll see the updated results here on HN as well!
Apologies, I did so elsewhere in the thread. I work at Timescale!