HammadB

You will always find a copy of McLuhan on the Chroma teams bookshelf !

Yes! It can - https://docs.trychroma.com/docs/querying-collections/metadat...

That is helpful feedback, thank you. We'll address this.

Hi! Hammad here - Chroma’s CTO. First off, I have an immense amount of respect for the Turbopuffer team, they’ve build a solid product.

I understand your point. Chroma Cloud has been quietly live in production for a year, and we have been discussing this architecture publicly for almost two years now. You can see this talk I gave at the CMU databases group - https://youtu.be/E4ot5d79jdA?si=i64ouoyFMevEgm3U. Some details have changed since then. But the core ideas remain the same.

The business model similarities mostly fall out of our architecture being similar, which mostly falls out of our constraints with respect to the workload being the same. There are only so many ways you can deliver a usage based billing model that is fair, understandable, and predictable. We aimed for a billing model that was all three, and this is what we arrived at.

On aesthetics, that’s always been our aesthetic, I think a lot of developer tools are leaning into the nostalgia of the early PC boom during this AI boom (fun fact, all the icons on our homepage are done by hand!).

On differences, we support optimized regexes vs full-scans, lending better performance. We also support trigram based full-text search which can often be useful for scenarios which need substring matches. We also support forking, which allows for cheap copy-on-write clones of your data, great for dataset versioning and tracking git repos with minimal cost. We've been building with support for generic sparse vectors (in beta) which enables techniques like SPLADE to be used, rather than just BM25. You can also run Chroma locally, enabling low-latency local workflows. This is great for AI apps where you need to iterate on a dataset until it passes evals, and then push it up to the cloud.

Chroma is Apache 2.0 open source - https://github.com/chroma-core/chroma and has a massive developer community behind it. Customers can run embedded, single-node and distributed Chroma themselves. We've suffered from depending on closed-source database startups and wanted to give developers using Chroma confidence in the longevity of their choice.

Lastly, we are building with AI workloads front and center and this changes what you build, how you build it and who you build for in the long term. We think search is changing and that the primary consumer of the search API for AI applications is shifting from human engineers, to language models. We are building some exciting things in this direction, more on that soon.

There is an abundance of discussion on this thread about whether models are intelligent or not.

This binary is an utter waste of time.

Instead focus on the gradient of intelligence - the set of cognitive skills any given system has and to what degree it has them.

This engineering approach is more likely to lead to practical utility and progress.

The view of intelligence as binary is incredibly corrosive to this field.

I've taken the linked Russ Tedrake class and have to say I loved this. Please make more!

It’s COOL

If segmented hsnw indices were O(N log N) - it would make no sense to build the index at all - brute force would be better as O(N log N) > O(N)

Sure, I see. I think this is an area where complexity analysis doesn’t lead to useful information.

To be more correct it’s O(N/C log C) where C is the capacity of a segment. In this case you can ignore 1/C and log C as constant. So now sure, you actually just have O(N). But this is not super useful as it says that a segmented hnsw approach and brute force approach are the same - when this is really not the case in practice.

Also O(N log N) > O(N) so I’m not sure why we would ever do anything with segmentation according to that analysis if it were correct.

a lot of game engines have higher level shading languages (often visually programmed) that are long-lived and widely used.

"The obstacle is that until now, off-the-shelf vector databases could not index a dataset larger than memory, because both the full-resolution vectors and the index (edge list) needed to be kept in memory during index construction. Larger datasets could be split into segments, but this means that at query time they need to search each segment separately, then combine the results, turning an O(log N) search per segment into O(N) overall."

How is a log N search over S segments O(N)?

+1 I was ready to drop $$$

Curious - what was the specific leaderless replication strategy?

I don't think that - I am speaking generally.

I don't think its the role of the government to penalize layoffs. If a company needs to do a layoff to remain viable it must be allowed to - the downside (everyone loses their job) is worse. And trying to apply some mechanism where the government decides if its ok or not seems too intractable.

I think the approach of things like the WARN (https://en.wikipedia.org/wiki/Worker_Adjustment_and_Retraini...) act are more plausible. Use legislation to make the workforce more informed, resilient and ahead-of layoffs. You don't want mass layoffs crippling your economy because people are unable to plan.

It's a direct consequence of the WARN act that most tech layoffs have extremely generous severance.

There is a fundamental disconnect between industry and academia here.

Yeah, was thinking the same thing. Something deeper to be explored here esp due to the consistency across domains. I wonder how far we can push "externalizing" higher level information the model wants to store while in the forward pass.

https://github.com/bijection/g9/blob/master/src/minimize.js

Maybe I am misunderstanding but the postgres buffer cache is LRU and will evict pages right? At times data may go to disk and then during serving it will have to be loaded into memory? So it is quite dependent on the size of your buffer cache as well as contention for that buffer cache.

Also the cache access patterns will vary between these implementations and is worth considering.

No one is discrediting (or even discussing really) the validity of the point that if you use postgres pgvector or pgembedding may makes sense for the reasons you mention. The question was about how the algorithms compare, I compared the algorithms. Comparing ANN algorithms is relatively apples-to-apples, but is still complicated and filled with nuance.

Sure, I don't disagree that there is more to vector database selection than raw performance. Any technical decision is filled with many considerations.

The commenter - thewataccount - asked about performance and I shared my intuition.

DiskANN construction is much much slower than HNSW, and the support around updates has been discussed in literature but has not been open sourced yet (They call this FreshDiskANN in the paper - DiskANN is from microsoft research btw).

HNSW-IF is an excellent extension to HNSW (that the vespa team has made easy to implement) that takes advantage of the speed/recall of HNSW in combination with the disk scalability of inverted indices - it is a hybrid approach.

pg_embedding is based on a different approximate nearest neighbors algorithm - HNSW, which is generally considered - and studied - to be faster as well as more accurate than pgvectors IVF (ignoring a lot of nuance).

However pg_embedding serves the index out of disk, whereas most vector databases opt to serve the index out of memory, or delegate to mmap'ed files. HNSW is a graph-based algorithm, thus access patterns are random and this does not lend itself to disk based access. I'd expect pg_embedding to be slower than memory-resident indices due to this fact. Also in general with a postgres index, my concern would be scalability and resource isolation. It's convenient to colocate these things but ANN indices have very different CPU/Memory/Disk usage patterns than what you may need for just your relational data.

For example

Chroma -> Serves HNSW out of memory, persists to disk with a WAL.

Weviate -> Serves HNSW out of memory, writes HNSW graph search to WAL and uses that for durability.

Milvus -> Serves HNSW out of memory, supports partial mmap, also supports another algorithm called DiskANN which is optimized for SSD. It uses cloud storage with a shared-everything architecture for durability (a lot of nuance here.)

QDrant -> Has a WAL, supports memory and mmap'ed indices.

I have become convinced that the only mass-marketable form of spatial computing is a neural implant or contact lens.

Announcements!

Nope, this is why you cannot use images as prompts without some workarounds! SD doesn’t use the shared CLIP space but the text encoded before projection

I think what’s worth mentioning is that for people who grew up with google, schools already were having a bit of a panic with resources like wolfram alpha and Wikipedia. For this cohort of users, myself included, the idea of fact checking ”found” material and weaving it into a proper answer is very familiar.

With this view, I think chatGPT is less of a novel UX shift than an accelerant on an existing one.

I think it proves the age old belief that as technology reaches maturity it loses its fancy monikers and we will probably just look at this as “very good, very specific, google” one day

Seems like this thread has highlighted at least two type of relationships people have with their job. Those who’ve accepted the seemingly Faustian bargain of working at something they accept is meaningless. These people treat work as just that and don’t attempt to derive much value from it other than the sustenance it provides. In the other camp are those who have found work they find deeply meaningful, often at the expense of their own financial interest. For those in the latter camp, can you share what you are working on? Curious to learn.

Projects like this are so inspiring.

You also cannot search for the short-squeezed stocks from this whole WSB push. They unlisted GME, AMC etc on Robinhood. Searching for them returns nothing. That feels really wrong.