redskyluan

Between Claude Code and my wife, only one enjoyed my late nights — a fun take on vibe-coding obsession and hard engineering lessons.

  Maintainer of Milvus here. A few thoughts from someone who lives this every day:

  1. The free user problem is real, and AI makes it worse. We serve a massive community of free Milvus users — and we're grateful for them, they make the project what it is. But we also feel the tension MinIO is describing. You invest serious engineering effort into stability and bug fixes, and most users will never become paying customers. In the AI era this ratio only gets harder — copy with AI becomes easier than ever

  2. We need better object storage options. As a heavy consumer of object storage, Milvus needs a reliable, performant, and truly open foundation. RustFS is a solid candidate — we've been evaluating it seriously. But we'd love to see more good options emerge. If the ecosystem can't meet our needs long-term, we may have to invest in building our own.

  3. Open source licensing deserves a serious conversation. The Apache 2.0 / Hadoop-era model served us well, but cracks are showing. Cloud vendors and AI companies consume enormous amounts of open-source infrastructure, and the incentives to contribute back are weaker than ever. I don't think the answer is closing the source — but I also don't think "hope enterprises pay for support" scales forever. We need the community to have an honest conversation about what sustainable open source looks like in the AI era. MinIO's move is a symptom worth paying attention to.

Using Hierarchical Clustering significantly reduces recall; this is a solution we used and abandoned three years ago.

dude you already missed the window.

nothing is better than sqlite as a library and don't use high perforamnce as your value for a python product

I really enjoyed this article. I have a lot of appreciation for PG, but some articles tend to exaggerate its capabilities, especially when it comes to PG vectors, which can be off-putting."

check https://github.com/zilliztech/VectorDBBench

By the way, if you’re not fully satisfied with S3Vector’s write, query, or recall performance, I’d encourage you to take a look at what we’ve built with Zilliz Cloud. It may not always be the lowest-cost option, but it will definitely meet your expectations when it comes to latency and recall.

Author of this article.

Yes, I’m the founder and maintainer of the Milvus project, and also a big fan of many AWS projects, including S3, Lambda, and Aurora. Personally, I don’t consider S3Vector to be among the best products in the S3 ecosystem, though I was impressed by its excellent latency control. It’s not particularly fast, nor is it feature-rich, but it seems to embody S3’s design philosophy: being “good enough” for certain scenarios.

In contrast, the products I’ve built usually push for extreme scalability and high performance. Beyond Milvus, I’ve also been deeply involved in the development of HBase and Oracle products. I hope more people will dive into the underlying implementation of S3Vector—this kind of discussion could greatly benefit both the search and storage communities and accelerate their growth.

Curious—why the shift from a Milvus-compatible API to a Chroma-compatible one? And of course, something in Python… because that’s obviously the fastest way to conquer the world.

Code Context is an MCP plugin that brings semantic code search to Claude Code, Gemini CLI, or any AI coding agent.

Full codebase indexing means richer context and better code generation.

100% open-source.

Same as cursor, We update every 5 minutes. You can pick a local storage or using cloud services for sure

Postgres users often hit scaling issues — whether it's with LISTEN/NOTIFY, PGVector, or even basic relational queries.

For startups, Postgres is a fantastic first choice. But plan ahead: as your workload grows, you’ll likely need to migrate or augment your stack.

Also let us know your ideas! How do we make code search easier

Most vector database benchmarks today focus on toy use cases: static data, pure ANN search, no filters, no writes.

That’s not how things work in production.

We built VectorDBBench to benchmark vector databases under real-world conditions. It’s open source and just hit v1.0.0.

What’s new in v1.0.0:

Label Filtering + vector search tests

Concurrent read/write under pressure

Customer dataset

Better webUI

It supports several popular vector DBs out of the box (Milvus, Weaviate, Qdrant, Chroma, etc.) and is easy to extend.

If you’re building anything with RAG, embeddings, or search infra, we’d love feedback.

Sharding is such a pain and no company should have to endure that.

What’s the largest deployment you’ve maintained in production?

If you’ve had the chance to manage one—especially while handling concurrent insert and search operations—you’ve probably felt the pain firsthand and understand just how critical scalability really is.

Insert into PGvector is super slow when data accumulate and the index build will take forever

sharding is a bad solution for any databases, especially vector database. See https://milvus.io/blog/why-manual-sharding-is-a-bad-idea-for...

Try ollama? It support several embedding models

Sharding can hinder your business growth, especially for all AI startups.

Prepare for scalability with a non-sharded architecture from the start.

How is pg performed on windows? Never tried before

Running everything on windows plaform is not easy... maybe just run with a container?

hah... we just try to name it after a bird.

"Woodpecker" is a good metaphorical name for a Write-Ahead Log (WAL) service for several reasons of Persistent and Reliable. Woodpeckers repeatedly and persistently peck trees, symbolizing the reliability and consistent logging of events or data.

Woodpecker is a modern, Kubernetes-native WAL storage implementation designed for stateless architectures, offering performance without disk overhead. If you've ever wondered what's behind Milvus’s robust log storage engine, now you can explore and deploy by yourself. (Note this is under heavy development yet)

I'm actually very disappointed about the performance of Mongo vector search after I test on it. Any vector database, is better than mongodb performance wise.

Amazing！

Search is not a problem . What to search is!

Using reasoning model, it is much easier to split task and focus on what to search

github https://github.com/zilliztech/deep-searcher

We have built a RAG calculator that, given a specific model and PDF format, can estimate the embedding and vector database costs. Additionally, it allows for visualizing the results of chunking. We’d love to hear your feedback on how we can simplify your RAG development process. Stay tuned as we continue to add more visualization tools!

Even SSD won't be fast enough for most indexes due to the random access nature. I've seen more than 1M iops on a huge nvme disk when use DiskANN index

why not try milvus? you get multiple index types, SIMD based brute force search, IVF, HNSW and DiskANN and you never bother to scale

That's true. Now we can offer the new dedicated Cost-optimized CU，pricing is

$0.318/hour ($228.96/month)

Which means you can store 20M 768 dimension data in 228$ per month