HN user

mjasay

378 karma
Posts0
Comments7
View on HN
No posts found.

I know I'm biased on this, but it has always seemed obvious that vector search would be subsumed into other databases. At MongoDB we've made it easy to manage operational and vector data in the same place, simplifying data architecture. While I think we do this better than others, it's also true that other vendors and communities (like Postgres with pgvector) have added vector capabilities and, frankly, always were going to do so. It's just a natural extension. I don't want to be dismissive of purpose-built vector databases, but they're going to have to evolve to suit more general-purpose workloads. This could happen as Neo4j has done, e.g., making graphs a more general way of thinking about data. It will be interesting to see how it plays out.

Correct. The focus is on 100% correctness. As I wrote in the post: "Over its 35 years in existence, SQL Server has evolved to meet a wide array of use cases. When first made available on GitHub, Babelfish won’t be able to handle every use case, but will be able to tackle the most common application scenarios. Most importantly, Babelfish will meet the correctness objective. That is, if Babelfish doesn’t yet support specific SQL Server functionality, it will return an error to the application, rather than defaulting to PostgreSQL behavior. Why? Because, again, developers (and the enterprises for which they work) must be able to depend absolutely on the correctness of SQL Server compatibility."

At launch Babelfish will be able to handle - with 100% correctness - the main semantics you'd want. HOWEVER, as you said (and as mentioned in the post), there's a large surface area, and a "long tail" of functionality that needs expertise from us and others to cover. So to do this right, it needs a community. It will be great for many use cases right from the start, but to ensure it's great/perfect for your use case...well, that just might take your help. Please work with us on this.

Totally fair to critique me as a marketer (though that's not really what I do. The only time I've had a true marketing title was when I ran marketing at MongoDB), but I don't think some of the other criticism sticks. We've been contributing to Apache Lucene and Solr (which feed into Elasticsearch), as well as Elasticsearch, for a long time. See, eg, https://aws.amazon.com/blogs/opensource/amazon-giving-back-a....

As for Headless Recorder, we're working with Tim now. That was a miss on our part, one that I (and the team involved) regret. But I don't regret you and others calling us out when you feel we've done wrong. AWS is not a perfect company, but I've been gratified that people here (in my experience) want to do the right thing. Sometimes we need help figuring out what's right in a given situation (or, rather, what's the right way to help customers. Increasingly you'll see teams understanding that more upstream contributions might be the best way to help customers in the medium- and long-term, even when it's not necessarily the obvious way to help them in the short-term). So please keep helping us do better.

To be very clear, it's not Amazon's language to take. We are contributing (and I think you'll see us increase our contributions), but we are one of many contributors.

And, I should add, the real contributors are the engineers who do the code. AWS is committing credits/cash/other, but when it comes to the engineering, we are hiring committers so that they have time/resources to continue to do so. But it's those engineers (and engineers from other companies) who are doing the work.

I'm very, very grateful that Facebook and others have also been hiring Rust committers. It's better as a community, not as one company's programming language, even a great company/org like Mozilla.

So, no, don't worry about AWS taking over Rust. We couldn't and don't want to. Rust has done so, so much right as a community-led project: welcoming, inclusive, kind. We hope to contribute to that, not commandeer anything.

I run the open source strategy and marketing team at AWS. As I told Tim privately and publicly (https://twitter.com/mjasay/status/1317084448119169024), I hadn't been aware of this but am talking with the relevant product team to see how we can improve in his regard.

AWS uses a lot of open source, and we contribute a lot, both in terms of code (first-party projects like Firecracker and Bottlerocket, but also third-party projects like Redis, GraphQL, Open Telemetry, etc.), testing, credits, foundation support, and more. But open source is ultimately about people and communities, and I personally feel we could have done more to acknowledge the great work Tim and his co-maintainers have done, and try to support their Headless Recorder work. We're talking with Tim now about this.

(While I think we do far better than sometimes acknowledged, we're also always looking to improve, and appreciate all the feedback that helps us toward that goal.)