FerretDB: open-source MongoDB alternative

https://blog.ferretdb.io/ferretdb-1-0-ga-opensource-mongodb-alternative/

by yawnxyz • 3 years ago

254 127 3 years ago

blog.ferretdb.io

singeezie• 3 years ago

if ferretDB truly does provides ad-hoc queries, indexing, aggregation, and real-time data processing, making it well-suited for a wide range of applications, then sure , it’s a great alternative . I can see all types of web apps even enterprise systems working well w it.

peterfarkas11• 3 years ago

FerretDB maintainer here.

We already offer basic support for aggregation and indexing, and we are adding as many features as we can, see our roadmap [1].

We are mainly building on our user's experience with running FerretDB and add features to our roadmap accordingly. We are not aiming to implement the entire feature set of MongoDB, of course, but the majority of MongoDB workloads are not utilizing the full feature set, either.

[1] https://github.com/orgs/FerretDB/projects/2/views/1

jzelinskie• 3 years ago

Congrats on the launch! Your GitHub Project board looks nice and clean. Do you have any posts or code for how your bot is managing issues there?

quasilyte• 3 years ago

Congrats with v1.0! I remember the times when it was called MangoDB. :D

aleksi11• 3 years ago

Yeah, those were easier times for sure :)

manishsharan• 3 years ago

This looks great. I had been using Document Layer for FoundationDB but Apple has abandoned the Document Layer for the FoundationDB ; so this looks promising .

richieartoul• 3 years ago

Tigris(https://github.com/tigrisdata/tigris) is one of the supported FerretDB backends and Tigris is backed by FoundationDB, so you can still have the Mongo interface with the reliability and scaling of FDB if that's what you're looking for

deepsun5• 3 years ago

At one company we had one-off requests to run get some statistics from our mongodb.

Writing the proper queries proven pretty tricky whenever GROUP BY/JOINs were involved, so I used online converters between SQL -> mongo query.

But then I realized that PostgreSQL has nice JSONB type (supports indices for subfields). So I put all the mongo data into tables with a single JSONB column (or two columns id+data, if you prefer).

Turned out that was much faster to run analytical queries :)

"document" is just row

"collection" is just table

skatanski2• 3 years ago

If there were JOINs it may be a sign MongoDB was not the best DB for you. It means the model was relational and a relational DB would be a better fit. MongoDB lookups are discouraged in general, and especially in analytical workloads, which cover a lot of data. MongoDB is better for the scenario, where all you need is already in the document.

mgfist2• 3 years ago

It may also mean poor schema design. You can absolutely model relational data in MongoDB without relying on joins. If you want to normalize your data don't pick Mongo.

forgetfulness• 3 years ago

Should it be used as a database in the way that Postgres would be used as a database? It seems pretty unfit for that role, with its past durability issues and that writes have to be tailored to how the data will be consumed, due to lack of ad-hoc queries.

As something layered on top of the actual source of truth it may be reasonable, writing a materialized version of a costly query that's read very often, sure, but that's an optimization and it'd be competing with Redis for what matters there.

deepsun5• 3 years ago

No, our app didn't use JOINs for the main work. But, as I wrote, every now and then we had some one-off requests that required either JOIN or GROUP BY or both.

With postgres+JSONB we could do both at the same time on live data.

packetlost2• 3 years ago

The only problem I have with PostgreSQL's JSONB type is it strictly implements the JSON standard, so things like nulls and integers aren't supported. Depending on your data, this could be a real problem.

aleksi11• 3 years ago

FWIW, JSONB supports JSON nulls, and all numbers are encoded as `numeric` type, storing integers and float64 values precisely (well, as much as possible for float64 values).

But you comment is correct for other data types like date-times and binary strings where some form of encoding/decoding is needed.

udp• 3 years ago

The json standard has nulls

phendrenad2• 3 years ago

MongoDB really only makes sense when you have an extremely large set of documents and you don't need to do this kind of statistics.

deepsun5• 3 years ago

Yep, although for large datasets I'd prefer multi-master architecture, depending what is needed, for example Blob storages or Cassandra for key-value, and BigQuery or Clickhouse for analytics.

We also thought we wouldn't need it, but turned out sometimes we wanted to run some stats.

softfalcon• 3 years ago

I would recommend against modeling too many relations in a document DB. It can work, but it gets bogged down very quickly for the reasons you've stated.

My off-the-cuff suggestion is that document databases are not built to model normalized relational data. If you try to do so, you bring a whole new level of pain upon yourself. It is do-able, but it is hard and annoying.

I know this from extensive personal experience dealing with a large, highly relational Mongo database.

I am very glad you found a solution within Postgres that works for you, and your mapping of document to row and collection to table is very apt!

If possible, would you care to tell me what the size of said documents (rows) and collections (tables) are in your solution? I am curious if in another life, we might have built our tech stack on Postgres instead of MongoDB and been much happier for doing so.

tasubotadas2• 3 years ago

Joins only make sense in analytical context as a tool to get some additional data into your report and almost never as domain modeling concept. People should pretty much hardly ever use RDBMS for their domain data... but since everyone learns RDBMS as their first DB, we have these horrible ORM frameworks on every corner.

deepsun5• 3 years ago

We had everything de-normalized, as you pointed out.

However, every now and then we had some one-off questions that required using Mongo for something it's not designed for. However, migrating to postgres+JSONB made it easy to do both.

dfragnito• 3 years ago

This is one of the reasons we created https://schemafreesql.com. We wanted SQL with some of the document store (NOSQL) features.

nextaccountic2• 3 years ago

Since FerretDB is built on top of Postgres, could it be an extension? So that one can mix some FerretDB tables and some general SQL tables in the same database (with proper foreign key constraints between them etc)

peterfarkas11• 3 years ago

Yes, building an extension is something we are looking at, and I am glad to hear that you would be interested in this kind of mixed use case. Would definitely make things interesting, and more performant.

that_guy_iain11• 3 years ago

I’m confused, to me it seems that MongoDB is also open-source. Their new license is basically just a better agpl. I only point this out because the blog post makes a point of saying it’s not open source.

peterfarkas11• 3 years ago

By definition, SSPL is not an open-source license [1].

One could argue that the SSPL brings justice to open-source, as cloud providers will not be able to monetize a project without giving anything back. SSPL forces them to pay a license fee.

On the other hand, this also creates a vendor lock-in situation for the user, simply because not all service providers will be able to negotiate a deal with the developer of the SSPL-licensed product. This limits choice, limits competition between providers, among other things.

This license fee may also be increased at the sole discretion of the developer, and the increased fee will be paid by the user in the end. This all doesn't sound open-source to me at all.

[1] https://blog.opensource.org/the-sspl-is-not-an-open-source-l...

that_guy_iain11• 3 years ago

Personally, that blog post is just arrogance from OSI. I read it. It didn’t make clear which freedoms were being removed. They are still able to run cloud hosting just release their custom code?

And OSI thinking they alone get to decide what is and what is not open source is arrogant.

vrtx09• 3 years ago

MongoDB’s source code is still freely available. It’s still actively developed in the open on GitHub. Unless you’re offering MongoDB as a service, it’s just as “open source” as ever.

If you want to offer MongoDB as a service, you can still do so free of charge, as long as the service infrastructure is also made openly available, right? And if you don’t want to make the source available, you can purchase a license and do so, right?

Furthermore, if you’re using MongoDB, be it self-hosted, with a vendor, or even some proprietary database that implements the wire protocol for compatibility, you’re likely using MongoDB-developed clients/drivers, which are Apache 2.0 (“OSI-approved open source”).

So it seems like the only way to be locked into a vendor is if you’re using MongoDB drivers to connect with a 3rd party database that doesn’t fully implement all functionality of MongoDB in a compatible way… Right?

I could be wrong, but as someone who contributed to MongoDB as an open source project, and was later hired by MongoDB based on said contributions, it kinda hurts to see the OSI’s “MongoDB isn’t open source anymore” campaign work so well.

That said, I sincerely wish the team behind this project all the best!

My complaints aren’t against anyone in the open source communities I’ve known and loved. Just this self-important legal organization that acts like it controls (and even gets to define) open source software.

P.S. I left MongoDB in 2015 due to a neurological disability, but it was one of the highlights of my career, with so many kind and brilliant people. But it’s also been a while, and my brain doesn’t work so well these days, so please correct me if I got anything wrong!

jchannon• 3 years ago

If it can run a replica set on an apple m1 which mongo can’t currently do then great!

vrtx09• 3 years ago

Er, why can’t you run a MongoDB replica set on your M1?

I mean, I wouldn’t recommend running a ReplicaSet on the same host on any production host (it defeats the purpose), but for testing, I’ve run a sharded cluster w/ 3 replicas per shard…

Happy to try and help!

hartator2• 3 years ago

Can you send me an email? Julien at/ serpapi.com

dang• 3 years ago

FerretDB 1.0 GA: open-source MongoDB alternative built on Postgres - https://news.ycombinator.com/item?id=35524235 - April 2023 (2 comments)

FerretDB: A truly open-source MongoDB alternative - https://news.ycombinator.com/item?id=29448906 - Dec 2021 (110 comments)

MangoDB has a new name - https://news.ycombinator.com/item?id=29407987 - Dec 2021 (5 comments)

Open source MongoDB drop-in replacement, built on top of Postgres - https://news.ycombinator.com/item?id=29096331 - Nov 2021 (2 comments)

MangoDB: An open-source MongoDB alternative - https://news.ycombinator.com/item?id=29071623 - Nov 2021 (200 comments)

Show HN: MangoDB - https://news.ycombinator.com/item?id=4139723 - June 2012 (2 comments)

yawnxyz4• 3 years ago

Kind of cool that MangoDB started out as a joke and ended up as a real, solid project

tristan9572• 3 years ago

One thing that I didn't see discussed was why FerretDB uses Postgres as a backend and not a storage engine. Perhaps I am just missing something though.

PeterZaitsev5• 3 years ago

Because PostgreSQL Is fantastic Open Source database engine, very popular with a lot of operational experience and tooling support.

This is Open Source way of doing things - having project focus on as narrow problem as possible (first) and leverage as much of existing componets as possible.

tristan9572• 3 years ago

Ok, that makes sense. I guess this project is targeting a compatible frontend for multiple backends. When I saw MongoDB alternative, I thought they were building both sides of the coin.

rmangi• 3 years ago

Cramming an api for one product on top of another is absolutely not “the open source way” I’d expect a much better explanation for this Frankenstein’s monster. This seems like it would work but never scale.

josevalerio• 3 years ago

Another day on the orange site where people don't understand that you can 100% have relationships in a NoSQL database - just don't try to do it the same way you do in a "relational" one or you're going to have a bad time. Storing everything in one collection is critical.

Here is an example using multi key indexes with Rick Houlihan from re:Invent 2022: https://youtu.be/eEENrNKxCdw?t=1131

kbd• 3 years ago

Ok, old man yelling at clouds moment finally coming for me. Now that we've been through the document-database heyday and are out the other end, what have we learned about where document databases are a good fit?

At the time I looked at them like a fad. "These script kiddies want to write javascript and ignore schemas. Let's see how well that works out for them." As expected, most of what I ever hear is regret.

Today, MongoDB has grown out of the reliability issues it had in the past, and Postgres has json features for the occasional times it's useful to store some loosely structured data along with otherwise relational data. Question is, what applications is a document-first database good for, outside of prototyping?

Edit: and to make sure I understand, FerretDB is a layer reimplementing MongoDB on top of Postgres and its json features?

peterfarkas11• 3 years ago

Yes, FerretDB is a layer which implements the MongoDB wire protocol on top of Postgres. Right now we are using JSONB, but this affects performance and we need to depart from this strategy in the long run. We have an article which explains the concept [1].

I wouldn't go into the document vs. relational argument, all arguments for and against would have merit. There are valid use cases for document databases (take e-commerce, for example), and we should not discount the fact that using a relational database is just more complicated. Using vanilla Postgres for a MongoDB use case will not be feasible for someone who's focus is, let's say, mobile application development. There is a reason behind MongoDB's popularity - it just provides a great developer experience. This is what we are aiming to recreate on top of Postgres.

[1] https://blog.ferretdb.io/pjson-how-to-store-bson-in-jsonb/

Zababa2• 3 years ago

I'm curious about what makes document databases a good fit for e-commerce. I've never worked in that industry.

yawnxyz4• 3 years ago

if we use Ferret for prototyping and eventually land on a schema, would it be possible to convert the FerretDB rows into more structured postgres? Would be so cool if it could just analyze and create a schema for you, you double check it, and it just works.

I think being able to convert back and forth would make it so worthwhile!

CodesInChaos2• 3 years ago

we need to depart from this strategy in the long run

What's the alternative you want to move to?

mitjam2• 3 years ago

A use case I have for FerretDB is migrating existing apps off MongoDB without needing to change the code. I find it funny that you could call these now „legacy“ apps.

peterfarkas11• 3 years ago

Exactly. Let's say a major company have a few applications which require MongoDB, and the rest of their applications are running on Postgres.

With FerretDB, they can migrate the app off MongoDB as you said, and keep Postgres only. Therefore they don't need to maintain internal knowledge on how to run MongoDB, or pay for MongoDB to run it for them (in which case they are not in control of their data, because it is all under MongoDB's account...).

This is one real world user example.

kiney• 3 years ago

Insert <look at me, I am the legacy app now> meme here

ocdtrekkie3• 3 years ago

Yeah, anxiously waiting for FerretDB to support Meteor over here.

tracker12• 3 years ago

Looks that way to me... I really liked a LOT about MongoDB, I don't think they had a good story for administration of scale + redundancy. RethinkDB, Cassandra and others have a ring + redundancy model which I think is easier to deal with. Where Mongo at least was limited to replication or sharding, and was a serious pain to deal with from the admin side imo.

Understanding the advantages and shortfalls is sometimes harder too. Mongo having multiple secondary indexes is pretty nice as well. Haven't dug into FerretDB, my first question is if it will run over the top of CockroachDB, as that's how I would probably want it configured. Then, I'm not sure if I wouldn't just use the JSONB surface in PostgreSQL/CockroachDB directly.

I think it just really depends on how/what you need to accomplish.

aranchelk• 3 years ago

Question is, what applications is a document-first database good for, outside of prototyping?

Some criteria I’d use: * Data is already naturally segregated and won’t be shared/joined much during normal usage * App already has transactions modeled in the application layer (or I guess if consistency really doesn’t matter) * App would benefit from being geographically distributed. * App is written in a language with a strong type system, has high code quality, test coverage, etc.

In my case, my company makes a distributed project management application. All changes to projects are canonically ordered and applied in the application layer.

Data is stored in Cloudflare Durable Objects and R2. Hard to classify DOs but they’re a lot closer to a document store than an SQL Db.

There are some nice benefits that would be hard to replicate with a traditional SQL Db setup.

brightball2• 3 years ago

I’m still waiting for the NoSQL crowd to realize that the best use case was always handled by CouchDB.

yawnxyz4• 3 years ago

I started picking up CouchDB and PouchDB and the DX is fantastic. Always wondered why there’s never been any hype or more examples around this stack?

brightball2• 3 years ago

Mongo got popular at the same time and people cared more about top end benchmarks than anything else.

yawnxyz4• 3 years ago

Apparently npm runs couchdb under the hood. There’s your answer to the “does it scale” question

vorticalbox• 3 years ago

Does ferret support transactions and document write locking?

peterfarkas11• 3 years ago

Our development roadmap [1] is based on the needs of the users and the software we are focusing on building compatibility with.

Interestingly, transactions are not commonly used and therefore it is not high on our priority list, it may take months until we get to it. Nevertheless, if enough users would request it, we would reconsider. The same goes for any other item on the roadmap, please check it out.

[1] https://github.com/orgs/FerretDB/projects/2

ahachete2• 3 years ago

I applaud this effort.

Many years ago I founded a project called ToroDB [1]. ToroDB had a vision very similar to that of FerretDB's: help MongoDB users feel at home on Postgres. This has far reaching implications, like allowing MongoDB applications to run without MongoDB (this is what FerretDB is essentially and what "ToroDB Server" was meant to be) or to replicate data from MongoDB to Postgres to improve the performance of analytical queries by several orders of magnitude (that was "ToroDB Stampede").

ToroDB ended up being discontinued. Timing was not right. At the time, NoSQL was exploding, and users "didn't want to look back to SQL" --until they learned the notable advantages, but it was a time consuming and hard job. Today, there's a much higher acceptance of SQL and most recognize that data querying in many cases goes through, or is significantly helped, by SQL.

I wish FerretDB a successful road and reach to where ToroDB didn't reach at the time. Good luck and congratulations on the 1.0 launch!

[1] https://torodb.com

nacs• 3 years ago

Looks like they mentioned your DB in a (probably SEO-bait) blog post here:

https://blog.ferretdb.io/5-database-alternatives-mongodb-202...

peterfarkas11• 3 years ago

FerretDB maintainer here. We consulted the ToroDB team before we started FerretDB, as we wanted to understand why they couldn't conquer the world. They were way ahead of their time. And what a cool logo!

satvikpendem• 3 years ago

As well, Postgres now has great support for jsonb and querying that data, so you can effectively use Postgres as if it were NoSQL, if you really wanted to.

audioheavy• 3 years ago

This sounds like an interesting hybrid implementation, and I understand its motivation. However, if distributed writes at low latency with the highest consistency guarantees are essential, combining those two layers (Mongo and PostgreSQL) isn't worth the complexity. It looks like you can administer it with both Mongo and PostgreSQL tools, but do you want to? Clusters, partitioning, replication? That's a lot of heavy lifting.

To fully disclose, I have a biased view on this given that I work for a (closed source) serverless, no-ops DB provider (Fauna) that implements a distributed transaction engine that is natively document-relational and doesn't compromise on relational (ACID, transactional) guarantees. Although not directly comparable to an OSS DB, The market expects software and services to abstract as much complexity as possible. It is hard to imagine how such a Mongo/Postgres hybrid could handle hyper-scale apps that require highly consistent distributed writes.

peterfarkas11• 3 years ago

Thanks for the feedback. Any managed Postgres-compatible database which offers autoscaling works with FerretDB. You don't have to manage the Postgres side if it is already managed [1].

One example would be Yugabyte which is not yet officially supported, but was tested by Yugabyte with FerretDB. Same goes for CockroachDB, which was also tested to some extent. Neon would work, too.

On the other hand, FerretDB may not be a solution for all possible use cases out there, but no database is.

[1] https://dev.to/aws-heroes/ferretdb-yugabytedb-on-kubernetes-...

https://dzone.com/articles/migrating-mongodb-collections-to-...

https://dzone.com/articles/using-cockroachdb-as-a-backend-fo...

https://dzone.com/articles/experimenting-with-unique-constra...

https://dzone.com/articles/cockroachdb-multiregion-abstracti...

chunsj• 3 years ago

I don’t think it is that matter whether a license is OSI compliant or not. The more important one is whether it is a Free Software License compliant like GPL3 preferably.

bitwize• 3 years ago

Sigh. Again.

"Free software" is not coterminous with copyleft. It includes copyleft and non copyleft free licenses like BSD. Even Stallman would happily call BSD- or MIT-licensed stuff free software.

The OSD was carefully chosen to incorporate what was already known as free software.

djha-skin• 3 years ago

When MongoDB came out it was sort of the anti-SQL database, so it is interesting that this flavor of the same kind of database is backed by postgreSQL.

0xbadcafebee• 3 years ago

But is it web scale?

coding123• 3 years ago

How compatible is it, supposedly the mongo drivers can connect to it. Can you connect spark mongo connector to it?

hartator2• 3 years ago

Any benchmarks against regular MongoDB?

mathattack• 3 years ago

Probably a huge cost savings to turn off Al those MongoDB nodes.

exabrial• 3 years ago

Somewhere there's an actual implementation that copies everything to /dev/null, with the justification that if you're using mongodb, you probably don't care about your data anyway. Can't seem to find it.

aleksi11• 3 years ago

I think you mean https://github.com/dcramer/mangodb

FerretDB: open-source MongoDB alternative

Related Stories