Gun – “Self-hosted Firebase”

http://gun.js.org/

by pimpl • 11 years ago

66 19 11 years ago

gun.js.org

jvehent• 11 years ago

koolkao• 11 years ago

I believe deepstream which received good feedback on HN also solves a similar problem.

With such proliferation of self hosted FireBase, can anyone in the know share their experiences with these solutions?

curiousjorge• 11 years ago

how does this not have more votes? im skeptical about the self hosted firebase claim, for instance, how would you scale this?

indrax• 11 years ago

With a two way dotted arrow, obviously.

yid2• 11 years ago

It requires zero maintenance and runs on your own infrastructure.

I don't understand how these two claims are not contradictory.

All conflict resolution happens locally in each peer using a deterministic algorithm.

How exactly does this happen? Via CRDTs? Consensus? How do you handle conflicts?

I'm very skeptical about this sort of largely content-free copy fronting what should be a pretty complicated distributed system.

avodonosov• 11 years ago

Yes, I am skeptical about automatic conflict resolution too. But the project is interesting and uses many right concepts.

marknadal6• 11 years ago

Good point, haven't heard that one before. Let me clarify. GUN requires no additional maintenance because it gets embedded into your app, compared to most databases which require you to run (and maintain) a database server.

If you do not want to even run your own app servers, I'm more then happy to let you use my free GUN backend so that way you do not even have to worry about that.

Consensus? Absolutely not. I definitely need to add more documentation and resources on this - in fact I'm doing a talk on how my conflict resolution algorithm works out in Berlin in a few weeks. Check the conference out at: https://2015.distributed-matters.org/ber/ . After that I should have much better materials on this available for people.

Briefly and naively, the way it works is a Vector Clock + Timestamp combo. Timestamps have bad exploits, and Vector Clocks don't work well in ephemeral environments. If you combine the two together they compensate for the vulnerabilities of the other. Every peer acts as a state machine operating inside of an open-closed boundary function (nothing fancy here, this is the same stuff you learned in middle/highschool). This does NOT give you Global Consistency, as different peers might have slightly different boundaries because of clock drift (you can use a separate service to minimize this), but they will all become Eventually Consistent.

This helpful? Any other questions?

yid2• 11 years ago

If you combine the two together they compensate for the vulnerabilities of the other.

Do you have a reference for this? I don't mean to keep sounding like a skeptic, but these are pretty big claims. I'm also genuinely curious about this claim and would really like to look into it further.

rubiquity2• 11 years ago

Because gun is not a database (NoDB), it is a persisted distributed cache. The fatal flaw with databases is that they assume some centralized authority. While this may be the case initially when you are small, it always ceases to be true when you become large enough that concurrency is unavoidable. No amount of leader election and consensus algorithms can patch this without facing an unjustified amount of complexity. Gun resolves all this by biting the bullet - it solves the hard problems first, not last. It gets data synchronization and conflict resolution right from the beginning, so it never has to rely on vulnerable leader election or consensus locking.

That's some of the most hand-wavy drivel I've ever read.

nathancahill• 11 years ago

I'd love to see Kyle Kingsbury put that through it's paces.

marknadal6• 11 years ago

Me too! I'm lucky enough to be talking alongside Kyle at https://2015.distributed-matters.org/ber/ in a few weeks. I'm hoping to get to know him and see if he'll bash up my database and help find flaws (I'm very confident in the mathematics of the system, however implementations are far from academic).

lobster_johnson• 11 years ago

I don't see what's hand-wavy about it. He makes a clear and succinct point: In a distributed database, managing consistency at scale gets complex because most systems are designed to assume a centralized authority [of consistency]. By removing the centralized authority and making conflict resolution the main mode of operation, the design can (according to the author) be made simpler.

I don't disagree with any of that, and it's not a new idea (Lotus Notes and CouchDB come to mind as databases that solve replication through conflict handling), although I'm not seeing a clear description of how this synchronization algorithm actually works. There's a description in the wiki [1], but it's unfinished. (Looking at the code, the entire project looks like quick-and-dirty, unfinished prototype, actually.)

[1] https://github.com/amark/gun/wiki/Conflict-Resolution-with-G...

rubiquity2• 11 years ago

Consenus, replication and consistency are potential solutions and their own problems to another entire set of really hard problems. If you're going to say you solved them you better say how, and most importantly, what tradeoffs you made to do so.

hiou• 11 years ago

Also see http://hood.ie for what appears to be a similar goal.

Lazare• 11 years ago

Seems a lot like CouchDB/PouchDB. Which makes the fact that the words "couch" and "pouch" do not appear anywhere on this page rather concerning to me, because the obvious questions I have are:

1. How is this better than CouchDB

2. How does this differ from CouchDB

3. Why did you not implement this on top of CouchDB

The lack of mention suggests the developers either have no clue what already exists, know that their solution can't compete with existing solutions, or don't really care about being better and are just having fun implementing their own system from scratch. None of those answers make me want to take a deeper look.

marknadal6• 11 years ago

You are right, it is very similar to Couch/PouchDB. Honestly, I tried Couch back in 2010 and wasn't very impressed and haven't really kept up with it since. Although I regularly hear good things about it and am aware it is one of the rare ones that is offline first. MongoDB wound up winning that market, so I didn't really think to provide a comparison to Couch.

So pardon my out-of-date ignorance:

1. Depends upon what you want. GUN can do dynamic queries and has realtime push notifications baked right in. What you didn't ask is how GUN is worse. Well, CouchDB is stable and has security - stuff which we're still working on.

2. GUN is a graph database which allows you to have key-value, relational, and document based data. Couch is a document based database. GUN gets embedded into your application server, while CouchDB has to run its own database server which then requires maintenance.

3. Mathematically speaking, any matrix (table) or tree (document) can be represented in a graph. But not all graphs can be represented as a matrix or tree. So building GUN ontop of CouchDB would have come with a lot of overhead, especially since CouchDB does not have dynamic queries.

Hope this helps! Let me know if you have any other questions.

marknadal6• 11 years ago

Just now noticed my project was on HN! Awesome, thanks to whoever submitted it. Will be replying to the comments now.

Gun – “Self-hosted Firebase”

Related Stories