HN user
One nice upside of having a single way to encode a value is fuzzing: when you work on an encoder/decoder, you can use a fuzzer and do round-trip comparison until you find crashes or inputs/outputs that don't match (and therefore issues in the code). But with LEB128 for example, the fuzzer quickly learn about those alternatives encoding and there is not much you can do from there.
https://github.com/git-bug/git-bug if that's your thing.
Any recommandation for a quality non-toy rust codebase to study?
On CRDTs: I assume tools like git-bug adopted CRDTs primarily to avoid merge conflicts, but "last-writer-wins" via timestamps is risky
FYI, git-bug doesn't use timestamps to figure out the ordering. It first uses the git DAG topology (it defines ancestors), then Lamport clocks (increment for any changes on any bugs), then order by hash if there is still an ambiguity. Note that the git DAG could also be signed, which would also provide some form of reliance against abuse.
I had an interesting discussion recently about how to handle conflict for bug trackers. In my opinion it's a great use-cases for CRDTs (as it avoids data corruption), as long as all user intents are visibly captured in a timeline and easily fixable. It turned out though that there is an interesting middle ground: as the CRDT captures *when* a concurrent editing happen, it's 100% doable to highlight in the UI which event are concurrent and might need attention.
Assuming that by "table" you mean another "document type" ... pretty easily. There is a reusable CRDT like datastructure that you can use to define your own thing. You do that by defining the operations that can happen on it and what they do. You don't have to handle the interaction with git or the conflict resolution.
Stuff like git-bug exists, but then you still need participation from other people.
The plan is to 1) finish the webUI and 2) accept external auth (e.g. github OAuth). Once done, anyone can trivially host publicly their own forge and accept public contribution without any buy-in effort. Then, if user wants to go native they just install git-bug locally.
Location: France
Remote: Yes (have been 100% remote for 6+ years)
Willing to relocate: No
Technologies: Go, networking, protocols, identity, cryptography, local-first, CRDTs, devops, AWS, kubernetes, IaC, free software, linux
Résumé/CV: see my Github profile for notable and open source work: https://github.com/MichaelMure
Email: on my Github profile
I'm a builder, a backend engineer with strong go proficiency, passionate about local-first and how we can build better software for a better world. Happy to talk about opportunities.
With some motivation you could port git-bug to another VCS without too much problem. You would need to implement those interfaces [1]. The one you care about especially is RepoData, which mainly imply you can store a DAG, have references and push/pull. I believe other VCS (say mercurial) have similar concepts.
Or you could just as well plug a generic database there.
[1] https://github.com/git-bug/git-bug/blob/master/repository/re...
Right now, yes, but the idea is to augment the webUI with external auth (e.g. Github OAuth and others) to make it a public portal where anyone can create issues and so on. In that case, the webUI would have access to the git repo, enforce any rules and prevent abuses.
With a single binary deployment, you'd just need a bit of config and a DNS, and you could host a forge-ish for your project.
We are not there yet but it's really not far.
So for example, git-bug already has a PR to add support for a project board: https://github.com/git-bug/git-bug/pull/843
The same way, one could add support for code review (aka PRs), todo list, custom entities that your workflow need (say, tracking documentation or custom requirement) ... It can also be entirely outside of the development process.
In theory it could happen but it's unlikely in practice, for multiple reasons:
- git-bug use a form of logical clock (not wall clock) that order an action in relation to other actions in the repo. Clock drifting doesn't matter.
- pushing to git usually require some access to the repo, and therefore abuse can be dealt with socially (aka you get kicked out)
What can happen for example is someone write a comment, shut down the computer and only push the next day, but in that case the comment showing up before yours is the correct merging.
Yes, this really meant to be some sort of framework for storing entities in git, handle the conflicts, and let you buld easily your own tool (or add more features to git-bug).
See also https://github.com/git-bug/git-bug/blob/master/doc/design/da... and https://github.com/git-bug/git-bug/blob/master/entity/dag/ex...
I'd love to see this used in the wild for other use cases.
Golang does that natively ;-)
The interesting thing to me is the stark difference between this and golang's approach.
With golang, you can run fuzzing as simply as you run tests, which means that it's trivial to target specific parts of your application or library. It obsoletes so much of those techniques.
I'm quite curious of techniques to guide more the fuzzing. It seems like the best you can do is provide a seed corpus and hope for the best.
Not a file storage but https://github.com/git-bug/git-bug push and sync with any git remote. There is a generic data structure you can use to build your conflict-free type.
A practical example of a op-based Merkle-DAG CRDT is (I believe) git-bug[1]. Some doc here[2].
I originally wrote something akin to an op-based CRDT and enforcing a purely linear series of commits in git's DAG, but eventually found that it doesn't really work in a multiplayer configuration. Eventually, I realized that I could instead have a real DAG to capture the concurrent editions, with "empty commits" as DAG merge operation.
The result is more or less what is described in that article, with some nice properties:
- written in go, I now have a generic implementation[3][4] that, given a set of operation, can easily support many practical use cases (bug tracker issue is the first, kanban and pull-requests coming).
- git itself is taking care of the storage and the synchronization with peers (aka git remotes). I get the full set of upsides described in the article.
- unfinished for now, but I can leverage some git construct to crypto sign each interaction with the data structure, to prove authenticity and later construct a proper access right system (who can edit a comment, who has admin right ...).
- additionally to the DAG structure, I also have lamport clock to give an order between each independent DAGs (last edited bug ...). They are also used as a help to compute the final order within a DAG if there is ambiguity (concurrent editing).
I'm much more an engineer than a researcher, so it'd be awesome to have the opinion of the community and especially iamwil, hecturchi or lxpz.
[1] https://github.com/MichaelMure/git-bug [2]: https://github.com/MichaelMure/git-bug/blob/master/doc/model... [3]: https://github.com/MichaelMure/git-bug/blob/master/entity/da... [4]: https://github.com/MichaelMure/git-bug/tree/master/entity/da...
I always wondered: is there actual study about this?
There is people arguing both ways with relatively good arguments, but what about quantifiable realities? As the author of GPL-licensed softwares, I didn't have much reasons to go that way or another, apart from hunch. Can we quantify those effects, so that we can properly align our licenses with the effect we want?
As a sort of spiritual successor to git-appraise, I've been working on git-bug[1] which support issues and will at some point support kanban and code review. There is a few notables improvements:
- CRDT-like reusable data structure [2][3] for true p2p workflow and easily create new entities (code review ...)
- bidirectional bridges to github, gitlab ... to ease the transition or just use git-bug as a complement of those platform
- CLI, terminal UI and web UI, for different taste and integrate into your tooling/workflow
[1] https://github.com/MichaelMure/git-bug
[2] https://github.com/MichaelMure/git-bug/blob/master/doc/model...
[3] https://github.com/MichaelMure/git-bug/blob/master/entity/da...
... of course. Mirror there: https://gitlab.com/MichaelMure/git-bug
Daily reminder that https://github.com/MichaelMure/git-bug could use some help :-D
If that makes you mad, I still need help with https://github.com/MichaelMure/git-bug ;-) Coming at some point, kanban and pull-request support, offline-first!
You might be interested by git-bug and https://github.com/MichaelMure/git-bug/blob/master/doc/model..., which seems to be exactly what you describe. (Disclaimer: author).
https://github.com/MichaelMure/git-bug
Offline-first bug tracker (and soon-ish forge) embedded in git.
I keep working on this because the idea makes a lot of sense to me, because I learn a lot from it and it benefits me indirectly.
I'm not especially looking to monetize, but I'm curious about what this community think about it.
Really cool. I have with git-bug[0] similar properties (offline first, identities, generic "crdt like" base data structure... ). Maybe you'd like to have a look and improve each other's design.
A bugtracker being "branch aware" and able to say where an issue started, where it was fixed ... can (and should IMHO) be a different concern than how the data is stored and how conflict resolution happen. By conflict resolution here, I mean: what happens when two user on different machines edit their issues and then states get merged? If you have this data into some format (let say JSON) and rely on git automatically merging, you eventually end up with corrupted formatting. Instead, you can do all that in your own branches, not pollute the normal code and not let git do any merging. Then, you can still have your bugtracker understand branches and track that information.
It's significantly more friction for little benefit compared to git-bug's solution. First, storing this data alongside the code means requiring everyone agreeing to use that tool (quite high ask). Then, that means that you let git merge said data in case of conflict, which leaves you two options: merge failure, or requiring the user to fix things. None of that is acceptable, and it's why many of previous attempt failed.
git-bug's author here with my 2cts. For what it's worth, I was aware of git notes during my original design process, but I found them clunky. I realized they would not be enough on their own to do conflict resolution and I'd be better making my own data structure with regular blobs, trees and commits. But when you have that, there is pretty much no point in using notes, as that would just grow the required features on which you sit on. What if I want to port git-bug to mercurial or whatever comes next? Even metadatas (the natural point of notes) are better expressed within that same unique data structure. Additionally, you then only need to sign those commits to cryptographically lock everything.
Thank you for that idea, I haven't considered that. I'm not too sure about how feasible that is as right now bridges sit on a sort of framework for configurations, credentials, events ... that seems uneasy to break into a plugin protocol. We'll see where that goes.
I haven't considered myself or git-bug worthy enough to bother him :-|
I was aware of fossil but never used it. It's not a new idea, it has been attempted many time before. Hopefully that one is the one reaching large usage :-) Regarding VSCode, the CLI commands are designed to be used for such integration. Hopefully someone makes it.