samuelstros

track the source of the asset and it works. take ui design. dont track the svg. track the design file itself

Simple left or right merge. One overwrites the other one.

The appeal or structured file formats like .docx, .json, etc. Images are unstructured and simple "do you want to keep the left or right image" is good enough.

Improving on "git not handling non-text files" is a semantic understanding aka parse step in between the file write.

Take a docx, write the file, parse it into entities e.g. paragraph, table, etc. and track changes on those entities instead of the binary blob. You can apply the same logic to files used in game development.

The hard part is making this fast enough. But I am working on this with lix [0].

[0] https://github.com/opral/lix

How do you get blob file writes fast?

I built lix [0] which stores AST’s instead of blobs.

Direct AST writing works for apps that are “ast aware”. And I can confirm, it works great.

But, the all software just writes bytes atm.

The binary -> parse -> diff is too slow.

The parse and diff step need to get out of the hot path. That semi defeats the idea of a VCS that stores ASTs though.

[0] https://github.com/opral/lix

Yep “stripe couldnt build it because they rely on http ingestion”. Lol.

Oh interesting. Checks sound similar to lix validation rules [1].

We were coming from a an application perspective where blocking the users intent is a no-go.

Do you have a link to a discussion where the JJ community is discussing checks?

[1] https://github.com/opral/lix/issues/239

oh i didn't know that claude code has a desktop app already

It's basically what Emdash (https://www.emdash.sh/), Conductor (https://www.conductor.build/) & CO have been building but as first class product from OpenAI.

Begs the question if Anthropic will follow up with a first-class Claude Code "multi agent" (git worktree) app themselves.

The SSIS use case is a good one for lix.

Write an SSIS plugin (essentially just parse the XML) and lix can track the xml props in the SSIS package. Opposed to line by line stuff that git does.

Knowing what the actual semantic deltas are in binary files (what your doc examples claimed) is hard.

Hm that is what lix provides?

SQL is just the interface to query the deltas.

Anyhow, lesson learned. The primary use case for lix is embedding. Positioning lix for binary files leading to existing systems. None of those support the embedded use case. Thus, don't position lix for binary files :)

I think our goals indeed differ.

How often are binary files being diffed? How long does it take to materialize? How long to run a diff algorithm?

If version control is embedded in an app, constantly.

Imagine a cell in a spreadsheet. An application wants to display a "blame" for a cell C43 i.e. how did the cell change over time?

The lix way is this SQL query

SELECT * from state_history WHERE file_id <the_spreadsheet> AND schema_key "excel_cell" AND entity_id C43;

Diffing on the fly is not possible. The information on what changed needs to be available without diffing. Otherwise, diffing an entire spreadsheet file for every commit on how cell C43 changed takes ages.

It can but the plugins are not developed for production readiness yet. I should clarify that.

The way to write a plugin:

Take an off the shelf parser for pdf, docx, etc. and write a lix plugin. The moment a plugin parses a binary file into structured data, lix can handle the version control stuff.

Does the following make more sense to you in respect to SQL?

Lix uses SQL databases as storage and query engine. Aka you get a filesystem on top of your SQL database that is version controlled.

Or, the analogy to git: Git uses the computers filesystem as storage layer. Lix uses SQL database (with the nice benefit of making everything queryable via SQL).

Lix doesn't reinvent databases — durability, ACID, and corruption recovery are handled by battle-tested SQL databases.

> This seems like a left turn from the value prop and why the value prop matters?

Better wording might be "Lix uses a SQL database as storage layer"?

The SQL part is crucial for two reasons. First, the hard part like data loss guarantees, transactions, etc. are taking care of by the database. We don't have to build custom stuff. Which secondly, reduces the risk for adapters that data loss can occur with lix.

As for how it works, both in the docs and in the comment I'm replying to, it's unclear how any of this interacts with the native version control embedded in M365 apps or why this tool can be trusted as effective at tracking M365 content changes.

It doesn't interact with version control in M365.

I'll update the positioning. Lix is a library to embed version control in whatever developers are building. Right now, lis is mostly interesting for startups that build AI-first solutions. They run into the problem "how do customers verify the changes AI agents make?".

The angle of universal version control, and using docx or Excel as an example, triggers the wrong comparisons. By no means is Lix competing with Sharepoint or existing version control solutions for MS Office.

Merge algebra is similar to git with a three way merge. Given that lix tracks individual changes, the three way merge is more fine grained.

In case of a conflict, you can either decide to do last write wins or surface the conflict to the user e.g. "Do you want to keep version A or version B?"

The SQL engine is merge unrelated. Lix uses SQL as storage and query engine, but not for merges.

Thanks for the feedback.

AI agents are the pull right now to why version control is needed outside of software engineering.

The mistake in the blog post is triggering comparisons to git, which leads to “why is this better/different than git?”.

If you have a custom binary file, you can write a plugin for it! :)

Lix runs on top of a SQL database because we initially built lix on top of git but needed:

- database semantics (transactions, acid, etc.)

- SQL to express history queries (diffing arbitrary file formats cant be solved with a simple diff() API)

Indeed, if lix were to target code version controlling, incompatibility with git is a “dead on arrival” situation.

But, Lix use case is not version controlling code.

It’s embedding version control in applications. Hence, the reason why lix runs within SQL databases. Apps have databases. Lix runs of top of them.

The benefit for the developer is a version control system within their database, and exposing version control to users.

But then the first thing it talks about is diffing files. Which honestly shouldn’t even be a feature of VCS. That’s just a separate layer.

There is nuance between git line by line diffing and what lix does.

For text diffing it holds true that diffing is a separate layer. Text files are small in size which allows on the fly diffing (that's what git does) by comparing two docs.

On the fly diffing doesn't work for structured file formats like xlsx, fig, dwg etc. It's too expensive. Both in terms of materializing two files at specific commits, and then diffing these two files.

What lix does under the hood is tracking individual changes, _which allows rendering a diff without on the fly diffing_. So lix is kind of responsible for the diffs but only in the sense that it provides a SQL API to query changes between two states. How the diff is rendered is up to the application.

Learnings from the comments so far: I need to refine the positioning of lix.

Lix is not a replacement for git. Nor does it target version controlling code as the primary use case.

A better positioning might be "version control system as a library". The primary use case is embedding lix into applications, AI agents, etc. that need version control.

I need to to bed now. I have a flight to catch in 6 hours.

PS I am open to suggestions regarding the positioning!

Yes. The tracking works via plugins to keep it generic. Here is a rough illustration:

File change -> Plugin (detects changes) -> Lix

It works surprisingly well because most standard file formats have off the shelf parsers. Parse a file format, and et voila, it is trivial to diff. Then pass on a standard schema for changes to lix and you end up with a generic API to query changes.

Hi,

I'm the creator of lix.

Lix doesn't target code version control. It can be used for it. But the primary use case is embedding version control in applications. Such an application can be an AI agent that modifies files which entails the need to show what the agent did in that file e.g. tracking the changes.

Git is good enough for code. I don't think there is space to gain much market share.

Correct. Lix has been developed with the embedded use-case in mind.

Someone can write a CLI for it. Though, the primary use case is not code version control but embedding into applications

Depends on the diff you render and what the plugin tracks.

In general, lix gives in API to track changes in any file format (via plugins). The "diff noise" thus depends on a) the plugin i.e. does it track them metadata? and b) what is rendered as the diff.

If the user doesn't care about seeing a diff of metadata in Excel, don't render the metadata in the diff. The latter is trivial because diffing in lix is just a SQL query.

Holy moly. I just went to bed. Checking my phone for last time. Opening hackernews for "one last scroll" and see lix, my project, popping up here.

Going through the questions now. So much for going to bed.

Git only works for text files. Everything else is a binary blob which, among other things, leads to merge conflicts, storage explosion, and slow git operations

RFC Hub has so much functionality that it's difficult to explain it all on the homepage

That's what I meant with overwhelming / too niche.

It seems like you intend to productize the RFC process e2e. But most "time consuming" parts of an RFC process is the human stuff "Did you read this?" "Did you update the RFC again?" etc. That back-and-forth seems to be expressed by all the features you have in RFC Hub but:

1. That makes RFC hub complicated.

2. Requires buy-in from every party to participate in all of RFC hubs feature like "Yes, I reviewed it and pressed the reviewed button in RFC Hub"

1 & 2 combined make RFC Hub (likely) a very niche product. New users are overwhelmed. Existing users need to onboard new users (their collegues) though. Otherwise, the RFC process will fallback to just DMs on Slack. Only a few teams will have sufficient buy in from all team members.

Initial reaction: Looks too complicated & too niche of a problem to appeal to a sustainably large user group.

GDocs might be annoying to track who read the RFC etc. etc. but everyone is familiar with it.

I write RFCs, I share RFCs and your tool seems to require a substantial amount of buy in

- register

- unclear what the writing experience is

- outdated / overloaded UI

The last RFC I wrote was in hackmd (https://hackmd.io/Jjy-afCWS4CAFlHa62anMQ) because

- I wanted Markdown to store the RFC in git eventually

- Google Docs has issues with Markdown rouundtrips

- I didn't want to use git to write with VSCode (although... I actually did. I let CLaude Code write most of the RFC under my guidance, then put it into hackmd for easy sharing)

I hope the feedback helps!

I recommend adding an installable executable. It seems like you are expecting people to clone your repo and then run it locally?

Take a look at Monitor Control how they structured their GitHub Readme and release page to download a dmg https://github.com/MonitorControl/MonitorControl

does a framework like vercel's ai sdk help, or is handling the loop + tool calling so straightforward that a framework is overcomplicating things?

for context, i want to build a claude code like agent in a WYSIWYG markdown app. that's how i stumbled on your blog post :)

can you elaborate on what versioning issues you are facing?

100% agree with the sentimment. markdown is hell as a format for editors :D

the effort it takes to serialize and parse markdown into an AST that rich text editor frameworks reliably operate on takes months. been there, done that. the majority of the engineering effort of building a markdown editor in the browser went into parsing and serializing markdown :/

Anyhow, we took the learnings from the Markdown editor app and created "zettel" as a result: https://github.com/opral/monorepo/tree/main/packages/zettel/.... The goal is to have an interoperable rich text AST—basically Markdown but with an AST spec.