We replaced traditional ORM migrations with a DAG and stopped breaking prod

by vswaroop04 • 4 months ago

5 1 4 months ago

vswaroop04• 4 months ago

If you have worked with any Object Relational Mapping tool like Drizzle, Prisma, TypeORM, Django, Rails you have probably come across this problem:

Two developers are working on branches and they both create a migration file named 004.

One of them renames their file to 005 then you merge the changes deploy them and hope that everything works out fine.

Sometimes it does work.. Sometimes a column just disappears because the migration files were run in the wrong order.

We had to deal with this problem many times, including a few times when we had to stay up late at night to fix things that we decided to think of a new way to handle migrations.

Most Object Relational Mapping tools use an numbered system for migrations. This system works fine when one developer is working on a project but it does not work well when there are multiple branches. You end up having to coordinate the migrations dealing with conflicts or worse you have to deal with issues that only show up when the project is live.

Rolling back changes is another problem. Writing rollback scripts is hard and people often skip this step or get it wrong especially when multiple changes are involved.

That is why we built Migratex.

Of using a numbered list migrations in Migratex form a directed acyclic graph, which is similar to how Git models commits.

Each migration file declares what it depends on so multiple branches can create migration files without any conflicts.

When you merge changes:

* If two migration files truly conflict, for example if they try to change the column to different types Migratex flags the problem.

* If the migration files are independent for example if they change tables or columns they merge cleanly.

You do not have to rename any files. You do not have to coordinate with anyone. You do not have to worry about deployments because of ordering issues.

Migratex also generates SQL code automatically:

* You update your Object Relational Mapping schema.

* You run the migratex generate command.

* It compares the schema to the database and generates the migration SQL code.

Migratex also generates rollback scripts alongside the migration scripts to make rollbacks safer.

The goal of Migratex is simple:

Make schema changes safe in an environment with developers, without relying on manual ordering or fragile rollbacks.

We replaced traditional ORM migrations with a DAG and stopped breaking prod

Related Stories