It’s time for me to tackle one of the “big ones”: database migrations. (cue dramatic music)
When you’re developing or prototyping, and not quite sure what your next schema will look like yet, it can be a major pain. Adding migrations one by one, rolling back and forth, etc. is not exactly pleasant.
The dream: I can only change a line in schema.rb, maybe run a one liner, and my local schema is up to date.
The approach makes me think about a major paradigm shift in Javascript in the 2010s: DOM diffing/declarative syntax. I remember the good old days before React and Vue (think before 2015), where we would use jQuery to add/remove every single DOM element manually. From the moment I switched, I never went back. I believe schema changes should aspire to the same.
When researching the issue the other day, I came across this provocatively-titled talk from djrobstep: Your migrations are bad, and you should feel bad, and its associated library, migra, a “comparison tool for Postgres”. In short, the tool diffs the two schemas and automatically infers a migration between the two.
I think this is an approach on which we could build upon.
Now, I’m sure there are many valid objections to it, an answer to some off the top of my head:
What if some changes/migrations are destructive? I don’t want to let a tool do that automatically. – I don’t think you should necessarily let the tool run wild. There could be a command “migrate my schema” which asks you to confirm that the inferred migrations are correct and that you want to run them. The design may vary here.
How do I know my migrations in production are going to be correct if they’re not explicitly defined? – There could be some tooling à la “convention over configuration” where you can either let the defaults run or specify them yourself. We could conceivably use this as a generator if one wishes in certain cases.
One thing in sure in my opinion, the process around migrations has a lot of space for improvement.
This is extra interesting to me right now because my current application uses Postgres materialized views heavily. We use scenic to manage them, and it works okay, but the zero-downtime migration path is complicated and view changes can be very hard to code review.
In a lot of ways it seems like you’re imagining something like Terraform for databases. That seems like an intensely ambitious project to me, but not in a bad way.
Interesting you mention views, I’m just now developing an app that’s using views, and I just discovered scenic. I was really hesitant to install it at first as I wasn’t sure it would be worth it, but turns out matching with a model and using associations is super useful. I’ll admit I was a bit disappointed to see that Rails doesn’t consider views in its model at all. (This would probably be another good topic to split on actually haha!)
About generated migrations, I do think this would be an ambitious project indeed. However, I think the corresponding benefits could also be huge. Not to write migrations anymore save some exceptions? Sign me up! – That’s why I’m testing the waters to see whether more folks are tempted by this too, mind you
I’m thinking at the moment, this might be a good idea of a side gem. Maybe it wouldn’t be too complicated to reuse the same kind of logic Migra has and just generate the migrations as a first step. For inclusion in core, well… we’ll see!
I don’t know how much capacity I would actually have to help out with a “Terraform for db migrations” style gem, but I’d be extremely interested in using one if it a solid one existed and would contribute where I could.
