Feature proposal: Sort database tables, columns, foreign keys, etc during schema.rb generation

For context: I am the original author of the fix-db-schema-conflicts gem.

It prevents the kind of annoying churn described in this earlier thread:

https://groups.google.com/forum/?fromgroups#!searchin/rubyonrails-core/sort$20columns$20schema|sort:date/rubyonrails-core/h6q8nGFzGKs/pztvg5sJrbgJ

Namely, when two people make changes to the same Rails codebase it can create a situation where they are forever locked into a contest of flip-flopping the order of columns/foreign-keys, etc in the db/schema.rb file each time they make changes.

The fix-db-schema-conflicts gem “fixes” this by prepending some code to the Rails schema generation that sorts everything before spitting it out to schema.rb.

Additionally it uses Rubocop to autoformat the generated file to use uniform spacing and column alignment.

Over the years I’ve maintained this gem the vast majority of issues surrounding it have come from Rubocop compatibility with Rails. The Rubocop autocorrect is really something that could be applied separately by any team and was only added to my gem as a convenience, which has turned into a huge mess in terms of gem maintenance. The real value of my gem comes from the ordering of the database schema.

I propose making a small change to Rails that integrates my gem’s schema sorting logic into Rails by default, with a configuration option to preserve the ordering if a specific application relies on column sort order for some reason.
This change would not affect structure.sql in any way.
In my mind this is a common source of frustration for every single Rails project that has more than contributor, and it is a simple, low-risk change (can be reverted with a configuration).

I would be very glad to do the work if it was desired.

What does the community think, before I spend time on a PR for this?

I’d rather see a public API that we could hook into safely. While the sorting sounds helpful, I’ve needed other behavior changes in the ruby schema dumper too.

What kind of other changes have you needed? What features would you like an API to provide?

For my usecase, I have a large monolith with over 100 team members. The churn and merge conflicts are constant and painful.

The schema file is over 13k lines with over 600 tables. I’ve been playing with some monkey patching to generate ruby schema dumps into multiple files in a subdir, one table per file.

It works, but I don’t look forward to future breaking changes. :slight_smile:

I use postgresql, and I’ve hit this a couple of times. I think it happens when I’m working locally on migrations that I end up reverting.

The public API suggestion feels separate from this. Adding your custom schema dumper may make sense, but the default schema dumper should be robust.

The column alignment problem Jake mentions was solved already in 5.1.0:

Sorting columns follows the same spirit of minimizing conflicts / surprises in the database schema file.

I think that this is a good idea.

I’ve been hit by this a handful of times as well, and it would be a welcome change IMO.

I really think it would help with the conflict resolution, at least.

I found a PR that was closed that tackles something similar, but it was closed due to lack of reproducibility (from what I understand).

Kind of wonder if this has to do with db:migrate being run on a case sensitive Mac.