Migrations and SVN branches (comments requested)

I have a tentative fix for the issue of using Rails migrations with Subversion branches. I wrote a plugin, announced here:

http://www.bradediger.com/blog/2006/11/subverted_migrations.html

It is available at:

svn://svn.madriska.com/plugins/subverted_migrations

The plugin changes Rails’s schema versioning to use version strings instead of integers, so you can have a version of “1-12,16-19” for example (with the missing versions being in another branch to be possibly merged in later). It also patches the db:migrate task to handle this versioning, as well as modifying the generate script to keep version numbers unique across all branches and trunk.

It is in alpha right now, but it is working well enough for me.

Is this something that would be a useful addition to Rails core? I’d appreciate some feedback / criticism.

Thanks,

Brad

Wow, this looks really complicated. I think the solution with all version control problems is communication, not complicated naming schemes. Whenever I have a problem with migration numbers I just: migrate down, merge, migrate up, commit.

Manfred

Thanks for the feedback. I don't think it's terribly complicated once you get used to it. With this change, you shouldn't have to think about merges when writing migrations. I just created kind of a contrived example to show that everything Just Works. Perhaps the complicated example obscures understanding.

The reason I wrote the plugin is to keep from having to fudge the migration version numbers when merging. I think it's a messy solution to have to back out schema changes in order to keep your versions straight. That sounds like a programmer serving his tools, and it doesn't work if you have even one irreversible migration.

Is there a particular part of the documentation you think is complicated, or is it a specific part of the workflow that is confusing?

Thanks again for your comments.
Brad

I agree, it does look complicated. I guess it depends on your
workflow though. Usually, I work solo or in a small team, so there
aren't a lot of branches. I haven't had any real issues with this as
long as I keep in contact.

I also work solo / in small teams, but I tend to have long-lived feature branches that don't get merged into production for a while (weeks to months) while developing and testing. These branches typically involve large schema changes that should be isolated from the production branch. Is there a more Rails-ish way to do schema branches like this?

>>>>
>>>> Is this something that would be a useful addition to Rails core?
>>>> I'd appreciate some feedback / criticism.
>>>
>>> Wow, this looks really complicated. I think the solution with all

Why can't we just have a ParallelMigration (IndependentMigration)
subclass that doesn't check for duplicates in the numbering?

My most common use case is where I'm taking someone else's app,
checking it in locally with svk, adding some fields or models here and
there, but my changes are really in parallel with the original
changes.

In that case I'd have

067_create_monkey_table
068_add_banana_field_to_monkey.rb
068_add_product_table.rb
069_add_monkey_field_to_product

We would assume from looking at the numbering that both 068_* are
somewhat dependent on 067, and come before 069. It would work only if
they are both this new independent migration class.

This opens up a whole new metaphor for thinking about migrations;
independent migration classes. So you'd add a few tables all at once,
but instead of putting then sequentially or in the same file, you make
3 parallel files.

I don't know if this solves the "long-running" branches issue, but
it's certainly simple..

self-reply here, pre-empting questions..

Courtenay wrote:

> >>>> Is this something that would be a useful addition to Rails core?
> >>>> I'd appreciate some feedback / criticism.
> >>>
> >>> Wow, this looks really complicated. I think the solution with all

Why can't we just have a ParallelMigration (IndependentMigration)
subclass that doesn't check for duplicates in the numbering?

My most common use case is where I'm taking someone else's app,
checking it in locally with svk, adding some fields or models here and
there, but my changes are really in parallel with the original
changes.

In that case I'd have

067_create_monkey_table
068_add_banana_field_to_monkey.rb
068_add_product_table.rb
069_add_monkey_field_to_product

We would assume from looking at the numbering that both 068_* are
somewhat dependent on 067, and come before 069. It would work only if
they are both this new independent migration class.

This opens up a whole new metaphor for thinking about migrations;
independent migration classes. So you'd add a few tables all at once,
but instead of putting then sequentially or in the same file, you make
3 parallel files.

I don't know if this solves the "long-running" branches issue, but
it's certainly simple..

I spend a lot of time doing branch-based development, where you make a
branch for each major feature. It's powerful, but merging migrations
is horrible.

What happens if you migrate up or down and you never applied one of the
parallel patches? or, if you merge someone's code and it contains old
migrations that you've progressed over?
For example, you're at version=52 but someone else's merged code adds
47 and 51, which you haven't run.. In this case you just migrate down
and up again..

What if you only applied one of the migrations 068, but you want to go
down anyway? I guess the IndependentMigration would have to specify if
you can skip it..otherwise, do what we all do when migrations half-run,
go in, comment out the offending code, migrate down, and uncomment it
again.

Damn, how'd it get to be 5am?

http://blog.caboo.se/articles/2006/12/08/simultaneous-migrations
http://dev.rubyonrails.org/ticket/6799

Here's the patch :slight_smile:

That looks really neat. It's certainly simpler than my solution, and I can't think of any problems with that off the top of my head.

be

I don't know. Maybe we should drop version numbers altogether ?

Take a look at my proposed solution:
http://blog.teksol.info/articles/2006/12/08/migrations-branches-and-dependencies

I'm in the process of writing it. I don't have tests yet, but I can
migrate up and down with no problem at this moment. I'll try and
release my patch as a plugin, so it can be tried.

Bye !

If I understand your suggestion correctly, it amounts to sorting and
applying migrations based on timestamp. I think real time is not the
right dimension for this as progress on different branches is not
coordinated by time. Rather, I suggest to apply migrations in the order
of their corresponding revision numbers. This approach doesn't
guarantee that there can be no conflicts when branches are merged, but
I think this can't be guaranteed anyway.

Michael

You are right, Michael. Sort by time, then apply. So, you suggest
that we use the revision number instead ? This would tie Rails to one
or more SCM. Is that advisable ?

Maybe we should sort by time the migration was committed ? But then
again, we tie Rails AR to SCM.

This is a hard problem.

The basic problem is endemic to SCM anyways. If SCM is not being used, there is generally no need for this fix. In my mind, there's nothing wrong with saying "if you use version control, Rails handles migration merging for you, but only if you use SVN". Nobody is worse off under that arrangement than they are now.

You think that tying rails features to specific dependencies of SCM is
a good thing?

I don't believe this is a hard problem.

The timestamp to me feels like "too much", and also, using SCM
revisions doesn't necessarily work, particularly where there are a few
branches, or where you're using (as what inspired this) something like
SVK.

The main issue here is, how do you keep track of multiple branches of
migrations per DB instance, particularly migrations added in the past.
I had thought about storing the migrations which have been run in the
schema table, and decided that it was also "too much", however, it
could be a good way to maintain the simplicity of numbering with
dependencies.

We already have the table of schema_info..

I don’t think there is anything wrong with tying certain features to a certain SCM system. However, looking back at this, I agree that that is not how this should be done. The problem is not that Rails itself would depend on a certain SCM, the problem is that the data would depend on a certain SCM and a certain set of version numbers. It would be fragile to SCM changes.

The original solution I gave (subverted_migrations) changes schema_info to reflect the set of migrations applied to the database (rather than just the latest migration). This is ideal because the totality of the versioning information is kept in the database rather than having to reference something external such as the version control repo.

The only part of the plugin that is tied to Subversion is the migration generator (and only then because I only work with Subversion). And it’s not tied to svn in a version-numbers sense; it only uses it for introspection into which files exist in which branches (to keep version numbers unique).

It is more complicated than migrations are now, but it has been working for me. The only time I have to manually merge is when two revisions in different branches make overlapping changes.

–be