Observers breaking migrations

We have developed an app with a number of observers and all seems well.

However, when deploying this to the live server, suddenly it failed to run
the migration to setup the database.

Turns out, when running the migrations it tries to load the observers. This
fails because whilst the models exist, the associated database tables don't
and the migration fails to start.

We commented out the lines in environment.rb in order to get the migrations
to run, and now it works fine, but that initial hurdle is annoying because
we have to do that manually.

Has anyone noticed this before and overcome the problem?

Should observers be loaded in migrations?