Note belongs_to Company or Contact?

Check out polymorphic associations. Should do exactly what you’re looking for.