Observing HABTM-associations with observers

I'm trying to write an observer which tracks changes to my data. This seems to
be an impossible for habtm-associations as all hooks in an observer just
works for models. As habtm-associatons gets saved when they're added to a
collection I haven't found a way to get an observer called. What's a good
solution to this problem?

I really wonder why there isn't a way of adding objects to a collection
without instantly creating an entry in the join table an just create these
entry when the parent object is saved?

Regards,
Timo