paperclip unit files created during tests left on system

I have a rails 3.0.3 app that uses paperclip (2.3.8) to store files on
s3 using the s3 gem. In my units tests I'm using the local file system
and I have noticed that the files are left on the system after the
tests have run. Looking at the logs I could see that the destroy
method was never called on the model with the attachment. I added a
teardown method that explicitly made the destroy call to ensure that
paperclip destroy callbacks were run and the files deleted- this
worked. I am pretty new to rails but I was a bit surprised by this.
Does the TestCase just wipe the database bypassing the model callbacks
or is there something else strange going on with my app and test
setup?