Factories give a lot more functionality, including inheritance and get really fancy when combined with stuff like Faker. I don’t think that the discussion here should be whether factories are useful or not, but if they are a good enough value added to rails.
I personally like the added functionality and expressability of factories and do not mind the extra time it takes to run the tests. But in my opinion what should really be taken in account is what we want someone that is just starting to test with rails to be using. Are fixtures really the best way to it? If so then maybe we could just add some reference in the documentation to other ways of doing it (factories), if not then changing the default to factories or extracting everything to separate gems and leave it to the user would be two possible solutions, IMO.