Having an OrderedHash that saves insertion order is useful enough and I personally don’t see a reason for a custom ordered hash.
Wouldn’t it be less expensive to order only the keys in a separate array and access the hash rather than creating a new hash?
Honestly, I’ve needed it many times and was surprised that it wasn’t implemented yet. Of course, there are a few ways you can do this (probably more efficient way than what I suggested) but we also need to keep in mind that we should keep our code as DRY as possible.