Hello list,
I’ve noticed ActiveStorage::Service::DiskService does two levels of directory sharding for Blob pathnames:
$ tree storage/ storage/ ±- aa/ ±- bb/ > ±- aabb00… ±- cc/ ±- aacc11…
However, when it comes to variants, the sharding is circumvented:
$ tree storage/ storage/ ±- aa/
±- bb/
±- aabb00… ±- cc/ ±- aacc11… ±- va/ ±- ri/ ±- variants/ ±- aabb00…/ ±- ±- aacc11…/ ±-
Should sharding exclude variant/ key prefixes? Maybe the directory layout can look like this:
$ tree storage/ storage/ ±- aa/
±- bb/
±- aabb00… ±- cc/ ±- aacc11… ±- variants/ ±- aa/ ±- bb/ ±- aabb00…/ ±- ±- cc/ ±- aacc11…/ ±-
This might not actually a problem at all, as the limiting factor on ext4 file systems is the inode index (which allow directories to contain ~10 million entries with 32 character long names (reference); the Base58 blob id is only 24 characters long). I guess with ZFS it’s even less of a problem.
I was going to create an issue for this, but this might actually be expected/desired behaviour. I’m also not sure if this is a bug or feature request either. I know changing this will cause some headache. I’m primarily looking for other opinions.
Kind Regards,
Dominik