I use sudo for system lib, but also vendor everything when deployed.
IMO gems in system lib should be treated like system software, could
be used by other programs in OS or other users. If you have another
local user they could overwrite the gem and cause problems for
everyone.
Unfortunately that also means system lib is volatile and could be
'helpfully upgraded' at any moment by admins. I sure hope Gemfile is
gem config tool that actually works...