how to assign edit/remove privileges for only own details and not on others?

In addition to using authorization gems like CanCan, you can also do
these things yourself. Before each action in each controller, get the
current user. Before any action that would modify, or show in such a
way as to allow modifying, any records, check if the current user is
the owner of those records. If not, redirect to an error page. Also
don't give him any *links* to such things! For instance, on the pages
where he can *view* other people's records, don't put an "Edit" link.