Are you creating subdirectories by hand and putting controllers in them like normal, or are you properly namespacing them? e.g.:
admin/incidents_controller.rb: IncidentsController < ApplicationController
-or-
admin/incidents_controller.rb: Admin::IncidentsController < ApplicationController
-eric