I’ve been doing web design since 1995, and this has always been an issue, always compounded by the wide array of browsers out there One thing that’s really been helpful for me is to do all development with Firefox as the target browser. It then makes it extremely easy to fix problems with IE (using conditional comments) or Safari (using browser hacks). I’ve found that if I develop using IE, I spend considerably much more time making it look right on other browsers.
You should strive for usability and readability. Your site should work on all browsers. It doesn’t have to look THE SAME across all browsers and platforms, but it should be easy to read and should have the same functionality. Most of your users will not be comparing your site on multiple browsers!