What characters need to be escaped?

Obviously, less and greater than's (< >) need to be escaped to prevent XSS, but what are the other potentially harmful characters?

im curious about this also due to letting users enter some html data, but not harmful scripts