Looking for an HTML parser

Why the Lucky Stiff has a great parser, hpricot

If you need to follow links or fill out forms as well, the trunk
of mechanize can use hpricot as it’s parser. Deadly combo!


Google for Rubyful Soup - it's a port (by the original author) of the
excellent Python parser "Beautiful Soup", which is explicitly designed
to deal with messy, badly-formed, awkward HTML - ie, the real-world
examples of it.