Scrapping Content from a website

Hey all,   I want to scrap some content from a website to rails application and wants to authenticate the url of the website which I want to scarp so please help me out....

Thank You!!!

I use Mechanize as user agent and Nokogiri as html/xml parser. http://mechanize.rubyforge.org/

Mechanize supports both http- and application- layer auth, so that could be what You are looking for.

I don't have a lot of experience but did some scraping recently. I used the nokogiri gem. found it to be pretty self explanatory and much easier to implement than i had anticipated. google it and i'm sure you can figure it out...

Max

Thanks Dear for the information.

Hey all,

Hi,

I want to scrap some content from a website to rails application and

wants to authenticate the url of the website which I want to scarp so

please help me out…

Mechanize, Nokogiri , Hpricot.

Some examples in ruby done by me

http://sathia27.wordpress.com/2011/09/04/version1-easy-dictionary/

http://sathia27.wordpress.com/2011/12/05/english-to-tamil-translator-script/

http://sathia27.wordpress.com/2011/09/06/tbus-version-1-search-bus-routes-from-terminal/

http://sathia27.wordpress.com/2011/05/30/update-foss-job-script/