Hey all, I want to scrap some content from a website to rails application and wants to authenticate the url of the website which I want to scarp so please help me out....
Thank You!!!
Hey all, I want to scrap some content from a website to rails application and wants to authenticate the url of the website which I want to scarp so please help me out....
Thank You!!!
I use Mechanize as user agent and Nokogiri as html/xml parser. http://mechanize.rubyforge.org/
Mechanize supports both http- and application- layer auth, so that could be what You are looking for.
I don't have a lot of experience but did some scraping recently. I used the nokogiri gem. found it to be pretty self explanatory and much easier to implement than i had anticipated. google it and i'm sure you can figure it out...
Max
Thanks Dear for the information.
Hey all,
Hi,
I want to scrap some content from a website to rails application and
wants to authenticate the url of the website which I want to scarp so
please help me out…
Mechanize, Nokogiri , Hpricot.
Some examples in ruby done by me
http://sathia27.wordpress.com/2011/09/04/version1-easy-dictionary/
http://sathia27.wordpress.com/2011/12/05/english-to-tamil-translator-script/
http://sathia27.wordpress.com/2011/09/06/tbus-version-1-search-bus-routes-from-terminal/
http://sathia27.wordpress.com/2011/05/30/update-foss-job-script/