HOWTO Let Rails login into another website

rubyonrails-talk
1

Hi guys,

I use Nokogiri to fetch some HTML pages and scrape it in the background. So far it works fine. But now I have to login to view some other pages in the background. How can I login in the background to fetch some HTML? In other words how can I login in the background using Rails?

Note that I'm not talking about some login form or something.

2

Hi!

I don’t know if I understood you well, but take a lookt at mechanize:

http://railscasts.com/episodes/191-mechanize

Regards,

Everaldo

3

Mechanize looks good, I'll definitely give it a try.

Where I'm basically searching for is a gem or library that can store a session / cookie, so Rails can login on a website in the background.

Anyone who know if there are other options besides mechanize?

4

Jeroen van Ingen wrote in post #1031839:

Mechanize looks good, I'll definitely give it a try.

Where I'm basically searching for is a gem or library that can store a session / cookie, so Rails can login on a website in the background.

Anyone who know if there are other options besides mechanize?

Hi Jeroen:

It seems you're talking about two separate things: (a) a mechanism to log in and scrape pages (b) a mechanism to run a task in the background

For (a) I use Mechanize for logging in and page scraping -- that's what it's designed for. For (b), I use DelayedJob. I've had no problem running a Mechanize task in the background using DelayedJob.

- ff