WGETting from within rails

rubyonrails-talk
1

i'm writing an app that wants to fetch a URL from a remote server and return the html text file to the app. i've been told there are some xml parsers for perl that have request functionality built right in. is there such a thing for Ruby or if nothing else the possibility of using WGET from within a Ruby program.

thanks in advance for any help.

2

you probably want #read in open-uri (or Net::HTTP) http://www.ruby-doc.org/stdlib/libdoc/open-uri/rdoc/index.html

or you could just backtick: `curl http::/yourURLhere`

3

plewizard,

  > i'm writing an app that wants to fetch a URL from a remote server and   > return the html text file to the app. i've been told there are some

If you just want to fetch the page text :

  require 'open-uri';   html = open("http://yahoo.com").readlines

  puts html

If you need to extract part(s) of the page, use Hpricot

    require 'rubygems';     require 'open-uri'     require 'hpricot'

    doc = Hpricot(open("http://yahoo.com"))

    title = doc.search('title').text     body = doc.search('body' ).text     links = doc .search("a").collect{|a| a['href'] }

    puts title     puts body     puts links.to_yaml

Hpricot also lets you modify the page contents : f.ex,     doc.search('a').each{|link| link.swap(link.inner_html)}

turns all the      <a href="....">something</a> into      something

Alain Ravet