Asp Forum
Home
|
Login
|
Register
|
Search
Forums
>
comp.lang.ruby
Re: page reader
Dan Zwell
7/26/2007 5:48:00 AM
Thilanka Marasinghe wrote:
> I was wondering if theres anything out there that can take web page full of
> text and search it for particular phrases I give it.. say for example I
> have A web page or several linked webpages.. I could say find "how to
> initiate a class".. or something like that.. This might seem likea silly
> question. also if I were to attempt something like that Would I have to use
> ruby rails?..
>
>
> thanks thilankka
>
Well, there are two parts to that question. One is how to get the body
of a webpage in a string,
require 'net/http'
str = Net::HTTP.get URI.parse('
http://www.ruby-lang.or...
)
and the second is how to search within that string.
str.scan(/.*ruby.*/) do |match|
puts "This line contained the word \"ruby\" in lower case:"
puts match
end
This type of program is called a "screen scraper", and there are some
tools/libraries that aid in their creation, if you want something
fancier than what I outlined above. WWW::Mechanize is the classic
(ported from Perl, I think), but the one I hear the most about is
Hpricot (based on Mechanize). These support things like using cookies
and automatically following redirects, and probably even the ability to
search a page and strip the HTML out of the result (though I may be
misremembering--I have only read about these programs). If you still
want to write your own stuff but get annoyed at the clunkiness and
obscure errors that Net::HTTP can be, have a look at http-access2 here:
http://raa.ruby-lang.org/project/htt...
.
There are plenty of other screen scraping libraries that I didn't
mention. Just use google. Your real problem is how to actually do the
searching. For that, you will need to read up on regular expressions.
One simple test might be
matches = 0
[/how/, /build/, /initiate|instantiate/, /class/].each do |regex|
matches += 1 if str =~ regex
end
if matches > 2
do something with str
end
The point is, you will need to be creative and research and probably
also ask for help. (The above does something with the string if it
matches 3 or 4 of the regular expressions in the list.)
Good luck,
Dan
Servizio di avviso nuovi messaggi
Ricevi direttamente nella tua mail i nuovi messaggi per
Re: page reader
Inserendo la tua e-mail nella casella sotto, riceverai un avviso tramite posta elettronica ogni volta che il motore di ricerca troverà un nuovo messaggio per te
Il servizio è completamente GRATUITO!
x
Login to ForumsZone
Login with Google
Login with E-Mail & Password