Asp Forum
Home
|
Login
|
Register
|
Search
Forums
>
comp.lang.ruby
Seeking for a ruby spider robot example
yuesefa
8/23/2006 5:09:00 PM
Hi, i want to write a little spider to do some web search
but have no idea how to start it. is there any example or something like that?
thanks :-)
yuesefa
3 Answers
Justin Collins
8/23/2006 5:37:00 PM
0
Haofei wrote:
> Hi, i want to write a little spider to do some web search
> but have no idea how to start it. is there any example or something
> like that?
> thanks :-)
>
> yuesefa
You can see an extremely simple/limited one that I made a while back:
http://students.seattleu.edu/collinsj/programs_netcr...
It may give you a place to start, but there are very good libraries for
getting and parsing websites, like Rubyful Soup, Mechanize, open-uri,
and so on.
Also, try searching the archives for more.
-Justin
Aaron Patterson
8/23/2006 11:27:00 PM
0
On Thu, Aug 24, 2006 at 02:09:28AM +0900, Haofei wrote:
> Hi, i want to write a little spider to do some web search
> but have no idea how to start it. is there any example or something like
> that?
> thanks :-)
>
> yuesefa
You can write one with WWW::Mechanize. I have an example on my blog:
http://tenderlovemaking.com/2006/05/26/mechanize-o...
There is also an example spider that comes along with Mechanize, just
look in the 'eg' directory.
Here is the spider for those that don't want to click (its not perfect,
but its small!):
(mech = WWW::Mechanize.new).get(ARGV[0])
(a = lambda { |p|
mech.page.links.each { |l|
mech.click(l) && p.call(p) if ! mech.visited? l
}
}).call(a)
--Aaron
Chris Lowis
8/25/2006 10:44:00 AM
0
I found hpricot easy to use :
http://code.whytheluckystiff.ne...
There's some good code examples on the site too.
Chris
--
Posted via
http://www.ruby-...
.
Servizio di avviso nuovi messaggi
Ricevi direttamente nella tua mail i nuovi messaggi per
Seeking for a ruby spider robot example
Inserendo la tua e-mail nella casella sotto, riceverai un avviso tramite posta elettronica ogni volta che il motore di ricerca troverà un nuovo messaggio per te
Il servizio è completamente GRATUITO!
x
Login to ForumsZone
Login with Google
Login with E-Mail & Password