[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

[ANN] WWW::Mechanize alike in Ruby

Michael Neumann

1/18/2005 1:05:00 PM

Hi there,

I just read the Web-testing thread in the latest "Ruby Weekly News" and want
to tell the interested people that I hacked up a little library which I use
in some small projects. Web::Unit didn't work for my purposes, and it was
easier to reinvent the wheel than to hack Web::Unit (of course I first tried
to hack it...).

It depends on narf-lib (>= 0.6.3) more exactly, it's version of htmltools, and
requires Ruby-current (due to limitations in the net/http library of 1.8.2).
If you want to try it out with Ruby 1.8.2, just get the net/*.rb files from
Ruby-current and use these instead (or use local overrides:
$LOAD_PATH.unshift './net-overrides' ... ).

Features:
* Cookies (very preliminary, no expire etc.)
* automatic redirect
* Forms, Links

Be aware that it's just a few-hour hack, to get my project done.

It's still very generic and misses lots of usability methods (I use lots of
"find" and use explicit parameters instead of implicit, e.g. last visited
page).

Here's a simple example, that logs into your rubyforge.org acount:

require 'mechanize'

agent = WWW::Mechanize.new {|a| a.log = Logger.new(STDERR) }
page = agent.get('http://rubyforge...)
link = page.links.find {|l| l.node.text =~ /Log In/ }
page = agent.click(link)
form = page.forms[1]
form.fields.find {|f| f.name == 'form_loginname'}.value = ARGV[0]
form.fields.find {|f| f.name == 'form_pw'}.value = ARGV[1]
page = agent.submit(form, form.buttons.first)
puts page.body

Here are the sources:

http://www.ntecs.de/viewcvs/vi...

I've realized that Ruby has client libraries for www/http (cgi.rb, net/http)
and server libraries for http (webrick), but not a common library that puts
those things together...

Regards,

Michael
4 Answers

Jamis Buck

1/18/2005 2:33:00 PM

0

On 22:04 Tue 18 Jan , Michael Neumann wrote:
> Hi there,
>
> I just read the Web-testing thread in the latest "Ruby Weekly News" and want
> to tell the interested people that I hacked up a little library which I use
> in some small projects. Web::Unit didn't work for my purposes, and it was
> easier to reinvent the wheel than to hack Web::Unit (of course I first tried
> to hack it...).
>
> It depends on narf-lib (>= 0.6.3) more exactly, it's version of htmltools, and
> requires Ruby-current (due to limitations in the net/http library of 1.8.2).

Isn't 1.8.2 the current Ruby? Or does "Ruby-current" mean something
other than what I'm understanding it to mean?

> If you want to try it out with Ruby 1.8.2, just get the net/*.rb files from
> Ruby-current and use these instead (or use local overrides:
> $LOAD_PATH.unshift './net-overrides' ... ).
>
> Features:
> * Cookies (very preliminary, no expire etc.)
> * automatic redirect
> * Forms, Links
>

[snip]

>
> I've realized that Ruby has client libraries for www/http (cgi.rb, net/http)
> and server libraries for http (webrick), but not a common library that puts
> those things together...

Nice job, Michael. I'm looking forward to trying this one out.

- Jamis

--
Jamis Buck
jamis_buck@byu.edu
http://www.jamisbuck...
------------------------------
"I am Victor of Borge. You will be assimil-nine-ed."

Austin Ziegler

1/18/2005 2:53:00 PM

0

On Tue, 18 Jan 2005 23:32:30 +0900, Jamis Buck <jamis_buck@byu.edu> wrote:
> On 22:04 Tue 18 Jan , Michael Neumann wrote:
> > Hi there,
> >
> > I just read the Web-testing thread in the latest "Ruby Weekly News" and want
> > to tell the interested people that I hacked up a little library which I use
> > in some small projects. Web::Unit didn't work for my purposes, and it was
> > easier to reinvent the wheel than to hack Web::Unit (of course I first tried
> > to hack it...).
> >
> > It depends on narf-lib (>= 0.6.3) more exactly, it's version of htmltools, and
> > requires Ruby-current (due to limitations in the net/http library of 1.8.2).
> Isn't 1.8.2 the current Ruby? Or does "Ruby-current" mean something
> other than what I'm understanding it to mean?

If it isn't 1.9 (CVS HEAD), then it's probably a post 1.8.2 release
stable snapshot.

-austin
--
Austin Ziegler * halostatue@gmail.com
* Alternate: austin@halostatue.ca

Michael Neumann

1/18/2005 2:56:00 PM

0

Jamis Buck wrote:
> On 22:04 Tue 18 Jan , Michael Neumann wrote:
>
>>Hi there,
>>
>>I just read the Web-testing thread in the latest "Ruby Weekly News" and want
>>to tell the interested people that I hacked up a little library which I use
>>in some small projects. Web::Unit didn't work for my purposes, and it was
>>easier to reinvent the wheel than to hack Web::Unit (of course I first tried
>>to hack it...).
>>
>>It depends on narf-lib (>= 0.6.3) more exactly, it's version of htmltools, and
>>requires Ruby-current (due to limitations in the net/http library of 1.8.2).
>
>
> Isn't 1.8.2 the current Ruby? Or does "Ruby-current" mean something
> other than what I'm understanding it to mean?

With -current I mean the development version of Ruby. Sorry for the
confusion. Maybe this could get back-ported into the stable-trunk?

>>If you want to try it out with Ruby 1.8.2, just get the net/*.rb files from
>>Ruby-current and use these instead (or use local overrides:
>>$LOAD_PATH.unshift './net-overrides' ... ).
>>
>>Features:
>> * Cookies (very preliminary, no expire etc.)
>> * automatic redirect
>> * Forms, Links
>>
>
>
> [snip]
>
>
>>I've realized that Ruby has client libraries for www/http (cgi.rb, net/http)
>>and server libraries for http (webrick), but not a common library that puts
>>those things together...
>
>
> Nice job, Michael. I'm looking forward to trying this one out.

Well, just download the net/ directory from the development branch of
Ruby, and put it into a net-overrides directory (you only need
net/http.rb, net/https.rb and net/protocol.rb). Then modify mechanize.rb:

# put this at the top of mechanize.rb
$LOAD_PATH.unshift "./net-overrides"

This should then work with Ruby 1.8.2.

Regards,

Michael

Michael Neumann

1/18/2005 2:59:00 PM

0

Austin Ziegler wrote:
> On Tue, 18 Jan 2005 23:32:30 +0900, Jamis Buck <jamis_buck@byu.edu> wrote:
>
>>On 22:04 Tue 18 Jan , Michael Neumann wrote:
>>
>>>Hi there,
>>>
>>>I just read the Web-testing thread in the latest "Ruby Weekly News" and want
>>>to tell the interested people that I hacked up a little library which I use
>>>in some small projects. Web::Unit didn't work for my purposes, and it was
>>>easier to reinvent the wheel than to hack Web::Unit (of course I first tried
>>>to hack it...).
>>>
>>>It depends on narf-lib (>= 0.6.3) more exactly, it's version of htmltools, and
>>>requires Ruby-current (due to limitations in the net/http library of 1.8.2).
>>
>>Isn't 1.8.2 the current Ruby? Or does "Ruby-current" mean something
>>other than what I'm understanding it to mean?
>
>
> If it isn't 1.9 (CVS HEAD), then it's probably a post 1.8.2 release
> stable snapshot.

I'm not sure whether it's in the stable branch. Just look for the
get_field method, which must be present (net/http*).

Regards,

Michael