[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

how to search a website

Li Chen

9/5/2008 2:11:00 PM

Hi all,

I want send a query to a website(which has a search button), wait for
the query result, then fetch the webpage. I wonder which library is good
for this.


Thanks,

Li
--
Posted via http://www.ruby-....

9 Answers

Michael Steinfeld

9/5/2008 2:46:00 PM

0

check out curb. http://curb.ruby...

mike

On Fri, Sep 5, 2008 at 10:11 AM, Li Chen <chen_li3@yahoo.com> wrote:
> Hi all,
>
> I want send a query to a website(which has a search button), wait for
> the query result, then fetch the webpage. I wonder which library is good
> for this.
>
>
> Thanks,
>
> Li
> --
> Posted via http://www.ruby-....
>
>

James Britt

9/5/2008 3:27:00 PM

0

Li Chen wrote:
> Hi all,
>
> I want send a query to a website(which has a search button), wait for
> the query result, then fetch the webpage. I wonder which library is good
> for this.


Look at Mechanize.


http://mechanize.rubyforge.org/...


--
James Britt

www.happycamperstudios.com - Wicked Cool Coding
www.jamesbritt.com - Playing with Better Toys
www.ruby-doc.org - Ruby Help & Documentation
www.rubystuff.com - The Ruby Store for Ruby Stuff

Li Chen

9/5/2008 5:10:00 PM

0

James Britt wrote:

>
> http://mechanize.rubyforge.org/...
>
>
> --
> James Britt


Hi James,

I download gems for both "mechanize" and "hoe" but still have problems
installing them. I am not sure what is going on.


Thanks,

Li

######################################
Here are the info I get:


C:\ruby\gem_download>gem install mechanize-0.7.8.gem --local
ERROR: While executing gem ... (RuntimeError)
Error instaling mechanize-0.7.8.gem:
mechanize requires hoe >= 1.7.0

C:\ruby\gem_download>gem install hoe-1.7.0.gem --local
ERROR: While executing gem ... (RuntimeError)
Error instaling hoe-1.7.0.gem:
hoe requires rubyforge >= 1.0.0
--
Posted via http://www.ruby-....

brabuhr

9/5/2008 5:37:00 PM

0

On Fri, Sep 5, 2008 at 1:10 PM, Li Chen <chen_li3@yahoo.com> wrote:
> James Britt wrote:
>> http://mechanize.rubyforge.org/...
>
> I download gems for both "mechanize" and "hoe" but still have problems
> installing them. I am not sure what is going on.
>
> ######################################
> Here are the info I get:
>
>
> C:\ruby\gem_download>gem install mechanize-0.7.8.gem --local
> ERROR: While executing gem ... (RuntimeError)
> Error instaling mechanize-0.7.8.gem:
> mechanize requires hoe >= 1.7.0
>
> C:\ruby\gem_download>gem install hoe-1.7.0.gem --local
> ERROR: While executing gem ... (RuntimeError)
> Error instaling hoe-1.7.0.gem:
> hoe requires rubyforge >= 1.0.0

If you are downloading the gems to install locally, you also need to
download the rubyforge gem. I'm not sure off-hand what the official
download location is for it. Did you/can you try:

gem install mechanize

to automatically download the dependencies for you?

brabuhr

9/5/2008 5:41:00 PM

0

On Fri, Sep 5, 2008 at 1:42 PM, <brabuhr@gmail.com> wrote:
> On Fri, Sep 5, 2008 at 1:10 PM, Li Chen <chen_li3@yahoo.com> wrote:
>> James Britt wrote:
>>> http://mechanize.rubyforge.org/...
>>
>> I download gems for both "mechanize" and "hoe" but still have problems
>> installing them. I am not sure what is going on.
>>
>> Error instaling mechanize-0.7.8.gem:
>> mechanize requires hoe >= 1.7.0
>> Error instaling hoe-1.7.0.gem:
>> hoe requires rubyforge >= 1.0.0
>
> If you are downloading the gems to install locally, you also need to
> download the rubyforge gem. I'm not sure off-hand what the official
> download location is for it.

http://gems.rubyforge...
http://gems.rubyforge.vm.bytemark.co.uk/gems/rubyforge...

Adam Akhtar

9/5/2008 5:58:00 PM

0

regarding mechanize, ive been tinkering with it and i want to knwo if
after you grab a webpage you can access the page`s html rather than
using the object returned from .get('...')

i wrote a huge script which took me hours and uses regular expressions
to scrape pages off ebay. It took me ages to write good regseps so i
dont want to thorw them away. I however ran into a problem regarding
logging into ebay so turned to mechanize. I basicaly want to login to
ebay using mechanize and then use my own regexps on a grabbed pages
html. Im not sure how to get that html though...

i checked the ruby core section for mechanize but its thin and the
examples dont cover this situtation, they all presume you will use
hrpicot etc.

any one help me on this one?
--
Posted via http://www.ruby-....

Lex Williams

9/5/2008 6:31:00 PM

0

Adam Akhtar wrote:
> regarding mechanize, ive been tinkering with it and i want to knwo if
> after you grab a webpage you can access the page`s html rather than
> using the object returned from .get('...')

you can access mechanize's current page using :

agent.page.body ( or whatever method you want to call on it )

and you wouldn't need to store the current page

> i wrote a huge script which took me hours and uses regular expressions
> to scrape pages off ebay. It took me ages to write good regseps so i
> dont want to thorw them away. I however ran into a problem regarding
> logging into ebay so turned to mechanize. I basicaly want to login to
> ebay using mechanize and then use my own regexps on a grabbed pages
> html. Im not sure how to get that html though...

interesting_stuff = agent.page.body.scan(/#{regex here}/i)

You only need the #{} if you have the regex stored in a variable

> i checked the ruby core section for mechanize but its thin and the
> examples dont cover this situtation, they all presume you will use
> hrpicot etc.
>
> any one help me on this one?

mechanize has an examples and a guide section , which pretty much covers
anything you would need to do .
If you have further questions , ask and I will do my best to answer them

Adam Akhtar

9/7/2008 3:26:00 AM

0

excellent lex that did the trick. I checked the core but that method
isnt listed there under page.

--
Posted via http://www.ruby-....

Lex Williams

9/7/2008 8:25:00 AM

0

Adam Akhtar wrote:
> excellent lex that did the trick. I checked the core but that method
> isnt listed there under page.

which method Adam ?
--
Posted via http://www.ruby-....