[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Running windows ruby, and watir under wine on linux?

Ezra Zygmuntowicz

7/11/2006 8:13:00 PM

Friends-

I have seen a few blog posts where folks mention that they have
gotten the ruby one click installer and watir/IE running under wine
on linux. Has anyone out there done this successfully? My company
would be willing to pay someone to help us get this up and running.
Anyone have any tips or any more information about making this kind
of thing happen?

We are using watir as a web scraper tool of sorts for scraping
crusty sites that use multiple framesets and java applets and all
kinds of other horrible stuff that defies any type of regular screen
scraping short of using watir to actually navigate the sites and
enter data into forms to get results we need from these companies
sites. And we need multiple instances of watir driving the same
companies site at once which is problematic because internet explorer
overwrites its cookies when more then one of these scrapes is run
against the same website at once. So we are trying to experiment with
running watir under wine so each scaper runs in its own env and
doesn't step on each others cookies.

Any win32 gurus have any ideas as to how we might be able to
dynamically from our ruby/watir scripts, manage the cookies written
by IE so we could control where they get written and be able to run
two or more concurrent scrapes of the same website via watir without
IE overwriting the cookie it uses? My company is willing to pay
someone to help solve this problem. Thoughts or ideas are very
welcome at this point. I have asked this same thing eslewhere but I
know there are many smart folks on this list and I hope someone might
have an idea to help us out. More info available if you need it.


Thanks-
-Ezra
5 Answers

Sean O'Halpin

7/11/2006 8:22:00 PM

0

On 7/11/06, Ezra Zygmuntowicz <ezmobius@gmail.com> wrote:
> Friends-
>
> I have seen a few blog posts where folks mention that they have
> gotten the ruby one click installer and watir/IE running under wine
> on linux. Has anyone out there done this successfully? My company
> would be willing to pay someone to help us get this up and running.
> Anyone have any tips or any more information about making this kind
> of thing happen?
>
> We are using watir as a web scraper tool of sorts for scraping
> crusty sites that use multiple framesets and java applets and all
> kinds of other horrible stuff that defies any type of regular screen
> scraping short of using watir to actually navigate the sites and
> enter data into forms to get results we need from these companies
> sites. And we need multiple instances of watir driving the same
> companies site at once which is problematic because internet explorer
> overwrites its cookies when more then one of these scrapes is run
> against the same website at once. So we are trying to experiment with
> running watir under wine so each scaper runs in its own env and
> doesn't step on each others cookies.
>
> Any win32 gurus have any ideas as to how we might be able to
> dynamically from our ruby/watir scripts, manage the cookies written
> by IE so we could control where they get written and be able to run
> two or more concurrent scrapes of the same website via watir without
> IE overwriting the cookie it uses? My company is willing to pay
> someone to help solve this problem. Thoughts or ideas are very
> welcome at this point. I have asked this same thing eslewhere but I
> know there are many smart folks on this list and I hope someone might
> have an idea to help us out. More info available if you need it.
>
>
> Thanks-
> -Ezra
>
>

You could try runas
(http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/...)
to run multiple Watir (and hence automated IE) processes as different
users, which would put the cookies in separates profiles.

Wouldn't work if you're trying to run multiple IE processes via
separate Ruby threads though.

Regards,
Sean

Ezra Zygmuntowicz

7/11/2006 8:41:00 PM

0


On Jul 11, 2006, at 1:21 PM, Sean O'Halpin wrote:

> On 7/11/06, Ezra Zygmuntowicz <ezmobius@gmail.com> wrote:
>> Friends-
>>
>> I have seen a few blog posts where folks mention that they
>> have
>> gotten the ruby one click installer and watir/IE running under wine
>> on linux. Has anyone out there done this successfully? My company
>> would be willing to pay someone to help us get this up and running.
>> Anyone have any tips or any more information about making this kind
>> of thing happen?
>>
>> We are using watir as a web scraper tool of sorts for
>> scraping
>> crusty sites that use multiple framesets and java applets and all
>> kinds of other horrible stuff that defies any type of regular screen
>> scraping short of using watir to actually navigate the sites and
>> enter data into forms to get results we need from these companies
>> sites. And we need multiple instances of watir driving the same
>> companies site at once which is problematic because internet explorer
>> overwrites its cookies when more then one of these scrapes is run
>> against the same website at once. So we are trying to experiment with
>> running watir under wine so each scaper runs in its own env and
>> doesn't step on each others cookies.
>>
>> Any win32 gurus have any ideas as to how we might be able to
>> dynamically from our ruby/watir scripts, manage the cookies written
>> by IE so we could control where they get written and be able to run
>> two or more concurrent scrapes of the same website via watir without
>> IE overwriting the cookie it uses? My company is willing to pay
>> someone to help solve this problem. Thoughts or ideas are very
>> welcome at this point. I have asked this same thing eslewhere but I
>> know there are many smart folks on this list and I hope someone might
>> have an idea to help us out. More info available if you need it.
>>
>>
>> Thanks-
>> -Ezra
>>
>>
>
> You could try runas
> (http://www.microsoft.com/resources/documentation/windo...
> proddocs/en-us/runas.mspx)
> to run multiple Watir (and hence automated IE) processes as different
> users, which would put the cookies in separates profiles.
>
> Wouldn't work if you're trying to run multiple IE processes via
> separate Ruby threads though.
>
> Regards,
> Sean


Thanks Sean. We are already using runas I should have mentioned. THe
reason this is failing is because some of these sites have modal
dialogs and the only way to deal with these in watir is via
click_no_wait. What click no wait does is uses system to start
another process so the modal dialog doesn't block the entire ruby
process. THen when the separate process tries to re-attach to the
window there is a security error that won't let a different process
then the one that started ie bind to the window. This all goes away
if we can get a way to dynamically manage the cookies somehow. Or
under wine each instance if wine will have its own environment that
skirts the issue as well. We have been up and down the wall with this
issue for the past week and have come up with no answers yet. I do
appreciate the suggestion though.

Thanks
-Ezra

Andrew Kreps

7/11/2006 10:11:00 PM

0

I've never used watir, but the best way I've found to isolate
environments is using virtualization software such as VMWare, Virtual
PC or the like. It sounds like it might be overkill for you
(especially if you want to run more than, say, 4 instances) but I
thought I'd throw it out there.

On 7/11/06, Ezra Zygmuntowicz <ezmobius@gmail.com> wrote:
>
> On Jul 11, 2006, at 1:21 PM, Sean O'Halpin wrote:
>
> Thanks Sean. We are already using runas I should have mentioned. THe
> reason this is failing is because some of these sites have modal
> dialogs and the only way to deal with these in watir is via
> click_no_wait. What click no wait does is uses system to start
> another process so the modal dialog doesn't block the entire ruby
> process. THen when the separate process tries to re-attach to the
> window there is a security error that won't let a different process
> then the one that started ie bind to the window. This all goes away
> if we can get a way to dynamically manage the cookies somehow. Or
> under wine each instance if wine will have its own environment that
> skirts the issue as well. We have been up and down the wall with this
> issue for the past week and have come up with no answers yet. I do
> appreciate the suggestion though.
>
> Thanks
> -Ezra
>
>
>
>
>

Daniel Berger

7/12/2006 12:44:00 AM

0

Ezra Zygmuntowicz wrote:
>
> On Jul 11, 2006, at 1:21 PM, Sean O'Halpin wrote:
>
>> On 7/11/06, Ezra Zygmuntowicz <ezmobius@gmail.com> wrote:
>>> Friends-
>>>
>>> I have seen a few blog posts where folks mention that they have
>>> gotten the ruby one click installer and watir/IE running under wine
>>> on linux. Has anyone out there done this successfully? My company
>>> would be willing to pay someone to help us get this up and running.
>>> Anyone have any tips or any more information about making this kind
>>> of thing happen?
>>>
>>> We are using watir as a web scraper tool of sorts for scraping
>>> crusty sites that use multiple framesets and java applets and all
>>> kinds of other horrible stuff that defies any type of regular screen
>>> scraping short of using watir to actually navigate the sites and
>>> enter data into forms to get results we need from these companies
>>> sites. And we need multiple instances of watir driving the same
>>> companies site at once which is problematic because internet explorer
>>> overwrites its cookies when more then one of these scrapes is run
>>> against the same website at once. So we are trying to experiment with
>>> running watir under wine so each scaper runs in its own env and
>>> doesn't step on each others cookies.
>>>
>>> Any win32 gurus have any ideas as to how we might be able to
>>> dynamically from our ruby/watir scripts, manage the cookies written
>>> by IE so we could control where they get written and be able to run
>>> two or more concurrent scrapes of the same website via watir without
>>> IE overwriting the cookie it uses? My company is willing to pay
>>> someone to help solve this problem. Thoughts or ideas are very
>>> welcome at this point. I have asked this same thing eslewhere but I
>>> know there are many smart folks on this list and I hope someone might
>>> have an idea to help us out. More info available if you need it.
>>>
>>>
>>> Thanks-
>>> -Ezra
>>>
>>>
>>
>> You could try runas
>> (http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/...)
>>
>> to run multiple Watir (and hence automated IE) processes as different
>> users, which would put the cookies in separates profiles.
>>
>> Wouldn't work if you're trying to run multiple IE processes via
>> separate Ruby threads though.
>>
>> Regards,
>> Sean
>
>
> Thanks Sean. We are already using runas I should have mentioned. THe
> reason this is failing is because some of these sites have modal
> dialogs and the only way to deal with these in watir is via
> click_no_wait. What click no wait does is uses system to start another
> process so the modal dialog doesn't block the entire ruby process.
> THen when the separate process tries to re-attach to the window there
> is a security error that won't let a different process then the one
> that started ie bind to the window. This all goes away if we can get a
> way to dynamically manage the cookies somehow. Or under wine each
> instance if wine will have its own environment that skirts the issue
> as well. We have been up and down the wall with this issue for the
> past week and have come up with no answers yet. I do appreciate the
> suggestion though.
>
> Thanks
> -Ezra
>
>
>
>
>
Rather than using system() you could try using Process.create in the
win32-process package. I'm guessing somewhat here, but give this a shot:

require 'win32/process'
include Win32

def click_no_wait
Process.create(
:app_name => "c:\ruby\bin\ruby ...",
:creation_flags => Process::CREATE_PRESERVE_CODE_AUTHZ_LEVEL
:inherited => true
)
end

You can futz around with the various creation flags and see if any of
them help you out.

HTH,

Dan

Ezra Zygmuntowicz

7/12/2006 4:00:00 PM

0

Hi-

On Jul 11, 2006, at 5:43 PM, Daniel Berger wrote:

> Ezra Zygmuntowicz wrote:
>>
>> On Jul 11, 2006, at 1:21 PM, Sean O'Halpin wrote:
>>
>>> On 7/11/06, Ezra Zygmuntowicz <ezmobius@gmail.com> wrote:
>>>> Friends-
>>>>
>>>> I have seen a few blog posts where folks mention that
>>>> they have
>>>> gotten the ruby one click installer and watir/IE running under
>>>> wine
>>>> on linux. Has anyone out there done this successfully? My company
>>>> would be willing to pay someone to help us get this up and running.
>>>> Anyone have any tips or any more information about making this kind
>>>> of thing happen?
>>>>
>>>> We are using watir as a web scraper tool of sorts for
>>>> scraping
>>>> crusty sites that use multiple framesets and java applets and all
>>>> kinds of other horrible stuff that defies any type of regular
>>>> screen
>>>> scraping short of using watir to actually navigate the sites and
>>>> enter data into forms to get results we need from these companies
>>>> sites. And we need multiple instances of watir driving the same
>>>> companies site at once which is problematic because internet
>>>> explorer
>>>> overwrites its cookies when more then one of these scrapes is run
>>>> against the same website at once. So we are trying to experiment
>>>> with
>>>> running watir under wine so each scaper runs in its own env and
>>>> doesn't step on each others cookies.
>>>>
>>>> Any win32 gurus have any ideas as to how we might be
>>>> able to
>>>> dynamically from our ruby/watir scripts, manage the cookies written
>>>> by IE so we could control where they get written and be able to run
>>>> two or more concurrent scrapes of the same website via watir
>>>> without
>>>> IE overwriting the cookie it uses? My company is willing to pay
>>>> someone to help solve this problem. Thoughts or ideas are very
>>>> welcome at this point. I have asked this same thing eslewhere but I
>>>> know there are many smart folks on this list and I hope someone
>>>> might
>>>> have an idea to help us out. More info available if you need it.
>>>>
>>>>
>>>> Thanks-
>>>> -Ezra
>>>>
>>>>
>>>
>>> You could try runas
>>> (http://www.microsoft.com/resources/documentation/windo...
>>> proddocs/en-us/runas.mspx)
>>> to run multiple Watir (and hence automated IE) processes as
>>> different
>>> users, which would put the cookies in separates profiles.
>>>
>>> Wouldn't work if you're trying to run multiple IE processes via
>>> separate Ruby threads though.
>>>
>>> Regards,
>>> Sean
>>
>>
>> Thanks Sean. We are already using runas I should have mentioned.
>> THe reason this is failing is because some of these sites have
>> modal dialogs and the only way to deal with these in watir is via
>> click_no_wait. What click no wait does is uses system to start
>> another process so the modal dialog doesn't block the entire ruby
>> process. THen when the separate process tries to re-attach to the
>> window there is a security error that won't let a different
>> process then the one that started ie bind to the window. This all
>> goes away if we can get a way to dynamically manage the cookies
>> somehow. Or under wine each instance if wine will have its own
>> environment that skirts the issue as well. We have been up and
>> down the wall with this issue for the past week and have come up
>> with no answers yet. I do appreciate the suggestion though.
>>
>> Thanks
>> -Ezra
>>
>>
>>
>>
>>
> Rather than using system() you could try using Process.create in
> the win32-process package. I'm guessing somewhat here, but give
> this a shot:
>
> require 'win32/process'
> include Win32
>
> def click_no_wait
> Process.create(
> :app_name => "c:\ruby\bin\ruby ...",
> :creation_flags => Process::CREATE_PRESERVE_CODE_AUTHZ_LEVEL
> :inherited => true
> )
> end
>
> You can futz around with the various creation flags and see if any
> of them help you out.
>
> HTH,
>
> Dan
>

Hey Dan-

Thanks a lot. That points us in the right direction. I really
appreciate it.

Thanks
-Ezra