Asp Forum - Mechanize and charset issues

John Schmitz

5/26/2009 10:20:00 PM

I'm not sure what is causing this error as I can successfully login, I
just can't submit this form without the script bombing out.

formatstring = "testing submission"

agent = WWW::Mechanize.new
page = agent.get 'hidden'
form = page.forms.first
if !(form.action.eql?('submit.php'))
p "logging in....."
form['username'] = 'hidden'
form['password'] = 'hidden'

page = agent.submit form
page = agent.click(page.link_with(:text => 'Add'))
end

page = agent.click(page.link_with(:text => '[Add Content]'))
uploadForm = page.forms[6]
uploadForm['format'] = formatstring
page = agent.submit uploadForm
#pp page

Gives me the error:

/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/util.rb:40:in
`iconv': "\342\202\254\305\223a condition"... (Iconv::IllegalSequence)
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/util.rb:40:in
`from_native_charset'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:151:in
`from_native_charset'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:143:in
`proc_query'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:142:in
`map'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:142:in
`proc_query'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:165:in
`build_query'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:164:in
`each'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:164:in
`build_query'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:213:in
`request_data'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize.rb:392:in
`post_form'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize.rb:335:in
`submit'
--
Posted via http://www.ruby-....

6 Answers

John Schmitz

5/26/2009 10:35:00 PM

It is caused by the following html â€&oelig in one of the
hidden form entries that is being submitted. I'm not sure how to avoid
this from bombing and still submit the form though?
--
Posted via http://www.ruby-....

The Higgs bozo

5/26/2009 10:50:00 PM

John Schmitz wrote:
>
> page = agent.click(page.link_with(:text => '[Add Content]'))
> uploadForm = page.forms[6]
> uploadForm['format'] = formatstring
> page = agent.submit uploadForm
> #pp page
>
> Gives me the error:
>
> /var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/util.rb:40:in
> `iconv': "\342\202\254\305\223a condition"... (Iconv::IllegalSequence)
> from

Running "ruby -KU ..." will probably fix it (at least it has worked for
me whenever I had errors from \nnn inside strings).
--
Posted via http://www.ruby-....

John Schmitz

5/26/2009 10:53:00 PM

The Higgs bozo wrote:
> John Schmitz wrote:
>>
>> page = agent.click(page.link_with(:text => '[Add Content]'))
>> uploadForm = page.forms[6]
>> uploadForm['format'] = formatstring
>> page = agent.submit uploadForm
>> #pp page
>>
>> Gives me the error:
>>
>> /var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/util.rb:40:in
>> `iconv': "\342\202\254\305\223a condition"... (Iconv::IllegalSequence)
>> from
>
> Running "ruby -KU ..." will probably fix it (at least it has worked for
> me whenever I had errors from \nnn inside strings).

Thank you for the response but it doesn't seem to solve the issue. I
think it's related to charsets and iconv, but I have no idea where to go
from there. I get a near duplicate error message with ruby -KU:

/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/util.rb:40:in
`iconv': "Ã¢Â¬Ã?a condition"... (Iconv::IllegalSequence)
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/util.rb:40:in
`from_native_charset'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:151:in
`from_native_charset'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:143:in
`proc_query'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:142:in
`map'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:142:in
`proc_query'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:165:in
`build_query'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:164:in
`each'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:164:in
`build_query'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize/form.rb:213:in
`request_data'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize.rb:392:in
`post_form'
from
/var/lib/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize.rb:335:in
`submit'
--
Posted via http://www.ruby-....

John Schmitz

5/27/2009 10:20:00 PM

If anyone comes across this problem, this is how I fixed it. Found a
method online and made some minor changes and additions. I just pass the
problem strings through this and it gives me back strings that don't
have issues.

def fix_quotes(c)
c.gsub!(/\342\200(?:\234|\235)/,'"')
c.gsub!(/\342\200(?:\230|\231)/,"'")
c.gsub!(/\342\200\223/,"-")
c.gsub!(/\342\200\246/,"...")
c.gsub!(/\303\242\342\202\254\342\204\242/,"'")
c.gsub!(/\303\242\342\202\254\302\235/,'"')
c.gsub!(/\303\242\342\202\254\305\223/,'"')
c.gsub!(/\303\242\342\202\254"/,'-')
c.gsub!(/\342\202\254\313\234/,'"')
end
--
Posted via http://www.ruby-....

Jarmo Pertman

5/28/2009 8:42:00 AM

Have you tried to set encoding for page something like this:
page.encoding = 'UTF-8'?

Jarmo

John Schmitz wrote:
> If anyone comes across this problem, this is how I fixed it. Found a
> method online and made some minor changes and additions. I just pass the
> problem strings through this and it gives me back strings that don't
> have issues.
>
> def fix_quotes(c)
> c.gsub!(/\342\200(?:\234|\235)/,'"')
> c.gsub!(/\342\200(?:\230|\231)/,"'")
> c.gsub!(/\342\200\223/,"-")
> c.gsub!(/\342\200\246/,"...")
> c.gsub!(/\303\242\342\202\254\342\204\242/,"'")
> c.gsub!(/\303\242\342\202\254\302\235/,'"')
> c.gsub!(/\303\242\342\202\254\305\223/,'"')
> c.gsub!(/\303\242\342\202\254"/,'-')
> c.gsub!(/\342\202\254\313\234/,'"')
> end

--
Posted via http://www.ruby-....

PATRICK

2/5/2013 2:14:00 AM

On Mon, 04 Feb 2013 15:29:21 -0800, Jeanne Douglas
<hlwdjsd2@NOSPAMgmail.com> wrote:

>In article <5q10h8phj7uptljbqs3llm76m1k85g9gu4@6ax.com>,
> SilentOtto@hotmail.com wrote:
>
>> On Mon, 04 Feb 2013 09:07:57 -0500, Patrick <pbarker001@woh.rr.com>
>> wrote:
>>
>> >On Sun, 03 Feb 2013 16:19:54 -0500, SilentOtto@hotmail.com wrote:
>> >
>> >>On Sun, 03 Feb 2013 14:06:34 -0500, Patrick <pbarker001@woh.rr.com>
>> >>wrote:
>> >>
>> >>>
>> >>>On Sun, 03 Feb 2013 03:27:21 -0500, SilentOtto@hotmail.com wrote:
>> >>>
>> >>>>On Sat, 02 Feb 2013 13:50:31 -0500, Patrick <pbarker001@woh.rr.com>
>> >>>>wrote:
>> >>>>
>> >>>>> SilentOtto@hotmail.com wrote:
>> >>>>> Patrick <pbarker001@woh.rr.com>>wrote:
>> >>>>>
>> >>>>>>>Why hasn't the Cardinal been charged with a crime?
>> >>>>>>
>> >>>>>>Because when the crime was committed, someone needed to file a
>> >>>>>>criminal complaint for criminal action to be taken.
>> >>>>>>Nobody has filed a criminal complaint.
>> >>>>>
>> >>>>>So, why do you think a crime ever took place?
>> >>>>
>> >>>>Because the documents show that a crime was being covered up.
>> >>>
>> >>>What documents?
>> >>
>> >>The documents mentioned in the article.
>> >>
>> >>>Cough them up.
>> >>
>> >>Read the article.
>> >
>> >Where is this document?
>>
>> It's not "document", rightard.
>>
>> It's "documents".
>
>12,000 of them.
>
>
>> The story is being covered heavily by the media.
>>
>> Your rightarded debate tactic of pretending that I'm making this up us
>> so childish that the only response you're going to get from me is
>> rolling eyes.
>>
>> -rolls eyes-
>
>Yeah, denial is strong in this one.

I'm waiting for a referene.....

comp.lang.ruby

Mechanize and charset issues

John Schmitz

John Schmitz

The Higgs bozo

John Schmitz

John Schmitz

Jarmo Pertman

PATRICK

x Login to ForumsZone