[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Net::HTTP.get_response hangs with particular url

Erik

4/3/2007 7:59:00 PM

This seems bizar, the first get_response() to apple.com goes fine, the
2nd hangs indefinitely. It hangs on two different machines in different
network segments.

require 'net/http'
require 'uri'

response = Net::HTTP.get_response( URI.parse('http://www.appl...) )
puts response.class # Net::HTTPOK
puts response.code # 200

response = Net::HTTP.get_response(
URI.parse('http://content.digitalwell.washington.edu/isilon/1/8/45/459ffc6f-3f91-46f5-af58-155791dad3b...)
)
puts response.class
puts response.code

(I have no affiliation with the hanging url, it's just a url to I came
across in my server logs while processing RSS feed enclosures).

Pressing ctrl-c gives:

/opt/ruby-1.8.4/lib/ruby/1.8/net/protocol.rb:133:in `sysread': Interrupt
from /opt/ruby-1.8.4/lib/ruby/1.8/net/protocol.rb:133:in
`rbuf_fill'
from /opt/ruby-1.8.4/lib/ruby/1.8/timeout.rb:56:in `timeout'
from /opt/ruby-1.8.4/lib/ruby/1.8/timeout.rb:76:in `timeout'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/protocol.rb:132:in
`rbuf_fill'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/protocol.rb:86:in `read'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:2180:in
`read_body_0'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:2141:in
`read_body'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:2166:in `body'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:2105:in
`reading_body'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:1048:in `request'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:944:in
`request_get'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:380:in
`get_response'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:545:in `start'
from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:379:in
`get_response'
from test.rb:8

Anyone?

--
Posted via http://www.ruby-....
2 Answers

Dan Zwell

4/3/2007 8:28:00 PM

0

I'm not positive, but I would point out that the URL in question is an
82MB mp3. I was under the impression that getting a HTTPResponse
actually downloaded the body of the requested page/file. So perhaps the
response is just taking a long time in arriving? My connection would
need more than 10 minutes for this to return (though I admit my error is
different when I press control-c)

Dan

Erik wrote:
> This seems bizar, the first get_response() to apple.com goes fine, the
> 2nd hangs indefinitely. It hangs on two different machines in different
> network segments.
>
> require 'net/http'
> require 'uri'
>
> response = Net::HTTP.get_response( URI.parse('http://www.appl...) )
> puts response.class # Net::HTTPOK
> puts response.code # 200
>
> response = Net::HTTP.get_response(
> URI.parse('http://content.digitalwell.washington.edu/isilon/1/8/45/459ffc6f-3f91-46f5-af58-155791dad3b...)
> )
> puts response.class
> puts response.code
>
> (I have no affiliation with the hanging url, it's just a url to I came
> across in my server logs while processing RSS feed enclosures).
>
> Pressing ctrl-c gives:
>
> /opt/ruby-1.8.4/lib/ruby/1.8/net/protocol.rb:133:in `sysread': Interrupt
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/protocol.rb:133:in
> `rbuf_fill'
> from /opt/ruby-1.8.4/lib/ruby/1.8/timeout.rb:56:in `timeout'
> from /opt/ruby-1.8.4/lib/ruby/1.8/timeout.rb:76:in `timeout'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/protocol.rb:132:in
> `rbuf_fill'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/protocol.rb:86:in `read'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:2180:in
> `read_body_0'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:2141:in
> `read_body'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:2166:in `body'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:2105:in
> `reading_body'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:1048:in `request'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:944:in
> `request_get'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:380:in
> `get_response'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:545:in `start'
> from /opt/ruby-1.8.4/lib/ruby/1.8/net/http.rb:379:in
> `get_response'
> from test.rb:8
>
> Anyone?
>

Erik

4/4/2007 6:34:00 AM

0

Dan Zwell wrote:
> I'm not positive, but I would point out that the URL in question is an
> 82MB mp3. I was under the impression that getting a HTTPResponse
> actually downloaded the body of the requested page/file. So perhaps the
> response is just taking a long time in arriving? My connection would
> need more than 10 minutes for this to return (though I admit my error is
> different when I press control-c)
>
> Dan

Hi Dan,

My connection to this file seems much worse than yours, I just tried it
with curl and it took 50 minutes! That's why it took so incredibly long!

By the way, I came across this issue while looking for a clean and
simple way to (1) catch redirects and (2) test whether an object is
available for download on a certain location.

I hereby post a snippet who does just that, maybe it helps other folks
who are looking for this solution and stumble on long waits for
get_response() to finish:

require 'open-uri'
require 'timeout'

def test( _url )
begin
content_length = nil
bytes_read = nil
Timeout::timeout( 10 ) do | length |
open( _url, "User-Agent" => "test",
:content_length_proc => lambda { |cl| content_length = cl },
:progress_proc => lambda { |br|
bytes_read = br
raise HadEnoughException if bytes_read > 0 } )
end
rescue HadEnoughException
puts "HadEnoughException"
rescue Timeout::Error
puts "Timed out on [#{_url}]"
rescue OpenURI::HTTPError
puts "Cannot open [#{_url}]"
rescue Exception => e
puts "Exception #{e.message}"
end
return content_length, bytes_read
end

class HadEnoughException < Exception
end

content_length, bytes_read = test('http://appl...)
puts content_length, bytes_read
# outputs HadEnoughException 31805 784 (values will be different on your
computer)

content_length, bytes_read =
test('http://content.digitalwell.washington.edu/isilon/1/8/45/459ffc6f-3f91-46f5-af58-155791dad3b...)
puts content_length, bytes_read
# outputs HadEnoughException 86136581 769 (values will be different on
your computer)

Note that not all web servers return the content_length in the http
header, so this value may be nil

--
Posted via http://www.ruby-....