Asp Forum - urllib slow on Leopard

jt

2/24/2008 7:57:00 PM

I've recently switched from Ubuntu to OS X Leopard. I have some python
scripts which download info from the web. The scripts were working fine
before on Ubuntu, but urllib seems to work really slowly on Leopard.

I tried an example from the docs:

>>> import urllib2
>>> f = urllib2.urlopen('http://www.python...)
>>> print f.read(100)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml...
>>>

The line f = ... takes about a minute to return - but otherwise returns
the predicted result. My web connection seems fine, so has anyone got
any ideas as to what the problem might be?

4 Answers

André

2/24/2008 9:07:00 PM

On Feb 24, 3:57 pm, mark <m...@privacy.net> wrote:
> I've recently switched from Ubuntu to OS X Leopard. I have some python
> scripts which download info from the web. The scripts were working fine
> before on Ubuntu, but urllib seems to work really slowly on Leopard.
>
> I tried an example from the docs:
>
> >>> import urllib2
> >>> f = urllib2.urlopen('http://www.python...)
> >>> print f.read(100)
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
> "http://www.w3.org/TR/xhtml...
> >>>
>
> The line f = ... takes about a minute to return - but otherwise returns
> the predicted result. My web connection seems fine, so has anyone got
> any ideas as to what the problem might be?

Nope, but it's not Leopard related. Just checked here and it loaded
in about 1 sec.

André

Stefan Behnel

2/24/2008 9:15:00 PM

mark wrote:
> I've recently switched from Ubuntu to OS X Leopard. I have some python
> scripts which download info from the web. The scripts were working fine
> before on Ubuntu, but urllib seems to work really slowly on Leopard.
>
> I tried an example from the docs:
>
>>>> import urllib2
>>>> f = urllib2.urlopen('http://www.python...)
>>>> print f.read(100)
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
> "http://www.w3.org/TR/xhtml...
>>>>
>
> The line f = ... takes about a minute to return - but otherwise returns
> the predicted result. My web connection seems fine, so has anyone got
> any ideas as to what the problem might be?

It likely tries to load the DTD in the background, which requires network access.

http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_d...

Stefan

jt

2/25/2008 9:40:00 PM

mark wrote:
> I've recently switched from Ubuntu to OS X Leopard. I have some python
> scripts which download info from the web. The scripts were working fine
> before on Ubuntu, but urllib seems to work really slowly on Leopard.
>
> I tried an example from the docs:
>
> >>> import urllib2
> >>> f = urllib2.urlopen('http://www.python...)
> >>> print f.read(100)
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
> "http://www.w3.org/TR/xhtml...
> >>>
>
> The line f = ... takes about a minute to return - but otherwise returns
> the predicted result. My web connection seems fine, so has anyone got
> any ideas as to what the problem might be?

I found that by configuring the network manually (specifying static IP
address, etc.) it sped the whole thing up. Go figure.

Paul Boddie

2/26/2008 12:07:00 PM

On 24 Feb, 22:14, Stefan Behnel <stefan...@behnel.de> wrote:
>
> It likely tries to load the DTD in the background, which requires network access.
>
> http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_d...

This is principally concerned with the standard library XML modules,
not urllib/urllib2 specifically. See the bug report in the tracker:

http://bugs.python.org...

I'd imagine that it's more likely to be a DNS problem than anything
else.

Paul

comp.lang.python

urllib slow on Leopard

jt

André

Stefan Behnel

jt

Paul Boddie

x Login to ForumsZone