[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

Can't Access ANY url from python (errno 61

galileo228

2/19/2010 9:49:00 PM

Hey all,

I've been working on a program that accesses my school's password
protected website and downloads directory names. I'm using mechanize.

Recently, the program has been unable to open the website, returning
the 'errno 61 connection refused' error. I presume the school's server
was blocking me because of many automated logins.

However, it turns out that I cannot now open ANY url from within
Python on my computer using mechanize (or urllib for that matter).
And I've tried in several places -- my place, a friend's place (who
also has comcast as an ISP) and the school -- but no dice, constant
errno 61's whenever I try to open a url.

The strangest thing about this is that firefox still works flawlessly
on any site.

However, at my friend's place, I was able to open url's from HIS
computer, no problem.

Can anyone think of anything I can do to solve this problem? Why would
firefox be working fine but not Python?

Any suggestions would be greatly appreciated.

Matt
14 Answers

Jonathan Gardner

2/19/2010 10:00:00 PM

0

On Fri, Feb 19, 2010 at 1:48 PM, MattB <mattbarkan@gmail.com> wrote:
>
> I've been working on a program that accesses my school's password
> protected website and downloads directory names. I'm using mechanize.
>
> Recently, the program has been unable to open the website, returning
> the 'errno 61 connection refused' error. I presume the school's server
> was blocking me because of many automated logins.
>
> However, it turns out that I cannot now open ANY url from within
> Python on my computer using mechanize (or urllib for that matter).
> And I've tried in several places -- my place, a friend's place (who
> also has comcast as an ISP) and the school -- but no dice, constant
> errno 61's whenever I try to open a url.
>
> The strangest thing about this is that firefox still works flawlessly
> on any site.
>
> However, at my friend's place, I was able to open url's from HIS
> computer, no problem.
>
> Can anyone think of anything I can do to solve this problem? Why would
> firefox be working fine but not Python?
>
> Any suggestions would be greatly appreciated.
>

Are you running behind a firewall? See if Firefox is configured with a
proxy. You'll have to use that to talk to any website.

--
Jonathan Gardner
jgardner@jonathangardner.net

Chris Rebert

2/19/2010 10:12:00 PM

0

On Fri, Feb 19, 2010 at 1:48 PM, MattB <mattbarkan@gmail.com> wrote:
> Hey all,
>
> I've been working on a program that accesses my school's password
> protected website and downloads directory names. I'm using mechanize.
>
> Recently, the program has been unable to open the website, returning
> the 'errno 61 connection refused' error. I presume the school's server
> was blocking me because of many automated logins.
>
> However, it turns out that I cannot now open ANY url from within
> Python on my computer using mechanize (or urllib for that matter).
> And I've tried in several places -- my place, a friend's place (who
> also has comcast as an ISP) and the school -- but no dice, constant
> errno 61's whenever I try to open a url.
>
> The strangest thing about this is that firefox still works flawlessly
> on any site.
<snip>
> Can anyone think of anything I can do to solve this problem? Why would
> firefox be working fine but not Python?
>
> Any suggestions would be greatly appreciated.

Based on what you've said, it's possible the school may have blocked
mechanize's User-Agent somehow.
Try spoofing mechanize as Firefox by setting the string for the
User-Agent header to that of Firefox.
See the section "Changing the automatically-added headers
(User-Agent)" on http://wwwsearch.sourceforge.net/mechaniz...
for how to do that.
To see what Firefox's User-Agent is, use http://whatsmyuser...

Cheers,
Chris
--
Disclaimer: This is not meant to in any way endorse violating AUPs.
http://blog.re...

Martin P. Hellwig

2/19/2010 11:03:00 PM

0

On 02/19/10 21:48, MattB wrote:
> Hey all,
>
> I've been working on a program that accesses my school's password
> protected website and downloads directory names. I'm using mechanize.
>
> Recently, the program has been unable to open the website, returning
> the 'errno 61 connection refused' error. I presume the school's server
> was blocking me because of many automated logins.

Being a former school BOFH, I can assure you that if I was annoyed by
your 'misuse' I would have tracked you down and made you aware of it.

>
> However, it turns out that I cannot now open ANY url from within
> Python on my computer using mechanize (or urllib for that matter).
> And I've tried in several places -- my place, a friend's place (who
> also has comcast as an ISP) and the school -- but no dice, constant
> errno 61's whenever I try to open a url.

As mentioned by Jonathan Gardener, this is most likely a proxy gateway.

>
> The strangest thing about this is that firefox still works flawlessly
> on any site.

Your system might have been centrally configure so that applications are
aware of the proxy, firefox probably has been piggybacking on those
settings (as it should). Most platforms can be made aware of a proxy by
a DHCP option send by the DHCP server (that is when you automatically
get an IP address).

> Any suggestions would be greatly appreciated.
>
> Matt

Google a bit around how you can figure out (from inside your script)
whether your used platform has a proxy configured and how to use it with
your application.

Good luck!

--
mph

galileo228

2/20/2010 12:21:00 AM

0

On Feb 19, 6:02 pm, "Martin P. Hellwig" <martin.hell...@dcuktec.org>
wrote:
> On 02/19/10 21:48, MattB wrote:
>
> > Hey all,
>
> > I've been working on a program that accesses my school's password
> > protected website and downloads directory names. I'm using mechanize.
>
> > Recently, the program has been unable to open the website, returning
> > the 'errno 61 connection refused' error. I presume the school's server
> > was blocking me because of many automated logins.
>
> Being a former school BOFH, I can assure you that if I was annoyed by
> your 'misuse' I would have tracked you down and made you aware of it.
>
>
>
> > However, it turns out that I cannot now open ANY url from within
> > Python on my computer using mechanize (or urllib for that matter).
> > And I've tried in several places -- my place, a friend's place (who
> > also has comcast as an ISP) and the school -- but no dice, constant
> > errno 61's whenever I try to open a url.
>
> As mentioned by Jonathan Gardener, this is most likely a proxy gateway.
>
>
>
> > The strangest thing about this is that firefox still works flawlessly
> > on any site.
>
> Your system might have been centrally configure so that applications are
> aware of the proxy, firefox probably has been piggybacking on those
> settings (as it should). Most platforms can be made aware of a proxy by
> a DHCP option send by the DHCP server (that is when you automatically
> get an IP address).
>
> > Any suggestions would be greatly appreciated.
>
> > Matt
>
> Google a bit around how you can figure out (from inside your script)
> whether your used platform has a proxy configured and how to use it with
> your application.
>
> Good luck!
>
> --
> mph

Hey all,

I've used httpfox to identify the precise headers being sent by
firefox, and then added them to my program using br.addheaders(), as
per the proper mechanize syntax. No dice. (In fact, these headers were
in the program when I ran it successfully from my friend's computer at
his apartment). So I'm pretty sure it's not a header issue.

I'll check and see whether firefox and my system are using a proxy.

Also, based on Martin's comment, I just wanted to make you all aware
that I intend no misuse, but rather am just trying to learn, as I'm a
programming noob. I am not doing anything that I can't do myself from
firefox (ie, I have an account at the school, and am allowed to sign
on with my name and password and look up information in the student
directory). If I do it for more than one student, it just becomes
repetitive, so I thought this was a first modest goal in learning to
do some programming.)

That said, I'm happy to discontinue the attempts, but I'd like to know
how it is that my computer (unless using firefox) is completely
blocked from opening urls from within python. (And how to fix it).

Thanks for the continued help.

galileo228

2/20/2010 1:07:00 AM

0

On Feb 19, 7:20 pm, MattB <mattbar...@gmail.com> wrote:
> On Feb 19, 6:02 pm, "Martin P. Hellwig" <martin.hell...@dcuktec.org>
> wrote:
>
>
>
> > On 02/19/10 21:48, MattB wrote:
>
> > > Hey all,
>
> > > I've been working on a program that accesses my school's password
> > > protected website and downloads directory names. I'm using mechanize.
>
> > > Recently, the program has been unable to open the website, returning
> > > the 'errno 61 connection refused' error. I presume the school's server
> > > was blocking me because of many automated logins.
>
> > Being a former school BOFH, I can assure you that if I was annoyed by
> > your 'misuse' I would have tracked you down and made you aware of it.
>
> > > However, it turns out that I cannot now open ANY url from within
> > > Python on my computer using mechanize (or urllib for that matter).
> > > And I've tried in several places -- my place, a friend's place (who
> > > also has comcast as an ISP) and the school -- but no dice, constant
> > > errno 61's whenever I try to open a url.
>
> > As mentioned by Jonathan Gardener, this is most likely a proxy gateway.
>
> > > The strangest thing about this is that firefox still works flawlessly
> > > on any site.
>
> > Your system might have been centrally configure so that applications are
> > aware of the proxy, firefox probably has been piggybacking on those
> > settings (as it should). Most platforms can be made aware of a proxy by
> > a DHCP option send by the DHCP server (that is when you automatically
> > get an IP address).
>
> > > Any suggestions would be greatly appreciated.
>
> > > Matt
>
> > Google a bit around how you can figure out (from inside your script)
> > whether your used platform has a proxy configured and how to use it with
> > your application.
>
> > Good luck!
>
> > --
> > mph
>
> Hey all,
>
> I've used httpfox to identify the precise headers being sent by
> firefox, and then added them to my program using br.addheaders(), as
> per the proper mechanize syntax. No dice. (In fact, these headers were
> in the program when I ran it successfully from my friend's computer at
> his apartment). So I'm pretty sure it's not a header issue.
>
> I'll check and see whether firefox and my system are using a proxy.
>
> Also, based on Martin's comment, I just wanted to make you all aware
> that I intend no misuse, but rather am just trying to learn, as I'm a
> programming noob. I am not doing anything that I can't do myself from
> firefox (ie, I have an account at the school, and am allowed to sign
> on with my name and password and look up information in the student
> directory). If I do it for more than one student, it just becomes
> repetitive, so I thought this was a first modest goal in learning to
> do some programming.)
>
> That said, I'm happy to discontinue the attempts, but I'd like to know
> how it is that my computer (unless using firefox) is completely
> blocked from opening urls from within python. (And how to fix it).
>
> Thanks for the continued help.

Breakthrough:

I tried switching from a wireless connection to my router, and instead
used an ethernet connection -- and now everything works.

Why would this make a difference? MAC address? Is it possible for an
external server to see my MAC address and block it? Clearly wasn't an
IP address issue!

Chris Rebert

2/20/2010 1:28:00 AM

0

On Fri, Feb 19, 2010 at 5:06 PM, MattB <mattbarkan@gmail.com> wrote:
> On Feb 19, 7:20 pm, MattB <mattbar...@gmail.com> wrote:
>> On Feb 19, 6:02 pm, "Martin P. Hellwig" <martin.hell...@dcuktec.org>
>> wrote:
>> > On 02/19/10 21:48, MattB wrote:
>> > > Hey all,
>>
>> > > I've been working on a program that accesses my school's password
>> > > protected website and downloads directory names. I'm using mechanize.
>>
>> > > Recently, the program has been unable to open the website, returning
>> > > the 'errno 61 connection refused' error. I presume the school's server
>> > > was blocking me because of many automated logins.
>>
>> > Being a former school BOFH, I can assure you that if I was annoyed by
>> > your 'misuse' I would have tracked you down and made you aware of it.
>>
>> > > However, it turns out that I cannot now open ANY url from within
>> > > Python on my computer using mechanize (or urllib for that matter).
>> > > And I've tried in several places -- my place, a friend's place (who
>> > > also has comcast as an ISP) and the school -- but no dice, constant
>> > > errno 61's whenever I try to open a url.
>>
>> > As mentioned by Jonathan Gardener, this is most likely a proxy gateway.
>>
>> > > The strangest thing about this is that firefox still works flawlessly
>> > > on any site.
>>
>> > Your system might have been centrally configure so that applications are
>> > aware of the proxy, firefox probably has been piggybacking on those
>> > settings (as it should). Most platforms can be made aware of a proxy by
>> > a DHCP option send by the DHCP server (that is when you automatically
>> > get an IP address).
>>
>> > > Any suggestions would be greatly appreciated.
<snip>
> Breakthrough:
>
> I tried switching from a wireless connection to my router, and instead
> used an ethernet connection -- and now everything works.
>
> Why would this make a difference? MAC address? Is it possible for an
> external server to see my MAC address and block it? Clearly wasn't an
> IP address issue!

If you're using the campus network and depending on the exact network
details, yes, they very likely can know your MAC address and thus
block it.
Since your Wi-Fi card and Ethernet card have different hardware MAC
addresses, yes, switching would change your visible MAC address, thus
circumventing any blocks based on it.

Cheers,
Chris
--
Hi ACMS!
http://blog.re...

galileo228

2/20/2010 2:33:00 AM

0

On Feb 19, 8:28 pm, Chris Rebert <c...@rebertia.com> wrote:
> On Fri, Feb 19, 2010 at 5:06 PM, MattB <mattbar...@gmail.com> wrote:
> > On Feb 19, 7:20 pm, MattB <mattbar...@gmail.com> wrote:
> >> On Feb 19, 6:02 pm, "Martin P. Hellwig" <martin.hell...@dcuktec.org>
> >> wrote:
> >> > On 02/19/10 21:48, MattB wrote:
> >> > > Hey all,
>
> >> > > I've been working on a program that accesses my school's password
> >> > > protected website and downloads directory names. I'm using mechanize.
>
> >> > > Recently, the program has been unable to open the website, returning
> >> > > the 'errno 61 connection refused' error. I presume the school's server
> >> > > was blocking me because of many automated logins.
>
> >> > Being a former school BOFH, I can assure you that if I was annoyed by
> >> > your 'misuse' I would have tracked you down and made you aware of it.
>
> >> > > However, it turns out that I cannot now open ANY url from within
> >> > > Python on my computer using mechanize (or urllib for that matter).
> >> > > And I've tried in several places -- my place, a friend's place (who
> >> > > also has comcast as an ISP) and the school -- but no dice, constant
> >> > > errno 61's whenever I try to open a url.
>
> >> > As mentioned by Jonathan Gardener, this is most likely a proxy gateway.
>
> >> > > The strangest thing about this is that firefox still works flawlessly
> >> > > on any site.
>
> >> > Your system might have been centrally configure so that applications are
> >> > aware of the proxy, firefox probably has been piggybacking on those
> >> > settings (as it should). Most platforms can be made aware of a proxy by
> >> > a DHCP option send by the DHCP server (that is when you automatically
> >> > get an IP address).
>
> >> > > Any suggestions would be greatly appreciated.
> <snip>
> > Breakthrough:
>
> > I tried switching from a wireless connection to my router, and instead
> > used an ethernet connection -- and now everything works.
>
> > Why would this make a difference? MAC address? Is it possible for an
> > external server to see my MAC address and block it? Clearly wasn't an
> > IP address issue!
>
> If you're using the campus network and depending on the exact network
> details, yes, they very likely can know your MAC address and thus
> block it.
> Since your Wi-Fi card and Ethernet card have different hardware MAC
> addresses, yes, switching would change your visible MAC address, thus
> circumventing any blocks based on it.
>
> Cheers,
> Chris
> --
> Hi ACMS!http://blog.re...

Chris,

I'm using the network in my own apartment. Not the campus's.
Moreover, my mac's MAC address is different from the MAC address shown
by my router, but as I said I'm also blocked when using my friend's
wireless router at his apartment.

So it must be my mac's MAC, and not the router's MAC, that's being
blocked, right?

But ALSO -- is it my ISP that's blocking the mac's MAC (and not the
school), since I can't raise ANY url's from python when I'm on
wireless?

Lie Ryan

2/20/2010 7:03:00 AM

0

On 02/20/10 13:32, MattB wrote:
>
> I'm using the network in my own apartment. Not the campus's.
> Moreover, my mac's MAC address is different from the MAC address shown
> by my router, but as I said I'm also blocked when using my friend's
> wireless router at his apartment.
>
> So it must be my mac's MAC, and not the router's MAC, that's being
> blocked, right?
>
> But ALSO -- is it my ISP that's blocking the mac's MAC (and not the
> school), since I can't raise ANY url's from python when I'm on
> wireless?

MAC or IP blocking can't be the reason, as the OP stated, he can use
Firefox just fine.

Can you access, say, http://www.... from urllib or mechanize?

If you can't access *any website* using urllib/mechanize but you can
with a browser and you're on a unix-based machine, you probably have the
same problem as I used to have. Check whether you used the same hostname
in /etc/conf.d/hostname and /etc/hosts (or wherever your distro saves
its hostname configurations, I use Gentoo); after editing those files
reboot (there are ways to avoid reboot, but rebooting guarantees the
conf file is reread).

Check the hostname by running this python script:

import socket
hn = socket.gethostname()
print hn
print socket.gethostbyname(hn) # should be 127.0.0.1

galileo228

2/20/2010 8:36:00 AM

0

On Feb 20, 2:02 am, Lie Ryan <lie.1...@gmail.com> wrote:
> On 02/20/10 13:32, MattB wrote:
>
>
>
> > I'm using the network in my own apartment. Not the campus's.
> > Moreover, my mac's MAC address is different from the MAC address shown
> > by my router, but as I said I'm also blocked when using my friend's
> > wireless router at his apartment.
>
> > So it must be my mac's MAC, and not the router's MAC, that's being
> > blocked, right?
>
> > But ALSO -- is it my ISP that's blocking the mac's MAC (and not the
> > school), since I can't raise ANY url's from python when I'm on
> > wireless?
>
> MAC or IP blocking can't be the reason, as the OP stated, he can use
> Firefox just fine.
>
> Can you access, say,http://www.goog... urllib or mechanize?
>
> If you can't access *any website* using urllib/mechanize but you can
> with a browser and you're on a unix-based machine, you probably have the
> same problem as I used to have. Check whether you used the same hostname
> in /etc/conf.d/hostname and /etc/hosts (or wherever your distro saves
> its hostname configurations, I use Gentoo); after editing those files
> reboot (there are ways to avoid reboot, but rebooting guarantees the
> conf file is reread).
>
> Check the hostname by running this python script:
>
> import socket
> hn = socket.gethostname()
> print hn
> print socket.gethostbyname(hn) # should be 127.0.0.1

Lie,

Wow. Strangely, that script returned 192.168.1.106. However, in Snow
Leopard's airport settings, if I click on 'advanced' and then
'proxies', the default proxy for 'http' is 127.0.0.1:4444 (and in
these settings, the 'use proxy for http' is checked).

I just tried checking the unix files you mentioned. In etc/hosts, the
following info is displayed:

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting. Do not change this entry.
##


127.0.0.1 localhost
255.255.255.255 broadcasthost
::1 localhost
fe80::1%lo0 localhost

Also found a file called ntp-restrict.conf, containing:

# Access restrictions documented in ntp.conf(5) and
# http://support.ntp.org/bin/view/Support/AccessRe...
# Limit network machines to time queries only

restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery

# localhost is unrestricted
restrict 127.0.0.1
restrict -6 ::1

includefile /private/etc/ntp.conf

Not sure if these are what I'm looking for -- I'm new to unix so I may
need a bit more hand-holding here.

I appreciate your time and effort.

Matt

Lie Ryan

2/20/2010 5:36:00 PM

0

On 02/20/10 19:36, MattB wrote:
> On Feb 20, 2:02 am, Lie Ryan <lie.1...@gmail.com> wrote:
>> On 02/20/10 13:32, MattB wrote:
>>
>>
>>
>>> I'm using the network in my own apartment. Not the campus's.
>>> Moreover, my mac's MAC address is different from the MAC address shown
>>> by my router, but as I said I'm also blocked when using my friend's
>>> wireless router at his apartment.
>>
>>> So it must be my mac's MAC, and not the router's MAC, that's being
>>> blocked, right?
>>
>>> But ALSO -- is it my ISP that's blocking the mac's MAC (and not the
>>> school), since I can't raise ANY url's from python when I'm on
>>> wireless?
>>
>> MAC or IP blocking can't be the reason, as the OP stated, he can use
>> Firefox just fine.
>>
>> Can you access, say,http://www.goog... urllib or mechanize?
>>
>> If you can't access *any website* using urllib/mechanize but you can
>> with a browser and you're on a unix-based machine, you probably have the
>> same problem as I used to have. Check whether you used the same hostname
>> in /etc/conf.d/hostname and /etc/hosts (or wherever your distro saves
>> its hostname configurations, I use Gentoo); after editing those files
>> reboot (there are ways to avoid reboot, but rebooting guarantees the
>> conf file is reread).
>>
>> Check the hostname by running this python script:
>>
>> import socket
>> hn = socket.gethostname()
>> print hn
>> print socket.gethostbyname(hn) # should be 127.0.0.1
>
> Lie,
>
> Wow. Strangely, that script returned 192.168.1.106. However, in Snow
> Leopard's airport settings, if I click on 'advanced' and then
> 'proxies', the default proxy for 'http' is 127.0.0.1:4444 (and in
> these settings, the 'use proxy for http' is checked).
>
> I just tried checking the unix files you mentioned. In etc/hosts, the
> following info is displayed:
>
> ##
> # Host Database
> #
> # localhost is used to configure the loopback interface
> # when the system is booting. Do not change this entry.
> ##
>
>
> 127.0.0.1 localhost
> 255.255.255.255 broadcasthost
> ::1 localhost
> fe80::1%lo0 localhost
>
>
> Not sure if these are what I'm looking for -- I'm new to unix so I may
> need a bit more hand-holding here.

Try the alternatives in this site: http://movealong.org/hos...


> Also found a file called ntp-restrict.conf, containing:
>
> # Access restrictions documented in ntp.conf(5) and
> # http://support.ntp.org/bin/view/Support/AccessRe...
> # Limit network machines to time queries only
>
> restrict default kod nomodify notrap nopeer noquery
> restrict -6 default kod nomodify notrap nopeer noquery
>
> # localhost is unrestricted
> restrict 127.0.0.1
> restrict -6 ::1
>
> includefile /private/etc/ntp.conf

Not sure if ntp-restrict.conf is a red herring here since its name
implies it's something for NTP (Network Time Protocol, used for time
synchronization)