[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

Memory errors with imaplib

Martey

1/20/2008 4:34:00 AM

Hi everyone,

I am trying to use imaplib to download messages, but I keep getting
memory errors when I try to download a large message (i.e. one with
attachments). Here is my test code (similar to the example in the
imaplib documentation):

import getpass, imaplib

M = imaplib.IMAP4_SSL("imap.gmail.com")
M.login("martey@gmail.com", getpass.getpass())
M.select("[Gmail]/All Mail")
typ, data = M.search(None, 'ALL')
for num in data[0].split():
typ, data = M.fetch(num, '(RFC822)')
print 'Message %s\n%s\n' % (num, data[0][1])
M.close()
M.logout()


And here is the error message:

Python(4521) malloc: *** mmap(size=6389760) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Traceback (most recent call last):
File "imaptest.py", line 8, in <module>
typ, data = M.fetch(num, '(RFC822)')
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/imaplib.py", line 437, in fetch
typ, dat = self._simple_command(name, message_set, message_parts)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/imaplib.py", line 1055, in _simple_command
return self._command_complete(name, self._command(name, *args))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/imaplib.py", line 885, in _command_complete
typ, data = self._get_tagged_response(tag)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/imaplib.py", line 986, in _get_tagged_response
self._get_response()
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/imaplib.py", line 948, in _get_response
data = self.read(size)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/imaplib.py", line 1150, in read
data = self.sslobj.read(size-read)
MemoryError

I am using Mac OS X 10.5 and Python 2.5.1 (downloaded from
python.org). I think it is possible that it is <http://bugs.p...
issue1092502>, but the workarounds suggested there do not seem to work
for me. Is this an actual Python bug, or am I doing something wrong?
6 Answers

Fredrik Lundh

1/20/2008 12:51:00 PM

0

Martey wrote:

> I am trying to use imaplib to download messages, but I keep getting
> memory errors when I try to download a large message (i.e. one with
> attachments). Here is my test code (similar to the example in the
> imaplib documentation):

> /.../

> I am using Mac OS X 10.5 and Python 2.5.1 (downloaded from
> python.org). I think it is possible that it is <http://bugs.p...
> issue1092502>, but the workarounds suggested there do not seem to work
> for me. Is this an actual Python bug, or am I doing something wrong?

looks like a known bug in imaplib:

http://bugs.p...issue1389051

"In a worst case scenario, you'll need some 13 gigabytes of
virtual memory to read a 15 megabyte message..."

</F>

Grant Edwards

1/20/2008 3:17:00 PM

0

On 2008-01-20, Fredrik Lundh <fredrik@pythonware.com> wrote:

> looks like a known bug in imaplib:
>
> http://bugs.python.org/is...
>
> "In a worst case scenario, you'll need some 13 gigabytes of
> virtual memory to read a 15 megabyte message..."

The problem and the one-line soulution have been known for over
two years and it's still an open bug?

--
Grant Edwards grante Yow! Yow! Is my fallout
at shelter termite proof?
visi.com

Christian Heimes

1/20/2008 4:10:00 PM

0

Grant Edwards wrote:
> The problem and the one-line soulution have been known for over
> two years and it's still an open bug?

Nobody was interested to provide a patch. I raised the level of the bug.
It's going to be fixed soonish.

Christian

Grant Edwards

1/20/2008 10:49:00 PM

0

On 2008-01-20, Christian Heimes <lists@cheimes.de> wrote:
> Grant Edwards wrote:
>> The problem and the one-line soulution have been known for over
>> two years and it's still an open bug?
>
> Nobody was interested to provide a patch. I raised the level of the bug.
> It's going to be fixed soonish.

If the solution shown in the bug report is correct, I'd be more
than happy to generate a patch.

--
Grant

Christian Heimes

1/21/2008 1:57:00 PM

0

Grant Edwards wrote:
> If the solution shown in the bug report is correct, I'd be more
> than happy to generate a patch.

The solution seems fine but IMO it should be fixed in the ssl socket and
not in imaplib. I like to get it *fixed* and not *worked around*. :)

Christian

Grant Edwards

1/21/2008 4:04:00 PM

0

On 2008-01-21, Christian Heimes <lists@cheimes.de> wrote:
> Grant Edwards wrote:
>
>> If the solution shown in the bug report is correct, I'd be
>> more than happy to generate a patch.
>
> The solution seems fine but IMO it should be fixed in the ssl
> socket and not in imaplib. I like to get it *fixed* and not
> *worked around*. :)

I've always been taught that you can't assume that a read on a
socket (ssl or otherwise) will return the number of bytes you
request. You've always got to code for the case where read()
returns smaller chunks. Maybe I've been taught wrongly, but
under Unix at least I've found that it's always a good practice
to assume that network reads will often return smaller chunks
than requested.

--
Grant Edwards grante Yow! Are we on STRIKE yet?
at
visi.com