[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

python 3.0 memory leaking?

rupert.thurner

2/10/2008 7:12:00 PM

the edgewall trac release 0.11 is blocked now since more than one
month for a memory leak nobody is able to find, see
http://groups.google.com/group/trac-dev/browse_thread/thread/116e5...
.

does python-3.0 improve something to avoid writing memory leaking
applications?

rupert.
4 Answers

Jeroen Ruigrok van der Werven

2/10/2008 8:05:00 PM

0

Hi Rupert,

-On [20080210 20:16], rupert.thurner (rupert.thurner@gmail.com) wrote:
>the edgewall trac release 0.11 is blocked now since more than one
>month for a memory leak nobody is able to find, see
>http://groups.google.com/group/trac-dev/browse_thread/thread/116e5...
>.

You are slightly mistaken that nobody is able to find anything. There is
currently a lot underway to fix this. The problem was not just one area, but
problems on a number of areas. Given the fact nobody uses a large enough
site with a 0.11 trunk installation these sort of things will start to show
during beta test, as it did. This is still beta period and I personally
would rather release a month later (even despite the already long release
time) than give our users a solution that eats away at their resources like
crazy. I am sure people will appreciate the latter more in the end. Already
the current fixes in trunk for Trac and Genshi stabilized a few large beta
deployment sites in terms of resource eating.

>does python-3.0 improve something to avoid writing memory leaking
>applications?

There are efforts underway to optimize the memory usage for 2.6 on a lot of
levels more than 2.5 already did. Check Python-dev of the last 2-3 weeks.
However, this is not interesting in the short term, since 2.4 and 2.5 (and
later on 2.6) will be versions that will be used for the coming 1-2 years
for sure. Avoidance is very difficult, especially given Python's dynamic
nature. But I'll let more knowledgeable minds talk about that.

For me, a relative newbie to Python, the entire memory allocation issue is
not transparent at all and information about it is scattered across the net.
One of the things I hope to contribute to in the coming year is to make sure
the entire area of Python memory use and proper coding is better understood.

--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
ã?¤ã?§ã?«ã?¼ã?³ ã?©ã?¦ã??ã?­ã??ã?¯ ã?´ã?¡ã?³ ã??ã?« ã?¦ã?§ã?«ã?´ã?§ã?³
http://www.in-n... | http://www.ra...
What is to be, will be. And what isn't to be sometimes happens...

Christian Heimes

2/10/2008 8:09:00 PM

0

rupert.thurner wrote:
> does python-3.0 improve something to avoid writing memory leaking
> applications?

No, it doesn't. Python 3.0 and maybe 2.6 are going to have some small
improvements but the improvements aren't related to memory leaking. I'm
working on the matter for a while now. I've limited some free lists and
I've a patch that gives allocated memory back to the OS earlier.

Have you followed my advice and checked all places that deal with
frames, tracebacks, exception objects and __del__ methods?

Christian

Christian Heimes

2/10/2008 8:25:00 PM

0

Jeroen Ruigrok van der Werven wrote:
> For me, a relative newbie to Python, the entire memory allocation issue is
> not transparent at all and information about it is scattered across the net.
> One of the things I hope to contribute to in the coming year is to make sure
> the entire area of Python memory use and proper coding is better understood.

Python uses its own memory allocator for small objecst (< 257 bytes).
Larger objects are allocated directly with malloc, smaller objects end
up in arenas. The code is well documented in
http://svn.python.org/view/python/trunk/Objects/obmalloc.c?rev=56476&...

Several objects keep a free list for performance reasons. Free list save
some extra mallocs and initialization of data structures. I've renamed
all free lists in Python 2.6 to "free_list".

Ints and floats are using their own block allocation algorithm. The code
predates Python's pymalloc code. I'm working on replacing the code with
pymalloc because pymalloc-ed memory is given back to the OS. The int
and float free lists keep their sizes until the Python process ends.

Christian

rupert.thurner

2/10/2008 10:23:00 PM

0

On Feb 10, 9:08 pm, Christian Heimes <li...@cheimes.de> wrote:
> rupert.thurner wrote:
> > does python-3.0 improve something to avoid writing memory leaking
> > applications?
>
> No, it doesn't. Python 3.0 and maybe 2.6 are going to have some small
> improvements but the improvements aren't related to memory leaking. I'm
> working on the matter for a while now. I've limited some free lists and
> I've a patch that gives allocated memory back to the OS earlier.
>
> Have you followed my advice and checked all places that deal with
> frames, tracebacks, exception objects and __del__ methods?
>
> Christian

many thanks christian! do you have any code examples how to handle
frames, tracebacks, exception objects and __del__ correctly?

btw, i put your hints on http://wiki.python.org/moin/....

rupert.