Asp Forum
Home
|
Login
|
Register
|
Search
Forums
>
comp.lang.ruby
Excessive Memory Usage with libxml
Daniel N
5/22/2007 5:28:00 PM
Hi,
I hope this is the right place for this.
I'm writing a markup mangler that I'm using libxml for. Primarily for
speed reasons.
I've got some basic functionality going, it's not finished, and when
I'm doing a benchmark I get some good figures.
I'm using a recursive function to process my data. The general gist
of it is as follows
http://pastie.cabo...
But...
When I run the benchmark I get weird things happening.
All benchmarks exhibit steadily increasing memory usage until the end
of the benchmark
10000 iterations were fine
very much above this and I started to get
malloc errors
or
segmentation fault
I changed
doc.root.to_s.gsub( /\<\/?#{artificial_root_tag}\>/, "" )
to
doc.root.to_a.join.gsub( /\<\/?#{artificial_root_tag}\>/, "" )
taking a hit on performance, but it allowed me to get through 100,000
iterations.
The memory usage though was stupid at 235Mb Real and 233Mb Virtual
The HTML I am parsing is a fairly small snippet
I'm running on Mac OsX.
Any ideas what and why?
thankyou
Daniel
1 Answer
Daniel N
5/22/2007 5:43:00 PM
0
On 5/23/07, Daniel N <has.sox@gmail.com> wrote:
> Hi,
>
> I hope this is the right place for this.
>
> I'm writing a markup mangler that I'm using libxml for. Primarily for
> speed reasons.
>
> I've got some basic functionality going, it's not finished, and when
> I'm doing a benchmark I get some good figures.
>
> I'm using a recursive function to process my data. The general gist
> of it is as follows
>
>
http://pastie.cabo...
>
>
> But...
>
> When I run the benchmark I get weird things happening.
>
> All benchmarks exhibit steadily increasing memory usage until the end
> of the benchmark
> 10000 iterations were fine
> very much above this and I started to get
>
> malloc errors
> or
> segmentation fault
>
>
> I changed
> doc.root.to_s.gsub( /\<\/?#{artificial_root_tag}\>/, "" )
>
> to
> doc.root.to_a.join.gsub( /\<\/?#{artificial_root_tag}\>/, "" )
>
> taking a hit on performance, but it allowed me to get through 100,000
> iterations.
> The memory usage though was stupid at 235Mb Real and 233Mb Virtual
>
> The HTML I am parsing is a fairly small snippet
>
> I'm running on Mac OsX.
>
> Any ideas what and why?
> thankyou
>
> Daniel
>
>
I've managed to reproduce the error by changing to Benchmark.bmbm. It is
ruby(3775) malloc: *** Deallocation of a pointer not malloced:
0x3903a60; This could be a double free(), or free() called with the
middle of an allocated block; Try setting environment variable
MallocHelp to see tools to help debug
ruby(3775) malloc: *** Deallocation of a pointer not malloced:
0x75b0750; This could be a double free(), or free() called with the
middle of an allocated block; Try setting environment variable
MallocHelp to see tools to help debug
ruby(3775) malloc: *** Deallocation of a pointer not malloced:
0x75c4950; This could be a double free(), or free() called with the
middle of an allocated block; Try setting environment variable
MallocHelp to see tools to help debug
/markup_mangler_benchmark.rb:79: [BUG] Segmentation fault
Servizio di avviso nuovi messaggi
Ricevi direttamente nella tua mail i nuovi messaggi per
Excessive Memory Usage with libxml
Inserendo la tua e-mail nella casella sotto, riceverai un avviso tramite posta elettronica ogni volta che il motore di ricerca troverà un nuovo messaggio per te
Il servizio è completamente GRATUITO!
x
Login to ForumsZone
Login with Google
Login with E-Mail & Password