Asp Forum - memory leak - comp.lang.ruby

Joe Van Dyk

4/27/2005 2:26:00 AM

Got a memory leak in the following code.. not sure if DRb is necessary
in it or not.

Put the code into a file named test.rb, btw.

I think I tracked it down to the SystemVIPC::MessageQueue#recv method.
If I don't give it a high receive length (like 8192), I don't see the
leak. If I do give it a high length, the program takes up a very
large amount of space pretty quickly... about an increase of a meg
every five seconds.

Any ideas? (On my linux system, kernel.msgmax is set to 10000 if it matters)

# test.rb

require 'drb'
require 'sysvipc'

class Foo
def initialize
@key = SystemVIPC::ftok("test.rb", 1)
@msg = SystemVIPC::MessageQueue.new(@key, 0777 | SystemVIPC::IPC_CREAT)
end

def recv
continue = true
while continue
begin
@msg.recv(1, 8192, SystemVIPC::IPC_NOWAIT)
rescue
continue = false
end
end
end

def send
@msg.send(1, "HELLO WORLD HELLO WORLD")
end
end

f = Foo.new
some_thread = Thread.new(f) do |foo|
loop do
foo.recv
sleep 1
end
end
some_thread = Thread.new(f) do |foo|
loop do
foo.send
sleep 1
end
end

DRb.start_service('druby://localhost:8000', f)
DRb.thread.join

def send
@msg.send(1, "HELLO WORLD HELLO WORLD")
end
end

f = Foo.new
some_thread = Thread.new(f) do |foo|
loop do
foo.recv
sleep 1
end
end
some_thread = Thread.new(f) do |foo|
loop do
foo.send
sleep 1
end
end

DRb.start_service('druby://localhost:8000', f)
DRb.thread.join

11 Answers

Joe Van Dyk

4/27/2005 2:46:00 AM

eek. copy/paste borked.

See below for correct version.

On 4/26/05, Joe Van Dyk <joevandyk@gmail.com> wrote:
> Got a memory leak in the following code.. not sure if DRb is necessary
> in it or not.
>
> Put the code into a file named test.rb, btw.
>
> I think I tracked it down to the SystemVIPC::MessageQueue#recv method.
> If I don't give it a high receive length (like 8192), I don't see the
> leak. If I do give it a high length, the program takes up a very
> large amount of space pretty quickly... about an increase of a meg
> every five seconds.
>
> Any ideas? (On my linux system, kernel.msgmax is set to 10000 if it matters)
>

# test.rb

require 'drb'
require 'sysvipc'

class Foo
def initialize
@key = SystemVIPC::ftok("test.rb", 1)
@msg = SystemVIPC::MessageQueue.new(@key, 0777 | SystemVIPC::IPC_CREAT)
end

def recv
continue = true
while continue
begin
@msg.recv(1, 8192, SystemVIPC::IPC_NOWAIT)
rescue
continue = false
end
end
end

def send
@msg.send(1, "HELLO WORLD HELLO WORLD")
end
end

f = Foo.new
some_thread = Thread.new(f) do |foo|
loop do
foo.recv
sleep 1
end
end
some_thread = Thread.new(f) do |foo|
loop do
foo.send
sleep 1
end
end

DRb.start_service('druby://localhost:8000', f)
DRb.thread.join

John Carter

4/27/2005 3:05:00 AM

Ara.T.Howard

4/27/2005 3:51:00 AM

Joe Van Dyk

4/27/2005 4:08:00 PM

On 4/26/05, Joe Van Dyk <joevandyk@gmail.com> wrote:
> eek. copy/paste borked.
>
> See below for correct version.

FYI, you'll need SysVIPC installed
(http://deisui.org/~ueno/ruby/sy...).

In essence, the code below starts a thread that uses a object to read
from a message queue repeatedly. If there's no messages left on the
message queue, the #recv method throws an exception, we catch that and
exit the loop. The reader thread then sleeps for a second and reads
more messages from the queue.

The problem doesn't seem to be with the sending of the message over
the message queue, rather its the receive. Somewhere, something isn't
being thrown away. I used ObjectSpace to monitor the number of
objects that existed. The total number of Objects in memory seem to
fluctuate, but don't increase over time (while memory is increasing).

I looked at the C code for the SysVIPC module but couldn't see any
obvious errors.

Any thoughts?

Joe

> On 4/26/05, Joe Van Dyk <joevandyk@gmail.com> wrote:
> > Got a memory leak in the following code.. not sure if DRb is necessary
> > in it or not.
> >
> > Put the code into a file named test.rb, btw.
> >
> > I think I tracked it down to the SystemVIPC::MessageQueue#recv method.
> > If I don't give it a high receive length (like 8192), I don't see the
> > leak. If I do give it a high length, the program takes up a very
> > large amount of space pretty quickly... about an increase of a meg
> > every five seconds.
> >
> > Any ideas? (On my linux system, kernel.msgmax is set to 10000 if it matters)
> >
>
> # test.rb
>
> require 'drb'
> require 'sysvipc'
>
> class Foo
> def initialize
> @key = SystemVIPC::ftok("test.rb", 1)
> @msg = SystemVIPC::MessageQueue.new(@key, 0777 | SystemVIPC::IPC_CREAT)
> end
>
> def recv
> continue = true
> while continue
> begin
> @msg.recv(1, 8192, SystemVIPC::IPC_NOWAIT)
> rescue
> continue = false
> end
> end
> end
>
> def send
> @msg.send(1, "HELLO WORLD HELLO WORLD")
> end
> end
>
> f = Foo.new
> some_thread = Thread.new(f) do |foo|
> loop do
> foo.recv
> sleep 1
> end
> end
> some_thread = Thread.new(f) do |foo|
> loop do
> foo.send
> sleep 1
> end
> end
>
> DRb.start_service('druby://localhost:8000', f)
> DRb.thread.join
>

Ara.T.Howard

4/27/2005 4:16:00 PM

ts

4/27/2005 4:17:00 PM

>>>>> "J" == Joe Van Dyk <joevandyk@gmail.com> writes:

J> I looked at the C code for the SysVIPC module but couldn't see any
J> obvious errors.

xcalloc() called without a free(), no ?

Guy Decoux

Joe Van Dyk

4/27/2005 4:35:00 PM

On 4/27/05, ts <decoux@moulon.inra.fr> wrote:
> >>>>> "J" == Joe Van Dyk <joevandyk@gmail.com> writes:
>
> J> I looked at the C code for the SysVIPC module but couldn't see any
> J> obvious errors.
>
> xcalloc() called without a free(), no ?
>
> Guy Decoux
>
>

I saw that. But I didn't see how it could be free'd (but my C-fu sucks).

Here's the relevant code:

============================
static VALUE
rb_msg_recv (argc, argv, obj)
int argc;
VALUE *argv, obj;
{
VALUE v_type, v_len, v_flags;
int flags = 0;
struct msgbuf *msgp;
struct ipcid_ds *msgid;
long type;
size_t rlen, len;

rb_scan_args (argc, argv, "21", &v_type, &v_len, &v_flags);
type = NUM2LONG (v_type);
len = NUM2INT (v_len);
if (!NIL_P (v_flags))
flags = NUM2INT (v_flags);

msgp = xcalloc (sizeof (long) + len, sizeof (char));
msgid = get_ipcid (obj);

retry:
TRAP_BEG;
rlen = msgrcv (msgid->id, msgp, len, type, flags);
TRAP_END;
if (rlen == (size_t)-1)
{
switch (errno)
{
case EINTR:
rb_thread_schedule ();
case EWOULDBLOCK:
#if EAGAIN != EWOULDBLOCK
case EAGAIN:
#endif
goto retry;
}
rb_sys_fail ("msgrcv(2)");
}

return rb_str_new (msgp->mtext, rlen);
}
============================

msgp is included in the return statement, so I'm not sure how it could
be free'd. Isn't its memory being used by the new Ruby string? And
shouldn't that memory be garbage collected at some point?

Joe

ts

4/27/2005 4:39:00 PM

>>>>> "J" == Joe Van Dyk <joevandyk@gmail.com> writes:

J> I saw that. But I didn't see how it could be free'd (but my C-fu sucks).

perhaps use alloca() (with memzero()) rather than malloc().

Guy Decoux

Joe Van Dyk

4/27/2005 4:51:00 PM

On 4/27/05, ts <decoux@moulon.inra.fr> wrote:
> >>>>> "J" == Joe Van Dyk <joevandyk@gmail.com> writes:
>
> J> I saw that. But I didn't see how it could be free'd (but my C-fu sucks).
>
> perhaps use alloca() (with memzero()) rather than malloc().
>

Is that standard practice with Ruby extensions?

ts

4/27/2005 4:54:00 PM

>>>>> "J" == Joe Van Dyk <joevandyk@gmail.com> writes:

J> Is that standard practice with Ruby extensions?

Only the author of the extension know the response to this question ...

Guy Decoux

comp.lang.ruby

memory leak

Joe Van Dyk

Joe Van Dyk

John Carter

Ara.T.Howard

Joe Van Dyk

Ara.T.Howard

ts

Joe Van Dyk

ts

Joe Van Dyk

ts

x Login to ForumsZone