[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Nested threading? implications to timeout

Sunshine On Leith

9/16/2006 3:16:00 AM

All,

I'm certainly not a ruby expert by any means. I'm trying to come up to
speed as fast as possible on ruby and rails. My last few months have
been reading, coding, reading, more reading, more coding, etc. :) So I
was doing some coding to try to come up to speed on ruby. I really
think it's one of the most phenominal languages I've ever found. I
never got into java, too verbose. That said I'd like to move on to the
question.

At this point, I'm trying to track down some sort of memory leak. In my
vast amount of digging through google and writing test code, I came
across the following interesting behavior. It seems that threads do not
have a parent child relationships. So when one kills a thread, ruby
doesn't actually kill any threads that were started inside that thread.
To clarify, consider the following:

irb(main):001:0> x=nil ; y = Thread.start { x = Thread.start { sleep 0.5
until false } ; sleep 0.5 until false }
=> #<Thread:0xb7abe20c run>
irb(main):002:0> x.status
=> "sleep"
irb(main):003:0> y.kill
=> #<Thread:0xb7abe20c dead>
irb(main):007:0> x.status
=> "sleep"
irb(main):008:0>

Clearly by killing, what I would call the "parent thread", did not kill
the child thread. And this would be okay, I think, except that the way
the timeout class works is by starting a thread and killing it if it
doesn't complete in the right amount of time. So if I were to wrap
anything in "timeout() {}" it's possible that something else calls
timeout inside that and then the thread is still alive, until some later
time. Is this normal? Can this cause memory leak? Am I confused? :)

Thanks much for your help and feedback,

Geff

--
Posted via http://www.ruby-....
20 Answers

Arnaud Bergeron

9/16/2006 7:40:00 AM

0

On 9/15/06, Geff Geff <boing@boing.com> wrote:
> All,
>
> I'm certainly not a ruby expert by any means. I'm trying to come up to
> speed as fast as possible on ruby and rails. My last few months have
> been reading, coding, reading, more reading, more coding, etc. :) So I
> was doing some coding to try to come up to speed on ruby. I really
> think it's one of the most phenominal languages I've ever found. I
> never got into java, too verbose. That said I'd like to move on to the
> question.
>
> At this point, I'm trying to track down some sort of memory leak. In my
> vast amount of digging through google and writing test code, I came
> across the following interesting behavior. It seems that threads do not
> have a parent child relationships. So when one kills a thread, ruby
> doesn't actually kill any threads that were started inside that thread.
> To clarify, consider the following:
>
> irb(main):001:0> x=nil ; y = Thread.start { x = Thread.start { sleep 0.5
> until false } ; sleep 0.5 until false }
> => #<Thread:0xb7abe20c run>
> irb(main):002:0> x.status
> => "sleep"
> irb(main):003:0> y.kill
> => #<Thread:0xb7abe20c dead>
[There are lines missing here...]
> irb(main):007:0> x.status
> => "sleep"
> irb(main):008:0>
>
> Clearly by killing, what I would call the "parent thread", did not kill
> the child thread. And this would be okay, I think, except that the way
> the timeout class works is by starting a thread and killing it if it
> doesn't complete in the right amount of time. So if I were to wrap
> anything in "timeout() {}" it's possible that something else calls
> timeout inside that and then the thread is still alive, until some later
> time. Is this normal?

Reading the timeout code I see that it creates a watcher thread to
provide the timeout. The block is executed on the current thread and
interruption is provided by raising an exception (from the watcher
tread) when the timeout expires. At the end of the method there is an
ensure to kill the watcher thread in any case so it would not
encounter the case you describe.

> Can this cause memory leak?

Due do what I described in the previous paragraph nested timeouts
won't cause any kind of leak.

> Am I confused? :)

Yes, I believe you are confused.

By the way, reading the code of the methods you have doubts about can
be a good way to clarify something.

> Thanks much for your help and feedback,
>
> Geff
>
> --
> Posted via http://www.ruby-....
>
>


--
I'm trying to launch the internet; so I open a terminal and go
"percent sign 'Internet'" at the prompt and it doesn't work. What
gives??!! -- random troll

Sunshine On Leith

9/16/2006 11:40:00 PM

0

Arnaud Bergeron wrote:
> On 9/15/06, Geff Geff <boing@boing.com> wrote:
>> At this point, I'm trying to track down some sort of memory leak. In my
>> => "sleep"
>> irb(main):003:0> y.kill
>> => #<Thread:0xb7abe20c dead>
> [There are lines missing here...]

sorry that was just me repeating the status method multiple times. :)

--
Posted via http://www.ruby-....

Sunshine On Leith

9/16/2006 11:49:00 PM

0

Francis Cianfrocca wrote:
> On 9/15/06, Geff Geff <boing@boing.com> wrote:
>>
>> vast amount of digging through google and writing test code, I came
>> across the following interesting behavior. It seems that threads do not
>> have a parent child relationships. So when one kills a thread, ruby
>> doesn't actually kill any threads that were started inside that thread.
>> To clarify, consider the following:
>
>
> Threads don't have any kind of parent-child relationship, as processes
> do.
> None of the threading models that have seen wide use have a feature like
> this. What was your basis for thinking that they do?

What was I thinking was the following: lets consider the case where you
try to timeout a HTTP connect with the net/http stuff. There are
already "timeouts" in the HTTP class. If my timeout times out before
the library's timeout, I end up killing the http thread that is watching
the tcpconnect thread. The tcpconnect thread lives on until it's done.
Maybe not such a big deal in thise case but the situation certainly
could have been worse. It could be kind of messy if you're using a
library in general because you don't know if it's going ot call timeout
of it's own and how that might impact resource allocation/deallocation.
As far as the programmer is concerned, he / she timed out the http class
and it should be (all) gone. But it's not. It's still hanging around.
Just like my nested sleep was.

I don't have any basis for thinking there is a parent / child
relationship. Just sorta counter intuitive in the case of a nested
timeout. Because what programmer probably wants is a timeout on the
thing that they are trying to timeout. The *WHOLE* thing, not part of
it, without verifying the libraries don't call timeout themselves.

Thoughts?

Seprately I'm having some issue with some of my activerecord objects
sticking around in a long running deamon. Not all, but some. It's odd.
I haven't found the problem yet, and I really don't know what's going
on. It's probably not timeout related because those would free up
eventually.

Thanks so much for the comments thus far.

Geff

--
Posted via http://www.ruby-....

Francis Cianfrocca

9/17/2006 12:19:00 AM

0

On 9/16/06, Geff Geff <boing@boing.com> wrote:
> What was I thinking was the following: lets consider the case where you
> try to timeout a HTTP connect with the net/http stuff. There are
> already "timeouts" in the HTTP class. If my timeout times out before
> the library's timeout, I end up killing the http thread that is watching
> the tcpconnect thread. The tcpconnect thread lives on until it's done.

I haven't read through the net/http code as it relates to this issue,
but I did read timeout.rb and as Arnaud says, there should be no
problem. If I may assume that your thread calls timeout on a net/http
call, which in turn calls timeout on a TCP connect-attempt, then if
your timeout fires first, you'll kill the thread that is doing the TCP
connect, because it's the same thread that called net/http. The
"internal" timer thread is left running. It will hold a memory
reference to the killed thread, which can hold resources longer than
you expect, but there's no leak. When the internal timer thread that
is left running eventually times out, it will try to raise an
exception on the killed thread, and nothing will happen.

Sunshine On Leith

9/17/2006 12:35:00 AM

0

Francis Cianfrocca wrote:
> On 9/16/06, Geff Geff <boing@boing.com> wrote:
> > What was I thinking was the following: lets consider the case where
> you
>> try to timeout a HTTP connect with the net/http stuff. There are
>> already "timeouts" in the HTTP class. If my timeout times out before
>> the library's timeout, I end up killing the http thread that is watching
>> the tcpconnect thread. The tcpconnect thread lives on until it's done.
>
> I haven't read through the net/http code as it relates to this issue,
> but I did read timeout.rb and as Arnaud says, there should be no
> problem. If I may assume that your thread calls timeout on a net/http
> call, which in turn calls timeout on a TCP connect-attempt, then if
> your timeout fires first, you'll kill the thread that is doing the TCP
> connect, because it's the same thread that called net/http. The
> "internal" timer thread is left running. It will hold a memory
> reference to the killed thread, which can hold resources longer than
> you expect, but there's no leak. When the internal timer thread that
> is left running eventually times out, it will try to raise an
> exception on the killed thread, and nothing will happen.

I think if I timeout a timeout that my timeout is killing the "waiter"
thread. And since the waiter thread started the TCPconnect attempt
thread that tcpconnect stays running until it ends, if it ends. Kinda
like my sleep thread example. Is that not accurate?

--
Posted via http://www.ruby-....

Francis Cianfrocca

9/17/2006 12:00:00 PM

0

On 9/16/06, Geff Geff <boing@boing.com> wrote:
> I think if I timeout a timeout that my timeout is killing the "waiter"
> thread. And since the waiter thread started the TCPconnect attempt
> thread that tcpconnect stays running until it ends, if it ends. Kinda
> like my sleep thread example. Is that not accurate?
>

I don't think so. Try the following:

---------------------------------------
require 'timeout'
begin
timeout(2) {
timeout(4) {
sleep 6
p "I finished"
}
}
rescue Timeout::Error
p "timeout fired"
end

sleep 10
------------------------------------------
You'll see the timeout message after two seconds, and the program will
end ten seconds later without saying "I finished." You should be able
to ignore the fact that timeout is implemented with threads in the
first place. Or were you asking a different question?

Jano Svitok

9/17/2006 12:30:00 PM

0

On 9/17/06, Francis Cianfrocca <garbagecat10@gmail.com> wrote:
> On 9/16/06, Geff Geff <boing@boing.com> wrote:
> > I think if I timeout a timeout that my timeout is killing the "waiter"
> > thread. And since the waiter thread started the TCPconnect attempt
> > thread that tcpconnect stays running until it ends, if it ends. Kinda
> > like my sleep thread example. Is that not accurate?
> >
>
> I don't think so. Try the following:
>
> ---------------------------------------
> require 'timeout'
> begin
> timeout(2) {
> timeout(4) {
> sleep 6
> p "I finished"
> }
> }
> rescue Timeout::Error
> p "timeout fired"
> end
>
> sleep 10
> ------------------------------------------
> You'll see the timeout message after two seconds, and the program will
> end ten seconds later without saying "I finished." You should be able
> to ignore the fact that timeout is implemented with threads in the
> first place. Or were you asking a different question?

FYI: Eric Hodel has written a post about nested timeouts:
http://blog.segment7.net/articles/2006/04/11/care-and-feeding-of-timeo...

Sunshine On Leith

9/17/2006 4:06:00 PM

0

Francis Cianfrocca wrote:
> require 'timeout'
> begin
> timeout(2) {
> timeout(4) {
> sleep 6
> p "I finished"
> }
> }
> rescue Timeout::Error
> p "timeout fired"
> end
>
> sleep 10

Ur very right. Thanks very much. Sorry I had a bad case of HUA. I
read the timeout library code backwards, so to speak. As if the child
thread (so to speak) was the actual code block, instead of just the
sleep and the raise.

Sorry for posting something I shouldn't have wasted your time with.
Thanks for the patience and the help.

Geff

--
Posted via http://www.ruby-....

Sunshine On Leith

9/17/2006 4:07:00 PM

0

Jan Svitok wrote:
> FYI: Eric Hodel has written a post about nested timeouts:
> http://blog.segment7.net/articles/2006/04/11/care-and-feeding-of-timeo...

Wow! Thanks very much.

--
Posted via http://www.ruby-....

Arnaud Bergeron

9/17/2006 4:56:00 PM

0

On 9/16/06, Francis Cianfrocca <garbagecat10@gmail.com> wrote:
> On 9/16/06, Geff Geff <boing@boing.com> wrote:
> > What was I thinking was the following: lets consider the case where you
> > try to timeout a HTTP connect with the net/http stuff. There are
> > already "timeouts" in the HTTP class. If my timeout times out before
> > the library's timeout, I end up killing the http thread that is watching
> > the tcpconnect thread. The tcpconnect thread lives on until it's done.
>
> I haven't read through the net/http code as it relates to this issue,
> but I did read timeout.rb and as Arnaud says, there should be no
> problem. If I may assume that your thread calls timeout on a net/http
> call, which in turn calls timeout on a TCP connect-attempt, then if
> your timeout fires first, you'll kill the thread that is doing the TCP
> connect, because it's the same thread that called net/http. The
> "internal" timer thread is left running. It will hold a memory
> reference to the killed thread, which can hold resources longer than
> you expect, but there's no leak. When the internal timer thread that
> is left running eventually times out, it will try to raise an
> exception on the killed thread, and nothing will happen.
>
The timer thread won't even remain because of the ensure at the end of
the timeout function that kills it ...


--
I'm trying to launch the internet; so I open a terminal and go
"percent sign 'Internet'" at the prompt and it doesn't work. What
gives??!! -- random troll