Asp Forum - [ANN] Multiplexer - linear non-blocking I/O

Mikael Brockman

11/26/2004 1:17:00 PM

Blocking I/O is really easy to use. But when you use it to write
servers, you run into problems: you can't run two blocking syscalls
simultaneously. So if you're writing a huge file to some guy, every
other client is stalled, and no one new can connect. Unacceptable, for
many types of servers. They need non-blocking I/O.

Non-blocking I/O is a lot more annoying to use. Instead of going

| write "Hello. What's your name?"
| name = read_line
| write "How do you do, #{name}?"
| state = read_line
| write "It's nice to hear that you're #{state}, #{name}."

we have to make weird state machines.

The good news: we can use callcc to make the non-blocking nature
practically invisible. Multiplexer does that. Here's how you'd write a
hello server:

| class Test < Multiplexer::Handler
| def handle
| write_line "Hello. What's your name?"
| name = read_line
| write_line "How do you do, #{name}?"
| state = read_line
| write_line "It's nice to hear that you're #{state}, #{name}."
| disconnect
| end
| end
|
| def test
| multiplexer = Multiplexer::Multiplexer.new 0.5
| multiplexer.listen 31337, Test
| multiplexer.run
| end
|
| test

It'll run in one thread. Multiplexer handles the select(3) calls.

Get it:
http://www.phubuh.org/Projects/nntpu/_darcs/current/mult...

31 Answers

Robert Klemme

11/26/2004 1:54:00 PM

"Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
news:87vfbszr5m.fsf@igloo.phubuh.org...
> Blocking I/O is really easy to use. But when you use it to write
> servers, you run into problems: you can't run two blocking syscalls
> simultaneously. So if you're writing a huge file to some guy, every
> other client is stalled, and no one new can connect. Unacceptable, for
> many types of servers. They need non-blocking I/O.
>
> Non-blocking I/O is a lot more annoying to use. Instead of going

<snip/>

> It'll run in one thread. Multiplexer handles the select(3) calls.

What is the advantage over a solution with threads? IOW, why should I use
multiplexer over individual threads per connection?

Kind regards

robert

Mikael Brockman

11/26/2004 2:20:00 PM

"Robert Klemme" <bob.news@gmx.net> writes:

> "Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
> news:87vfbszr5m.fsf@igloo.phubuh.org...
>
> > Blocking I/O is really easy to use. But when you use it to write
> > servers, you run into problems: you can't run two blocking syscalls
> > simultaneously. So if you're writing a huge file to some guy, every
> > other client is stalled, and no one new can connect. Unacceptable,
> > for many types of servers. They need non-blocking I/O.
> >
> > Non-blocking I/O is a lot more annoying to use. Instead of going
>
> <snip/>
>
> > It'll run in one thread. Multiplexer handles the select(3) calls.
>
> What is the advantage over a solution with threads? IOW, why should I
> use multiplexer over individual threads per connection?

Since Ruby's threads aren't native, you can't do I/O from several at a
time. So one IO#read call blocking for a long time will block the other
threads, too. You could loop a read with a time-out, I guess. But with
a single thread running select, the whole process can stall completely
while waiting for I/O. And it's more elegant. :-)

Dave Thomas

11/26/2004 3:39:00 PM

On Nov 26, 2004, at 8:19, Mikael Brockman wrote:

> Since Ruby's threads aren't native, you can't do I/O from several at a
> time. ]

This is not true. Ruby goes non-blocking I/O from threads, so in
general you'll see overlapped execution.

Thread.new do
loop do
puts "You said #{gets}"
end
end

10.times do
sleep(1)
puts "Say something"
end

Cheers

Dave

Robert Klemme

11/26/2004 3:41:00 PM

"Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
news:87r7mgzo8x.fsf@igloo.phubuh.org...
> "Robert Klemme" <bob.news@gmx.net> writes:
>
> > "Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
> > news:87vfbszr5m.fsf@igloo.phubuh.org...
> >
> > > Blocking I/O is really easy to use. But when you use it to write
> > > servers, you run into problems: you can't run two blocking syscalls
> > > simultaneously. So if you're writing a huge file to some guy, every
> > > other client is stalled, and no one new can connect. Unacceptable,
> > > for many types of servers. They need non-blocking I/O.
> > >
> > > Non-blocking I/O is a lot more annoying to use. Instead of going
> >
> > <snip/>
> >
> > > It'll run in one thread. Multiplexer handles the select(3) calls.
> >
> > What is the advantage over a solution with threads? IOW, why should I
> > use multiplexer over individual threads per connection?
>
> Since Ruby's threads aren't native, you can't do I/O from several at a
> time.

That's not true.

> So one IO#read call blocking for a long time will block the other
> threads, too.

Also wrong: execute this script

tickers = [$stdout, $stderr].map do |io|
Thread.new do
100.times do |i|
io.puts "#{io.fileno}: #{Time.now}: Tick #{i}"
sleep 1
end
end
end

puts "PROMPT"
# blocks in next line:
input = gets
puts "ENTERED #{input}"

tickers.each {|th| th.join }

16:39:44 [ruby]: ruby ticker.rb
1: Fri Nov 26 17:40:05 GMT+2:00 2004: Tick 0
2: Fri Nov 26 17:40:05 GMT+2:00 2004: Tick 0
PROMPT
2: Fri Nov 26 17:40:06 GMT+2:00 2004: Tick 1
1: Fri Nov 26 17:40:06 GMT+2:00 2004: Tick 1
2: Fri Nov 26 17:40:07 GMT+2:00 2004: Tick 2
1: Fri Nov 26 17:40:07 GMT+2:00 2004: Tick 2
2: Fri Nov 26 17:40:08 GMT+2:00 2004: Tick 3
1: Fri Nov 26 17:40:08 GMT+2:00 2004: Tick 3
foo
ENTERED foo
2: Fri Nov 26 17:40:09 GMT+2:00 2004: Tick 4
1: Fri Nov 26 17:40:09 GMT+2:00 2004: Tick 4
2: Fri Nov 26 17:40:10 GMT+2:00 2004: Tick 5
1: Fri Nov 26 17:40:10 GMT+2:00 2004: Tick 5
2: Fri Nov 26 17:40:11 GMT+2:00 2004: Tick 6
1: Fri Nov 26 17:40:11 GMT+2:00 2004: Tick 6
2: Fri Nov 26 17:40:12 GMT+2:00 2004: Tick 7
1: Fri Nov 26 17:40:12 GMT+2:00 2004: Tick 7

> You could loop a read with a time-out, I guess. But with
> a single thread running select, the whole process can stall completely
> while waiting for I/O. And it's more elegant. :-)

IMHO threads are more elegant.

Regards

robert

Mikael Brockman

11/26/2004 4:05:00 PM

"Robert Klemme" <bob.news@gmx.net> writes:

> "Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
> news:87r7mgzo8x.fsf@igloo.phubuh.org...
> > "Robert Klemme" <bob.news@gmx.net> writes:
> >
> > > "Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
> > > news:87vfbszr5m.fsf@igloo.phubuh.org...
> > >
> > > > Blocking I/O is really easy to use. But when you use it to
> > > > write servers, you run into problems: you can't run two blocking
> > > > syscalls simultaneously. So if you're writing a huge file to
> > > > some guy, every other client is stalled, and no one new can
> > > > connect. Unacceptable, for many types of servers. They need
> > > > non-blocking I/O.
> > > >
> > > > Non-blocking I/O is a lot more annoying to use. Instead of
> > > > going
> > >
> > > <snip/>
> > >
> > > > It'll run in one thread. Multiplexer handles the select(3) calls.
> > >
> > > What is the advantage over a solution with threads? IOW, why
> > > should I use multiplexer over individual threads per connection?
> >
> > Since Ruby's threads aren't native, you can't do I/O from several at
> > a time.
>
> That's not true.
>
> > So one IO#read call blocking for a long time will block the other
> > threads, too.
>
> Also wrong: execute this script
> [snip]

Sorry: faulty generalization. The problem is demonstrated in this
script:

| require 'socket'
|
| server = TCPServer.new 12345
|
| t_a = Thread.start do
| a = server.accept
| data = "foo" * 10000000
| a << data
| a.close
| end
|
| b = server.accept
| b << "b"
| b.close
|
| t_a.join

The second client isn't accepted until the huge batch of data is sent.
I guess you could solve the problem by splitting it into a bunch of
smaller batches.

Another appeal could be that by keeping it single-threaded, you have
fewer concurrency issues to worry about.

Robert Klemme

11/26/2004 4:15:00 PM

"Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
news:87brdkzjd8.fsf@igloo.phubuh.org...
> "Robert Klemme" <bob.news@gmx.net> writes:
>
> > "Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
> > news:87r7mgzo8x.fsf@igloo.phubuh.org...
> > > "Robert Klemme" <bob.news@gmx.net> writes:
> > >
> > > > "Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
> > > > news:87vfbszr5m.fsf@igloo.phubuh.org...
> > > >
> > > > > Blocking I/O is really easy to use. But when you use it to
> > > > > write servers, you run into problems: you can't run two blocking
> > > > > syscalls simultaneously. So if you're writing a huge file to
> > > > > some guy, every other client is stalled, and no one new can
> > > > > connect. Unacceptable, for many types of servers. They need
> > > > > non-blocking I/O.
> > > > >
> > > > > Non-blocking I/O is a lot more annoying to use. Instead of
> > > > > going
> > > >
> > > > <snip/>
> > > >
> > > > > It'll run in one thread. Multiplexer handles the select(3)
calls.
> > > >
> > > > What is the advantage over a solution with threads? IOW, why
> > > > should I use multiplexer over individual threads per connection?
> > >
> > > Since Ruby's threads aren't native, you can't do I/O from several at
> > > a time.
> >
> > That's not true.
> >
> > > So one IO#read call blocking for a long time will block the other
> > > threads, too.
> >
> > Also wrong: execute this script
> > [snip]
>
> Sorry: faulty generalization. The problem is demonstrated in this
> script:
>
> | require 'socket'
> |
> | server = TCPServer.new 12345
> |
> | t_a = Thread.start do
> | a = server.accept
> | data = "foo" * 10000000
> | a << data
> | a.close
> | end
> |
> | b = server.accept
> | b << "b"
> | b.close
> |
> | t_a.join
>
> The second client isn't accepted until the huge batch of data is sent.
> I guess you could solve the problem by splitting it into a bunch of
> smaller batches.
>
> Another appeal could be that by keeping it single-threaded, you have
> fewer concurrency issues to worry about.

The typical pattern for TCPserver looks different: You create a thread per
accepted connection.

require 'socket'

server = TCPServer.new 12345

loop do
Thread.new(server.accept) do |a|
begin
data = "foo" * 10000000
a << data
ensure
a.close
end
end
end

Regards

robert

Mikael Brockman

11/26/2004 4:30:00 PM

"Robert Klemme" <bob.news@gmx.net> writes:

> "Mikael Brockman" <mikael@phubuh.org> schrieb im Newsbeitrag
> news:87brdkzjd8.fsf@igloo.phubuh.org...
>
> The typical pattern for TCPserver looks different: You create a thread per
> accepted connection.
>
> require 'socket'
>
> server = TCPServer.new 12345
>
> loop do
> Thread.new(server.accept) do |a|
> begin
> data = "foo" * 10000000
> a << data
> ensure
> a.close
> end
> end
> end

You're right, but I get the same results. Only one client at a time is
accepted.

Lloyd Zusman

11/26/2004 5:27:00 PM

"Robert Klemme" <bob.news@gmx.net> writes:

> [ ... ]
>
> What is the advantage over a solution with threads? IOW, why should I
> use multiplexer over individual threads per connection?

The issues raised in subsequent posts illustrate the classic arguments
between one-thread-per-connection proponents and select-loop proponents
that I have heard since the mid 1990's.

Given good thread and select/non-blocking-io implementations, both
methods can solve the same problems and can work just fine.

As for me, I believe that it's good to have both methods to choose from.

The java folks have recently offered a 'select' methodology in addition
to their traditional thread-based approach. Perl has not-so-recently
added thread support on top of its traditional preference for
'select'.

And I'm glad that I also have both options in ruby.

--
Lloyd Zusman
ljz@asfast.com
God bless you.

Bill Kelly

11/26/2004 6:10:00 PM

From: "Mikael Brockman" <mikael@phubuh.org>
>
> "Robert Klemme" <bob.news@gmx.net> writes:
>
> > server = TCPServer.new 12345
> >
> > loop do
> > Thread.new(server.accept) do |a|
> > begin
> > data = "foo" * 10000000
> > a << data
> > ensure
> > a.close
> > end
> > end
> > end
>
> You're right, but I get the same results. Only one client at a time is
> accepted.

Strange... I'm surprised there are too many differences between
your select multiplexer using continuations and a threads solution,
since: a) ruby uses select() to multiplex behind the scenes when
multiple threads are doing I/O; and b) ruby continuations are
implemented using threads.

That said I haven't studied your continuations solution so maybe
my surprise is misplaced...

But I wrote this simplistic threaded socket IO performance test
last month http://bwk.homeip.net/ftp/ruby/... and I've
watched it accept and handle 100+ clients without delay. Each
client is pushing bytes as fast as it can (at the specified
chunk size), and each server thread handling a client is in
return replying with bytes as fast as it can.

. . Just chiming in in case this is somehow helpful... if not,
please disregard. =D

Regards,

Bill

Gyoung-Yoon Noh

11/26/2004 6:21:00 PM

Until one thread terminates a system call like IO-related task, other
threads will be blocked. Kernel does not know ruby's (userland)
threads, so if your application needs concurrency in massive IO tasks,
maybe you should implement a kind of thread scheduler by yourself. OK,
Kernel#fork will be an another choice.

If 'thread' is only sufficient for all, why did Sun adopt NIO in Java2 1.4?

Run following server example, and execute 'telnet localhost 31337' on
two each shell, no client connection will be blocked.

<code>
require 'multiplexer'

class Test2 < Multiplexer::Handler
def initialize
super()
end

def handle
begin
write_line "ooo" * 10000000
rescue EOFError
puts "My client closed its socket."
end
end
end

def test
multiplexer = Multiplexer::Multiplexer.new 0.5
multiplexer.client_data = []
multiplexer.listen 31337, Test2
puts "Listening on port 31337."
multiplexer.run
end

if $0 == __FILE__
test
end
</code>

IMHO, multiplexinig is a good choice for concurrent programming in
ruby, insofar ruby VM does not support native threads.

Best regards,

--
http://nohmad.su...

comp.lang.ruby

[ANN] Multiplexer - linear non-blocking I/O

Mikael Brockman

Robert Klemme

Mikael Brockman

Dave Thomas

Robert Klemme

Mikael Brockman

Robert Klemme

Mikael Brockman

Lloyd Zusman

Bill Kelly

Gyoung-Yoon Noh

x Login to ForumsZone