Asp Forum - Rev/actor TCP monkey patching

fedzor

2/11/2008 11:26:00 PM

Short and sweet -

How exactly does one monkey patch Rev's TCP over Net::HTTP-s?

Thanks,
-------------------------------------------------------|
~ Ari
seydar: it's like a crazy love triangle of Kernel commands and C code

12 Answers

Tony Arcieri

2/12/2008 2:43:00 AM

[Note: parts of this message were removed to make it a legal post.]

On Feb 11, 2008 4:26 PM, fedzor <fedzor@gmail.com> wrote:

> Short and sweet -
>
> How exactly does one monkey patch Rev's TCP over Net::HTTP-s?
>

Check out the SVN trunk:

svn checkout http://rev.rubyforg...

There's now a Rev::SSL module you can extend any subclass of Rev::TCPSocket
with (this includes Revactor::TCP::Socket).

To use it with Rev::HttpClient, just:

require 'rev/ssl'

and subclass HttpClient, defining the following callbacks:

def on_connect
super
extend(Rev::SSL)
ssl_client_start
end

You'll also need to:

def on_ssl_connect
request(...) # Make the HTTP request after the SSL handshake completes
end

...to initiate the HTTP request after the SSL handshake is complete.

There's a cooresponding #on_ssl_error(ex) method you can define to capture
any errors which occur during the SSL handshake (or any attempts to
negotiate a new session).
You can also:

def ssl_context
OpenSSL::SSL::SSLContext.new(...)
end

...to create an OpenSSL context to use for the SSL session. By default no
certificate verification takes place, so if you'd like to do any add the
appropriate certificates to the SSL context object.

If you're having any trouble, you might take a look at the Revactor SVN
trunk:

svn checkout http://revactor.rubyforg...

Revactor's HttpClient is already configured for SSL support, and wraps
everything up a lot better, e.g.:

response = Revactor::HttpClient.get("https://your.ssl.server.h...)

This also takes care of some of the other annoyances of HTTP, like following
redirects.

If you don't want to use Revactor directly, you can at least use it as a
reference for using Rev::HttpClient in conjunction with SSL. Just have a
look at lib/revactor/http_client.rb

I'd like to wrap up SSL support in Rev's HttpClient a bit more nicely, but
for now my real priority was getting it working in Revactor.

--
Tony Arcieri
ClickCaster, Inc.
tony@clickcaster.com

Tony Arcieri

2/12/2008 4:05:00 AM

[Note: parts of this message were removed to make it a legal post.]

On Feb 11, 2008 4:26 PM, fedzor <fedzor@gmail.com> wrote:

> Short and sweet -
>
> How exactly does one monkey patch Rev's TCP over Net::HTTP-s?
>

I probably should've mentioned this up front, but you can save yourself a
lot of grief by just using Revactor:

>> require 'revactor'
=> true
>> response = Revactor::HttpClient.get("https://www.google...)
=> #<Revactor::HttpResponse:0x5d30f4
@client=#<Revactor::HttpClient:0x5d5020>, @status=200, @reason="OK",
@version="HTTP/1.1", @content_length=nil, @chunked_encoding=true,
@header_fields={"Cache-Control"=>"private",
"Set-Cookie"=>"PREF=ID=b7f27e80b36a740b:TM=1202788671:LM=1202788671:S=CIZI75tKEMuS7q3X;
expires=Thu, 11-Feb-2010 03:57:51 GMT; path=/; domain=.google.com",
"Server"=>"gws", "Transfer-Encoding"=>"chunked", "Date"=>"Tue, 12 Feb 2008
03:57:51 GMT", "Connection"=>"Close"}, @content_type="text/html;
charset=ISO-8859-1">
>> response.body
=> "<html><head><meta http-equiv=\"content-type\" content=\"text/html;
charset=ISO-8859-1\"><title>Google</title> ...

That's it.

Just grab the Rev and Revactor svn repositories, "rake gem" in both, and
install the gems. You get all the benefits of an evented HTTP client
without the inversion-of-control headaches typically associated with evented
programming.

--
Tony Arcieri
ClickCaster, Inc.
tony@clickcaster.com

fedzor

2/12/2008 8:39:00 PM

Thank you so much!

This has really saved me a lot of speed and given me some more speed ;-)

On a side note, would you be able to post some more example with
Actors to achieve concurrency? Because at first I understand your
echo server, and then I tested out actors myself and everything I
knew fell apart....

Thanks,
Ari
--------------------------------------------|
If you're not living on the edge,
then you're just wasting space.

Tony Arcieri

2/12/2008 10:31:00 PM

[Note: parts of this message were removed to make it a legal post.]

On Feb 12, 2008 1:38 PM, fedzor <fedzor@gmail.com> wrote:

> On a side note, would you be able to post some more example with
> Actors to achieve concurrency? Because at first I understand your
> echo server, and then I tested out actors myself and everything I
> knew fell apart....
>

Here's a simple example. It will create a new Actor for each specified URL,
which probably isn't the behavior you want. You'll instead want something
like a queue. Also, this will gib if you have duplicates in the URL list.
And as another bonus, I haven't tested it, but I think it will work :)

However, it should be enough to get you started:

require 'revactor'

# A list of URLs we want to fetch
url_list = ['http://www.googl..., ...]

# Capture the parent Actor to send messages to
parent = Actor.current

# Spawn a new Actor for each URL we want to fetch
url_list.each do |u|
Actor.spawn do
# Request the URL
response = Revactor::HttpClient.get(u)

# Consume and store the body if the response status was 200
response.body if response.status == 200

# Close the connection
response.close

parent << [:http_response, u, response]
end
end

# Store all the responses in a hash mapping URLs to the responses
responses = {}

# Consume all the responses
while responses.size < url_list.size
Actor.receive do |filter|
# Catch messages which start with :http_response and contain two objects
filter.when(Case[:http_response, Object, Object]) do |_, url, response|
responses[url] = response
end
end
end

# Inspect the responses
p responses

--
Tony Arcieri
ClickCaster, Inc.
tony@clickcaster.com

fedzor

2/13/2008 1:11:00 AM

On Feb 12, 2008, at 5:30 PM, Tony Arcieri wrote:

> # Capture the parent Actor to send messages to
> parent = Actor.current
>
> # Spawn a new Actor for each URL we want to fetch
> url_list.each do |u|
> Actor.spawn do
> # Request the URL
> response = Revactor::HttpClient.get(u)
>
> # Consume and store the body if the response status was 200
> response.body if response.status == 200
>
> # Close the connection
> response.close
>
> parent << [:http_response, u, response]
> end
> end

Here is where I've found a problemo will occur. In my *ahem*
extensive *ahem* research with your fine fine actors ;-), concurrency
only exists when run within an actor - Actor.current has failed me
here! Example:

foods = ['chocolate', 'seltzer', 'awesome sauce']
foods.each do |food|

Actor.spawn(food) do |f|
sleep 1
puts f
end
end
puts "foods rounded up"

# chocolate
# foods rounded up

Uhoh! Not quite. Lemme try something different -

Well, whatever, I couldn't reproduce that code, because I updated to
your svn code. The one released as a gem required me to embed it all
within an actor before getting any sort of concurrency. But
basically, I sent some snail mail to an actor and it waited on
quitting until all of the internal actors finished working
concurrently. It was really awesome and I understood it! And then I
reinstalled revactor. oof! Better work something out.

Do you think you could help me understand what I would need to do to
get awesome sauce printed?

Thanks,
-------------------------------------------------------|
~ Ari
if god gives you lemons
YOU FIND A NEW GOD

fedzor

2/13/2008 1:39:00 AM

As a follow up, I realized why it wasn't working: bug in filters. I'm
too scared to patch it up, but I think I'll do that anyways. Example
problem:

myactor = Actor.spawn do
Actor.receive do |filter|
filter.when(:dog) { puts "I got a dog!" }
end
end

myactor << :dog

Run it and see..... NOTHING. it exits.... so..... what up G.

-------------------------------------------------------|
~ Ari
seydar: it's like a crazy love triangle of Kernel commands and C code

Tony Arcieri

2/13/2008 1:48:00 AM

[Note: parts of this message were removed to make it a legal post.]

On Feb 12, 2008 6:10 PM, fedzor <fedzor@gmail.com> wrote:

> Here is where I've found a problemo will occur. In my *ahem*
> extensive *ahem* research with your fine fine actors ;-), concurrency
> only exists when run within an actor - Actor.current has failed me
> here! Example:
>
> foods = ['chocolate', 'seltzer', 'awesome sauce']
> foods.each do |food|
>
> Actor.spawn(food) do |f|
> sleep 1
> puts f
> end
> end
> puts "foods rounded up"
>

I should really document this better, but: In Revactor, Actors are
Fiber-based, so anything that blocks for prolonged periods of time will hang
all Actors in the system. It's the same sort of thing you'll encounter with
an evented framework like EventMachine.

Fortunately, there's "Actor-safe" replacements for most of the blocking
tasks you'll perform in a networked application, namely: DNS resolution,
opening connections, SSL handshakes, and reading from and writing to the
network.

In the case of sleep, there's the handy:

Actor.sleep 1

Which is just shorthand for Actor.receive { |filter| filter.after 1 }

For anything related to networking, you need to use Actor::TCP or
Actor::HttpClient (which uses the fully asynchronous HTTP client from Rev)

To execute long-running blocks of code which aren't related to networking,
the next release of Revactor will be thread safe. This means you can spin
the long running task off in a thread, and have it send a message when it
completes.

--
Tony Arcieri
ClickCaster, Inc.
tony@clickcaster.com

Tony Arcieri

2/13/2008 1:58:00 AM

[Note: parts of this message were removed to make it a legal post.]

On Feb 12, 2008 6:39 PM, fedzor <fedzor@gmail.com> wrote:

>
> As a follow up, I realized why it wasn't working: bug in filters. I'm
> too scared to patch it up, but I think I'll do that anyways. Example
> problem:
>
> myactor = Actor.spawn do
> Actor.receive do |filter|
> filter.when(:dog) { puts "I got a dog!" }
> end
> end
>
> myactor << :dog
>
> Run it and see..... NOTHING. it exits.... so..... what up G.
>

The problem here is that you need to do something which yields control back
to the Actor scheduler before it can run. The simplest thing you can do is:

Actor.sleep 0

This will yield control to the scheduler, which will process any outstanding
messages then return control back to you (since you told it you wanted to
sleep for 0 seconds)

It's pretty confusing from irb, I'll admit...

When doing anything with Actors, just remember you're "queuing up"
operations which will run later... later being whenever you call
Actor.receive. Actor.receive is the only way to defer control to other
Actors (keeping in mind Actor.sleep is just shorthand for Actor.receive)

All of the "blocking" operations in Revactor, things like Actor::TCP.connect,
Actor::HttpClient.get, etc. are all calling Actor.receive underneath.

--
Tony Arcieri
ClickCaster, Inc.
tony@clickcaster.com

fedzor

2/13/2008 2:12:00 AM

On Feb 12, 2008, at 8:57 PM, Tony Arcieri wrote:

> The problem here is that you need to do something which yields
> control back
> to the Actor scheduler before it can run. The simplest thing you
> can do is:
>
> Actor.sleep 0
>
> This will yield control to the scheduler, which will process any
> outstanding
> messages then return control back to you (since you told it you
> wanted to
> sleep for 0 seconds)
>
> It's pretty confusing from irb, I'll admit...
>
> When doing anything with Actors, just remember you're "queuing up"
> operations which will run later... later being whenever you call
> Actor.receive. Actor.receive is the only way to defer control to
> other
> Actors (keeping in mind Actor.sleep is just shorthand for
> Actor.receive)

BTW, do you hang on IRC? #ruby-lang, seydar.

So I need to do the Actor.sleep to dish out control, but how come I
didn't have to do that with the version of revactor actually released?

-------------------------------------------------------|
~ Ari
if god gives you lemons
YOU FIND A NEW GOD

fedzor

2/13/2008 2:13:00 AM

On Feb 12, 2008, at 8:48 PM, Tony Arcieri wrote:

> To execute long-running blocks of code which aren't related to
> networking,
> the next release of Revactor will be thread safe. This means you
> can spin
> the long running task off in a thread, and have it send a message
> when it
> completes.

Next release or next svn update? I'm running the svn version.

~ Ari
English is like a pseudo-random number generator - there are a
bajillion rules to it, but nobody cares.

comp.lang.ruby

Rev/actor TCP monkey patching

fedzor

Tony Arcieri

Tony Arcieri

fedzor

Tony Arcieri

fedzor

fedzor

Tony Arcieri

Tony Arcieri

fedzor

fedzor

x Login to ForumsZone