Asp Forum - [ANN] BackgrounDRb release 1.0 available now

hemant

12/17/2007 12:23:00 PM

Hi Folks,

I am glad to announce 1.0 release of BackgrounDRb.

This would be a major release since 0.2 release of BackgrounDRb.

Here is a brief summary of changes:

- BackgrounDRb is DRb no longer. It makes use of EventDriven network
programming library packet ( http://packet.goog... ).

- Since we moved to packet, many nasty thread issues, result hash
corruption issues are totally gone. Lots of work has went in
making scheduler rock solid stable.

- Each worker, still runs in its own process, but each worker has a
event loop of its own and all the events are triggered by the internal
reactor loop. In a nutshell, you are not encouraged to use threads
in your workers now. All the workers are already concurrent, but you
are encouraged to use co-operative multitasking, rather than
pre-emptive. A simple example is,

For implement something like progress bar in old version of bdrb, you would:
- start your processing a thread (so as your worker can receive
further request from rails ) and have a instance
variable ( protected by mutex ) which is updated on progress and
can be send to rails.

- With new backgroundrb, progress bar would be:
process your damn request and just use register_status() to
register status of your worker. Just because
you are doing some processing won't mean that your worker will
block. It can still receive requests from rails.

- Now, you can schedule multiple methods with their own triggers.

| :schedules:
| :foo_worker:
| :foobar:
| :trigger_args: */5 * * * * * *
| :data: Hello World
| :barbar:
| :trigger_args: */10 * * * * * *

- Inside each worker, you can start tcp server or connect to a
external server. Two important methods available in all workers are:

start_server("localhost",port,ModuleName)
connect("localhost",port,ModuleName)

Connected client or outgoing connection would be integrated with
Event Loop and you can process requests from these guys
asynchronously. This mouse trap can allow you to build truly
distributed workers across your network.

- Each worker comes with a "thread_pool" object, which can be used
to run tasks concurrently. For example:

thread_pool.defer(url) { |url| scrap_wiki_content(url) }

- Each worker has access to method "register_status" which can be
used to update status of worker or store results. Results of a worker
can be retrieved even after a worker has died.

By default the results would be saved in master process memory, but
you can configure BackgrounDRb to store these results in a memcache
server or a cluster using following option in configuration file:

# backgroundrb.yml

| :backgroundrb:
| :port: 11006
| :ip: 0.0.0.0
| :log: foreground
| :result_storage:
| :memcache: "10.10.10.2:11211,10.10.10.6:11211"

- Relevant URLs:
** Home Page: http://backgroundrb.rub...
** SVN : http://svn.devjavu.com/backgrou...
** Bug Reports/Ticks: http://backgroundrb.devjavu....

- Credits :
** Ezra Zygmuntowicz,skaar for taking BackgrounDRb so far.
** Kevin for helping out with OSX issues.
** Andy for patches and initial testing.
** Paul for patching up README.
** Other initial users.
** Matz, Francis for general inspiration.

--
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://g...

4 Answers

hemant

12/17/2007 2:59:00 PM

Hi

On Dec 17, 2007 7:47 PM, Josh Symonds <veraticus@gmail.com> wrote:
> Awesome, thanks for all the work, Hemant! Does this release also fix the
> issue with BackgrounDRb writing ActiveRecord messages to the development
> log?

Yes, John,

The issue you mention has been fixed in the latest release.

>
>
>
> On Dec 17, 2007 6:23 AM, hemant < gethemant@gmail.com> wrote:
> >
> >
> >
> > Hi Folks,
> >
> > I am glad to announce 1.0 release of BackgrounDRb.
> >
> > This would be a major release since 0.2 release of BackgrounDRb.
> >
> > Here is a brief summary of changes:
> >
> > - BackgrounDRb is DRb no longer. It makes use of EventDriven network
> > programming library packet ( http://packet.goog... ).
> >
> > - Since we moved to packet, many nasty thread issues, result hash
> > corruption issues are totally gone. Lots of work has went in
> > making scheduler rock solid stable.
> >
> > - Each worker, still runs in its own process, but each worker has a
> > event loop of its own and all the events are triggered by the internal
> > reactor loop. In a nutshell, you are not encouraged to use threads
> > in your workers now. All the workers are already concurrent, but you
> > are encouraged to use co-operative multitasking, rather than
> > pre-emptive. A simple example is,
> >
> > For implement something like progress bar in old version of bdrb, you
> would:
> > - start your processing a thread (so as your worker can receive
> > further request from rails ) and have a instance
> > variable ( protected by mutex ) which is updated on progress and
> > can be send to rails.
> >
> > - With new backgroundrb, progress bar would be:
> > process your damn request and just use register_status() to
> > register status of your worker. Just because
> > you are doing some processing won't mean that your worker will
> > block. It can still receive requests from rails.
> >
> >
> > - Now, you can schedule multiple methods with their own triggers.
> >
> > | :schedules:
> > | :foo_worker:
> > | :foobar:
> > | :trigger_args: */5 * * * * * *
> > | :data: Hello World
> > | :barbar:
> > | :trigger_args: */10 * * * * * *
> >
> > - Inside each worker, you can start tcp server or connect to a
> > external server. Two important methods available in all workers are:
> >
> > start_server("localhost",port,ModuleName)
> > connect("localhost",port,ModuleName)
> >
> > Connected client or outgoing connection would be integrated with
> > Event Loop and you can process requests from these guys
> > asynchronously. This mouse trap can allow you to build truly
> > distributed workers across your network.
> >
> > - Each worker comes with a "thread_pool" object, which can be used
> > to run tasks concurrently. For example:
> >
> > thread_pool.defer(url) { |url| scrap_wiki_content(url) }
> >
> > - Each worker has access to method "register_status" which can be
> > used to update status of worker or store results. Results of a worker
> > can be retrieved even after a worker has died.
> >
> > By default the results would be saved in master process memory, but
> > you can configure BackgrounDRb to store these results in a memcache
> > server or a cluster using following option in configuration file:
> >
> > # backgroundrb.yml
> >
> > | :backgroundrb:
> > | :port: 11006
> > | :ip: 0.0.0.0
> > | :log: foreground
> > | :result_storage:
> > | :memcache: "10.10.10.2:11211,10.10.10.6:11211"
> >
> >
> > - Relevant URLs:
> > ** Home Page: http://backgroundrb.rub...
> > ** SVN : http://svn.devjavu.com/backgrou...
> > ** Bug Reports/Ticks: http://backgroundrb.devjavu....
> >
> > - Credits :
> > ** Ezra Zygmuntowicz,skaar for taking BackgrounDRb so far.
> > ** Kevin for helping out with OSX issues.
> > ** Andy for patches and initial testing.
> > ** Paul for patching up README.
> > ** Other initial users.
> > ** Matz, Francis for general inspiration.
> >
> >
> > --
> > Let them talk of their oriental summer climes of everlasting
> > conservatories; give me the privilege of making my own summer with my
> > own coals.
> >
> > http://g...
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel@rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgrou...
> >
>
>

--
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://g...

James Herdman

12/28/2007 7:52:00 PM

On Dec 17, 7:23 am, hemant <gethem...@gmail.com> wrote:
> Hi Folks,
>
> I am glad to announce 1.0 release ofBackgrounDRb.
>
> This would be a major release since 0.2 release ofBackgrounDRb.
>
> Here is a brief summary of changes:
>
> -BackgrounDRbis DRb no longer. It makes use of EventDriven network
> programming library packet (http://packet.goog...).
>
> - Since we moved to packet, many nasty thread issues, result hash
> corruption issues are totally gone. Lots of work has went in
> making scheduler rock solid stable.
>
> - Each worker, still runs in its own process, but each worker has a
> event loop of its own and all the events are triggered by the internal
> reactor loop. In a nutshell, you are not encouraged to use threads
> in your workers now. All the workers are already concurrent, but you
> are encouraged to use co-operative multitasking, rather than
> pre-emptive. A simple example is,
>
> For implement something like progress bar in old version of bdrb, you would:
> - start your processing a thread (so as your worker can receive
> further request from rails ) and have a instance
> variable ( protected by mutex ) which is updated on progress and
> can be send to rails.
>
> - With newbackgroundrb, progress bar would be:
> process your damn request and just use register_status() to
> register status of your worker. Just because
> you are doing some processing won't mean that your worker will
> block. It can still receive requests from rails.
>
> - Now, you can schedule multiple methods with their own triggers.
>
> | :schedules:
> | :foo_worker:
> | :foobar:
> | :trigger_args: */5 * * * * * *
> | :data: Hello World
> | :barbar:
> | :trigger_args: */10 * * * * * *
>
> - Inside each worker, you can start tcp server or connect to a
> external server. Two important methods available in all workers are:
>
> start_server("localhost",port,ModuleName)
> connect("localhost",port,ModuleName)
>
> Connected client or outgoing connection would be integrated with
> Event Loop and you can process requests from these guys
> asynchronously. This mouse trap can allow you to build truly
> distributed workers across your network.
>
> - Each worker comes with a "thread_pool" object, which can be used
> to run tasks concurrently. For example:
>
> thread_pool.defer(url) { |url| scrap_wiki_content(url) }
>
> - Each worker has access to method "register_status" which can be
> used to update status of worker or store results. Results of a worker
> can be retrieved even after a worker has died.
>
> By default the results would be saved in master process memory, but
> you can configureBackgrounDRbto store these results in a memcache
> server or a cluster using following option in configuration file:
>
> #backgroundrb.yml
>
> | :backgroundrb:
> | :port: 11006
> | :ip: 0.0.0.0
> | :log: foreground
> | :result_storage:
> | :memcache: "10.10.10.2:11211,10.10.10.6:11211"
>
> - Relevant URLs:
> ** Home Page:http://backgroundrb.rub...
> ** SVN :http://svn.devjavu.com/backgrou...
> ** Bug Reports/Ticks:http://backgroundrb.devjavu....
>
> - Credits :
> ** Ezra Zygmuntowicz,skaar for takingBackgrounDRbso far.
> ** Kevin for helping out with OSX issues.
> ** Andy for patches and initial testing.
> ** Paul for patching up README.
> ** Other initial users.
> ** Matz, Francis for general inspiration.
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://g...

A bit of a late question, but is it fairly difficult to migrate old
workers and tests to this new version of BDrb?

James H.

hemant

12/29/2007 7:33:00 AM

Hi

On Dec 29, 2007 1:25 AM, James H. <james.herdman@gmail.com> wrote:
> A bit of a late question, but is it fairly difficult to migrate old
> workers and tests to this new version of BDrb?

No, it shouldn't be difficult. There are incompatibilities and all are
documented. You should read the README file and probably find out
yourselves.

James Herdman

1/1/2008 6:26:00 PM

On Dec 29 2007, 2:33 am, hemant <gethem...@gmail.com> wrote:
> Hi
>
> On Dec 29, 2007 1:25 AM, James H. <james.herd...@gmail.com> wrote:
>
> > A bit of a late question, but is it fairly difficult to migrate old
> > workers and tests to this new version of BDrb?
>
> No, it shouldn't be difficult. There are incompatibilities and all are
> documented. You should read the README file and probably find out
> yourselves.

Thanks Hemant. I look forward to investigating further. I'm just
trying to figure out how valuable it is for me to re-code my workers.

Happy new year, and thank-you again for your time and hard work!

James

comp.lang.ruby

[ANN] BackgrounDRb release 1.0 available now

hemant

hemant

James Herdman

hemant

James Herdman

x Login to ForumsZone