[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

Advice needed on parallel processing in python

Glazner

3/24/2010 3:28:00 PM

Hi!

I need to replace an app that does number crunching over a local
network.
it have about 50 computers as slaves
each computer needs to run COM that will do the "job"
right now the system uses MFC threads and DCOM to distribute the load.

as i said, Now i'm trying to replace this system with python.
I already use win32all and read about Pareller Python and Pyro.

a requirement is that messages can be sent to the remote worker.

If anyone can share experience I'll love to hear


Many Thanks,
Yoav Glazner
2 Answers

Jon Clements

3/24/2010 6:13:00 PM

0

On 24 Mar, 15:27, Glazner <yoavglaz...@gmail.com> wrote:
> Hi!
>
> I need to replace an app that does number crunching over a local
> network.
> it have about 50 computers as slaves
> each computer needs to run COM that will do the "job"
> right now the system uses MFC threads and DCOM to distribute the load.
>
> as i said, Now i'm trying to replace this system with python.
> I already use win32all and read about Pareller Python and Pyro.
>
> a requirement is that messages can be sent to the remote worker.
>
> If anyone can share experience I'll love to hear
>
> Many Thanks,
> Yoav Glazner

Would Celery suit? http://celerypr...

hth

Jon.

bobicanprogram

3/25/2010 1:18:00 PM

0

On Mar 24, 1:13 pm, Jon Clements <jon...@googlemail.com> wrote:
> On 24 Mar, 15:27, Glazner <yoavglaz...@gmail.com> wrote:
>
>
>
> > Hi!
>
> > I need to replace an app that does number crunching over a local
> > network.
> > it have about 50 computers as slaves
> > each computer needs to run COM that will do the "job"
> > right now the system uses MFC threads and DCOM to distribute the load.
>
> > as i said, Now i'm trying to replace this system with python.
> > I already use win32all and read about Pareller Python and Pyro.
>
> > a requirement is that messages can be sent to the remote worker.
>
> > If anyone can share experience I'll love to hear
>
> > Many Thanks,
> > Yoav Glazner
>
> Would Celery suit?http://celerypr...
>
> hth
>
> Jon.


If you can add just one Linux server onto that network you could use
SIMPL-Python to do what you want.

http://www.icanpr...06py/lesson1/le...

ie. SIMPL messages going seamlessly from Python (Windows) to Python
(Windows) with the SIMPL sandbox being hosted on the Linux node.

Of course if you wanted to add Linux nodes as processing nodes.
Python (Windows) modules could seamlessly exchange messages with those
as well. Same would be true if you wanted to distribute part of your
app into the cloud. All your Python code would look virtually
identical in each of these cases.

As an example of an inexpensive Linux node the SIMPL toolkit has been
ported to the ~$100 Sheeva Plug computer (http://www.icanpr...
simpl/plugsimplbin.self.html) (http://www.plugco...).

bob