[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

removeall() in list

Aaron Brady

1/11/2008 8:23:00 PM

Any ideas for a thread-safe list.removeall( X ): removing all
occurrences of X within list L, when L might be modified concurrently?

Sincerely,
Aaron
25 Answers

Paul Rubin

1/11/2008 8:58:00 PM

0

castironpi@gmail.com writes:
> Any ideas for a thread-safe list.removeall( X ): removing all
> occurrences of X within list L, when L might be modified concurrently?

That way lies madness. Do something sensible instead. Put a lock
around the list, or put all mutators for the list into a single
thread, or whatever. Don't do what you're describing.

Aaron Brady

1/11/2008 10:30:00 PM

0

On Jan 11, 2:57 pm, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
> castiro...@gmail.com writes:
> > Any ideas for a thread-safe list.removeall( X ): removing all
> > occurrences of X within list L, when L might be modified concurrently?
>
> That way lies madness. Do something sensible instead. Put a lock
> around the list, or put all mutators for the list into a single
> thread, or whatever. Don't do what you're describing.

This function just wants X out of the list. It doesn't matter if this
happens before, during, or after something else; so long as it happens.

Paul Rubin

1/11/2008 11:27:00 PM

0

castironpi@gmail.com writes:
> This function just wants X out of the list. It doesn't matter if this
> happens before, during, or after something else; so long as it happens.

If you're asking in a general way how to do something like that,
there are several basic methods:

1. Put a single thread in charge of the list, and communicate with it
by message passing through Queues. To get X out of the list, you'd
send the mutator thread a message asking for removal. The mutator
thread would loop reading and processing messages from the queue,
blocking when no requests are pending. This is sort of the preferred
Python style and is pretty simple to get correct, but if there are
many such objects you can end up with more threads than you really
want.

2. Associate a lock with the list. Anything wanting to access the
list should acquire the lock, do its stuff, then release the lock.
This gets confusing after a while.

3. Various other schemes involving finer grained locks etc. that
get horrendously error prone (race conditions etc).

There is probably a good tutorial somewhere about programming with
threads. It's sometimes a tricky subject, so it's worth taking
the time to study various approaches rather than re-inventing the wheeel.

Aaron Brady

1/11/2008 11:44:00 PM

0

On Jan 11, 5:26 pm, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
> castiro...@gmail.com writes:
> > This function just wants X out of the list. It doesn't matter if this
> > happens before, during, or after something else; so long as it happens.
>
> 2. Associate a lock with the list. Anything wanting to access the
> list should acquire the lock, do its stuff, then release the lock.
> This gets confusing after a while.

To keep it generic, how about:

listA.op( insert, x )
listA.op( remove, x )

Aaron Brady

1/11/2008 11:50:00 PM

0

On Jan 11, 5:43 pm, castiro...@gmail.com wrote:
> On Jan 11, 5:26 pm, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
>
> > castiro...@gmail.com writes:
> > > This function just wants X out of the list. It doesn't matter if this
> > > happens before, during, or after something else; so long as it happens.
>
> > 2. Associate a lock with the list. Anything wanting to access the
> > list should acquire the lock, do its stuff, then release the lock.
> > This gets confusing after a while.
>
> To keep it generic, how about:
>
> listA.op( insert, x )
> listA.op( remove, x )

However, in reality, your rock and hard place are:
listA.op( listA.insert, x )
listA.op( listA.remove, x )

or

listA.op( 'insert', x )
listA.op( 'remove', x )

Paul Rubin

1/11/2008 11:51:00 PM

0

castironpi@gmail.com writes:
> > 2. Associate a lock with the list. Anything wanting to access the
> > list should acquire the lock, do its stuff, then release the lock.
> > This gets confusing after a while.
>
> To keep it generic, how about:
>
> listA.op( insert, x )
> listA.op( remove, x )

Sure, there are various ways you can make the code look uniform. What
gets messy is if you want to (say) operate on several lists at the
same time, which means you need to hold multiple locks simultaneously,
and some other thread is also trying to do the same thing. If you
acquire the locks in the wrong order, you can get a situation where
both threads deadlock forever.

Aaron Brady

1/12/2008 12:17:00 AM

0

On Jan 11, 5:51 pm, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
> castiro...@gmail.com writes:
> > listA.op( insert, x )
> > listA.op( remove, x )
>
> Sure, there are various ways you can make the code look uniform. What
> gets messy is if you want to (say) operate on several lists at the
> same time, which means you need to hold multiple locks simultaneously,
> and some other thread is also trying to do the same thing. If you
> acquire the locks in the wrong order, you can get a situation where
> both threads deadlock forever.
And:
> However, in reality, your rock and hard place are:
> listA.op( listA.insert, x )
> listA.op( listA.remove, x )
>
> or
>
> listA.op( 'insert', x )
> listA.op( 'remove', x )

For a standard library, you may not want to be exposing the potential
for deadlock-- not that users can't do that themselves.

lockerA.op( listA.extend, [ x ] )
lockerB.op( listB.reverse )
def thA():
gui.listbox.append( lockerA.op( listA.pop ) )

Aaron Brady

1/12/2008 12:40:00 AM

0

On Jan 11, 6:17 pm, castiro...@gmail.com wrote:
> On Jan 11, 5:51 pm, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
>
>
>
> > castiro...@gmail.com writes:
> > > listA.op( insert, x )
> > > listA.op( remove, x )
>
> > Sure, there are various ways you can make the code look uniform. What
> > gets messy is if you want to (say) operate on several lists at the
> > same time, which means you need to hold multiple locks simultaneously,
> > and some other thread is also trying to do the same thing. If you
> > acquire the locks in the wrong order, you can get a situation where
> > both threads deadlock forever.
> And:
> > However, in reality, your rock and hard place are:
> > listA.op( listA.insert, x )
> > listA.op( listA.remove, x )
>
> > or
>
> > listA.op( 'insert', x )
> > listA.op( 'remove', x )
>
> For a standard library, you may not want to be exposing the potential
> for deadlock-- not that users can't do that themselves.
>
> lockerA.op( listA.extend, [ x ] )
> lockerB.op( listB.reverse )
> def thA():
> gui.listbox.append( lockerA.op( listA.pop ) )

Could you:

lockerA= Locker( listA, listB )
lockerA.op( listB.reverse )
lockerA.op( listA.pop )

Where lockerA ops acquire the locks on all its threads?

Paul Rubin

1/12/2008 2:04:00 AM

0

castironpi@gmail.com writes:
> Could you:
>
> lockerA= Locker( listA, listB )
> lockerA.op( listB.reverse )
> lockerA.op( listA.pop )
>
> Where lockerA ops acquire the locks on all its threads?

I don't understand that question. The main thing to understand is
that multi-threaded programming is complicated (especially if you're
after high performance), and very difficult to get right without
knowing exactly what you're doing. The generally preferred approach
in Python is to keep things simple at some performance cost.

Your Locker approach above looks sort of reasonable if you can be
absolutely sure that nothing else can mess with listA or listB
concurrently with those locker operations. Normally you would put
listA and listB into a single object along with a lock, then do
operations on that object.

You might check the Python Cookbook for some specific recipes and
sample code for this stuff. If you've used Java, Python's general
threading mechanisms are similar, but they are in the library rather
than built into the language (i.e. there is no "synchronized"
keyword, you have to do that locking explicitly).

What is the actual application, if you don't mind saying? Are you
sure that you really need concurrency?

Aaron Brady

1/12/2008 8:37:00 AM

0

On Jan 11, 8:04 pm, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
> castiro...@gmail.com writes:
> > Could you:
>
> > lockerA= Locker( listA, listB )
> > lockerA.op( listB.reverse )
> > lockerA.op( listA.pop )
>
> > Where lockerA ops acquire the locks on all its threads?
>
> I don't understand that question. The main thing to understand is
> that multi-threaded programming is complicated (especially if you're
> after high performance), and very difficult to get right without
> knowing exactly what you're doing. The generally preferred approach
> in Python is to keep things simple at some performance cost.
>
> Your Locker approach above looks sort of reasonable if you can be
> absolutely sure that nothing else can mess with listA or listB
> concurrently with those locker operations. Normally you would put
> listA and listB into a single object along with a lock, then do
> operations on that object.
>
> You might check the Python Cookbook for some specific recipes and
> sample code for this stuff. If you've used Java, Python's general
> threading mechanisms are similar, but they are in the library rather
> than built into the language (i.e. there is no "synchronized"
> keyword, you have to do that locking explicitly).
>
> What is the actual application, if you don't mind saying? Are you
> sure that you really need concurrency?

I'm writing an NxN observer pattern, mostly for my own personal
exploration. Two threads -might- be calling 'Disconnect' at the same
time, and I can't even guarantee that the function runs properly.

for emelem in [ e for e in emlist if e.func is func ]:
try:
emlist.remove( emelem )
except ValueError:
pass