Veloz wrote:
> On Mar 2, 1:18 pm, Raymond Hettinger <pyt...@rcn.com> wrote:
>> On Mar 2, 8:29 am, Veloz <michaelve...@gmail.com> wrote:
>>
>>> Hi all
>>> I'm looking for a queue that I can use with multiprocessing, which has
>>> a peek method.
>>> I've seen some discussion about queue.peek but don't see anything in
>>> the docs about it.
>>> Does python have a queue class with peek semantics?
>> Am curious about your use case? Why peek at something
>> that could be gone by the time you want to use it.
>>
>> val = q.peek()
>> if something_i_want(val):
>> v2 = q.get() # this could be different than val
>>
>> Wouldn't it be better to just get() the value and return if you don't
>> need it?
>>
>> val = q.peek()
>> if not something_i_want(val):
>> q.put(val)
>>
>> Raymond
>
> Yeah, I hear you. Perhaps queue is not the best solution. My highest
> level use case is this: The user visits a web page (my app is a
> Pylons app) and requests a "report" be created. The report takes too
> long to create and display on the spot, so the user expects to visit
> some url "later" and see if the specific report has completed, and if
> so, have it returned to them.
>
> At a lower level, I'm thinking of using some process workers to create
> these reports in the background; there'd be a request queue (into
> which requests for reports would go, each with an ID) and a completion
> queue, into which the workers would write an entry when a report was
> created, along with an ID matching the original request.
>
> The "peek" parts comes in when the user comes back later to see if
> their report has done. That is, in my page controller logic, I'd like
> to look through the complete queue and see if the specific report has
> been finished (I could tell by matching up the ID of the original
> request to the ID in the completed queue). If there was an item in the
> queue matching the ID, it would be removed.
>
> It's since occurred to me that perhaps a queue is not the best way to
> handle the completions. (We're ignoring the file system as a solution
> for the time being, and focusing on in-memory structures). I'm
> wondering now if a simple array of completed items wouldn't be better.
> Of course, all the access to the array would have to be thread/process-
> proof. As you pointed out, for example, multi-part operations such as
> "is such-and-such an ID in the list? If so, remove it and return in"
> would have to be treated atomically to avoid concurrency issues.
>
> Any thoughts on this design approach are welcomed :-)
>
A set of completed reports, or a dict with the ID as the key? The
advantage of a dict is that the value could contain several bits of
information, such as when it was completed, the status (OK or failed),
etc. You might want to wrap it in a class with locks (mutexes) to ensure
it's threadsafe.