[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Mayayana

7/26/2011 1:33:00 AM

Synopsis:

I'm working on a mime filter aimed at blind people
and those who may be stuck with IE. The basics
are now done and working, but I may need DLL
multithreading. The gist of it is that the mime filter
functionality is a bit like a bucket brigade. When IE
asks for another bucket, I ask urlmon. But my bucket
gets filled a cup at a time. Then I need to filter the
"water". Once I'm ready I started handing the water off
to IE a cup at a time. So far it works fine. It's designed
to be asynchronous. But I think that I may need to be
able to respond to IE calls while I'm processing. In other
words, when my bucket is half-full, if the server is slow,
IE might give up. I can return the self-explanatory
E_PENDING message to keep IE on the line, but only if
my operations are on another thread. This has to be within
a DLL because of the repeated reads from urlmon. I can't
use an AxEXE.

I started researching multithreading. I've got Matthew
Curland's code, but it's very bulky, and I don't even know
if it's still up-to-date. I also found discussions between Olaf
and others, but that all seemed to be about Ax EXEs.

So, my question: Does anyone know of a tried and true,
reasonably simple multithreading option inside a DLL?
(Please stop sighing, Ralph. :)


31 Answers

ralph

7/26/2011 5:29:00 PM

0

On Mon, 25 Jul 2011 21:32:38 -0400, "Mayayana"
<mayayana@invalid.nospam> wrote:

>Synopsis:
>
> I'm working on a mime filter aimed at blind people
>and those who may be stuck with IE. The basics
>are now done and working, but I may need DLL
>multithreading. The gist of it is that the mime filter
>functionality is a bit like a bucket brigade. When IE
>asks for another bucket, I ask urlmon. But my bucket
>gets filled a cup at a time. Then I need to filter the
>"water". Once I'm ready I started handing the water off
>to IE a cup at a time. So far it works fine. It's designed
>to be asynchronous. But I think that I may need to be
>able to respond to IE calls while I'm processing. In other
>words, when my bucket is half-full, if the server is slow,
>IE might give up. I can return the self-explanatory
>E_PENDING message to keep IE on the line, but only if
>my operations are on another thread. This has to be within
>a DLL because of the repeated reads from urlmon. I can't
>use an AxEXE.
>

Not convinced that is true.

> I started researching multithreading. I've got Matthew
>Curland's code, but it's very bulky, and I don't even know
>if it's still up-to-date. I also found discussions between Olaf
>and others, but that all seemed to be about Ax EXEs.
>

The one nice thing about a platform that has been dropped - everything
is still up-to-date as there is nothing newer. <g>

Yes it is bulky - but you want 'robust', correct?

Many developers pass over ActiveX Exes simply because they don't want
an extra exe/process. Seems a reasonable complaint - but it is often
the most robust and simplest method for supporting multitasking in VB.
The results are seldom as much of a performance-killer as one
imagines.

> So, my question: Does anyone know of a tried and true,
>reasonably simple multithreading option inside a DLL?
> (Please stop sighing, Ralph. :)
>

Ha. No sighs. While the subject itself has been beaten to death, some
interesting new nugget often appears in any long discussion.

Multithreading in Windows is not that complex nor as much of a 'Black
Art' as it may first appear or often portrayed. Multithreading is
Windows' middle name. The multitasking ability of Windows is built
around managing threads not processes. All VB applications are
multithreaded for example - each has a UI thread and a process thread.

The problem with multitasking in VB is 1) it doesn't have any native*
support of multithreading; 2) every running 'VB unit' is fundamentally
a STA - thus only one thread is alive at any one time anyway; and 3)
the VBVM/Runtime is not thread-safe (the primary reason for crashes).

* One often hears the ActiveX Exe as being VB's "native" support for
multithreading, and in a sense it is, since that is the one
mutltithreading model VB does support quite well.

The problem with multitasking in general is wrapped around that last
term - "the model". Simply saying one would like something else
perform some task is analogous to the boss coming up and saying "Here
is an new assistant. Put him to work." Never as easy as it sounds.

What do you want him to do?
That is key. Often what you may initially think is the best use of her
time is not.
Is what he's doing going to take a long time (infrequent "finishes")
or do you need to plan for frequent interuptions during the day?
What do you do when she is finished?
Stop what you are doing and take care of it, or queue for later?
If the former how do you stop and save so you can go back to where you
where? If the later how do you remind yourself new stuff is available?
How does she let you know she is finished? Ring a bell? Send and
EMail? Drop it in your In Box?
All this amounts to a pile of BRs and protocols that make up a "model"
for your problem domain - and guess what? There is no OSFA! The Devil
is in the details.

Here is probably the simplest most robust way to do "multitasking in
VB": (Note: I don't care for his snide remarks but the code does show
how simple it can be - *Assuming* an infrequent callback will fit into
YOUR model.)
http://www.planet-source-code.com/vb/scripts/ShowCode.asp?txtCodeId=24695&am...

Don't over concern yourself with any one mechanism. Don't saddle
yourself with arbitrary pre-conditions. Build your 'model' (legal pad,
whiteboard, pizza box, ...) at that point - which technology or
mechanism you need to use will become self-evident. As always, the
"what" before the "how".

-ralph

Mayayana

7/26/2011 6:06:00 PM

0

| > This has to be within
| >a DLL because of the repeated reads from urlmon. I can't
| >use an AxEXE.
| >
|
| Not convinced that is true.
|

I wish it weren't. An ActvieX EXE would probably be
serviceable otherwise. But I have two things to do:
Reading data from urlmon and editing the webpage
bytes after I have them. The latter is straightforward.
I could put that external. But the read is a looping
call with pointers:

Obj.Read(ByVal pv As Long, ByVal cb As Long, pcbRead As Long)

Obj is a pointer to an IInternetProtocol interface that
urlmon sends in. I don't see how I could send that pointer
to an external process.

The page you linked looks interesting. If I just wrap
that redirect in the C++ DLL it's safe? I don't have any
experience with this. How do I release the thread when
I'm done? Can I somehow call an object? What I need
to do is to call a function to do the data read and then
return me a byte array. That function will call sub-functions.
In other words, the 2nd thread needs to make the
repeated calls to IInternetProtocol.Read and then process
the entire file, while the first thread answers the IE calls
with E_PENDING. It would be easiest to do all that in
a class. Though it doesn't have to be in a class.
If I have it in a .bas, can the 2nd thread call other functions
in the .bas? Or even maybe call into a class on its own?
Can it pass an array to the 1st thread? Or perhaps it could
pass a pointer to array(0) and I could copy the array out?
I'm not clear about how scope fits with threading. Is the
2nd thread in the same scope, at all levels, as the first
thread? Can it access, or be passed, the IInternetProtocol
pointer?


(Mike Mitchell)

7/27/2011 6:44:00 AM

0

On Mon, 25 Jul 2011 21:32:38 -0400, "Mayayana"
<mayayana@invalid.nospam> wrote:

> I started researching multithreading. I've got Matthew
>Curland's code, but it's very bulky, and I don't even know
>if it's still up-to-date. I also found discussions between Olaf
>and others, but that all seemed to be about Ax EXEs.
>
> So, my question: Does anyone know of a tried and true,
>reasonably simple multithreading option inside a DLL?
> (Please stop sighing, Ralph. :)

Why not ask Olaf Schmidt? He helped me enormously with his
DirectCOM.Dll for one of my apps. See http://www.thecommon.n...

MM

ralph

7/27/2011 6:45:00 AM

0

On Wed, 27 Jul 2011 07:43:36 +0100, MM <kylix_is@yahoo.co.uk> wrote:

>On Mon, 25 Jul 2011 21:32:38 -0400, "Mayayana"
><mayayana@invalid.nospam> wrote:
>
>> I started researching multithreading. I've got Matthew
>>Curland's code, but it's very bulky, and I don't even know
>>if it's still up-to-date. I also found discussions between Olaf
>>and others, but that all seemed to be about Ax EXEs.
>>
>> So, my question: Does anyone know of a tried and true,
>>reasonably simple multithreading option inside a DLL?
>> (Please stop sighing, Ralph. :)
>
>Why not ask Olaf Schmidt? He helped me enormously with his
>DirectCOM.Dll for one of my apps. See http://www.thecommon.n...
>

I second that motion.

-ralph

Mayayana

7/27/2011 1:52:00 PM

0


| Why not ask Olaf Schmidt? He helped me enormously with his
| DirectCOM.Dll for one of my apps. See http://www.thecommon.n...
|

I certainly welcome any help from Olaf. :)
I don't think his existing tools can help, though.
They don't really have documentation, so I'm
not sure, but as far as I can see all of his
threading functionality is about getting an AxDLL
class running on a separate thread. If I could use
that I could use an AxEXE. But the problem is that
I need to pass an interface pointer to the function
in thread 2. I'm not certain about variable scope
in that case, but I don't think I can pass that
pointer to an external library. I'm assuming that
it gets put into a relative address that won't
translate.


Jim Mack

7/27/2011 2:44:00 PM

0

Mayayana wrote:
>> Why not ask Olaf Schmidt? He helped me enormously with his
>> DirectCOM.Dll for one of my apps. See
>> http://www.thecommon.n...
>>
>
> I certainly welcome any help from Olaf. :)
> I don't think his existing tools can help, though.
> They don't really have documentation, so I'm
> not sure, but as far as I can see all of his
> threading functionality is about getting an AxDLL
> class running on a separate thread. If I could use
> that I could use an AxEXE. But the problem is that
> I need to pass an interface pointer to the function
> in thread 2. I'm not certain about variable scope
> in that case, but I don't think I can pass that
> pointer to an external library. I'm assuming that
> it gets put into a relative address that won't
> translate.

The issue with separate processes is a valid one, but an AxDLL runs
in-process. In that sense it isn't an "external" library. Objects /
handles are valid across threads, just not across processes.

--
Jim

Mayayana

7/27/2011 3:20:00 PM

0


| The issue with separate processes is a valid one, but an AxDLL runs
| in-process. In that sense it isn't an "external" library. Objects /
| handles are valid across threads, just not across processes.
|

Ah, thanks. I wondered about that but didn't
want to spend a half day experimenting to figure
it out. So it looks like I can use Olaf's DirectCom.Dll
STARTCOMOBJECT method in an external DLL if
I don't get internal threading solved.


Mayayana

7/27/2011 10:20:00 PM

0

To Olaf, or anyone else who knows about
his dhRichClient3.dll:

I'm able to call a function in a DLL from
within a class of the first DLL using
cThreadHandler. The function can show
me a msgbox. But I can't get a return either
async or synchronous. Is it possible that
this can only be done from an EXE?


Schmidt

7/30/2011 7:35:00 PM

0

Am 28.07.2011 00:19, schrieb Mayayana:
> To Olaf, or anyone else who knows about
> his dhRichClient3.dll:
>
> I'm able to call a function in a DLL from
> within a class of the first DLL using
> cThreadHandler. The function can show
> me a msgbox. But I can't get a return either
> async or synchronous. Is it possible that
> this can only be done from an EXE?

I would not use this "high-level" threading-approach,
which is built into the RichClient3 - since this
one is thought more for something like "asynchronously
running Background-Workers" - then throwing Events for
"Progress" and "Finished" from the BackGround-Thread.

For working more near "to the metal" (which I think,
is important in your case) you should use the
approach which is built into DirectCOM.dll
(no dependency to the larger RichClient-lib is
involved then).

And as you already guessed correctly -
the STARTCOMOBJECT-call of DirectCOM.dll is what
you perhaps need (or should try).
This call is used in conjunction with your own
"MyThreadHandler.dll", which only hosts one
Public Class. And this Public Class needs to
provide (by convention) a Public Function:

Public Function ThreadMain(ByVal P As Long) As Long

as an entry-point for the thread - and
in 'P' you could provide a Pointer to Shared
memory for example (memory which you previously
allocated within your Main-Thread).

What's nice is - if you have your "normal unthreaded"
code already within an ActiveX-Dll - then you can
easily place such an additional (Public) ThreadClass
within that same Dll - and then call:
STARTCOMOBJECT against your very own DllFileName...
"from within" (your normal Dll-Class).

Example:
If your current Dll is named "MyIEFilter.dll"
and currently has one Public Class defined:
cMyFilter

you could then simply add an additional ThreadClass:
cMyFilterThread (this is the one which runs threaded,
and has this Public ThreadMain-Function defined).

Plus maybe an additional Private-Class (that's the way
I usually do it), which is responsible for starting up -
and later on also properly cleanup the additional
ThreadClass - this private Handler-Class could be named:
cMyFilterThreadHandler

Within this Private Class cMyFilterThreadHandler
(which runs in your normal MainThread) you can
ensure proper Startup (in Class_Initialize)
and proper Teardown (in Class_Terminate)
against your real ThreadClass (cMyFilterThread).
So these two classes behave somewhat like "linked
Classes" then - if you instantitate
cMyFilterThreadHandler in your MainThread,
then the "threaded ShadowClass" (cMyFilterThread)
is automatically also "alive" ... if you
set cMyFilterThreadHandler to Nothing, then
also cMyFilterThread (and of course also the thread
it is running on) will be destroyed.

So, if you want to use a Thread from within
cMyFilter, you would simply start your
Thread with:
Dim ThreadHandler As cMyFilterThreadHandler
Set ThreadHandler = New cMyFilterThreadHandler

after that the additional thread is running

and as soon as the ThreadHandler-Variable
goes out of scope - or is set to Nothing,
also the Thread (which runs the cMyFilterThread-
Class) will terminate.

All that, as said - possible directly within
one and the same Dll (and by placing the
appropriate Startup- and Teardown-Code within
_Initialize and _Terminate of your Handler-Class.

What (as always with threads, no matter what
language) makes the whole thing tricky is,
that you will need to find a good way, to cancel
an actually running operation as fast and clean
as possible, when it's a longer one, and performed
"in the thread" - this is, when you set the
Private Handler-Class to Nothing - and have
to ensure a proper teardown of the upstarted
ThreadClass by signaling this class, that
the MainThread "goes down" - or just wants to
get rid of the thread as fast as possible -
and signaling from the Handler-CLass to its
associated ThreadClass is usually not the
problem (e.g. in Shared Memory you can set
a simple Flag really immediately) - the
question is, if the currently (possibly a
long-run-task-performing) thread, does
have the time, to react to this "ShutDOwn-
Flag which was set "from outside" (from the
Mainthreads THreadHandler-Class).
When the ThreadClass just performed an internal
Call, which takes "seconds" to return, then
it can of course not look at this ShutDown-
Flag - and exit the ThreadMain-Function -
the Handler-Class has then (in its Class_Terminate)
either to wait appropriately - or - after some
TimeOut you find useful, perform a hard shutdown
using TerminateThread (which is not recommended,
whilst working with COM-stuff, COM-routines which
the thread currently was yet "diving in").

I don't know, what you have available in
these Filter-APIs, to cancel the processing
within the thread - or if they are "granular"
in a way, that you don't have that long
"taskrunner-calls", but only relative short
callbacks (which perform some tasks on a relative
short buffer and then return, leaving the thread
free (idling), to take a look at the ShutDown-Flag.

But perhaps you don't need any thread at all -
I cannot imagine that the APIs (the interfaces
to implement) didn't thought about possibly
slow-incoming data (from the server-end).
Without the filter, the IE would also have
to wait for the next small chunk, when it's
only "trickling in".
And *if* there's a filter inbetween, then
you're acting, as if you would be the server -
and if this, your "virtual socket" (your filter)
has no new data available at the moment
(when requested from the IE-end) - then
you should simply pass a (the same) value back,
which otherwise the "async-Downloader-Class which
would talk directly with the server" also would
have to pass back to the IE-request-handler.

Hmm, I'd like to have a look at your code...

Olaf

Mayayana

7/30/2011 10:22:00 PM

0

Thanks, Olaf.
After spending a couple of days trying to get
multi-threading working, I did more testing. (The
multi-threading was difficult because I don't really
understand the APIs involved. And even if I got
it working, I'm needing to pass in pointers and
return a byte array. It's fairly complex.)

After more testing I think you may be right
about the multi-threading not being necessary.
Unfortunately, MS has very little documentation
of all this. The basic deal is that IE calls in for
data, specifiying the number of bytes it wants.
The MS mimefilt sample code then calls the
incoming IInternetProtocol object for that number
of bytes and passes them on. It continues like that
until the page is downloaded. But for my purposes
I need to get the whole page in one piece, for
processing, before I give it to IE. Common sense would
say that the E_PENDING message I can send to IE
would tell it, "don't go away...be right with you".
But noooooooo. :) When the incoming object sends
me an E_PENDING it's the same as a success message
in that it has filled the buffer with data! That makes me
think that sending E_PENDING to IE probably serves
no purpose, as you suggested. So at this point I'm
proceeding accordingly. In tests online I've loaded a
140 KB webpage and sent it through my filter class
(which only takes a few ms) and IE has no trouble
loading that page.

I'll save your notes in case it looks like the multi-
threading will be needed, but at this point I don't
even see how I could use it. If IE expects data with
an E_PENDING return then there seems to be no way
to send a "be right with you" message.

I was going to put a package on my website once
this is done, with basic mime filter code. While mime
filters and protocol handlers provide vast possibilities,
they're obscure, with very little documentation. The little
that I've found online is mainly for protocol handlers,
which work a bit differently. An operation called "bsalsa"
has mime filter code in Delphi, but it's so radically different
from what MS does -- and from what works for me --
that I can't imagine their code actually works. That means
the mimefilt sample is really the only existing sample code.

At this point the basic program I'm making involves the
mime filter DLL and a browser extension to allow for settings
access within IE. The filtering and GUI are still in the works.
But I'd be happy to send you what I have for a filter now,
if you like. The actual filter itself is mainly just a fairly
simple class, a .bas that contains redirection back inside
for the redirected vTable pointers, and a few declarations.

At some point I hope to write up the details, but I haven't
got that far yet. There seem to be a lot of things that either
don't work as advertised or are simply not documented.
Fortunately, most of the class functions will tolerate a msgbox
without causing trouble, and the DLL can be recompiled
as long as no Explorer windows are open. That allows for
relatively easy debugging.