Question about Tcl event loop/Tcl_EventQueue and friends
in [TCL]
|
From: APN on 28 Oct 2009 04:20 Background: in my TWAPI extension, I call back the application to notify it of certain events, such as hotkeys, file changes, service control events etc. Depending on the source of the notification, the trigger may be in the same thread as the interpreter or a different thread. The notification is therefore implemented through the use of Tcl_AsyncMark in my system handler for the event. Then when the Tcl event loop calls back into my AsyncProc, I call Tcl_QueueEvent. It's only when Tcl calls my event handler, I actually do the Tcl_Eval of a script to notify the application. This all seems to work well but I had the following questions - Do I really need to do a Tcl_QueueEvent from my AsyncProc or can I directly do the Tcl_Eval in the asyncproc itself? The documentation seems to imply I could do that provided I am careful to save/restore the existing interp result but I'm still nervous. Note that asyncproc may get passed a NULL interp, but I actually stash away the interp that had originally made the notification request and would invoke that, not what was passed into asyncproc. In any case the question is, am I unnecessarily adding a layer of Tcl_QueueEvent/handle event ? More important, I now want to add the capability for the callback script to return a value to be passed to the system. For example, when a request to remove CD-ROM media is sent by the system, I want the application to be able to allow/deny the request. What currently happens is that when the application wants media notifications, a hidden window is created. The hidden window proc (WndProc) gets WM_DEVICECHANGE notifications as a result of the standard Tcl windows GetMessage/TranslateMessage/DispatchMessage loop in Tcl_WaitForEvent. My window handler then follows the AsyncMark-AsyncProc-QueueEvent- HandleEvent sequence described above. The problem is this does not allow for the window handler WndProc to return the script result value as its return code to allow/deny the media change request. So I need to know if the following is a feasible alternative - Directly call Tcl_Eval (using the interp stashed away in the window private area) from my WndProc and return the result of that call as my WndProc return value to allow/denyerequest. The stack at that point looks like Tcl_WaitForEvent - DispatchMessage - os stuff - WndProc - Tcl_Eval. No AsyncMark/Tcl_EventQueue etc. This would be the simplest alternative. If I take care to save and restore interp state, is this kosher ? Or am I going to break some event loop assumptions ? My only other alternative is to move the device change notification window to another thread which can then block awaiting the Tcl event loop to do its thing. This is not attractive for a lot of reasons. Comments/advice much appreciated. /Ashok
From: David Gravereaux on 28 Oct 2009 05:37 APN wrote: ... > Do I really need to do a Tcl_QueueEvent from my AsyncProc or can I > directly do the Tcl_Eval in the asyncproc itself? The documentation > seems to imply I could do that provided I am careful to save/restore > the existing interp result but I'm still nervous. Be nervous. I don't believe the documentation. You don't know what the stack frame depth is or how strange it may be from [Incr Tcl] having applied its magic. The granularity is rather fine for where the interp has been yielded. You just don't know where you are. >... In any case the question is, > am I unnecessarily adding a layer of Tcl_QueueEvent/handle event ? No, that layer is supposed to be there. You want your outside work to get inserted into the event loop. Tk puts its work there, sockets, pipes, etc.. It is the proper place to insert new background work that eval scripts. > More important, I now want to add the capability for the callback > script to return a value to be passed to the system. For example, when > a request to remove CD-ROM media is sent by the system, I want the > application to be able to allow/deny the request. What currently > happens is that when the application wants media notifications, a > hidden window is created. The hidden window proc (WndProc) gets > WM_DEVICECHANGE notifications as a result of the standard Tcl windows > GetMessage/TranslateMessage/DispatchMessage loop in Tcl_WaitForEvent. > My window handler then follows the AsyncMark-AsyncProc-QueueEvent- > HandleEvent sequence described above. The problem is this does not > allow for the window handler WndProc to return the script result value > as its return code to allow/deny the media change request. > > So I need to know if the following is a feasible alternative - > Directly call Tcl_Eval (using the interp stashed away in the window > private area) from my WndProc and return the result of that call as my > WndProc return value to allow/denyerequest. The stack at that point > looks like Tcl_WaitForEvent - DispatchMessage - os stuff - WndProc - > Tcl_Eval. No AsyncMark/Tcl_EventQueue etc. This would be the simplest > alternative. If I take care to save and restore interp state, is this > kosher ? Or am I going to break some event loop assumptions ? > > My only other alternative is to move the device change notification > window to another thread which can then block awaiting the Tcl event > loop to do its thing. This is not attractive for a lot of reasons. > > Comments/advice much appreciated. > > /Ashok > From your stack trace, you show you are already in Tcl's thread context. IOW, you called CreateWindowEx() from the thread that is driving Tcl, thus Tcl_WaitForEvent() will service the wndProc for it. If that's true, you don't need Tcl_AsyncMark at all. Just skip to Tcl_QueueEvent(), but... I don't think it would be safe to directly call Tcl_Eval there either. The script callback could get you into trouble by not allowing the stack to unwind. Generally, it would go against the normal way things are run and you'd put putting yourself at a higher priority than everyone else. I see the problem, though. You can't block on an event/semaphore waiting for the eventProc to come around to answer the question as you would be blocking the main processing thread, thus the entire world comes to a halt. Sounds like a job for more threads, sorry. Put your hidden window in a new thread that you can block on an event waiting for the answer. Use the Tcl_AsyncMark/asyncProc/Tcl_QueueEvent/eventProc method to get the job over to Tcl so it can reply back to the blocked thread. Yeah, that is a lot of code, but well worth it, IMO. Thanks for the explanation, too, it rings (rather old) bells for me. --
From: Donal K. Fellows on 28 Oct 2009 06:15 On 28 Oct, 08:20, APN <palm...(a)yahoo.com> wrote: > Do I really need to do a Tcl_QueueEvent from my AsyncProc or can I > directly do the Tcl_Eval in the asyncproc itself? There has been fighting over that API (the AsyncProc stuff) as to what it actually means between those who use it for dealing with Unix signals and those who use it for munging stuff with threads. I don't think we've ever really resolved this either; it's just that there are very few people who understand all the issues so the discussion only rarely proceeds at all. FWIW, the classic single-threaded (Tcl 7.*) use of Tcl_AsyncMark was in a Unix signal handler to tell the interpreter that it would need to run some code when it got to a predictable state. That code (the AsyncProc) could use Tcl_Eval if it chose and could generate an error that would be seen by the normal execution of the interpreter. (The key use case was being able to break into busy loops with SIGINT, i.e., Ctrl+C.) Donal.
From: APN on 28 Oct 2009 10:26 Thanks for the confirmation though it's not the answer I wanted to hear! The code for doing the whole async/queue dance from another thread is already in twapi as it's needed for other components like services so doing device notifications that way should not be too hard. But I don't like the idea of two thread switches and four queuing operations just to tell Windows not to eject a CD-ROM just yet! But if that's how it's gotta be, that's how it's going to be. /Ashok On Oct 28, 2:37 pm, David Gravereaux <davyg...(a)pobox.com> wrote: > APN wrote: > > ... > > > Do I really need to do a Tcl_QueueEvent from my AsyncProc or can I > > directly do the Tcl_Eval in the asyncproc itself? The documentation > > seems to imply I could do that provided I am careful to save/restore > > the existing interp result but I'm still nervous. > > Be nervous. I don't believe the documentation. You don't know what the > stack frame depth is or how strange it may be from [Incr Tcl] having > applied its magic. The granularity is rather fine for where the interp > has been yielded. You just don't know where you are. > > >... In any case the question is, > > am I unnecessarily adding a layer of Tcl_QueueEvent/handle event ? > > No, that layer is supposed to be there. You want your outside work to > get inserted into the event loop. Tk puts its work there, sockets, > pipes, etc.. It is the proper place to insert new background work that > eval scripts. > > > > > More important, I now want to add the capability for the callback > > script to return a value to be passed to the system. For example, when > > a request to remove CD-ROM media is sent by the system, I want the > > application to be able to allow/deny the request. What currently > > happens is that when the application wants media notifications, a > > hidden window is created. The hidden window proc (WndProc) gets > > WM_DEVICECHANGE notifications as a result of the standard Tcl windows > > GetMessage/TranslateMessage/DispatchMessage loop in Tcl_WaitForEvent. > > My window handler then follows the AsyncMark-AsyncProc-QueueEvent- > > HandleEvent sequence described above. The problem is this does not > > allow for the window handler WndProc to return the script result value > > as its return code to allow/deny the media change request. > > > So I need to know if the following is a feasible alternative - > > Directly call Tcl_Eval (using the interp stashed away in the window > > private area) from my WndProc and return the result of that call as my > > WndProc return value to allow/denyerequest. The stack at that point > > looks like Tcl_WaitForEvent - DispatchMessage - os stuff - WndProc - > > Tcl_Eval. No AsyncMark/Tcl_EventQueue etc. This would be the simplest > > alternative. If I take care to save and restore interp state, is this > > kosher ? Or am I going to break some event loop assumptions ? > > > My only other alternative is to move the device change notification > > window to another thread which can then block awaiting the Tcl event > > loop to do its thing. This is not attractive for a lot of reasons. > > > Comments/advice much appreciated. > > > /Ashok > > From your stack trace, you show you are already in Tcl's thread context. > IOW, you called CreateWindowEx() from the thread that is driving Tcl, > thus Tcl_WaitForEvent() will service the wndProc for it. If that's > true, you don't need Tcl_AsyncMark at all. Just skip to > Tcl_QueueEvent(), but... > > I don't think it would be safe to directly call Tcl_Eval there either. > The script callback could get you into trouble by not allowing the stack > to unwind. Generally, it would go against the normal way things are run > and you'd put putting yourself at a higher priority than everyone else. > > I see the problem, though. You can't block on an event/semaphore > waiting for the eventProc to come around to answer the question as you > would be blocking the main processing thread, thus the entire world > comes to a halt. > > Sounds like a job for more threads, sorry. Put your hidden window in a > new thread that you can block on an event waiting for the answer. Use > the Tcl_AsyncMark/asyncProc/Tcl_QueueEvent/eventProc method to get the > job over to Tcl so it can reply back to the blocked thread. > > Yeah, that is a lot of code, but well worth it, IMO. Thanks for the > explanation, too, it rings (rather old) bells for me. > > -- > > signature.asc > < 1KViewDownload
From: David Gravereaux on 28 Oct 2009 17:06
APN wrote: > Thanks for the confirmation though it's not the answer I wanted to > hear! The code for doing the whole async/queue dance from another > thread is already in twapi as it's needed for other components like > services so doing device notifications that way should not be too > hard. But I don't like the idea of two thread switches and four > queuing operations just to tell Windows not to eject a CD-ROM just > yet! > > But if that's how it's gotta be, that's how it's going to be. Maybe the whole dance should be a new extended set of procedure for Tcl itself? For example, You can skip Tcl_AsyncMark altogether if the core was compiled for thread support and use Tcl_QueueThreadEvent directly and being compiled in, it could know this. That's one less thread-safe linkedlist in use. What would a new set APIs do for us? I'm trying to brain stretch, but can't quite do it right now. Generally, programmers like to package up complexity into manageable little units of simplicity. I haven't written any C/C++ code for almost a year now, so I'm seriously out of practice and I painfully can almost imagine it. -- |