IPC based on name pipe FIFO and transaction log file
in [Unix Programming]
|
Prev: LBW 0.1: Linux Binaries on Windows
Next: socket
From: Peter Olcott on 1 Apr 2010 21:52 "Ersek, Laszlo" <lacos(a)caesar.elte.hu> wrote in message news:Pine.LNX.4.64.1004020234430.11352(a)login01.caesar.elte.hu... > On Thu, 1 Apr 2010, Peter Olcott wrote: > >> I don't see how this does not guarantee all of the >> atomicity that I need. Could you propose a concrete >> example that meets the standard and causes problems? It >> seems like it is saying the entire append must complete >> before any other file modifications take place. > > write() itself (with or without O_APPEND) is not required > to write all bytes at once, even to a regular file. > > http://www.opengroup.org/onlinepubs/9699919799/functions/write.html > > Consider a signal delivered to the thread, a file size > limit reached, or being temporarily out of space on the fs > hosting the file; all after some but not all bytes were > written. (These would return -1 and set errno to EINTR > (without SA_RESTART), EFBIG, ENOSPC, respectively, if no > data could have been written before encountering the > condition in question.) This list is not exhaustive. > > (Signal delivery is plausible -- suppose you submit a > write request of 1G bytes on a system with little buffer > cache and a slow disk. If SSIZE_MAX equals LONG_MAX, for > example, the result of such a request is not even > implementation-defined.) I want to somehow define this process for maximum reliability. Each request will be a customer's purchase, I don't want to lose any requests. The reason that I defined the log file as part of this process was to make these customer requests persistent. The purpose of the named pipe was to provide event driven notification to each process. > > > > ----v---- > > Write requests to a pipe or FIFO shall be handled in the > same way as a regular file with the following exceptions: > > [...] > > * Write requests of {PIPE_BUF} bytes or less shall not be > interleaved with data from other processes doing writes on > the same pipe. [...] > > * If the O_NONBLOCK flag is clear, a write request may > cause the thread to block, but on normal completion it > shall return nbyte. > > [...] > > ----^---- > > Both quoted guarantees (exclusion of interleaved writes, > plus completeness of writes) are exceptional behavior of > pipes in relation to regular files. > > Perhaps an interpretation request should be submitted: if > write() returns nbyte and O_APPEND was set, was the block > written atomically then? > > <http://www.kernel.org/doc/man-pages/online/pages/man2/write.2.html> > does say > > ----v---- > > If the file was open(2)ed with O_APPEND, the file offset > is first set to the end of the file before writing. The > adjustment of the file offset and the write operation are > performed as an atomic step. > > ----^---- > > which seems to imply that the write operation itself is > atomic. (... if it returns "count", in my interpretation.) > > lacos Great thanks.
From: Vitus Jensen on 2 Apr 2010 06:33 Hi David, On Thu, 1 Apr 2010, David Schwartz wrote: > On Apr 1, 4:23 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote: .... > Seems kind of silly to have a thread for each request that spends most > of its time just waiting for another program. You don't need a thread > to wait. Just assign a thread to do what needs to be done when you're > notified that the request is finished being processed by the OCR > software. I'm coming from another platform where the maxime was "a thread should do nothing very well" (PF), which I always interpreted as to code your thread so that they spend most of their time waiting. So yes, you need a thread to wait. 99% of the time it should do nothing else but wait. > I'm saying, don't have one thread waiting to do X when X is possible > and one waiting to do Y when Y is possible and so on. First, this > wastes a lot of threads. Second, it forces a lot of context switches > to get the "right thread for the job" running. Are threads a sparse resource in linux? I thought the limit for a typical system is well above 1000. And if a thread is waiting for data appearing on a filehandle how could it create context switches? It's just lying there, the only thing it's occupying is address space and some kernel memory. Now if all those threads will be crunching data this would be another case. To avoid overload just increase/decrease a counter and wait (!) if it's getting too high. > Instead, have one thread that waits until anything is possible. When > something is possible, it wakes another thread to wait for the next > thing to be possible and it does X, Y, Z, or whatever it was just told > is now possible to do. In this case you need code to decide what to do when woken up in your application. A switch and a call to the corresponding worker routine, passing matching context data via stack to that routine. > This results in far fewer context switches and better utilization of > CPU code and data caches. (Of course, if the web part is an > insignificant fraction of resource usage, it might not matter.) This doesn't result in more context switches (see above) but in more application code which puts a heavier load on code and data caches. If you start those worker routines as threads, the decision making about what worker to run is moved into the kernel which is highly optimised for that kind of work. Additionally your worker threads keep their context data local and may hide that data structure from other threads/modules which give a much cleaner, simpler and safer design. Give threads a chance, they are there for a reason. Vitus -- Vitus Jensen, Hannover, Germany, Earth, Universe (current)
From: Peter Olcott on 2 Apr 2010 09:20 "Vitus Jensen" <vitus(a)alter-schwede.de> wrote in message news:alpine.LNX.2.00.1004021202200.5338(a)asterix.crazy-teaparty.dyndns.org... > Hi David, > > On Thu, 1 Apr 2010, David Schwartz wrote: > >> On Apr 1, 4:23 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com> >> wrote: > > ... > >> Seems kind of silly to have a thread for each request >> that spends most >> of its time just waiting for another program. You don't >> need a thread >> to wait. Just assign a thread to do what needs to be done >> when you're >> notified that the request is finished being processed by >> the OCR >> software. > > I'm coming from another platform where the maxime was "a > thread should do nothing very well" (PF), which I always > interpreted as to code your thread so that they spend most > of their time waiting. So yes, you need a thread to wait. > 99% of the time it should do nothing else but wait. You probably have that correctly, when I first read it, it seemed that it was saying that threads can not be relied upon because they keep screwing things up. I thought that I might add this little bit of humor. > >> I'm saying, don't have one thread waiting to do X when X >> is possible >> and one waiting to do Y when Y is possible and so on. >> First, this >> wastes a lot of threads. Second, it forces a lot of >> context switches >> to get the "right thread for the job" running. > > Are threads a sparse resource in linux? I thought the > limit for a typical system is well above 1000. > > And if a thread is waiting for data appearing on a > filehandle how could it create context switches? It's > just lying there, the only thing it's occupying is address > space and some kernel memory. > > Now if all those threads will be crunching data this would > be another case. To avoid overload just increase/decrease > a counter and wait (!) if it's getting too high. > >> Instead, have one thread that waits until anything is >> possible. When >> something is possible, it wakes another thread to wait >> for the next >> thing to be possible and it does X, Y, Z, or whatever it >> was just told >> is now possible to do. > > In this case you need code to decide what to do when woken > up in your application. A switch and a call to the > corresponding worker routine, passing matching context > data via stack to that routine. > >> This results in far fewer context switches and better >> utilization of >> CPU code and data caches. (Of course, if the web part is >> an >> insignificant fraction of resource usage, it might not >> matter.) > > This doesn't result in more context switches (see above) > but in more application code which puts a heavier load on > code and data caches. > > If you start those worker routines as threads, the > decision making about what worker to run is moved into the > kernel which is highly optimised for that kind of work. > Additionally your worker threads keep their context data > local and may hide that data structure from other > threads/modules which give a much cleaner, simpler and > safer design. > > Give threads a chance, they are there for a reason. > > Vitus > > -- > Vitus Jensen, Hannover, Germany, Earth, Universe (current)
From: Ersek, Laszlo on 2 Apr 2010 09:48 On Fri, 2 Apr 2010, Vitus Jensen wrote: > Hi David, (I'll try to follow up on this, though I'm sure David will offer you much better answers.) > On Thu, 1 Apr 2010, David Schwartz wrote: > >> On Apr 1, 4:23�pm, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote: > > ... > >> Seems kind of silly to have a thread for each request that spends most >> of its time just waiting for another program. You don't need a thread >> to wait. Just assign a thread to do what needs to be done when you're >> notified that the request is finished being processed by the OCR >> software. > > I'm coming from another platform where the maxime was "a thread should > do nothing very well" (PF), which I always interpreted as to code your > thread so that they spend most of their time waiting. So yes, you need > a thread to wait. 99% of the time it should do nothing else but wait. What's the name of that platform? I don't understand the "PF" abbreviation. Perhaps you mean "FP" as in functional programming. The programmer-accessible thread concept in "such languages" tends to be lightweight ("green threads" or "managed threads"), and the runtime environment of the language provides a scheduler, implemented in user space, that multiplexes such green threads over kernel threads (kernel schedulable entities). This is called the M:N thread model. >> I'm saying, don't have one thread waiting to do X when X is possible >> and one waiting to do Y when Y is possible and so on. First, this >> wastes a lot of threads. Second, it forces a lot of context switches to >> get the "right thread for the job" running. > > Are threads a sparse resource in linux? I thought the limit for a > typical system is well above 1000. GNU/Linux switched to a 1:1 thread model not so long ago (at least when you program in C), AFAICT. http://people.redhat.com/drepper/glibcthreads.html http://people.redhat.com/drepper/nptl-design.pdf Furthermore, IIRC the default stack dedicated to a single thread is 2M (at least?), so 1000 threads in a 32-bit process would eat up (per default) the usable address space of the process quite quickly. So yes, in the 1:1 thread model (which seems to me kind of "proven optimal" for system programming languages), a thread is a heavy-weight resource. You implement manually what the user-space green threads scheduler does for you elsewhere. > And if a thread is waiting for data appearing on a filehandle how could > it create context switches? It's just lying there, the only thing it's > occupying is address space and some kernel memory. I recommend reading "High-Performance Server Architecture" by Jeff Darcy: <http://pl.atyp.us/content/tech/servers.html>. > Now if all those threads will be crunching data this would be another > case. To avoid overload just increase/decrease a counter and wait (!) if > it's getting too high. You create heavy-weight threads not because you want to express the work to be done differently, ie. in separate logical sections (eg. for I/O multiplexing), but because you want to scale CPU-intensive work to multiple cores, or you wish to overlap waiting for IO with CPU-intensive work and don't want to use AIO or signals for some reason. The C10K problem, by Dan Kegel: <http://kegel.com/c10k.html>. >> Instead, have one thread that waits until anything is possible. When >> something is possible, it wakes another thread to wait for the next >> thing to be possible and it does X, Y, Z, or whatever it was just told >> is now possible to do. > > In this case you need code to decide what to do when woken up in your > application. A switch and a call to the corresponding worker routine, > passing matching context data via stack to that routine. > >> This results in far fewer context switches and better utilization of >> CPU code and data caches. (Of course, if the web part is an >> insignificant fraction of resource usage, it might not matter.) > > This doesn't result in more context switches (see above) but in more > application code which puts a heavier load on code and data caches. The user-space scheduler of your runtime environment does exactly what David tells you to implement manually in C. You don't have to dispatch yourself based on event types because you register them beforehand with your user-space scheduler (probably written in C), and it does this X, Y, Z or whatever for you. X, Y, Z are things (event handlers) you registered implicitly when you defined your green threads and the resources they wait for. "Homogeneous threads" are easier to schedule for the kernel, and they adapt better to a changing load between different task types. Consider a pharmacy with five windows. Compare the following two setups: - you have a single queue of patients, with the patient at the head of the queue dispatched to whichever window becomes free -- each window (teller?) handles all kinds of requests, - you have five queues and patients can't move between queues once they've chosen one queue, according to their types of requests. Which one seems to handle better (a) smoother servicing / minimal wait times / wildly varying individual servicing times, (b) adding more windows, (c) adding more patients? Your runtime environment may make the impression of five separate queues (eg. windows by task types), but internally, it has, say, two real windows (by CPU cores) and it keeps reassembling your patients into queues corresponding to different concepts. > If you start those worker routines as threads, the decision making about > what worker to run is moved into the kernel which is highly optimised > for that kind of work. It surely is, but then the kernel *must* work (the user/kernel boundary *must* be crossed) for doing nothing more than dispatching (selecting a worker based on task type). A switch statement is much cheaper. > Additionally your worker threads keep their context data local and may > hide that data structure from other threads/modules which give a much > cleaner, simpler and safer design. I agree that keeping request-specific (temporary) data on the stack improves locality, but I believe this should matter little if a request, once dispatched, requires a massive amount of computation (relative to ensuring cache coherence etc). And eavy-weight threads appear most eligible to me when jobs do need intensive crunching. (We're talking OCR, right?) > Give threads a chance, they are there for a reason. Yes, they're there so you can place one independent instruction stream (or a very low number of them) on each individual core. .... Please feel free to pick this apart. I kind of sense some gaping holes in my reasoning, but I intend to prod knowledgeable subscribers into explaining, so that I can learn. Thanks. lacos
From: David W Noon on 2 Apr 2010 10:32
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 2 Apr 2010 15:48:13 +0200, Ersek, Laszlo wrote about Re: IPC based on name pipe FIFO and transaction log file: >On Fri, 2 Apr 2010, Vitus Jensen wrote: [snip] >> I'm coming from another platform where the maxime was "a thread >> should do nothing very well" (PF), [snip] >What's the name of that platform? I don't understand the "PF" >abbreviation. Vitus was referring to OS/2, and PF is Peter Fitzsimmons. Peter was a well known developer on OS/2 some 15 or 20 years ago. - -- Regards, Dave [RLU #314465] ======================================================================= dwnoon(a)spamtrap.ntlworld.com (David W Noon) Remove spam trap to reply by e-mail. ======================================================================= -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) iEYEARECAAYFAku1//AACgkQ9MqaUJQw2MnsvgCfcDJeJ9yFcddhdFqN1TgBZ6Ol VgoAn2aqUChiOHBhv72GGTHWYpXFumMt =vfkN -----END PGP SIGNATURE----- |