How reliable are named pipes?
in [Unix Programming]
|
From: Rainer Weikusat on 9 Apr 2010 07:44 "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes: > "Ian Collins" <ian-news(a)hotmail.com> wrote in message > news:8277c2Fp8jU14(a)mid.individual.net... >> On 04/ 9/10 11:12 AM, Peter Olcott wrote: >>> "Ian Collins"<ian-news(a)hotmail.com> wrote in message >>> news:82726lFp8jU7(a)mid.individual.net... >>>> On 04/ 9/10 04:58 AM, Peter Olcott wrote: >>>>> Are there any reliability issues or other caveats with >>>>> using >>>>> named pipes? >>>> >>>> In what context? >>> >>> One respondent in another group said that named pipes are >>> inherently very unreliable and I think that he switched >>> to >>> some sort of sockets. In any case he did not choose named >>> pipes for his IPC because of significant reliability >>> issues >>> that he encountered. >> >> Care to cite a reference? >> >> -- >> Ian Collins > > http://groups.google.com/group/microsoft.public.vc.mfc/msg/e1b2fc1b0ff39589?hl=en A 'named pipe' on Windows is something completely different than a named pipe on UNIX(*). Specifically a 'named pipe' on Windows is a service endpoint which can be used for datagram-style or bytestream-style communication among networked hosts using the 'standard', proprietary Microsoft networking protocols which grew 'organically' (this should really be 'carcinomically' :->) out of the original DOS file and printer sharing support (OS/2 has/had a similar facility). Not even Microsoft has (as far as I know) complete and accurate documentation about that. In contrast to this, a 'named pipe' on UNIX(*) is just that: A pipe (unidirectional[*] bytestream-style communication channel for IPC on a single computer) with a name (in the filesystem namespace). Basically, this is just a memory buffer in the kernel which can be accessed by doing an open(2) on the pathname of the pipe and processes can then sequentially copy data into the buffer with write(2) and read data sequentially (in FIFO order) with read(2). [*] Traditionally, pipes were unidirectional. "On many systems" (Linux being a notable exception), they are actually bidirectional nowadays. The main problem of FIFOs is that they are not suitable for anything except 1:1 communication because there is neither a way to determine which of possibly many processes wrote a particular sequence of bytes nor one to send a reply to a specific recipient.
From: Casper H.S. Dik on 9 Apr 2010 08:36 Rainer Weikusat <rweikusat(a)mssgmbh.com> writes: >A 'named pipe' on Windows is something completely different than a >named pipe on UNIX(*). Specifically a 'named pipe' on Windows is a >service endpoint which can be used for datagram-style or >bytestream-style communication among networked hosts using the >'standard', proprietary Microsoft networking protocols which grew >'organically' (this should really be 'carcinomically' :->) out of the >original DOS file and printer sharing support (OS/2 has/had a similar >facility). Not even Microsoft has (as far as I know) complete and >accurate documentation about that. Originally, they were actually buffered in the filesystem >In contrast to this, a 'named pipe' on UNIX(*) is just that: A pipe >(unidirectional[*] bytestream-style communication channel for IPC on a >single computer) with a name (in the filesystem namespace). Basically, >this is just a memory buffer in the kernel which can be accessed by >doing an open(2) on the pathname of the pipe and processes can then >sequentially copy data into the buffer with write(2) and read data >sequentially (in FIFO order) with read(2). > [*] Traditionally, pipes were unidirectional. "On many > systems" (Linux being a notable exception), they are actually > bidirectional nowadays. >The main problem of FIFOs is that they are not suitable for anything >except 1:1 communication because there is neither a way to determine >which of possibly many processes wrote a particular sequence of bytes >nor one to send a reply to a specific recipient. For a systems with STREAMS, there's a connld module which creates a new STREAM everytime the pipe is opened. (You get a pipe which works like accept) But without that, multiple messages from multiple processes can be concatenated and they need to be picked apart. Casper -- Expressed in this posting are my opinions. They are in no way related to opinions held by my employer, Sun Microsystems. Statements on Sun products included here are not gospel and may be fiction rather than truth.
From: Ersek, Laszlo on 9 Apr 2010 08:58 On Fri, 9 Apr 2010, Rainer Weikusat wrote: > The main problem of FIFOs is that they are not suitable for anything > except 1:1 communication because there is neither a way to determine > which of possibly many processes wrote a particular sequence of bytes > nor [...] I apologize for splitting hairs and parroting banalities, but if all producers agree to write only messages not bigger than _POSIX_PIPE_BUF (== 512) bytes, then those messages will be atomic, in a portable way, and a single consumer can parse the stream. This enables n-to-1 communication. (Admittedly, not a very general kind.) Messages may carry a fixed length header, designating the complete (or remaining) length of the message, and identifying the sender. lacos
From: Peter Olcott on 9 Apr 2010 10:43 "Rainer Weikusat" <rweikusat(a)mssgmbh.com> wrote in message news:87hbnkx33z.fsf(a)fever.mssgmbh.com... > "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes: >> "Ian Collins" <ian-news(a)hotmail.com> wrote in message >> news:8277c2Fp8jU14(a)mid.individual.net... >>> On 04/ 9/10 11:12 AM, Peter Olcott wrote: >>>> "Ian Collins"<ian-news(a)hotmail.com> wrote in message >>>> news:82726lFp8jU7(a)mid.individual.net... >>>>> On 04/ 9/10 04:58 AM, Peter Olcott wrote: >>>>>> Are there any reliability issues or other caveats >>>>>> with >>>>>> using >>>>>> named pipes? >>>>> >>>>> In what context? >>>> >>>> One respondent in another group said that named pipes >>>> are >>>> inherently very unreliable and I think that he switched >>>> to >>>> some sort of sockets. In any case he did not choose >>>> named >>>> pipes for his IPC because of significant reliability >>>> issues >>>> that he encountered. >>> >>> Care to cite a reference? >>> >>> -- >>> Ian Collins >> >> >> http://groups.google.com/group/microsoft.public.vc.mfc/msg/e1b2fc1b0ff39589?hl=en > > A 'named pipe' on Windows is something completely > different than a > named pipe on UNIX(*). Specifically a 'named pipe' on > Windows is a > service endpoint which can be used for datagram-style or > bytestream-style communication among networked hosts using > the > 'standard', proprietary Microsoft networking protocols > which grew > 'organically' (this should really be 'carcinomically' :->) > out of the > original DOS file and printer sharing support (OS/2 > has/had a similar > facility). Not even Microsoft has (as far as I know) > complete and > accurate documentation about that. > > In contrast to this, a 'named pipe' on UNIX(*) is just > that: A pipe > (unidirectional[*] bytestream-style communication channel > for IPC on a > single computer) with a name (in the filesystem > namespace). Basically, > this is just a memory buffer in the kernel which can be > accessed by > doing an open(2) on the pathname of the pipe and processes > can then > sequentially copy data into the buffer with write(2) and > read data > sequentially (in FIFO order) with read(2). > > [*] Traditionally, pipes were unidirectional. "On many > systems" (Linux being a notable exception), they are > actually > bidirectional nowadays. > > The main problem of FIFOs is that they are not suitable > for anything > except 1:1 communication because there is neither a way to > determine > which of possibly many processes wrote a particular > sequence of bytes > nor one to send a reply to a specific recipient. Unless the ProcessID is passed back and forth. In my case I have many threads of one process that will communicate with four processes. I was considering named pipes from the threads to each of the four processes (depending on the process priority of the job). To communicate back I would have four more named pipes that communicate with the web server's many threads. They would pass the ThreadID back and forth. Someone else proposed that I use some form of sockets.
From: Peter Olcott on 9 Apr 2010 10:45
"Casper H.S. Dik" <Casper.Dik(a)Sun.COM> wrote in message news:4bbf1f6a$0$22940$e4fe514c(a)news.xs4all.nl... > Rainer Weikusat <rweikusat(a)mssgmbh.com> writes: > >>A 'named pipe' on Windows is something completely >>different than a >>named pipe on UNIX(*). Specifically a 'named pipe' on >>Windows is a >>service endpoint which can be used for datagram-style or >>bytestream-style communication among networked hosts using >>the >>'standard', proprietary Microsoft networking protocols >>which grew >>'organically' (this should really be 'carcinomically' :->) >>out of the >>original DOS file and printer sharing support (OS/2 >>has/had a similar >>facility). Not even Microsoft has (as far as I know) >>complete and >>accurate documentation about that. > > Originally, they were actually buffered in the filesystem > >>In contrast to this, a 'named pipe' on UNIX(*) is just >>that: A pipe >>(unidirectional[*] bytestream-style communication channel >>for IPC on a >>single computer) with a name (in the filesystem >>namespace). Basically, >>this is just a memory buffer in the kernel which can be >>accessed by >>doing an open(2) on the pathname of the pipe and processes >>can then >>sequentially copy data into the buffer with write(2) and >>read data >>sequentially (in FIFO order) with read(2). > >> [*] Traditionally, pipes were unidirectional. "On many >> systems" (Linux being a notable exception), they are >> actually >> bidirectional nowadays. > >>The main problem of FIFOs is that they are not suitable >>for anything >>except 1:1 communication because there is neither a way to >>determine >>which of possibly many processes wrote a particular >>sequence of bytes >>nor one to send a reply to a specific recipient. > > For a systems with STREAMS, there's a connld module which > creates > a new STREAM everytime the pipe is opened. (You get a pipe > which works like accept) > > But without that, multiple messages from multiple > processes can be > concatenated and they need to be picked apart. Fixed length messages with ThreadID and/or ProcessID. > > Casper > > -- > Expressed in this posting are my opinions. They are in no > way related > to opinions held by my employer, Sun Microsystems. > Statements on Sun products included here are not gospel > and may > be fiction rather than truth. |