Prev: clients, was Re: Details about how process scheduler works?
Next: processor time
From: Rachit Agrawal on 15 Apr 2010 17:12
On Apr 16, 2:06 am, Chris Friesen <cbf...(a)mail.usask.ca> wrote:
> On 04/15/2010 03:04 PM, Rachit Agrawal wrote:
>
> > But in that case, won't it be better to use Message Queues? As
>
> >http://www.bluedonkey.org/cgi-bin/twiki/bin/view/Books/VxWorksCookboo...
>
> > says pipes are a wrapper to Message queues and are bit slower than
> > them.
>
> That is specific to vxworks.
>
> Chris

Oh ok, just out of curiosity, I wanted to know what are pros and cons
of both MQs and Pipes (POSIX). Are there any comparison available on
the net? I am not getting any.

Thanks
From: Chris Friesen on 15 Apr 2010 17:24
On 04/15/2010 12:52 PM, Rachit Agrawal wrote:
> On Apr 15, 9:57 pm, Chris Friesen <cbf...(a)mail.usask.ca> wrote:

>> Okay, so you've got a linux system, ext3, unspecified processor, and a
>> single spindle rotating media hard disk.
> Intel Xeon 24-core machine, forgot to mention that.

>> From your other email you suggested you wanted to write 7 bytes every
>> 100 instructions or so. Assuming 2 billion instructions per second,
>> that works out to about 140MB/sec that you want to write. That's pretty
>> much at the limit of consumer-grade hardware.
>>
>> Do you have any rules about how the chunks of 7 bytes need to be
>> ordered? Is it enough that each chunk is atomic or do they need to be
>> ordered somehow as well?
> Each chunk need to be atomic.

Given the fact that you want to write many small chunks and the ordering
doesn't matter, I'd probably do this:

1) Make a fifo (aka "named pipe") for interprocess communication.

2) Put each of your data-generating apps on a separate core. Each of
the data-generating processes accumulates 7-byte chunks to a local buffer.

3) Once they fill up the buffer, they write it to the shared pipe. On
linux this will be atomic as long as the buffer is 4096 bytes or less.

4) Have a process running on another core that just reads from the pipe
and writes out to disk.

The goal here is to minimize the overhead of library and system calls.
You could use a circular buffer in shared memory to reduce it even more
but the savings probably wouldn't be worth the extra complexity.

Chris
From: Chris Friesen on 15 Apr 2010 17:34
On 04/15/2010 03:12 PM, Rachit Agrawal wrote:

> Oh ok, just out of curiosity, I wanted to know what are pros and cons
> of both MQs and Pipes (POSIX). Are there any comparison available on
> the net? I am not getting any.

Generally posix message queues are more fully featured than pipes.

One major difference is that posix message queues support message
priority, where if there is a backlog the higher priority messages are
delivered first.

Chris
From: Rachit Agrawal on 16 Apr 2010 01:41
On Apr 16, 2:34 am, Chris Friesen <cbf...(a)mail.usask.ca> wrote:
> On 04/15/2010 03:12 PM, Rachit Agrawal wrote:
>
> > Oh ok, just out of curiosity, I wanted to know what are pros and cons
> > of both MQs and Pipes (POSIX). Are there any comparison available on
> > the net? I am not getting any.
>
> Generally posix message queues are more fully featured than pipes.
>
> One major difference is that posix message queues support message
> priority, where if there is a backlog the higher priority messages are
> delivered first.
>
> Chris

Thanks Chris, but whenever I looking for comparison of message queues
and named pipes (FIFO), in almost all the places, they say MQs are
faster. For e.g.
http://beej.us/guide/bgipc/output/html/singlepage/bgipc.html#fifos

So, this is what adding to my confusion. :(
From: Chris Friesen on 16 Apr 2010 02:17
On 04/15/2010 11:41 PM, Rachit Agrawal wrote:
> On Apr 16, 2:34 am, Chris Friesen <cbf...(a)mail.usask.ca> wrote:

>> One major difference is that posix message queues support message
>> priority, where if there is a backlog the higher priority messages are
>> delivered first.

> Thanks Chris, but whenever I looking for comparison of message queues
> and named pipes (FIFO), in almost all the places, they say MQs are
> faster. For e.g.
> http://beej.us/guide/bgipc/output/html/singlepage/bgipc.html#fifos

That site has nothing about the performance of the different options.
Also, it's dealing with system V message queues and not posix message
queues. They are two different things.

It's hard to speak in generalities about the speeds of different options
since they can vary depending on the specific platform. Shared memory
will generally be fastest if done efficiently, but after that it may
vary and the only way to know for sure is to benchmark it.

Chris

First  |  Prev  | 
Pages: 1 2 3 4
Prev: clients, was Re: Details about how process scheduler works?
Next: processor time