Do named pipes have disk I/O ?? [Unix Programming]

Prev: Learn about proxy sites and how to use them to open blocked sites unlimited downloads from RapidShare and megaupload and increase the speed of the Internet with new sites for free
Next: Why can't I catch a SIGBUS on my Mac (when strlen(NULL) is used)?

From: Peter Olcott on 7 Apr 2010 21:24

If you don't explain your reasoning then your advice is
useless to me.

Whether or not my idea is possibly reasonable depends upon
exactly how a higher and lower priority tasks are scheduled.

If there is any time slicing between the higher and lower
priority tasks then my idea might possibly be superior to
yours, (in the particular instance of my specific problem)
if you think not please try to explain this with reasoning
not dogma or ad hominem attacks.

If you can show me the error of my proposal using reasoning,
then I will learn more deeply. I am not the kind of guy that
learns by memorizing a bunch of facts. I learn by testing
the boundaries of ideas by exploring the details of the
reasoning behind them.

"David Schwartz" <davids(a)webmaster.com> wrote in message
news:994e90fc-b8be-4cbb-bf13-998ce7a23abd(a)x12g2000yqx.googlegroups.com...
On Apr 7, 2:42 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com>
wrote:

> The one thing that I know is that my fundamental process
> is
> about as memory intensive as a process can get. I also
> know
> in advance what the memory access patterns are much better
> than any cache algorithm could possibly infer. Even if a
> cache algorithm could infer this pretty well it would
> necessarily waste a lot of time gaining this info.

Bluntly, you have no idea what you're talking about.

> All that really needs to happen is to load my DFA
> recognizer
> directly into cache and not unload it from cache until
> recognition is complete.

That's sheer idiocy. If your recognizer hits, say, a page
fault, how
does the page fault handler running ridiculously slow help
your
recognizer run faster?

> This occurs naturally (reasonably
> well) if my process is the only one running. If another
> memory intensive process is time sliced in it screws up
> the
> cache resulting in a tenfold degradation to performance.

No, it doesn't. I already explained why.

> This analysis of the actual conditions would seem to
> indicate that the optimization that I propose might be
> better than the optimization that you propose.

You really are starting to seem uneducatable. You might be
right 99%
of the time, but the system will be right 100% of the time.
Just make
sure the system knows what you want, *not* how you think it
should
give it to you.

DS

From: Peter Olcott on 7 Apr 2010 21:26

"Phil Carmody" <thefatphil_demunged(a)yahoo.co.uk> wrote in
message news:87ochugby0.fsf(a)kilospaz.fatphil.org...
> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>> "Phil Carmody" <thefatphil_demunged(a)yahoo.co.uk> wrote in
>> message news:8739z7gdb6.fsf(a)kilospaz.fatphil.org...
>>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>>>> "Phil Carmody" <thefatphil_demunged(a)yahoo.co.uk> wrote:
>>>>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>>> ...
>>>> The only issue is that the minimum
>>>> sleep period seems to be one second, I could really use
>>>> it
>>>> to be 100 ms. I might have to build my own sleep system
>>>> for
>>>> this process.
>>>
>>> So you're attempting to optimising the system and
>>> reinvent
>>> the
>>> scheduler, and yet you've never heard of poll(2)? Why do
>>> I
>>> feel
>>> that's an unsurprising combination?
>>
>> I am working on providing a preemptive scheduling system
>> whereby a process puts itself to sleep when another
>> higher
>> priority job becomes available.
>
> Why do I find it unsurprising that you misuse the standard
> terminology?

The proposal has changed such that its original terminology
does not so aptly apply.

>
> Why do you think that you, who doesn't even know about the
> operating system you're working under, can implement its
> features better than the OS itself?

Even if I fail at this I will succeed in knowing the
reasoning why I failed.
>
> Phil
> --
> I find the easiest thing to do is to k/f myself and just
> troll away
> -- David Melville on r.a.s.f1

From: Peter Olcott on 7 Apr 2010 21:42

Documentation indicates that they are atomic.

One simple way to handle multiple threads or processes
needing to append to the same file would be to sequence
these operations.

If pwrites are guaranteed to seek and write as a single
atomic operation, then this might require some sequencing,
or locking or both, especially if to the same location
within the file. I am just beginning to think these things
through again after many years.

"David Schwartz" <davids(a)webmaster.com> wrote in message
news:67a7c2a3-c7b1-4555-89c4-ae6dffe40fbc(a)r18g2000yqd.googlegroups.com...
On Apr 7, 8:40 am, "Peter Olcott" <NoS...(a)OCR4Screen.com>
wrote:

> Here is a specific concrete example:
> I am assuming that all pwrite(), pread() and append()
> operations are atomic specifically because the kernel
> forces
> these to be executed sequentially. Is this the means that
> the kernel uses to make these operations atomic?

They are not atomic, and the kernel does not force them to
be executed
sequentially.

It would be absolutely idiotic in the extreme for the kernel
to delay
a process that attempted to 'pread' one byte from the
beginning of a
file until a 'pwrite' to the end of that file from another
process of
100MB from file 'mmap'ed from a slow NFS server finished.

DS

From: Peter Olcott on 7 Apr 2010 23:24

"Phil Carmody" <thefatphil_demunged(a)yahoo.co.uk> wrote in
message news:87sk77exjd.fsf(a)kilospaz.fatphil.org...
> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>> The one thing that I know is that my fundamental process
>> is
>> about as memory intensive as a process can get.
>
> It's quite possibly very badly designed, then.
>
>> I also know
>> in advance what the memory access patterns are much
>> better
>> than any cache algorithm could possibly infer.
>
> What on earth does 'better' mean? If a cache algorithm
> cannot
> infer them, then your memory access patterns are complete
> shite.
>
>> Even if a
>> cache algorithm could infer this pretty well it would
>> necessarily waste a lot of time gaining this info.
>
> And you're proud of this inefficiency? Why on earth didn't
> you design your code to be efficient instead?

My technology is already 872-fold faster than the next best
alternative:
http://www.ocr4screen.com/Unique.html

>
> Phil
> --
> I find the easiest thing to do is to k/f myself and just
> troll away
> -- David Melville on r.a.s.f1

From: Peter Olcott on 7 Apr 2010 23:43

Actually I hope that you are right and I am wrong, it would
make my job much simpler.

I also figured out how you could be mostly right, and thus
have the superior solution, this would mostly depend upon
the details of exactly how time slicing is implemented: The
size and frequency of each time slice. I am estimating that
both the size and the frequency may vary over some range of
values.

My original solution might still provide slightly better
performance for the high priority jobs, at the expense of a
lot of trouble and potential for error, and substantial
reduction in the performance of the other jobs.

The performance degradation that I might expect would be a
small fraction of the high priority jobs might take about
twice as long. If it is as bad as this, this would be
acceptable. I doubt that it would be worse than this.

I do know from extensive benchmark testing that cache
spatial locality of reference does provide a ten-fold
increase in speed. What I failed to take into account is how
quickly cache could be refilled. If it has to be refilled
exactly once, this could double processing time.

"David Schwartz" <davids(a)webmaster.com> wrote in message
news:994e90fc-b8be-4cbb-bf13-998ce7a23abd(a)x12g2000yqx.googlegroups.com...
On Apr 7, 2:42 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com>
wrote:

> The one thing that I know is that my fundamental process
> is
> about as memory intensive as a process can get. I also
> know
> in advance what the memory access patterns are much better
> than any cache algorithm could possibly infer. Even if a
> cache algorithm could infer this pretty well it would
> necessarily waste a lot of time gaining this info.

Bluntly, you have no idea what you're talking about.

> All that really needs to happen is to load my DFA
> recognizer
> directly into cache and not unload it from cache until
> recognition is complete.

That's sheer idiocy. If your recognizer hits, say, a page
fault, how
does the page fault handler running ridiculously slow help
your
recognizer run faster?

> This occurs naturally (reasonably
> well) if my process is the only one running. If another
> memory intensive process is time sliced in it screws up
> the
> cache resulting in a tenfold degradation to performance.

No, it doesn't. I already explained why.

> This analysis of the actual conditions would seem to
> indicate that the optimization that I propose might be
> better than the optimization that you propose.

You really are starting to seem uneducatable. You might be
right 99%
of the time, but the system will be right 100% of the time.
Just make
sure the system knows what you want, *not* how you think it
should
give it to you.

DS