Do named pipes have disk I/O ?? [Unix Programming]

Prev: Learn about proxy sites and how to use them to open blocked sites unlimited downloads from RapidShare and megaupload and increase the speed of the Internet with new sites for free
Next: Why can't I catch a SIGBUS on my Mac (when strlen(NULL) is used)?

From: David Schwartz on 6 Apr 2010 21:23

On Apr 6, 6:06 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote:

> If you think that priorities are not the solution to my
> problem then how can I make sure that certain HTTP requests
> get executed immediately in strict FIFO order, and other
> HTTP requests only get executed when there are no top
> priority HTTP requests?

> The first requests take 50 ms the second requests take 3.5
> minutes. Although there may be periods of 100 ms free quite
> often periods of 3.5 minutes might never be free.

> Maybe the 3.5 minute job needs its own process. Is it easy
> to put a process to sleep and wake it up again?

If you can easily process the high-priority jobs in a separate process
from the low-priority jobs and elevate the high-priority job's process
(or alternatively, lower the lower-priority jobs process priority)
that would be a good solution. At that point, you might as well just
use one process per request (possibly having the process go on to
serve other requests when it finishes). Unless you specifically code
the processes to access some shared resource, you won't have priority
inversion problems between processes.

In that case, you'd have two queues feeding two pools of processes.
Each pool should have at least as many processes as the system has
cores.

That seems like a simple solution. Since you really don't need any of
the benefits of multi-threading in this case, there's really no reason
you should pay the costs of multi-threading.

DS

From: Peter Olcott on 6 Apr 2010 22:31

Since I will only have one core initially, I want to have
the one process somehow put the other process to sleep when
there are any items in the high priority queue. Can
Linux/Unix do this easily?

"David Schwartz" <davids(a)webmaster.com> wrote in message
news:374f11c0-0947-434a-ba1e-eed9859470b7(a)b33g2000yqc.googlegroups.com...
On Apr 6, 6:06 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com>
wrote:

> If you think that priorities are not the solution to my
> problem then how can I make sure that certain HTTP
> requests
> get executed immediately in strict FIFO order, and other
> HTTP requests only get executed when there are no top
> priority HTTP requests?

> The first requests take 50 ms the second requests take 3.5
> minutes. Although there may be periods of 100 ms free
> quite
> often periods of 3.5 minutes might never be free.

> Maybe the 3.5 minute job needs its own process. Is it easy
> to put a process to sleep and wake it up again?

If you can easily process the high-priority jobs in a
separate process
from the low-priority jobs and elevate the high-priority
job's process
(or alternatively, lower the lower-priority jobs process
priority)
that would be a good solution. At that point, you might as
well just
use one process per request (possibly having the process go
on to
serve other requests when it finishes). Unless you
specifically code
the processes to access some shared resource, you won't have
priority
inversion problems between processes.

In that case, you'd have two queues feeding two pools of
processes.
Each pool should have at least as many processes as the
system has
cores.

That seems like a simple solution. Since you really don't
need any of
the benefits of multi-threading in this case, there's really
no reason
you should pay the costs of multi-threading.

DS

From: David Schwartz on 7 Apr 2010 01:35

On Apr 6, 7:31 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote:

> Since I will only have one core initially, I want to have
> the one process somehow put the other process to sleep when
> there are any items in the high priority queue. Can
> Linux/Unix do this easily?

Set the priority significantly higher.

DS

From: Phil Carmody on 7 Apr 2010 02:00

"Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
> "Rainer Weikusat" <rweikusat(a)mssgmbh.com> wrote in message
> news:87d3ycnwcl.fsf(a)fever.mssgmbh.com...
>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>>> "Scott Lurndal" <scott(a)slp53.sl.home>: wrote in message
>>>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>>
>> [...]
>>
>>>>>I am only focusing on the design of the preemptive
>>>>>scheduling now.
>>>>>Also the initial hardware will only have a single core
>>>>>with
>>>>>hyperthreading.
>>>>>
>>>>
>>>> You are making this far more complicated than necessary.
>>>> Have fun.
>>>>
>>>> scott
>>>
>>> So then do you have a simple way to implement preemptive
>>> scheduling?
>>
>> If you are convinced that this is the solution to your
>> problem, the
>> kernel already provides exactly that. You just have to use
>> the
>> existing facilities.
>
> So the OS will put a thread to sleep, saving its
> multi-megabyte state upon a process triggered event?

What, *precisely*, do you mean by "saving" the state,
and why are you so worried by that?

> I would save the state with a simple change of the integer
> subscript into an array of each threads data. Could the OS
> be told that this is what is needed?

In which case you have to simultaniously maintain the multi-megabyte
state of _all_ of the "thread data".

You're frightened of ghosts.

Phil
--
I find the easiest thing to do is to k/f myself and just troll away
-- David Melville on r.a.s.f1

From: Jasen Betts on 7 Apr 2010 07:43

On 2010-04-04, Peter Olcott <NoSpam(a)OCR4Screen.com> wrote:
>
> "Mark Hobley" <markhobley(a)hotpop.donottypethisbit.com> wrote
> in message news:v4nl87-ula.ln1(a)neptune.markhobley.yi.org...
>> In comp.unix.programmer Peter Olcott
>> <NoSpam(a)ocr4screen.com> wrote:
>>> I will be receiving up to 100 web requests per second
>>> (that
>>> is the maximum capacity) and I want to begin processing
>>> them
>>> as soon as they arrive hopefully without polling for
>>> them.
>>
>> That will happen. If 100 web requests come down the pipe,
>> the receiving process
>> will get them.
>>
>> It is only when no requests come down the pipe that the
>> receiving process will
>> have to wait for a request to come in. This is no big
>> deal.
>>
>> Mark.
>>
>> --
>> Mark Hobley
>> Linux User: #370818 http://markhobley.yi.org/
>>
>
> So it is in an infinite loop eating up all of the CPU time
> only when there are no requests to process?

no it's a read() that doesn't return or a select() that times out
while there are no requests inbound.

as soon as something arives and the kernel scheduler gets around to
activiating your progcess (immediately if the cpu is not busy)
the read() will return or select indiocate an non-blocking file
descriptor.

> I don't think
> that I want this either because I will have two priorities
> of requests. If there are no high priority requests I want
> it to begin working on the low priority requests.

select() prefers the lowest numbered file descriptor it's asked to
test/watch so that should be easy to arrange,

--- news://freenews.netfront.net/ - complaints: news(a)netfront.net ---