Details about how process scheduler works?
in [Unix Programming]
|
From: Peter Olcott on 13 Apr 2010 11:04 "Chris Friesen" <cbf123(a)mail.usask.ca> wrote in message news:as6dnVw3GMrIGVnWnZ2dnUVZ_v2dnZ2d(a)posted.sasktel... > On 04/13/2010 08:18 AM, Peter Olcott wrote: >> "Chris Friesen" <cbf123(a)mail.usask.ca> wrote in message > >>> Have you considered reading up on the design of current >>> linux/unix >>> OS's rather than asking random questions on newsgroups >>> based on >>> what random people on the internet say? > >> Yes I bought a dozen books for this purpose. The one that >> covers this topic has not arrived yet. Also the earlier >> version of this book that is available online does not >> seem >> to go into the details that I need. >> >> http://oreilly.com/catalog/linuxkernel/chapter/ch10.html > > Generally speaking it's a bad idea to code applications > based on > internal knowledge of the OS algorithms. > > The section you describe gives a substantial amount of > detail on the > workings of the scheduler used by linux until relatively > recently. It > certainly describes how priorities, timeslices, etc. are > handled. > > What information are you looking for that isn't described > in that section? > > Chris Ultimately what I am looking for is a way to provide absolute priority to one kind of job over three other kinds of jobs. Each one of these jobs is invoked by another process through some form of IPC. I am trying to derive the best design to achieve these requirements.
From: Chris Friesen on 13 Apr 2010 11:52 On 04/13/2010 09:04 AM, Peter Olcott wrote: > Ultimately what I am looking for is a way to provide > absolute priority to one kind of job over three other kinds > of jobs. If you really mean "absolute" priority then you should consider one of the soft-realtime scheduler policies--SCHED_FIFO or SCHED_RR. (Of course by default on current linux systems even those are limited to 95% of the system, though that limit is tuneable.) Your earlier statements suggested you wanted to give tuneable percentages though. On current linux the best way to do that would be to create two different scheduler groups. Give the first group a weight corresponding to 80% and the other group a weight corresponding to 20%, put the first kind of job in the first group and the other jobs in the second group. This is highly specific to linux though. Chris
From: Peter Olcott on 13 Apr 2010 12:10 "Chris Friesen" <cbf123(a)mail.usask.ca> wrote in message news:3tednYgc4c8iDlnWnZ2dnUVZ_rydnZ2d(a)posted.sasktel... > On 04/13/2010 09:04 AM, Peter Olcott wrote: > >> Ultimately what I am looking for is a way to provide >> absolute priority to one kind of job over three other >> kinds >> of jobs. > > If you really mean "absolute" priority then you should > consider one of > the soft-realtime scheduler policies--SCHED_FIFO or > SCHED_RR. (Of > course by default on current linux systems even those are > limited to 95% > of the system, though that limit is tuneable.) > > Your earlier statements suggested you wanted to give > tuneable > percentages though. On current linux the best way to do > that would be > to create two different scheduler groups. Give the first > group a weight > corresponding to 80% and the other group a weight > corresponding to 20%, > put the first kind of job in the first group and the other > jobs in the > second group. This is highly specific to linux though. > > Chris The first process is a web server that has been adapted so that it can directly interface with four OCR processes or one OCR process with four threads. There has also been a big debate over whether or not I should implement the priority of the four types of OCR jobs using threads or processes. I am thinking that something as simple as lowering the priority of three of the four OCR jobs might provide everything that I need since this is a server dedicated for the sole purpose of running the web server / OCR processes. Either that or raising the priority of the web server and one of the four OCR processes.
From: Chris Friesen on 13 Apr 2010 13:00 On 04/13/2010 10:10 AM, Peter Olcott wrote: > There has also been a big debate over whether or not I > should implement the priority of the four types of OCR jobs > using threads or processes. Technically nice() and setpriority() both only work on processes. There is no standard way to specify the nice level of a single thread within a process. In linux if you specify PRIO_PROCESS and a "who" of 0, the current thread will be changed but there is no guarantee this will work on other systems. > I am thinking that something as simple as lowering the > priority of three of the four OCR jobs might provide > everything that I need since this is a server dedicated for > the sole purpose of running the web server / OCR processes. > Either that or raising the priority of the web server and > one of the four OCR processes. That would be the simplest. Worth a try. Chris
From: David Schwartz on 13 Apr 2010 13:25
On Apr 12, 8:48 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote: > Someone in another group said that the idea that you > mentioned in your last sentence had been universally > abandoned several decades ago because there was too much OS > overhead involved with this. That's how progress goes. Ideas are adopted, abandoned, and then rediscovered. How much OS overhead matters in the scheduler, compared to how important it is that you do the most important work, has changed significantly, mostly due to the increasing complexity of CPU topology. > I came here to confirm or deny > the truth of this alternative view. Of course I could be > misparaphrasing what they said. It's going to depend on people's areas of experience. Someone who works with Windows a lot will have a different idea of what's common scheduler behavior than someone who only works with Linux. Linux's scheduling algorithm was recently completely replaced and the way priority works was completely changed. In most typical cases, the observable behavior will be the same. If the process with the highest priority always runs/pre-empts, then processes with close static priorities will wind up getting proportional CPU anyway as their dynamic priorities cause them to pre- empt each other. Processes with vastly differing static priorities will wind up giving the vast majority of the CPU to the higher- priority process (assuming it can use it) with either model. I honestly think your requirements are not unusual and you will have a hard time getting things to not work if you use the process priority model. But the best approach is going to be to test on the hardware you plan to use. Schedulers vary from OS to OS more than just about anything else. DS |