Can extra processing threads help in this case?
in [MFC]
|
From: Joseph M. Newcomer on 11 Apr 2010 19:18 See below... On Sun, 11 Apr 2010 17:05:01 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >> >> >> He now got it down to 50 TPS, 3000 per minute, 18,000 per >> hour, 144,000 per work day after he was told that he can't >> do his 10 ms OCR in 0 ms OVERHEAD :) > >Bullshit again. I already said that I estimated the overhead >to not exceed 10 ms. You stupidly read this to mean that I >said that the overhead will take 0 ms. I did say that my >goal is to get as close to zero time as possible, and this >might still be possible because of hypertheading. Worst >case scenario is that it will double my processing time. **** Ihave NO IDEA how hyperthreading can help in this case, since you are not using any form of threading, and some of the actions, such as transacted file system, passing messages via IPC, etc. are fundamentally serializing actions that require waiting for one thread to complete before another thread is allowed to proceed (and as I pointed out, whether threse threads are in the same process or different processes matters not in the slightest; you have this weird idea that threads in separate processes solve nonexisten problems that you claim exist if the threads share a single process. Indeed, there are such problems but none of the problems you have described change no matter how many processes are involved!) Maybe this is a use of concurrency we have not seen before... OK, if you overhead is 10ms, which is actually pretty generous, then it makes sense. Note that doubling your processing time does not account for hundreds of ms of TCP/IP or HTTP transaction overhead (including network delays). But you still don't get why a SQMS architecture as I described makes more sense. **** > >If you weren't so damn rude I would not be so harsh. >Although I could have written what I said more clearly, it >is not my fault for you getting it wrong. All you had to do >to get it right is to read exactly what I said. **** We are rude because you keep ignoring what WE are saying while busily accusing us of ignoring what YOU are saying! You demanded "sound reasoning" from me, and I showed that a few seconds' thought and third-grade arithmetic prove your design is faulty. You could have done this "sound reasoning" on your own! It is not only not Rocket Science, it is so basic that, literally, a child can do it. And we never get a single coherent picture; we get constantly-revised pieces, one at a time, without any way to fit them back together to whatever the requirements, specification, or implementation strategy might be. And what we see of the latter, we know to be largely nonsensical. I'd fail freshman programmers for such a design. joe **** > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 11 Apr 2010 19:36 "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message news:%23%23iEA1c2KHA.4332(a)TK2MSFTNGP02.phx.gbl... > Peter Olcott wrote: > >>> So how do your HTTP request get delegated? Four >>> separate IP address, sub domains? >>> >>> free.peter.com >>> 1penny.peter.com >>> nickle.peter.com >>> peso.peter.com >>> >>> What happens when they cross domain attempts occur? >> >> I would not be using the complex design that you are >> referring to. One domain, one web server, four OCR >> processes. > > > So you back to a Many to One Fifo queue. And what happens > with the HTTP responses? They have another FIFO queue in the opposite direction. > >>> >>> Right, two different memory locations can not possible >>> overflow each other. >> >> One writer of a single unsigned 32-bit integer at a fixed >> shared memory location and three readers. >> if (NumberOfHighPriorityJobs != 0) >> nanosleep(20); > > > By the web server needs to do a +1 and one of the OCR has > to do a -1. No not on this memory location. This memory location is a copy of another memory location that does these things to the original value. I don't want to slow down the read of this value by using a lock because it will be read in a very tight loop. In other words the original memory location may be read or written to as often as once every 10 ms. It has to be locked to make sure that it is updated correctly. This copy of the memory location could easily be read a million times a second or more, don't want to slow this down with a lock. > > No conflicts, no reader/locker locks? No Interlock > increments and decrements? As long as a simultaneous read and write can not garble each other there is no need for any of these things, on this copy of the original memory location. > > > -- > HLS
From: Joseph M. Newcomer on 11 Apr 2010 19:37 See below... On Sat, 10 Apr 2010 09:22:28 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:peuvr59s085vrj20t8b7ddse13f71sv4qo(a)4ax.com... >> See below... >> On Fri, 9 Apr 2010 20:49:05 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>>Oh right now I know why that wont work. the 3.5 minute >>>jobs >>>(whenever they occur) would completely kill my 100 MS goal >>>for the high priority jobs. >> **** >> THis is called "priority inversion". Of course, if you >> used an ordered queue and multiple >> servers, you would have several other servers to handle >> the other jobs. >> **** > >That is not in the budget in the foreseeable future. One >single core server with Hyperthreading. > >>> I could keep killing these jobs >>>whenever a high priority job arrives, but, I might not >>>ever >>>have a whole 3.5 minutes completely free so this 3.5 >>>minute >>>jobs may never get executed. Giving these 3.5 minutes jobs >>>a >>>tiny time slice is the simplest way to solve this problem. >> **** >> No, and if you think it is so, you have missed the point >> entirely. Ultimately, you have >> to schedule things appropriately, and delegating this to >> the OS scheduler is NOT going to >> solve this problem! And handling multiple queues one at a >> time is not going to solve this >> problem. You need to look at how >> priority-inversion-protection systems work and go that >> way (for example, the way you schedule in the one-and-only >> queue is that you never let >> more than N large jobs get in the queue in sequence). >> This problem has been around for as > >Even one large job take 2,100- fold too long. > >> long as realtime systems have been around, and the >> solutions form an entire body of >> realtime literature. RTOS systems deal with this, and >> they have documented the RTOS >> algorithms for preventing priority inversion. Spent a >> little time actually reading about >> these systems you claim expertise on, and you will find >> how to order your queue to prevent >> priority inversion situations from arising. And it is all >> done in a single queue. > >I did and see no possible way the priority inversion could >possibly occur in either of the two designs that I am >considering: >(1) Four queues each with their own OCR process, one of >these processes has much more process priority than the >rest. **** Did we not explain that messing with thread priorities gets you in trouble? And you seem to have this totally weird idea that "process" and "thread" have meaning. Get this: THERE ARE ONLY THREADS! Processes are a fiction that provide resource management; a process holds the address space, file system handles, and other resources. A process IS NOT SCHEDULABLE! ONLY THREADS ARE SCHEDULABLE! And therefore it doesn't matter if a set of threads are distributed across a dozen processes or all live in one process, the problems DO NOT CHANGE (except that processes add kmore overhead) **** > >(2) My original design four processes that immediately yield >putting themselves to sleep as soon as a higher priority >process occurs. The highest one of these never yields. **** How? You are postulating mechanisms that do not exist in any operating system I am aware of. They do not exist in linux, or Unix, or Solaris, or Mac OS X, or Windows. Of course, with a LOT of work, you could build such mechanisms out of low-level pieces in SOME of these systems (but not easily in Windows, that lacks 'signal'). You are making one of the most fundamental design failures: that thread priority can be a substitute for synchronization! So your design basically sucks. The reason you don't get priority inversion is that you can't block the processing thread by a slow job when there is one demanding immediate attention. But in a 4-CPU system, you give up most of your CPU power to accomplish this, which is another reason the design sucks. You have a design that will NOT use concurrency when it is available, and will not scale up to larger multiprocessors. It works, by accident, on a single-core CPU, and from this you generalize that it is an acceptable design. ROTFL! **** > >Also I don't see that there is much difference between four >queues one for each priority level and a single priority >ordered queue. The priority ordered queue would have a more >complex insert. Looking for a higher priority job would be >easier, only look at the head of the queue. Named pipes can >not handle this. **** Watch carefully: 1 + 1 = 2. 2 + 2 = 4; 1 / 4 = 0.25. Read the third-grade arithmetic that I used to demonstrate that a SQMS architecture scales up quite well, and maximizes concurrency while minimizing delay, whereas your MQMS architecture is guaranteed to maximize delays because, basically, it allows no concurrency. If you can't follow the simple arithmetic, try to find a reasonably bright child to explain addition, subtraction, multiplication and division to you. You demanded "sound reasoning" when I thought you could do simple arithmetic all on your own, without me to lead you every step of the way. > >> **** >>> >>>A possibly better way to handle this would be to have the >>>3.5 minute job completely yield to the high priority jobs, >>>and then pick up exactly where it left off. >> **** >> No, the better way is to use priority-inversion-prevention >> algorithms as documented in the >> realtime literature. I went to a lecture on these last >> year some time, and learned in 40 >> minutes what progress had been made in >> priority-inversion-prevention. Clever stuff. >> **** > >From what I have been told, the only possible way that >priority inversion can possibly occur is if there is some >sort of dependency on a shared resource. I don't see any >shared resource in my new design with four different >independent processes. **** You don't have the faintest clue what you are talking about, do you? You have solved the priority inversion problem by creating a design that minimizes utlization of computing resources and which, by its very nature, guarantees absolutely worst-case response under load! According to my training, we call this "bad design". **** > >>> >>>>>Just the pipe name itself it part of the disk, nothing >>>>>else >>>>>hits the disk. There are many messages about this on the >>>>>Unix/Linux groups, I stated a whole thread on this: >>>> **** >>>> And the pipe grows until what point? It runs out of >>>> memory? Ohh, this is a new >>>> interpretation of "unlimited pipe growth" of which I >>>> have >>>> been previously unaware! >>> >>>It does not ever simply discard input at some arbitrary >>>queue length such as five items. One of the FIFO models >>>did >>>just that. >> **** >> Well, that is not exactly "robust" or "reliable" now, is >> it? So it either blocks, or >> discards data. If it blocks, it doesn't matter whether it >> grows or not. If it discards >> data, it is not a very good design. >> joe > >The fundamental OS IPC mechanisms are what is discarding the >data. I think that this pertained to TCP sockets discarding >data because their buffer of very few elements had been >exceeded. **** TCP sockets will NEVER discard data; obviously, you cannot tell the difference between TCP/IP and UDP. TCP uses a positive-acknowledgement protocol and essentially what we might call a "distributed semaphore" model of queue management, which if you had ever read anything about it, you would immediately have known. I have no idea why you create imaginary problems due to your lack of understanding and solve them with fantasy solutions whose reliability is problematic at best. TCP/IP is a reliable protocol in that you will receive EVERY byte the sender sends, IN ORDER, NO DUPLICATES, NO LOST DATA, EVER! or both the sender and the receiver will receive a notication of failure (usually a lost connection because of failure of the sender or receiver). Where, in this specification, is there anything about it being allowed to whimsically throw away data? (Hint: the correct answer is "nowhere") Again, you have rejected a reasonable design alternative because of complete ignorance of the details! And how do you detect/recover from ANY IPC mechanism that can throw data away? Or does reliability no longer matter? joe **** > >> **** >>> >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 11 Apr 2010 20:05 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:vgl4s5ppn64lkfr7a46ihkm2h4lrbqi6dg(a)4ax.com... > See below... > On Sun, 11 Apr 2010 17:05:01 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>> >>> >>> He now got it down to 50 TPS, 3000 per minute, 18,000 >>> per >>> hour, 144,000 per work day after he was told that he >>> can't >>> do his 10 ms OCR in 0 ms OVERHEAD :) >> >>Bullshit again. I already said that I estimated the >>overhead >>to not exceed 10 ms. You stupidly read this to mean that I >>said that the overhead will take 0 ms. I did say that my >>goal is to get as close to zero time as possible, and this >>might still be possible because of hypertheading. Worst >>case scenario is that it will double my processing time. > **** > Ihave NO IDEA how hyperthreading can help in this case, > since you are not using any form > of threading, and some of the actions, such as transacted > file system, passing messages The main this that I want to occur in parallel is the disk accesses. The disk drive controller might already do this for me. Another big source of parallelism is the multiple processes. Certainly the web server could be updating the FIFO queues, including writing the (transaction log) audit trail to disk, while one or more OCR processes are running. Because of hyperthreading is might take 10 + 10 ms per transaction, yet still process 100 TPS. > via IPC, etc. are fundamentally serializing actions that > require waiting for one thread to > complete before another thread is allowed to proceed (and > as I pointed out, whether threse > threads are in the same process or different processes > matters not in the slightest; you > have this weird idea that threads in separate processes > solve nonexisten problems that you > claim exist if the threads share a single process. > Indeed, there are such problems but Yes and you say this from your decade of experience with pthreads, right? > none of the problems you have described change no matter > how many processes are involved!) > Maybe this is a use of concurrency we have not seen > before... > > OK, if you overhead is 10ms, which is actually pretty > generous, then it makes sense. Note > that doubling your processing time does not account for > hundreds of ms of TCP/IP or HTTP > transaction overhead (including network delays). Of course not. I have another 400 ms allocated to that. Also I already know completely that at any time unforeseen conditions could arise to make this utterly infeasible for the duration of these unforeseen conditions. If network traffic suddenly spikes. > > But you still don't get why a SQMS architecture as I > described makes more sense. My goal is to provide absolute priority of the high priority jobs over all the other jobs, SQMS can't do that. I can't see how it isn't a more complex design lacking any incremental benefits. Four Queues each with their own process only involves each process seeing if it has a job in its own queue. My purpose of IPC in this case to tell the process that it does have a new job in its queue. > **** >> >>If you weren't so damn rude I would not be so harsh. >>Although I could have written what I said more clearly, it >>is not my fault for you getting it wrong. All you had to >>do >>to get it right is to read exactly what I said. > **** > We are rude because you keep ignoring what WE are saying > while busily accusing us of > ignoring what YOU are saying! You demanded "sound > reasoning" from me, and I showed that a > few seconds' thought and third-grade arithmetic prove your > design is faulty. You could > have done this "sound reasoning" on your own! It is not > only not Rocket Science, it is so > basic that, literally, a child can do it. You never explained (or that may have been in the messages that I ignored) why my design of MQMS was inferior to what you proposed. In the future any statements lacking supporting reasoning will be taken to be pure malarkey at least until complete and sound reasoning is provided. Mostly what both you and Hector have been doing is forming an erroneous view of what I am saying based on false assumptions of what I am meaning and then point out how bad this misconception of what I am saying is without pointing out even why the misconception itself is bad. When you do this I have no way to correct the misconception. Here is a really good heuristic for you never make any assumptions at all. If you never make any assumptions then you also never make any false assumptions. There is NEVER a good reason to make an assumption in an ongoing dialogue. > > And we never get a single coherent picture; we get > constantly-revised pieces, one at a > time, without any way to fit them back together to > whatever the requirements, > specification, or implementation strategy might be. And > what we see of the latter, we > know to be largely nonsensical. I'd fail freshman > programmers for such a design. > joe > **** >> > > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 11 Apr 2010 20:20
"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:m8m4s55k6vsqr36lrkfo5b70al9sa86hos(a)4ax.com... > See below... > On Sat, 10 Apr 2010 09:22:28 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>(1) Four queues each with their own OCR process, one of >>these processes has much more process priority than the >>rest. > **** > Did we not explain that messing with thread priorities > gets you in trouble? > > And you seem to have this totally weird idea that > "process" and "thread" have meaning. Get Oh like the complete fiction of a separate address space for processes and not for threads? This group really needs to be moderated. > this: THERE ARE ONLY THREADS! Processes are a fiction > that provide resource management; a > process holds the address space, file system handles, and > other resources. A process IS > NOT SCHEDULABLE! ONLY THREADS ARE SCHEDULABLE! And > therefore it doesn't matter if a set > of threads are distributed across a dozen processes or all > live in one process, the > problems DO NOT CHANGE (except that processes add kmore > overhead) > **** >> >>(2) My original design four processes that immediately >>yield >>putting themselves to sleep as soon as a higher priority >>process occurs. The highest one of these never yields. > **** > How? You are postulating mechanisms that do not exist in > any operating system I am aware // shared memory location if (NumberOfHighPriorityJobsPending !=0) nanosleep(20); Is seems like every other message you switch into jerk mode. > of. They do not exist in linux, or Unix, or Solaris, or > Mac OS X, or Windows. Of course, > with a LOT of work, you could build such mechanisms out of > low-level pieces in SOME of > these systems (but not easily in Windows, that lacks > 'signal'). You are making one of the > most fundamental design failures: that thread priority can > be a substitute for > synchronization! > > So your design basically sucks. > > The reason you don't get priority inversion is that you > can't block the processing thread > by a slow job when there is one demanding immediate > attention. But in a 4-CPU system, you > give up most of your CPU power to accomplish this, which > is another reason the design > sucks. You have a design that will NOT use concurrency > when it is available, and will not > scale up to larger multiprocessors. It works, by > accident, on a single-core CPU, and from > this you generalize that it is an acceptable design. > > ROTFL! > **** >> >>Also I don't see that there is much difference between >>four >>queues one for each priority level and a single priority >>ordered queue. The priority ordered queue would have a >>more >>complex insert. Looking for a higher priority job would be >>easier, only look at the head of the queue. Named pipes >>can >>not handle this. > **** > Watch carefully: 1 + 1 = 2. 2 + 2 = 4; 1 / 4 = 0.25. > Read the third-grade arithmetic > that I used to demonstrate that a SQMS architecture scales > up quite well, and maximizes > concurrency while minimizing delay, whereas your MQMS > architecture is guaranteed to > maximize delays because, basically, it allows no > concurrency. If you can't follow the > simple arithmetic, try to find a reasonably bright child > to explain addition, subtraction, > multiplication and division to you. You demanded "sound > reasoning" when I thought you > could do simple arithmetic all on your own, without me to > lead you every step of the way. >> >>> **** >>>> >>>>A possibly better way to handle this would be to have >>>>the >>>>3.5 minute job completely yield to the high priority >>>>jobs, >>>>and then pick up exactly where it left off. >>> **** >>> No, the better way is to use >>> priority-inversion-prevention >>> algorithms as documented in the >>> realtime literature. I went to a lecture on these last >>> year some time, and learned in 40 >>> minutes what progress had been made in >>> priority-inversion-prevention. Clever stuff. >>> **** >> >>From what I have been told, the only possible way that >>priority inversion can possibly occur is if there is some >>sort of dependency on a shared resource. I don't see any >>shared resource in my new design with four different >>independent processes. > **** > You don't have the faintest clue what you are talking > about, do you? You have solved the > priority inversion problem by creating a design that > minimizes utlization of computing > resources and which, by its very nature, guarantees > absolutely worst-case response under > load! According to my training, we call this "bad > design". > **** >> >>>> >>>>>>Just the pipe name itself it part of the disk, nothing >>>>>>else >>>>>>hits the disk. There are many messages about this on >>>>>>the >>>>>>Unix/Linux groups, I stated a whole thread on this: >>>>> **** >>>>> And the pipe grows until what point? It runs out of >>>>> memory? Ohh, this is a new >>>>> interpretation of "unlimited pipe growth" of which I >>>>> have >>>>> been previously unaware! >>>> >>>>It does not ever simply discard input at some arbitrary >>>>queue length such as five items. One of the FIFO models >>>>did >>>>just that. >>> **** >>> Well, that is not exactly "robust" or "reliable" now, is >>> it? So it either blocks, or >>> discards data. If it blocks, it doesn't matter whether >>> it >>> grows or not. If it discards >>> data, it is not a very good design. >>> joe >> >>The fundamental OS IPC mechanisms are what is discarding >>the >>data. I think that this pertained to TCP sockets >>discarding >>data because their buffer of very few elements had been >>exceeded. > **** > TCP sockets will NEVER discard data; obviously, you cannot > tell the difference between > TCP/IP and UDP. TCP uses a positive-acknowledgement > protocol and essentially what we > might call a "distributed semaphore" model of queue > management, which if you had ever read > anything about it, you would immediately have known. I > have no idea why you create > imaginary problems due to your lack of understanding and > solve them with fantasy solutions > whose reliability is problematic at best. > > TCP/IP is a reliable protocol in that you will receive > EVERY byte the sender sends, IN > ORDER, NO DUPLICATES, NO LOST DATA, EVER! or both the > sender and the receiver will receive > a notication of failure (usually a lost connection because > of failure of the sender or > receiver). Where, in this specification, is there > anything about it being allowed to > whimsically throw away data? (Hint: the correct answer is > "nowhere") > > Again, you have rejected a reasonable design alternative > because of complete ignorance of > the details! > > And how do you detect/recover from ANY IPC mechanism that > can throw data away? Or does > reliability no longer matter? > joe > **** >> >>> **** >>>> >>> Joseph M. Newcomer [MVP] >>> email: newcomer(a)flounder.com >>> Web: http://www.flounder.com >>> MVP Tips: http://www.flounder.com/mvp_tips.htm >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm |