Can extra processing threads help in this case?
in [MFC]
|
From: Hector Santos on 25 Mar 2010 10:35 Peter Olcott wrote: >> Even though, given his limitations by his own design, he >> picked the wrong software - he wants SQLITE3! It fits in >> perfectly with a single ACCESSOR queuing concept. :) >> >> -- >> HLS > > You got any other idea for a 100% portable FIFO queue that > has maximum fault tolerance? Yes, the world has already put aside this Kindergarden concept! and NO ONE IN THE WORLD uses SQL for a FIFO! If they do, they are PRIMITIVE. I already provided you an example C++ class but you can also use standard file I/O or an ISAM file or better yet a Memory Mapped FILE! > (By fault tolerance I mean yank the power plug from the wall > and when the machine is re-started it (as much as possible) > picks up right where it left off) Then you have NO CHOICE but to avoid (reduce) Cache and buffer I/O, i.e. keep all your memory and data virtualized on disk to minimize your integrity issue and to maximize restarts with least data lost. KIDDIE STUFF! -- HLS
From: Peter Olcott on 25 Mar 2010 10:35 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:otnmq553u2c9hdhabfe3ss74uct7pv0flu(a)4ax.com... > See below... > On Wed, 24 Mar 2010 23:14:32 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>message news:snnlq5lfv1a59n003k5gkk8a6plb41ruj1(a)4ax.com... >>> See below... >>> >>> On Tue, 23 Mar 2010 14:11:28 -0500, "Peter Olcott" >>> <NoSpam(a)OCR4Screen.com> wrote: >>> >>>>Ah so this is the code that you were suggesting? >>>>I won't be able to implement multi-threading until >>>>volume >>>>grows out of what a single core processor can >>>>accomplish. >>>>I was simply going to use MySQL for the inter-process >>>>communication, building and maintaining my FIFO queue. >>> **** >>> Well, I can think of worse ways. For example, writing >>> the >>> data to a floppy disk. Or >>> punching it to paper tape and asking the user to >>> re-insert >>> the paper tape. MySQL for >>> interprocess communication? Get serious! >> >>Can you think of any other portable way that this can be >>done? > **** > EnqueueRequest() which takes a description of the request. > And is implemented on each > platform with platform-specific code that is optimized for > that platform. Your concept of > "Portable" is incorrect here. you want the FASTEST way > that the platform supports, and > that may be completely different from every other > platform. But if you code has a single > abstraction, you are free to implement it in the best way > on each platform! Is this supported on every platform? > > Of course, this is so screamingly obvious that it is also > clear that you wouldn't want to > listen to any idea that doesn't fit your preconceived > notion. > *** >>I would estimate that MySQL would actually keep the FIFO >>queue resident in RAM cache. > **** > Oh, it's the guesswork-and-wishful-thinking approach ALL > OVER AGAIN! DUH! And what part > of "don't guess, measure" did you fail to understand from > the last week of messages? WHY > do you believe MySQL will behave this way? Cite the > appropriate chapter from the MySQL > manual, or, since it is open-source, show the subroutine > that guarantees this! > > But feel free to live in your fantasy world! When the > Real World accidentally matches > your fantasies, you will win. > joe > > **** >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Hector Santos on 25 Mar 2010 10:40 Peter Olcott wrote: > "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message >> Its a moot point now. You can't run multiple socket >> servers on the same port 80 Peter. Oh, they will start >> up, but you won't get any secondary processors trying to >> bind on port 80 to listen on it. That is still among one >> of #1 support questions when a new customer purchases our >> service products, installs it and runs it and he test it >> with his browser and it doesn't connect - he has IIS >> running in the background. So either he turns it off or >> he uses a different port. >> >> You have no choice but to follow the proper multi-thread >> design server models. :) > That should not be any issue because data processing time > will always be a large multiple of HTTP processing time. Huh? What does that have to do what you can't have multiple EXE running with their own web server? > Web server listens at port 80, and creates a separate thread > (possibly from a thread pool) to handle each HTTP request. > It places each request in some sort of FIFO queue. OCR > process has one or more threads that pull requests from this > FIFO queue. But thats a SINGLE PROCESS with 1 OCR. You can not run this multiple times because the OS will not allow a 2nd or more web server to bind on the same port. You need to go back to separating the WEB SERVER from the OCR.EXE process and now we go back to my early post where you need to work out the MIDDLE WARE, the OCR Protocol Interface logic. WEB <--> INTERFACE <--> 4 instances of OCR.EXE -- HLS
From: Peter Olcott on 25 Mar 2010 10:46 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:otnmq553u2c9hdhabfe3ss74uct7pv0flu(a)4ax.com... > See below... > On Wed, 24 Mar 2010 23:14:32 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>message news:snnlq5lfv1a59n003k5gkk8a6plb41ruj1(a)4ax.com... >>> See below... >>> >>> On Tue, 23 Mar 2010 14:11:28 -0500, "Peter Olcott" >>> <NoSpam(a)OCR4Screen.com> wrote: >>> >>>>Ah so this is the code that you were suggesting? >>>>I won't be able to implement multi-threading until >>>>volume >>>>grows out of what a single core processor can >>>>accomplish. >>>>I was simply going to use MySQL for the inter-process >>>>communication, building and maintaining my FIFO queue. >>> **** >>> Well, I can think of worse ways. For example, writing >>> the >>> data to a floppy disk. Or >>> punching it to paper tape and asking the user to >>> re-insert >>> the paper tape. MySQL for >>> interprocess communication? Get serious! >> >>Can you think of any other portable way that this can be >>done? > **** > EnqueueRequest() which takes a description of the request. > And is implemented on each > platform with platform-specific code that is optimized for > that platform. Your concept of > "Portable" is incorrect here. you want the FASTEST way > that the platform supports, and I know what portable means. > that may be completely different from every other > platform. But if you code has a single > abstraction, you are free to implement it in the best way > on each platform! I already knew that. One respondent on the forum that is supposed to know this stuff comp.programming.threads suggested the disk file approach. If EnqueueRequest() is portable across at least Windows/Unix/Linux then this may be the way that I go. This will require a learning curve, but, this learning curve may be justified. Do you have any ideas on maximizing fault tolerance? I was thinking that the only possible basis for fault tolerance would require some sort of persistent storage. > > Of course, this is so screamingly obvious that it is also > clear that you wouldn't want to > listen to any idea that doesn't fit your preconceived > notion. > *** >>I would estimate that MySQL would actually keep the FIFO >>queue resident in RAM cache. > **** > Oh, it's the guesswork-and-wishful-thinking approach ALL > OVER AGAIN! DUH! And what part > of "don't guess, measure" did you fail to understand from > the last week of messages? WHY > do you believe MySQL will behave this way? Cite the > appropriate chapter from the MySQL > manual, or, since it is open-source, show the subroutine > that guarantees this! > > But feel free to live in your fantasy world! When the > Real World accidentally matches > your fantasies, you will win. > joe > > **** >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 25 Mar 2010 10:48
"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:tmomq5tj59i46ufn9hvv2pf9gho5tedhoa(a)4ax.com... > See below... > On Wed, 24 Mar 2010 23:15:49 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>message news:snnlq5lfv1a59n003k5gkk8a6plb41ruj1(a)4ax.com... >>> See below... >>> >>> On Tue, 23 Mar 2010 14:11:28 -0500, "Peter Olcott" >>> <NoSpam(a)OCR4Screen.com> wrote: >>> >>>>Ah so this is the code that you were suggesting? >>>>I won't be able to implement multi-threading until >>>>volume >>>>grows out of what a single core processor can >>>>accomplish. >>>>I was simply going to use MySQL for the inter-process >>>>communication, building and maintaining my FIFO queue. >>> **** >>> Well, I can think of worse ways. For example, writing >>> the >>> data to a floppy disk. Or >>> punching it to paper tape and asking the user to >>> re-insert >>> the paper tape. MySQL for >>> interprocess communication? Get serious! >> >>This was suggested on the comp.programming,threads group >>Cross Platform FIFO Queue > **** > Which apparently is inhabited by equally clueless people. > Or by people who clearly have > no interest in performance! Or maybe they decided to give > you a facile answer so you > would stop bothering them....one which was just pulling > your chain. > > Or they were testing the limits of your credulity. > Reminds me of the Calvin & Hobbs > cartoon: The family is in the card. Calvin: "Dad, how do > they determine the weight limit > of a bridge?" Dad: "They run bigger and bigger trucks > over it until it collapses, then > they rebuild it exactly and post the weight limit" > > Can you explain why you would accept, without question, > such a patently absurd suggestion > from one newsgroup while ignoring all the good advice > you've been getting in this one? > joe Can you provide any other basis for maximizing fault tolerance that does not require some sort of persistent storage? > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm |