Can extra processing threads help in this case?
in [MFC]
|
From: Joseph M. Newcomer on 26 Mar 2010 20:33 I filed three (actually four, but we withdrew one when a later one superseded it) patents in the early 1990s, and there is no product yet. Am I a patent troll? You can't use failure to produce a product as evidence of being a patent troll. Not that we didn't try, for years, and finally gave up. Peter may be clueless with respect to modern technology (or in some cases, decades-old technology or any existing instance of the technology) but I think you may be unreasonable in referring to him this way. I don't mind calling him clueless (he demonstrates it time after time) or even being denser than depleted uranium (his failure to listen to us makes this obvious) but "patent troll" implies a level of unethicality that I would be loathe to ascribe to someone without substantiating evidence. I read the patent claims, and they seem sound. Key here is if one of the open source OCR systems uses an identical technique and can be demonstrated to have existed prior to his patent application, is there an issue, and even then, all it would do in the most extreme case is invalidate the patent; it does not prove he has behaved in an unethical fashion. joe On Fri, 26 Mar 2010 16:14:18 -0400, Hector Santos <sant9442(a)nospam.gmail.com> wrote: >Liviu wrote: > >> "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote... >>> "Liviu" <lab2k1(a)gmail.c0m> wrote... >>>> "Hector Santos" <sant9442(a)nospam.gmail.com> wrote... >>>>> You're not going to get anything done because you don't have >>>>> the capacity to do so. You haven't yet in what 2-3 years? > > >>> > >>>> "I filed a provisional patent last August" - Peter Olcott, 12/14/2001 >>>> >>>> (message #584 in thread of 881 at >>>> http://groups.google.com/group/comp.lang.c++/msg/f8161ee71a584326?hl=en) >>> This patent issued in 2005. The task that I am undertaking is very >>> large. >> >> I am not even trying to argue that now. But you are talking _years_ in >> the works, yet demonstrated deep confusion over elementary matters >> and ignored most of the sound advice volunteered here. Then you say >> "I do not have the time to learn inessential new things". Nothing >> personal, of course, and don't know that you even realize it, but that >> paints you somewhere between utterly arrogant and a complete kook. > >He doesn't realize it, thats the problem. > >He filed a frivolous less than $100 or so provisional patent in 2001. >A one page application. It gives you one year to file the full >patent. He was granted the patent in 2006. > >Yet STILL NO PRODUCT in 9 years. Thats a classic definition of a >Patent Troll and he still behavior like one today. > >I doubt he will have it done in the next 10 when it expires but that >shouldn't stop anyone from providing a web OCR service or using the >technology today - hmmmmmmm, it doesn't :) Makes you wonder why he >has no licensees, no VCs taken him up, no company using his product or >services - he has absolute nothing. The patent was frivolous and all >he did was waste his money. Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 26 Mar 2010 20:36 Be careful about what you mean by "real-time priority". Note that if you push a process to Priority 15 you have a severe negative effect on almost everything else in the entire system; if you move up to the priority range 16..31, you have the potential to block the file system indefinitely. None of these are particularly good ideas. joe On Fri, 26 Mar 2010 15:48:56 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:6j3qq5hbdgrtotu8k91pm5mb3db4dgd228(a)4ax.com... >> See below... >> On Thu, 25 Mar 2010 19:20:38 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>> >>>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>message news:64onq5dabohqabp4htku20ajb8q96r06s1(a)4ax.com... >>>> See below... >>>> On Thu, 25 Mar 2010 10:12:56 -0500, "Peter Olcott" >>>> <NoSpam(a)OCR4Screen.com> wrote: >>>> >>>>> >>>>>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>>message >>>>>news:00rmq5hctllab7ursv8q64pq5eiv8s82ad(a)4ax.com... >>>>>> See below... >>>>>> On Thu, 25 Mar 2010 00:01:37 -0500, "Peter Olcott" >>>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>>> >>>>>>> >>>>>>>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>>>>message >>>>>>>news:rdqlq5dv2u8bh308se0td53rk7lqmv0bki(a)4ax.com... >>>>>>>> Make sure the addresses are completely independent >>>>>>>> of >>>>>>>> where the vector appears in memory. >>>>>>>> >>>>>>>> Given you have re-implemented std::vector >>>>>>>> (presumably >>>>>>>> as >>>>>>>> peter::vector) and you have done >>>>>>>> all the good engineering you claim, this shouldn't >>>>>>>> take >>>>>>>> very much time at all. Then you >>>>>>>> can use memory-mapped files, and share this massive >>>>>>>> footprint across multiple processes, >>>>>>>> so although you might have 1.5GB in each process, it >>>>>>>> is >>>>>>>> the SAME 1.5GB because every >>>>>>>> process SHARES that same data with every other >>>>>>>> process. >>>>>>>> >>>>>>>> Seriously, this is one of the exercises in my >>>>>>>> Systems >>>>>>>> Programming course; we do it >>>>>>>> Thursday afternoon. >>>>>>>> joe >>>>>>> >>>>>>>But all that this does is make page faults quicker >>>>>>>right? >>>>>>>Any page faults at can only degrade my performance. >>>>>> *** >>>>>> Denser than depleted uranium. Fewer page faults, >>>>>> quicker. >>>>>> For an essay, please explain >>>>>> in 500 words or less why I am right (it only requires >>>>>> THINKING about the problem) and why >>>>>> these page faults happen only ONCE even in a >>>>>> multiprocess >>>>>> usage! Compare to the ReadFile >>>>>> solution. Compare and contrast the two approaches. >>>>>> Talk >>>>>> about storage allocation >>>>>> bottlenecks. >>>>>> >>>>>> I'm sorry, but you keep missing the point. DId you >>>>>> think >>>>>> your approach has ZERO page >>>>>> faults? You even told us it doesn't! >>>>> >>>>>I was making a conservative estimate, actual measurement >>>>>indicated zero page faults after all data was loaded, >>>>>even >>>>>after waiting 12 hours. >>>> *** >>>> And a memory-mapped file would not show the same >>>> performance? You know this HOW? >>>> **** >>>>> >>>>>> Why do you think a memory-mapped file is going to >>>>>> be different? Oh, I forgot, you don't WANT to >>>>>> understand >>>>>> how they work, or how paging >>>>>> works! >>>>> >>>>>Not if testing continues to show that paging is not >>>>>occurring. >>>> **** >>>> SCALABILITY! SCALABILITY! MAXIMIZE THROUGHPUT! MEET >>>> 500MS PERFORMANCE GOALS! >>>> SCALABILITY! >>>> >>> >>>If there are no page faults without a memory mapped file >>>and >>>there are no page faults with a memory mapped file, then >>>exactly and precisely what is the specific incremental >>>benefit of a memory mapped file? (The performance and >>>memory usage would have to be the same in both cases). >> ***** >> Did you say there were no page faults? I thought you said >> there were 27,000 while the >> data is being loaded! And is it not screamingly obvious >> that if you have the same file >> mapped into 20 processes, that you did not take 27,000 >> page faults in EACH of them? >> Instead, you took a flurry of page faults when you mapped >> the file in and touched the >> pages, and thereafter there are no page faults in ANY >> process that is sharing that >> segment! ZERO! In contrast, in your single-thread >> multiple-process model, each process >> has to undergo 27,000 page faults as it starts up. >> >> Have I not repeatedly said "amortized over ALL processes"? >> **** >>> >>>I don't want a really quick way to load more data when >>>needed, because there is no possible way that this could >>>be >>>nearly as fast as having the data already loaded. >> **** >> I have no idea why you thought I said anything like this. >> Instead, I gave you a >> ZERO-page-fault way to load the data into subsequent >> processes. > >Yet since that is almost never going to occur it helps >almost not at all. > >One process that probably will only have a single thread for >years will load the data exactly once and continue to run >indefinitely unless something goes wrong. This single >process is to as much as possible on Linux or Windows have >real-time priority. > >> >> Note also that the page faults might be what are called >> "soft" faults, that is, when the >> OS goes to look for the page, it discovers it is already >> in memory, and doesn't waste time >> bringing in another copy. So even if the page faults >> occur, they are very CHEAP page >> faults. >> **** >>> >>>> Your method does not scale; our suggestions give you >>>> scalability, maximize throughput, and >>>> probably makes it possible to meet your wonderful 500ms >>>> goal consistently. >>>> joe >>> >>>Yet you have consistently, time after time, again and >>>again >>>never backed this up with reasoning. As soon as you back >>>this up with verifiably correct reasoning, thenn (then and >>>only then) will I agree. I absolutely and positively >>>reject >>>dogma as a source of truth. >> **** >> Hmm.. "Verifiable correct reasoning"....how about ZERO >> PAGE FAULTS ARE BETTER THAN LOTS OF >> PAGE FAULTS? Doesn't that sound good? As to the rest of >> the reasoning, it is in the > >I want zero page faults from the time that my application >has loaded all of its data on. I don't care about the three >minutes a month start up time. > >> MSDN, which I have read, which I have told you that YOU >> should read, and I'm not going to > >You have not given me even enough of an outline of your >reasoning to see if it could possibly make sense. Most of >the reasoning that you have provided about this specific >matter has been based on false assumptions. Most of your >advice has been superb. On this one point we fail to agree. > >> retype it all from memory here. There is no "dogma" here. >> Just obvious conclusions >> anyone who bothered to understand the technology could >> arrive at, just from reading the >> documentation that is available! >> joe >> **** >>> >>>> >>>> **** >>>>> >>>>>> joe >>>>>> **** >>>>>> >>>>>> Joseph M. Newcomer [MVP] >>>>>> email: newcomer(a)flounder.com >>>>>> Web: http://www.flounder.com >>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>>>> >>>> Joseph M. Newcomer [MVP] >>>> email: newcomer(a)flounder.com >>>> Web: http://www.flounder.com >>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>> >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Hector Santos on 26 Mar 2010 22:05 Thats now two people now telling him his wish list for a real time process with a 500ms resident and response time requirement is flawed. Any good developer or just real "Software Engineer" who is beyond programming would already know about these considerations and even if he didn't the new "buzz word" of the day (new to him) is just another thing in his "Wish List" without knowing how it integrated with the rest his wish list. -- HLS Joseph M. Newcomer wrote: > Be careful about what you mean by "real-time priority". Note that if you push a process > to Priority 15 you have a severe negative effect on almost everything else in the entire > system; if you move up to the priority range 16..31, you have the potential to block the > file system indefinitely. None of these are particularly good ideas. > joe > > On Fri, 26 Mar 2010 15:48:56 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >> message news:6j3qq5hbdgrtotu8k91pm5mb3db4dgd228(a)4ax.com... >>> See below... >>> On Thu, 25 Mar 2010 19:20:38 -0500, "Peter Olcott" >>> <NoSpam(a)OCR4Screen.com> wrote: >>> >>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>> message news:64onq5dabohqabp4htku20ajb8q96r06s1(a)4ax.com... >>>>> See below... >>>>> On Thu, 25 Mar 2010 10:12:56 -0500, "Peter Olcott" >>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>> >>>>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>>> message >>>>>> news:00rmq5hctllab7ursv8q64pq5eiv8s82ad(a)4ax.com... >>>>>>> See below... >>>>>>> On Thu, 25 Mar 2010 00:01:37 -0500, "Peter Olcott" >>>>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>>>> >>>>>>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>>>>> message >>>>>>>> news:rdqlq5dv2u8bh308se0td53rk7lqmv0bki(a)4ax.com... >>>>>>>>> Make sure the addresses are completely independent >>>>>>>>> of >>>>>>>>> where the vector appears in memory. >>>>>>>>> >>>>>>>>> Given you have re-implemented std::vector >>>>>>>>> (presumably >>>>>>>>> as >>>>>>>>> peter::vector) and you have done >>>>>>>>> all the good engineering you claim, this shouldn't >>>>>>>>> take >>>>>>>>> very much time at all. Then you >>>>>>>>> can use memory-mapped files, and share this massive >>>>>>>>> footprint across multiple processes, >>>>>>>>> so although you might have 1.5GB in each process, it >>>>>>>>> is >>>>>>>>> the SAME 1.5GB because every >>>>>>>>> process SHARES that same data with every other >>>>>>>>> process. >>>>>>>>> >>>>>>>>> Seriously, this is one of the exercises in my >>>>>>>>> Systems >>>>>>>>> Programming course; we do it >>>>>>>>> Thursday afternoon. >>>>>>>>> joe >>>>>>>> But all that this does is make page faults quicker >>>>>>>> right? >>>>>>>> Any page faults at can only degrade my performance. >>>>>>> *** >>>>>>> Denser than depleted uranium. Fewer page faults, >>>>>>> quicker. >>>>>>> For an essay, please explain >>>>>>> in 500 words or less why I am right (it only requires >>>>>>> THINKING about the problem) and why >>>>>>> these page faults happen only ONCE even in a >>>>>>> multiprocess >>>>>>> usage! Compare to the ReadFile >>>>>>> solution. Compare and contrast the two approaches. >>>>>>> Talk >>>>>>> about storage allocation >>>>>>> bottlenecks. >>>>>>> >>>>>>> I'm sorry, but you keep missing the point. DId you >>>>>>> think >>>>>>> your approach has ZERO page >>>>>>> faults? You even told us it doesn't! >>>>>> I was making a conservative estimate, actual measurement >>>>>> indicated zero page faults after all data was loaded, >>>>>> even >>>>>> after waiting 12 hours. >>>>> *** >>>>> And a memory-mapped file would not show the same >>>>> performance? You know this HOW? >>>>> **** >>>>>>> Why do you think a memory-mapped file is going to >>>>>>> be different? Oh, I forgot, you don't WANT to >>>>>>> understand >>>>>>> how they work, or how paging >>>>>>> works! >>>>>> Not if testing continues to show that paging is not >>>>>> occurring. >>>>> **** >>>>> SCALABILITY! SCALABILITY! MAXIMIZE THROUGHPUT! MEET >>>>> 500MS PERFORMANCE GOALS! >>>>> SCALABILITY! >>>>> >>>> If there are no page faults without a memory mapped file >>>> and >>>> there are no page faults with a memory mapped file, then >>>> exactly and precisely what is the specific incremental >>>> benefit of a memory mapped file? (The performance and >>>> memory usage would have to be the same in both cases). >>> ***** >>> Did you say there were no page faults? I thought you said >>> there were 27,000 while the >>> data is being loaded! And is it not screamingly obvious >>> that if you have the same file >>> mapped into 20 processes, that you did not take 27,000 >>> page faults in EACH of them? >>> Instead, you took a flurry of page faults when you mapped >>> the file in and touched the >>> pages, and thereafter there are no page faults in ANY >>> process that is sharing that >>> segment! ZERO! In contrast, in your single-thread >>> multiple-process model, each process >>> has to undergo 27,000 page faults as it starts up. >>> >>> Have I not repeatedly said "amortized over ALL processes"? >>> **** >>>> I don't want a really quick way to load more data when >>>> needed, because there is no possible way that this could >>>> be >>>> nearly as fast as having the data already loaded. >>> **** >>> I have no idea why you thought I said anything like this. >>> Instead, I gave you a >>> ZERO-page-fault way to load the data into subsequent >>> processes. >> Yet since that is almost never going to occur it helps >> almost not at all. >> >> One process that probably will only have a single thread for >> years will load the data exactly once and continue to run >> indefinitely unless something goes wrong. This single >> process is to as much as possible on Linux or Windows have >> real-time priority. >> >>> Note also that the page faults might be what are called >>> "soft" faults, that is, when the >>> OS goes to look for the page, it discovers it is already >>> in memory, and doesn't waste time >>> bringing in another copy. So even if the page faults >>> occur, they are very CHEAP page >>> faults. >>> **** >>>>> Your method does not scale; our suggestions give you >>>>> scalability, maximize throughput, and >>>>> probably makes it possible to meet your wonderful 500ms >>>>> goal consistently. >>>>> joe >>>> Yet you have consistently, time after time, again and >>>> again >>>> never backed this up with reasoning. As soon as you back >>>> this up with verifiably correct reasoning, thenn (then and >>>> only then) will I agree. I absolutely and positively >>>> reject >>>> dogma as a source of truth. >>> **** >>> Hmm.. "Verifiable correct reasoning"....how about ZERO >>> PAGE FAULTS ARE BETTER THAN LOTS OF >>> PAGE FAULTS? Doesn't that sound good? As to the rest of >>> the reasoning, it is in the >> I want zero page faults from the time that my application >> has loaded all of its data on. I don't care about the three >> minutes a month start up time. >> >>> MSDN, which I have read, which I have told you that YOU >>> should read, and I'm not going to >> You have not given me even enough of an outline of your >> reasoning to see if it could possibly make sense. Most of >> the reasoning that you have provided about this specific >> matter has been based on false assumptions. Most of your >> advice has been superb. On this one point we fail to agree. >> >>> retype it all from memory here. There is no "dogma" here. >>> Just obvious conclusions >>> anyone who bothered to understand the technology could >>> arrive at, just from reading the >>> documentation that is available! >>> joe >>> **** >>>>> **** >>>>>>> joe >>>>>>> **** >>>>>>> >>>>>>> Joseph M. Newcomer [MVP] >>>>>>> email: newcomer(a)flounder.com >>>>>>> Web: http://www.flounder.com >>>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>>>> Joseph M. Newcomer [MVP] >>>>> email: newcomer(a)flounder.com >>>>> Web: http://www.flounder.com >>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>> Joseph M. Newcomer [MVP] >>> email: newcomer(a)flounder.com >>> Web: http://www.flounder.com >>> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm -- HLS
From: Joseph M. Newcomer on 26 Mar 2010 22:41 I see a lengthy debate about thread priority coming... As I tell my students, "Mucking around with thread priorities is your last resort in engineering a system. SOMETIMES it is what you need, but playing around with thread prioitites is certain to get you into trouble, and must be done VERY CAREFULLY if at all." But I see another lengthy discussion coming...with critical concepts like "priority inversion" being ignored. joe On Fri, 26 Mar 2010 22:05:42 -0400, Hector Santos <sant9442(a)nospam.gmail.com> wrote: >Thats now two people now telling him his wish list for a real time >process with a 500ms resident and response time requirement is flawed. > >Any good developer or just real "Software Engineer" who is beyond >programming would already know about these considerations and even if >he didn't the new "buzz word" of the day (new to him) is just another >thing in his "Wish List" without knowing how it integrated with the >rest his wish list. > >-- >HLS > >Joseph M. Newcomer wrote: > >> Be careful about what you mean by "real-time priority". Note that if you push a process >> to Priority 15 you have a severe negative effect on almost everything else in the entire >> system; if you move up to the priority range 16..31, you have the potential to block the >> file system indefinitely. None of these are particularly good ideas. >> joe >> >> On Fri, 26 Mar 2010 15:48:56 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: >> >>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>> message news:6j3qq5hbdgrtotu8k91pm5mb3db4dgd228(a)4ax.com... >>>> See below... >>>> On Thu, 25 Mar 2010 19:20:38 -0500, "Peter Olcott" >>>> <NoSpam(a)OCR4Screen.com> wrote: >>>> >>>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>> message news:64onq5dabohqabp4htku20ajb8q96r06s1(a)4ax.com... >>>>>> See below... >>>>>> On Thu, 25 Mar 2010 10:12:56 -0500, "Peter Olcott" >>>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>>> >>>>>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>>>> message >>>>>>> news:00rmq5hctllab7ursv8q64pq5eiv8s82ad(a)4ax.com... >>>>>>>> See below... >>>>>>>> On Thu, 25 Mar 2010 00:01:37 -0500, "Peter Olcott" >>>>>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>>>>> >>>>>>>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>>>>>> message >>>>>>>>> news:rdqlq5dv2u8bh308se0td53rk7lqmv0bki(a)4ax.com... >>>>>>>>>> Make sure the addresses are completely independent >>>>>>>>>> of >>>>>>>>>> where the vector appears in memory. >>>>>>>>>> >>>>>>>>>> Given you have re-implemented std::vector >>>>>>>>>> (presumably >>>>>>>>>> as >>>>>>>>>> peter::vector) and you have done >>>>>>>>>> all the good engineering you claim, this shouldn't >>>>>>>>>> take >>>>>>>>>> very much time at all. Then you >>>>>>>>>> can use memory-mapped files, and share this massive >>>>>>>>>> footprint across multiple processes, >>>>>>>>>> so although you might have 1.5GB in each process, it >>>>>>>>>> is >>>>>>>>>> the SAME 1.5GB because every >>>>>>>>>> process SHARES that same data with every other >>>>>>>>>> process. >>>>>>>>>> >>>>>>>>>> Seriously, this is one of the exercises in my >>>>>>>>>> Systems >>>>>>>>>> Programming course; we do it >>>>>>>>>> Thursday afternoon. >>>>>>>>>> joe >>>>>>>>> But all that this does is make page faults quicker >>>>>>>>> right? >>>>>>>>> Any page faults at can only degrade my performance. >>>>>>>> *** >>>>>>>> Denser than depleted uranium. Fewer page faults, >>>>>>>> quicker. >>>>>>>> For an essay, please explain >>>>>>>> in 500 words or less why I am right (it only requires >>>>>>>> THINKING about the problem) and why >>>>>>>> these page faults happen only ONCE even in a >>>>>>>> multiprocess >>>>>>>> usage! Compare to the ReadFile >>>>>>>> solution. Compare and contrast the two approaches. >>>>>>>> Talk >>>>>>>> about storage allocation >>>>>>>> bottlenecks. >>>>>>>> >>>>>>>> I'm sorry, but you keep missing the point. DId you >>>>>>>> think >>>>>>>> your approach has ZERO page >>>>>>>> faults? You even told us it doesn't! >>>>>>> I was making a conservative estimate, actual measurement >>>>>>> indicated zero page faults after all data was loaded, >>>>>>> even >>>>>>> after waiting 12 hours. >>>>>> *** >>>>>> And a memory-mapped file would not show the same >>>>>> performance? You know this HOW? >>>>>> **** >>>>>>>> Why do you think a memory-mapped file is going to >>>>>>>> be different? Oh, I forgot, you don't WANT to >>>>>>>> understand >>>>>>>> how they work, or how paging >>>>>>>> works! >>>>>>> Not if testing continues to show that paging is not >>>>>>> occurring. >>>>>> **** >>>>>> SCALABILITY! SCALABILITY! MAXIMIZE THROUGHPUT! MEET >>>>>> 500MS PERFORMANCE GOALS! >>>>>> SCALABILITY! >>>>>> >>>>> If there are no page faults without a memory mapped file >>>>> and >>>>> there are no page faults with a memory mapped file, then >>>>> exactly and precisely what is the specific incremental >>>>> benefit of a memory mapped file? (The performance and >>>>> memory usage would have to be the same in both cases). >>>> ***** >>>> Did you say there were no page faults? I thought you said >>>> there were 27,000 while the >>>> data is being loaded! And is it not screamingly obvious >>>> that if you have the same file >>>> mapped into 20 processes, that you did not take 27,000 >>>> page faults in EACH of them? >>>> Instead, you took a flurry of page faults when you mapped >>>> the file in and touched the >>>> pages, and thereafter there are no page faults in ANY >>>> process that is sharing that >>>> segment! ZERO! In contrast, in your single-thread >>>> multiple-process model, each process >>>> has to undergo 27,000 page faults as it starts up. >>>> >>>> Have I not repeatedly said "amortized over ALL processes"? >>>> **** >>>>> I don't want a really quick way to load more data when >>>>> needed, because there is no possible way that this could >>>>> be >>>>> nearly as fast as having the data already loaded. >>>> **** >>>> I have no idea why you thought I said anything like this. >>>> Instead, I gave you a >>>> ZERO-page-fault way to load the data into subsequent >>>> processes. >>> Yet since that is almost never going to occur it helps >>> almost not at all. >>> >>> One process that probably will only have a single thread for >>> years will load the data exactly once and continue to run >>> indefinitely unless something goes wrong. This single >>> process is to as much as possible on Linux or Windows have >>> real-time priority. >>> >>>> Note also that the page faults might be what are called >>>> "soft" faults, that is, when the >>>> OS goes to look for the page, it discovers it is already >>>> in memory, and doesn't waste time >>>> bringing in another copy. So even if the page faults >>>> occur, they are very CHEAP page >>>> faults. >>>> **** >>>>>> Your method does not scale; our suggestions give you >>>>>> scalability, maximize throughput, and >>>>>> probably makes it possible to meet your wonderful 500ms >>>>>> goal consistently. >>>>>> joe >>>>> Yet you have consistently, time after time, again and >>>>> again >>>>> never backed this up with reasoning. As soon as you back >>>>> this up with verifiably correct reasoning, thenn (then and >>>>> only then) will I agree. I absolutely and positively >>>>> reject >>>>> dogma as a source of truth. >>>> **** >>>> Hmm.. "Verifiable correct reasoning"....how about ZERO >>>> PAGE FAULTS ARE BETTER THAN LOTS OF >>>> PAGE FAULTS? Doesn't that sound good? As to the rest of >>>> the reasoning, it is in the >>> I want zero page faults from the time that my application >>> has loaded all of its data on. I don't care about the three >>> minutes a month start up time. >>> >>>> MSDN, which I have read, which I have told you that YOU >>>> should read, and I'm not going to >>> You have not given me even enough of an outline of your >>> reasoning to see if it could possibly make sense. Most of >>> the reasoning that you have provided about this specific >>> matter has been based on false assumptions. Most of your >>> advice has been superb. On this one point we fail to agree. >>> >>>> retype it all from memory here. There is no "dogma" here. >>>> Just obvious conclusions >>>> anyone who bothered to understand the technology could >>>> arrive at, just from reading the >>>> documentation that is available! >>>> joe >>>> **** >>>>>> **** >>>>>>>> joe >>>>>>>> **** >>>>>>>> >>>>>>>> Joseph M. Newcomer [MVP] >>>>>>>> email: newcomer(a)flounder.com >>>>>>>> Web: http://www.flounder.com >>>>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>>>>> Joseph M. Newcomer [MVP] >>>>>> email: newcomer(a)flounder.com >>>>>> Web: http://www.flounder.com >>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>>> Joseph M. Newcomer [MVP] >>>> email: newcomer(a)flounder.com >>>> Web: http://www.flounder.com >>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Hector Santos on 26 Mar 2010 23:25
Joe, there really doesn't have to be any lengthy discussion if he simply listen to experts or if he still doesn't believe us, he should at the very least research what raising the process thread priority means. Just look at what WIN32 API SetThreadPriority() says: http://msdn.microsoft.com/en-us/library/ms686277(VS.85).aspx When manipulating priorities, be very careful to ensure that a high-priority thread does not consume all of the available CPU time. A thread with a base priority level above 11 interferes with the normal operation of the operating system. Using REALTIME_PRIORITY_CLASS may cause disk caches to not flush, hang the mouse, and so on. Really, it can end right there. He can not expect a real time priority class process with a guarantee 500ms resident and response time and expect his machine to function right. -- HLS Joseph M. Newcomer wrote: > I see a lengthy debate about thread priority coming... > > As I tell my students, "Mucking around with thread priorities is your last resort in > engineering a system. SOMETIMES it is what you need, but playing around with thread > prioitites is certain to get you into trouble, and must be done VERY CAREFULLY if at all." > > But I see another lengthy discussion coming...with critical concepts like "priority > inversion" being ignored. > joe > > On Fri, 26 Mar 2010 22:05:42 -0400, Hector Santos <sant9442(a)nospam.gmail.com> wrote: > >> Thats now two people now telling him his wish list for a real time >> process with a 500ms resident and response time requirement is flawed. >> >> Any good developer or just real "Software Engineer" who is beyond >> programming would already know about these considerations and even if >> he didn't the new "buzz word" of the day (new to him) is just another >> thing in his "Wish List" without knowing how it integrated with the >> rest his wish list. >> >> -- >> HLS >> >> Joseph M. Newcomer wrote: >> >>> Be careful about what you mean by "real-time priority". Note that if you push a process >>> to Priority 15 you have a severe negative effect on almost everything else in the entire >>> system; if you move up to the priority range 16..31, you have the potential to block the >>> file system indefinitely. None of these are particularly good ideas. >>> joe >>> >>> On Fri, 26 Mar 2010 15:48:56 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: >>> >>>> One process that probably will only have a single thread for >>>> years will load the data exactly once and continue to run >>>> indefinitely unless something goes wrong. This single >>>> process is to as much as possible on Linux or Windows have >>>> real-time priority. |