Can extra processing threads help in this case?
in [MFC]
|
From: Hector Santos on 14 Apr 2010 19:24 Well, otherwise you stuck teaching him next about Query Dissemination Theory. :) http://www.google.com/search?q=Query+Dissemination+Theory Look, understand one thing - he is stuck with a single thread process OCR.exe. He has proven he can not capable of programming for threads, for any OS. He has asked how to make it thread safe. That is why he is persistent with this single FIFO for OCR process thing. He must frame the environment around an "one job at a time" OCR process. So at best, he has to use four instances to separate the job types. Which is fine, a waste, but he will need a web proxy to delegate it all and this proxy should be the one that starts the other four processes to get some parent/child controls. -- Jerry Coffin wrote: > In article <e4S4Gt$2KHA.4540(a)TK2MSFTNGP04.phx.gbl>, sant9442 > @nospam.gmail.com says... > > [ ... ] > > > Jerry, I don't wish to act or play moderator, by no means do I wish to > > show any disrespect here. He got his answers to a wide degree, but no > > amount of insight by scientists, engineers and experts in the field is > > good enough. Its really time to ignore this troll. > > As much as I prefer to give people the benefit of the doubt, I'm > quickly realizing that you're probably right. > > -- > Later, > Jerry.
From: Joseph M. Newcomer on 25 Apr 2010 15:04 See below.. On Wed, 14 Apr 2010 08:04:28 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Jerry Coffin" <jerryvcoffin(a)yahoo.com> wrote in message >news:MPG.262f14c976a813a1989878(a)news.sunsite.dk... >> In article >> <XIqdne3OYaoRX1nWnZ2dnUVZ_qOdnZ2d(a)giganews.com>, >> NoSpam(a)OCR4Screen.com says... >>> >>> "Jerry Coffin" <jerryvcoffin(a)yahoo.com> wrote in message >>> news:MPG.262e5c39c3902417989876(a)news.sunsite.dk... >> >> [ ... The buffer built into the hard disc: ] >> >>> Because of required fault tolerance they must be >>> immediately >>> flushed to the actual platters. >>> >>> > Though it depends somewhat on the disk, most drives >>> > store >>> > enough power on board (in a capacitor) that if the >>> > power dies, >>> > they can still write the data in the buffer out to the >>> > platter. >>> > As such, you generally don't have to worry about >>> > bypassing it to >>> > assure your data gets written. >>> >>> When you are dealing with someone else's money >>> (transactions >>> are dollars) this is not recommended. >> >> [ ... ] >> >>> Buffer must be shut off, that is exactly and precisely >>> what >>> I meant by [all writes are forced to disk immediately]. >> >> Quite the contrary. Disabling the buffer will *hurt* the >> system's >> dependability -- a lot. The buffer allows the disc to use >> an elevator >> seeking algorithm, which minimizes head movement. The >> voice coil that >> drives the head is relatively fragile, so minimizing >> movement >> translates directly to reduced wear and better >> dependability. >> >> Disabling the buffer will lead almost directly to data >> corruption. >> None of the banks, insurance companies, etc., that I've >> worked with >> would even *consider* doing what you think is necessary >> for financial >> transactions. > >Several experts disagree with this statement. From what I >recall including Joe. (He may have been intentionally >leading me astray, though) **** I deeply resent that suggestion. I may be rude to someone who fails to listen, but I do not give erroneous information! **** >One expert stated that many of >the high quality drives come with buffering turned off by >default. I already know that truth is not a democracy and >these four or five people could all be wrong. ***** Read the disk controller manual. Most of these are approximately as thick as a telephone directory for a major city (e.g., New York, Washington DC). I once spent three days just finding the right PAGE to figure out what the ATAPI command string was! And in those drives, the buffering was on by default. And who cares? What you CARE about is whether or not your device driver enables or disables the feature! You clearly come form some fantasy world where all software works just the way you would like it to; I wonder what the Device Driver Requirements for PFOS actually says about this! ***** > >I do remember another line-of-reasoning that may be the one >that you are referring to. This only works if the source of >the problem is a power loss. There are some RAID controllers >that have a battery backup directly on the controller. When >it goes into battery backup mode, it immediately flushes its >buffers to disk. What happens if the source of the problem >has another cause such as either the program or the >operating system locking up? Is this simply a case where the >bank gets the transaction wrong? **** Banks have more sophisticated models of transaction processing going on. When I wrote banking software, drives were as big as washing machines and did not have buffering, so I can't tell what has to be done today. But I do know what one of the tech guys told me at a trade show (which is why you should never allow tech guys on the trade show floor: they are likely to tell inconvenient truths) ***** > >>> I think that the figure that I quoted may have already >>> included than, it might really be access time rather than >>> seek time. I am so unused to the c/c++ library lseek and >>> fseek meaning that, that I may have related the incorrect >>> term. >> >> I doubt it -- for the sake of "nice" numbers, most drive >> manufacturers like to quote the fastest sounding things >> they can. In >> fact, they'll often quote the time only for the actual >> head movement, >> even leaving out the time for the controller hardware to >> translate >> the incoming command into signals sent to the servo (which >> makes no >> real sense at all, since there's no way for you to >> actually bypass >> the controller). > >You are playing with me here? One of these is in >milliseconds and the other one is probably in nanoseconds. **** Actually, I have talked with the people who write the servo software, and it is not anywhere near as straightforward as you might think. There are LOTS of internal queues, which serially form a long pipeline between the computer interface and the platters (no, it is not MQMS; it is SQ[M|S]S->SQ[M|S]S->SQ[M|S]S, and I helped write a simulator that modeled this, and I cannot say more without violating NDA (the architecture of controller queues is well-documented in the literature, but the individual details are proprietary). **** > >> >> [ ... ] >> >>> In any case access time still looks like it is the >>> binding >>> constraint on my TPS. >> >> Perhaps -- and perhaps not. Right now, the "binding >> constraint" isn't >> access time; it's complete lack of basis for even making >> informed >> guesses. I'll say it again: for your guesses to mean >> anything at all, >> you need to put together at least a really minimal system >> and do some >> measurements on it. Without that, it's just hot air. >> >> -- >> Later, >> Jerry. > >I like to stick with analysis until most everything is >known. The analytical answer that would determine how long >(the disk access aspect of) a transaction takes would seem >to most crucially depend on whether you or five other >experts are correct on the issue of turning off the hard >drive cache. **** But the point we have been making for weeks is that you are trying to outguess the behavior of very complex sets of interacting systems. The people who have cared about this for decades KNOW you can't "analyze" the behavior without building a real system and measuring it! Your mythology of disk access time (for example, it appears you confuse the fseek/_seek operations with disk arm motion, although there is absolutely NOTHING that suggests such a correlation!) is an example of pure guesswork with ZERO analysis possible. So you keep flailing about, pulling numbers from out of various parts of your anatomy, and then having derived these numbers from Tarot cards, use them as parameters in a poorly-understood operating environment. This is not a viable design methodology, and those of us who have built large, complex systems KNOW this! Perhaps my several years at the SEI biased me against your methodology; DoD was very good at contracting out projects to people who used random-number-generators to obtain performance parameters, and delivered systems that, after tens of millions of dollars of investment, did not work, and were so interwoven with their fanciful design parameters that they could not be changed! NOBODY who has the slightest pretension to sanity would take your approach; we would instead build a flexible system, run it, measure it under real or synthetic load, and then start tuning it! DIdn't you read that essay about the capabilities added to linux to support Web delivery? It siad that that was EXACTLY what they did! joe ***** > >Since no amount of testing can possibly provide this answer, >and the test results completely depend upon this answer, I >must await this answer before testing becomes feasible. > >Perhaps you can provide complete and sound reasoning (even a >good link or two would do) showing that these other five >experts are wrong. You have already made a good start of it >by referring to the fragility of the read/write head. Joe >also mentioned this. > >The biggest missing piece is how a transaction can be >reversed even when the application or operating system >crashes. I might have figured that one out. It would not >matter if the application crashes if the database manager is >in a separate process. I still don't know what could be done >about an OS crash in the middle of a transaction besides >turning off hard disk write caching. > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 26 Apr 2010 01:15 Unforunately, the link below goes to a site which attempted to attack my machine by running a client-side script, and three layers of firewall stripped out the JavaVirus attack. So I have no idea what it might have said, because the site obviously does not care to have people who care about security read it. joe On Tue, 13 Apr 2010 14:36:36 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:rog9s55uk4kddce41ia7p8lt7cvooee5a1(a)4ax.com... >> See below... >> On Mon, 12 Apr 2010 22:15:39 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>>A single queue is much more difficult. Write at whatever >>>location is appropriate at the moment and read from the >>>head >>>of whichever portion of the queue applies to this >>>priority. >>>What do you do a linear search for the head? If you don't >>>do >>>a linear search and keep track of the different heads >>>separately then this is essentially four queues, simply >>>strung together. How can that be better than four queues? >> **** >> No, you keep the queue in sorted order. You, of course, >> have performance numbers which >> (since you won't experiment) you can derive from sound >> reasoning to demonstrate that what >> is going on there is ultimately a provably correct design >> decision. >> >> Otherwise, you are just ranting. You can't prove you are >> right, so you tell us we must be >> wrong because we disagree with you. >> >> I want to see this sound reasoning laid out in detail, >> with numbers to go with it. >> joe >> **** > > >http://docs.google.com/viewer?a=v&q=cache:Hb_P22Cj9OAJ:citeseerx.ist.psu.edu/viewdoc/download%3Fdoi%3D10.1.1.93.429%26rep%3Drep1%26type%3Dpdf+multi+queue+multi+server&hl=en&gl=us&pid=bl&srcid=ADGEESh1kerH3RGqAvIEul4ryHpwxxU5HdWzS3edrtXW764CJUPudOBFnvTmUvl7W3uBXqe046N1tNkirmGqVOkUlmWQWTZQgLLwQHf5LolcXX43mvOEc3k0wR55vXqYAklq8Fu2-qgL&sig=AHIEtbSrBAf6HW8XDtNinTOdsNx5lf9tNQ > >Google below if the above link does not work >On the stability of the multi-queue multi-server processor >sharing with limited service > > >>> >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 26 Apr 2010 01:27
See below... On Wed, 14 Apr 2010 07:30:50 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message >news:uoQRoC02KHA.5420(a)TK2MSFTNGP05.phx.gbl... >> Joseph M. Newcomer wrote: >> >>> >>> So how is it that your unsubstantiated opinions are so >>> reliable and nobody else's opinions >>> can be trusted? >>> joe >> >> Joe, he has have the same battles and issues, I mean the >> SAME, with the Linux group. Read the thread here: >> >> http://www.groupsrv.com/linux/post-923554.html >> >> Same mis-understandings, same answers, even the "Great" >> David Schwartz and practically everyone else have told him >> same things. And all of them are recognizing how >> "uneducable" he is. Read it all six pages of it and see >> the Deja Vu. I loved David's statement: > >Yes there were a small number of cases in those groups where >my communication style proved to be less than ideal. None of >these people were rude though. > >It would have probably been much more effective if I started >off with [What is the best way to achieve this functional >objective?] > >I tried that here and Joe completely ignored this correct >[functional objective] statement and went on and on about >how [functional objective] statements should never have >implementation details, even though this one had no >implementation details. > >He said that he was not talking about what I just said (the >correct functional objective statement immediately above his >comments) but some of the other times that I did not form >correct functional objective statements, and did include too >many implementation details. > >He completely ignored anything at all that I said that was >correct, and ever resorted to deceitful means if that is >what it took to continue to be hyper critical of anything >that I said. > >At this point I concluded that Joe's intended purpose was >not to be helpful, but hurtful. > **** Probably because the "functional objective" stated was one of the Magic Morphing Requirements that changed on every post, and often changed in ways that were inconsistent with previously stated "non-negotiable" functional objectives. It is hard to track exactly what is going on when the requirements change with each question. My purpose is to hit you upside the head and tell you to go out and LEARN SOMETHING about the systems you want to use! I have spent my entire life trying to understand how these complex systems work, and you walk in and want trivial answers to unanswerable questions, and It Ain't Gonna Happen. The longer you persist in asking questions for which you expect simple answers, the ruder I become, because you are obviously not paying any attention to the meta-lesson, which is There Ain't Any Simple Answers. We know this; you obviously cannot begin to grasp it, and instead want to build "optimal" solutions in a space in which it is simply not possible without actually measuring real systems and understanding how they work. Instead, you start spewing nonsense, for which I have no patience, and often your "functional objectives" aren't even making sense. As someone in the linux group commented, you are not even capable of asking a coherent question (part of the problem is that your questions, with their functional objectives, embody an amazing amount of nonsensical premises and are not even internally consistent, and the functional objectives are often unrealizable, which is why we tend to ignore them; for example, if you state that your functional objective is to grow wings and fly, we are not likely to focus on the answering details of how pinion feathers should work) joe Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm |