Can extra processing threads help in this case?
in [MFC]
|
From: Peter Olcott on 12 Apr 2010 19:49 "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message news:%2375%23fYo2KHA.5820(a)TK2MSFTNGP06.phx.gbl... > Peter Olcott wrote: > >> The ONLY purpose of the free jobs is to get paying jobs. >> The only way that a free job would never get done is it >> my website is earning $10 per second 24/7/365. $864,000 >> per day. Long before that ever happens I will set up a >> cluster of servers just for the free jobs. > > > Geez, practically $1 Million a day, nearly 1/3 billion a > year and you you still reading books nor written a "Hello > World!" program? > > I got a great name for your vapor product - Peter Meter! > > Tell ya what, write me a check for $432K and I will have > it done for you, in, hmmmm lets say, 3 days. Ok, ok, > multiply that by 3. > > Look at this way, your break even point would be 4 hours > once done! > > -- > HLS Of course I doubt that I will ever make that much money by any means, but, that is the only way that the free jobs would never get done. This is only one of three different products that I will eventually have. Each of the subsequent products are developed on the prior products fully implemented technology. It is only my final product that I ever have much of a chance of making big "mass market" sized money. All in all it may be on the order of twenty man-years worth of work all done by one man.
From: Joseph M. Newcomer on 12 Apr 2010 20:18 See below... On Mon, 12 Apr 2010 11:10:08 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Jerry Coffin" <jerryvcoffin(a)yahoo.com> wrote in message >news:MPG.262c6206211fe8d9989864(a)news.sunsite.dk... >> In article >> <abidnZAfALb5HF_WnZ2dnUVZ_jydnZ2d(a)giganews.com>, >> NoSpam(a)OCR4Screen.com says... >> >> [ ... ] >> >>> No. Joe was and continues to be wrong that a machine with >>> plenty of extra RAM ever needs to page out either a >>> process >>> or its data. >> >> It's not a question of whether it *needs* to -- it's a >> simple fact >> that with both Windows and Linux, it will *try* to whether >> it needs >> to or not. Windows it's called the working set trimmer -- >> it's a task > >(1) It makes no sense at all that when the system has an >extra 4 GM of RAM that it would need to page out processes >or their data to disk. >(2) In fact it does not do this paging to disk from two >empirical tests one Windows and Linux after twelve hours. *** Which assertions ignore working set trimmers and other techniques, but unlike the other assertions, he actuall MEASURED this and got REAL DATA to show that there is no paging traffic. And I accepted that, so I'm not sure why I am accused on "continuing" to be wrong about this. **** > > >>> No, the latest analysis indicates that I am back up to >>> 100 >>> because the webserver and the OCR execute in parallel. >> >> On a single core machine? There are a few pieces that can >> execute in >> parallel (the OCR can use the CPU while the network >> adapter is >> reading or writing data), but with only one core, very >> little really >> happens in parallel -- the whole point of multiple cores >> (or multiple >> processors) is to allow them to *really* do things in >> parallel, >> instead of just switching between processes quickly enough >> for it to >> *look* like they're running in parallel. > >A PENTIUM 4 is the design constraint. **** I think the observation that the Computer History Museum is deaccessioning some of its exhibits is probably the best explanation of why a Pentium 4 is the design constraint. I junked my last one years ago. **** >After further analysis I may still get 100 TPS because the >heavy loads are: >(1) Writing the transactions to disk >(2) Processing the OCR CPU intensive > >> Keeping track of average vs. peak load is easy -- dealing >> with it >> (given a task as processor intensive as you've suggested) >> is not. > >There are many simple ways to deal with it. Postpone all non >critical jobs to off peak periods. If this is not enough >then I will use the $10 per second to buy a better solution. *** I love the sales projections. Throughput potential != income. **** > >> Seriously, you'd be a lot better off with a "cloud" >> computing >> provider than one that gives you only a single core. >> Assuming your >> OCR really works, there's a pretty fair chance that the >> work pattern >> will be substantially different than you seem to >> imagine -- instead >> of a page or two at a time, you're (reasonably) likely to >> receive >> scans of an entire book at a time. > >If these are sent as individual pages they are provided the >highest priority, if these are sent as one very huge PNG >file, they may await off peak processing. **** Which is why the SQMS priority-ordered queue model works better **** > >> >> This is a scenario where something like Amazon's EC2 would >> work well >> -- you pay only for processor time you actually use, but >> if you get a >> big job, it can run your task on dozens or even thousands >> of >> processors so (for example) all the pages from a book are >> OCRed in >> parallel, the results put back together, and your customer >> gets his >> result back quickly. Then your system goes idle again, and >> you quit >> paying for *any* processor time until another task comes >> along. **** Gee, sounds like SQMS to me! **** > >I don't think that this model on my current service provider >provides enough storage. > >> >> -- >> Later, >> Jerry. > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 12 Apr 2010 20:24 See below.. On Mon, 12 Apr 2010 14:12:18 -0400, Hector Santos <sant9442(a)nospam.gmail.com> wrote: > >Joseph M. Newcomer wrote: > >>>> >>>>> Alternative (a) There are four processes with four queues >>>>> one for each process. These processes only care about >>>>> executing the jobs from their own queue. They don't care >>>>> about the jobs in any other queue. The high priority >>>>> process >>>>> is given a relative process priority that equates to 80% >>>>> of >>>>> the CPU time of these four processes. The remaining three >>>>> processes get about 7% each. This might degrade the >>>>> performance of the high priority jobs more than the next >>>>> alternative. >>>> There is no such thing with any OS of which I'm aware. At >>>> least with >>>> a typical OS, the highest priority task is the *only* one >>>> that will >>>> run at any given time. Windows (for one example) does >>>> attempt to >>>> prevent starvation of lower priority threads by waking one >>>> lower >>>> priority thread every four seconds. >>> The alternative that you show quoted above is called time >>> slicing and has been available for many decades. >> **** >> And therefore timeslicing takes ZERO overhead, right? And you are still thinking that >> mucking with thread priorities is going to result in a flawless design (and if you start >> ranting about how "thread" and "processes" are different, you will only confirm that you >> are completely and utterly clueless) > > >I'll tell ya Joe, this guy needs a Step 12 program! > >>> My scheduler will signal all of the low priority jobs that >>> they need to sleep now. When the high priority queue is >>> empty, and all of the high priority jobs are completed the >>> low priority jobs get a signal to wake up now. >> **** >> Oh, a whole NEW mechanism! Has anyone said "this is without a doubt the dumbest design I >> have ever seen"? Well, it is. You really don't understand operating systems, because the >> ideas you state here show a fundamental confusion about the right way to build >> applications. > > > > >> I presume you mean the linux 'signal' operation. Have you really studied the problems >> involved in using it? >> **** > > >I think he really wants a > > HttpThread() > { > WaitForSingleOjbject(hHighPriorityProcessInProgress,INFINITE); > .... > } > >or > > WebServer() > { > for each thread handle, suspend all low priority > threads if new request is high priority. > > start HttpThread() thread. > } > >>> Block IP long before that. >> **** >> WHat IP? You clearly have some bizarre notion that the IP is unforgeable. If I wanted to >> do a D-O-S on your site, I'd change the IP with every transmission. You have some awfully >> naive views about the goodness of the world. Or are we talking about Peter's Fantasy >> Internet, where everyone is well-behaved, there are not D-O-S attacks, and nobody EVER >> emails a virus? >> **** > >No Joe, he will require every customer to have a verifiable static IP >before signing up. That IP will be part of the user record and will >never change. :) **** Verizon is my pipe provider; I use their FIOS pipe. I have a dynamically-assigned IP address, which changes at least once a day. So he is going to preclude everyone using a FIOS connection? Not to mention those using cable-TV providers, who also use dynamic IP addresses? Hasn't anyone explained to him that using the IP address for validation doesn't work? Oh, never mind, I remember now, *I* explained it a while ago. I remember talking about NAT servers and how they transform IP addresses and why IP validation doesn't work if NAT is in the way, and nearly every ISP uses some form of NAT to go to *their* backbone provider.... joe **** Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 12 Apr 2010 20:29 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:11s6s51ufidbrigq8g1u0nl1jkj6hhfkmf(a)4ax.com... > See below.... > On Sun, 11 Apr 2010 21:33:33 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>What is the exact scenario that produces this massive >>delay? > *** > Sorry, I already explained this, complete with the > arithmetic to prove it. I don't feel > llike repeating the obvious. I don't believe you, this is merely a ruse. If you are going to go on and on and on without ever getting to the point then you are a harasser and not at all a helper. > ***** >> >>> whereas a SQMS model with priority-inversion prevention >>> can MINIMIZE end-to-end delays in >>> the server. >> >>On a single core processor? If it does I don't see how. >>You >>have to explain these details. On a quad core it is almost >>obvious how it could help. > **** > Yep, on a single-core processor! One of the little > details is one of the most > carefully-guarded secrets of modern operating systems, so > I;m not surprised you haven't > heard about it. It is called "time slicing", and I could > tell you more about it, but then > I'd have to kill you, because this top-secret technique is > known only to very few > initiates. I may have violated my sacred oaths even by > hinting at its existence, and I > will have to watch out for the high priests of operating > systems, who may declare me > excommunicated for revealing it. > **** Yes yet another ruse. You are merely flatly wrong again and hiding it behind rhetoric. >> >>> >>> But then, performance clearly is not an important >>> consideration, or you would want a >>> design that minimizes end-to-end transaction time in the >>> server under high load >>> conditions. And you would not be so insistent that we >>> acknowlege your design must be >> >>No it is just you ignoring design constraints again. >>Single >>core not quad core. > **** > I am curious where you are finding these single-core > machines? The antique sales on eBay? > Is your ISP really willing to support these for you? > > But SQMS works better on single-core machines (see > reference to that secret technique > called "time slicing") So I am already doing that with my MQMS, and because I have been told that Linux does not do a very good job with threads, my implementation is likely superior to yours. I have seen no reasoning to the contrary. I equate the absence of reasoning with the lack of truth because the ONLY correct measure of truth is sound reasoning. Credibility is a pretty crappy stand-in for sound reasoning (valid reasoning based on true premises). > joe > **** >> >>> right, when I was able to demonstrate, with third-grade >>> arithmetic, that it isn't very >>> good. >>> joe >>> >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 12 Apr 2010 20:28
See below... On Mon, 12 Apr 2010 15:53:23 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:aqk6s5h3hjvp5drio1iqfqimj2b269r3ad(a)4ax.com... >> See below... >> On Mon, 12 Apr 2010 10:05:29 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> > >>>The alternative that you show quoted above is called time >>>slicing and has been available for many decades. >> **** >> And therefore timeslicing takes ZERO overhead, right? And >> you are still thinking that >> mucking with thread priorities is going to result in a >> flawless design (and if you start >> ranting about how "thread" and "processes" are different, >> you will only confirm that you >> are completely and utterly clueless) > >It looks like using nice to provide both the webserver and >exactly one of four OCR processes a priority of exactly -1 >is the best solution. So far no one has provide sound >reasoning to the contrary. **** The optimism of naivete knows no bounds! Have fun! You will be quite suprised to discover what really happens! I hope you will be able to use "sound reasoning" to explain what is going on when it doesn't work... **** > >Likewise with four queues and four OCR processes on a >PENTIUM 4. > >The priority queue may work very well on a quad-core, I see >no way to implement it to meet my design goals on a PENTIUM >4. **** Mostly, I think, due to the difficulty of finding a working Pentium 4 processor to run anything on... Actually, the priority queue works better on a single core machine, but I've tried to explain why this works, and you have ignored the explanations. **** > > >>>Block IP long before that. >> **** >> WHat IP? You clearly have some bizarre notion that the IP >> is unforgeable. If I wanted to >> do a D-O-S on your site, I'd change the IP with every >> transmission. You have some awfully >> naive views about the goodness of the world. Or are we >> talking about Peter's Fantasy >> Internet, where everyone is well-behaved, there are not >> D-O-S attacks, and nobody EVER >> emails a virus? > >How else do you block a DoS attack? **** YOU don't. You let your ISP block it for you. Note that your belief that the IP address is going to be a reasonable approach shows how little you understand about Internet protocols or (as I have explained in the past) how D-O-S attacks work (each packet has a forged IP address which is fictional, PRECISELY so you can't use the IP address to detect and block the attack!) joe **** > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm |