Can extra processing threads help in this case?
in [MFC]
|
From: Joseph M. Newcomer on 30 Mar 2010 13:21 On Mon, 29 Mar 2010 21:30:29 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:0fn1r513fgotrpqe1u5puurd977jgrb240(a)4ax.com... >> See below... >> On Mon, 29 Mar 2010 10:19:07 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> > >It is only considered a transaction if the client request is >committed to disk. **** And you are willing to pay the performance penalty this incurs? I though performance was EVERYTHING and ZERO page faults was the accepted upper bound! And I presume you know the costs, in milliseconds, for the database commit. If I were going down this design route, I would already have that number. I would either get it from the documentation, from a benchmark test supplied by the vendor which I would run on my machine, or my own test. But I would not proceed until I had that number in hand. **** > >> >> Your failure to deal with the concept that "no client will >> be charged for an uncompleted >> transaction" is going to get you into trouble (seriously). >> You have not suggested a >> mechanism by which a lost transaction (e.g., due to >> catastrophic server failure) will be >> sent back to the client, > >Sure I have you just haven't gotten to it yet, email. I >haven't got any more time for this discussion. I am >convinced that I will get it right. I will get back to you >on this when I am pretty sure that I have it right and you >can double check my final design. **** Oh, in your fantasy world, email is a reliable delivery mechanism? It isn't out here in the real world. And did you read Hector's comments on email acknowledgement (and he's far more an expert on email than I am; all I know is that there are serious reliability issues with email; the evidence is that people send me email which I never receive, and which never even arrive at my ISP, although there is a record that they were sent). Apparently, you once read an article on email, and now believe you understand it completely. Even your concept of a "veriiable email address" shows a degree of cluenessness that is, alas, unsurprising. My ISP, who makes his living delivering email to and from me, doesn't understand it completely. Go read a Unix email manual (e.g., from O'Reilly Press) to find out how complex it is, and how hard it is to make it reliable. Oh, the ugliness of reality. I know LOTS of people who can tell horror stories about failing email servers, because they have been responsible for running them (one friend ran the email router that was responsible for all email arriving on the East Coast of the U.S.; it was located in the Pittsburgh Supercomputer Center at that time) but you seem to think email is actually reliable. I guess this is the advantage of using the wishful-thinking approach to design. All those ugly problems go away. joe **** > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: BobF on 30 Mar 2010 13:34 This was amusing for a while, but I'm getting bored now ... <yaaaawwwwwwwwwn ....>
From: Joseph M. Newcomer on 30 Mar 2010 13:41 The fact that I told you about VirtualLock in December is ALSO a matter of public record! I think you should carefully read what I told you about memory-mapped files. It is actually easy; it is in the public record! joe On Tue, 30 Mar 2010 12:14:22 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:ipa4r55sc1jr36e27ub95g7uiot8c6g36g(a)4ax.com... >> See below... >> On Tue, 30 Mar 2010 11:41:12 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>> >>>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>message news:q3v3r594leecok45c018d1m5mcljg48i0i(a)4ax.com... >>>> >>>> >>>> On Mon, 29 Mar 2010 22:33:57 -0500, "Peter Olcott" >>>> <NoSpam(a)OCR4Screen.com> wrote: >>>> >>>>> >>>>>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>>message >>>>>news:uk12r5die24557r1evjkvrqovj58qjvc9e(a)4ax.com... >>>>>> See below... >>>>>> On Sun, 28 Mar 2010 23:03:42 -0500, "Peter Olcott" >>>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>>> >>>>>>>>But you can run >>>>>>>> with virtual memory WITHOUT paging being a part of >>>>>>>> it, >>>>>>> >>>>>>>How is that not like running an automobile's engine >>>>>>>without >>>>>>>any form of fuel? >>>>>> **** >>>>>> Because you are being dense and stupid. For example, >>>>>> a >>>>>> better analogy would be running an >>>>> >>>>>Calling me stupid (an opinion) >>>> **** >>>> Actually, it is a demonstrated fact, as anyone who has >>>> been following this thread can >>>> attest. We keep telling you "X" and you keep saying >>>> "not >>>> X", not because you have any >>> >>>I kept telling you that I wanted a way to prevent page >>>faults, and you kept providing memory mapped files as the >>>solution, knowing that memory mapped files do not prevent >>>page faults, and also knowing that VirtualLock() does >>>prevent pages faults. So who is acting stupidly in this >>>case? >> **** >> Oh. A new fantasy. Memory-mapped files do not prevent >> page faults, but that is not your >> question. > >> Your question was, could you create a situation in which >> page faults did not >> occur. > >and you kept providing memory mapped files as the solution, >and refrained from providing VirtualLock(). This is a matter >of public record. > >> You demonstrated, by conducting a proper experiment, that >> in a memory-rich system, >> you had zero page faults. You were not using VirtualLock. >> Why is it you think that >> memory-mapped files are any more subject to paging faults >> than a system which does not use >> VirtualLock? Oh, I missed that leap of logic. > >Maybe it was partly my fault, I did not explicitly use the >term [prevent page faults] until recently. > >> >> Furthermore, you did not justify why you must have zero >> page faults, other than some >> bizarre reasoning that said if you have page faults, you >> cannot meet your realtime window. > >It would take far longer than 100 ms to load 4 GB of data. >If the OS (in a memory rich environment) found any reason to >page any of the data out, then this same reason could >possibly page all of this data out. All this is moot now. > >> I don't recall that you measured this; if you did, then it >> is true (and the quantity of >> page faults was known); if you did not see any impact from >> a small number of page faults, >> then you cannot claim that zero is the upper limit of >> acceptable page faults. The prorper >> scientific approach to this is to create experiments which >> induce higher and higher number > >One has to begin with the right categories and reasoning >first, otherwise one must test every combination of >everything, including the combinations that don't make any >sense. I began with the premise that it would be absurd for >a memory rich system to have page faults, and testing >confirmed this. This separate testing turned out to be the >waste of time that I expected it to be. There was a benefit >derived from this testing that made it worthwhile. I now >know that my algorithm redesign results in a ten-fold gain >in speed. > >> of page faults until active interference with the realtime >> window is detected, then back >> off (remember the Calvin and Hobbs cartoon I cited?) and >> back off enough that you have >> "headroom" in case there are mechanisms that induce >> additional page faults. >> >> VirtualLock reduces the probability of page faults by >> locking the pages you specify into >> memory. Did you think that applies only to your data? >> You will have to VirtualLock your >> code, also. >> joe >> >>> > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 30 Mar 2010 13:45 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:gbb4r5d7i62ue6b3lkelnksollka78digm(a)4ax.com... > See below... > On Mon, 29 Mar 2010 23:27:42 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>message news:s5n2r5lj03ktt52logni11mhgsudskmkgn(a)4ax.com... >>> See below... >>> On Mon, 29 Mar 2010 16:35:54 -0500, "Peter Olcott" >>> <NoSpam(a)OCR4Screen.com> wrote: >>> >>> >>>> >>>>It is not that I am stupid it is that for some reason (I >>>>am >>>>thinking intentionally) you fail to get what I am >>>>saying. >>>>I needed something the PREVENTS page faults**, MMF does >>>>not >>>>do that, VirtualLock() does. >>> **** >>> I never promised that memory mapped files would give you >>> ZERO page faults; I only pointed >>> out that it can reduce the total number of page faults, >>> and distribute the cost of them >>> differently than your simplistic model that takes >>> several >>> thousand page faults to load the >>> data. And I said that in any real world experiment, it >>> is >>> essential to gather the data to >>> show the exact impact of the architecture. >>> **** >> >>The whole goal of the original architecture was to find a >>way to prevent pages faults to my several GB of data >>because >>it would have taken several minutes to load this data, >>thus >>it was to be kept loaded. All this time I am sure that you >>were aware of VirtualLock() but said nothing. > *** > I am reminded that I pointed out VirtualLock to you MONTHS > previously. > *** >> >>I kept saying that I need zero page faults and you kept >>push >>MMF as the solution. I kept saying that MMF are no good >>because they don't prevent pages faults and I need >>something >>that prevents pages faults, and you kept push MMF all the >>while knowing about VirtualLock(). > **** > No, I pointed out that MMF reduce page faults during > loading. Of course, the addresses > which are mapped are subject to VirtualLock. I also > pointed out that MMF reduce page > fault overheads to zero if the segment is shared with > other processes, which is a Good > Thing. YOU were the one that kept saying MMF are no good > because they don't prevent page > faults, and I kept saying that you were wrong in this > opinion. You are still wrong. You > were not paying any attention to my saying that you got to > AMORTIZE any page faults across > ALL processes, and these > page faults would only occur during loading into the first > process, (1) I never saw that you ever said anything like this. (2) Is it really true, even in the case where pages loaded exceeds physical RAM? > no different than what you are already experiencing. Oh > well, I'm sorry reality > doesn't conform to your fantasies, but I can tell you > repeatedly you are wrong. Your > insistence that you are right does not make you right. It > only proves you don't > understand what anyone is telling you. > joe > **** > > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Tom Serface on 30 Mar 2010 13:48
Yeah. I have never marked a thread to ignore in this forum, but this is the first... "BobF" <nothanks(a)no.spam> wrote in message news:#9ig$8C0KHA.3264(a)TK2MSFTNGP06.phx.gbl... > > This was amusing for a while, but I'm getting bored now ... > <yaaaawwwwwwwwwn ....> |