Can extra processing threads help in this case?
in [MFC]
|
From: Peter Olcott on 27 Mar 2010 23:55 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:e9jtq5pav683bb9qgcfrvmkg439nribmno(a)4ax.com... > Seee below... > > On Thu, 25 Mar 2010 17:27:37 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: >> >>So I have been convinced for several days now that >>multi-threads could be a good idea. I still see no benefit >>of adding memory mapped files to this mix. You have >>explicitly failed to point out any incremental benefit >>that >>I get from memory mapped files over multiple threads >>sharing >>the same memory in the case where data remains resident in >>memory as my testing has shown that it does. (I will also >>test this on Linux). > *** > I believe I pointed out that it is likely to result in > FEWER page faults, since parts of As I have said over a dozen times; GIVEN the immutable premise that there are no page faults, then within the explicit context of this immutable premise ONLY, how can memory mapped files help? AND apparently the answer is they can not help in this case. If you don't answer this time I will give up on you as playing head games with me. > the DFA you don't access don't actually have to be brought > in. And the initialization is > going to be faster than several minutes, because there's > nothing to "read in" and no > storage allocation is going to be required! > > Is this not screamingly obvious? Just from basic > principles? > **** It is screamingly utterly pointless when as I have said the initialization is a three minutes every three months thing. It is the 24/7/365 that needs to be optimized, NOT the three minutes every three months. >> >>What you do instead is continue to insist that data does >>not >>remain resident in memory. I will test this in Linux very >>soon. > **** > Interesting that in a later post you reveal that it > DOESN'T remain resident in memory, I don't remember ever saying this. Substantial testing has confirmed that ALL the data from my process does remain in RAM for both Windows and Linux. AND if nothing else, in Linux I can issue a simple command to forbid the OS from paging out my data. I can apply this same command to every aspect of the process including stack space. > proving that I was right; I stated that you have no > guarantees. Or are those numbers > about how linux slows down pure fiction? I think they > prove what I said, and that it is > true in linux (and it looks like a Working Set issue, > which I pointed out somewhere in my > first response to your post). > joe > **** >> >>>> >>> Joseph M. Newcomer [MVP] >>> email: newcomer(a)flounder.com >>> Web: http://www.flounder.com >>> MVP Tips: http://www.flounder.com/mvp_tips.htm >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 27 Mar 2010 23:57 The models for the assessment are: A Webserver at a hosting site A Webserver at your site The parameters are: Firewalls implemented using conventional operating systems (such as Unix/linux) Firewalls implemented using network operating systems (e.g. Cisco) Firewalls implemented in proprietary dedicated hardware (e.g., SonicWall, LinkSys) Firewalls run on the hosting operating system (Black Ice, Microsoft Firewall) what the firewalls detect and protect you from Harware protection against common exploits (e.g., no-execute stack segments) Software protection against common exploits Note that a configuration with a oouple layers of front-end firewalls is going to be more secure than a configuration without these, no matter where it is located. If it is at your site, YOU are responsible for reading the event logs from all these firewalls, every day, to see what attcks are being mounted against you. If you host at an ISP, the ISP is doing this for you. And they armortize the cost of this over all their customers, meaning whatever they are charging you, it is a whopping lot less than it would cost you to do it yourself. Physical security at the hosting site is not your problem. They supposedly give you guarantees about their physical security. I've had a tour of my ISP's hosting site. There's a lot of physical security, including several levels of code-locked doors between the street and my server. And to physically attack my server, you not only have to pass through all the doors, you need to figure out which of the several hundred machines on that site is my server, and whatever you plan to do to it will take more time than the response time of the police to a report of a security breach. Or have you rededfined "physical security" to mean something other than what the rest of us recognize? (Note that you are talking to someone who was responsible for the physical security of our computer room at our company; I specified and supervised the installation of the system, as well as the HVAC, power conditioning, and fire supression systems, and I was the one who specified, ordered, and supervised the installation of a secure development facility in our site that had to pass rigid third-party security standards [specifically, those of IBM who was giving us a top-secret development contract for an unannounced product], so I know something about physical security) joe On Fri, 26 Mar 2010 15:38:38 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:hr2qq5tb9p5rlkk8vdetjlkusv3fk9n7cd(a)4ax.com... >> See below... >> On Fri, 26 Mar 2010 13:31:36 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>> >>>"Tom Serface" <tom(a)camaswood.com> wrote in message >>>news:OxTzNBRzKHA.264(a)TK2MSFTNGP05.phx.gbl... >>>> We just use IIS for our intranet web applications. We >>>> can >>>> install it on our servers as they are manufactured and >>>> it's really easy to set up. >>>> >>>> There are restrictions to doing web applications of >>>> course, but you can do a lot of coding server-side if >>>> you >>>> know the server will have access and permission to >>>> update >>>> areas on the intranet. >>>> >>>> Tom >>> >>>My trick is to make this web server installed a the client >>>site have as close to as possible impenetrable security >>>because I am adding trade secret to my intellectual >>>property >>>protection. I think that I have this essentially figured >>>out. >>> >>>The one part that I do not have completely figured out is >>>exactly how I can make it close to impossible for anyone >>>to >>>physically open the machine without detection. I have >>>figured out ways to make this mostly (but not completely) >>>moot. I have also figured out ways to make case access >>>security extremely difficult to circumvent. >> *** >> The Standard Model of physical security goes like this: >> >> It is buried twelve feet deep in a 3-foot-thick concrete >> vault in the middle of an active >> artillery range in the middle of a secure military base. >> >> For dead-certain security, there are no wires leading into >> it, and it is turned off, and >> there is no nearby power supply. >> >> Another model surrounds it by vicious pit bulls, but >> unless you have done background >> checks on the pit bulls, you cannot really be sure this is >> safe. One of them might be in >> the pay of your competitor. A pound of prime rib can buy >> a lot of loyalty in a pit bull. >> >> The simplest model is that you, personally, guard the >> machine, armed with some illegal >> automatic weapon. That's 24/7, so don't plan on sleeping, >> eating, or dealing with bodily >> functions. >> >> All other methods are not considered trustworthy in the >> absolute sense. >> >> [A friend once had a tour of the NSA. In one room, inside >> a Faraday cage, there was a >> battery power supply running a PDP-11 with a hard drive >> attached. And a Marine guard, >> armed with a .45 pistol. His orders: should anyone >> attempt to enter the room by force, he >> was to draw his weapon, and shoot the disk. Now THAT'S >> physical security!] >> >> Physical access is not your problem; this is the problem >> of your ISP, and their physical >> security system. This is what you are paying for. Not >> just a CPU, a bit of wire, and a >> UPS system! >> joe > >Two different things are being assessed: >(1) Webserver on the internet >(2) A webserver rented to the client at the client's >location. > >I spoke to Phil Zimmerman about his stuff once, this seems >like it could form one essential element of a sufficient >basis. > >> >>> >>>> >>>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote in message >>>> news:toydnSprNqitejHWnZ2dnUVZ_sidnZ2d(a)giganews.com... >>>>> >>>>> "Tom Serface" <tom(a)camaswood.com> wrote in message >>>>> news:uiYs3GQzKHA.5040(a)TK2MSFTNGP02.phx.gbl... >>>>>> Hi Peter, >>>>>> >>>>>> I don't think the proximity is the only gating factor. >>>>>> Bandwidth for connected servers, the path of the data, >>>>>> etc. all matter. If your users are directly >>>>>> connecting >>>>>> to your server it may make a difference. It must make >>>>>> some sort of difference because every professional >>>>>> download site I know of has multiple "mirror" sites so >>>>>> that you can select one closest to you. If nothing >>>>>> else, distributing it may make some people use other >>>>>> servers and spread the cycles needed to read and send >>>>>> the data off a little less arduous. >>>>>> >>>>>> Tom >>>>> >>>>> Yes, and it also just occured to me that in special >>>>> cases >>>>> I may even be able to provide an intranet webserver. >>>>> >>> >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 28 Mar 2010 00:22 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:vtjtq5152v2tvhdut0oabr2ssn9bd3rfk3(a)4ax.com... > The models for the assessment are: > > A Webserver at a hosting site > A Webserver at your site No that's not it. The models for the assessment are: 1) A Webserver at a hosting site 2) A Webserver rented to the client at the client's site. Here is how you tell Windows not to ever swap this memory out. http://msdn.microsoft.com/en-us/library/aa366895(VS.85).aspx Because you didn't get the spec right, your advice could not apply to the key part that is missing: Making a machine at the client site such that it is close to impossible for the client to get direct access to the executable files. The big missing piece here is how to make the physical security of the computer's physical encasement as impenetrable as possible. Even most of this is figured out, the key is simply detecting penetration in a very reliable way. This last sentence is the biggest missing piece. The only way that I figured out to make this work is much more clumsy than I would like. So if you have any ideas on how to make physical access to the computer internals infeasibly undetectable for anyone lacking a $100,000 budget, I would like to know. I will also carefully study your other advice below: > > The parameters are: > > Firewalls implemented using conventional operating systems > (such as Unix/linux) > Firewalls implemented using network operating systems > (e.g. Cisco) > Firewalls implemented in proprietary dedicated hardware > (e.g., SonicWall, LinkSys) > Firewalls run on the hosting operating system (Black Ice, > Microsoft Firewall) > > what the firewalls detect and protect you from > > Harware protection against common exploits (e.g., > no-execute stack segments) > Software protection against common exploits > > Note that a configuration with a oouple layers of > front-end firewalls is going to be more > secure than a configuration without these, no matter where > it is located. If it is at > your site, YOU are responsible for reading the event logs > from all these firewalls, every > day, to see what attcks are being mounted against you. If > you host at an ISP, the ISP is > doing this for you. And they armortize the cost of this > over all their customers, meaning > whatever they are charging you, it is a whopping lot less > than it would cost you to do it > yourself. > > Physical security at the hosting site is not your problem. > They supposedly give you > guarantees about their physical security. I've had a tour > of my ISP's hosting site. > There's a lot of physical security, including several > levels of code-locked doors between > the street and my server. And to physically attack my > server, you not only have to pass > through all the doors, you need to figure out which of the > several hundred machines on > that site is my server, and whatever you plan to do to it > will take more time than the > response time of the police to a report of a security > breach. Or have you rededfined > "physical security" to mean something other than what the > rest of us recognize? (Note that > you are talking to someone who was responsible for the > physical security of our computer > room at our company; I specified and supervised the > installation of the system, as well as > the HVAC, power conditioning, and fire supression systems, > and I was the one who > specified, ordered, and supervised the installation of a > secure development facility in > our site that had to pass rigid third-party security > standards [specifically, those of IBM > who was giving us a top-secret development contract for an > unannounced product], so I know > something about physical security) > joe Yes for my web server I will simply be using conventional means. I will trust my provider for the physical security. > > On Fri, 26 Mar 2010 15:38:38 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>message news:hr2qq5tb9p5rlkk8vdetjlkusv3fk9n7cd(a)4ax.com... >>> See below... >>> On Fri, 26 Mar 2010 13:31:36 -0500, "Peter Olcott" >>> <NoSpam(a)OCR4Screen.com> wrote: >>> >>>> >>>>"Tom Serface" <tom(a)camaswood.com> wrote in message >>>>news:OxTzNBRzKHA.264(a)TK2MSFTNGP05.phx.gbl... >>>>> We just use IIS for our intranet web applications. We >>>>> can >>>>> install it on our servers as they are manufactured and >>>>> it's really easy to set up. >>>>> >>>>> There are restrictions to doing web applications of >>>>> course, but you can do a lot of coding server-side if >>>>> you >>>>> know the server will have access and permission to >>>>> update >>>>> areas on the intranet. >>>>> >>>>> Tom >>>> >>>>My trick is to make this web server installed a the >>>>client >>>>site have as close to as possible impenetrable security >>>>because I am adding trade secret to my intellectual >>>>property >>>>protection. I think that I have this essentially figured >>>>out. >>>> >>>>The one part that I do not have completely figured out >>>>is >>>>exactly how I can make it close to impossible for anyone >>>>to >>>>physically open the machine without detection. I have >>>>figured out ways to make this mostly (but not >>>>completely) >>>>moot. I have also figured out ways to make case access >>>>security extremely difficult to circumvent. >>> *** >>> The Standard Model of physical security goes like this: >>> >>> It is buried twelve feet deep in a 3-foot-thick concrete >>> vault in the middle of an active >>> artillery range in the middle of a secure military base. >>> >>> For dead-certain security, there are no wires leading >>> into >>> it, and it is turned off, and >>> there is no nearby power supply. >>> >>> Another model surrounds it by vicious pit bulls, but >>> unless you have done background >>> checks on the pit bulls, you cannot really be sure this >>> is >>> safe. One of them might be in >>> the pay of your competitor. A pound of prime rib can >>> buy >>> a lot of loyalty in a pit bull. >>> >>> The simplest model is that you, personally, guard the >>> machine, armed with some illegal >>> automatic weapon. That's 24/7, so don't plan on >>> sleeping, >>> eating, or dealing with bodily >>> functions. >>> >>> All other methods are not considered trustworthy in the >>> absolute sense. >>> >>> [A friend once had a tour of the NSA. In one room, >>> inside >>> a Faraday cage, there was a >>> battery power supply running a PDP-11 with a hard drive >>> attached. And a Marine guard, >>> armed with a .45 pistol. His orders: should anyone >>> attempt to enter the room by force, he >>> was to draw his weapon, and shoot the disk. Now THAT'S >>> physical security!] >>> >>> Physical access is not your problem; this is the problem >>> of your ISP, and their physical >>> security system. This is what you are paying for. Not >>> just a CPU, a bit of wire, and a >>> UPS system! >>> joe >> >>Two different things are being assessed: >>(1) Webserver on the internet >>(2) A webserver rented to the client at the client's >>location. >> >>I spoke to Phil Zimmerman about his stuff once, this seems >>like it could form one essential element of a sufficient >>basis. >> >>> >>>> >>>>> >>>>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote in >>>>> message >>>>> news:toydnSprNqitejHWnZ2dnUVZ_sidnZ2d(a)giganews.com... >>>>>> >>>>>> "Tom Serface" <tom(a)camaswood.com> wrote in message >>>>>> news:uiYs3GQzKHA.5040(a)TK2MSFTNGP02.phx.gbl... >>>>>>> Hi Peter, >>>>>>> >>>>>>> I don't think the proximity is the only gating >>>>>>> factor. >>>>>>> Bandwidth for connected servers, the path of the >>>>>>> data, >>>>>>> etc. all matter. If your users are directly >>>>>>> connecting >>>>>>> to your server it may make a difference. It must >>>>>>> make >>>>>>> some sort of difference because every professional >>>>>>> download site I know of has multiple "mirror" sites >>>>>>> so >>>>>>> that you can select one closest to you. If nothing >>>>>>> else, distributing it may make some people use other >>>>>>> servers and spread the cycles needed to read and >>>>>>> send >>>>>>> the data off a little less arduous. >>>>>>> >>>>>>> Tom >>>>>> >>>>>> Yes, and it also just occured to me that in special >>>>>> cases >>>>>> I may even be able to provide an intranet webserver. >>>>>> >>>> >>> Joseph M. Newcomer [MVP] >>> email: newcomer(a)flounder.com >>> Web: http://www.flounder.com >>> MVP Tips: http://www.flounder.com/mvp_tips.htm >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Hector Santos on 28 Mar 2010 02:29 Peter Olcott wrote: >> Almost every introduction-to-operating-systems book gives >> only general principles and >> simplified-for-students overviews. Real systems are never >> as simple as the textbooks > > But then (I guess I will ask explicitly) How much has this > stuff changed in 25 years? Nothing has change. Everything is the exactly the same, exactly how it was 25 years ago. Finish reading the 2nd half, and you will catch up with modern technology. > The only thing that I really need to know about VM right > now, is how to minimize its impact on my performance. I told you already. When you scale up, don't be a hog! Scale out if you want to be a hog. Its like eating pie. With a single pie, each hog with not share well with others. But if you get a whole pie for each hog, they will be happy. But if you stay with a single pie, then you better tame the hog's eating habits. Now isn't that so simple? If that doesn't make sense or don't believe it, and need more intellectual challenging reading, then study concepts like Pareto Optimality which in simple layman terms is: One can not gain without hurting others. When you are pareto optimal, you can apply it to have reached a maximum efficiency of shared resources, a steady state of smooth operation. No entropy, No Chaos. No pressure points. Perfect delegation of work. Its almost like socialism. But if one demands more, it can not do so without taking away from others. You have entropy, chaos, riots, revolutions!. You know Pareto also suggest you can get at least 80% (actually more) of the information you need on Google.NET. > #include <sys/mman.h> > int mlock(const void *addr, size_t len); > int munlock(const void *addr, size_t len); > int mlockall(int flags); > int munlockall(void); > > Is there anything like this in Windows? I told you already. AWE (Address Windowing Extension). But you need to first upgrade from the Windows 7 Home Edition. -- HLS
From: Pete Delgado on 28 Mar 2010 03:23
"Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote in message news:GpGdnebGT-QCRDPWnZ2dnUVZ_h2dnZ2d(a)giganews.com... > Here is how you tell Windows not to ever swap this memory out. > http://msdn.microsoft.com/en-us/library/aa366895(VS.85).aspx Weren't you the one who insisted that your application wasn't using *virtual memory*? So why would you use this API for your program that doesn't use virtual memory? <grin> PS: I don't suppose you read the "remarks" section that explained about the maximum number of pages a process can lock? -Pete |