code_seg / alloc_text pragma - static library
in [Device Drivers]
|
Prev: WinUSB Error 121 - semaphore timeout
Next: STATUS_ACCESS_DENIED causes file deletion (minifilter driver)
From: Doron Holan [MSFT] on 5 Jan 2010 12:59 well, come and argue with the windows perf team then ;). every page counts and we can see measurable improvements as well NP pool usage. it is a facetious argument to say that other limits in the system will be hit before you reach NP pool limits, a piggy set of drivers can easily consume a ton of NP pool while being idle. also, it is not about exhausting the limits of NP pool, it is the effects that NP pool have on the rest of the memory manager in terms of how much flexibility it has to manage paged pool. you are also thinking of just the client. now take your thinking and apply it to N VMs running on one machine, each overcommitted in terms of the memory assigned to each. d -- This posting is provided "AS IS" with no warranties, and confers no rights. "Alexander Grigoriev" <alegr(a)earthlink.net> wrote in message news:#HFeAGbjKHA.2164(a)TK2MSFTNGP02.phx.gbl... > I don't buy that at all. You know that Vista/2008 doesn't have an explicit > limit on NP pool size anymore. In any case, if you hit NP pool limit, > you've perhaps already hit other bottlenecks, such as IO throughput, etc. > > And it doesn't matter if you hit it 20MB earlier or later. If it happens, > your system was already fragile. I don't believe, also that amount of > loaded pageable KM code in a x86 system is any more than 10 MB. Not worth > trouble. These paged code/paged stack bugs happen to be quite obscure and > hard to diagnose. > > "Doron Holan [MSFT]" <doron.holan(a)online.microsoft.com> wrote in message > news:OhwzGBXjKHA.4912(a)TK2MSFTNGP02.phx.gbl... >> it does not quite work that way. by consuming NP pool you take away the >> ability to use that memory for anything else. the friendlier you are >> with NP pool (e.g. the less you consume), the better the system can >> respond to memory pressure. a few megabytes make a huge difference when >> a machine is running a lot of apps or is under load >> >> d >> >> -- >> >> This posting is provided "AS IS" with no warranties, and confers no >> rights. >> >> >> "Alexander Grigoriev" <alegr(a)earthlink.net> wrote in message >> news:u#seTjMhKHA.2164(a)TK2MSFTNGP02.phx.gbl... >>> I've had my share of troubles because of paged code. The most recent was >>> paged code in win2008 acpi.sys which happened to be at DISPATCH_LEVEL, >>> when DriverVerifier marked all such pages non-present because a spinlock >>> was being acquired by another processor. Happened during DTM test for >>> hot processor plug simulation. >>> >>> 2GB of RAM costs 50$. 20MB costs $0.50 then. I'm not sure it's worth the >>> trouble. >>> >>> "Don Burn" <burn(a)stopspam.windrvr.com> wrote in message >>> news:eU1W2RLhKHA.1456(a)TK2MSFTNGP06.phx.gbl... >>>>I have to disagree. The 20MB figure is big even for todays systems, and >>>>if your driver can potentially be used in an embedded system or similar >>>>limited then forgetting PAGE sections is a bad idea. I know of a major >>>>vendor who lost a number of lucrative contracts thanks to this approach. >>>>In one case a client of mine acquired the sources, and by simple work on >>>>using PAGE sections produced an overall improvment in their embedded >>>>system by well over 50%. >>>> >>>> >>>> -- >>>> Don Burn (MVP, Windows DKD) >>>> Windows Filesystem and Driver Consulting >>>> Website: http://www.windrvr.com >>>> Blog: http://msmvps.com/blogs/WinDrvr >>>> >>>> >>>> "Alexander Grigoriev" <alegr(a)earthlink.net> wrote in message >>>> news:%23A2k4NLhKHA.2184(a)TK2MSFTNGP04.phx.gbl... >>>>> Using paged code is not worth trouble. Unless the code is over 20MB. >>>>> Save yourself from grief and forget about using PAGE sections. >>>>> >>>>> "David F." <df2705(a)community.nospam> wrote in message >>>>> news:059BB54E-A6FB-477C-A525-368ACAECA83C(a)microsoft.com... >>>>>> Hi, >>>>>> >>>>>> If I have a stoic library built that I want to link and say it's part >>>>>> of the "PAGE" code segment how do I do that? The library could be >>>>>> used in either case (non-paged or paged) when I know it's okay to use >>>>>> in PAGE segment I want to tell the build to make it paged code to not >>>>>> allocate non-paged pool space for the code. >>>>>> >>>>>> Also how exactly does the code_seg with class information work? Can >>>>>> I provide the segment and class name in the header file or in a .CPP >>>>>> module and does that make that class always go to that segment even >>>>>> if that class is compiled to its own static library too? >>>>>> >>>>>> TIA!! >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> __________ Information from ESET NOD32 Antivirus, version of virus >>>>> signature database 4715 (20091224) __________ >>>>> >>>>> The message was checked by ESET NOD32 Antivirus. >>>>> >>>>> http://www.eset.com >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> __________ Information from ESET NOD32 Antivirus, version of virus >>>> signature database 4715 (20091224) __________ >>>> >>>> The message was checked by ESET NOD32 Antivirus. >>>> >>>> http://www.eset.com >>>> >>>> >>>> >>>> >>> >>> > >
From: alberto on 5 Jan 2010 18:37 It depends on the application mix, no ? In my line of work, graphics frames per second is the one overriding performance figure, and I'm not shy of grabbing page frames if that helps me achieve my target spinning rate. If the processor runs a tad slower, so be it! My chip doesn't. :-) I can afford that luxury because my environment is quite specialized and my application suite can be expected to be by far the most important thing - if not the only one - running in that machine; I know that people won't be going to the web or doing email on the same server. Still, different environments may have different needs, and after all the NP pool is just a resource. Alberto. On Jan 5, 12:59 pm, "Doron Holan [MSFT]" <doron.ho...(a)online.microsoft.com> wrote: > well, come and argue with the windows perf team then ;). every page counts > and we can see measurable improvements as well NP pool usage. it is a > facetious argument to say that other limits in the system will be hit before > you reach NP pool limits, a piggy set of drivers can easily consume a ton of > NP pool while being idle. also, it is not about exhausting the limits of NP > pool, it is the effects that NP pool have on the rest of the memory manager > in terms of how much flexibility it has to manage paged pool. you are also > thinking of just the client. now take your thinking and apply it to N VMs > running on one machine, each overcommitted in terms of the memory assigned > to each. > > d > -- > > This posting is provided "AS IS" with no warranties, and confers no rights. > > "Alexander Grigoriev" <al...(a)earthlink.net> wrote in message > > news:#HFeAGbjKHA.2164(a)TK2MSFTNGP02.phx.gbl... > > > > > I don't buy that at all. You know that Vista/2008 doesn't have an explicit > > limit on NP pool size anymore. In any case, if you hit NP pool limit, > > you've perhaps already hit other bottlenecks, such as IO throughput, etc. > > > And it doesn't matter if you hit it 20MB earlier or later. If it happens, > > your system was already fragile. I don't believe, also that amount of > > loaded pageable KM code in a x86 system is any more than 10 MB. Not worth > > trouble. These paged code/paged stack bugs happen to be quite obscure and > > hard to diagnose. > > > "Doron Holan [MSFT]" <doron.ho...(a)online.microsoft.com> wrote in message > >news:OhwzGBXjKHA.4912(a)TK2MSFTNGP02.phx.gbl... > >> it does not quite work that way. by consuming NP pool you take away the > >> ability to use that memory for anything else. the friendlier you are > >> with NP pool (e.g. the less you consume), the better the system can > >> respond to memory pressure. a few megabytes make a huge difference when > >> a machine is running a lot of apps or is under load > > >> d > > >> -- > > >> This posting is provided "AS IS" with no warranties, and confers no > >> rights. > > >> "Alexander Grigoriev" <al...(a)earthlink.net> wrote in message > >>news:u#seTjMhKHA.2164(a)TK2MSFTNGP02.phx.gbl... > >>> I've had my share of troubles because of paged code. The most recent was > >>> paged code in win2008 acpi.sys which happened to be at DISPATCH_LEVEL, > >>> when DriverVerifier marked all such pages non-present because a spinlock > >>> was being acquired by another processor. Happened during DTM test for > >>> hot processor plug simulation. > > >>> 2GB of RAM costs 50$. 20MB costs $0.50 then. I'm not sure it's worth the > >>> trouble. > > >>> "Don Burn" <b...(a)stopspam.windrvr.com> wrote in message > >>>news:eU1W2RLhKHA.1456(a)TK2MSFTNGP06.phx.gbl... > >>>>I have to disagree. The 20MB figure is big even for todays systems, and > >>>>if your driver can potentially be used in an embedded system or similar > >>>>limited then forgetting PAGE sections is a bad idea. I know of a major > >>>>vendor who lost a number of lucrative contracts thanks to this approach. > >>>>In one case a client of mine acquired the sources, and by simple work on > >>>>using PAGE sections produced an overall improvment in their embedded > >>>>system by well over 50%. > > >>>> -- > >>>> Don Burn (MVP, Windows DKD) > >>>> Windows Filesystem and Driver Consulting > >>>> Website:http://www.windrvr.com > >>>> Blog:http://msmvps.com/blogs/WinDrvr > > >>>> "Alexander Grigoriev" <al...(a)earthlink.net> wrote in message > >>>>news:%23A2k4NLhKHA.2184(a)TK2MSFTNGP04.phx.gbl... > >>>>> Using paged code is not worth trouble. Unless the code is over 20MB.. > >>>>> Save yourself from grief and forget about using PAGE sections. > > >>>>> "David F." <df2...(a)community.nospam> wrote in message > >>>>>news:059BB54E-A6FB-477C-A525-368ACAECA83C(a)microsoft.com... > >>>>>> Hi, > > >>>>>> If I have a stoic library built that I want to link and say it's part > >>>>>> of the "PAGE" code segment how do I do that? The library could be > >>>>>> used in either case (non-paged or paged) when I know it's okay to use > >>>>>> in PAGE segment I want to tell the build to make it paged code to not > >>>>>> allocate non-paged pool space for the code. > > >>>>>> Also how exactly does the code_seg with class information work? Can > >>>>>> I provide the segment and class name in the header file or in a .CPP > >>>>>> module and does that make that class always go to that segment even > >>>>>> if that class is compiled to its own static library too? > > >>>>>> TIA!! > > >>>>> __________ Information from ESET NOD32 Antivirus, version of virus > >>>>> signature database 4715 (20091224) __________ > > >>>>> The message was checked by ESET NOD32 Antivirus. > > >>>>>http://www.eset.com > > >>>> __________ Information from ESET NOD32 Antivirus, version of virus > >>>> signature database 4715 (20091224) __________ > > >>>> The message was checked by ESET NOD32 Antivirus. > > >>>>http://www.eset.com- Hide quoted text - > > - Show quoted text -
From: Alexander Grigoriev on 5 Jan 2010 22:47 I would call this micro-optimization, rudiment from NT 3.1, which ran on 32 MB. I don't argue against paged pool, but paged kernel code should just go away. And paged kernel stacks, too, though I didn't see swapped-out stacks in W2008. Especially in kernel workitem threads. Yes, yes, kernel worker threads that serve IoQueueWorkitem are waiting with UserMode and their stacks could get swapped out, which I saw in Win2003. Too bad that our driver stack that occasionally needed a kernel workitem to restore a target connection, was in paging path. Bammm - deadlock. By the way, HyperV doesn't use overcommit for virtual machines. And in any case, 20 MB doesn't buy you anything. 512MB would, but only in low end configs. Better memory usage optimization would be to fix file caching. Which is supposedly, at last, (though I have not run stress tests on that), done in 2008/Vista+. Before it was so awful, that a large file copy would just steal all memory from under other processes and cause them to page-in like mad. Looks like someone messed up MRU/LRU really bad. "Doron Holan [MSFT]" <doron.holan(a)online.microsoft.com> wrote in message news:OJIlmDjjKHA.5520(a)TK2MSFTNGP06.phx.gbl... > well, come and argue with the windows perf team then ;). every page > counts and we can see measurable improvements as well NP pool usage. it is > a facetious argument to say that other limits in the system will be hit > before you reach NP pool limits, a piggy set of drivers can easily consume > a ton of NP pool while being idle. also, it is not about exhausting the > limits of NP pool, it is the effects that NP pool have on the rest of the > memory manager in terms of how much flexibility it has to manage paged > pool. you are also thinking of just the client. now take your thinking > and apply it to N VMs running on one machine, each overcommitted in terms > of the memory assigned to each. > > d > -- > > This posting is provided "AS IS" with no warranties, and confers no > rights. > > > "Alexander Grigoriev" <alegr(a)earthlink.net> wrote in message > news:#HFeAGbjKHA.2164(a)TK2MSFTNGP02.phx.gbl... >> I don't buy that at all. You know that Vista/2008 doesn't have an >> explicit limit on NP pool size anymore. In any case, if you hit NP pool >> limit, you've perhaps already hit other bottlenecks, such as IO >> throughput, etc. >> >> And it doesn't matter if you hit it 20MB earlier or later. If it happens, >> your system was already fragile. I don't believe, also that amount of >> loaded pageable KM code in a x86 system is any more than 10 MB. Not worth >> trouble. These paged code/paged stack bugs happen to be quite obscure and >> hard to diagnose. >> >> "Doron Holan [MSFT]" <doron.holan(a)online.microsoft.com> wrote in message >> news:OhwzGBXjKHA.4912(a)TK2MSFTNGP02.phx.gbl... >>> it does not quite work that way. by consuming NP pool you take away the >>> ability to use that memory for anything else. the friendlier you are >>> with NP pool (e.g. the less you consume), the better the system can >>> respond to memory pressure. a few megabytes make a huge difference when >>> a machine is running a lot of apps or is under load >>> >>> d >>> >>> -- >>> >>> This posting is provided "AS IS" with no warranties, and confers no >>> rights. >>> >>> >>> "Alexander Grigoriev" <alegr(a)earthlink.net> wrote in message >>> news:u#seTjMhKHA.2164(a)TK2MSFTNGP02.phx.gbl... >>>> I've had my share of troubles because of paged code. The most recent >>>> was paged code in win2008 acpi.sys which happened to be at >>>> DISPATCH_LEVEL, when DriverVerifier marked all such pages non-present >>>> because a spinlock was being acquired by another processor. Happened >>>> during DTM test for hot processor plug simulation. >>>> >>>> 2GB of RAM costs 50$. 20MB costs $0.50 then. I'm not sure it's worth >>>> the trouble. >>>> >>>> "Don Burn" <burn(a)stopspam.windrvr.com> wrote in message >>>> news:eU1W2RLhKHA.1456(a)TK2MSFTNGP06.phx.gbl... >>>>>I have to disagree. The 20MB figure is big even for todays systems, >>>>>and if your driver can potentially be used in an embedded system or >>>>>similar limited then forgetting PAGE sections is a bad idea. I know of >>>>>a major vendor who lost a number of lucrative contracts thanks to this >>>>>approach. In one case a client of mine acquired the sources, and by >>>>>simple work on using PAGE sections produced an overall improvment in >>>>>their embedded system by well over 50%. >>>>> >>>>> >>>>> -- >>>>> Don Burn (MVP, Windows DKD) >>>>> Windows Filesystem and Driver Consulting >>>>> Website: http://www.windrvr.com >>>>> Blog: http://msmvps.com/blogs/WinDrvr >>>>> >>>>> >>>>> "Alexander Grigoriev" <alegr(a)earthlink.net> wrote in message >>>>> news:%23A2k4NLhKHA.2184(a)TK2MSFTNGP04.phx.gbl... >>>>>> Using paged code is not worth trouble. Unless the code is over 20MB. >>>>>> Save yourself from grief and forget about using PAGE sections. >>>>>> >>>>>> "David F." <df2705(a)community.nospam> wrote in message >>>>>> news:059BB54E-A6FB-477C-A525-368ACAECA83C(a)microsoft.com... >>>>>>> Hi, >>>>>>> >>>>>>> If I have a stoic library built that I want to link and say it's >>>>>>> part of the "PAGE" code segment how do I do that? The library could >>>>>>> be used in either case (non-paged or paged) when I know it's okay to >>>>>>> use in PAGE segment I want to tell the build to make it paged code >>>>>>> to not allocate non-paged pool space for the code. >>>>>>> >>>>>>> Also how exactly does the code_seg with class information work? Can >>>>>>> I provide the segment and class name in the header file or in a .CPP >>>>>>> module and does that make that class always go to that segment even >>>>>>> if that class is compiled to its own static library too? >>>>>>> >>>>>>> TIA!! >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> __________ Information from ESET NOD32 Antivirus, version of virus >>>>>> signature database 4715 (20091224) __________ >>>>>> >>>>>> The message was checked by ESET NOD32 Antivirus. >>>>>> >>>>>> http://www.eset.com >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> __________ Information from ESET NOD32 Antivirus, version of virus >>>>> signature database 4715 (20091224) __________ >>>>> >>>>> The message was checked by ESET NOD32 Antivirus. >>>>> >>>>> http://www.eset.com >>>>> >>>>> >>>>> >>>>> >>>> >>>> >> >>
From: David F. on 5 Jan 2010 23:25 "Kalle Olavi Niemitalo" <kon(a)iki.fi> wrote in message news:87y6ks7wxy.fsf(a)Astalo.kon.iki.fi... > "David F." <df2705(a)community.nospam> writes: > >> If I have a stoic library built that I want to link and say it's part >> of the "PAGE" code segment how do I do that? The library could be >> used in either case (non-paged or paged) when I know it's okay to use >> in PAGE segment I want to tell the build to make it paged code to not >> allocate non-paged pool space for the code. > > I suppose you could specify a custom segment name > (e.g. yourlib.text) when building the static library, then use > LINK /MERGE to rename it to PAGETEXT or .text in each driver that > uses the static library. Dunno how to best do that in BUILD. > I'm delaying trying to figure this out - I could probably move 40 to 50K out of NPP by doing it.
From: Pavel Lebedinsky [MSFT] on 7 Jan 2010 01:43
> I would call this micro-optimization, rudiment from NT 3.1, which ran > on 32 MB. I don't argue against paged pool, but paged kernel code > should just go away. And paged kernel stacks, too If you ask the Windows perf team, they will probably say that saving 10 MB of nonpaged memory is a major achievement, not a micro-optimization. During win7 the perf team filed hundreds of bugs against all kinds of components across Windows (not just drivers), asking them to reduce nonpaged footprint, copy-on-write pages in user space etc. Sometimes the savings were just a few pages per component, but together they are a big part of why win7 is the only version of Windows (except maybe NT 3.5/3.51) to have a smaller memory footprint than the previous one. > Better memory usage optimization would be to fix file caching. That's like saying that you shouldn't try to make a plane lighter until you can figure out how to make the engines more powerful. Why not try to do both at the same time? -- Pavel Lebedinsky/Windows Fundamentals Test This posting is provided "AS IS" with no warranties, and confers no rights. |