Measure amount of leaked memory?
in [Windows Servers]
|
Prev: Recycle Bin Not Empting
Next: svchost.exe
From: henrikm on 6 May 2010 07:28 Hi. Some of our servers behave rather strangely and I suspect that we have programs that crash/leak memory. Is there any way to measure the total amount of RAM lost due to poorly designed programs? Currently I'm not looking for a way to check a specific process - that I can do with Windows Performace Monitor. The only way I can come up with is checking the amount of RAM (not swapped out memory) that windows says is allocated and compare it to the sum of all the processes allocated RAM. I've automated it with VBS/WMI but a manual check can be done with "Windows Task Manager". Under the tab "Performance" and the group "Physical Memory (K)", subtract "Available" from "Total" and we should have what Windows thinks is allocated RAM, right? Then switch to the tab "Processes", make sure "Show processes from all users" is checked and summarize every value in the column "Mem Usage". Now, shouldn't those two numbers match each other rather well? They don't! And on some troublesome servers the difference between "what windows thinks is allocated" and "the sum of all processes Mem Usage" is 5 gigs of RAM... Could someone please shed some light on this and find a flaw in my reasoning? /Henrik
From: dennis on 6 May 2010 11:03 On 06-05-2010 13:28, henrikm wrote: > Could someone please shed some light on this and find a flaw in my > reasoning? The only way is to know how much each process usually use. If it uses way more than it should be using then it might be leaking. Once a leaking process is terminated, all memory becomes available for others processes.
From: henrikm on 6 May 2010 15:38 > Once a leaking process is terminated, all memory becomes available for > others processes. I thought that was only true when you had some kind of garbage collection. If you allocate (and use) memory but loose your pointer and do not free the memory, does Windows really reclaim that memory upon termination without garbage collection? But it's been seven long years since I worked as a C/C++ developer, so I'm probably wrong... ;-) But lets not get sidetracked. I still wonder if it is possible to compare "what windows thinks is allocated" and "the sum of all processes Mem Usage". (see my first post) I feel like an old stupid man, not being able to understand why they differ, and in some cases differ a _lot_. That is the main question I need an answer to.
From: John John - MVP on 6 May 2010 15:47 henrikm wrote: >> Once a leaking process is terminated, all memory becomes available for >> others processes. > > I thought that was only true when you had some kind of garbage > collection. > If you allocate (and use) memory but loose your pointer and do not > free the memory, does Windows really reclaim that memory upon > termination without garbage collection? > But it's been seven long years since I worked as a C/C++ developer, so > I'm probably wrong... ;-) > > But lets not get sidetracked. > I still wonder if it is possible to compare "what windows thinks is > allocated" and "the sum of all processes Mem Usage". (see my first > post) > I feel like an old stupid man, not being able to understand why they > differ, and in some cases differ a _lot_. > That is the main question I need an answer to. It is difficult, if not impossible, to account for exact memory usage on NT operating systems but by using Perfmon and select performance counters you can get a fairly close accounting of memory usage. To obtain the physical memory usage first determine memory usage by the Operating System and Device Drivers by adding the following Memory Object counters: + Pool Nonpaged Bytes + Pool Paged Resident Bytes + System Cache Resident Bytes + System Code Resident Bytes + System Driver Resident Bytes ------------------------------ = Total Bytes used by the Operating System and Device Drivers Now that we know the memory usage by the Operating System and Device Drivers we can use the Memory Available Bytes counter and simple arithmetic to obtain the correct size of the Total Process Working Set size: + Total RAM installed in the computer - Available Bytes - Total Bytes used by the Operating System and Device Drivers (above) ------------------------------------- = Total Process Working Set Bytes** The Total Process Working Set Bytes can also be obtained from Perfmon's Process(_Total) Working Set Bytes counter, but the value reported by the counter will differ from the calculated value. Running processes share memory pages, these shared pages hold shared DLLs, DLLs that can be used by many different processes. Although the shared DLLs are only loaded into memory once, the Total Process Working Set Bytes counter counts every "usage" instance of shared DLLs. If a shared DLL is used by four different applications the Total Process Working Set Bytes will count the memory usage by the DLL four times, the DLL will be counted in each of the individual Process Working Sets thus inflating the actual value of the shared memory pages. The calculation above returns a more accurate value for the Total Process Working Set Bytes. To obtain RAM usage we simply sum up the two totals above: + Total Bytes used by the Operating System and Device Drivers + Total Process Working Set Bytes --------------------------------------- = Total RAM Usage In short, a quick but less accurate RAM usage figure can be obtained by adding up the following four Perfmon counters: + Process(_Total) Working Set Bytes + Cache Bytes (Total System Working Set)* + Pool Nonpaged Bytes --------------------------------------- = Total RAM Usage + Available Bytes --------------------------------------- = Total RAM installed in the Computer ======================================= As explained earlier, because of shared pages the total may differ to the actual RAM installed. * The Cache Bytes counter is the Total System Working Set, the Cache Bytes counter is an aggregate of the following Memory Counters: + Pool Paged Resident Bytes + System Cache Resident Bytes + System Code Resident Bytes + System Driver Resident Bytes ---------------------------- = Cache Bytes or System Working Set Bytes **The above calculations will yield a close estimate of memory usage on NT operating systems, bear in mind that certain Operating System components have no associated Memory Object Counters and obtaining values for these items is difficult, in the above examples memory usage by these components is accounted by default in the Total Process Working Set Bytes calculations. John
From: henrikm on 6 May 2010 16:17
> It is difficult, if not impossible, to account for exact memory usage on > NT operating systems Thank you. Suddenly I don't feel half as daft as before. :-) > but by using Perfmon and select performance > counters you can get a fairly close accounting of memory usage. And thanks once again for an exhaustive answer. Here in Sweden it's time to get som sleep but tomorrow (with more caffeine in my bloodstream) I'm gonna read it more carefully. Henrik |