From: Vera Noest [MVP] on
"Jim" <nobody(a)nospam.edu> wrote on 03 jan 2008 in
microsoft.public.windows.terminal_services:

>
> "Vera Noest [MVP]" <vera.noest(a)remove-this.hem.utfors.se> wrote
> in message
> news:Xns9A17DB1B5989Everanoesthemutforsse(a)207.46.248.16...
>> You have probably seen KB 142676, but for Windows 2003 you'll
>> want this one:
>>
>> 126962 - "Out of Memory" error message appears when you have a
>> large number of programs running
>> http://support.microsoft.com/?kbid=126962
>>
>> Also check JSI Tip nr 8057:
>>
>> Some applications may not run correctly in a Windows Server
>> 2003 Terminal Services environment
>> http://www.jsifaq.com/SF/Tips/Tip.aspx?id=8057
>>
>> _________________________________________________________
>> Vera Noest
>> MCSE, CCEA, Microsoft MVP - Terminal Server
>> TS troubleshooting: http://ts.veranoest.net
>> ___ please respond in newsgroup, NOT by private email ___
>>
>
> What values do you recommend?

I've no idea, depends on the current / default value. I'd probably
increment by small amounts.

> It would help if I could get dheapmon to run, otherwise I think
> I'm shooting in the dark. I can't find the proper symbol table
> in order to get it to install. Any ideas here?

Did you download the symbol tables from Microsoft symbol server, as
described in the download page for desktop Heap Monitor?
http://www.microsoft.com/whdc/DevTools/Debugging/symbolpkg.mspx

> Regards,
> Jim

_________________________________________________________
Vera Noest
MCSE, CCEA, Microsoft MVP - Terminal Server
TS troubleshooting: http://ts.veranoest.net
___ please respond in newsgroup, NOT by private email ___
From: Jim on

"Vera Noest [MVP]" <vera.noest(a)remove-this.hem.utfors.se> wrote in message
news:Xns9A1AE448DD77Everanoesthemutforsse(a)207.46.248.16...
> "Jim" <nobody(a)nospam.edu> wrote on 03 jan 2008 in
> microsoft.public.windows.terminal_services:
>
>>
>> "Vera Noest [MVP]" <vera.noest(a)remove-this.hem.utfors.se> wrote
>> in message
>> news:Xns9A17DB1B5989Everanoesthemutforsse(a)207.46.248.16...
>>> You have probably seen KB 142676, but for Windows 2003 you'll
>>> want this one:
>>>
>>> 126962 - "Out of Memory" error message appears when you have a
>>> large number of programs running
>>> http://support.microsoft.com/?kbid=126962
>>>
>>> Also check JSI Tip nr 8057:
>>>
>>> Some applications may not run correctly in a Windows Server
>>> 2003 Terminal Services environment
>>> http://www.jsifaq.com/SF/Tips/Tip.aspx?id=8057
>>>
>>> _________________________________________________________
>>> Vera Noest
>>> MCSE, CCEA, Microsoft MVP - Terminal Server
>>> TS troubleshooting: http://ts.veranoest.net
>>> ___ please respond in newsgroup, NOT by private email ___
>>>
>>
>> What values do you recommend?
>
> I've no idea, depends on the current / default value. I'd probably
> increment by small amounts.
>
>> It would help if I could get dheapmon to run, otherwise I think
>> I'm shooting in the dark. I can't find the proper symbol table
>> in order to get it to install. Any ideas here?
>
> Did you download the symbol tables from Microsoft symbol server, as
> described in the download page for desktop Heap Monitor?
> http://www.microsoft.com/whdc/DevTools/Debugging/symbolpkg.mspx

Yes, I think so. I downloaded the windows 2003 sp2 package (105 mb) and
extracted them into the proper folder (kktools). Dheapinst.exe fails with
a message that the symbol table is the wrong version.

This server has intel processors, specifically, it has 2 dual cores, giving
4 logical processors. I did select the x86 folder but I don't know for
certain if this was the correct choice. This may be unrelated (or not), but
system information hangs trying to do a system summary. Therefore, I cannot
assume that this is a 32bit system.

Regards,
Jim


From: TP on
Hi Jim,

I reviewed your registry and boot.ini settings and there
is nothing abnormal that would cause the error you are
seeing. Please gather more information so that hopefully
we can track down the cause. To that end I will ask
you more questions and give suggestions.

Dheapmon may help here so perhaps you should start
by getting that working. It is normally not too difficult.

1. Logon to the console (we will start by looking at desktop
heap usage for session 0)
2. Download and install dheapmon 8.1
3. Open up a command prompt and change to the
dheapmon folder that is applicable, usually c:\kktools\dheapmon8.1\x86
4. Install dheapmon and specify to download symbols from
the ms symbol server with this command:

dheapinst.exe -y srv*http://msdl.microsoft.com/download/symbols

You should get a success message after it is finished. If not,
make sure you have the command correct and try again. I
have had times when the download doesn't work and it will
not give you any message, on second attempt it works.

This document may help you if you continue to have issues:

Error message that is similar to the following when you try to
install the Desktop Heap Monitor tool: "dheapinst - Win32k.sys
symbol load error, correct symbol required"

http://support.microsoft.com/kb/914216

5. Load the dheapmon driver with this command:

dheapmon -l

6. Run a summary and detailed list to txt files with these commands:

dheapmon>dheapsum_0.txt
dheapmon -v>dheapverb_0.txt

7. Unload the dheapmon driver with this command:

dheapmon -u

8. Logon to a session and repeat steps 5-7, only this time name
the two files something like dheapsum_rdp.txt and dheapverb_rdp.txt

You can post the contents of the above files if you like and I will
comment. Please examine them and remove any information you
don't want the world to know about.

It is possible that you need to tweak your settings for your
specific running applications/situation but I think it best to
try and track down the cause first.

How many sessions normally logged on to this server prior to
installing SP2?

If you logon to the server but do *not* open any programs are
you able to get more than two sessions on without error?

You mentioned before that WMI is crashing, this might just be
a symptom of heap exhaustion or it may be important. Any
other errors in the system and application logs that may give
a clue as to the cause of this or the problems in general?

Are you connecting with very high resolution and high color depth?

Were there any other software changes?

What software do you have running on this machine? Think about
also things that are running as services, perhaps a custom application,
and/or security/antivirus/etc., special backup agents, server mirroring
agents, etc.

Open up Task Manager and enable some of the additional columns,
specifically User Objects and GDI Objects, VM Size, Paged Pool.
Check "Show processes from all users"

Anything stand out with high user objects, etc?

In Task Manager--Performance tab, what is your total physical
memory, and under kernel memory what do you normally see for
paged?

Thanks.

-TP

Jim wrote:
> Sorry about the delay, couldn't get access to the machine over New
> Years. Windows Registry Editor Version 5.00
>
> [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session
> Manager\Memory Management] "ClearPageFileAtShutdown"=dword:00000000
>
> "DisablePagingExecutive"=dword:00000000
>
> "LargeSystemCache"=dword:00000001
>
> "NonPagedPoolQuota"=dword:00000000
>
> "NonPagedPoolSize"=dword:00000000
>
> "PagedPoolQuota"=dword:00000000
>
> "PagedPoolSize"=dword:00000000
>
> "SecondLevelDataCache"=dword:00000000
>
> "SystemPages"=dword:00000000
>
> "PagingFiles"=hex(7):43,00,3a,00,5c,00,70,00,61,00,67,00,65,00,66,00,69,00,6c,\
>
> 00,65,00,2e,00,73,00,79,00,73,00,20,00,32,00,30,00,34,00,36,00,20,00,34,00,\
>
> 30,00,39,00,32,00,00,00,00,00
>
> "PhysicalAddressExtension"=dword:00000000
>
> "WriteWatch"=dword:00000001
>
> [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session
> Manager\Memory Management\PrefetchParameters]
> "VideoInitTime"=dword:0000055f
>
> "EnablePrefetcher"=dword:00000002
>
> "AppLaunchMaxNumPages"=dword:00000fa0
>
> "AppLaunchMaxNumSections"=dword:000000aa
>
> "AppLaunchTimerPeriod"=hex:80,69,67,ff,ff,ff,ff,ff
>
> "BootMaxNumPages"=dword:0001f400
>
> "BootMaxNumSections"=dword:00000ff0
>
> "BootTimerPeriod"=hex:00,f2,d8,f8,ff,ff,ff,ff
>
> "MaxNumActiveTraces"=dword:00000008
>
> "MaxNumSavedTraces"=dword:00000008
>
> "RootDirPath"="Prefetch"
>
> "HostingAppList"="DLLHOST.EXE,MMC.EXE,RUNDLL32.EXE"
>
>
>
>
>
> Windows Registry Editor Version 5.00
>
> [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session
>
> Manager\SubSystems]
>
> "Debug"=hex(2):00,00
>
> "Kmode"=hex(2):25,00,53,00,79,00,73,00,74,00,65,00,6d,00,52,00,6f,00,6f,00,74,\
>
> 00,25,00,5c,00,73,00,79,00,73,00,74,00,65,00,6d,00,33,00,32,00,5c,00,77,00,\
>
> 69,00,6e,00,33,00,32,00,6b,00,2e,00,73,00,79,00,73,00,00,00
>
> "Optional"=hex(7):50,00,6f,00,73,00,69,00,78,00,00,00,00,00
>
> "Posix"=hex(2):25,00,53,00,79,00,73,00,74,00,65,00,6d,00,52,00,6f,00,6f,00,74,\
>
> 00,25,00,5c,00,73,00,79,00,73,00,74,00,65,00,6d,00,33,00,32,00,5c,00,70,00,\
>
> 73,00,78,00,73,00,73,00,2e,00,65,00,78,00,65,00,00,00
>
> "Required"=hex(7):44,00,65,00,62,00,75,00,67,00,00,00,57,00,69,00,6e,00,64,00,\
>
> 6f,00,77,00,73,00,00,00,00,00
>
> "Windows"=hex(2):25,00,53,00,79,00,73,00,74,00,65,00,6d,00,52,00,6f,00,6f,00,\
>
> 74,00,25,00,5c,00,73,00,79,00,73,00,74,00,65,00,6d,00,33,00,32,00,5c,00,63,\
>
> 00,73,00,72,00,73,00,73,00,2e,00,65,00,78,00,65,00,20,00,4f,00,62,00,6a,00,\
>
> 65,00,63,00,74,00,44,00,69,00,72,00,65,00,63,00,74,00,6f,00,72,00,79,00,3d,\
>
> 00,5c,00,57,00,69,00,6e,00,64,00,6f,00,77,00,73,00,20,00,53,00,68,00,61,00,\
>
> 72,00,65,00,64,00,53,00,65,00,63,00,74,00,69,00,6f,00,6e,00,3d,00,31,00,30,\
>
> 00,32,00,34,00,2c,00,33,00,30,00,37,00,32,00,2c,00,35,00,31,00,32,00,20,00,\
>
> 57,00,69,00,6e,00,64,00,6f,00,77,00,73,00,3d,00,4f,00,6e,00,20,00,53,00,75,\
>
> 00,62,00,53,00,79,00,73,00,74,00,65,00,6d,00,54,00,79,00,70,00,65,00,3d,00,\
>
> 57,00,69,00,6e,00,64,00,6f,00,77,00,73,00,20,00,53,00,65,00,72,00,76,00,65,\
>
> 00,72,00,44,00,6c,00,6c,00,3d,00,62,00,61,00,73,00,65,00,73,00,72,00,76,00,\
>
> 2c,00,31,00,20,00,53,00,65,00,72,00,76,00,65,00,72,00,44,00,6c,00,6c,00,3d,\
>
> 00,77,00,69,00,6e,00,73,00,72,00,76,00,3a,00,55,00,73,00,65,00,72,00,53,00,\
>
> 65,00,72,00,76,00,65,00,72,00,44,00,6c,00,6c,00,49,00,6e,00,69,00,74,00,69,\
>
> 00,61,00,6c,00,69,00,7a,00,61,00,74,00,69,00,6f,00,6e,00,2c,00,33,00,20,00,\
>
> 53,00,65,00,72,00,76,00,65,00,72,00,44,00,6c,00,6c,00,3d,00,77,00,69,00,6e,\
>
> 00,73,00,72,00,76,00,3a,00,43,00,6f,00,6e,00,53,00,65,00,72,00,76,00,65,00,\
>
> 72,00,44,00,6c,00,6c,00,49,00,6e,00,69,00,74,00,69,00,61,00,6c,00,69,00,7a,\
>
> 00,61,00,74,00,69,00,6f,00,6e,00,2c,00,32,00,20,00,50,00,72,00,6f,00,66,00,\
>
> 69,00,6c,00,65,00,43,00,6f,00,6e,00,74,00,72,00,6f,00,6c,00,3d,00,4f,00,66,\
>
> 00,66,00,20,00,4d,00,61,00,78,00,52,00,65,00,71,00,75,00,65,00,73,00,74,00,\
>
> 54,00,68,00,72,00,65,00,61,00,64,00,73,00,3d,00,31,00,36,00,00,00
>
> [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session
>
> Manager\SubSystems\CSRSS]
>
> "CsrSrvSharedSectionBase"=dword:7f6f0000
>
> Boot.ini
>
> [boot loader]
>
> timeout=30
>
> default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
>
> [operating systems]
>
> multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003,
> Standard"
>
> /fastdetect /NoExecute=OptOut
>
> ----
>
> What parameter needs to be changed in your opinion?
>
> Regards,
>
> Jim
From: Jim on

"TP" <tperson.knowspamn(a)mailandnews.com> wrote in message
news:%23xSm6GlTIHA.4712(a)TK2MSFTNGP04.phx.gbl...
> Hi Jim,
>
> I reviewed your registry and boot.ini settings and there is nothing
> abnormal that would cause the error you are seeing. Please gather more
> information so that hopefully we can track down the cause. To that end I
> will ask you more questions and give suggestions.
>
> Dheapmon may help here so perhaps you should start by getting that
> working. It is normally not too difficult.
>
> 1. Logon to the console (we will start by looking at desktop heap usage
> for session 0)
> 2. Download and install dheapmon 8.1
> 3. Open up a command prompt and change to the dheapmon folder that is
> applicable, usually c:\kktools\dheapmon8.1\x86
> 4. Install dheapmon and specify to download symbols from the ms symbol
> server with this command:
>
> dheapinst.exe -y srv*http://msdl.microsoft.com/download/symbols

Done, successfull


>
> You should get a success message after it is finished. If not, make sure
> you have the command correct and try again. I have had times when the
> download doesn't work and it will not give you any message, on second
> attempt it works.
>
> This document may help you if you continue to have issues:
>
> Error message that is similar to the following when you try to install the
> Desktop Heap Monitor tool: "dheapinst - Win32k.sys symbol load error,
> correct symbol required"
>
> http://support.microsoft.com/kb/914216
>
> 5. Load the dheapmon driver with this command:
>
> dheapmon -l
>
> 6. Run a summary and detailed list to txt files with these commands:
>
> dheapmon>dheapsum_0.txt
> dheapmon -v>dheapverb_0.txt

Done.

>
> 7. Unload the dheapmon driver with this command:
>
> dheapmon -u

Unloaded.

>
> 8. Logon to a session and repeat steps 5-7, only this time name the two
> files something like dheapsum_rdp.txt and dheapverb_rdp.txt

Done. The failure to login in did not recur.

>
> You can post the contents of the above files if you like and I will
> comment. Please examine them and remove any information you don't want
> the world to know about.
>
> It is possible that you need to tweak your settings for your specific
> running applications/situation but I think it best to try and track down
> the cause first.

I'm going to have to wait until this problem recurs. It occured this morning
but is not occuring now.

>
> How many sessions normally logged on to this server prior to installing
> SP2?

Usually, no more than 2.

This time, I logged in 3 remote sessions, one using a VPN. Sessions were
logged off and back on without incident.

Don't you just hate it when you're trying to get something to fail to
document it, and then it fails to fail?

>
> If you logon to the server but do *not* open any programs are you able to
> get more than two sessions on without error?

Yes. Though the failure to log in occurs before the applicaton is started.

>
> You mentioned before that WMI is crashing, this might just be a symptom of
> heap exhaustion or it may be important. Any other errors in the system
> and application logs that may give a clue as to the cause of this or the
> problems in general?

While this test was going on, there were repeated events for WMI

"Event filter with query "SELECT * FROM __InstanceModificationEvent WITHIN
60 WHERE TargetInstance isa "Win32_LogicalMemoryConfiguration" AND
TargetInstance.AvailableVirtualMemory < 613869" could not be (re)activated
in namespace "//./root/CIMV2" because of error 0x8004106c. Events may not be
delivered through this filter until the problem is corrected."


>
> Are you connecting with very high resolution and high color depth?

Whatever the default is.

>
> Were there any other software changes?

The application involved was upgraded using detailed assistance from the
vendor. The update (actually a product replacement) needed SP2 installed as
a prerequesite on the server.

>
> What software do you have running on this machine? Think about also
> things that are running as services, perhaps a custom application, and/or
> security/antivirus/etc., special backup agents, server mirroring agents,
> etc.

The applications are MS office 2002 and "Software Motif"'s EMR database,
ChiroSoft, proprietary applications.

>
> Open up Task Manager and enable some of the additional columns,
> specifically User Objects and GDI Objects, VM Size, Paged Pool.
> Check "Show processes from all users"
>
> Anything stand out with high user objects, etc?

No.

>
> In Task Manager--Performance tab, what is your total physical memory, and
> under kernel memory what do you normally see for paged?

This appears to be normal. 31596.

Now that the problem is intermittent, I'm probably going to have to wait for
it to start failing and what dheapmon turns up.

Thanks for all your help.

Regards,
Jim