From: jolteroli on
Hee Rob,

> reg add "HKLM\SYSTEM\CurrentControlSet\Control\Session
> Manager \Environment" /v _NT_SYMBOL_PATH /d
> "SRV*c:\windows\symbols*http://msdl.microsoft.com/download/symbols" /f

Real men don't click, cool... :)

> ntoskrnl.exe!KiSwapContext+0x26
> ntoskrnl.exe!KiSwapThread+0x2e5
> ntoskrnl.exe!KeWaitForSingleObject+0x346
> ntoskrnl.exe!KiSuspendThread+0x18
> ntoskrnl.exe!KiDeliverApc+0x117
> ntoskrnl.exe!KiSwapThread+0x300
> ntoskrnl.exe!KeWaitForMultipleObjects+0x3d7
> win32k.sys!RawInputThread+0x4e0
> win32k.sys!xxxCreateSystemThreads+0x60
> win32k.sys!NtUserCallOneParam+0x23
> ntoskrnl.exe!KiFastCallEntry+0xfc
> ntdll.dll!KiFastSystemCallRet
> winsrv.dll!NtUserCallOneParam+0xc
>
> Does this tell you anything?

Since our servers (not having this issue) doesn't fork to KiDeliverApc, I
guess there is constantly arriving keyb data (that is what RawInputThread
RIT "polls" for) and that's causing the IO. I have no clue what is causing
the keyb data...

In german we have the Systemcontrol "Eingabehilfen" with the symbol of Steve
Balmers wheelchair. Check if any enhanced services is turned on there...

ATI's stupid HotKeyPolling service comes to mind too...

May be something in this direction. For me it seems the data comes from the
keyb/driver.

> To me it tells nothing as I don't have the sources :-| (Almost a complete
> waste of time.)

Any additional help would need a debugger, so it was a waste of time. Sorry
Rob.

-jolt

From: TP on
Robert and Vincent,

What are the exact problem symptoms you are experiencing? Is
the server running slow? Are users complaining? If yes, what
lead you to the conclusion that this particular reading is the
source of the problems you are having?

I have looked at the numbers you are seeing and this must be
a bug. I see this on every 2003 server I have examined so far
(since reading your post). If these numbers were true the
servers would grind to a halt.

Thanks.

-TP

Robert Premuž wrote:
> On 2008-10-31 I wrote:
>>
>> On Oct 17, 3:56 pm, Vincent <Vinc...(a)discussions.microsoft.com>
>> wrote:
>>>
>>> Currently I'm running into some issues which occur on each and
>>> every windows 2003 server that I'm managing:
>>>
>>> When someone logs in on a server the csrss.exe process starts to
>>> use up a lot of disk I/O. It seems to have a minimum of 10 MB/s
>>> with a maximum of 80 MB/s although the average seems to be around
>>> 30 MB/s. The high I/O continues until the user logs off. The high
>>> I/O also stops completely when (and this is especially weird) the
>>> Terminal service client screen is minimized, but high disk activity
>>> will resume as soon as the screen is restored.
>>
>> Vincent, I don't know the solution to the problem but I can confirm
>> that I experience the same on two MS Windows Server 2003 SP2
>> (Standard Edition) with all the latest patches (the servers don't
>> have the same hardware or software configuration).
>>
>> Here is how I can get extremely high I/O for the csrss.exe: when I'm
>> connected to the server through the Remote Desktop I press and hold a
>> key (e.g. Shift or Alt) on the keyboard: the Process Explorer shows
>> that Other Bytes Delta is about 200 MB in that case. See picture on
>> http://www.hotshare.net/image/91284-64755441d6.html
>>
>> This is really a major issue. It there any MSMVP who has seen that
>> before and knows the solution?
>
> I installed the 934330 hotfix received from MS Support (see
> http://support.microsoft.com/kb/934330 that explains a similar problem
> with csrss.exe) but it didn't solved the problem.
>
> -- rpr.

From: rpr on
On Nov 13, 9:07 pm, "jolteroli" <jolt1...(a)gmx.net> wrote:
>
> > reg add "HKLM\SYSTEM\CurrentControlSet\Control\Session
> > Manager \Environment" /v _NT_SYMBOL_PATH /d
> > "SRV*c:\windows\symbols*http://msdl.microsoft.com/download/symbols" /f
>
> Real men don't click, cool... :)

I also live in Unix world and hence I'm familiar with the command
line. Unfortunately MS KBAs rarely suggest using command line tools.
Fortunately we have this groups for writing our own knowledge base
articles :-)

> Since our servers (not having this issue) doesn't fork to KiDeliverApc, I
> guess there is constantly arriving keyb data (that is what RawInputThread
> RIT "polls" for) and that's causing the IO. I have no clue what is causing
> the keyb data...
>
> In german we have the Systemcontrol "Eingabehilfen" with the symbol of Steve
> Balmers wheelchair. Check if any enhanced services is turned on there...
>
> ATI's stupid HotKeyPolling service comes to mind too...
>
> May be something in this direction. For me it seems the data comes from the
> keyb/driver.

Any of the Accessibility Options is not enabled on the server.
Also, no ATI services.

-- rpr.
From: rpr on
I discovered the high I/O problem in csrss.exe while investigating a
problem with the system clock on one of DCs in the domain. The
server's system clock is quite inaccurate.
I tried to solve that by reconfiguring the Windows Time Service
(w32time) using the procedures suggested at
http://technet.microsoft.com/en-us/library/cc773061.aspx
but it was not satisfactory as the Windows Time Service cannot take
into account the clock drift (as a good NTP service should).

At http://answers.google.com/answers/threadview?id=438195
you can find complains that suggest this high I/O slows down the
Windows Server 2003. Note that the question is written on 2006-01-26.

At http://forum.sysinternals.com/forum_posts.asp?TID=12894
Mark Russinovich said the following regarding this issue on
2008-03-06:
"This behavior is a known bug in the w2k3 terminal server I/O
counters."
But it was not explained any further.

Even if this bug causes no real harm it should have been solved long
ago. It certainly creates confusion and increases dissatisfaction with
MS software.

-- rpr.

On Nov 13, 9:27 pm, "TP" <tperson.knowsp...(a)mailandnews.com> wrote:
> Robert and Vincent,
>
> What are the exact problem symptoms you are experiencing?  Is
> the server running slow?  Are users complaining?  If yes, what
> lead you to the conclusion that this particular reading is the
> source of the problems you are having?
>
> I have looked at the numbers you are seeing and this must be
> a bug.  I see this on every 2003 server I have examined so far
> (since reading your post).  If these numbers were true the
> servers would grind to a halt.
>
> Thanks.
>
> -TP