Prev: KiFastSystemCall in the call stack
Next: WM_QUERYENDSESSION behavior
From: "Jialiang Ge [MSFT]" on 12 Oct 2009 03:02
Hello Bob

I agree. It's most likely a stack corruption that results in the problem.
There are generally three causes of stack corruption:

A. Stack Overrun
1. Array indexing errors cause the stack overrun.
2. Static buffer overrun on the stack.

B. Calling Convention Mismatch

We can detect stack corruption in compile time (if you have the source code
of the shell extension) or in runtime, but most of these skills require
that you have the source code of the shell extension, which may not be
feasible in this case. Therefore, Kerem's suggestion to disable shell
extension one by one may be the most effective and efficient way to find
the culprit module. Please try it. After locating the module, if it's a
third party module, you can consider checking for the product's update or
contacting the product's supports.

----------------------------------------------
Detecting Stack Corruption:

1. Detect stack overrun at compile time.

Because stack buffer overruns are such common problems, there is a tool
that can help detect these errors at compile time: PREfast.

PREfast is the codename for the /analyze compiler switch and SAL
annotations:
http://blogs.msdn.com/vcblog/archive/2008/02/05/prefast-and-sal-annotations.
aspx
To enable PREfast, open the project's properties, and turn to the Code
Analysis / General node. Change the value of "Enable Code Analysis For
C/C++ on Build" to "Yes (/analyze)", then rebuild the project. You will see
warning messages of buffer overrun in the Error List window of Visual
Studio. For example:

warning C6386: Buffer overrun: accessing 'n', the writable size is '8'
bytes, but '12' bytes might be written
warning C6201: Index '2' is out of valid index range '0' to '1' for
possibly stack allocated buffer 'n'
warning C6201: Index '3' is out of valid index range '0' to '1' for
possibly stack allocated buffer 'n'

, and these warnings for StaticBufferOverrun:

warning C6204: Possible buffer overrun in call to 'wcscpy': use of
unchecked parameter 'pszSource'
warning C4996: 'wcscpy': This function or variable may be unsafe. Consider
using wcscpy_s instead. To disable deprecation, use
_CRT_SECURE_NO_WARNINGS.

2. Detect stack overrun and calling convention mismatch at run-time.

Unfortunately, Application Verifier cannot detect stack overrun and calling
convention mismatch effectively at run-time.

In VC++ compiler, there is an option to enable the stack frame runtime
error checking: Project Properties / C/C++ / Code Generation / Basic
Runtime Check. Stack frame checking is enabled when the option is set to
"Stack Frames (/RTCs)". The stack frame checking switch is on in Debug
build, and it is off in Release build by default. The option helps protect
against a number of different stack corruptions:

Each time a function is called, it initializes all local variables to
nonzero values to prevent them from retaining old values from prior
function calls.

It verifies the stack pointer (esp register) to ensure that stack
corruptions caused by calling convention mismatches do not occur.

Protects against buffer overruns and underruns of local variables.

When any of the three corruptions happens, the application is broken into a
debugger with a break instruction exception. The callstack is like this:

0:000> k
ChildEBP RetAddr
0014efc4 00ac1ba9 ntdll!DbgBreakPoint
0014fe18 00ac1bef CppStackCorruption!failwithmessage+0x1ea
0014fe28 00ac18a6 CppStackCorruption!_RTC_Failure+0x37
0014fe4c 00ac1091 CppStackCorruption!_RTC_CheckEsp+0x18
0014fe54 00ac11fc CppStackCorruption!wmain+0x11
0014fe98 75c436d6 CppStackCorruption!__tmainCRTStartup+0x10f
0014fea4 770b883c kernel32!BaseThreadInitThunk+0xe
0014fee4 770b880f ntdll!__RtlUserThreadStart+0x70
0014fefc 00000000 ntdll!_RtlUserThreadStart+0x1b

It is important to note that the /RTC compiler options are designed to work
with debug builds and, as such, have no impact on released builds. The /RTC
switch is meant solely to test your code during development.

Other viable options to detect stack corruption at run-time include
Rational's Purify or NuMega's BoundsChecker.

Regards,
Jialiang Ge
Microsoft Online Community Support

=================================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msdnmg(a)microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.
=================================================


First  |  Prev  | 
Pages: 1 2 3
Prev: KiFastSystemCall in the call stack
Next: WM_QUERYENDSESSION behavior