Prev: TreeCtrl and Images
Next: How to get crash dump when a unhandled CException is thrown by a MFC app
From: Joseph M. Newcomer on 7 Jan 2010 10:39
See below...
On Wed, 6 Jan 2010 14:27:01 -0800, Frank Perry <FrankPerry(a)discussions.microsoft.com>
wrote:

>Howdy,
>
>I'll try and cover several questions.
>
>The program is not Unicode.
>
>The CString in the case here could be generated one of serveral ways. The
>most likey is:
>const char *DefOperators[] =
>{
> "OR",
> "AND",
> "ADJ",
> "NEAR",
> "WITH",
> "SAME"
>};
****
The above declaration is erroneous. If you are using CString, the only correct format
would have been

LPCTSTR DefOperators[] =
{
_T("OR"),
_T("AND"),
....etc.
};

Otherwise, your code will not be correct if it is compiled as Unicode. It is extremely
important to realize that in 2010, the data type 'char' should be considered obsolete
except in very rare and exotic circumstances, and not only is this not one, but you have
asserted that CString must necessarily be 8 bit characters, which is simply not true.
****
>
>CString ValidateDefOperator(CString Value)
>{
> Value.MakeUpper();
> int Size = sizeof(DefOperators)/sizeof(DefOperators[0]);
****
Starting in VS2008, the poorly-documented _countof could be used:
int Size = _countof(DefOperators);
It is not clear why you have to introduce a separate variable for this, since the
expression is a compile time constant and takes no runtime cost to evaluate.
****
> for(int i=0; i<Size; i++)
> if(DefOperators[i] == Value)
> return Value;
> return "OR";
****
return _T("OR");
****
>}
>When it comes from some other source, it will be processed here so the end
>result would be the same.
>
>I can't trace it because it is a rare occurrence. I'm afraid that logic is
>about all I can apply to the problem.
****
The "rare occurrence" part suggests memory damage. See my essay on memory damage
detection on my MVP Tips site.
****
>
>I don't see instances of GetBuffer applied to the string in question. The
>data is a class variable. If something is clobbering the string, it will be
>almost impossible to sort out of the hundreds of C++ files that make up the
>code.
****
Note that I also asked "Did you every use GetBuffer/ReleaseBuffer on *any* string?"
(emphasis added).

It looks like a data overrun problem, but it could also be an uninitialized-pointer
problem. But an uninitialized-pointer problem should show up in debug mode as an access
fault to some weird-looking address.
****
>
>One reason for asking about this is I had a similar problem show up when a
>CString was being returned from or Oracle ODBC driver. That would sometimes
>return a bad length for the string in a field of the rowset. It was returned
>from the driver with a problem before anything in the program itself touched
>it. I was wondering if this was similar.
>
>The string itself is corrupted because the serialized data for it is FF FF
>FF 02 00 00 DD, which indicates it was returned to the serialize string
>function as 0xDD000002 bytes long.
****
Not sure what the FF FF FF represents. I don't use MFC serialization because I consider
it deeply unreliable in the presence of schema evolution. But it still looks like memory
damage, unless there is perhaps a bad serialization call that has set the serialization
stream off-by-one, or something else weird.
joe
****
Joseph M. Newcomer [MVP]
email: newcomer(a)flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 7 Jan 2010 10:57
0xDD is the byte used to represent free storage in the debug heap (see crt\src\dbgheap.c
in the VC directory, for example). So if one of these bytes were copied accidentally into
the space, for example, by using a dead pointer, who knows what is going to happen? If a
pointer which used to point to a structure is used to obtain a byte, you might get 0xDD,
and if another pointer to what used to point to another structure is used to store it, and
that pointer now points to what is now a CString, well, that's certain death, and the data
you see could result. Note that sometimes the data might be a 0 in which case a 0
overwrites a 0, so the bug only shows up when the overwritten data is nonzero.

Try running under the Application Verifier with all possible storage tests turned on. It
might show up something.
joe

On Thu, 7 Jan 2010 07:09:01 -0800, Frank Perry <FrankPerry(a)discussions.microsoft.com>
wrote:

>Howdy,
>
>I looked at the serialization code for serializing a CString. It gets the
>length from CString and based on that length writes out the length of the
>length (if that makes sense) in the form of a mask before adding the string.
>From that, I think the GetLength is the problem. Based on the length, it
>prefaces the length with either no bytes if the length is 0 - 0xfe, or FF if
>it's 0xff to 0xfffe, etc. It prefaces the string length with 0xff 0xff 0xff
>so it clearly believes the length is 0xDD000002 (e.i. requiring 4 bytes to
>express).
>
>I am not sure what I think about the buffer overrun. On the one hand, it is
>an obvious possibility that something else is clobbering the data. But on
>the other hand, almost everything we write is a string and 0xDD isn't in the
>normal character set. If it was something like 0x41 or 0x20 it would make
>much more sense.
>
>I am not familiar with the format of a CString but is the length someplace
>where it could be clobbered by an overrun while leaving the actual 2 inplace
>and also leave enought of the rest of the header to still function as a
>string? Assuming it's 'little endian' I would think the 2 would have been
>clobbered before an overwrite would leave an 0xdd three bytes deeper in the
>CString header.
>
>I find the idea of a copy function going bad hopeful. (If only because I
>can change that quickly and see what happens.) In my experience, copying a
>string with a bad length will result in the new string being just as bad as
>the old one. It copies by the string's stated length and not the actual
>length. (My ODBC Cstring problem was correctable by saving a new string with
>LockBuffer which stopped at the first 0x00 and not the GetLength value.)
>
Joseph M. Newcomer [MVP]
email: newcomer(a)flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Frank Perry on 11 Jan 2010 13:09
Howdy,

I haven't been able to use Application Verify. It seems to block my access
to the database. I haven't had a chance to see why or how but when I have my
program listed in it, the program failes to return data from the ACE dll that
interacts with the database.

--
Frank Perry
LavaLeaf Software


"Joseph M. Newcomer" wrote:

> 0xDD is the byte used to represent free storage in the debug heap (see crt\src\dbgheap.c
> in the VC directory, for example). So if one of these bytes were copied accidentally into
> the space, for example, by using a dead pointer, who knows what is going to happen? If a
> pointer which used to point to a structure is used to obtain a byte, you might get 0xDD,
> and if another pointer to what used to point to another structure is used to store it, and
> that pointer now points to what is now a CString, well, that's certain death, and the data
> you see could result. Note that sometimes the data might be a 0 in which case a 0
> overwrites a 0, so the bug only shows up when the overwritten data is nonzero.
>
> Try running under the Application Verifier with all possible storage tests turned on. It
> might show up something.
> joe
>
> On Thu, 7 Jan 2010 07:09:01 -0800, Frank Perry <FrankPerry(a)discussions.microsoft.com>
> wrote:
>
> >Howdy,
> >
> >I looked at the serialization code for serializing a CString. It gets the
> >length from CString and based on that length writes out the length of the
> >length (if that makes sense) in the form of a mask before adding the string.
> >From that, I think the GetLength is the problem. Based on the length, it
> >prefaces the length with either no bytes if the length is 0 - 0xfe, or FF if
> >it's 0xff to 0xfffe, etc. It prefaces the string length with 0xff 0xff 0xff
> >so it clearly believes the length is 0xDD000002 (e.i. requiring 4 bytes to
> >express).
> >
> >I am not sure what I think about the buffer overrun. On the one hand, it is
> >an obvious possibility that something else is clobbering the data. But on
> >the other hand, almost everything we write is a string and 0xDD isn't in the
> >normal character set. If it was something like 0x41 or 0x20 it would make
> >much more sense.
> >
> >I am not familiar with the format of a CString but is the length someplace
> >where it could be clobbered by an overrun while leaving the actual 2 inplace
> >and also leave enought of the rest of the header to still function as a
> >string? Assuming it's 'little endian' I would think the 2 would have been
> >clobbered before an overwrite would leave an 0xdd three bytes deeper in the
> >CString header.
> >
> >I find the idea of a copy function going bad hopeful. (If only because I
> >can change that quickly and see what happens.) In my experience, copying a
> >string with a bad length will result in the new string being just as bad as
> >the old one. It copies by the string's stated length and not the actual
> >length. (My ODBC Cstring problem was correctable by saving a new string with
> >LockBuffer which stopped at the first 0x00 and not the GetLength value.)
> >
> Joseph M. Newcomer [MVP]
> email: newcomer(a)flounder.com
> Web: http://www.flounder.com
> MVP Tips: http://www.flounder.com/mvp_tips.htm
> .
>
From: Joseph M. Newcomer on 11 Jan 2010 16:53
Hmmm...that's interesting, and may already be symptomatic of a problem. I've not used
databases, so don't know about potential problems that might exist.
joe

On Mon, 11 Jan 2010 10:09:01 -0800, Frank Perry <FrankPerry(a)discussions.microsoft.com>
wrote:

>Howdy,
>
>I haven't been able to use Application Verify. It seems to block my access
>to the database. I haven't had a chance to see why or how but when I have my
>program listed in it, the program failes to return data from the ACE dll that
>interacts with the database.
Joseph M. Newcomer [MVP]
email: newcomer(a)flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Hector Santos on 12 Jan 2010 01:08
Frank Perry wrote:

> ....
> I can't trace it because it is a rare occurrence. I'm afraid that logic is
> about all I can apply to the problem.


In general when rare and intermittent things like this rear its head,
it means its a PDE - programming design error.

It could be buffer overflow, underflow or whatever, a corruption is
occurring somewhere. This basically means, you need to isolate your
code, black box it and make sure its perfect in isolated testing and
if you still have problems when you plug it back into your
application, then you have a PE somewhere. Its that plain and simple. :)

Divide and conquer is one way to get to this problems. A good heap
manager tool may help too.

Happy Hunting!

--
HLS
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: TreeCtrl and Images
Next: How to get crash dump when a unhandled CException is thrown by a MFC app