Super Subtle
-
We've got some MFC code that was started when VC6 was still a good idea. This code is sprinkled with the obligatory use of BOOL (afterall, MFC likes the BOOL type). Today, I was running through our app, and noticed that when I clicked a button in the toolbar, the app would crash with the message
The stack around 'bCanClose' has become corruptedThe code was hanging up on the following block of code:bool bCanClose = true
HWND hWnd = m_pcDllView->GetSafeHwnd();
if(IsWindow(hWnd))
{
AfxCallWndProc( m_pcDllView,
hWnd,
WM_QUERY_CLOSE, //messages.h #define WM_QUERY_CLOSE = WM_APP + 1
(WPARAM) &bCanClose,
0);
if(! bCanClose) // <----------- Crash!
{
//...
}
}Here's the code that was handling the message being sent in the code shown above:
LRESULT CMyFormView::WindowProc(UINT message, WPARAM wParam, LPARAM lParam)
{
switch(message)
{
case WM_QUERY_CLOSE:
*((BOOL*) wParam) = IsOkToClose();
return 0;
}return CFormView::WindowProc(message, wParam, lParam);}
bool CMyFormView::IsOkToClose()
{
bool bResult = false;
// ... some code
return bResult;
}Can anyone spot the problem? I saw it right away, but I think my spidey senses are working overtime because I'm in bug-fix mode. After I discovered this, I had two other equally experienced C++ programmers look at it, and they just shrugged their shoulders. The answer is in the wParam. When the message is sent, a
BOOLis used. When the message is handled, it's expecting abool, and its value is set accordingly. When the value is returned to the calling function, the stack is corrupted because the types in the casts don't match."Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001 -
We've got some MFC code that was started when VC6 was still a good idea. This code is sprinkled with the obligatory use of BOOL (afterall, MFC likes the BOOL type). Today, I was running through our app, and noticed that when I clicked a button in the toolbar, the app would crash with the message
The stack around 'bCanClose' has become corruptedThe code was hanging up on the following block of code:bool bCanClose = true
HWND hWnd = m_pcDllView->GetSafeHwnd();
if(IsWindow(hWnd))
{
AfxCallWndProc( m_pcDllView,
hWnd,
WM_QUERY_CLOSE, //messages.h #define WM_QUERY_CLOSE = WM_APP + 1
(WPARAM) &bCanClose,
0);
if(! bCanClose) // <----------- Crash!
{
//...
}
}Here's the code that was handling the message being sent in the code shown above:
LRESULT CMyFormView::WindowProc(UINT message, WPARAM wParam, LPARAM lParam)
{
switch(message)
{
case WM_QUERY_CLOSE:
*((BOOL*) wParam) = IsOkToClose();
return 0;
}return CFormView::WindowProc(message, wParam, lParam);}
bool CMyFormView::IsOkToClose()
{
bool bResult = false;
// ... some code
return bResult;
}Can anyone spot the problem? I saw it right away, but I think my spidey senses are working overtime because I'm in bug-fix mode. After I discovered this, I had two other equally experienced C++ programmers look at it, and they just shrugged their shoulders. The answer is in the wParam. When the message is sent, a
BOOLis used. When the message is handled, it's expecting abool, and its value is set accordingly. When the value is returned to the calling function, the stack is corrupted because the types in the casts don't match."Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001 -
We've got some MFC code that was started when VC6 was still a good idea. This code is sprinkled with the obligatory use of BOOL (afterall, MFC likes the BOOL type). Today, I was running through our app, and noticed that when I clicked a button in the toolbar, the app would crash with the message
The stack around 'bCanClose' has become corruptedThe code was hanging up on the following block of code:bool bCanClose = true
HWND hWnd = m_pcDllView->GetSafeHwnd();
if(IsWindow(hWnd))
{
AfxCallWndProc( m_pcDllView,
hWnd,
WM_QUERY_CLOSE, //messages.h #define WM_QUERY_CLOSE = WM_APP + 1
(WPARAM) &bCanClose,
0);
if(! bCanClose) // <----------- Crash!
{
//...
}
}Here's the code that was handling the message being sent in the code shown above:
LRESULT CMyFormView::WindowProc(UINT message, WPARAM wParam, LPARAM lParam)
{
switch(message)
{
case WM_QUERY_CLOSE:
*((BOOL*) wParam) = IsOkToClose();
return 0;
}return CFormView::WindowProc(message, wParam, lParam);}
bool CMyFormView::IsOkToClose()
{
bool bResult = false;
// ... some code
return bResult;
}Can anyone spot the problem? I saw it right away, but I think my spidey senses are working overtime because I'm in bug-fix mode. After I discovered this, I had two other equally experienced C++ programmers look at it, and they just shrugged their shoulders. The answer is in the wParam. When the message is sent, a
BOOLis used. When the message is handled, it's expecting abool, and its value is set accordingly. When the value is returned to the calling function, the stack is corrupted because the types in the casts don't match."Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001I wish you'd left the answer off this one. I spotted it straight off, mainly because I was hosed by this back when I was still a real C++ developer. Oh well.:-D
Deja View - the feeling that you've seen this post before.
-
We've got some MFC code that was started when VC6 was still a good idea. This code is sprinkled with the obligatory use of BOOL (afterall, MFC likes the BOOL type). Today, I was running through our app, and noticed that when I clicked a button in the toolbar, the app would crash with the message
The stack around 'bCanClose' has become corruptedThe code was hanging up on the following block of code:bool bCanClose = true
HWND hWnd = m_pcDllView->GetSafeHwnd();
if(IsWindow(hWnd))
{
AfxCallWndProc( m_pcDllView,
hWnd,
WM_QUERY_CLOSE, //messages.h #define WM_QUERY_CLOSE = WM_APP + 1
(WPARAM) &bCanClose,
0);
if(! bCanClose) // <----------- Crash!
{
//...
}
}Here's the code that was handling the message being sent in the code shown above:
LRESULT CMyFormView::WindowProc(UINT message, WPARAM wParam, LPARAM lParam)
{
switch(message)
{
case WM_QUERY_CLOSE:
*((BOOL*) wParam) = IsOkToClose();
return 0;
}return CFormView::WindowProc(message, wParam, lParam);}
bool CMyFormView::IsOkToClose()
{
bool bResult = false;
// ... some code
return bResult;
}Can anyone spot the problem? I saw it right away, but I think my spidey senses are working overtime because I'm in bug-fix mode. After I discovered this, I had two other equally experienced C++ programmers look at it, and they just shrugged their shoulders. The answer is in the wParam. When the message is sent, a
BOOLis used. When the message is handled, it's expecting abool, and its value is set accordingly. When the value is returned to the calling function, the stack is corrupted because the types in the casts don't match."Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001I may be missing something really obvious here, but I fail to understand why this will crash or corrupt the stack. sizeof(BOOL) = 4 sizeof(bool) = 1 So &bCanClose will point to 4 bytes of stack allocated memory. *((bool*)&bCanClose) will modify the first 1 byte of memory at the address pointed by bCanClose (which already has 4 bytes of allocated memory). So it should be safe.:confused: However the reverse will obviously cause a stack corruption.
-
I may be missing something really obvious here, but I fail to understand why this will crash or corrupt the stack. sizeof(BOOL) = 4 sizeof(bool) = 1 So &bCanClose will point to 4 bytes of stack allocated memory. *((bool*)&bCanClose) will modify the first 1 byte of memory at the address pointed by bCanClose (which already has 4 bytes of allocated memory). So it should be safe.:confused: However the reverse will obviously cause a stack corruption.
Thanks, I saw the difference in the code immediately. I just do not see why it should corrupt anything. :confused:
INTP "Program testing can be used to show the presence of bugs, but never to show their absence."Edsger Dijkstra
-
Yeah I saw it right away too. This code is pretty ugly. Does this use MFC doc/view architecture because the document has much better ways of asking its views if its OK to close.
I don't know why this code was written the way it was. It's actually in a COM DLL.
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001 -
I may be missing something really obvious here, but I fail to understand why this will crash or corrupt the stack. sizeof(BOOL) = 4 sizeof(bool) = 1 So &bCanClose will point to 4 bytes of stack allocated memory. *((bool*)&bCanClose) will modify the first 1 byte of memory at the address pointed by bCanClose (which already has 4 bytes of allocated memory). So it should be safe.:confused: However the reverse will obviously cause a stack corruption.
Sorry - had the types switched. Now does it make sense?
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001 -
We've got some MFC code that was started when VC6 was still a good idea. This code is sprinkled with the obligatory use of BOOL (afterall, MFC likes the BOOL type). Today, I was running through our app, and noticed that when I clicked a button in the toolbar, the app would crash with the message
The stack around 'bCanClose' has become corruptedThe code was hanging up on the following block of code:bool bCanClose = true
HWND hWnd = m_pcDllView->GetSafeHwnd();
if(IsWindow(hWnd))
{
AfxCallWndProc( m_pcDllView,
hWnd,
WM_QUERY_CLOSE, //messages.h #define WM_QUERY_CLOSE = WM_APP + 1
(WPARAM) &bCanClose,
0);
if(! bCanClose) // <----------- Crash!
{
//...
}
}Here's the code that was handling the message being sent in the code shown above:
LRESULT CMyFormView::WindowProc(UINT message, WPARAM wParam, LPARAM lParam)
{
switch(message)
{
case WM_QUERY_CLOSE:
*((BOOL*) wParam) = IsOkToClose();
return 0;
}return CFormView::WindowProc(message, wParam, lParam);}
bool CMyFormView::IsOkToClose()
{
bool bResult = false;
// ... some code
return bResult;
}Can anyone spot the problem? I saw it right away, but I think my spidey senses are working overtime because I'm in bug-fix mode. After I discovered this, I had two other equally experienced C++ programmers look at it, and they just shrugged their shoulders. The answer is in the wParam. When the message is sent, a
BOOLis used. When the message is handled, it's expecting abool, and its value is set accordingly. When the value is returned to the calling function, the stack is corrupted because the types in the casts don't match."Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001One never stops learning! Anyway, compiler flag /Zp4 should solve this easily! :-D
-
Sorry - had the types switched. Now does it make sense?
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001Now that makes since! Thanks for clearing up the confusion. :-D
INTP "Program testing can be used to show the presence of bugs, but never to show their absence."Edsger Dijkstra
-
We've got some MFC code that was started when VC6 was still a good idea. This code is sprinkled with the obligatory use of BOOL (afterall, MFC likes the BOOL type). Today, I was running through our app, and noticed that when I clicked a button in the toolbar, the app would crash with the message
The stack around 'bCanClose' has become corruptedThe code was hanging up on the following block of code:bool bCanClose = true
HWND hWnd = m_pcDllView->GetSafeHwnd();
if(IsWindow(hWnd))
{
AfxCallWndProc( m_pcDllView,
hWnd,
WM_QUERY_CLOSE, //messages.h #define WM_QUERY_CLOSE = WM_APP + 1
(WPARAM) &bCanClose,
0);
if(! bCanClose) // <----------- Crash!
{
//...
}
}Here's the code that was handling the message being sent in the code shown above:
LRESULT CMyFormView::WindowProc(UINT message, WPARAM wParam, LPARAM lParam)
{
switch(message)
{
case WM_QUERY_CLOSE:
*((BOOL*) wParam) = IsOkToClose();
return 0;
}return CFormView::WindowProc(message, wParam, lParam);}
bool CMyFormView::IsOkToClose()
{
bool bResult = false;
// ... some code
return bResult;
}Can anyone spot the problem? I saw it right away, but I think my spidey senses are working overtime because I'm in bug-fix mode. After I discovered this, I had two other equally experienced C++ programmers look at it, and they just shrugged their shoulders. The answer is in the wParam. When the message is sent, a
BOOLis used. When the message is handled, it's expecting abool, and its value is set accordingly. When the value is returned to the calling function, the stack is corrupted because the types in the casts don't match."Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001Spotted it right away. I must have spent way too much time stepping through message map code (those sort of return value mismatches could happen all too easily with VC6) before I jumped onto the WTL bandwagon. :doh:
Anna :rose: Linting the day away :cool: Anna's Place | Tears and Laughter "If mushy peas are the food of the devil, the stotty cake is the frisbee of God"