Local Variable changes suddenly
-
I am currently analyzing some nasty application crashes. Fortunately I can already reproduce these crashes within a few minutes and I get a dump that is supposed to be useful (note that it is a crash dump from an optimized release version). Let me outline the code where the application crashes (pseudo code):
int myNumberOfElements = myCalculation();
[...] // some other code, but no assignments or referencing to myNumber
if(myNumberOfElements < 1 || myNumberOfElements > 100000)
{
return;
}MyArray* pArray = AllocArray(myNumberOfElements); // dump shows that myNumberOfElements is greater than 10^9
AllocArrayunfortunately belongs to a 3rd party library, but it basically callsmalloc. It then crashes as it does not check accordingly if malloc succeeded. However, the root cause of this problem seems to be the large number of elements (which is definitely not the result of my calculation). Further there is only one assignment tomyNumberOfElementsand if used as parameter 'call by value' is used. What do you guys think, is the dump file corrupt? Is there a way that the stack gets corrupted, causing this problem?So somewhere between assigning
myNumberOfElementsa value and callingAllocArray(),myNumberOfElementsis changing? Have you stepped through those statements while watchingmyNumberOfElements?"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Man who follows car will be exhausted." - Confucius
-
I am currently analyzing some nasty application crashes. Fortunately I can already reproduce these crashes within a few minutes and I get a dump that is supposed to be useful (note that it is a crash dump from an optimized release version). Let me outline the code where the application crashes (pseudo code):
int myNumberOfElements = myCalculation();
[...] // some other code, but no assignments or referencing to myNumber
if(myNumberOfElements < 1 || myNumberOfElements > 100000)
{
return;
}MyArray* pArray = AllocArray(myNumberOfElements); // dump shows that myNumberOfElements is greater than 10^9
AllocArrayunfortunately belongs to a 3rd party library, but it basically callsmalloc. It then crashes as it does not check accordingly if malloc succeeded. However, the root cause of this problem seems to be the large number of elements (which is definitely not the result of my calculation). Further there is only one assignment tomyNumberOfElementsand if used as parameter 'call by value' is used. What do you guys think, is the dump file corrupt? Is there a way that the stack gets corrupted, causing this problem?So the test for myNumberOfElements being out of range happens right before the call to AllocArray? That's not a lot of time for it to be getting corrupted by your code before the call. It seems definitely possible the value seen in dump file is corrupt as a result of the crash rather than the cause of the crash. But... - As already asked, how are you looking at the dump file? - Does the debug version of the program work OK? - Have you tried turning off or lowering optimization? Try this first if you haven't! - Are you sure that AllocArray is capable of handling 100000 elements? Some things to maybe try: - Can you modify your program to output myNumberOfElements right before the call? - Create a global variable to hold the result of myCalculation and pass that in to AllocArray. If this is memory/stack corruption happening in your function, it's unlikely it would step on the global var in the same way. - Hardcode the value being passed in to AllocArray. If you can find out and use the real result of myCalculation, even better. Does this still crash? Either way, this will tell you a lot about what's going on.
-
So the test for myNumberOfElements being out of range happens right before the call to AllocArray? That's not a lot of time for it to be getting corrupted by your code before the call. It seems definitely possible the value seen in dump file is corrupt as a result of the crash rather than the cause of the crash. But... - As already asked, how are you looking at the dump file? - Does the debug version of the program work OK? - Have you tried turning off or lowering optimization? Try this first if you haven't! - Are you sure that AllocArray is capable of handling 100000 elements? Some things to maybe try: - Can you modify your program to output myNumberOfElements right before the call? - Create a global variable to hold the result of myCalculation and pass that in to AllocArray. If this is memory/stack corruption happening in your function, it's unlikely it would step on the global var in the same way. - Hardcode the value being passed in to AllocArray. If you can find out and use the real result of myCalculation, even better. Does this still crash? Either way, this will tell you a lot about what's going on.
Yes, the test is right before the AllocArray-call. I use Visual Studio for analyzing the dump. I have done this a thousand times before but in my experience it rather displayed no value instead of a wrong value. I will try to reproduce the problem on a Debug (or at least less optimized) version. The AllocArray basically can handle any size as long as enough memory is available (100000 should be no problem, 10^9 is definitely a problem). I will also try your other suggestions, if applicable. The code runs within the rendering of a openGL frame and gets executed many thousand times before a crash may occur... Thank you.
-
So somewhere between assigning
myNumberOfElementsa value and callingAllocArray(),myNumberOfElementsis changing? Have you stepped through those statements while watchingmyNumberOfElements?"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Man who follows car will be exhausted." - Confucius
-
How are you analysing the dump file? Can show me a stack track for a start?
Steve
As posted in my other reply I use Visual Studio for analyzing the dumps. I would like to avoid posting a call stack as I think that my superiors would not be very happy finding parts of our code on codeproject (even if they are small and not useful to anyone).
-
As posted in my other reply I use Visual Studio for analyzing the dumps. I would like to avoid posting a call stack as I think that my superiors would not be very happy finding parts of our code on codeproject (even if they are small and not useful to anyone).
A call stack is unlikely to give away trade secrets.
Steve
-
I cannot step through this particular case, because the code runs perfect many thousand times before it fails a single time :-(
How about using a conditional breakpoint?
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Man who follows car will be exhausted." - Confucius
-
Yes, the test is right before the AllocArray-call. I use Visual Studio for analyzing the dump. I have done this a thousand times before but in my experience it rather displayed no value instead of a wrong value. I will try to reproduce the problem on a Debug (or at least less optimized) version. The AllocArray basically can handle any size as long as enough memory is available (100000 should be no problem, 10^9 is definitely a problem). I will also try your other suggestions, if applicable. The code runs within the rendering of a openGL frame and gets executed many thousand times before a crash may occur... Thank you.
Ahh, the fact that it runs thousands of times OK before crashing changes things a bit. I don't suppose it's so convenient as to always crash at the exact same point/iteration? If you really think the local variable is getting suddenly changed, and that's causing the problem, you should try to prove that or disprove it: - Definitely try moving it to a global. - Do the other local variables in the function all look OK when you look at the in the dump? Is myNumberOfElements the last variable declared in the function? It would be odd if only myNumberOfElements was getting corrupted and nothing else around it (unless AllocArray is taking a reference to it and corrupting it there). You could try placing "guard" variables or the same size, with known values, declared right before and after myNumberOfElements and seeing if those get alterred in any way during the crash. - store the result of myCalculation in two or three separate variables (in different locations - a global, a heap var, and the stack) that you can compare in the dump. No way they can all get corrupted in the same way without everything else in the program also doing so. - log the value to a file when received from myCalculation and again immediately before the AllocArray call. One or more of thse things should be sufficient to prove or disprove your theory and give you enough tot go on for the next step
-
Ahh, the fact that it runs thousands of times OK before crashing changes things a bit. I don't suppose it's so convenient as to always crash at the exact same point/iteration? If you really think the local variable is getting suddenly changed, and that's causing the problem, you should try to prove that or disprove it: - Definitely try moving it to a global. - Do the other local variables in the function all look OK when you look at the in the dump? Is myNumberOfElements the last variable declared in the function? It would be odd if only myNumberOfElements was getting corrupted and nothing else around it (unless AllocArray is taking a reference to it and corrupting it there). You could try placing "guard" variables or the same size, with known values, declared right before and after myNumberOfElements and seeing if those get alterred in any way during the crash. - store the result of myCalculation in two or three separate variables (in different locations - a global, a heap var, and the stack) that you can compare in the dump. No way they can all get corrupted in the same way without everything else in the program also doing so. - log the value to a file when received from myCalculation and again immediately before the AllocArray call. One or more of thse things should be sufficient to prove or disprove your theory and give you enough tot go on for the next step
Thank you for your hints, they helped me to find out that the actual problem seems to be a memory leak. So the actual reason was not the wrong parameter for the malloc (which also caused the memory to be exhausted), but the malloc seems to have corrupted the stack if there was no more memory available, and hence the value of the variable was no longer accurate. I will now try to verify this thesis, the fix itself will be obvious.
-
Thank you for your hints, they helped me to find out that the actual problem seems to be a memory leak. So the actual reason was not the wrong parameter for the malloc (which also caused the memory to be exhausted), but the malloc seems to have corrupted the stack if there was no more memory available, and hence the value of the variable was no longer accurate. I will now try to verify this thesis, the fix itself will be obvious.