Ominous bug from DLL world [modified]
-
Ok. C++ project that consist out of the 3 subprojects: Executable, DLL, and Static library. Dependencies as follows: Executable linked to Static library, DLL linked to static library. Static library contains set of C++ classes which are used in DLL and the EXE. DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". The DLL is loaded on the fly with LoadLibrary and then unloaded with FreeLibrary API (this is done to conserve memory). Classes in the static lib usually contain arbitrary set of virtual functions ( very usual stuff: Dtors, overrides etc). Here we go: 1. Executable loads DLL into memory with LoadLibrary. 2. Executable calls function in DLL say "GetClassA" which in turn looks like this for simplicity:
CMyClass* GetClassA()
{
return new CMyClass;
}3. Executable receives pointer to CMyClass. 4. Executable frees DLL with FreeLibrary. 5. Application at some point goes on to use CMyClass pointer and dies miserable death with Access violation. What has happened? Answer later.
modified on Tuesday, July 22, 2008 11:19 PM
Some pointer issue?
"The clue train passed his station without stopping." - John Simmons / outlaw programmer "Real programmers just throw a bunch of 1s and 0s at the computer to see what sticks" - Pete O'Hanlon
-
The executable and DLL are using different C runtimes, which means you can't pass pointers (among other things) across the executable/DLL boundary. See http://msdn.microsoft.com/en-us/library/ms235460(VS.80).aspx[^] for a full explanation.
-
Ok. C++ project that consist out of the 3 subprojects: Executable, DLL, and Static library. Dependencies as follows: Executable linked to Static library, DLL linked to static library. Static library contains set of C++ classes which are used in DLL and the EXE. DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". The DLL is loaded on the fly with LoadLibrary and then unloaded with FreeLibrary API (this is done to conserve memory). Classes in the static lib usually contain arbitrary set of virtual functions ( very usual stuff: Dtors, overrides etc). Here we go: 1. Executable loads DLL into memory with LoadLibrary. 2. Executable calls function in DLL say "GetClassA" which in turn looks like this for simplicity:
CMyClass* GetClassA()
{
return new CMyClass;
}3. Executable receives pointer to CMyClass. 4. Executable frees DLL with FreeLibrary. 5. Application at some point goes on to use CMyClass pointer and dies miserable death with Access violation. What has happened? Answer later.
modified on Tuesday, July 22, 2008 11:19 PM
Graham Bradshaw and ied hitting it pretty close. The explanation is not that simple though as why access violation happens. Hint: If that static library was DLL instead, the whole problem goes completely away. When I post answer unless someone beats me to it I will post a link to a small sample project so you can see that bug first hand through the debugger. Have fun!!!
modified on Tuesday, July 22, 2008 11:44 PM
-
Ok. C++ project that consist out of the 3 subprojects: Executable, DLL, and Static library. Dependencies as follows: Executable linked to Static library, DLL linked to static library. Static library contains set of C++ classes which are used in DLL and the EXE. DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". The DLL is loaded on the fly with LoadLibrary and then unloaded with FreeLibrary API (this is done to conserve memory). Classes in the static lib usually contain arbitrary set of virtual functions ( very usual stuff: Dtors, overrides etc). Here we go: 1. Executable loads DLL into memory with LoadLibrary. 2. Executable calls function in DLL say "GetClassA" which in turn looks like this for simplicity:
CMyClass* GetClassA()
{
return new CMyClass;
}3. Executable receives pointer to CMyClass. 4. Executable frees DLL with FreeLibrary. 5. Application at some point goes on to use CMyClass pointer and dies miserable death with Access violation. What has happened? Answer later.
modified on Tuesday, July 22, 2008 11:19 PM
Because the vtable is referencing methods in the DLL address space.
Anyone who thinks he has a better idea of what's good for people than people do is a swine. - P.J. O'Rourke
-
Because the vtable is referencing methods in the DLL address space.
Anyone who thinks he has a better idea of what's good for people than people do is a swine. - P.J. O'Rourke
Yup, that's my bet, too. He unloaded the code that he wants to call.
We are a big screwed up dysfunctional psychotic happy family - some more screwed up, others more happy, but everybody's psychotic joint venture definition of CP
blog: TDD - the Aha! | Linkify!| FoldWithUs! | sighist -
Which one? (Though this is relevant only for Grahams suggestions, not what I think it is...)
We are a big screwed up dysfunctional psychotic happy family - some more screwed up, others more happy, but everybody's psychotic joint venture definition of CP
blog: TDD - the Aha! | Linkify!| FoldWithUs! | sighist -
Because the vtable is referencing methods in the DLL address space.
Anyone who thinks he has a better idea of what's good for people than people do is a swine. - P.J. O'Rourke
Correct. :-D Even though the address of the class itself lays in the global process heap, the vtable address lays withing the DLL address space. Call to the FreeLibrary deletes all vtable entries of any class allocated inside the DLL. Every time you go to use any virtual function including virtual destructor (via "delete"), the program crashes. Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested. If you allocate same class in EXE it will have completely different vtable addresses because EXE has its own copy too. For example you could have just but that duplicate class code into EXE and the DLL to achive the same result.By changing static library configuration into DLL solves that problem because neither EXE nor factory DLL has any copies of the class. Classes allocated this way will all have vtable adreesses lay in that third DLL. If that DLL has lifetime of EXE the problem goes away. A bit about the project where I witnessed that bug. Description I gave was over simplified. That dynamic DLL was compiled on the fly by the executable with C++ compiler, then loaded, then ran a for loop inserting new classes into the sequence container inside the EXE. Also it has rather complex exchage of information back and forth. After that the dll was freed and deleted from hard disk. DLL was serving as a scripting language for the EXE. That bug never manifested itself until we added virtual functions to the shared classes.
-
Correct. :-D Even though the address of the class itself lays in the global process heap, the vtable address lays withing the DLL address space. Call to the FreeLibrary deletes all vtable entries of any class allocated inside the DLL. Every time you go to use any virtual function including virtual destructor (via "delete"), the program crashes. Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested. If you allocate same class in EXE it will have completely different vtable addresses because EXE has its own copy too. For example you could have just but that duplicate class code into EXE and the DLL to achive the same result.By changing static library configuration into DLL solves that problem because neither EXE nor factory DLL has any copies of the class. Classes allocated this way will all have vtable adreesses lay in that third DLL. If that DLL has lifetime of EXE the problem goes away. A bit about the project where I witnessed that bug. Description I gave was over simplified. That dynamic DLL was compiled on the fly by the executable with C++ compiler, then loaded, then ran a for loop inserting new classes into the sequence container inside the EXE. Also it has rather complex exchage of information back and forth. After that the dll was freed and deleted from hard disk. DLL was serving as a scripting language for the EXE. That bug never manifested itself until we added virtual functions to the shared classes.
So can we see the sample project you mentioned?
-
Correct. :-D Even though the address of the class itself lays in the global process heap, the vtable address lays withing the DLL address space. Call to the FreeLibrary deletes all vtable entries of any class allocated inside the DLL. Every time you go to use any virtual function including virtual destructor (via "delete"), the program crashes. Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested. If you allocate same class in EXE it will have completely different vtable addresses because EXE has its own copy too. For example you could have just but that duplicate class code into EXE and the DLL to achive the same result.By changing static library configuration into DLL solves that problem because neither EXE nor factory DLL has any copies of the class. Classes allocated this way will all have vtable adreesses lay in that third DLL. If that DLL has lifetime of EXE the problem goes away. A bit about the project where I witnessed that bug. Description I gave was over simplified. That dynamic DLL was compiled on the fly by the executable with C++ compiler, then loaded, then ran a for loop inserting new classes into the sequence container inside the EXE. Also it has rather complex exchage of information back and forth. After that the dll was freed and deleted from hard disk. DLL was serving as a scripting language for the EXE. That bug never manifested itself until we added virtual functions to the shared classes.
steveb wrote:
withing the DLL address space
Is this terminology correct? In my definition, DLLs don't have address spaces, processes do. The DLL is loaded into the address space for the process.
steveb wrote:
Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested.
This sounds more and more like C runtimes to me. Are you certain the DLL and executable are using *exactly* the same CRT? Specifically, which CRT(s) are they using?
-
steveb wrote:
withing the DLL address space
Is this terminology correct? In my definition, DLLs don't have address spaces, processes do. The DLL is loaded into the address space for the process.
steveb wrote:
Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested.
This sounds more and more like C runtimes to me. Are you certain the DLL and executable are using *exactly* the same CRT? Specifically, which CRT(s) are they using?
DLLs do have its own address space which starts at the base of the DLLs HINSTANCE and upward. Usually 0x1000000 or any other base address returned by the LoadLibrary. Open the memory window in debugger and paste HINSTANCE value returned by LoadLibrary into it-There is your DLL code. That section is nuked after FreeLibrary called. Vtable is oddly located in that space instead of the process heap.
-
So can we see the sample project you mentioned?
-
DLLs do have its own address space which starts at the base of the DLLs HINSTANCE and upward. Usually 0x1000000 or any other base address returned by the LoadLibrary. Open the memory window in debugger and paste HINSTANCE value returned by LoadLibrary into it-There is your DLL code. That section is nuked after FreeLibrary called. Vtable is oddly located in that space instead of the process heap.
To quote the gospel according to St Redmond[^], The LoadLibrary Function loads the specified module into the address space of the calling process.
-
To quote the gospel according to St Redmond[^], The LoadLibrary Function loads the specified module into the address space of the calling process.
Thats is exactly right. LoadLibrary is loaded into the calling process. In case of exe this space lays between 0x00400000 and INT_MAX. Well not exactly. INT_MAX/2 to be pecise because space above INT_MAX/2 reserved for OS DLLs. default value for DLL would be 0x10000000 + DLL size. This area is nuked with FreeLibrary call. And this is not the point at all. Point is that the VTable address of the class created inside this dll is in this area even for the heap objects created with new operator :omg:
modified on Wednesday, July 23, 2008 10:16 AM
-
DLLs do have its own address space which starts at the base of the DLLs HINSTANCE and upward. Usually 0x1000000 or any other base address returned by the LoadLibrary. Open the memory window in debugger and paste HINSTANCE value returned by LoadLibrary into it-There is your DLL code. That section is nuked after FreeLibrary called. Vtable is oddly located in that space instead of the process heap.
steveb wrote:
Vtable is oddly located in that space instead of the process heap.
Not odd at all, the vtable contains addresses that map to addresses within the DLL. Even if the vtable itself were located in the global heap, the program would still fail when it tried to jump to the addresses which are now gone.
Anyone who thinks he has a better idea of what's good for people than people do is a swine. - P.J. O'Rourke
-
steveb wrote:
withing the DLL address space
Is this terminology correct? In my definition, DLLs don't have address spaces, processes do. The DLL is loaded into the address space for the process.
steveb wrote:
Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested.
This sounds more and more like C runtimes to me. Are you certain the DLL and executable are using *exactly* the same CRT? Specifically, which CRT(s) are they using?
Graham Bradshaw wrote:
Is this terminology correct? In my definition, DLLs don't have address spaces, processes do.
DLLs use the address space of the loading process. However, the portion of memory which the DLL itself occupies can be called its address space. In this case, the vtable resolves to an address in the memory space occupied by the DLL, which is why the vtable is not located in the global heap, but in the private heap maintained by the DLL. When the DLL is unloaded, the vtable is nuked because it's invalid anyway. (Even if linking to a static library, you must use the DLL version of the CRT IF you intend to allocate memory in the DLL and deallocate it elsewhere. I personally think this is a terrible idea from a design standpoint. In the future, I may make the object use a fixed memory allocator or otherwise customize how it allocates, and by having the allocating module do the unallocating, it prevents problems.)
Anyone who thinks he has a better idea of what's good for people than people do is a swine. - P.J. O'Rourke
-
Ok. C++ project that consist out of the 3 subprojects: Executable, DLL, and Static library. Dependencies as follows: Executable linked to Static library, DLL linked to static library. Static library contains set of C++ classes which are used in DLL and the EXE. DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". The DLL is loaded on the fly with LoadLibrary and then unloaded with FreeLibrary API (this is done to conserve memory). Classes in the static lib usually contain arbitrary set of virtual functions ( very usual stuff: Dtors, overrides etc). Here we go: 1. Executable loads DLL into memory with LoadLibrary. 2. Executable calls function in DLL say "GetClassA" which in turn looks like this for simplicity:
CMyClass* GetClassA()
{
return new CMyClass;
}3. Executable receives pointer to CMyClass. 4. Executable frees DLL with FreeLibrary. 5. Application at some point goes on to use CMyClass pointer and dies miserable death with Access violation. What has happened? Answer later.
modified on Tuesday, July 22, 2008 11:19 PM
-
The executable and DLL are using different C runtimes, which means you can't pass pointers (among other things) across the executable/DLL boundary. See http://msdn.microsoft.com/en-us/library/ms235460(VS.80).aspx[^] for a full explanation.
-
The executable and DLL are using different C runtimes, which means you can't pass pointers (among other things) across the executable/DLL boundary. See http://msdn.microsoft.com/en-us/library/ms235460(VS.80).aspx[^] for a full explanation.
Graham Bradshaw wrote:
you can't pass pointers (among other things) across the executable/DLL boundary.
What? Sure you can. It's just that if you have different heaps between the modules, you must delete where you've newed, and free where you've malloced.
-- Kein Mitleid Für Die Mehrheit
-
Graham Bradshaw wrote:
you can't pass pointers (among other things) across the executable/DLL boundary.
What? Sure you can. It's just that if you have different heaps between the modules, you must delete where you've newed, and free where you've malloced.
-- Kein Mitleid Für Die Mehrheit
Jörgen Sigvardsson wrote:
It's just that if you have different heaps between the modules, you must delete where you've newed, and free where you've malloced
Yes, I know. And that was the point of the original post. The OP said: DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". Which is my point. The memory was allocated in the DLL and freed in the executable, which you can't do if the DLL and executable use different run-times. Which part of my response do you not agree with?
-
Jörgen Sigvardsson wrote:
It's just that if you have different heaps between the modules, you must delete where you've newed, and free where you've malloced
Yes, I know. And that was the point of the original post. The OP said: DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". Which is my point. The memory was allocated in the DLL and freed in the executable, which you can't do if the DLL and executable use different run-times. Which part of my response do you not agree with?
Graham Bradshaw wrote:
Which part of my response do you not agree with?
This part: which means you can't pass pointers (among other things) across the executable/DLL boundary.