Ominous bug from DLL world [modified]
-
Because the vtable is referencing methods in the DLL address space.
Anyone who thinks he has a better idea of what's good for people than people do is a swine. - P.J. O'Rourke
Yup, that's my bet, too. He unloaded the code that he wants to call.
We are a big screwed up dysfunctional psychotic happy family - some more screwed up, others more happy, but everybody's psychotic joint venture definition of CP
blog: TDD - the Aha! | Linkify!| FoldWithUs! | sighist -
Which one? (Though this is relevant only for Grahams suggestions, not what I think it is...)
We are a big screwed up dysfunctional psychotic happy family - some more screwed up, others more happy, but everybody's psychotic joint venture definition of CP
blog: TDD - the Aha! | Linkify!| FoldWithUs! | sighist -
Because the vtable is referencing methods in the DLL address space.
Anyone who thinks he has a better idea of what's good for people than people do is a swine. - P.J. O'Rourke
Correct. :-D Even though the address of the class itself lays in the global process heap, the vtable address lays withing the DLL address space. Call to the FreeLibrary deletes all vtable entries of any class allocated inside the DLL. Every time you go to use any virtual function including virtual destructor (via "delete"), the program crashes. Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested. If you allocate same class in EXE it will have completely different vtable addresses because EXE has its own copy too. For example you could have just but that duplicate class code into EXE and the DLL to achive the same result.By changing static library configuration into DLL solves that problem because neither EXE nor factory DLL has any copies of the class. Classes allocated this way will all have vtable adreesses lay in that third DLL. If that DLL has lifetime of EXE the problem goes away. A bit about the project where I witnessed that bug. Description I gave was over simplified. That dynamic DLL was compiled on the fly by the executable with C++ compiler, then loaded, then ran a for loop inserting new classes into the sequence container inside the EXE. Also it has rather complex exchage of information back and forth. After that the dll was freed and deleted from hard disk. DLL was serving as a scripting language for the EXE. That bug never manifested itself until we added virtual functions to the shared classes.
-
Correct. :-D Even though the address of the class itself lays in the global process heap, the vtable address lays withing the DLL address space. Call to the FreeLibrary deletes all vtable entries of any class allocated inside the DLL. Every time you go to use any virtual function including virtual destructor (via "delete"), the program crashes. Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested. If you allocate same class in EXE it will have completely different vtable addresses because EXE has its own copy too. For example you could have just but that duplicate class code into EXE and the DLL to achive the same result.By changing static library configuration into DLL solves that problem because neither EXE nor factory DLL has any copies of the class. Classes allocated this way will all have vtable adreesses lay in that third DLL. If that DLL has lifetime of EXE the problem goes away. A bit about the project where I witnessed that bug. Description I gave was over simplified. That dynamic DLL was compiled on the fly by the executable with C++ compiler, then loaded, then ran a for loop inserting new classes into the sequence container inside the EXE. Also it has rather complex exchage of information back and forth. After that the dll was freed and deleted from hard disk. DLL was serving as a scripting language for the EXE. That bug never manifested itself until we added virtual functions to the shared classes.
So can we see the sample project you mentioned?
-
Correct. :-D Even though the address of the class itself lays in the global process heap, the vtable address lays withing the DLL address space. Call to the FreeLibrary deletes all vtable entries of any class allocated inside the DLL. Every time you go to use any virtual function including virtual destructor (via "delete"), the program crashes. Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested. If you allocate same class in EXE it will have completely different vtable addresses because EXE has its own copy too. For example you could have just but that duplicate class code into EXE and the DLL to achive the same result.By changing static library configuration into DLL solves that problem because neither EXE nor factory DLL has any copies of the class. Classes allocated this way will all have vtable adreesses lay in that third DLL. If that DLL has lifetime of EXE the problem goes away. A bit about the project where I witnessed that bug. Description I gave was over simplified. That dynamic DLL was compiled on the fly by the executable with C++ compiler, then loaded, then ran a for loop inserting new classes into the sequence container inside the EXE. Also it has rather complex exchage of information back and forth. After that the dll was freed and deleted from hard disk. DLL was serving as a scripting language for the EXE. That bug never manifested itself until we added virtual functions to the shared classes.
steveb wrote:
withing the DLL address space
Is this terminology correct? In my definition, DLLs don't have address spaces, processes do. The DLL is loaded into the address space for the process.
steveb wrote:
Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested.
This sounds more and more like C runtimes to me. Are you certain the DLL and executable are using *exactly* the same CRT? Specifically, which CRT(s) are they using?
-
steveb wrote:
withing the DLL address space
Is this terminology correct? In my definition, DLLs don't have address spaces, processes do. The DLL is loaded into the address space for the process.
steveb wrote:
Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested.
This sounds more and more like C runtimes to me. Are you certain the DLL and executable are using *exactly* the same CRT? Specifically, which CRT(s) are they using?
DLLs do have its own address space which starts at the base of the DLLs HINSTANCE and upward. Usually 0x1000000 or any other base address returned by the LoadLibrary. Open the memory window in debugger and paste HINSTANCE value returned by LoadLibrary into it-There is your DLL code. That section is nuked after FreeLibrary called. Vtable is oddly located in that space instead of the process heap.
-
So can we see the sample project you mentioned?
-
DLLs do have its own address space which starts at the base of the DLLs HINSTANCE and upward. Usually 0x1000000 or any other base address returned by the LoadLibrary. Open the memory window in debugger and paste HINSTANCE value returned by LoadLibrary into it-There is your DLL code. That section is nuked after FreeLibrary called. Vtable is oddly located in that space instead of the process heap.
To quote the gospel according to St Redmond[^], The LoadLibrary Function loads the specified module into the address space of the calling process.
-
To quote the gospel according to St Redmond[^], The LoadLibrary Function loads the specified module into the address space of the calling process.
Thats is exactly right. LoadLibrary is loaded into the calling process. In case of exe this space lays between 0x00400000 and INT_MAX. Well not exactly. INT_MAX/2 to be pecise because space above INT_MAX/2 reserved for OS DLLs. default value for DLL would be 0x10000000 + DLL size. This area is nuked with FreeLibrary call. And this is not the point at all. Point is that the VTable address of the class created inside this dll is in this area even for the heap objects created with new operator :omg:
modified on Wednesday, July 23, 2008 10:16 AM
-
DLLs do have its own address space which starts at the base of the DLLs HINSTANCE and upward. Usually 0x1000000 or any other base address returned by the LoadLibrary. Open the memory window in debugger and paste HINSTANCE value returned by LoadLibrary into it-There is your DLL code. That section is nuked after FreeLibrary called. Vtable is oddly located in that space instead of the process heap.
steveb wrote:
Vtable is oddly located in that space instead of the process heap.
Not odd at all, the vtable contains addresses that map to addresses within the DLL. Even if the vtable itself were located in the global heap, the program would still fail when it tried to jump to the addresses which are now gone.
Anyone who thinks he has a better idea of what's good for people than people do is a swine. - P.J. O'Rourke
-
steveb wrote:
withing the DLL address space
Is this terminology correct? In my definition, DLLs don't have address spaces, processes do. The DLL is loaded into the address space for the process.
steveb wrote:
Vtable addresses located in the DLL space because DLL is linked to static library and has its own copy of the class requested.
This sounds more and more like C runtimes to me. Are you certain the DLL and executable are using *exactly* the same CRT? Specifically, which CRT(s) are they using?
Graham Bradshaw wrote:
Is this terminology correct? In my definition, DLLs don't have address spaces, processes do.
DLLs use the address space of the loading process. However, the portion of memory which the DLL itself occupies can be called its address space. In this case, the vtable resolves to an address in the memory space occupied by the DLL, which is why the vtable is not located in the global heap, but in the private heap maintained by the DLL. When the DLL is unloaded, the vtable is nuked because it's invalid anyway. (Even if linking to a static library, you must use the DLL version of the CRT IF you intend to allocate memory in the DLL and deallocate it elsewhere. I personally think this is a terrible idea from a design standpoint. In the future, I may make the object use a fixed memory allocator or otherwise customize how it allocates, and by having the allocating module do the unallocating, it prevents problems.)
Anyone who thinks he has a better idea of what's good for people than people do is a swine. - P.J. O'Rourke
-
Ok. C++ project that consist out of the 3 subprojects: Executable, DLL, and Static library. Dependencies as follows: Executable linked to Static library, DLL linked to static library. Static library contains set of C++ classes which are used in DLL and the EXE. DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". The DLL is loaded on the fly with LoadLibrary and then unloaded with FreeLibrary API (this is done to conserve memory). Classes in the static lib usually contain arbitrary set of virtual functions ( very usual stuff: Dtors, overrides etc). Here we go: 1. Executable loads DLL into memory with LoadLibrary. 2. Executable calls function in DLL say "GetClassA" which in turn looks like this for simplicity:
CMyClass* GetClassA()
{
return new CMyClass;
}3. Executable receives pointer to CMyClass. 4. Executable frees DLL with FreeLibrary. 5. Application at some point goes on to use CMyClass pointer and dies miserable death with Access violation. What has happened? Answer later.
modified on Tuesday, July 22, 2008 11:19 PM
-
The executable and DLL are using different C runtimes, which means you can't pass pointers (among other things) across the executable/DLL boundary. See http://msdn.microsoft.com/en-us/library/ms235460(VS.80).aspx[^] for a full explanation.
-
The executable and DLL are using different C runtimes, which means you can't pass pointers (among other things) across the executable/DLL boundary. See http://msdn.microsoft.com/en-us/library/ms235460(VS.80).aspx[^] for a full explanation.
Graham Bradshaw wrote:
you can't pass pointers (among other things) across the executable/DLL boundary.
What? Sure you can. It's just that if you have different heaps between the modules, you must delete where you've newed, and free where you've malloced.
-- Kein Mitleid Für Die Mehrheit
-
Graham Bradshaw wrote:
you can't pass pointers (among other things) across the executable/DLL boundary.
What? Sure you can. It's just that if you have different heaps between the modules, you must delete where you've newed, and free where you've malloced.
-- Kein Mitleid Für Die Mehrheit
Jörgen Sigvardsson wrote:
It's just that if you have different heaps between the modules, you must delete where you've newed, and free where you've malloced
Yes, I know. And that was the point of the original post. The OP said: DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". Which is my point. The memory was allocated in the DLL and freed in the executable, which you can't do if the DLL and executable use different run-times. Which part of my response do you not agree with?
-
Jörgen Sigvardsson wrote:
It's just that if you have different heaps between the modules, you must delete where you've newed, and free where you've malloced
Yes, I know. And that was the point of the original post. The OP said: DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". Which is my point. The memory was allocated in the DLL and freed in the executable, which you can't do if the DLL and executable use different run-times. Which part of my response do you not agree with?
Graham Bradshaw wrote:
Which part of my response do you not agree with?
This part: which means you can't pass pointers (among other things) across the executable/DLL boundary.
-
Graham Bradshaw wrote:
Which part of my response do you not agree with?
This part: which means you can't pass pointers (among other things) across the executable/DLL boundary.
And there is, of course, an implied "in the case which you are discussing" in there. You allocate in the DLL, pass the pointer across, delete in the executable, you're in trouble. Technically, yes, its posssible. "You can't do it" doesn't mean it's physically impossible. It means you won't get the results you expect.
-
And there is, of course, an implied "in the case which you are discussing" in there. You allocate in the DLL, pass the pointer across, delete in the executable, you're in trouble. Technically, yes, its posssible. "You can't do it" doesn't mean it's physically impossible. It means you won't get the results you expect.
If you do not use virtual functions inside the classes which are created inside the dll (which is later unloaded on the fly) then everything works fine. Please note that this bug manifetst itself only when the factory DLL that created the class is destroyed while the class still lives. If the factory DLL has the span of the executable, then everything works fine. I think that is why ATL explicitly declares all classes with ATL_NO_VTABLE (I could be wrong).
-
Ok. C++ project that consist out of the 3 subprojects: Executable, DLL, and Static library. Dependencies as follows: Executable linked to Static library, DLL linked to static library. Static library contains set of C++ classes which are used in DLL and the EXE. DLL acts as some sort of Factory for the classes by allocating them with "new" and returning to executable. Executable uses them throughout app and deallocates them with the "delete". The DLL is loaded on the fly with LoadLibrary and then unloaded with FreeLibrary API (this is done to conserve memory). Classes in the static lib usually contain arbitrary set of virtual functions ( very usual stuff: Dtors, overrides etc). Here we go: 1. Executable loads DLL into memory with LoadLibrary. 2. Executable calls function in DLL say "GetClassA" which in turn looks like this for simplicity:
CMyClass* GetClassA()
{
return new CMyClass;
}3. Executable receives pointer to CMyClass. 4. Executable frees DLL with FreeLibrary. 5. Application at some point goes on to use CMyClass pointer and dies miserable death with Access violation. What has happened? Answer later.
modified on Tuesday, July 22, 2008 11:19 PM
The allocation of the CMyClass is performed by the the DLL yes? Well then the class instance is allocated in the DLLs heap space. Once the DLL is unloaded then that heap space is no longer available. Another senario that falls under this problem is the following (which is similar by not the same as the static lib will use the heap space of the calling exe or dll): -Exe links against DLL A and DLL B. -Exe calls function in DLL A which in turn constructs instance of class C defined in DLL B on the heap and returns it to Exe. -DLL A is unloaded but DLL B is not The same effect will be caused in this case as the new operator (that assigns the memory) allocated it in the address space of DLL A. This can be overcome by adding a static create function to class C, which just calls "return new C()" in it's implementation. It is also possible to do this via overloading the new operator in the class but in a much more convoluted way. So going back to your issue, it basically depends how you want to fix it... - Keep DLL in existance - or turn your static lib (which I'm assuming is where classes are defined) into a DLL and add static create functions [Edit - I must admit that I didn't see the other posts concerning the vtable links due to the static library. This is probably also an issue. I'm pretty certain that the workarounds above will also work]
-
Graham Bradshaw wrote:
you can't pass pointers (among other things) across the executable/DLL boundary.
What? Sure you can. It's just that if you have different heaps between the modules, you must delete where you've newed, and free where you've malloced.
-- Kein Mitleid Für Die Mehrheit
Ok, Maybe you can tell me what is wrong with this. When I run this, it dies with memory corruption in the find statement in the Execute function. I pass the pointer to keep the map from being copied (since it might get huge in a real project) To me there is no obvious reason why. // The DLL #include #include using namespace std; typedef map<string,string> param_t; __declspec(dllexport) bool Execute(param_t *params){ params->find("xyz"); return true; } // The EXE #include #include using namespace std; typedef map<string,string> param_t; __declspec(dllimport) bool Execute(param_t *params); int main() { param_t params; params["xyz"]="abc"; params.find("xyz"); // test it here before we call the other function return Execute(¶ms); } I was using Visual C++ 6.0. Compiling it with any other compiler seems to work fine (including newer versions of Visual C++)