How to avoid thrid party lib(no source codes) to allocate memory from physical memory?
-
I'm not sure what this has to do with OP's question. He asked how to modify a DLL without the source code.
Use the best guess
If you're dynamically creating the objects in your app, you can overload the 'new' operator. But that's not really going to do the trick. As others have mentioned data has to be in memory to be used.
-
Thank you for your advice. Yes, maybe I can do some cleanup work.
That's the best place to start. Make sure you have no memory leaks. If there is some reason that you need gigabytes of data in memory to do some heavy complex processing (perhaps DNA analysis, simulations etc...,) then upgrade your hardware and go 64 bit.
-
If you're dynamically creating the objects in your app, you can overload the 'new' operator. But that's not really going to do the trick. As others have mentioned data has to be in memory to be used.
-
I don't think you meant to post this to me, or (maybe) even to this thread.
Use the best guess
I meant it for the thread anyway...
-
Falconapollo wrote:
My problem is how to avoid thrid party lib to allocate memory from physical memory.
No, that is not your problem - that is a suggestion on how to solve your (as of yet unknown) problem. And it won't work for several reasons: 1. The operating system will do all memory allocations; most importantly even the most basic function you can call to allocate memory will allocate that within physical memory. There's no way to avoid that short of writing your own OS. 2. The OS will also move memory from and to hard disk as needed in order to keep the entire system running smoothly. Repeatedly. At runtime! Again, you cannot influence that short of rewriting the OS. 3. Technically, no application or library actually allocates or processes data in physical memory; instead they work on virtual memory. Under the hood, the OS will put the actual data anywhere into physical memory or on hard disk, but neither your application nor that library would even be able to tell where their data are currently residing, because from their point of view all data always resides in virtual memory. There is no way (and no sensible reason) to change that. Well, short of rewriting the OS ;) So, please start again, and start by describing your problem, not by suggesting a solution. Are you running out of memory? Are you experiencing performance issues? Are you suffering from memory fragmentation? Any of these could be a problem, and there may be reasonable solutions. But either solution will not solve all of these problems, so knowing the problem would help.
Stefan_Lang wrote:
2. The OS will also move memory from and to hard disk as needed in order to keep the entire system running smoothly. Repeatedly. At runtime! Again, you cannot influence that short of rewriting the OS
I am rather certain that all desktop OSes provide access to some very raw memory management APIs. Without that there are whole classifications of applications that could not be written. As an example of one API for windows, I presume there are others, the following insures that memory is not written to the hard drive. http://msdn.microsoft.com/en-us/library/aa366895%28VS.85%29.aspx[^]
-
Suppose this scenario: I refer to a third party lib in my C++ app, but I don't want the third party lib to use my physical memory at all. Instead, I want it to only allocate memory from hard disk. I don't know source codes of third party lib, however as it run in the Windows platform, so I think it's possible to control the memory management with Win32 API. My problem is how to avoid thrid party lib to allocate memory from physical memory. Am I going in the wrong direction? Anybody can help me? PS: I'm using Visual C++ 2010.
If your problem is that your application fails because over time this library keeps consuming memory then the solution is to remove the library from your application. You can do that as follows. 1. Create an executable that wraps the library. 2. Provide a communication idiom: files, stdio or sockets. That provides the necessary API for your business needs (basically a proxy for what you are already doing.) 3. In your actual application use WinProcess (or whatever the method is called) to execute 1 and manage it. 4. Use the communication from 2 from your application in place of the straight to library calls you are currently using. Other variations on the above depend on your needs - Create multiple instance - Stop each instance after X (1 or more) uses - Have min/max instances and use a locked queue to send requests for processing - Detect failures, restart and retry a request. - Detect excessive memory usage, terminate and restart.
-
Stefan_Lang wrote:
2. The OS will also move memory from and to hard disk as needed in order to keep the entire system running smoothly. Repeatedly. At runtime! Again, you cannot influence that short of rewriting the OS
I am rather certain that all desktop OSes provide access to some very raw memory management APIs. Without that there are whole classifications of applications that could not be written. As an example of one API for windows, I presume there are others, the following insures that memory is not written to the hard drive. http://msdn.microsoft.com/en-us/library/aa366895%28VS.85%29.aspx[^]
Quote:
I am rather certain that all desktop OSes provide access to some very raw memory management APIs.
yes, i can't agree more. But I don't find what i need.
-
If your problem is that your application fails because over time this library keeps consuming memory then the solution is to remove the library from your application. You can do that as follows. 1. Create an executable that wraps the library. 2. Provide a communication idiom: files, stdio or sockets. That provides the necessary API for your business needs (basically a proxy for what you are already doing.) 3. In your actual application use WinProcess (or whatever the method is called) to execute 1 and manage it. 4. Use the communication from 2 from your application in place of the straight to library calls you are currently using. Other variations on the above depend on your needs - Create multiple instance - Stop each instance after X (1 or more) uses - Have min/max instances and use a locked queue to send requests for processing - Detect failures, restart and retry a request. - Detect excessive memory usage, terminate and restart.
Thank you for the specific suggestions. i will have a try.
-
What is the problem that you are trying to solve?
i'm trying to solve "out of memory" problem, it's the no-source third party lib who eat up my memory. but i'm rather my usage is right, and i have to do this way. so, i'm trying to find a way to make the third patry lib occupy less memory.
-
i'm trying to solve "out of memory" problem, it's the no-source third party lib who eat up my memory. but i'm rather my usage is right, and i have to do this way. so, i'm trying to find a way to make the third patry lib occupy less memory.
If the leak is due to a bug in a third party library then there may be no workaround to your issue. The operating system already caches out memory to a page file on disk. If you're exceeding the 1 or 3gig limit set by the linker, then you're hitting the wall on addressable memory. You can't allocate more memory than you can address. With 32 bit pointers the theoretical limit is 2^32 bytes or 4gig. In practical terms it's less because some of the bits in the upper end of the pointer can have a special meaning. I've seen server side systems live with leaks like this, by periodically restarting. If you can prove that the third party library is leaking the memory with a simple test case, then I would raise this issue with your vendor.