Memory allocation related query
-
The remaining memory is reserved for the system and cannot be accessed from your program.
Ovidiu Cucu Microsoft MVP - Visual C++
But I even tried increasing the PF size to 6.5 GB from 4 GB. Still my program was giving 'memory full' error after using approximately 2.8 GB of PF.
-
I have written one small program in Visual Studio 6.0 which just goes on allocating the memory using 'new' operator. My thinking was that it will give 'memory full' error once all the physical memory and page file memory is full. So I executed the program nad start observing the Task Manager. As the program is executing, the task manager was showing reduced available size of the physical memory. Once the physical memory is full, it start showing higher PF Usage. When my program game me the 'Memory full' error, the PF usage was just 2.8 GB. My PF size is 4 GB. So still 1.2 GB was unused. Why should the program give 'Memory full' error when there is still 1.2 GB available. My physical memory (RAM) is 512 MB. Any idea why is it so? Following is the program. int main(int argc, char* argv[]) { double dMemAllocated=0; char *c[64003]; char *s[64003]; int i=0; while (1) { try { c[i] = new char[40*1024]; s[i] = new char[40*1024]; if(i > 64000) break; if((c[i] == NULL) || (s[i] == NULL)) { cout << "Memory full. Memory Allocated = " << dMemAllocated; getchar(); } cout << i++ << "\n"; } catch(...) { cout << "Memory Allocated = " << dMemAllocated; getchar(); } dMemAllocated += 80*1024; } cout << "Memory Allocated = " << dMemAllocated; getchar(); return 0; }
-
There is a limititation because there is a VC-Project (compiler/linker) setting to limit the working set to 2 GB. Which can be changed, but suddenly I dont remember.:confused:
Greetings from Germany
If it can be changed, then it is great. Can you please let me know when you remember it? Is it achieved by changing the compiler option? Or changes required are in registry etc?
-
But I even tried increasing the PF size to 6.5 GB from 4 GB. Still my program was giving 'memory full' error after using approximately 2.8 GB of PF.
You can increase it to 200GB or more... the virtual memory size for a Win32 process is 4GB.
Ovidiu Cucu Microsoft MVP - Visual C++
-
If it can be changed, then it is great. Can you please let me know when you remember it? Is it achieved by changing the compiler option? Or changes required are in registry etc?
Well, there is the /LARGEADDRESSAWARE option. But I've never actually tried it :~ See http://msdn2.microsoft.com/en-us/library/wz223b1z(vs.80).aspx You can also start reading this: http://blogs.msdn.com/oldnewthing/archive/2004/08/12/213468.aspx – I'm too lazy to read it myself :zzz: Why do you need that much memory, anyway? And why not switch to 64-bit if you really need it?
Florin Crişan
-
Well, there is the /LARGEADDRESSAWARE option. But I've never actually tried it :~ See http://msdn2.microsoft.com/en-us/library/wz223b1z(vs.80).aspx You can also start reading this: http://blogs.msdn.com/oldnewthing/archive/2004/08/12/213468.aspx – I'm too lazy to read it myself :zzz: Why do you need that much memory, anyway? And why not switch to 64-bit if you really need it?
Florin Crişan
Salut Florin, :) As stated in MSDN "The /LARGEADDRESSAWARE option tells the linker that the application can handle addresses larger than 2 gigabytes" but AFAIK you cannot pass beyond how much memory a process can handle, i.e. 4GB for Win32(see my first answer).
Ovidiu Cucu Microsoft MVP - Visual C++
Cofounder CODEXPERT.RO -
Salut Florin, :) As stated in MSDN "The /LARGEADDRESSAWARE option tells the linker that the application can handle addresses larger than 2 gigabytes" but AFAIK you cannot pass beyond how much memory a process can handle, i.e. 4GB for Win32(see my first answer).
Ovidiu Cucu Microsoft MVP - Visual C++
Cofounder CODEXPERT.ROI guess I need another cup of coffee :)
Florin Crişan
-
I guess I need another cup of coffee :)
Florin Crişan
Me too! ;)
Ovidiu Cucu Microsoft MVP - Visual C++ Cofounder CODEXPERT.RO
-
Well, there is the /LARGEADDRESSAWARE option. But I've never actually tried it :~ See http://msdn2.microsoft.com/en-us/library/wz223b1z(vs.80).aspx You can also start reading this: http://blogs.msdn.com/oldnewthing/archive/2004/08/12/213468.aspx – I'm too lazy to read it myself :zzz: Why do you need that much memory, anyway? And why not switch to 64-bit if you really need it?
Florin Crişan
I think only Vista (Amongst microsoft OS) is 64 bit OS. And I heard that it has problems. So I am try to avoid it. So much memory is required becuase it is multiuser application and each user can fetch upto big set of data in memory for processing. So even if one user takes 20 MB, then 100 simultaneous users can occupy 2 GB.
-
I think only Vista (Amongst microsoft OS) is 64 bit OS. And I heard that it has problems. So I am try to avoid it. So much memory is required becuase it is multiuser application and each user can fetch upto big set of data in memory for processing. So even if one user takes 20 MB, then 100 simultaneous users can occupy 2 GB.
Actually, there are 64-bit editions of XP Professional, Server 2003 and Vista. Anyway, you may want to consider processing the data in smaller chunks (you may not need to load whole files in the memory). Allocating more than the physical memory will seriously degrade performance (virtual memory is much slower). You might also want to try distributing the workload to several machines ;-) But, of course, this is just generic advice – I don't really know what your application does :)
Florin Crişan
-
Actually, there are 64-bit editions of XP Professional, Server 2003 and Vista. Anyway, you may want to consider processing the data in smaller chunks (you may not need to load whole files in the memory). Allocating more than the physical memory will seriously degrade performance (virtual memory is much slower). You might also want to try distributing the workload to several machines ;-) But, of course, this is just generic advice – I don't really know what your application does :)
Florin Crişan
... and by files I mean data, even if it comes from a database :)
Florin Crişan
-
Actually, there are 64-bit editions of XP Professional, Server 2003 and Vista. Anyway, you may want to consider processing the data in smaller chunks (you may not need to load whole files in the memory). Allocating more than the physical memory will seriously degrade performance (virtual memory is much slower). You might also want to try distributing the workload to several machines ;-) But, of course, this is just generic advice – I don't really know what your application does :)
Florin Crişan
Yes. I have to explore distributing the workload on multiple machines using web farming if the 2 GB limitation could not be overcome. However I am trying to avoid it as it will add up the cost.
-
I have written one small program in Visual Studio 6.0 which just goes on allocating the memory using 'new' operator. My thinking was that it will give 'memory full' error once all the physical memory and page file memory is full. So I executed the program nad start observing the Task Manager. As the program is executing, the task manager was showing reduced available size of the physical memory. Once the physical memory is full, it start showing higher PF Usage. When my program game me the 'Memory full' error, the PF usage was just 2.8 GB. My PF size is 4 GB. So still 1.2 GB was unused. Why should the program give 'Memory full' error when there is still 1.2 GB available. My physical memory (RAM) is 512 MB. Any idea why is it so? Following is the program. int main(int argc, char* argv[]) { double dMemAllocated=0; char *c[64003]; char *s[64003]; int i=0; while (1) { try { c[i] = new char[40*1024]; s[i] = new char[40*1024]; if(i > 64000) break; if((c[i] == NULL) || (s[i] == NULL)) { cout << "Memory full. Memory Allocated = " << dMemAllocated; getchar(); } cout << i++ << "\n"; } catch(...) { cout << "Memory Allocated = " << dMemAllocated; getchar(); } dMemAllocated += 80*1024; } cout << "Memory Allocated = " << dMemAllocated; getchar(); return 0; }
Hi, all objects are allocated in a single address space, which on a 32-bit operating system cannot extend beyond 4GB. Actual systems such as Win XP do impose further restrictions; XP normally deals with 2GB of address space, you can optionally increase that to 3GB. You want all objects get the number of bytes (hence address range) that their sizes indicate, and you want all objects non-overlapping, so once their sizes add up to more than 2 or 3GB game is over. The size of physical memory plays no role in this; whatever spills over goes into a disk file where the non-physical memory is temporarily stored; making that file larger than 2 or 3GB does not make much sense: all addresses (32-bit pointers) must still be unique and the objects non-overlapping. On a 64-bit operating system, all pointers occupy twice the amount of space, but the available address space is theoretically squared, in practice much much larger than you will ever need. :)
Luc Pattyn [Forum Guidelines] [My Articles]
this months tips: - before you ask a question here, search CodeProject, then Google - the quality and detail of your question reflects on the effectiveness of the help you are likely to get - use PRE tags to preserve formatting when showing multi-line code snippets