Is there a way to handle OutOfMemory exceptions?
-
Hello, I am working on a project that requires me to generate long reports that can have 150 pages or more - Each of these pages are generated as an A4 image (~3500x2480pixels), which is then inserted into an ArrayList... The problem however is that I'm repeatedly getting OutOfMemory exceptions that I seem to be unable to get rid of despite extensively clearing all objects i'm not using (using the Image.Dispose() method for example) as well as forcing the garbage collector (GC.Collect()) every now and then... I'm constructing each of these images in the 1bpp pixel format that would mean that each of these images would be ~1MB on average... For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)?? Any one has any ideas on how I can get rid of, or even handle, these OutOfMemory exceptions?? Thanks, Sarab
-
Hello, I am working on a project that requires me to generate long reports that can have 150 pages or more - Each of these pages are generated as an A4 image (~3500x2480pixels), which is then inserted into an ArrayList... The problem however is that I'm repeatedly getting OutOfMemory exceptions that I seem to be unable to get rid of despite extensively clearing all objects i'm not using (using the Image.Dispose() method for example) as well as forcing the garbage collector (GC.Collect()) every now and then... I'm constructing each of these images in the 1bpp pixel format that would mean that each of these images would be ~1MB on average... For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)?? Any one has any ideas on how I can get rid of, or even handle, these OutOfMemory exceptions?? Thanks, Sarab
How are you using the report when it's finished, as you create the pages as images? Do they really have to be images? Do you really have to create all images at the same time? Is there no way of handling the data of the report in some other way, and generate each image when you need it? You could convert each image into GIF or PNG format, that would substantially reduce the size without losing any information. --- b { font-weight: normal; }
-
Hello, I am working on a project that requires me to generate long reports that can have 150 pages or more - Each of these pages are generated as an A4 image (~3500x2480pixels), which is then inserted into an ArrayList... The problem however is that I'm repeatedly getting OutOfMemory exceptions that I seem to be unable to get rid of despite extensively clearing all objects i'm not using (using the Image.Dispose() method for example) as well as forcing the garbage collector (GC.Collect()) every now and then... I'm constructing each of these images in the 1bpp pixel format that would mean that each of these images would be ~1MB on average... For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)?? Any one has any ideas on how I can get rid of, or even handle, these OutOfMemory exceptions?? Thanks, Sarab
-
Hello, I am working on a project that requires me to generate long reports that can have 150 pages or more - Each of these pages are generated as an A4 image (~3500x2480pixels), which is then inserted into an ArrayList... The problem however is that I'm repeatedly getting OutOfMemory exceptions that I seem to be unable to get rid of despite extensively clearing all objects i'm not using (using the Image.Dispose() method for example) as well as forcing the garbage collector (GC.Collect()) every now and then... I'm constructing each of these images in the 1bpp pixel format that would mean that each of these images would be ~1MB on average... For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)?? Any one has any ideas on how I can get rid of, or even handle, these OutOfMemory exceptions?? Thanks, Sarab
Heritos is right. I would save the images to and only hold a reference to them.
sarabjs wrote:
For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)??
No. Because .Net will use much more memory, as the code is managed and everything is encapsulate in Objects. I tried it with a 3D-scene of 100 MB and old C++ only needs a little bit more than 100 MB for the whole programm while .Net used more than 200 MB. Another Point is the Garbage Collection, which isn't a real one. If you generate an object and drop it afterwards its hold in the memory quiet a time. So there are objects which aren't used anymore. Greetings, Ingo
-
Heritos is right. I would save the images to and only hold a reference to them.
sarabjs wrote:
For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)??
No. Because .Net will use much more memory, as the code is managed and everything is encapsulate in Objects. I tried it with a 3D-scene of 100 MB and old C++ only needs a little bit more than 100 MB for the whole programm while .Net used more than 200 MB. Another Point is the Garbage Collection, which isn't a real one. If you generate an object and drop it afterwards its hold in the memory quiet a time. So there are objects which aren't used anymore. Greetings, Ingo
ihoecken wrote:
Another Point is the Garbage Collection, which isn't a real one. If you generate an object and drop it afterwards its hold in the memory quiet a time. So there are objects which aren't used anymore.
That's not an accurate statement, you appear to be laboring under one of two incorrect assumptions. .net's garbage collector is generational which means that it attempts to collect younger objects far more frequently than older ones. If memory is needed it will however collect the older generations. Secondly when an object is GCed the memory is returned to the application stack/heap, not to the operating system. Standard c/c++ runtimes behave the same way because requesting/returning memory to the OS is an expensive operation. Consequently the task manager doesn't display the actual ammount of ram the program is using. This is more apparent in .net since it's runtime requests a larger default chunk size than standard C++ ones. Excess memory is returned if the OS asks for it (ie appB requests more than the OS has available so the OS asks other running apps if they can return thier excess), minimizing the app will also trigger this release under .net. IIRC it will eventually do this on it's own if it concludes it asked for more than it'll ever likely need. If the systems apps are using more total ram than is physically available, the excess will be parked on the swap file because of inactivity so while it may make the task manager numbers look scary it doesn't have a real effect on performance.
-
ihoecken wrote:
Another Point is the Garbage Collection, which isn't a real one. If you generate an object and drop it afterwards its hold in the memory quiet a time. So there are objects which aren't used anymore.
That's not an accurate statement, you appear to be laboring under one of two incorrect assumptions. .net's garbage collector is generational which means that it attempts to collect younger objects far more frequently than older ones. If memory is needed it will however collect the older generations. Secondly when an object is GCed the memory is returned to the application stack/heap, not to the operating system. Standard c/c++ runtimes behave the same way because requesting/returning memory to the OS is an expensive operation. Consequently the task manager doesn't display the actual ammount of ram the program is using. This is more apparent in .net since it's runtime requests a larger default chunk size than standard C++ ones. Excess memory is returned if the OS asks for it (ie appB requests more than the OS has available so the OS asks other running apps if they can return thier excess), minimizing the app will also trigger this release under .net. IIRC it will eventually do this on it's own if it concludes it asked for more than it'll ever likely need. If the systems apps are using more total ram than is physically available, the excess will be parked on the swap file because of inactivity so while it may make the task manager numbers look scary it doesn't have a real effect on performance.
A good garbage collection begins to release unused memory when there is time, .Net Garbage collections releases memory when memory is needed. So its much slower and sometimes such memory isn't released properly, even if the object isn't used any more. Greetings, Ingo
-
A good garbage collection begins to release unused memory when there is time, .Net Garbage collections releases memory when memory is needed. So its much slower and sometimes such memory isn't released properly, even if the object isn't used any more. Greetings, Ingo
The .NET GC is self-tuning. It runs on both a semi-regular (self-adusted) schedule and on-demand when either the applications calls for it or the operating system does and wants memory returned. Memory is never released "improperly" as you call it. The GC tries, and does a very good job, at keeping a resonably sized pool of memory available to the application for future allocations. Like I said, the GC is self-tuning and can only improve it's allocation strategy by over-estimating what it needs to start with then watching how your application allocates and releases memory. But, yes, the GC is a bit lazy when application activity is low, and for good reason. In order for the GC to run, the threads of your application must be stopped before the collection takes place. This is because objects in memory can be moved by the GC to compact and defrag the heaps and pointers to those objects must be updated to reflect their new locations. This cannot be done while the code is still running! So, yeah, the GC tries to be as lazy as possible, giving your app more time to execute. Garbage Collection and object allocation, when dealing with the Managed Heap, goes VERY, VERY fast. It slows down considerably if memory has to be allocated from the system first, adding it to the Managed Heap. BTW, this is all documented on MSDN: Programming for Garbage Collection[^] Garbage Collection: Automatic Memory Management in the Microsoft .NET Framework[^] Garbage Collection—Part 2: Automatic Memory Management in the Microsoft .NET Framework[^] Improving .NET Application Performance and Scalability[
-
If it were me, I'd try just saving each bitmap as it's created, and then just keeping an arraylist of filenames (files in a temporary location), accessing individual files as needed.
Just tried this... I write each image to disk after it is created, dispose it and force the garbage collector after every 20 images... No success however; I keep the Windows Task Manager on during processing, and the page file size still continues to gradually increase till it hits the limit, and once again I get hit with OutofMemory exceptions, which only lead to more when handled... Interestingly however, the moment the control comes out of the routine - the page file size immediately drops back to normal...
-
Hello, I am working on a project that requires me to generate long reports that can have 150 pages or more - Each of these pages are generated as an A4 image (~3500x2480pixels), which is then inserted into an ArrayList... The problem however is that I'm repeatedly getting OutOfMemory exceptions that I seem to be unable to get rid of despite extensively clearing all objects i'm not using (using the Image.Dispose() method for example) as well as forcing the garbage collector (GC.Collect()) every now and then... I'm constructing each of these images in the 1bpp pixel format that would mean that each of these images would be ~1MB on average... For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)?? Any one has any ideas on how I can get rid of, or even handle, these OutOfMemory exceptions?? Thanks, Sarab
Hi Sarab, I think that your problem is regarding the number of handles you create!!! If you know, Windows restricts each process in creating handles. For example, it is 10,000 handles for XP. But my experience says that this equation is that straight forward!!! It is also dependent on the memory the process occupies and the number of GDI objects being created by your app. To confirm this, try seeing the number of GDI objects and Handles being created by your app through the taskmanager->View->Select Columns... And if the number of Handles is going beyond 500 and GDI objects more than 300 or 400, then i think that you have located the problem!!! Even i had a similar problem, but not yet found the solution as such!!! The work around which i did was not storing such a huge data in memory which creates lots of GDI objects!!!! Though im able to get control over it, i feel that this is not the correct solution to it!!! Regards, Rajesh
-
Hello, I am working on a project that requires me to generate long reports that can have 150 pages or more - Each of these pages are generated as an A4 image (~3500x2480pixels), which is then inserted into an ArrayList... The problem however is that I'm repeatedly getting OutOfMemory exceptions that I seem to be unable to get rid of despite extensively clearing all objects i'm not using (using the Image.Dispose() method for example) as well as forcing the garbage collector (GC.Collect()) every now and then... I'm constructing each of these images in the 1bpp pixel format that would mean that each of these images would be ~1MB on average... For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)?? Any one has any ideas on how I can get rid of, or even handle, these OutOfMemory exceptions?? Thanks, Sarab
Thanks folks!! I appreciate everyone's help... Turns out, I didn't have to cross the Seven Seas: I managed to solve the issue by explicitly disposing (Image.Dispose()) some images I was creating during processing... I used to think that .NET automatically clears the memory being held by an image object once all references to it are removed - Guess that isn't the case... Learnt a lot from these discussions though - Thanks! -- modified at 16:03 Friday 3rd February, 2006