Memory allocation / GC issues???

Dave Kreskowiak

Xpnctoc wrote:

Using Task Manager to monitor my memory use,

Stop right there. You're looking in the wrong place. Managed code (.NET CLR) apps run inside a virtual machine, just like a Java app. What Task Manager is showing you is the RAM reserved for the entire virtual machine your app is running in, not just your app. So, even though your app isn't using that memory, it's still being held by the virtual machine for future use. That's why the memory doesn't get freed.

Xpnctoc wrote:

Is my Garbage Collector broken?

Nope. It's just being lazy. It will hang on to that memory, so long as Windows doesn't want it back. When Windows starts to get low on memory, it'll ask the virtual machine to release whatever it can back to Windows.

Xpnctoc wrote:

Should I somehow manually invoke garbage collection to clear some memory?

This won't solve the "problem". This will tell the GC to free up any managed memory that hasn't already been freed. But, this memory goes back into the Managed Heap (reserved for future use), NOT back to Windows.

Xpnctoc wrote:

Does anyone know if the GC works on a time interval or a certain percentage of consumed memory, or what?

The GC is extremely good at it's job and runs whenever it detects the need to. There is no set schedule and it tunes it's own performance based on how it sees your app running and what it expects your app to do in the future.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007

Xpnctoc

So in other words, I'm paranoid. The application I'm writing requires a potentially unlimited amount of user data. Well of course this is impossible because a computer only has a finite amount of memory. But what I've been worried about is that someday I'll be using the app and I'll hit memory capacity because the GC didn't reallocate 100MB+ that was used up in temporary dialog boxes in menus. But if it's just because the GC is lazy and doesn't kick in until absolutely necessary, then I guess I should be OK.

Xpnctoc

Thanks for the insight, Dave. I have never heard that .NET was actually running on a virtual machine. I knew there was JIT and such involved, but that puts things into a little different perspective. And like I told Luc, I think I'm coming to the realization that I'm just paranoid because of the extreme data memory requirement of the app I'm writing. But I'd even take it one step farther and suggest to myself that I'm also only working in the theoretical, because it would take a long time using the app to develop the need for so much memory. By then we'll probably be talking terabytes of memory on standard computer models anyway :-)

Dave Kreskowiak

Well, that doesn't exactly guarantee you won't run out of memory. You can still cause leaks, like resources and handles. For example, with all these dialogs, are you showing them by calling .ShowDialog()?? If so, are you then calling .Dispose() on those dialogs when you're done with the data in them?? If not, you'll eventually run the system out of resources and cause an OutOfMemory exception.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007

Luc Pattyn

Hi, there are good reasons why the gc is lazy, and should not be called explicitly: 1. it is an expensive operation, since it needs to check all the data in your app, figuring out which objects still have references to them (are "alive") and which don't. There can be a lot of data involved, it does not fit in the cache, and it trashes the cache (i.e. whatever was rightfully in there now is gone). 2. while doing its job, the gc suspends all your threads, to make sure the stacks and objects don't change while it is examining them; so your app is temporarily dead. :)

Luc Pattyn [Forum Guidelines] [My Articles]

this weeks tips: - make Visual display line numbers: Tools/Options/TextEditor/... - show exceptions with ToString() to see all information - before you ask a question here, search CodeProject, then Google

Luc Pattyn

You may want to display (and periodically update) the value of Environment.WorkingSet inside your app; that allows you to see memory usage by your process.

Luc Pattyn [Forum Guidelines] [My Articles]

this weeks tips: - make Visual display line numbers: Tools/Options/TextEditor/... - show exceptions with ToString() to see all information - before you ask a question here, search CodeProject, then Google

realJSOP

A side note - don't allocate more than about 85k of memory to any single object unless you're prepare to handle cleanup manually. Anything bigger than that gets shoved onto the "large object heap", and the GC doesn't clean that up for you.

"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001

Xpnctoc

Thanks for the tip. I wasn't aware of that one. I'm sure that will help.

Xpnctoc

Got that covered. All of my core logic I'm writing in unmanaged code. I found early on that I didn't like the overhead of some of the System.Collections.Generic classes -- especially the bulky disk usage in serialization compared to doing a straight binary write of unmanaged data structures. So I chose to write the app in C++/CLI, using .NET for the ease of creating Windows forms and controls while keeping my core logic and processing in native C++. So I'm fully prepared to do my unmanaged allocation and cleanup. Like I said, the behavior I'm calling into question occurs with things I have little or no control over, such as Menu controls and dialogs.

Luc Pattyn

Hi John, I don't fully agree, AFAIK objects on the large object heap are managed like all other managed objects, with a single difference: the LOH does not get reshuffled, hence there is a risk of fragmentation. Example: allocate 1MB, then 100MB, repeat this N times until memory is almost full, then throw away the references to all 100MB objects and try to get one really big object of 300MB. IMO the threshold (85KB?) is not documented, is not available through some API, may vary over time, and may depend on your system's memory situation. I feel the Framework should be more specific about this, it should guarantee a reasonable minimum value for the threshold, so one can make sure fragmentation will not become an issue by keeping objects smaller than that. :)

Luc Pattyn [Forum Guidelines] [My Articles]

this weeks tips: - make Visual display line numbers: Tools/Options/TextEditor/... - show exceptions with ToString() to see all information - before you ask a question here, search CodeProject, then Google

Lutoslaw

Hey, I though that a code

openFileDialog.ShowDialog();
openFileDialog.Dispose();
openFileDialog.ShowDialog();

throws "object disposed" excpt or sth like that, doesn't it? So if I use this dialog all the time...

Greetings - Gajatko

Luc Pattyn

of course you only dispose of things you don't need any longer.

Luc Pattyn [Forum Guidelines] [My Articles]

this weeks tips: - make Visual display line numbers: Tools/Options/TextEditor/... - show exceptions with ToString() to see all information - before you ask a question here, search CodeProject, then Google

Scott Dorman

Luc Pattyn wrote:

the threshold (85KB?) is not documented, is not available through some API, may vary over time, and may depend on your system's memory situation.

In general this is true, the 85KB limit isn't publicly documented. I did find it in Rotor[^] (SSCLI) in the gc.h header file for the garbage collector's native code:

/* misc defines */
#define LARGE_OBJECT_SIZE 85000

.

Scott.

—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]

Dave Kreskowiak

gajatko wrote:

So if I use this dialog all the time...

You create a new instance of it...

A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007

Vaibhav Sharma

Hi, I am amazed that after reading all the replies and posts on this article no one mentions of the Using...End Using code block. I am sure you must be knowing it but just a little bit of detailing on the using statements: 'VB Version, C# no different, just lang. changes Using dlgOpen As New OpenFileDialog() dlgOpen.Filter="*.*" 'Any other properties dlg.ShowDialog() End Using Now what the above code fragment does is that the CLR will create the dlgOpen object and will be responsible for its disposal, not YOU. This has serious benefits in terms of Performance & Memory Utilization since as soon as the code execution reaches End Using the dlgOpen will be Disposed off without requiring you to do any thing at all. :::One Catch Though While using the Using statements::: the dlgOpen will not be accessible outside the Using block since its scope is limited is to the Using...End Using block. So i strongly recommend you to go ahead and use the Using...End Using blocks for all the DialogBoxes since most of them times they return only one value which can easily be stored on a local/global variable. Awaiting reply eagerly.

Regards, Vaibhav Sharma

Xpnctoc

I appreciate the tip. I am aware of the "using" syntax, but in my case that won't work. As I mentioned in my response to John Simmons's post above, my application is written in C++/CLI, which has no such construct. There were multiple reasons for that language choice, some of which I mentioned and others that are irrelevant to this conversation. And even if I was in C#/VB, I'd still have the same problem with the Menu control. Can't use "using" for something like a menu that has to be present for the duration of execution. Regardless, I think it was one of Luc Pattyn's responses that directed me to try examining memory usage through the Environment.WorkingSet (thank's again, Luc). This provides memory use much closer to what I would expect based on my interaction with the application. I still find it alarming that the .NET virtual machine continues to eat up additional memory when it's not needed, but there I just have to trust that the GC will do its job when push comes to shove.