small, slow memory leak
-
a typical GC works its way through the maze of objects, starting at the ones it knows are alive: references found on the stacks (and in CPU registers) and the static objects; then it looks for references inside those objects, recursively. Those are the only ones that could ever be accessed again. Two objects that reference each other but are floating in space, detached from stacks and statics, cannot be accessed, you don't have a reference to access them. Here is a little test you might want to run to convince yourself empirically:
public class Floater {
private Floater other;
private static long count;public Floater() { count++; if(count&0xFFFFFF)==0) Console.WriteLine("Already created "+count+" Floaters"); } public static void Link(Floater f1, Floater f2) { f1.other=f2; f2.other=f1; }}
public class Test {
public static void Test() {
for(;;) Floater.Link(new Floater(), new Floater());
}
}Now run Test.Test() and wonder why it never runs out of memory. :)
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
Thanks. Looks like we fixed bugs by actually getting some things wrong :)
"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverb -
Interesting. Tomorrow at work I will get an unfixed version of a leaky application out of source control and try to find out what actually caused the leak. Obviously wrong reasoning brought me to the right answer back then.
"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverbSetting some references to null inside an object you will no longer need isn't wrong of course; and if you somehow are keeping that object alive inadvertently, it would reduce the damage. That is probably why it happened to solve your problem. :)
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
-
Setting some references to null inside an object you will no longer need isn't wrong of course; and if you somehow are keeping that object alive inadvertently, it would reduce the damage. That is probably why it happened to solve your problem. :)
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
One application was simply requesting memory faster than the GC was releasing it. According to ProcessExplorer a thorough implementation of Dispose() and clearing all references corrected this. The other one, which I want to look at again tomorrow, is far trickier. Someone crated controls on an ASP .Net page dynamically and stored them in the session state in order to easily restore them after postbacks. Bad idea. Memory consumption went up with every page request. Ok, the controls, the SessionState object and the page itself formed loops. But where was the anchor that kept them alive? After finishing its lifecycle, the page should be released by the server and the whole construct should be floating. Edit: Reading my own post, the prime suspect right now would be the SessionState object. The server might actually keep it until the session expires and reuse it at every new request.
"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverb -
One application was simply requesting memory faster than the GC was releasing it. According to ProcessExplorer a thorough implementation of Dispose() and clearing all references corrected this. The other one, which I want to look at again tomorrow, is far trickier. Someone crated controls on an ASP .Net page dynamically and stored them in the session state in order to easily restore them after postbacks. Bad idea. Memory consumption went up with every page request. Ok, the controls, the SessionState object and the page itself formed loops. But where was the anchor that kept them alive? After finishing its lifecycle, the page should be released by the server and the whole construct should be floating. Edit: Reading my own post, the prime suspect right now would be the SessionState object. The server might actually keep it until the session expires and reuse it at every new request.
"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverbCDP1802 wrote:
requesting memory faster than the GC was releasing it
without finalizers, I would not expect that as most of the GC runs on the thread that needs the memory; finalizers however get handled by a separate (and low-priority) thread, which indeed can be an issue.
CDP1802 wrote:
the controls, the SessionState object and the page itself formed loops
I don't know how that got implemented exactly, and yes I would assume the SessionState is kept in say a static collection (a Dictionary?) so it persists until it has been decided the session is over. :)
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
-
is the most annoying bug to address. I think I may lose my mind today. I've never written an application that suffered from one, and I've written numerous applications similar to this current one. I fear it may be due to my interaction with a legacy set of libraries.
Visual Studio Profiler cannot do the job for you? I mean, to track where the memory is used.
-
I worked on an application that puts together huge download files from a database. Even the delay caused by the garbage collection may be too much. First, look that you implement (and use) Dispose() on all relevant large data objects. Without having to wait for garbage collection the real behavior of your application gets much clearer to see. And if there really is a leak, then look out for objects which hold references to each other or in a circle. Those would survive garbage collection, so it would be better to set all references to other objects to null whuile disposing them. We had a tricky situation in an ASP .Net app, where some fool stored controls in the session state. The page held references to the controls, the controls held references to the page. They were never released by the garbage collection and memory got fuller with every page request.
"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverb"We had a tricky situation in an ASP .Net app, where some fool stored controls in the session state." Do you work for the SQL Server SSRS team? They store everything they can possibly come up with session state (or the cache depending on how you configure it). 1.2GB in 2min? No problemo.
He who asks a question is a fool for five minutes. He who does not ask a question remains a fool forever. [Chineese Proverb] Jonathan C Dickinson (C# Software Engineer)
-
"We had a tricky situation in an ASP .Net app, where some fool stored controls in the session state." Do you work for the SQL Server SSRS team? They store everything they can possibly come up with session state (or the cache depending on how you configure it). 1.2GB in 2min? No problemo.
He who asks a question is a fool for five minutes. He who does not ask a question remains a fool forever. [Chineese Proverb] Jonathan C Dickinson (C# Software Engineer)
Not that I know of, but perhaps they made me drunk and dragged me aboard last night :) At least sailors where once recruited that way.
"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverb -
is the most annoying bug to address. I think I may lose my mind today. I've never written an application that suffered from one, and I've written numerous applications similar to this current one. I fear it may be due to my interaction with a legacy set of libraries.
Hi, I've got no idea what could cause your C# memory leak. But you may want to have a look at this blog entry considering the dangers of the large object heap. It is about how allocating alternate big lumps of memory and smaller lumps then freeing the bigger lump to allocating a slightly bigger lump repeatedly may lead to memory loss. http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/[^] It took me a long time to find this error and as it happens when adding stuff to lists in a quite simple way it could happen to you! Good luck. Erik
-
is the most annoying bug to address. I think I may lose my mind today. I've never written an application that suffered from one, and I've written numerous applications similar to this current one. I fear it may be due to my interaction with a legacy set of libraries.
I hate those. A couple years ago I had a GDI resource leak (caused by a leaked MFC object) that only showed up after my application had been running a week or more. It took months to find that one.
Software Zen:
delete this; -
I hate those. A couple years ago I had a GDI resource leak (caused by a leaked MFC object) that only showed up after my application had been running a week or more. It took months to find that one.
Software Zen:
delete this;I'd say resource leaks are one kind of the two most likely causes of memory leaks in managed code. The other would be using static collections. But resource held by any object would be released by finaliser after GC - so unless one uses a buggy library, it can only be a result of using unmanaged code without implementing proper finaliser. So I would start by searching for "static" in the code.
-
I'd say resource leaks are one kind of the two most likely causes of memory leaks in managed code. The other would be using static collections. But resource held by any object would be released by finaliser after GC - so unless one uses a buggy library, it can only be a result of using unmanaged code without implementing proper finaliser. So I would start by searching for "static" in the code.
In this case, it was a native-mode application (no GC, no finalizer, etc.). One of our test technicians found a sequence of actions that made the problem happen more likely to occur: in a couple of hours, rather than several days. That eventually let me track it down and fix it.
Software Zen:
delete this; -
is the most annoying bug to address. I think I may lose my mind today. I've never written an application that suffered from one, and I've written numerous applications similar to this current one. I fear it may be due to my interaction with a legacy set of libraries.
You can track down the memory leak using the Signal Flare debugging pattern: Increase the size of your objects (one at a time) 50-fold, and rerun your program. When the memory leak jumps 50-fold, you've found your problem.
-
is the most annoying bug to address. I think I may lose my mind today. I've never written an application that suffered from one, and I've written numerous applications similar to this current one. I fear it may be due to my interaction with a legacy set of libraries.
Hello, This might not be applicable in your particular scenario, and there are performance issues that should be accounted for (starting an exe is relatively slow), but I have dealt with leaky third-party libraries I can't circumvent by wrapping their use on a different executable file. Instead of directly linking the library to my application, I execute the wrapper process and pass the parameters on the command line. The thing is that *all* resources are freed once the wrapper process dies. You could improve performance by creating a service-like app (probably using WCF) that is restarted once a memory usage threshold is exceeded. This might be a little harder to code but avoids the performance penalty incurred when a process starts. This should only be considered as a last option. Hope this might help, and good luck! :cool: Sergio
-
is the most annoying bug to address. I think I may lose my mind today. I've never written an application that suffered from one, and I've written numerous applications similar to this current one. I fear it may be due to my interaction with a legacy set of libraries.
"small and slow" are definitely annoying. Sometimes the big ones can be fun, though. I have a side-project which consists of many parts. They all centre around this GDI based display (like a custom powerpoint) and originally I planned to draw to a buffer (Bitmap) from a separate thread and the display would read from that buffer. For performance, there were two buffers and they swapped. Unfortunately I forgot about the whole de-allocation thing and on a 1024x768 display it leaked memory at 50MB/s.
Ninja (the Nerd)
Confused? You will be... -
That is no fun. Do you have to fully restart, or can you clean it up, sort of reinitialize? I have had coworkers do similar things to get things by and into beta. Restarting weekly. (in my industry nothing happens on Saturdays, ever) I also just realized this issue only happens on the production server facing the production database. Running it on my corporate dev box, against the dev database produces level memory. It is subscribed to the exact same feed of live data. The prod server is running Windows Server SP1 and my desktop is running Windows 7. I've also seen an allusive error on the prod box that has never happened on my corporate box running the same program simultaneously.
-
is the most annoying bug to address. I think I may lose my mind today. I've never written an application that suffered from one, and I've written numerous applications similar to this current one. I fear it may be due to my interaction with a legacy set of libraries.
As well as the suggestions from others, look out for objects (including controls etc) referencing each other via events (connecting to an event handler is normally a strong reference unless you deliberately code it as a weak one). This is particularly bad for events on static objects (some of these exist in the Windows Forms libraries for example, for detecting session/colour depth/resolutions changes etc).