small, slow memory leak

Ian Shlasko

Nah, the leak is in a Java library accessed via RCW... I don't want to put it into its own application domain, as that'd kill my performance... So the only way to release the memory is to restart the process. Granted, I already have an infrastructure in place to do just that (I can restart the service remotely, by entering a password into the client app)... I just hate relying on the old "Have you tried restarting?" tech support fix.

Proud to have finally moved to the A-Ark. Which one are you in?
Author of the Guardians Saga (Sci-Fi/Fantasy novels)

CPallini

wizardzz wrote:

I fear it may be due to my interaction with a legacy set of libraries.

Yes, the legacy libraries do leak memory while attempting to escape from your code... ;P

If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler. -- Alfonso the Wise, 13th Century King of Castile.
This is going on my arrogant assumptions. You may have a superb reason why I'm completely wrong. -- Iain Clarke
[My articles]

Luc Pattyn

CDP1802 wrote:

look out for objects which hold references to each other or in a circle. Those would survive garbage collection

Not correct. They would all survive if at least one of them is really alive, otherwise the cluster dies as it should. :)

Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum

Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.

Lost User

Why? Each member of the circle is still being referenced by another member.

"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverb

Luc Pattyn

a typical GC works its way through the maze of objects, starting at the ones it knows are alive: references found on the stacks (and in CPU registers) and the static objects; then it looks for references inside those objects, recursively. Those are the only ones that could ever be accessed again. Two objects that reference each other but are floating in space, detached from stacks and statics, cannot be accessed, you don't have a reference to access them. Here is a little test you might want to run to convince yourself empirically:

public class Floater {
private Floater other;
private static long count;

public Floater() {
    count++;
    if(count&0xFFFFFF)==0) Console.WriteLine("Already created "+count+" Floaters");
}

public static void Link(Floater f1, Floater f2) {
    f1.other=f2;
    f2.other=f1;
}

}

public class Test {
public static void Test() {
for(;;) Floater.Link(new Floater(), new Floater());
}
}

Now run Test.Test() and wonder why it never runs out of memory. :)

Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum

Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.

Andy Brummer

The circle has to be reachable from the stack or a root object. If .net used reference counting then you could have that issue.

Curvature of the Mind now with 3D

Lost User

Interesting. Tomorrow at work I will get an unfixed version of a leaky application out of source control and try to find out what actually caused the leak. Obviously wrong reasoning brought me to the right answer back then.

"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverb

Lost User

Thanks. Looks like we fixed bugs by actually getting some things wrong :)

"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverb

Luc Pattyn

Setting some references to null inside an object you will no longer need isn't wrong of course; and if you somehow are keeping that object alive inadvertently, it would reduce the damage. That is probably why it happened to solve your problem. :)

Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum

Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.

Lost User

One application was simply requesting memory faster than the GC was releasing it. According to ProcessExplorer a thorough implementation of Dispose() and clearing all references corrected this. The other one, which I want to look at again tomorrow, is far trickier. Someone crated controls on an ASP .Net page dynamically and stored them in the session state in order to easily restore them after postbacks. Bad idea. Memory consumption went up with every page request. Ok, the controls, the SessionState object and the page itself formed loops. But where was the anchor that kept them alive? After finishing its lifecycle, the page should be released by the server and the whole construct should be floating. Edit: Reading my own post, the prime suspect right now would be the SessionState object. The server might actually keep it until the session expires and reuse it at every new request.

"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverb

Luc Pattyn

CDP1802 wrote:

requesting memory faster than the GC was releasing it

without finalizers, I would not expect that as most of the GC runs on the thread that needs the memory; finalizers however get handled by a separate (and low-priority) thread, which indeed can be an issue.

CDP1802 wrote:

the controls, the SessionState object and the page itself formed loops

I don't know how that got implemented exactly, and yes I would assume the SessionState is kept in say a static collection (a Dictionary?) so it persists until it has been decided the session is over. :)

Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum

Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.

tec goblin

Visual Studio Profiler cannot do the job for you? I mean, to track where the memory is used.

Jonathan C Dickinson

"We had a tricky situation in an ASP .Net app, where some fool stored controls in the session state." Do you work for the SQL Server SSRS team? They store everything they can possibly come up with session state (or the cache depending on how you configure it). 1.2GB in 2min? No problemo.

He who asks a question is a fool for five minutes. He who does not ask a question remains a fool forever. [Chineese Proverb] Jonathan C Dickinson (C# Software Engineer)

Lost User

Not that I know of, but perhaps they made me drunk and dragged me aboard last night :) At least sailors where once recruited that way.

"I have what could be described as the most wide-open sense of humor on the site, and if I don't think something is funny, then it really isn't." - JSOC, 2011 -----
"Friar Modest never was a prior" - Italian proverb

Erik Rude

Hi, I've got no idea what could cause your C# memory leak. But you may want to have a look at this blog entry considering the dangers of the large object heap. It is about how allocating alternate big lumps of memory and smaller lumps then freeing the bigger lump to allocating a slightly bigger lump repeatedly may lead to memory loss. http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/[^] It took me a long time to find this error and as it happens when adding stuff to lists in a quite simple way it could happen to you! Good luck. Erik

Gary Wheeler

I hate those. A couple years ago I had a GDI resource leak (caused by a leaked MFC object) that only showed up after my application had been running a week or more. It took months to find that one.

Software Zen: delete this;

User 4223959

I'd say resource leaks are one kind of the two most likely causes of memory leaks in managed code. The other would be using static collections. But resource held by any object would be released by finaliser after GC - so unless one uses a buggy library, it can only be a result of using unmanaged code without implementing proper finaliser. So I would start by searching for "static" in the code.

Gary Wheeler

In this case, it was a native-mode application (no GC, no finalizer, etc.). One of our test technicians found a sequence of actions that made the problem happen more likely to occur: in a couple of hours, rather than several days. That eventually let me track it down and fix it.

Software Zen: delete this;

Alan Balkany

You can track down the memory leak using the Signal Flare debugging pattern: Increase the size of your objects (one at a time) 50-fold, and rerun your program. When the memory leak jumps 50-fold, you've found your problem.

sgorozco

Hello, This might not be applicable in your particular scenario, and there are performance issues that should be accounted for (starting an exe is relatively slow), but I have dealt with leaky third-party libraries I can't circumvent by wrapping their use on a different executable file. Instead of directly linking the library to my application, I execute the wrapper process and pass the parameters on the command line. The thing is that *all* resources are freed once the wrapper process dies. You could improve performance by creating a service-like app (probably using WCF) that is restarted once a memory usage threshold is exceeded. This might be a little harder to code but avoids the performance penalty incurred when a process starts. This should only be considered as a last option. Hope this might help, and good luck! :cool: Sergio