Luc, I've been running ANTS Memory Profiler 8.8 on the service... the Large Object Heap size is actually only about 40MB according to this profiler. It shows the "Private Bytes" and "Working Set - Private" as the one that has all the memory. When I took a snapshot this is what it is saying: -> Generation 1 0 bytes -> Generation 2 -> 1.105MB -> Large Object Heap -> 2.645MB -> Unused memory allocated to .NET -> 108.6MB -> Unmanaged -> 618.6MB It also shows this for class list (Live size (bytes)): -> ConditionalWeakTable+Entry<Object, PSMemberInfoInternalCollection> (8,073,936 bytes) -> Int32[] (2,726,312 bytes) -> string (337,040) -> AdsValueHelper (151,956) and it just goes down from there. It does show "string" has 4,782 live instances and "AdsValueHelper" has 4,221 live instances

Just wanted to update you that i've tried a couple different things... The first thing i've tried is completely disabling all Exchange actions (message tracking logs, mailbox sizes, etc). Now it basically just processes the Active Directory options which has a total of 4533 that can be put in a list. What I am finding is the memory usage is still up to 1GB now even with all the tasks disabled and growing. I've had this service working without memory issues in the past. I completely rewrote it changing from Entity Framework to Linq to SQL because I didn't want to worry about the "context" being different. My goal was to make it where the scheduler version could last multiple version of the primary application. I'm really starting to wonder if its Linq to SQL because nothing should be over 5000 in a list now after disabling those other options. I may try switching to using SqlConnection and SqlCommand for a test (BTW I updated my code if you want to check it out again at the current state)

I have no issues moving it to x64 except it seems like it would use way more memory than it should even if I switched it right? I have shortened the intervals (just a xml file) to every 30 minutes... i'm querying all of this from Exchange so I don't want to run it too often. I may need to just create some fake data and seed it into a list and bypass Exchange for testing. This is a Windows Service... The restart would technically help but I don't like that idea lol I'm testing your ListOfLists class right now.. worse case I could combine my powershell command sql insert statements so i'm not passing around Lists. However the powershell command does return a ICollection of PSObjects so most likely I guess we would end up in the same situation?

Hi Luc! I think after reading your questions/answers and researching that I have a much better understanding of what is going on. Basically when a List has more than 10,000 objects in it then it goes into LOH. Now from what I understand the garbage collector doesn't normally "compact" these as it expects to reuse this space? Now I read in 4.5.1 they introduced this: whi[^] which gives you the ability to tell it TO compact once and then it resets to default which doesn't compact LOH. Now what you posted creates a List of Lists (as named) and keeps each list under 10,000 so it never enters LOH. I'm testing your class right now and recompiling.. Edit: (Sorry forgot to answer your questions): Questions: a) how many users satisfy x.MailboxPlan > 0 Right now it is under 5000 but technically it could be well over 10,000 one day b) what would be a good upper limit for the Count of this allMailboxes? Same thing.. right now under 5000 but could be over 10,000 one day c) how many users would there typically be in the below a.Users (sent or received) This is how many messages a single user has sent. Most of the time it will be under 1000 but if someone spams it could be larger. The list of ALL users for sent messages is well over 10,000. I can put in some logs to get the exact data

Ok it is running with that new code is currently at 677MB. Just a few minutes ago it went up to 1.5GB and back down... however it is slowly climbing

Ok code is updated and running now. I changes the triggers that were every 60 minutes to every 30 minutes to hopefully reproduce quicker. Updated code is here: KnowMoreIT/CloudPanel-Service · GitHub[^]

Luc, Sorry for the delay. Yesterday I discovered another issue in the code. The Quartz library i'm using would let the job fire multiple times. So it didn't stop the timer while the job was executing and restart the timer after it was done. I discovered after that the memory leak still exists and i'm trying your suggestions now. HOWEVER the memory leak is MUCH slower now because the jobs are not occurring so often. From yesterday morning to now the memory has grown to 1.5GB

Luc, Thank you for assisting me with this. So I'm going to update the code and push it out tomorrow morning with the pre-allocated list sizes. Also just to let you know the process is at 386MB of memory right now. I know you said don't worry about the using statement but when I started this at 6pm I already put them in place. The code on github is still showing the try/catch/finally but I can update it with what I changed if you think it is necessary. Again, I really appreciate you helping!

Hey Luc! Glad to hear you like my coding style... its been completely self taught (with help from random people on the web) so never really knew LOL. 1) I am running 32-bit process. Memory is a virtual machine running 2008 R2 with 16GB of memory (4GB is used normally) 2) Here is the breakdown: Users: 4333 rows Message Tracking Logs: 1432284 rows Mailbox Sizes: 2209537 rows Database Sizes: 10520 History Stats table: 69806 rows 3) Get_MessageTrackingLogsTask normally would run every 720 minutes. To speed up the testing phase I changed it to every 60 minutes (along with the rest of the tasks) 4) The service starts out at 5MB of memory. From 6pm to 9pm tonight it has climbed to 314MB. I usually end up stopping the service around 2.8GB used. I've let it just go once and it crashed when it hit the max memory for 32-bit process) Here is intervals: GetDisabledUsers has ran 180 times (every 10 min) [Users table] GetLockedUsers has ran 180 times (Every 10 min) [Users table] GetMailboxSizes has ran 3 times (every 60 min) [Mailbox sizes table] GetMailboXDatabaseSizes has ran 6 times (every 30 min) [Database sizes table] GetMessageTrackingLogs has ran 3 times (every 60 min) [Message Tracking table] FindMissingData has ran 9 times (every 20 min) [Users table] UpdateDatabaseHistory has ran 3 times (every 60 min) [History Stats table] 5) I've only let it crash once so I don't know where it crashed. I don't see any errors and it continues to function fine till it runs out of memory. This does happen everytime I run it.. I've tried removing the db.Connection.Open() and currently (right now) i'm trying using the "using" statement. 6) I can make the memory climb by increasing the intervals. So one of my tasks is causing the issue but examining the code over and over I can't find where it possibly is doing it :-( 7) It probably would take about 2 days for it to fully climb and crash. I would setup something to capture a memory dump when it crashes but it would take a few days. One thing I can try is to increase the interval for all tasks to like 999999 minutes except for one (set at 10 min). Then I can slowly include each task until I see the memory problem. But right now it has gone from 5MB to 314MB in 3 hours. It should just be querying from Exchange/AD and dumbing the data to SQL and disposing of all of it... so technically the memory should go back down right?

So I thought it was looking promising but the memory maxed out again. I could see the memory going up and down but over time it just steadily cliimbed. I posted the code here: KnowMoreIT/CloudPanel-Service · GitHub[^]

Ok as a test I took out all of the following:

db.Connection.Open();

I kept my Try/Catch/Finally which calls Dispose on the DataContext for testing. Since I am not manually opening connections it should open and close after each query/update but it should handle the connections by itself. I'm going to keep an eye on memory today. If this doesn't work then I will try the USING statement. I still want to handle the connection because I don't want it opening and closing all the time since it causes a delay. Once I narrow down what it is I'll call Close() before disposing of the DataContext. I will let you all know once I have an update

ClousePanelDbContext is actually from DataContext since this is Linq to SQL and not actually Entity Framework. (System.Data.Linq.DataContext) As far as the using statement I see you are putting that in the try/catch so if what is INSIDE the using statement throws an exception will it be caught in the catch area so I can see what the exception was? It was my understand that the using statement wouldn't throw the error where you could catch it unless you put the try/catch INSIDE the using statement. But then again I could of understood that wrong.

I will look into the memory profiler. The powershell commands i'm using is the same code I'm using in a web application and haven't noticed any problems. I thought it was the timers but I think its more LINQ to SQL since I swapped out the timers for the Quartz library. The GC.Collect didn't actually help because when it ran I could see the memory go up and go down... however after a couple days i'm seeing i'm at 1.5GB now. So it is possible that 1 timer is causing all this.... the logging is log4net and it logs to a text file

Anytime I'm doing multiple queries I manually open the connection. From my understanding each query (without manually opening) is in essence doing this: -> Open Connection -> Query -> Close connection -> Open Connection -> Query -> Close connection So by me calling: db.Database.Connection.Open() it should be doing this: -> Open Connection -> Query -> Query -> Close connection

I'm disposing of "db" which should close the connection shouldn't it? I can try using but I usually steer away from it because I can't really catch exceptions without putting another try/catch inside the finally which is in effect making it a nested try/catch isn't it?

I'm having some issue with my code in my Windows Service not releasing memory. In short my Windows Service basically created about 8 timers that elapsed every X minutes which called a method that queried data from Exchange and stored in the database. What I have noticed is the memory always climbed to the point it would crash the service when it reached maximum (32-bit process) I switched from using timers to using the Quartz library but experienced the same results. Here is an example of one of the timers jobs:

public class Get_MailboxDatabaseSizesTask : IJob
{
private static readonly ILog logger = LogManager.GetLogger("Get_MailboxDatabaseSizesTask");

    public void Execute(IJobExecutionContext context)
    {
        CloudPanelDbContext db = null;
        ExchActions powershell = null;
        int processedCount = 0;

        try
        {
            db = new CloudPanelDbContext(Config.ServiceSettings.SqlConnectionString);
            db.Connection.Open();

            powershell = new ExchActions();
            var mailboxDatabases = powershell.Get\_MailboxDatabaseSizes();
            db.StatMailboxDatabaseSizes.InsertAllOnSubmit(mailboxDatabases);
            db.SubmitChanges();

            processedCount = mailboxDatabases.Count;
            mailboxDatabases = null;
        }
        catch (Exception ex)
        {
            logger.ErrorFormat("Failed to retrieve mailbox database sizes: {0}", ex.ToString());
        }
        finally
        {
            if (powershell != null)
                powershell.Dispose();

            if (db != null)
                db.Dispose();

            logger.InfoFormat("Processed a total of {0} mailbox databases", processedCount);
        }
        
    }
}

As you can see I am disposing of my context and my ExchActions class... I solved the memory issue by adding this to the end of each Execute method:

        GC.Collect();
        GC.WaitForPendingFinalizers();

Its not ideal but its working.... but why do I have such a memory problem without it? Where should I look?

Well it turns out the code actually does work. I'm not able to get it to work running it from my Server 2012 R2 lab server but from my 2008 R2 lab server it runs fine. I'll have to do some more investigating but at least I know the code is right! Thanks!

But thats the issue. I've compared it with numerous code examples and this should be working. Example: https://com2kid.wordpress.com/2011/09/22/remotely-executing-commands-in-powershell-using-c/[^] [^] The only thing I can think of is that this is a Server 2012 R2 box and wondering if there is something funky with the security. I haven't tried to a 2008 R2 box yet but plan on doing that today.

I've hit a road block and can't figure out why I am getting this response back. It appears that it connects to the server but it returns: "System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server server2.domain.local failed with the following error message: An invalid argument was supplied". I enabled everything I thought on the remote computer:

winrm set winrm/config/service/auth @{Basic="true"}
winrm set winrm/config/service @{AllowUnencrypted="true"}
winrm set winrm/config/client @{TrustedHosts="*"}

I'm trying to remote powershell so I can run Citrix powershell commands from another computer and here is what I have:

try
{
XenDesktop7 xd = new XenDesktop7("http://server2.domain.local:5985/wsman", @"DOMAIN\Administrator", "#######");
string[] blah = xd.GetCatalogs();

            foreach (var b in blah)
            {
                Console.WriteLine(b);
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.ToString());
        }

public CitrixPowershell(string uri, string username, string password)
{
this._connection = GetConnection(uri, username, password);
PSSnapInException snapinException;
_runspace = RunspaceFactory.CreateRunspace(_connection);
_runspace.Open();

        \_runspace.RunspaceConfiguration.AddPSSnapIn("Citrix.\*.Admin.V\*", out snapinException);

        \_powershell = PowerShell.Create();
        \_powershell.Runspace = \_runspace;
        
    }

    private WSManConnectionInfo GetConnection(string uri, string username, string password)
    {
        SecureString pwd = new SecureString();
        foreach (char x in password)
            pwd.AppendChar(x);

        PSCredential ps = new PSCredential(username, pwd);

        WSManConnectionInfo wsinfo = new WSManConnectionInfo(false, "server2.domain.local", 5985, "/wsman", "http://schemas.microsoft.com/powershell/Microsoft.PowerShell", ps);
        wsinfo.AuthenticationMechanism = AuthenticationMechanism.Basic;
        wsinfo.ProxyAuthentication = AuthenticationMechanism.Negotiate;
        return wsinfo;
    }

Here is what i'm trying to do but having issues getting it to work since SecureAuthHeader is always null:

[DataContract]
public class SecureAuthHeader
{
[DataMember]
public string UserKey { get; set; }

    \[DataMember\]
    public string UserSecret { get; set; }
}

\[AspNetCompatibilityRequirements(RequirementsMode=AspNetCompatibilityRequirementsMode.Allowed)\]
\[ServiceBehavior(InstanceContextMode=InstanceContextMode.PerSession)\]
public class CPApi : ICPApi
{
    public SecureAuthHeader Authentication;

    public void SetAuthentication(string \_userkey, string \_usersecret)
    {
        Authentication = new SecureAuthHeader();
        Authentication.UserKey = \_userkey;
        Authentication.UserSecret = \_usersecret;
    }
            
    public List GetAllUsers()
    {
        string companyCode = IsAuthenticated;


        // MORE HERE
    }

    private string IsAuthenticated
    {
        get
        {
            if (Authentication == null)
                throw new FaultException("You must authenticate first.");

            try
            {
                DataTable ds = SqlLibrary.ReadSql("SELECT CompanyID FROM ApiAccess WHERE CustomerKey=@CustomerKey AND CustomerSecret=@CustomerSecret",
                    new SqlParameter\[\] { new SqlParameter("CustomerKey", Authentication.UserKey), new SqlParameter("CustomerSecret", Authentication.UserSecret) });

                if (ds == null || ds.Rows.Count == 0)
                    throw new FaultException("Invalid authentication.");

                return ds.Rows\[0\]\["CompanyID"\].ToString();
            }
            catch (Exception ex)
            {
                throw new FaultException(ex.Message);
            }
        }
    }