Help spawing more then 25 threads per processor in a thread pool
-
need some guidance, I have a collection of properties 166 to be spacific, each object in the collection will spawn a thread to perform a long running task, I also want to pump meessages back to the calling thread. threadpool will provide me the solution unfortunatly MS has limited the number of threads to 25 per processor, simple math will tell me that I need a huge box. so that another approach would be to loop from 0 to max threadpool and spawn an instance of a thread pulling the objectid from the collection based on the threadid. that's fine or the first 50 objects. how would I now pull the remaing objects from the collection. example for i as 0 to threadpool.maxthreads -1 dim node as basenode = basecollection(i) dim rem as new workernode(node) addhandler rem.messageback new messagehandler(addressof OnMessagesReceived) rem.dowork next assuming dual CPU will make the maxthread value be 50 and my basecollection will hold 166 objects I am stuck spawning objects 51-165
-
need some guidance, I have a collection of properties 166 to be spacific, each object in the collection will spawn a thread to perform a long running task, I also want to pump meessages back to the calling thread. threadpool will provide me the solution unfortunatly MS has limited the number of threads to 25 per processor, simple math will tell me that I need a huge box. so that another approach would be to loop from 0 to max threadpool and spawn an instance of a thread pulling the objectid from the collection based on the threadid. that's fine or the first 50 objects. how would I now pull the remaing objects from the collection. example for i as 0 to threadpool.maxthreads -1 dim node as basenode = basecollection(i) dim rem as new workernode(node) addhandler rem.messageback new messagehandler(addressof OnMessagesReceived) rem.dowork next assuming dual CPU will make the maxthread value be 50 and my basecollection will hold 166 objects I am stuck spawning objects 51-165
With .NET 2.0 SP1, the default was changed from 25 threads per core to 250. This is not to say that using 250 is a good design. Seriously, if you think you need more than 25 threads per core running, you really need to reconsider your design. You'll be wasting so much time in context switching, it's silly. Also, the ThreadPool throttles it's new thread creation to 2 per second once the thread count exceeds the number of cores running the threads. So, on a single core machine, it'll still take well over a minute to create all of your threads. Now, having said all that, you can't change the MaxThreadPoolThreads limit in managed code.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008 -
With .NET 2.0 SP1, the default was changed from 25 threads per core to 250. This is not to say that using 250 is a good design. Seriously, if you think you need more than 25 threads per core running, you really need to reconsider your design. You'll be wasting so much time in context switching, it's silly. Also, the ThreadPool throttles it's new thread creation to 2 per second once the thread count exceeds the number of cores running the threads. So, on a single core machine, it'll still take well over a minute to create all of your threads. Now, having said all that, you can't change the MaxThreadPoolThreads limit in managed code.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008this is good to hear, even though the collection will hold 165 objects, it's unlikly I will actualy spawn all 165 threads at once, each object will represent a pop3 client, that will pop a mailbox return all it's mail perform some formating, auto alerting and store the results in a DB then die. the reason for spawing as many as possible at once is reduce the time between poping, also the reason for passing messages back is for the management interface to watch activity through ipclient....