ThreadPool not executing queued jobs
-
Hi, I'm working on a calculation module which handles a lot of small calculations. A sqeuencial approach is not feasible due to the amount of calculations (the biggest batch is approximately 15 million calculations) and my initial run projected 83 days in the worst case scenario. This is too long. Since the threadpool is described as a good solution for many short calculations, I decided to give it a try. My implementation works on a small scale (15k calculations) and performs acceptable given 40 threads (dual-core, so max ideal number is max 50 according to the msdn documentation, but then the system threads are starving and timers run out of sync and GC freezes). My only problem is, when I feed my small jobs to the queue, it fills the first batch of empty threads and then continues queuing, but *not* executing the jobs which are queued for processing. At a certain point, on the small scale, it has completed queuing, and *then* it starts processing the requests very fast. This is not sustainable, since when filling the queue, the system slowly grinds to a halt queuing slower per job. To get an idea of a run: one calculation costs 25-500ms to complete, around 15 calculations per context, and about 1 million contexts to be processed per run. Anyone can help me find an answer why the threadpool continues to queue but not process the queued jobs until the queuing is complete? Thanks in advance.
The consumer isn't a moron; she is your wife.
-
Hi, I'm working on a calculation module which handles a lot of small calculations. A sqeuencial approach is not feasible due to the amount of calculations (the biggest batch is approximately 15 million calculations) and my initial run projected 83 days in the worst case scenario. This is too long. Since the threadpool is described as a good solution for many short calculations, I decided to give it a try. My implementation works on a small scale (15k calculations) and performs acceptable given 40 threads (dual-core, so max ideal number is max 50 according to the msdn documentation, but then the system threads are starving and timers run out of sync and GC freezes). My only problem is, when I feed my small jobs to the queue, it fills the first batch of empty threads and then continues queuing, but *not* executing the jobs which are queued for processing. At a certain point, on the small scale, it has completed queuing, and *then* it starts processing the requests very fast. This is not sustainable, since when filling the queue, the system slowly grinds to a halt queuing slower per job. To get an idea of a run: one calculation costs 25-500ms to complete, around 15 calculations per context, and about 1 million contexts to be processed per run. Anyone can help me find an answer why the threadpool continues to queue but not process the queued jobs until the queuing is complete? Thanks in advance.
The consumer isn't a moron; she is your wife.
Helfdane wrote:
Anyone can help me find an answer why the threadpool continues to queue but not process the queued jobs until the queuing is complete? Thanks in advance.
Even if they can, you are chasing the wrong problem. With only two cores you are not gaining anything by trying to pool a million operations. There is a logical maximum benefit of running twice as fast by using both cores equally. However you cannot attain the maximum because there is overhead of managing the pool and context switching between threads and other resources being used by the system. You might be better off just creating two threads and splitting your operations in half running half in each thread. Since you did not mention it, I should ask if you know about distributed computing? This would be the most common solution using PC's to reduce a 83 day workload.
led mike
-
Hi, I'm working on a calculation module which handles a lot of small calculations. A sqeuencial approach is not feasible due to the amount of calculations (the biggest batch is approximately 15 million calculations) and my initial run projected 83 days in the worst case scenario. This is too long. Since the threadpool is described as a good solution for many short calculations, I decided to give it a try. My implementation works on a small scale (15k calculations) and performs acceptable given 40 threads (dual-core, so max ideal number is max 50 according to the msdn documentation, but then the system threads are starving and timers run out of sync and GC freezes). My only problem is, when I feed my small jobs to the queue, it fills the first batch of empty threads and then continues queuing, but *not* executing the jobs which are queued for processing. At a certain point, on the small scale, it has completed queuing, and *then* it starts processing the requests very fast. This is not sustainable, since when filling the queue, the system slowly grinds to a halt queuing slower per job. To get an idea of a run: one calculation costs 25-500ms to complete, around 15 calculations per context, and about 1 million contexts to be processed per run. Anyone can help me find an answer why the threadpool continues to queue but not process the queued jobs until the queuing is complete? Thanks in advance.
The consumer isn't a moron; she is your wife.
Helfdane wrote:
performs acceptable given 40 threads
40 threads on a dual core? It can perform better if you use few threads. Under that load, you're spending an awful lot of time doing context switching going from thread to thread to thread.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008 -
Helfdane wrote:
performs acceptable given 40 threads
40 threads on a dual core? It can perform better if you use few threads. Under that load, you're spending an awful lot of time doing context switching going from thread to thread to thread.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008 -
Sometimes is multiple people tell him the same thing, it might sink in that it's not a good idea. :-D
A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008 -
Sometimes is multiple people tell him the same thing, it might sink in that it's not a good idea. :-D
A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008 -
Helfdane wrote:
Anyone can help me find an answer why the threadpool continues to queue but not process the queued jobs until the queuing is complete? Thanks in advance.
Even if they can, you are chasing the wrong problem. With only two cores you are not gaining anything by trying to pool a million operations. There is a logical maximum benefit of running twice as fast by using both cores equally. However you cannot attain the maximum because there is overhead of managing the pool and context switching between threads and other resources being used by the system. You might be better off just creating two threads and splitting your operations in half running half in each thread. Since you did not mention it, I should ask if you know about distributed computing? This would be the most common solution using PC's to reduce a 83 day workload.
led mike
Thank you and Dave for your replies. Distributed computing is not an option, since it's an application that is installed at the customer and has to run standalone (it's a critical application where most clients prefer to hook it off the network). About the threads, I was only following the MSDN documentation where they mention that 25 threads per core is the max ideal number where the overhead would still be smaller than the improvement per thread. I forgot to mention, each calculation *might* need some additional information from the DB, hence the threading. My problem is not the load or the number of threads, they are parameters on an existing infrastructure. My question is, why the threads are not working (are they on hold/in wait?) while queuing the jobs (is this infrastructure the right one?). Since you and Dave Kreskowiak propose less threads which I can agree on, the queuing will still be there, thus my problem still exists as the queuing still takes time while the threads do nothing (cpu load during queuing is not above 40%, so the other 60% should be used to do the first calculations but it doesn't use the 60% unless queuing is completed).
The consumer isn't a moron; she is your wife.
-
Thank you and Dave for your replies. Distributed computing is not an option, since it's an application that is installed at the customer and has to run standalone (it's a critical application where most clients prefer to hook it off the network). About the threads, I was only following the MSDN documentation where they mention that 25 threads per core is the max ideal number where the overhead would still be smaller than the improvement per thread. I forgot to mention, each calculation *might* need some additional information from the DB, hence the threading. My problem is not the load or the number of threads, they are parameters on an existing infrastructure. My question is, why the threads are not working (are they on hold/in wait?) while queuing the jobs (is this infrastructure the right one?). Since you and Dave Kreskowiak propose less threads which I can agree on, the queuing will still be there, thus my problem still exists as the queuing still takes time while the threads do nothing (cpu load during queuing is not above 40%, so the other 60% should be used to do the first calculations but it doesn't use the 60% unless queuing is completed).
The consumer isn't a moron; she is your wife.
Helfdane wrote:
the MSDN documentation where they mention that 25 threads per core is the max ideal number where the overhead would still be smaller than the improvement per thread.
Yes but for what use model? Threading is mostly related to multi-user processing. You want to maximize threads in that use model to share the compute cycles across users so that they don't wait one at a time for a reply. This is not your use model.
Helfdane wrote:
why the threads are not working (are they on hold/in wait?)
Debugging threaded code is difficult when you are there, we are not. We are mostly guessing at this based on the information you have provided. One thing we do not know is how many jobs you queued. You threw out some numbers like 15 million. That's a large number for a PC. Given your use model I would research using two threads and a design that allows each thread to process half the calculations. Resources are another issue when you start talking numbers like 15 million. I would wonder how much resources each calculation requires, I mean if there is some source data used for each calculation and you load all 15 million calculation resources at once you probably start having a lot of paging which slows everything down, as opposed to getting the resources only when needed. I don't know what your experience and background is but writing efficient code requires a lot of in depth knowledge and experience. You have to understand hardware as well as software. Most of that is beyond my expertise, but I do understand you can't just read a threading article to get up to speed.
led mike
-
Helfdane wrote:
the MSDN documentation where they mention that 25 threads per core is the max ideal number where the overhead would still be smaller than the improvement per thread.
Yes but for what use model? Threading is mostly related to multi-user processing. You want to maximize threads in that use model to share the compute cycles across users so that they don't wait one at a time for a reply. This is not your use model.
Helfdane wrote:
why the threads are not working (are they on hold/in wait?)
Debugging threaded code is difficult when you are there, we are not. We are mostly guessing at this based on the information you have provided. One thing we do not know is how many jobs you queued. You threw out some numbers like 15 million. That's a large number for a PC. Given your use model I would research using two threads and a design that allows each thread to process half the calculations. Resources are another issue when you start talking numbers like 15 million. I would wonder how much resources each calculation requires, I mean if there is some source data used for each calculation and you load all 15 million calculation resources at once you probably start having a lot of paging which slows everything down, as opposed to getting the resources only when needed. I don't know what your experience and background is but writing efficient code requires a lot of in depth knowledge and experience. You have to understand hardware as well as software. Most of that is beyond my expertise, but I do understand you can't just read a threading article to get up to speed.
led mike
You have valid points, thanks for your input. We are talking a multi-user environment, that is, multiple users can do a request for a number of calculations (there is one dispatcher which processes the requests by transforming it into the format used by the calculation engine. This dispatcher will probably have a WCF component for the thin clients to talk to, this is not final yet. In addition, the dispatcher will have the possibility of throttling the requests if the queue becomes too big, but that is something I haven't worked out yet; it's still on the drawing board). The dispatcher is designed to also monitor the system resources, as on which it will base his throttling and batching strategy. This should lessen the preassure on the hardware as well. The test-case I had was 15k calculations (1000 base calculations with 15 subcalculations each in testing data). This means that when the calculation starts, 15k calculations are in the queue. The thing which hints me that nothing is happening is, that when queuing the first action of a calculation, the calculation routine is sending a notification to the console that it is starting, but the starting message never gets there unless queuing is done and so is the load. I am not sure if my current infrastructure will be sufficient, hence my research and these forum postings, as I am not sure if I want to use a component (the threadpool in this case) where I cannot predict exactly what will happen. I think I do need threading since missing parts required for a calculation need to be fetched from the DB and/or are dependant on other calculations. My background is saas with an average load of 500+ simultanious connections for clients who operate globally. My background is not primarily with .net.
The consumer isn't a moron; she is your wife.