Threads / Class / Array
-
I have a form that gets a list of files for processing I am storing the path in an sting array then passing the array to a class that does the work on the files. the single thread approach takes about 30 mins to process 100 files and update an access DB (will be ported to SQL soon). Currently the single thread makes the form not responsive. I want to at a minimum move the processing to another thread, possibly setup a thread pool. from what I can tell the time consuming part is updating the DB. I have a few questions before I jump to conclusion of adding 1 thread or a thread pool. 1. will the thread pool solution run into locked file problems when writing to the DB from multiple threads? currently using Jet OLEDB to connect to the DB 2. What is the best way to handle passing the files to the class with threads? Currently I am reading the directory and storing the files in a string array in the form creating the object for the class passing the array to the StoreArray method call the Process method the process method calls another method that handles the interaction with the array I think the answer to my 2nd question with the least amount of rewriting code is to create the object of the class as a form variable, but I'm not sure that that is really the best solution. I'm new to threads and not sure how I can get the data from the Form to the class. below is two of the methods that should show how i am currently doing this (could have errors I didn't cut and paste) ## form1 ### private void btnStart_Click(object sender, EventArgs e) { int count = 0; string dirpath = this.txtxmldir.Text.ToString(); if (dirpath != "") { //new Dir Info object DirectoryInfo di = new DirectoryInfo(dirpath); //load files into array FileInfo[] rgFiles = di.GetFiles("*.xml"); foreach (FileInfo fi in rgFiles) { file[count] = dirpath + "\\" + fi.Name; count++; } } RRRxml ProcessXml = new RRRxml(); ProcessXml.SetFiles(file, count); ProcessXml.Process(); } ## RRRxml Class ### public void Process() { while(Arraycounter > 0) { string FiletoProcess = ReadFiles(); //Read files is in charge of controlling the array counter and returns the file to load xml.Load(FiletoProcess);
-
I have a form that gets a list of files for processing I am storing the path in an sting array then passing the array to a class that does the work on the files. the single thread approach takes about 30 mins to process 100 files and update an access DB (will be ported to SQL soon). Currently the single thread makes the form not responsive. I want to at a minimum move the processing to another thread, possibly setup a thread pool. from what I can tell the time consuming part is updating the DB. I have a few questions before I jump to conclusion of adding 1 thread or a thread pool. 1. will the thread pool solution run into locked file problems when writing to the DB from multiple threads? currently using Jet OLEDB to connect to the DB 2. What is the best way to handle passing the files to the class with threads? Currently I am reading the directory and storing the files in a string array in the form creating the object for the class passing the array to the StoreArray method call the Process method the process method calls another method that handles the interaction with the array I think the answer to my 2nd question with the least amount of rewriting code is to create the object of the class as a form variable, but I'm not sure that that is really the best solution. I'm new to threads and not sure how I can get the data from the Form to the class. below is two of the methods that should show how i am currently doing this (could have errors I didn't cut and paste) ## form1 ### private void btnStart_Click(object sender, EventArgs e) { int count = 0; string dirpath = this.txtxmldir.Text.ToString(); if (dirpath != "") { //new Dir Info object DirectoryInfo di = new DirectoryInfo(dirpath); //load files into array FileInfo[] rgFiles = di.GetFiles("*.xml"); foreach (FileInfo fi in rgFiles) { file[count] = dirpath + "\\" + fi.Name; count++; } } RRRxml ProcessXml = new RRRxml(); ProcessXml.SetFiles(file, count); ProcessXml.Process(); } ## RRRxml Class ### public void Process() { while(Arraycounter > 0) { string FiletoProcess = ReadFiles(); //Read files is in charge of controlling the array counter and returns the file to load xml.Load(FiletoProcess);
Hi Planker, i will try to answer your question. 1. I don't know it exactly but it seems that there is only one connection to the db at one time (I think because it is file based). This issue will be solved if you move to a database supporting multiple connections at once. To use multiple threads with the access database you can think about using the producer-consumer pattern. The threads processing your files will produce data. This data will be stored in a single datastructure (attention: must be thread-safe!). If data is passed into this structure the thread containing the db connection will "consume" this data and store it within the db. (search for the producer-consumer-pattern to get an idea of it). 2. Here I would nearly do the same using the producer-consumer-pattern. Store the files in a thread-safe datastructure (single static instance, so that it can be accessed from everywhere in your application), the threads working with the files will "consume" the filenames stored within the datastructure. The perfect datastructure for this is a Queue. For using the producer-consumer-pattern take a look here: http://msdn.microsoft.com/en-us/library/yy12yx1f(VS.80).aspx[^][^]. Let me know if this helps you. Regards Sebastian
It's not a bug, it's a feature! Check out my CodeProject article Permission-by-aspect. Me in Softwareland.