Some program designing problems
-
Hello, Since I'm a trully begginer in .NET technology, I need advice for the following problem: I developed a windows service, who has the meaning to perform complex processing, over some data. Itself is an independent application. The data came from users of a web site (file uploads, or HTML posts). A web service read the data and save it in files, on harddisk. The windows service, use FileSystemWatcher to monitor the arrival of data, and when a new file is there, start to process it. The question: is this a good way to implement data exchange between web service and windows service? I mean, does not exist other ways to Interprocess-communication/data exchange? The reason I have concerns is that under heavy stress, the filesystemwatcher could behave badly, I know it have some inertia... Does anybody could point me to another kind of "watcher" to use? Some global events or ... what else??? I really have just a partial knowledge about .NET capabilities, so any advice is wellcome. Regards, Don Miguel.
-
Hello, Since I'm a trully begginer in .NET technology, I need advice for the following problem: I developed a windows service, who has the meaning to perform complex processing, over some data. Itself is an independent application. The data came from users of a web site (file uploads, or HTML posts). A web service read the data and save it in files, on harddisk. The windows service, use FileSystemWatcher to monitor the arrival of data, and when a new file is there, start to process it. The question: is this a good way to implement data exchange between web service and windows service? I mean, does not exist other ways to Interprocess-communication/data exchange? The reason I have concerns is that under heavy stress, the filesystemwatcher could behave badly, I know it have some inertia... Does anybody could point me to another kind of "watcher" to use? Some global events or ... what else??? I really have just a partial knowledge about .NET capabilities, so any advice is wellcome. Regards, Don Miguel.
If all you're looking for is a different file watching mechanism, you can try this: Store the names of the files you've already processed (if you delete or move the file you just processed, that'll work too). Then, instead of subscribing to the file system notifications, get a directory listing and see which items are new. (If you need to see changed files, store the timestamp and compare those as well.) Now that you have a list of files to process, go ahead and run through them. Update your list of processed files, wait x seconds, and repeat. If you were wanting to decide on a completely different way of getting the data, you'd need to tell us a lot more information. Do you have control of the application creating the files? Are these on the same machine or separate? How much data is coming through? How often does new data arrive? How critical is it that every single bit of data be processed? Must they be processed in order? Etc. Have fun finding the answer! John :D
-
If all you're looking for is a different file watching mechanism, you can try this: Store the names of the files you've already processed (if you delete or move the file you just processed, that'll work too). Then, instead of subscribing to the file system notifications, get a directory listing and see which items are new. (If you need to see changed files, store the timestamp and compare those as well.) Now that you have a list of files to process, go ahead and run through them. Update your list of processed files, wait x seconds, and repeat. If you were wanting to decide on a completely different way of getting the data, you'd need to tell us a lot more information. Do you have control of the application creating the files? Are these on the same machine or separate? How much data is coming through? How often does new data arrive? How critical is it that every single bit of data be processed? Must they be processed in order? Etc. Have fun finding the answer! John :D
Hello John, First of all, thanks a lot for your time. You are absolutely right, I can list myself the files which are new, so I will not depend on file watcher event. (but I really liked the event notification style...) But also, I want some advice about different mechanisms to receive data. The application which create the data is on different server, and the single way I thinked to pass data to my windows service was through a file. But first tests, reveal a much lower performance as expected, so sure another mechanism could help, at least if it bypass harddisk operations. I don't have control over the application which create the data. I just know it can output a stream of data, in text, xml, or binary. I know the format. It can send data over network sockets. As a must, data have to be processed in order. The chalenge for developers, is to find the fastest way for all above communication. The .NET framework is new for almost all of us, so feedback from experienced people, which faced real performance issues, is greatly appreciated. Thanks again, Don Miguel
-
Hello John, First of all, thanks a lot for your time. You are absolutely right, I can list myself the files which are new, so I will not depend on file watcher event. (but I really liked the event notification style...) But also, I want some advice about different mechanisms to receive data. The application which create the data is on different server, and the single way I thinked to pass data to my windows service was through a file. But first tests, reveal a much lower performance as expected, so sure another mechanism could help, at least if it bypass harddisk operations. I don't have control over the application which create the data. I just know it can output a stream of data, in text, xml, or binary. I know the format. It can send data over network sockets. As a must, data have to be processed in order. The chalenge for developers, is to find the fastest way for all above communication. The .NET framework is new for almost all of us, so feedback from experienced people, which faced real performance issues, is greatly appreciated. Thanks again, Don Miguel
If you have no control over the application generating the data, then your solution is less difficult to find. Choose the option provided by that application which is likely to be the fastest. Learn what that requires, then research/learn/ask how to implement that in the platform/language needed. John John :D
-
Hello, Since I'm a trully begginer in .NET technology, I need advice for the following problem: I developed a windows service, who has the meaning to perform complex processing, over some data. Itself is an independent application. The data came from users of a web site (file uploads, or HTML posts). A web service read the data and save it in files, on harddisk. The windows service, use FileSystemWatcher to monitor the arrival of data, and when a new file is there, start to process it. The question: is this a good way to implement data exchange between web service and windows service? I mean, does not exist other ways to Interprocess-communication/data exchange? The reason I have concerns is that under heavy stress, the filesystemwatcher could behave badly, I know it have some inertia... Does anybody could point me to another kind of "watcher" to use? Some global events or ... what else??? I really have just a partial knowledge about .NET capabilities, so any advice is wellcome. Regards, Don Miguel.
Finally, found the proper solution, if anyone interested: - using native Win32 interprocess comunication methods. Unfortunately, it took me years to finally conclude that this is the true solution. I hoped, and hoped, and hoped... but now, after .NET 2.0 arrival, and looking at the plans for next version, I can safely advice anyone having the same problem: use native way!!!