need some codeproject c++ expertise...
-
hey cpians :) i need some suggestions / tips on diagnosing a problem w/ one of my programs / operating environment... Heres is the data i have to go by so far. I have a c++ program , which loops forever. It looks in a particular folder every 30 seconds looking for new files, that it reads what its found, and processes the info for my needs, than deletes the file. The program is running on a windows 2000 box. The problem I am having is that the process just disappears on heavy usage. No errors , or exceptions that I could try to throw for further data collection, the programs process just disappears and the program stops running and nothing is mentioned of it to me from windows. Just dont know really where to start diagnosing / building diagnostic data. Sometimes it will run fine for 2 days looping the same code 20,000 times, and sometimes it dies after 2 hours. Its the same code looping over and over, so in theory it would be more of a circumstancial / environmental problem the program is running into, than a code itself problem. any tips / suggestions on places to collect more diagnostic data , or other things to check would be highly appreciated :) thanks, josh todo.... :: insert inpirational text here ::
-
hey cpians :) i need some suggestions / tips on diagnosing a problem w/ one of my programs / operating environment... Heres is the data i have to go by so far. I have a c++ program , which loops forever. It looks in a particular folder every 30 seconds looking for new files, that it reads what its found, and processes the info for my needs, than deletes the file. The program is running on a windows 2000 box. The problem I am having is that the process just disappears on heavy usage. No errors , or exceptions that I could try to throw for further data collection, the programs process just disappears and the program stops running and nothing is mentioned of it to me from windows. Just dont know really where to start diagnosing / building diagnostic data. Sometimes it will run fine for 2 days looping the same code 20,000 times, and sometimes it dies after 2 hours. Its the same code looping over and over, so in theory it would be more of a circumstancial / environmental problem the program is running into, than a code itself problem. any tips / suggestions on places to collect more diagnostic data , or other things to check would be highly appreciated :) thanks, josh todo.... :: insert inpirational text here ::
jDawwg wrote: The problem I am having is that the process just disappears on heavy usage. Maybe it is just gone fetching a beer, as everybody is busy and noone is looking its way? ;P Seriously, does it still appear in the task manager? Is it a service? How do you wait this 30 seconds? Simply have it touch some file somewhere in every iteration, so you can at least see on the access time when your process has died.
Who is 'General Failure'? And why is he reading my harddisk?!?
-
hey cpians :) i need some suggestions / tips on diagnosing a problem w/ one of my programs / operating environment... Heres is the data i have to go by so far. I have a c++ program , which loops forever. It looks in a particular folder every 30 seconds looking for new files, that it reads what its found, and processes the info for my needs, than deletes the file. The program is running on a windows 2000 box. The problem I am having is that the process just disappears on heavy usage. No errors , or exceptions that I could try to throw for further data collection, the programs process just disappears and the program stops running and nothing is mentioned of it to me from windows. Just dont know really where to start diagnosing / building diagnostic data. Sometimes it will run fine for 2 days looping the same code 20,000 times, and sometimes it dies after 2 hours. Its the same code looping over and over, so in theory it would be more of a circumstancial / environmental problem the program is running into, than a code itself problem. any tips / suggestions on places to collect more diagnostic data , or other things to check would be highly appreciated :) thanks, josh todo.... :: insert inpirational text here ::
This is a design flaw. Instead of wasting CPU cycles polling a folder, let the OS notify you when something has changed. Use
FindFirstChangeNotification()orReadDirectoryChangesW()instead.
Five birds are sitting on a fence. Three of them decide to fly off. How many are left?
-
hey cpians :) i need some suggestions / tips on diagnosing a problem w/ one of my programs / operating environment... Heres is the data i have to go by so far. I have a c++ program , which loops forever. It looks in a particular folder every 30 seconds looking for new files, that it reads what its found, and processes the info for my needs, than deletes the file. The program is running on a windows 2000 box. The problem I am having is that the process just disappears on heavy usage. No errors , or exceptions that I could try to throw for further data collection, the programs process just disappears and the program stops running and nothing is mentioned of it to me from windows. Just dont know really where to start diagnosing / building diagnostic data. Sometimes it will run fine for 2 days looping the same code 20,000 times, and sometimes it dies after 2 hours. Its the same code looping over and over, so in theory it would be more of a circumstancial / environmental problem the program is running into, than a code itself problem. any tips / suggestions on places to collect more diagnostic data , or other things to check would be highly appreciated :) thanks, josh todo.... :: insert inpirational text here ::
I would think if there are no exceptions or errors when the program disappears, it must have terminated properly. Or, if it is a console app, you might need to run it from the console to see the errors when the program terminates. I suggest you have the program produce a very detailed log of its activities in a file somewhere. This data should get you started. Lorenz Prem Microsoft Corporation
-
jDawwg wrote: The problem I am having is that the process just disappears on heavy usage. Maybe it is just gone fetching a beer, as everybody is busy and noone is looking its way? ;P Seriously, does it still appear in the task manager? Is it a service? How do you wait this 30 seconds? Simply have it touch some file somewhere in every iteration, so you can at least see on the access time when your process has died.
Who is 'General Failure'? And why is he reading my harddisk?!?
I'm using Sleep() to make it wait. I used the term '30 seconds' loosely before, as you can see in my code posted below I dont quite wait that long even :) It doesnt still appear in task manager, it just disappears from everywhere nice and sneaky and strange like! Its not a service. I know I could build data by coding some logging / RT- data analysis into the program , and then use process of elimination to narraw the problem down between the program or o/s / environment conditions, which I guess seems to be one of my few remaining options so I may bite the bullet and do it this way, i was just hoping i was doing it the hard way or something :) #include "stdafx.h" #include "FileInfo.h" #include #include int __cdecl main(int argc, char* argv[]) { CString s; char c1[6000]; // Text char c2[255]; // Storage char cSpoolFile[300]; char cOutFile[300]; char cOutFileTwo[300]; char cIgnoreList[21000]; char testhold[21000]; char c4[]="blah\0"; // key int len; // int result; int numread; int sorted = 0; FILE *input; fstream outfile; // read in configuration ifstream inStream("config.txt"); inStream.getline(cSpoolFile, 300, '\n'); inStream.getline(cOutFile, 300, '\n'); inStream.getline(cOutFileTwo, 300, '\n'); inStream.close(); // read ignore ifstream inStreamtwo("ignore.txt"); while (!inStreamtwo.eof()) { inStreamtwo.getline(testhold, 21000, '\n'); s += testhold; if (!inStreamtwo.eof()) { s += "|"; } } inStreamtwo.close(); // null terminate s += "\0"; // make string to char for var passed into .dll strcpy(cIgnoreList, s); // notification... printf ("Starting Loop.....\n\n"); // loop forever here while (1) { // Open directory for read CFileInfoArray fia; fia.AddDir( cSpoolFile, // Directory "*.*", // Filemask (all files) FALSE, // Recurse subdirs CFileInfoArray::AP_SORTBYNAME | CFileInfoArray::AP_SORTASCENDING, // Sort by name and ascending FALSE // Do not add array entries for directories (only for files) ); /// here will start the directory iteration for (int ji=0;ji
-
This is a design flaw. Instead of wasting CPU cycles polling a folder, let the OS notify you when something has changed. Use
FindFirstChangeNotification()orReadDirectoryChangesW()instead.
Five birds are sitting on a fence. Three of them decide to fly off. How many are left?
actually, the way it works in production here, we get 100's of small txt files added to the folder every few seconds. so it really is never polling the folder. by the time it finishes processing files, than goes through its wait state, their is gauranteed to be more files there for it to find on the need look :) thanks tho, but your suggestion is probably not relevant to what causes my current program to crash. todo.... :: insert inpirational text here ::
-
I would think if there are no exceptions or errors when the program disappears, it must have terminated properly. Or, if it is a console app, you might need to run it from the console to see the errors when the program terminates. I suggest you have the program produce a very detailed log of its activities in a file somewhere. This data should get you started. Lorenz Prem Microsoft Corporation
-
hey cpians :) i need some suggestions / tips on diagnosing a problem w/ one of my programs / operating environment... Heres is the data i have to go by so far. I have a c++ program , which loops forever. It looks in a particular folder every 30 seconds looking for new files, that it reads what its found, and processes the info for my needs, than deletes the file. The program is running on a windows 2000 box. The problem I am having is that the process just disappears on heavy usage. No errors , or exceptions that I could try to throw for further data collection, the programs process just disappears and the program stops running and nothing is mentioned of it to me from windows. Just dont know really where to start diagnosing / building diagnostic data. Sometimes it will run fine for 2 days looping the same code 20,000 times, and sometimes it dies after 2 hours. Its the same code looping over and over, so in theory it would be more of a circumstancial / environmental problem the program is running into, than a code itself problem. any tips / suggestions on places to collect more diagnostic data , or other things to check would be highly appreciated :) thanks, josh todo.... :: insert inpirational text here ::
In case you cannot (don't have time to) fix the problem, you can use XYNTService[^] to run your program, which will check your program automatically and restart it if neccesary (You need to set CheckProcess to 5 and set Restart to Yes in the XYNTService.ini file to have it check your program every 5 minutes). Good luck. [Edit] If you haven't had any idea what a service is about, may be you don't want to waste time on my suggestion. Not all programs are sutible to run from a service. [/Edit][
My articles and software tools
-
I'm using Sleep() to make it wait. I used the term '30 seconds' loosely before, as you can see in my code posted below I dont quite wait that long even :) It doesnt still appear in task manager, it just disappears from everywhere nice and sneaky and strange like! Its not a service. I know I could build data by coding some logging / RT- data analysis into the program , and then use process of elimination to narraw the problem down between the program or o/s / environment conditions, which I guess seems to be one of my few remaining options so I may bite the bullet and do it this way, i was just hoping i was doing it the hard way or something :) #include "stdafx.h" #include "FileInfo.h" #include #include int __cdecl main(int argc, char* argv[]) { CString s; char c1[6000]; // Text char c2[255]; // Storage char cSpoolFile[300]; char cOutFile[300]; char cOutFileTwo[300]; char cIgnoreList[21000]; char testhold[21000]; char c4[]="blah\0"; // key int len; // int result; int numread; int sorted = 0; FILE *input; fstream outfile; // read in configuration ifstream inStream("config.txt"); inStream.getline(cSpoolFile, 300, '\n'); inStream.getline(cOutFile, 300, '\n'); inStream.getline(cOutFileTwo, 300, '\n'); inStream.close(); // read ignore ifstream inStreamtwo("ignore.txt"); while (!inStreamtwo.eof()) { inStreamtwo.getline(testhold, 21000, '\n'); s += testhold; if (!inStreamtwo.eof()) { s += "|"; } } inStreamtwo.close(); // null terminate s += "\0"; // make string to char for var passed into .dll strcpy(cIgnoreList, s); // notification... printf ("Starting Loop.....\n\n"); // loop forever here while (1) { // Open directory for read CFileInfoArray fia; fia.AddDir( cSpoolFile, // Directory "*.*", // Filemask (all files) FALSE, // Recurse subdirs CFileInfoArray::AP_SORTBYNAME | CFileInfoArray::AP_SORTASCENDING, // Sort by name and ascending FALSE // Do not add array entries for directories (only for files) ); /// here will start the directory iteration for (int ji=0;ji
2 possible cases: 1. As I understand you have 2 processes accessing the files: first process writes the file and your program reads the file -- therefore: You maybe failing here:
if ( (input = fopen( fia[ji].GetFilePath(), "r" ) ) == NULL ) { //printf( "Could not open file.......%s\n", fia[ji].GetFilePath() ); return 0; }Because, you return ==> your process exits; 2. Yourchar c1[6000]; // Textis declared as 6000 bytes long. Therefore, in the following statements:numread = fread( c1, sizeof( char ), sizeof(c1), input ); c1[++numread] = '\0'; // null terminateYou maybe getting Buffer overrun if file size is larger then 6000 bytes. I "...Ability to type is not enough to become a Programmer. Unless you type in VB. But then again you have to type really fast..." Me -
I would think if there are no exceptions or errors when the program disappears, it must have terminated properly. Or, if it is a console app, you might need to run it from the console to see the errors when the program terminates. I suggest you have the program produce a very detailed log of its activities in a file somewhere. This data should get you started. Lorenz Prem Microsoft Corporation
ahhh!! you got me thinking, and I got less lazy, and did some looking into the way the admins were running the program (which i should care, but usually to busy to care about) :) they run a mover script (batch file) to move the neccesary files from one folder, to 2 seperate locations. Then they run 2 copies of my app , on the 2 seperate folders. So in those off instances that a copy of my program was trying to access the same files that were still being manipulated by the mover program, i was getting NULL returns on my file open. which , i was doing a natural exit in that case. I modified the program to not exit if it recieves NULL, and to just be patient and retry to go about its business w/ the next file and no more disappearing app! :) thanks for the time / brain power, everyone :) we have solved 'the case of the disappearing app'.. todo.... :: insert inpirational text here ::
-
2 possible cases: 1. As I understand you have 2 processes accessing the files: first process writes the file and your program reads the file -- therefore: You maybe failing here:
if ( (input = fopen( fia[ji].GetFilePath(), "r" ) ) == NULL ) { //printf( "Could not open file.......%s\n", fia[ji].GetFilePath() ); return 0; }Because, you return ==> your process exits; 2. Yourchar c1[6000]; // Textis declared as 6000 bytes long. Therefore, in the following statements:numread = fread( c1, sizeof( char ), sizeof(c1), input ); c1[++numread] = '\0'; // null terminateYou maybe getting Buffer overrun if file size is larger then 6000 bytes. I "...Ability to type is not enough to become a Programmer. Unless you type in VB. But then again you have to type really fast..." Me -
actually, the way it works in production here, we get 100's of small txt files added to the folder every few seconds. so it really is never polling the folder. by the time it finishes processing files, than goes through its wait state, their is gauranteed to be more files there for it to find on the need look :) thanks tho, but your suggestion is probably not relevant to what causes my current program to crash. todo.... :: insert inpirational text here ::
"we get 100's of small txt files added to the folder every few seconds." Hmm... I think maybe you should reconsider your whole design for this application, creating/deleting hundreds of small files every few seconds is a bad move, it fragments your hard drive and slows down your system performance. What are you using those small files for? Lemme guess... communicate among processes, am I right? I think you may want to check out some IPC technique, there are many articles about IPC on code project.
-
"we get 100's of small txt files added to the folder every few seconds." Hmm... I think maybe you should reconsider your whole design for this application, creating/deleting hundreds of small files every few seconds is a bad move, it fragments your hard drive and slows down your system performance. What are you using those small files for? Lemme guess... communicate among processes, am I right? I think you may want to check out some IPC technique, there are many articles about IPC on code project.
nope, they are not for communication among processes. really cant disclose details about it, but they are created this way by a program i cannot replace(without massive time /$$, upper management would rather have my time somewhere else :) ). so basically , i have to engage the tiny files. I know about the fragmentation situation, my trusty admins have already configured the specifically for this kind of usage. if u read the other threads too, the problem with my current solution are solved. it'd be nice if I had time and money to sit and code redundant solutions to programs i just didnt like the implementation of (there again, anything can be improved upon thought), but this wouldnt make me a productive career programmer , I would be more of an unemployed artist type. todo.... :: insert inpirational text here ::