Executing external DOS command with output redirecting to file

[Rajesh_Parameswaran]

Hi there, I'm trying to find a string inside a text file, using VC++ 6.0 application. The logs contained in the text file can grow up to MBs (50 - 70 MB) My aim here is to search a string in that and need to get the time stamp of last occurrence of the string. I cant use the sequential search as this process runs as a thread and it runs for every 2 seconds. Sequential search will take too much of time, if the size of the log file grows to MBs. I thought of using the findstr.exe command to find out the occurrence of the string "Error #1028" by this, all the lines matching the string will be redirected to the file, thus makes easy to find the last occurrence of the matching string. WinExec("findstr   /C:\"ERROR #1028\" C:\\SERVER.log > C:\\temp_log.txt", SW_HIDE) But, the output seems to be not re-directing to the temp_log.txt file. I even tried creating the file temp.txt before executing the same, but the contents will be blank. Even I tried the >> operator too. If i place this same command in a batch file, and invoke the batch file using WinExec(), it works fine. Please pass on your comments on how this could be achieve or is there any other way to achieve the same, without having a dependency on external DOS command findstr.exe Thanks in Advance, Rajesh

[Stuart Dootson]

Rajesh_Parameswaran wrote:

Sequential search will take too much of time, if the size of the log file grows to MBs.

How do you think findstr works? It does a sequential search through the file... Even with a naive string search, you can search through 60MB in around 500ms (2.4GHz Core 2Duo, using a file mapping to load the file). If you apply some algorithm optimisation (we're looking for the last occurrence of a string - start at the end of the file rather than beginning, that's likely to get a lot smaller.

Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p

[Rajesh_Parameswaran]

Hi Stuart, Thanks for your response. Is there a way to do backward read in a file? I searched in google and i couldn't find useful article on backward reading a file. Each line in the log file is dynamic. So reading a fixed length of character will end up in middle of a line. The time stamp in the log file is at be beginning of each line and there are changes that it might read half the line, if we try to read as a fixed chunk of data from bottom. please pass on your comments. Thanks in Advance, Rajesh

[Stuart Dootson]

I would read the file by mapping it into memory[^], not reading it using file I/O functions. That allows me to treat the file as a continuous block of memory, so searching the file for a string is almost as as simple as using strstr[^] - I actually used this code to search backwards through a file (specified as the first argument on the command line:

int main(int argc, char* argv[])
{
DefaultFileReader fr(argv[1]);

const char* lookBegin = argv[2];
const size_t lookLen = strlen(argv[2]);
const char* lookEnd = argv[2] + lookLen;

const char* p = (const char*)fr.end()-lookLen;
while (p!=(const char*)fr.begin())
{
if (!strncmp(lookBegin, p, lookLen)) break;
--p;
}
size_t firstPos = p - (const char*)fr.begin();
std::cout << firstPos << std::endl;
return 0;
}

DefaultFileReader is a class I've written that memory-maps a file. It has begin() and end() methods to access the memory that has been mapped. You can see that my code isn't looking for lines in the file - that's not really necessary once you know what text you're looking for (but could be added - line terminators are just characters!).

Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p

[David Crow]

Rajesh_Parameswaran wrote:

WinExec("findstr /C:\"ERROR #1028\" C:\\SERVER.log > C:\\temp_log.txt", SW_HIDE) But, the output seems to be not re-directing to the temp_log.txt file. I even tried creating the file temp.txt before executing the same, but the contents will be blank. Even I tried the >> operator too.

This all has to do with > and >> not being passed on to the command processor. As you've noted, if you want to do it this way, you'll need to write that string to a BAT file and WinExec() that instead. How is the file formatted? Is it in any order? How often is the file updated?

"Old age is like a bank account. You withdraw later in life what you have deposited along the way." - Unknown

"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons

[Rajesh_Parameswaran]

Hi David, This log file is plain text file, a Sybase generated one for replication server and the logging will be very frequent and dynamic. Sample content of the file listed below: ================================================================================== I. 2006/03/15 06:50:48. The first transaction for database 'JAGUAR1.blue' has been logged into the exceptions log and skipped. I. 2006/03/15 06:50:49. A grouped transaction of 2 individual transactions has failed in database 'JAGUAR1.blue'. Each transaction in the group will be executed individually. E. 2006/03/15 06:50:51. ERROR #1028 DSI EXEC(104(1) JAGUAR1.blue) - dsiqmint.c(3062)      Message from server: Message: 2601, State 1, Severity 14 -- 'Attempt to insert duplicate key row in object 'NIiAppStat' with unique index 'XPKNIiAppStat' '. I. 2006/03/15 06:50:51. Message from server: Message: 3621, State 0, Severity 10 -- 'Command has been aborted. '. H. 2006/03/15 06:50:51. THREAD FATAL ERROR #5049 DSI EXEC(104(1) JAGUAR1.blue) - dsiqmint.c(3069)      The DSI thread for database 'JAGUAR1.blue' is being shutdown. DSI received data server error #2601 which is mapped to STOP_REPLICATION. See logged data server errors for more information. The data server error was caused by output command #1 mapped from input command #2 of the failed transaction. I. 2006/03/15 06:50:51. The DSI thread for database 'JAGUAR1.blue' is shutdown. ============================================================================== I'm looking for time stamp for   "ERROR #1028 DSI EXEC(104(1)" Thanks in Advance, Rajesh