Optimisation Brag
-
Hey Guys Just had to come on and brag about my optimisation i wrote a simple parser funtion today and the first run it took 75sec to parse a 199Kb text file. The latest optimised version is doing the same job in 0.06sec how good is that for optimisation. Peter
pretty good :cool:
43 68 65 65 72 73 2c 4d 69 63 68 61 65 6c
-
Hey Guys Just had to come on and brag about my optimisation i wrote a simple parser funtion today and the first run it took 75sec to parse a 199Kb text file. The latest optimised version is doing the same job in 0.06sec how good is that for optimisation. Peter
Got Perl? :)
Bruce Duncan, CP#9088, CPUA 0xA1EE, Sonork 100.10030
'ugly naked women are good, when i'm not around, in front of someone else' - Shog9 -
75sec to parse a 199Kb text file ? I did better times when I had a PC XT 4.77Mhz using Turbo Pascal 3 on 5 1/4" disks! Concussus surgo. When struck I rise.
Yeah No Prob The bottleneck was basically memory access i was using strlen in the while() loops. I just assigned the value to a dword once and used that rather then strlen so one memory access rather then having to go through each byte of a string to work out its size. I also then played about with the compiler settings and switched from debug and enabled Pentium Pro, fastcall, 16bit allignment. I also used the stricmp rather then using CString == operator which gave me a performance boost. The strlen fix made it go from 75sec to 2.5sec so big bottleneck and the stricmp() made it go to 0.7sec from 2.5sec and then the compile optimistions made it go from 0.7 to 0.06. heres the code: before:-
QueryPerformanceCounter(&liBeforeClock); DWORD dwCurWord = 0; DWORD dwNumMatches = 0; UINT uiCurChar = 0; DWORD dwCurPart = 0; DWORD dwStringPart = 0; CString sCurWord; CString sKeyWord; do { while(pStringtoParse[dwStringPart] != ' ') { if(dwStringPart == strlen(pStringtoParse)) { break; } sCurWord += pStringtoParse[dwStringPart]; dwStringPart++; } //AfxMessageBox(sCurWord); for(DWORD i = 0; i < dwNumOfWords; i++) { sKeyWord = pWordToCheck[i].sKeyWord; if(sKeyWord == sCurWord) { //AfxMessageBox(sCurWord); dwNumMatches++; } } sCurWord.Empty(); dwStringPart++; } while(dwStringPart < strlen(pStringtoParse)); QueryPerformanceCounter(&liAfterClock); return dwNumMatches;after:-DWORD BasicParser(char *pStringtoParse) { QueryPerformanceCounter(&liBeforeClock); DWORD dwCurWord = 0; DWORD dwNumMatches = 0; UINT uiCurChar = 0; DWORD dwCurPart = 0; DWORD dwStringPart = 0; CString sCurWord; CString sKeyWord; //Find length of string once at start this can save hugh //amounts of time rather then doing strlen in loop checks DWORD dwStringLength = strlen(pStringtoParse); do { while(pStringtoParse[dwStringPart] != ' ') { if(dwStringPart == dwStringLength) { break; } sCurWord += pStringtoParse[dwStringPart]; dwStringPart++; } //AfxMessageBox(sCurWord); for(DWORD i = 0; i < dwNumOfWords; i++) { //sKeyWord = pWordToCheck[i].sKeyWord; if(_stricmp(sCurWord, pWordToCheck[i].sKeyWord) == 0) { //AfxMessageBox(sCurWord); dwNumMatches++; } } sCurWord.Empty(); dwStringPart++; } while(dwStringPart < dwStringLength); QueryPerforma -
Yeah No Prob The bottleneck was basically memory access i was using strlen in the while() loops. I just assigned the value to a dword once and used that rather then strlen so one memory access rather then having to go through each byte of a string to work out its size. I also then played about with the compiler settings and switched from debug and enabled Pentium Pro, fastcall, 16bit allignment. I also used the stricmp rather then using CString == operator which gave me a performance boost. The strlen fix made it go from 75sec to 2.5sec so big bottleneck and the stricmp() made it go to 0.7sec from 2.5sec and then the compile optimistions made it go from 0.7 to 0.06. heres the code: before:-
QueryPerformanceCounter(&liBeforeClock); DWORD dwCurWord = 0; DWORD dwNumMatches = 0; UINT uiCurChar = 0; DWORD dwCurPart = 0; DWORD dwStringPart = 0; CString sCurWord; CString sKeyWord; do { while(pStringtoParse[dwStringPart] != ' ') { if(dwStringPart == strlen(pStringtoParse)) { break; } sCurWord += pStringtoParse[dwStringPart]; dwStringPart++; } //AfxMessageBox(sCurWord); for(DWORD i = 0; i < dwNumOfWords; i++) { sKeyWord = pWordToCheck[i].sKeyWord; if(sKeyWord == sCurWord) { //AfxMessageBox(sCurWord); dwNumMatches++; } } sCurWord.Empty(); dwStringPart++; } while(dwStringPart < strlen(pStringtoParse)); QueryPerformanceCounter(&liAfterClock); return dwNumMatches;after:-DWORD BasicParser(char *pStringtoParse) { QueryPerformanceCounter(&liBeforeClock); DWORD dwCurWord = 0; DWORD dwNumMatches = 0; UINT uiCurChar = 0; DWORD dwCurPart = 0; DWORD dwStringPart = 0; CString sCurWord; CString sKeyWord; //Find length of string once at start this can save hugh //amounts of time rather then doing strlen in loop checks DWORD dwStringLength = strlen(pStringtoParse); do { while(pStringtoParse[dwStringPart] != ' ') { if(dwStringPart == dwStringLength) { break; } sCurWord += pStringtoParse[dwStringPart]; dwStringPart++; } //AfxMessageBox(sCurWord); for(DWORD i = 0; i < dwNumOfWords; i++) { //sKeyWord = pWordToCheck[i].sKeyWord; if(_stricmp(sCurWord, pWordToCheck[i].sKeyWord) == 0) { //AfxMessageBox(sCurWord); dwNumMatches++; } } sCurWord.Empty(); dwStringPart++; } while(dwStringPart < dwStringLength); QueryPerformaPeter Liddle wrote: i was using strlen in the while() loops. Sort of pessimization, isn't it :) Tomasz Sowinski -- http://www.shooltz.com
"Yields falsehood when preceded by its quotation" yields falsehood when preceded by its quotation.
-
Yeah No Prob The bottleneck was basically memory access i was using strlen in the while() loops. I just assigned the value to a dword once and used that rather then strlen so one memory access rather then having to go through each byte of a string to work out its size. I also then played about with the compiler settings and switched from debug and enabled Pentium Pro, fastcall, 16bit allignment. I also used the stricmp rather then using CString == operator which gave me a performance boost. The strlen fix made it go from 75sec to 2.5sec so big bottleneck and the stricmp() made it go to 0.7sec from 2.5sec and then the compile optimistions made it go from 0.7 to 0.06. heres the code: before:-
QueryPerformanceCounter(&liBeforeClock); DWORD dwCurWord = 0; DWORD dwNumMatches = 0; UINT uiCurChar = 0; DWORD dwCurPart = 0; DWORD dwStringPart = 0; CString sCurWord; CString sKeyWord; do { while(pStringtoParse[dwStringPart] != ' ') { if(dwStringPart == strlen(pStringtoParse)) { break; } sCurWord += pStringtoParse[dwStringPart]; dwStringPart++; } //AfxMessageBox(sCurWord); for(DWORD i = 0; i < dwNumOfWords; i++) { sKeyWord = pWordToCheck[i].sKeyWord; if(sKeyWord == sCurWord) { //AfxMessageBox(sCurWord); dwNumMatches++; } } sCurWord.Empty(); dwStringPart++; } while(dwStringPart < strlen(pStringtoParse)); QueryPerformanceCounter(&liAfterClock); return dwNumMatches;after:-DWORD BasicParser(char *pStringtoParse) { QueryPerformanceCounter(&liBeforeClock); DWORD dwCurWord = 0; DWORD dwNumMatches = 0; UINT uiCurChar = 0; DWORD dwCurPart = 0; DWORD dwStringPart = 0; CString sCurWord; CString sKeyWord; //Find length of string once at start this can save hugh //amounts of time rather then doing strlen in loop checks DWORD dwStringLength = strlen(pStringtoParse); do { while(pStringtoParse[dwStringPart] != ' ') { if(dwStringPart == dwStringLength) { break; } sCurWord += pStringtoParse[dwStringPart]; dwStringPart++; } //AfxMessageBox(sCurWord); for(DWORD i = 0; i < dwNumOfWords; i++) { //sKeyWord = pWordToCheck[i].sKeyWord; if(_stricmp(sCurWord, pWordToCheck[i].sKeyWord) == 0) { //AfxMessageBox(sCurWord); dwNumMatches++; } } sCurWord.Empty(); dwStringPart++; } while(dwStringPart < dwStringLength); QueryPerformaVery minor change but a massive improvement!!! :-D
Regards, Brian Dela :-)
-
pretty good :cool:
43 68 65 65 72 73 2c 4d 69 63 68 61 65 6c
Michael Mac wrote: 43 68 65 65 72 73 2c 4d 69 63 68 61 65 6c Cheers, Michael ? Concussus surgo. When struck I rise.
-
Michael Mac wrote: 43 68 65 65 72 73 2c 4d 69 63 68 61 65 6c Cheers, Michael ? Concussus surgo. When struck I rise.
Now I need to change my signature because everybody knows what it means. :) :) :)
43 68 65 65 72 73 2c 4d 69 63 68 61 65 6c
-
Peter Liddle wrote: how good is that for optimisation. Fairly good.. considering it's 1250 times faster... Feel like posting the code for both versions???
Regards, Brian Dela :-)
Removed .Net and used gcc /O3 instead ?
And I swallow a small raisin.
-
Hey Guys Just had to come on and brag about my optimisation i wrote a simple parser funtion today and the first run it took 75sec to parse a 199Kb text file. The latest optimised version is doing the same job in 0.06sec how good is that for optimisation. Peter
I found the exact same thing in code I wrote for a University assignment (part of the AI system for a scrabble playing game - I was using strlen to check if a string was empty. When I changed it to just check if the string contents was null, it sped up increadibly (about the same kind of change as you had - a few seconds to parse the word list rather than a few minutes) -- Help me! I'm turning into a grapefruit!
-
Peter Liddle wrote: i was using strlen in the while() loops. Sort of pessimization, isn't it :) Tomasz Sowinski -- http://www.shooltz.com
"Yields falsehood when preceded by its quotation" yields falsehood when preceded by its quotation.
Sort of pessimization, isn't it? I had a similar problem years ago at my first job. I was writing an input filter for Targa images. Targas have a case where you need to interpret blocks of data based on the value of their first byte. Naively, I coded is as "read a byte, read the chunk, read a byte, read a chunk." It took over a minute to process something like a 320x240 image (on a 386, of course, but still ridiculously slow). Changing to a buffered i/o mechanism reduced it to reasonable times. When I proudly told my boss of the speedup, he said, "So you really didn't optimize it, you just depessimized it." *jab* *deflate* "Uhh... yeah, I guess." :-) Tim Lesher http://www.lesher.ws