Getting the filesize of all files inside of a folder, quicker
-
I am wondering if anyone has any other ways of tallying up the filesize of all files in a folder than the following:
Using CFileFind recursively to get a list of all files in the specified folder and its subfolders, and calling CFile::GetStatus() on each file found to tally the total filesize.I've discovered that this is quite quick to do for folders with thousands of files if it is on a local HD, but if it's across the LAN on a seperate box it is rather slow to do this. I need a different way of doing this that will be much quicker: I have a set of many folders to check filesizes on, and if just one folder is taking 20 seconds, it's killer to wait for a set of 50 folders, heheheh. If anyone has a better way, let me know! :) : Dean 'Karnatos' Michaud -
I am wondering if anyone has any other ways of tallying up the filesize of all files in a folder than the following:
Using CFileFind recursively to get a list of all files in the specified folder and its subfolders, and calling CFile::GetStatus() on each file found to tally the total filesize.I've discovered that this is quite quick to do for folders with thousands of files if it is on a local HD, but if it's across the LAN on a seperate box it is rather slow to do this. I need a different way of doing this that will be much quicker: I have a set of many folders to check filesizes on, and if just one folder is taking 20 seconds, it's killer to wait for a set of 50 folders, heheheh. If anyone has a better way, let me know! :) : Dean 'Karnatos' MichaudDoing the same thing across the network is going to be slower than doing it on the local machine, you can't stop that. You might make it marginally faster by having a service on the remote machine do all the disk\file work for you and return just the folder totals. There will still be overhead of network communications but the service will be accessing the disk locally and so might be somewhat faster depending on the layout of the files and folders on the disk. A further step would be to have the service monitor the folders and keep an semi-real-time total of the folders which it could return without scanning the disk at request time.
"No matter where you go, there your are." - Buckaroo Banzai
-pete
-
I am wondering if anyone has any other ways of tallying up the filesize of all files in a folder than the following:
Using CFileFind recursively to get a list of all files in the specified folder and its subfolders, and calling CFile::GetStatus() on each file found to tally the total filesize.I've discovered that this is quite quick to do for folders with thousands of files if it is on a local HD, but if it's across the LAN on a seperate box it is rather slow to do this. I need a different way of doing this that will be much quicker: I have a set of many folders to check filesizes on, and if just one folder is taking 20 seconds, it's killer to wait for a set of 50 folders, heheheh. If anyone has a better way, let me know! :) : Dean 'Karnatos' MichaudThis http://support.microsoft.com/?id=835601[^] talks of a similar problem? Why not make a socket-based app at the other end of the connection, and make that app do the CFileFind thing on the remote comupter. This would take some effort no doubt, but I suppose it should be faster than using CFileFind on a local pc with a remote directory... Bikram Singh
-
This http://support.microsoft.com/?id=835601[^] talks of a similar problem? Why not make a socket-based app at the other end of the connection, and make that app do the CFileFind thing on the remote comupter. This would take some effort no doubt, but I suppose it should be faster than using CFileFind on a local pc with a remote directory... Bikram Singh
Yeap - I've thought about having a service on the other end that keeps a tally in real-time and have my app just ask it for the folder sizes. If I have to I'll take that approach. I'm just wondering if there is any other way that might even be marginally faster than what I am doing. I was wondering if there's a way to query a folder's size rather than querying each and every file as I am doing. I'll have a look at that MS article now... perhaps it might shed some light on things for me. Thanks to both you and palbano for quick replies. If anyone thinks of something else I'm all ears :-D
-
Yeap - I've thought about having a service on the other end that keeps a tally in real-time and have my app just ask it for the folder sizes. If I have to I'll take that approach. I'm just wondering if there is any other way that might even be marginally faster than what I am doing. I was wondering if there's a way to query a folder's size rather than querying each and every file as I am doing. I'll have a look at that MS article now... perhaps it might shed some light on things for me. Thanks to both you and palbano for quick replies. If anyone thinks of something else I'm all ears :-D
>> If anyone thinks of something else I'm all ears Well you might also take a look at using FTP. At least that way you don't have to build your own service just the client to communicate with FTP. FTP without getting into file transfers is fairly simple.
"No matter where you go, there your are." - Buckaroo Banzai
-pete
-
I am wondering if anyone has any other ways of tallying up the filesize of all files in a folder than the following:
Using CFileFind recursively to get a list of all files in the specified folder and its subfolders, and calling CFile::GetStatus() on each file found to tally the total filesize.I've discovered that this is quite quick to do for folders with thousands of files if it is on a local HD, but if it's across the LAN on a seperate box it is rather slow to do this. I need a different way of doing this that will be much quicker: I have a set of many folders to check filesizes on, and if just one folder is taking 20 seconds, it's killer to wait for a set of 50 folders, heheheh. If anyone has a better way, let me know! :) : Dean 'Karnatos' MichaudI managed to cut the time it takes down by half. I was being dense, I did not realise that CFileFind can return the filesize without having to call CFile::GetStatus(). It's still slow, so if anyone else knows of a quicker way, other than using a service on the networked PC, feel free to let me know! :D : Dean 'Karnatos' Michaud
-
I managed to cut the time it takes down by half. I was being dense, I did not realise that CFileFind can return the filesize without having to call CFile::GetStatus(). It's still slow, so if anyone else knows of a quicker way, other than using a service on the networked PC, feel free to let me know! :D : Dean 'Karnatos' Michaud
No, but if you DO end up writing a service, then you can also use the file change notifications, if you are only wathcing a few directories, and the OS will let you know if anything has changed in the folder, so you won't even really need to periodically scan the folder, you will be told when something is different about it! We ran into this same network issue, and went the 'remote app' route.
-
I managed to cut the time it takes down by half. I was being dense, I did not realise that CFileFind can return the filesize without having to call CFile::GetStatus(). It's still slow, so if anyone else knows of a quicker way, other than using a service on the networked PC, feel free to let me know! :D : Dean 'Karnatos' Michaud
CFileFind might be converting the date and times to c style dates and times. In your case, I doubt it matters much, but I have seen applications where the conversions where taking up over 20% of the process time. You might try using the WIN32 functions directly. Tim Smith I'm going to patent thought. I have yet to see any prior art.