Finding files
-
When you first format a NTFS volume, some 12% of that is pre-allocted for the MFT ( which FindFile's can't find ). Try running your test again but manualy open "C:\$MFT" and "C:\$MFTMirr" and account for the sizes ;) The other NTFS specific file are small enough to forget about.
Waldermort
WalderMort wrote:
Try running your test again but manualy open "C:\$MFT" and "C:\$MFTMirr" and account for the sizes
c:\$mft = 125.5MB c:\$mftmirr = 4KB
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
-
WalderMort wrote:
Try running your test again but manualy open "C:\$MFT" and "C:\$MFTMirr" and account for the sizes
c:\$mft = 125.5MB c:\$mftmirr = 4KB
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
Well that accounts for some 5%... That's got me thinking now, what does windows need all that space for? Just out of curiousity, how does your test work?
Waldermort
-
Well that accounts for some 5%... That's got me thinking now, what does windows need all that space for? Just out of curiousity, how does your test work?
Waldermort
WalderMort wrote:
Just out of curiousity, how does your test work?
Nothing special. I just used a
CFileFindobject to round up the sizes of all the files on the C: drive.
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
-
WalderMort wrote:
Just out of curiousity, how does your test work?
Nothing special. I just used a
CFileFindobject to round up the sizes of all the files on the C: drive.
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
ahh, did you take into account that file size on disk is usually higher than that CFileFind reports? Also many of the smaller files are stored inside the MFT rather than waste cluster space. Through tests, I have found some 30% of my C: volume contains files that CFileFind cannot find. I think the only way to find out what those files are is to read the MFT directly and compare to the results of CFileFind.
Waldermort
-
ahh, did you take into account that file size on disk is usually higher than that CFileFind reports? Also many of the smaller files are stored inside the MFT rather than waste cluster space. Through tests, I have found some 30% of my C: volume contains files that CFileFind cannot find. I think the only way to find out what those files are is to read the MFT directly and compare to the results of CFileFind.
Waldermort
WalderMort wrote:
did you take into account that file size on disk is usually higher than that CFileFind reports?
Yes, I accounted for slack space.
WalderMort wrote:
Also many of the smaller files are stored inside the MFT rather than waste cluster space.
I thought of that, but I've got files that are a few bytes in length, and they are found by
CFileFind.
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
-
WalderMort wrote:
did you take into account that file size on disk is usually higher than that CFileFind reports?
Yes, I accounted for slack space.
WalderMort wrote:
Also many of the smaller files are stored inside the MFT rather than waste cluster space.
I thought of that, but I've got files that are a few bytes in length, and they are found by
CFileFind.
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
DavidCrow wrote:
I thought of that, but I've got files that are a few bytes in length, and they are found by CFileFind.
CfileFind will return the size regardless of which part of the disk it is stored. A more realistic method would be to pass each file into DeviceIoControl and count the cluster/fragment usage. 0 fragments indicates that it is stored in the MFT. Also, I have just accounted for a further 4GB by giving myself access rights to the "System Volume Information". If you know of a way to do this programaticaly I would be glad to hear. My next step is to call CreateFile for every file/folder found, if it fails then I should attempt to adjust the access rights. This is where CFindFile fails the most, it gives no indication of where it skipped a file/directory simply because it had no rights.
Waldermort
-
DavidCrow wrote:
I thought of that, but I've got files that are a few bytes in length, and they are found by CFileFind.
CfileFind will return the size regardless of which part of the disk it is stored. A more realistic method would be to pass each file into DeviceIoControl and count the cluster/fragment usage. 0 fragments indicates that it is stored in the MFT. Also, I have just accounted for a further 4GB by giving myself access rights to the "System Volume Information". If you know of a way to do this programaticaly I would be glad to hear. My next step is to call CreateFile for every file/folder found, if it fails then I should attempt to adjust the access rights. This is where CFindFile fails the most, it gives no indication of where it skipped a file/directory simply because it had no rights.
Waldermort
WalderMort wrote:
...giving myself access rights to the "System Volume Information".
How?
WalderMort wrote:
My next step is to call CreateFile for every file/folder found, if it fails then I should attempt to adjust the access rights.
Going back to your "...for each file I then perform a few tasks." comment, do you need to perform said tasks on those files for which you have insufficient access?
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
-
WalderMort wrote:
...giving myself access rights to the "System Volume Information".
How?
WalderMort wrote:
My next step is to call CreateFile for every file/folder found, if it fails then I should attempt to adjust the access rights.
Going back to your "...for each file I then perform a few tasks." comment, do you need to perform said tasks on those files for which you have insufficient access?
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
DavidCrow wrote:
How?
A simple command prompt[^] But this method is not temporary.
DavidCrow wrote:
comment, do you need to perform said tasks on those files for which you have insufficient access?
I'm creating a defragger, so I would prefer to have access to as many files as possible, especially those on the windows install volume.
Waldermort
-
DavidCrow wrote:
How?
A simple command prompt[^] But this method is not temporary.
DavidCrow wrote:
comment, do you need to perform said tasks on those files for which you have insufficient access?
I'm creating a defragger, so I would prefer to have access to as many files as possible, especially those on the windows install volume.
Waldermort
WalderMort wrote:
I'm creating a defragger,
There might be something here you can use.
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
-
WalderMort wrote:
I'm creating a defragger,
There might be something here you can use.
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
Thanks, that may come in handy. At a glance it works similar to my method, but I have also built in a DiskMap viewer.
Waldermort
-
DavidCrow wrote:
I thought of that, but I've got files that are a few bytes in length, and they are found by CFileFind.
CfileFind will return the size regardless of which part of the disk it is stored. A more realistic method would be to pass each file into DeviceIoControl and count the cluster/fragment usage. 0 fragments indicates that it is stored in the MFT. Also, I have just accounted for a further 4GB by giving myself access rights to the "System Volume Information". If you know of a way to do this programaticaly I would be glad to hear. My next step is to call CreateFile for every file/folder found, if it fails then I should attempt to adjust the access rights. This is where CFindFile fails the most, it gives no indication of where it skipped a file/directory simply because it had no rights.
Waldermort
WalderMort wrote:
Also, I have just accounted for a further 4GB by giving myself access rights to the "System Volume Information".
Same here. It now seems that my numbers have gone the other way.
GetDiskFreeSpaceEx()reports 28GB used bytes, but I am counting 30.3GB bytes used by files, a difference of 231.4MB. Hmmm...
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
-
WalderMort wrote:
Also, I have just accounted for a further 4GB by giving myself access rights to the "System Volume Information".
Same here. It now seems that my numbers have gone the other way.
GetDiskFreeSpaceEx()reports 28GB used bytes, but I am counting 30.3GB bytes used by files, a difference of 231.4MB. Hmmm...
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
Haha, sounds like your loop could do with a little refining. Try the DeviceIoApproach as I mentioned earlier, you should get some more realistic values.
Waldermort
-
Haha, sounds like your loop could do with a little refining. Try the DeviceIoApproach as I mentioned earlier, you should get some more realistic values.
Waldermort
WalderMort wrote:
Try the DeviceIoApproach as I mentioned earlier...
Which control code did you have in mind?
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
-
WalderMort wrote:
Try the DeviceIoApproach as I mentioned earlier...
Which control code did you have in mind?
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
you would need to use the FSCTL_GET_EXTENT_BUFFERS. The trouble is with this operation is that you don't know how large the buffer should be. I wrapped the call up in a class with the buffer as a static member. Each time the buffer is too small, I double the size and keep calling until successful. The Extent count indicates how many fragments the file has, 0 indicates that it is located in the MFT. Each extent returned will then tell you how many clusters each fragment has. By adding these and multiplying by your drives 'bytes per cluster' you will find out exactly how much space a particular file/folder is taking. Also be aware of NT's compressed runs which is indicated by a startLCN of -1. I still haven't managed to get my head around what this actualy means.
Waldermort
-
DavidCrow wrote:
I thought of that, but I've got files that are a few bytes in length, and they are found by CFileFind.
CfileFind will return the size regardless of which part of the disk it is stored. A more realistic method would be to pass each file into DeviceIoControl and count the cluster/fragment usage. 0 fragments indicates that it is stored in the MFT. Also, I have just accounted for a further 4GB by giving myself access rights to the "System Volume Information". If you know of a way to do this programaticaly I would be glad to hear. My next step is to call CreateFile for every file/folder found, if it fails then I should attempt to adjust the access rights. This is where CFindFile fails the most, it gives no indication of where it skipped a file/directory simply because it had no rights.
Waldermort
WalderMort wrote:
If you know of a way to do this programaticaly I would be glad to hear.
I can 'get' the access rights, but haven't quite got the 'set' part down yet.
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
-
WalderMort wrote:
If you know of a way to do this programaticaly I would be glad to hear.
I can 'get' the access rights, but haven't quite got the 'set' part down yet.
"A good athlete is the result of a good and worthy opponent." - David Crow
"To have a respect for ourselves guides our morals; to have deference for others governs our manners." - Laurence Sterne
You still working on that, lol. To be honest, I didn't touch that part of my code yet, I'm still working on my class hierarchy. I was thinking to use ShellExec() with the "runas" flag, change the security settings and store them into a file, which I believe cacls is able to do. This should, in theory, allow the origional settings to be replaced once complete. But like I said, I didn't try this yet and probably won't for another week or so. If you manage to do it using another method, I would be interested to see how.
Waldermort