Creating un-fragmented files
-
Hi There, I am busy with an application that needs to pre-allocate all available space on a clean-formatted dedicated drive to datafiles of a fixed size. At this stage I am doing something like:
while (freediskspace < allocatespace) { // Create new file // Write to file 10 times, (allocatespace/10) bytes at a time // Close the file }Creating the files like this, one after the other, I would expect that there would be minimal (if any) fragmentation on the disk. However, when I then run Microsoft Defragment to analyze the disk it reports that just about all the created files are fragmented and some of them even into 93 fragments. I have experimented with different allocation size, writing different chunk size at a time, etc, but all of them result in fragmented files. My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Thanks OD -
Hi There, I am busy with an application that needs to pre-allocate all available space on a clean-formatted dedicated drive to datafiles of a fixed size. At this stage I am doing something like:
while (freediskspace < allocatespace) { // Create new file // Write to file 10 times, (allocatespace/10) bytes at a time // Close the file }Creating the files like this, one after the other, I would expect that there would be minimal (if any) fragmentation on the disk. However, when I then run Microsoft Defragment to analyze the disk it reports that just about all the created files are fragmented and some of them even into 93 fragments. I have experimented with different allocation size, writing different chunk size at a time, etc, but all of them result in fragmented files. My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Thanks ODWow, that's fascinating. Those results surprise me. I would guess that the severe fragmenting is due to the delayed write cache that Windows uses. It sees you madly writing data, so it waits before it actually commits anything to disk. But when it does, whatever algorithm it uses causes the files to get written haphazardly, resulting in fragmentation. I made all that up as I went along, by the way. Anyway, there are Flush functions that may force the file buffers to disk. If you're using
fopen()there is afflush()that may do what I'm suggesting. There is alsoCFile::Flush()if you swing that way. I think every method of file writing has an associated flush function. I would recommend flushing each file at the end of each loop, maybe at the end of each block write. Just to see what happens. Let us know what you find! -
Hi There, I am busy with an application that needs to pre-allocate all available space on a clean-formatted dedicated drive to datafiles of a fixed size. At this stage I am doing something like:
while (freediskspace < allocatespace) { // Create new file // Write to file 10 times, (allocatespace/10) bytes at a time // Close the file }Creating the files like this, one after the other, I would expect that there would be minimal (if any) fragmentation on the disk. However, when I then run Microsoft Defragment to analyze the disk it reports that just about all the created files are fragmented and some of them even into 93 fragments. I have experimented with different allocation size, writing different chunk size at a time, etc, but all of them result in fragmented files. My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Thanks ODod@ananzi.co.za wrote: My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? There's no guarantee that you can. It's totally up to the file system as to how it splits up a file.
-
Hi There, I am busy with an application that needs to pre-allocate all available space on a clean-formatted dedicated drive to datafiles of a fixed size. At this stage I am doing something like:
while (freediskspace < allocatespace) { // Create new file // Write to file 10 times, (allocatespace/10) bytes at a time // Close the file }Creating the files like this, one after the other, I would expect that there would be minimal (if any) fragmentation on the disk. However, when I then run Microsoft Defragment to analyze the disk it reports that just about all the created files are fragmented and some of them even into 93 fragments. I have experimented with different allocation size, writing different chunk size at a time, etc, but all of them result in fragmented files. My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Thanks ODtry instead of od@ananzi.co.za wrote: // Write to file 10 times, (allocatespace/10) bytes at a time write to memory until done (alloc/realloc all you want) dump the memory to the file at once. Otherwise I do not think you have contol over it, unless you want to write kernel driver.
-
Hi There, I am busy with an application that needs to pre-allocate all available space on a clean-formatted dedicated drive to datafiles of a fixed size. At this stage I am doing something like:
while (freediskspace < allocatespace) { // Create new file // Write to file 10 times, (allocatespace/10) bytes at a time // Close the file }Creating the files like this, one after the other, I would expect that there would be minimal (if any) fragmentation on the disk. However, when I then run Microsoft Defragment to analyze the disk it reports that just about all the created files are fragmented and some of them even into 93 fragments. I have experimented with different allocation size, writing different chunk size at a time, etc, but all of them result in fragmented files. My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Thanks ODod@ananzi.co.za wrote: My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Use FAT32... NTFS should be renamed to SFFS or self fragmenting file system... Actually I think there is a way by setting the size of the file when you create it but I am not sure... John
-
od@ananzi.co.za wrote: My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Use FAT32... NTFS should be renamed to SFFS or self fragmenting file system... Actually I think there is a way by setting the size of the file when you create it but I am not sure... John
Indeed, setting the file size might help. I don't know the api call, but for MFC it is: CFile::SetLength()
A student knows little about a lot. A professor knows a lot about little. I know everything about nothing.
-
Hi There, I am busy with an application that needs to pre-allocate all available space on a clean-formatted dedicated drive to datafiles of a fixed size. At this stage I am doing something like:
while (freediskspace < allocatespace) { // Create new file // Write to file 10 times, (allocatespace/10) bytes at a time // Close the file }Creating the files like this, one after the other, I would expect that there would be minimal (if any) fragmentation on the disk. However, when I then run Microsoft Defragment to analyze the disk it reports that just about all the created files are fragmented and some of them even into 93 fragments. I have experimented with different allocation size, writing different chunk size at a time, etc, but all of them result in fragmented files. My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Thanks ODI dont know about the new versions of NTFS, but in NT4 it reserved the same amount as the file use, as free space after the file. That way a file could grow to double size without being fragmented. But... When the disk got more that half full, the reserved spaces was taked to use, and some files became very fragmented because some of the reserved spaces was quite small... Try to create file at half size, and then expand it to full size, if NTFS still use the same algorithm, this should help :) - Anders Money talks, but all mine ever says is "Goodbye!"
-
Hi There, I am busy with an application that needs to pre-allocate all available space on a clean-formatted dedicated drive to datafiles of a fixed size. At this stage I am doing something like:
while (freediskspace < allocatespace) { // Create new file // Write to file 10 times, (allocatespace/10) bytes at a time // Close the file }Creating the files like this, one after the other, I would expect that there would be minimal (if any) fragmentation on the disk. However, when I then run Microsoft Defragment to analyze the disk it reports that just about all the created files are fragmented and some of them even into 93 fragments. I have experimented with different allocation size, writing different chunk size at a time, etc, but all of them result in fragmented files. My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Thanks ODThat is very fascinating! I offer you one of my scheme although it perhaps sounds not very OK 1.Marked the fragments with fragment serial number. Send the fragments with serial number, the fragment size and the total size. 2.When receiving a fragment analize it and then allocate the total space and the fragment postion. After this, store the fragment in the special position 3.The task is finished when the last fragment is stored. This way maybe minimize the number of fragments
-
Hi There, I am busy with an application that needs to pre-allocate all available space on a clean-formatted dedicated drive to datafiles of a fixed size. At this stage I am doing something like:
while (freediskspace < allocatespace) { // Create new file // Write to file 10 times, (allocatespace/10) bytes at a time // Close the file }Creating the files like this, one after the other, I would expect that there would be minimal (if any) fragmentation on the disk. However, when I then run Microsoft Defragment to analyze the disk it reports that just about all the created files are fragmented and some of them even into 93 fragments. I have experimented with different allocation size, writing different chunk size at a time, etc, but all of them result in fragmented files. My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Thanks ODCreate the file, seek to the position corresponding to how long you want the file and call
SetEndOfFile()to set the new size of the file:hFile = CreateFile(...);
SetFilePointer(hFile, 1048576, NULL, FILE_BEGIN); // 1MB file here
SetEndOfFile(hFile);Of course, you'll need to perform error checking as well, and you'll need to change the value in
SetFilePointer()to be the desired size of the file. I can't gaurantee that it will create unfragmented files (this is really up to Windows), but it's probably the most likely to if there is enough unfragmented space. Just keep in mind that the contents of the file are undefined, so you might want to go back and write over the file to initialise it. Some of the other guys have good suggestions, so try them as well :) Hope this helps,Ryan
"Punctuality is only a virtue for those who aren't smart enough to think of good excuses for being late" John Nichol "Point Of Impact"
-
Indeed, setting the file size might help. I don't know the api call, but for MFC it is: CFile::SetLength()
A student knows little about a lot. A professor knows a lot about little. I know everything about nothing.
SetFilePointer()/SetEndOfFile()for Win32 :)Ryan
"Punctuality is only a virtue for those who aren't smart enough to think of good excuses for being late" John Nichol "Point Of Impact"
-
Hi There, I am busy with an application that needs to pre-allocate all available space on a clean-formatted dedicated drive to datafiles of a fixed size. At this stage I am doing something like:
while (freediskspace < allocatespace) { // Create new file // Write to file 10 times, (allocatespace/10) bytes at a time // Close the file }Creating the files like this, one after the other, I would expect that there would be minimal (if any) fragmentation on the disk. However, when I then run Microsoft Defragment to analyze the disk it reports that just about all the created files are fragmented and some of them even into 93 fragments. I have experimented with different allocation size, writing different chunk size at a time, etc, but all of them result in fragmented files. My question is now very simply this: How do I create files on a clean-formatted drive without them becoming fragmented ?? Thanks ODHi Guys, Thanks for all the comments. Just one or two other things: 1. I can not write the file in one shot, as the files are 250MB in size. I can not allocate 250MB to write to a file, there is just not enough ram. 2. We've found FAT32 wholly unsuitable for big partitions (120GB and up). FAT32 does not even format reliably on these big partitions. Cheers OD