How to copy file much faster?
-
I have to copy files to movable devices using simple file operation such as fread and fwrite in a big loop. And I found some software do such works more quickly. I want to know if I used multithread it would be much faster?If it so,multithread would eate much CPU time resource.So,I think it perhaps no good to adding multithread to copy files. But I also want to know how the softwares can copy more fast? Thanks. GOOD LUCK.
-
I have to copy files to movable devices using simple file operation such as fread and fwrite in a big loop. And I found some software do such works more quickly. I want to know if I used multithread it would be much faster?If it so,multithread would eate much CPU time resource.So,I think it perhaps no good to adding multithread to copy files. But I also want to know how the softwares can copy more fast? Thanks. GOOD LUCK.
-
I have to copy files to movable devices using simple file operation such as fread and fwrite in a big loop. And I found some software do such works more quickly. I want to know if I used multithread it would be much faster?If it so,multithread would eate much CPU time resource.So,I think it perhaps no good to adding multithread to copy files. But I also want to know how the softwares can copy more fast? Thanks. GOOD LUCK.
A friend of mine tried to write the fastest Win32 file copy a few years ago. He used 2 threads and 2 32K buffers (read one then switch and read other while writing the one ) and the Win32 API calls as opposed to the C Library file functions. I don't know if it was the best but it was 4 times faster than using the C library calls, mostly because they use small buffers, and 10 times faster than using Explorer on large files :wtf:
Nothing is exactly what it seems but everything with seems can be unpicked.
-
I have to copy files to movable devices using simple file operation such as fread and fwrite in a big loop. And I found some software do such works more quickly. I want to know if I used multithread it would be much faster?If it so,multithread would eate much CPU time resource.So,I think it perhaps no good to adding multithread to copy files. But I also want to know how the softwares can copy more fast? Thanks. GOOD LUCK.
-
If you're copying multiple files, you could spawn a thread for each file to copy so that, with respect to your program, the copying is taking place in parallel. Judy
There's just one problem here. First and foremost, it depends on WHERE and TO you copy. The HD is far slower than the processor and memory, so the bottleneck is THERE. Copying two files (or more) from the same HD and to the same HD, then it will slow down the operation at least 2x. If the source and destination is different for each file, then yes, you can use threads. Otherwise you're just slowing down things.
-
There's just one problem here. First and foremost, it depends on WHERE and TO you copy. The HD is far slower than the processor and memory, so the bottleneck is THERE. Copying two files (or more) from the same HD and to the same HD, then it will slow down the operation at least 2x. If the source and destination is different for each file, then yes, you can use threads. Otherwise you're just slowing down things.
Depends on how the program is written. If it is a series of CopyFile, CopyFile, CopyFile functions calls, I agree 100% with you. If however, the OP is doing things between each CopyFile call, this may speed his process up by _always_ having data at the HD level waiting to be written while the OPs other code is executing. This is the kind of thing where you need to code it different ways and measure each ways performance in different source files / target disks scenarios. Sometimes you just have to say "sorry, hard drives are slow - you've just got to wait." That's why they made a wait cursor. Judy
-
Depends on how the program is written. If it is a series of CopyFile, CopyFile, CopyFile functions calls, I agree 100% with you. If however, the OP is doing things between each CopyFile call, this may speed his process up by _always_ having data at the HD level waiting to be written while the OPs other code is executing. This is the kind of thing where you need to code it different ways and measure each ways performance in different source files / target disks scenarios. Sometimes you just have to say "sorry, hard drives are slow - you've just got to wait." That's why they made a wait cursor. Judy
Ah, of course. You're absolutely right. One thread can take care of copying and one thread can take care of everything else that needs to be done meanwhile... But then again, you don't need to use synchronus file copy either... You can asynchronous. While the file is copying, the program can do other stuff like readying more data copy or something, as long as it doesn't read or write to the same HD, since that would slow things down.
-
does your software run under a windows os? if so why don't you use the copyfile api from kernel32.dll? cheers markus
-
A friend of mine tried to write the fastest Win32 file copy a few years ago. He used 2 threads and 2 32K buffers (read one then switch and read other while writing the one ) and the Win32 API calls as opposed to the C Library file functions. I don't know if it was the best but it was 4 times faster than using the C library calls, mostly because they use small buffers, and 10 times faster than using Explorer on large files :wtf:
Nothing is exactly what it seems but everything with seems can be unpicked.
-
If you're copying multiple files, you could spawn a thread for each file to copy so that, with respect to your program, the copying is taking place in parallel. Judy