The year, 2006. The problem, we still can't copy files reliably
-
Paul Watson wrote:
And in the past I have found Windows handles fewer larger files better than many thousands of smaller files.
True, that. Next time you might want to just use XCOPY or an equivalent utility to avoid all of the extra busywork the GUI copy goes through. XCOPY also has the advantage of providing an option to keep working when a single file fails:
> XCOPY /S /I /C /Y <sourcedir> <destdir>
(that'll copy a directory recursively, create the destination if it doesn't already exist, keep going if a single file fails, and not bug you about overwriting anything) And if you're copying to/from a network drive, the
/Zoption can be a huge frustration-reducer as well. :)---- Scripts i’ve known... CPhog 1.8.2 - make CP better. Forum Bookmark 0.2.5 - bookmark forum posts on Pensieve Print forum 0.1.2 - printer-friendly forums Expand all 1.0 - Expand all messages In-place Delete 1.0 - AJAX-style post delete Syntax 0.1 - Syntax highlighting for code blocks in the forums
-
The task: Copy 2,300 files equaling 16gb from one laptop HD to an external HD. This should be easy, just copy the parent directory and paste in the destination directory. This I do. The dialog takes 3 minutes just to appear and then another 5 minutes to calculate how long it is going to copy all the files. It estimates it will take 89 minutes to copy which is fine by me. So I leave it for 30 minutes, come back and find the dialog is now saying 120 minutes. Hmmm. OK, maybe 2,300 files is too much for the copy app. So I decide to ZIP the files (without compression), copy the resulting 13gig file and then unzip on the destination directory. It takes 30 minutes to ZIP, 20 minutes to copy the file across and then I start to unzip. 30 minutes in it is about 15% through, WTF? OK, who cares, just leave it while I am off moving a boat. I come back 3 hours later and my girlfriend, who was told "not to touch the shiny laptop", is pointing at the external HD saying "It is making a clicking sound." Oh bugger. I lift the lid and see a screen full of "Delayed write failure" dialogs. 50 clicks later WinZIP tells me at 97% it failed. *sigh* We have computers more powerful than what launched man to the moon and yet we can't bleeding well reliably copy files from one HD to another.
regards, Paul Watson Ireland & South Africa
Shog9 wrote:
And with that, Paul closed his browser, sipped his herbal tea, fixed the flower in his hair, and smiled brightly at the multitude of cute, furry animals flocking around the grassy hillside where he sat coding Ruby on his Mac...
I never do massive copies with Explorer, because the GUI preparation steps take forever. I really like Robocopy, which has built-in support for retrying when errors happen. It comes with Vista, I think it's in a resource kit for XP.
--Mike-- Visual C++ MVP :cool: LINKS~! Ericahist | PimpFish | CP SearchBar v3.0 | C++ Forum FAQ Ford, what's this fish doing in my ear?
-
Paul Watson wrote:
And in the past I have found Windows handles fewer larger files better than many thousands of smaller files.
True, that. Next time you might want to just use XCOPY or an equivalent utility to avoid all of the extra busywork the GUI copy goes through. XCOPY also has the advantage of providing an option to keep working when a single file fails:
> XCOPY /S /I /C /Y <sourcedir> <destdir>
(that'll copy a directory recursively, create the destination if it doesn't already exist, keep going if a single file fails, and not bug you about overwriting anything) And if you're copying to/from a network drive, the
/Zoption can be a huge frustration-reducer as well. :)---- Scripts i’ve known... CPhog 1.8.2 - make CP better. Forum Bookmark 0.2.5 - bookmark forum posts on Pensieve Print forum 0.1.2 - printer-friendly forums Expand all 1.0 - Expand all messages In-place Delete 1.0 - AJAX-style post delete Syntax 0.1 - Syntax highlighting for code blocks in the forums
Shog9 wrote:
extra busywork the GUI copy goes through
Yes, indeed. I'm always reminded of a line from that immortal Mel Brooks epic Spaceballs[^]:
Why are we always preparing? Just go!
Software Zen:
delete this; -
The task: Copy 2,300 files equaling 16gb from one laptop HD to an external HD. This should be easy, just copy the parent directory and paste in the destination directory. This I do. The dialog takes 3 minutes just to appear and then another 5 minutes to calculate how long it is going to copy all the files. It estimates it will take 89 minutes to copy which is fine by me. So I leave it for 30 minutes, come back and find the dialog is now saying 120 minutes. Hmmm. OK, maybe 2,300 files is too much for the copy app. So I decide to ZIP the files (without compression), copy the resulting 13gig file and then unzip on the destination directory. It takes 30 minutes to ZIP, 20 minutes to copy the file across and then I start to unzip. 30 minutes in it is about 15% through, WTF? OK, who cares, just leave it while I am off moving a boat. I come back 3 hours later and my girlfriend, who was told "not to touch the shiny laptop", is pointing at the external HD saying "It is making a clicking sound." Oh bugger. I lift the lid and see a screen full of "Delayed write failure" dialogs. 50 clicks later WinZIP tells me at 97% it failed. *sigh* We have computers more powerful than what launched man to the moon and yet we can't bleeding well reliably copy files from one HD to another.
regards, Paul Watson Ireland & South Africa
Shog9 wrote:
And with that, Paul closed his browser, sipped his herbal tea, fixed the flower in his hair, and smiled brightly at the multitude of cute, furry animals flocking around the grassy hillside where he sat coding Ruby on his Mac...
That is yet another feature I have enjoyed in Vista. They upgraded a few features in the file copy area that makes it much more usable. I copy large groups of files easily and with prompts in case of errors that allow me to continue. Much nicer :)
Rocky <>< Latest Code Blog Post: OpenID/CardSpace - Is it time? Latest Tech Blog Post: Scratch: fun for all ages for free!
-
That is yet another feature I have enjoyed in Vista. They upgraded a few features in the file copy area that makes it much more usable. I copy large groups of files easily and with prompts in case of errors that allow me to continue. Much nicer :)
Rocky <>< Latest Code Blog Post: OpenID/CardSpace - Is it time? Latest Tech Blog Post: Scratch: fun for all ages for free!
Cool though I think Chris was complaining about Vista's copy just the other week.
regards, Paul Watson Ireland & South Africa
Shog9 wrote:
And with that, Paul closed his browser, sipped his herbal tea, fixed the flower in his hair, and smiled brightly at the multitude of cute, furry animals flocking around the grassy hillside where he sat coding Ruby on his Mac...
-
Thanks Marc, I'll give it a try. Would MS be sued if it improved its file copy routines?
regards, Paul Watson Ireland & South Africa
Shog9 wrote:
And with that, Paul closed his browser, sipped his herbal tea, fixed the flower in his hair, and smiled brightly at the multitude of cute, furry animals flocking around the grassy hillside where he sat coding Ruby on his Mac...
You can't sell a OS on "copying files just got better".
Developers, Developers, Developers, Developers, Developers, Developers, Velopers, Develprs, Developers!
We are a big screwed up dysfunctional psychotic happy family - some more screwed up, others more happy, but everybody's psychotic joint venture definition of CP
Linkify!|Fold With Us! -
So the OS/X filecopy sucks as much as the Windows version? Or are you slumming... ;P Still, sounds like you're blaming a hardware failure on software. Oh, and since it has to be written, uncompressed, to the external harddrive, you're not really saving anything by zipping it. That only helps when you can move the thing compressed across the bottleneck and then uncompress on the other side.
---- Scripts i’ve known... CPhog 1.8.2 - make CP better. Forum Bookmark 0.2.5 - bookmark forum posts on Pensieve Print forum 0.1.2 - printer-friendly forums Expand all 1.0 - Expand all messages In-place Delete 1.0 - AJAX-style post delete Syntax 0.1 - Syntax highlighting for code blocks in the forums
Heh - I was going to suggest to Paul that OS X would probably do a better job - I backed up my home directory on my iBook a couple of weeks ago (the motherboard/wi-fi is hosed, new motherboard being fitted) - just drag-dropped the directory onto a USB drive and it copied 40GB in 180,000+ files quite happily. It took a while (an hour or so? Maybe longer), but it just worked...
-
Paul Watson wrote:
And in the past I have found Windows handles fewer larger files better than many thousands of smaller files.
True, that. Next time you might want to just use XCOPY or an equivalent utility to avoid all of the extra busywork the GUI copy goes through. XCOPY also has the advantage of providing an option to keep working when a single file fails:
> XCOPY /S /I /C /Y <sourcedir> <destdir>
(that'll copy a directory recursively, create the destination if it doesn't already exist, keep going if a single file fails, and not bug you about overwriting anything) And if you're copying to/from a network drive, the
/Zoption can be a huge frustration-reducer as well. :)---- Scripts i’ve known... CPhog 1.8.2 - make CP better. Forum Bookmark 0.2.5 - bookmark forum posts on Pensieve Print forum 0.1.2 - printer-friendly forums Expand all 1.0 - Expand all messages In-place Delete 1.0 - AJAX-style post delete Syntax 0.1 - Syntax highlighting for code blocks in the forums
Thanks Shog9 from outer space. I have just kicked that command off for a copy of 80gb from an external HD to a networked drive in a Windows XP VM in Parallels on Mac OS X... if this works easily I'll eat your famous pickle.
regards, Paul Watson Ireland & South Africa
Shog9 wrote:
And with that, Paul closed his browser, sipped his herbal tea, fixed the flower in his hair, and smiled brightly at the multitude of cute, furry animals flocking around the grassy hillside where he sat coding Ruby on his Mac...
-
Paul Watson wrote:
And in the past I have found Windows handles fewer larger files better than many thousands of smaller files.
True, that. Next time you might want to just use XCOPY or an equivalent utility to avoid all of the extra busywork the GUI copy goes through. XCOPY also has the advantage of providing an option to keep working when a single file fails:
> XCOPY /S /I /C /Y <sourcedir> <destdir>
(that'll copy a directory recursively, create the destination if it doesn't already exist, keep going if a single file fails, and not bug you about overwriting anything) And if you're copying to/from a network drive, the
/Zoption can be a huge frustration-reducer as well. :)---- Scripts i’ve known... CPhog 1.8.2 - make CP better. Forum Bookmark 0.2.5 - bookmark forum posts on Pensieve Print forum 0.1.2 - printer-friendly forums Expand all 1.0 - Expand all messages In-place Delete 1.0 - AJAX-style post delete Syntax 0.1 - Syntax highlighting for code blocks in the forums
Yeah, I use XCOPY all the time for copying large numbers of files. The gui copy function is horrible dealing with slow connections or large numbers of files.
Using the GridView is like trying to explain to someone else how to move a third person's hands in order to tie your shoelaces for you. -Chris Maunder
-
The task: Copy 2,300 files equaling 16gb from one laptop HD to an external HD. This should be easy, just copy the parent directory and paste in the destination directory. This I do. The dialog takes 3 minutes just to appear and then another 5 minutes to calculate how long it is going to copy all the files. It estimates it will take 89 minutes to copy which is fine by me. So I leave it for 30 minutes, come back and find the dialog is now saying 120 minutes. Hmmm. OK, maybe 2,300 files is too much for the copy app. So I decide to ZIP the files (without compression), copy the resulting 13gig file and then unzip on the destination directory. It takes 30 minutes to ZIP, 20 minutes to copy the file across and then I start to unzip. 30 minutes in it is about 15% through, WTF? OK, who cares, just leave it while I am off moving a boat. I come back 3 hours later and my girlfriend, who was told "not to touch the shiny laptop", is pointing at the external HD saying "It is making a clicking sound." Oh bugger. I lift the lid and see a screen full of "Delayed write failure" dialogs. 50 clicks later WinZIP tells me at 97% it failed. *sigh* We have computers more powerful than what launched man to the moon and yet we can't bleeding well reliably copy files from one HD to another.
regards, Paul Watson Ireland & South Africa
Shog9 wrote:
And with that, Paul closed his browser, sipped his herbal tea, fixed the flower in his hair, and smiled brightly at the multitude of cute, furry animals flocking around the grassy hillside where he sat coding Ruby on his Mac...
Even worse is when it doesn't tell you that the copy has failed. I recently installed a new larger HDD in my computer, so I moved all my music over to it. I thought everything had gone fine. I looked in one of my folders and saw some files with weird names (like temp files). I figured that they had been artifacts of the encoding process so I deleted them. Then I looked closer and found that I had deleted actual songs, but that the copy procedure in Windows Explorer never got around to renaming them back to the original names!
-- Marcus Kwok
-
The task: Copy 2,300 files equaling 16gb from one laptop HD to an external HD. This should be easy, just copy the parent directory and paste in the destination directory. This I do. The dialog takes 3 minutes just to appear and then another 5 minutes to calculate how long it is going to copy all the files. It estimates it will take 89 minutes to copy which is fine by me. So I leave it for 30 minutes, come back and find the dialog is now saying 120 minutes. Hmmm. OK, maybe 2,300 files is too much for the copy app. So I decide to ZIP the files (without compression), copy the resulting 13gig file and then unzip on the destination directory. It takes 30 minutes to ZIP, 20 minutes to copy the file across and then I start to unzip. 30 minutes in it is about 15% through, WTF? OK, who cares, just leave it while I am off moving a boat. I come back 3 hours later and my girlfriend, who was told "not to touch the shiny laptop", is pointing at the external HD saying "It is making a clicking sound." Oh bugger. I lift the lid and see a screen full of "Delayed write failure" dialogs. 50 clicks later WinZIP tells me at 97% it failed. *sigh* We have computers more powerful than what launched man to the moon and yet we can't bleeding well reliably copy files from one HD to another.
regards, Paul Watson Ireland & South Africa
Shog9 wrote:
And with that, Paul closed his browser, sipped his herbal tea, fixed the flower in his hair, and smiled brightly at the multitude of cute, furry animals flocking around the grassy hillside where he sat coding Ruby on his Mac...
Turn off power management on your USB ports for starters. Then get syncbackse. I've been able to move 500 gigs of data 4 times in a row with it and never had a single problem. Something else to consider would be putting the external drive on another machine and mapping a drive. Don't ask me why but I just think this works better. I get some weird things happening when using USB at times... SyncBackSE would copy anything to anywhere and it just rules.
-
Thanks Shog9 from outer space. I have just kicked that command off for a copy of 80gb from an external HD to a networked drive in a Windows XP VM in Parallels on Mac OS X... if this works easily I'll eat your famous pickle.
regards, Paul Watson Ireland & South Africa
Shog9 wrote:
And with that, Paul closed his browser, sipped his herbal tea, fixed the flower in his hair, and smiled brightly at the multitude of cute, furry animals flocking around the grassy hillside where he sat coding Ruby on his Mac...
> I have just kicked that command off for a copy of 80gb from an external HD to a networked drive As an alternative to xcopy with all the trimmings, I've come to rely on robocopy.exe from the Win2K (?) resource kit. The nice thing about it is that it can retry failed files, but better yet, if you re-run the entire job, it'll skip identical files. I now use it to regularly back up about 100GB worth of data off of my main PC to my secondary one across my LAN with nothing more than a batch file. Since it only copies changed files, updates can typically be done within half an hour, even through my LAN. I've found that more often than not, using the GUI to copy any number of files reliably is asking for trouble.