Free File synchronization software status update.. [modified]
-
Yesterday I asked for advice about free file synchronization software here: http://www.codeproject.com/Lounge.aspx?msg=3405379#xx3405379xx[^] Anyways. I tried Microsoft SyncToy and that was worse than the tool I had previously used FreeFileSync http://sourceforge.net/projects/freefilesync/[^] The interface was prettier but it had about the same features and speed but did not work as well. I mean after clicking preview and waiting the hour then clicking run it appeared to start the scan over. I say appeared because after 20 minutes where it had not copied a single file in run mode after the 1 hour I had already waited I gave up. FreeFileSync instantly copies files after it shows you the preview. After this I went searching for RoboCopy and I found RichCopy. http://blogs.technet.com/ken/archive/2009/06/10/build-4-0-216-has-been-posted-to-the-microsoft-download-center.aspx[^] This looked very very promising given its feature set and multithreaded operation however I do not think it will work because of a bug. If I enable the option to include all files with time changes it tries to synchronize files that have the same minute. I assume this is a timestamp issue between filesystems not supporting the same resolution of timestamps. I have seen this before where some tools had a button to ignore 2 second differences. Sadly this tool had no such feature that I could find. So I told it to ignore timestamps and I enabled copy all files that are different sizes expecting this to copy any files with size difference and also copy all files that do not exist on the destination. Sadly it misses the mark on the second part. If the file does not exist on the destination it posts an error and skips the file. :mad: I think I will try robocopy next. [EDIT]I think there may be a workable usage of RichCopy. I first need to sync all files that don't exist on the destination. And then s
-
I can not run this because my linux machines are not permitted on the company network (well not without a lot of regulation and red tape) so the server would not be able to communicate with the source files.
John
-
I can not run this because my linux machines are not permitted on the company network (well not without a lot of regulation and red tape) so the server would not be able to communicate with the source files.
John
If I'm not mistaken, the entire thing is written in C#, including the server component, that's why it needs Suze, for Mono support. Might be worthwhile trying to recompile it for something else...that was another avenue I was going to explore...the source files can be found on SourceForge[^]
-
If I'm not mistaken, the entire thing is written in C#, including the server component, that's why it needs Suze, for Mono support. Might be worthwhile trying to recompile it for something else...that was another avenue I was going to explore...the source files can be found on SourceForge[^]
I will put that on the list of things to try. For now I need to get back to programming.
John
-
Yesterday I asked for advice about free file synchronization software here: http://www.codeproject.com/Lounge.aspx?msg=3405379#xx3405379xx[^] Anyways. I tried Microsoft SyncToy and that was worse than the tool I had previously used FreeFileSync http://sourceforge.net/projects/freefilesync/[^] The interface was prettier but it had about the same features and speed but did not work as well. I mean after clicking preview and waiting the hour then clicking run it appeared to start the scan over. I say appeared because after 20 minutes where it had not copied a single file in run mode after the 1 hour I had already waited I gave up. FreeFileSync instantly copies files after it shows you the preview. After this I went searching for RoboCopy and I found RichCopy. http://blogs.technet.com/ken/archive/2009/06/10/build-4-0-216-has-been-posted-to-the-microsoft-download-center.aspx[^] This looked very very promising given its feature set and multithreaded operation however I do not think it will work because of a bug. If I enable the option to include all files with time changes it tries to synchronize files that have the same minute. I assume this is a timestamp issue between filesystems not supporting the same resolution of timestamps. I have seen this before where some tools had a button to ignore 2 second differences. Sadly this tool had no such feature that I could find. So I told it to ignore timestamps and I enabled copy all files that are different sizes expecting this to copy any files with size difference and also copy all files that do not exist on the destination. Sadly it misses the mark on the second part. If the file does not exist on the destination it posts an error and skips the file. :mad: I think I will try robocopy next. [EDIT]I think there may be a workable usage of RichCopy. I first need to sync all files that don't exist on the destination. And then s
How much are you syncing?!! I use SyncToy to sync my source directory to a USB drive and find it perfectly adequate. Are you sure you have you inclusions/exclusions set up correctly? In my 1.70GB source directory, the preview just took less than 5s and a sync I haven't done in a six months (i.e., lots of changes) took less than 3 mins. (IIRC the first run is a little slower while it sets up whatever "difference database" it uses for a folder pair). Dan
-
How much are you syncing?!! I use SyncToy to sync my source directory to a USB drive and find it perfectly adequate. Are you sure you have you inclusions/exclusions set up correctly? In my 1.70GB source directory, the preview just took less than 5s and a sync I haven't done in a six months (i.e., lots of changes) took less than 3 mins. (IIRC the first run is a little slower while it sets up whatever "difference database" it uses for a folder pair). Dan
The dataset is a little over 1TB containing over 1 million files. The connection is a 800 Kbit network link. Per day 2 to 5 GB is added. I just want it to pick up the addition from the source and mirror that to the dest. Right now the source and dest contain the same files but in the future I will have to trim files from the source but still keep them on the dest.
John
-
The dataset is a little over 1TB containing over 1 million files. The connection is a 800 Kbit network link. Per day 2 to 5 GB is added. I just want it to pick up the addition from the source and mirror that to the dest. Right now the source and dest contain the same files but in the future I will have to trim files from the source but still keep them on the dest.
John
The thing about Sync Framework is that it uses the FileSyncProvider which has one hell of a limitation. If you attempt to sync from one master folder of hundreds of thousands of files in its directory tree, then it takes a very, very long time. You can work round this by syncing from multiple lower level folders, but this is a bit of a faff. As we were using the Sync Framework for a load of file transfers, this became a real pain; so much so that we wrote our own version of the FileSyncProvider which absolutely blazes. Part of the problem is that the FileSyncProvider seems to use the BinaryFormatter to serialize the metadata (updating the base metadata on every change), we used a custom formatter and deferred the write time, which makes it so much faster.
"WPF has many lovers. It's a veritable porn star!" - Josh Smith
As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.
-
The thing about Sync Framework is that it uses the FileSyncProvider which has one hell of a limitation. If you attempt to sync from one master folder of hundreds of thousands of files in its directory tree, then it takes a very, very long time. You can work round this by syncing from multiple lower level folders, but this is a bit of a faff. As we were using the Sync Framework for a load of file transfers, this became a real pain; so much so that we wrote our own version of the FileSyncProvider which absolutely blazes. Part of the problem is that the FileSyncProvider seems to use the BinaryFormatter to serialize the metadata (updating the base metadata on every change), we used a custom formatter and deferred the write time, which makes it so much faster.
"WPF has many lovers. It's a veritable porn star!" - Josh Smith
As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.
Thanks for the detailed explanation of the problem.. I think for the time being I will investigate other options.
John
-
Yesterday I asked for advice about free file synchronization software here: http://www.codeproject.com/Lounge.aspx?msg=3405379#xx3405379xx[^] Anyways. I tried Microsoft SyncToy and that was worse than the tool I had previously used FreeFileSync http://sourceforge.net/projects/freefilesync/[^] The interface was prettier but it had about the same features and speed but did not work as well. I mean after clicking preview and waiting the hour then clicking run it appeared to start the scan over. I say appeared because after 20 minutes where it had not copied a single file in run mode after the 1 hour I had already waited I gave up. FreeFileSync instantly copies files after it shows you the preview. After this I went searching for RoboCopy and I found RichCopy. http://blogs.technet.com/ken/archive/2009/06/10/build-4-0-216-has-been-posted-to-the-microsoft-download-center.aspx[^] This looked very very promising given its feature set and multithreaded operation however I do not think it will work because of a bug. If I enable the option to include all files with time changes it tries to synchronize files that have the same minute. I assume this is a timestamp issue between filesystems not supporting the same resolution of timestamps. I have seen this before where some tools had a button to ignore 2 second differences. Sadly this tool had no such feature that I could find. So I told it to ignore timestamps and I enabled copy all files that are different sizes expecting this to copy any files with size difference and also copy all files that do not exist on the destination. Sadly it misses the mark on the second part. If the file does not exist on the destination it posts an error and skips the file. :mad: I think I will try robocopy next. [EDIT]I think there may be a workable usage of RichCopy. I first need to sync all files that don't exist on the destination. And then s
-
You could try XXCOPY at WWW.XXCOPY.COM[^] In some respects a better RoboCopy.
Candy: Here's the plan: we changes our names, move to a distant island, and disguise ourselves as a family of traveling donkey polishers.
I have actually used that in the past. Just did not think about it for this operation..
John