Working from home... need to backup a large set of files each day.
-
I did something like that with an overseas connection a few years ago between Houston and France. I did some testing with the connection using various strategies. We tried copying files in native form and also dumped into 'split' zip files. We used a scheduled batch file to do the hard work. The zip file strategy got the best throughput if we were copying all files but our files typically had less than 1% change per 24 hour period... so we used "xcopy /d" (which does not delete files on the remote side which get deleted on the local side). There was a considerable difference in elapsed time on whether we used xcopy on the sending end or receiving end. I've forgotten now which was faster but I think we did one of the following:
xcopy /a /d /s /e /c /i /q /h /r /k /o /x /y D:\localfiles \\remote\blahblah
xcopy /a /d /s /e /c /i /q /h /r /k /o /x /y \\remote\blahblah D:\localfiles
I'm retired. There's a nap for that... - Harvey
For future reference, you should look into something like the rsync protocol (usually available by default on linux, but there are versions for windows too). This is designed to synchronise large files over slow connections, and hence exchanges CPU power for line speed, by hashing the contents of large files and only sending changed blocks within files: thus, if you have a 4Gb file where only 10 bytes gets added every day, instead of transmitting the whole file each time it is updated, it will only sync the block(s) that contain the changed bytes. I've used this to keep many Gb of data backed up off-site when I only had a 512k upload speed to do it. The initial backup can take a long time, but once its done...
-
Hello all, We have some big files that need to be sent from a remote computer to the company server each day. The work computer has a 10 MB DSL connection and the server has an optical fiber (50MB) connection. The remote worker is not tech savvy. I can imagine three ways of handling that: 1. Files stored in the local computer and send them each day to the remote server (via a scheduled backup that will run automatically at a certain hour). The computer user can shut the computer down at the evening. 2. Files stored in the local computer synchronized automatically using a kind of "Google Drive" that the server has natively included. 3. Files in the remote server and access them remotely via WebDAV or similar. Knowing that... what would you choose? And why? Thank you very much! :thumbsup:
www.robotecnik.com[^] - robots, CNC and PLC programming
Have you considered using Dropbox or one of the other "cloud" drives? They keep the content synchronized pretty continuously.
-
Hello all, We have some big files that need to be sent from a remote computer to the company server each day. The work computer has a 10 MB DSL connection and the server has an optical fiber (50MB) connection. The remote worker is not tech savvy. I can imagine three ways of handling that: 1. Files stored in the local computer and send them each day to the remote server (via a scheduled backup that will run automatically at a certain hour). The computer user can shut the computer down at the evening. 2. Files stored in the local computer synchronized automatically using a kind of "Google Drive" that the server has natively included. 3. Files in the remote server and access them remotely via WebDAV or similar. Knowing that... what would you choose? And why? Thank you very much! :thumbsup:
www.robotecnik.com[^] - robots, CNC and PLC programming
RDP is an excellent tool BUT if you decide to allow the remote working then have an FTP Server (I use FileZilla) at the hosting end and use something like "winscp" to feed the modified files back winscp runs a sync across the ftp protocol and therefore only transfer modified files and can be run multiple times to make sure that transfers aborted due to PC restarts get completed you can run over ssl/tls and providing you have decent usernames and passwords then it would be reasonably secure for added/alternative security factor in a VPN FileZilla would also allow you to set different transfer rates for different times of day so that the DSL wouldn't be maxed out at busy times
-
Have you considered using Dropbox or one of the other "cloud" drives? They keep the content synchronized pretty continuously.
-
RDP is an excellent tool BUT if you decide to allow the remote working then have an FTP Server (I use FileZilla) at the hosting end and use something like "winscp" to feed the modified files back winscp runs a sync across the ftp protocol and therefore only transfer modified files and can be run multiple times to make sure that transfers aborted due to PC restarts get completed you can run over ssl/tls and providing you have decent usernames and passwords then it would be reasonably secure for added/alternative security factor in a VPN FileZilla would also allow you to set different transfer rates for different times of day so that the DSL wouldn't be maxed out at busy times
I'll have to look for the VPN thing (which sounds good when it increases security), but, the RDP (VNC, TeamViewer, the one that comes with windows...)... will help a lot and then, no files need to be stored into the remote computer as everything will stay in the server always... Thank you!
www.robotecnik.com[^] - robots, CNC and PLC programming
-
Hello all, We have some big files that need to be sent from a remote computer to the company server each day. The work computer has a 10 MB DSL connection and the server has an optical fiber (50MB) connection. The remote worker is not tech savvy. I can imagine three ways of handling that: 1. Files stored in the local computer and send them each day to the remote server (via a scheduled backup that will run automatically at a certain hour). The computer user can shut the computer down at the evening. 2. Files stored in the local computer synchronized automatically using a kind of "Google Drive" that the server has natively included. 3. Files in the remote server and access them remotely via WebDAV or similar. Knowing that... what would you choose? And why? Thank you very much! :thumbsup:
www.robotecnik.com[^] - robots, CNC and PLC programming
-
Step 1. Calculate how many floppy disks you will need...
- I would love to change the world, but they won’t give me the source code.
-
A 10MB DSL connection is enough to RDP with. Unless they're doing multimedia work... problem solved.
Jeremy Falcon
Agreed. It's how I work from home - citrix connection to remote terminal which RDPs onto my desktop machine (and it could just as equally be onto a VM) and then no files ever have to get to my home computer and none to transfer back, everything is kept secure in the company network. Both screens on my office PC visible on both screens at home, works very well.
-
NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO! :laugh: :laugh: :laugh: :laugh: :laugh:
www.robotecnik.com[^] - robots, CNC and PLC programming
-
Hello all, We have some big files that need to be sent from a remote computer to the company server each day. The work computer has a 10 MB DSL connection and the server has an optical fiber (50MB) connection. The remote worker is not tech savvy. I can imagine three ways of handling that: 1. Files stored in the local computer and send them each day to the remote server (via a scheduled backup that will run automatically at a certain hour). The computer user can shut the computer down at the evening. 2. Files stored in the local computer synchronized automatically using a kind of "Google Drive" that the server has natively included. 3. Files in the remote server and access them remotely via WebDAV or similar. Knowing that... what would you choose? And why? Thank you very much! :thumbsup:
www.robotecnik.com[^] - robots, CNC and PLC programming
If they're not tech savvy, then they're probably using windows (you didn't say), so I'd suggest OneDrive if the files have to available locally to the user. With that, the files are locally cached and automatically synced to the cloud as they are changed. Just don't let the user put files elsewhere on their system, so maybe move all the microsoft directories to it. You'll get all their personal crap too, but hey, they're getting free backups of their person stuff, so they shouldn't complain. If the files don't need to be local with the user, then I agree with others -- have them remote desktop into a system at the company. That way you limit their person stuff, and you get to control what's installed on that machine, when (if) its virus checked, etc. If they're using a fruit, then you'll have to find an equivalent for that OS.
We can program with only 1's, but if all you've got are zeros, you've got nothing.
-
If they're not tech savvy, then they're probably using windows (you didn't say), so I'd suggest OneDrive if the files have to available locally to the user. With that, the files are locally cached and automatically synced to the cloud as they are changed. Just don't let the user put files elsewhere on their system, so maybe move all the microsoft directories to it. You'll get all their personal crap too, but hey, they're getting free backups of their person stuff, so they shouldn't complain. If the files don't need to be local with the user, then I agree with others -- have them remote desktop into a system at the company. That way you limit their person stuff, and you get to control what's installed on that machine, when (if) its virus checked, etc. If they're using a fruit, then you'll have to find an equivalent for that OS.
We can program with only 1's, but if all you've got are zeros, you've got nothing.
-
Nah, the remote option will do it... Even the onedrive, google drive... options are there, there's too much data and it will be much easier to handle it by remoting. Thank you!
www.robotecnik.com[^] - robots, CNC and PLC programming
I'd be impressed if an unsavvy user managed to touch 3 GB worth of files faster than it could be backed up. If they will alter 3 GB worth of files every day, and you need them backed up every night, and they can turn off their computer, then there really isn't any choice -- the files have to live on a server you control at your company or you won't reliably have backups of them.
We can program with only 1's, but if all you've got are zeros, you've got nothing.
-
I'd be impressed if an unsavvy user managed to touch 3 GB worth of files faster than it could be backed up. If they will alter 3 GB worth of files every day, and you need them backed up every night, and they can turn off their computer, then there really isn't any choice -- the files have to live on a server you control at your company or you won't reliably have backups of them.
We can program with only 1's, but if all you've got are zeros, you've got nothing.
Non tech-savvy people have strange super powers that grant them the ability to destroy everything in seconds... You know... like leaving a bottle of tequila on the backspace key of a computer... :laugh:
www.robotecnik.com[^] - robots, CNC and PLC programming
-
Hello all, We have some big files that need to be sent from a remote computer to the company server each day. The work computer has a 10 MB DSL connection and the server has an optical fiber (50MB) connection. The remote worker is not tech savvy. I can imagine three ways of handling that: 1. Files stored in the local computer and send them each day to the remote server (via a scheduled backup that will run automatically at a certain hour). The computer user can shut the computer down at the evening. 2. Files stored in the local computer synchronized automatically using a kind of "Google Drive" that the server has natively included. 3. Files in the remote server and access them remotely via WebDAV or similar. Knowing that... what would you choose? And why? Thank you very much! :thumbsup:
www.robotecnik.com[^] - robots, CNC and PLC programming
Use Resilio Sync and forget about it. Resilio: File Synchronization Software for Enterprise, Business & Individuals[^]
-
And think about security. The company's data remains at the company, it is never stored at the remote worker's home. For that alone, I prefer RDP for all remote workers by FAR.
Yeah, that too.
Jeremy Falcon
-
Use Resilio Sync and forget about it. Resilio: File Synchronization Software for Enterprise, Business & Individuals[^]
It's a good suggestion, but given there will be only two positions... all the sending/receiving would be done between two computers so the p2p magic would be reduced to a kind of dropbox... and this solution is already in the server the company has. Thank you!
www.robotecnik.com[^] - robots, CNC and PLC programming
-
Non tech-savvy people have strange super powers that grant them the ability to destroy everything in seconds... You know... like leaving a bottle of tequila on the backspace key of a computer... :laugh:
www.robotecnik.com[^] - robots, CNC and PLC programming
True. And there's far better places for that bottle of Tequila, like filling our glasses, something a lot of non tech-savvy people do seem to understand :) Good luck with the backup strategy for that user. Something tells me you're going to need it.. or a lot of that said Tequila :)
We can program with only 1's, but if all you've got are zeros, you've got nothing.
-
For future reference, you should look into something like the rsync protocol (usually available by default on linux, but there are versions for windows too). This is designed to synchronise large files over slow connections, and hence exchanges CPU power for line speed, by hashing the contents of large files and only sending changed blocks within files: thus, if you have a 4Gb file where only 10 bytes gets added every day, instead of transmitting the whole file each time it is updated, it will only sync the block(s) that contain the changed bytes. I've used this to keep many Gb of data backed up off-site when I only had a 512k upload speed to do it. The initial backup can take a long time, but once its done...
We were not interested in continuous updates, but periodic updates upon demand. My understanding of rsync is that it runs continuously. Also, we weren't dealing with large files, but a mix, mostly small files.
I'm retired. There's a nap for that... - Harvey
-
We were not interested in continuous updates, but periodic updates upon demand. My understanding of rsync is that it runs continuously. Also, we weren't dealing with large files, but a mix, mostly small files.
I'm retired. There's a nap for that... - Harvey
Fair enough. My experience is that rsync still works well. You can schedule when it runs easily enough - backups were only done at night to avoid loading the slow broadband during the day when it was in use for news feeds etc.