Backup schedule / Media recommendation
-
We are slowly growing out of our current semi-manual backup mechanism (problems being: time to restore, manual part, and incomplete coverage, and time to restore). We have to back up a file server (80G and growing) which is inactive during the night plus two system drives that rarely change (manual backup after install seems OK) So when do you back up what, how, and which media? ---- My thoughts: As far as I understand, a typical schedule would be (with the times choosen according to your data/requirements): - Monthly full backup - Weekly diff to the full backup - daily incremental backup How would you chose these times? Is this to much? Are there some rules of thumb for the sizes of the diff / incremental data? Where should I put the full backup on? Another Harddrive? Rotating two harddrives? 15 DVDs? Tape????
Pandoras Gift #44: Hope. The one that keeps you on suffering.
aber.. "Wie gesagt, der Scheiss is' Therapie"
boost your code || Fold With Us! || sighist | doxygenActually TRY to restore and USE your backup every while and then. Shut down the original server and see if someone complains about "lost data". I lost the count of companies I saw that stablished the "perfect" backup system and, a few months later, discovered that someone created a new folder somewhere else and it was not backed up. We backup our CVS folders and, from time to time, we try to build our software from the backup, instead of building from the server. I see dead pixels Yes, even I am blogging now!
-
I installed and administrated networks at all sorts of businesses for many years and the standard practice for optimal disaster recovery is this: 10 TAPE rotation (yes tapes, not burned cd's or dvd's etc) Take 10 tapes and label them as follows: Monday, Tuesday, Wednesday, Thursday, Friday 1, Friday 2, Friday 3, Month 1, Month 2, Month 3 Have a backup program that runs automatically every evenening and can alert you to any problems, test it to ensure it's backing up and restoring properly on an ongoing basis. Insert the daily tapes on the morning of those days, cycle through the friday tapes so that you use them in sequence and at the end of the month or first day of the new month substitute the Month X tape for that day's backup and cycle through them each month. This gives you ability to restore data from as recently as the last working day or as far back as three months ago. Both are critical because it's often that something goes amiss or is damaged but not noticed for a long period of time. At the end of each working day you absolutely *MUST* take the previous nights backup tape with you and store it offsite. There are services that will do that for you in larger cities but the critical point is that it is taken away from the building physically. We've used such a system to get many businesses up and running after disasters of all kinds including fire, flooding, complete theft of all computer equipment (including the tapes that were left laying beside the server) etc etc. Tape capacity must be enough to store everything on one single tape and each backup must be complete, do not do incremental backups as they spread your risk over many tapes which gives you an exponentially greater chance of losing data for each tape you add to the increment. DVD's and CD's are inadequate because they do not hold enough data and they are much more expensive to backup in this manner. In a very critical situation we've added a noon backup and added tapes accordingly but that's pretty rare.
"A preoccupation with the next world pretty clearly signals an inability to cope credibly with this one."
Just to add to the importance of physically removing backups from the building... my sister's workplace burned to the ground a few months back. While the company had made good backups they hadnt stored them offsite, and consequently they lost everything. Ouch.
-
Could you recommend a Starting Kit or similar?
Pandoras Gift #44: Hope. The one that keeps you on suffering.
aber.. "Wie gesagt, der Scheiss is' Therapie"
boost your code || Fold With Us! || sighist | doxygen??? Starting kit? This is a case where you should really call in a competent network technician with experience and ask them what they recommend even if you do it yourself. If your organization is really cheap then you can do the following (which I hated people doing to me when I was still doing that kind of work): Phone every networking company you can find, ask them to send a qualified tech to inspect your hardware and needs and and provide a detailed quote on a backup system including prices for hardware, installation etc. Then take the info and implement it yourself. It's very revealing what more than one place will quote and gives you an idea of what's out there etc. Backup is too critical to fart around with cheaply though, when the big earthquake happened in california there was a very good study done on the impacts of improper disaster recovery for business and a very high percentage of business that lost all their data actually never recovered and went bankrupt in the end. A lot of the others lost a *lot* of money and customers over lost data, it's really an area worth investing some real money and professional expertise in. If you don't do it now, chances are you will do it later. ;) I've been coding for a living for many years now and I'm not up on the latest backup technologies to recommend any hardware, but the process is the same regardless of the hardware involved.
"A preoccupation with the next world pretty clearly signals an inability to cope credibly with this one."
-
Actually TRY to restore and USE your backup every while and then. Shut down the original server and see if someone complains about "lost data". I lost the count of companies I saw that stablished the "perfect" backup system and, a few months later, discovered that someone created a new folder somewhere else and it was not backed up. We backup our CVS folders and, from time to time, we try to build our software from the backup, instead of building from the server. I see dead pixels Yes, even I am blogging now!
Yes, many years ago I was called in to do some work on a Lantastic network at a carpet retail store. I was more Novell and NT at the time, but Lantastic was pretty easy to work with. As I was finishing up I was chatting with the owner and asked the inevitable "how is your backup" question I always asked because 95% of every new business we took over didn't have any kind of backup at all. He said "oh it's great, really great, it takes only about 30 seconds" "WHAT?" I said. (there was no backup method at the time that could take anything less than an hour) "Sure, it's super fast, we've been using it for years" I take a look, it's an old Colorado tape drive and when I run the backup it indeed takes 30 seconds, 29 seconds to load the software then load the tape, then 1 second to report that there is no tape in the drive (even though there was). Apparently some dumbass technician had installed it for them years before and had not once tested it before leaving the place.
"A preoccupation with the next world pretty clearly signals an inability to cope credibly with this one."
-
Just to add to the importance of physically removing backups from the building... my sister's workplace burned to the ground a few months back. While the company had made good backups they hadnt stored them offsite, and consequently they lost everything. Ouch.
We do software now, no technical stuff, but we were in chatting with our business accountant the other day and the subject of his network came up and it turned out he didn't take the tapes offsite at all either. We chastised him pretty agressively about that, in particular since all our own data is stored there as well. We asked him how long he thought it would take to completely recover his business with the loss of every bit of his client's accounting data for all the years he had been looking after them and he was pretty shaken by the thought. It's amazing how many really intelligent people don't stop to even consider such a simple and common possibility, every day in the city an office is burned or broken into and everything stolen etc. It's got to be the most easily forseeable disaster than can befall a business, yet I can't count the number of times I've seen the backup media (if there is even a backup at all) stored in a nice lockable (highly combustable) plastic box right on top of the server they are backing up!
"A preoccupation with the next world pretty clearly signals an inability to cope credibly with this one."
-
Good but definitely not good enough, I have had many clients over the years not notice that a file was damaged or missing for more than 8 weeks, you also should have at least three monthly backups kept separately so you can go back a "quarter".
"A preoccupation with the next world pretty clearly signals an inability to cope credibly with this one."
I meant to add that at the end of every month, a full backup is copied to another Rev disk, the "monthly archive" and that is kept *forever* (ie the disk is used once, and stored).
-
I meant to add that at the end of every month, a full backup is copied to another Rev disk, the "monthly archive" and that is kept *forever* (ie the disk is used once, and stored).
-
Maybe a bit on the pricey side, but it's an interesting option. May I pester you with additional questions? How much "raw" data you get on a REV disk - i.e. what compression rate to be expected? (IOMEGA assumes 2.6:1) Does a drive include the software? How does it handle? Graham Bradshaw wrote: We just plug the Iomega drive into any computer with a USB port, put a Rev disk in, and all data is readable as part of the file system. you don't need to install any software to access the (compressed?) data? (now THAT would be cool!) Do you use the RAID 0/1 just so you don't need to plug HDD's in case of failure?
Pandoras Gift #44: Hope. The one that keeps you on suffering.
aber.. "Wie gesagt, der Scheiss is' Therapie"
boost your code || Fold With Us! || sighist | doxygenpeterchen wrote: Maybe a bit on the pricey side True, but nowhere near as pricey as rewriting all that source code from scratch! peterchen wrote: How much "raw" data you get on a REV disk Native, it's 35GB. peterchen wrote: you don't need to install any software to access the (compressed?) data? The one in the office is mounted in a server chassis, and is connected to an IDE channel (just like a CD-ROM). Without installing the Iomega sofwtare, Windows sees it as a CD-ROM (ie read, but no write). Once you install the Iomega software, it becomes a removable disk. We treat it like a (very, very large) floppy disk. We put the disk in the drive, and copy files to it using Explorer, and eject the disk. 15GB takes about 28 minutes to copy, to give you some idea of speed. On the backup server, we don't use the compression option (or the Iomega backup software other that as a driver. To read the data off, using the USB drive (note this is kept offsite, with the offsite backups themselves, for use in a disaster recovery scenario), you just plug the drive into a USB port, and you can pull the data off (I've tried this with Windows XP, and it does work). peterchen wrote: Do you use the RAID 0/1 just so you don't need to plug HDD's in case of failure Yes. We have 6 18GB SCSI disks on the chain, and 4 are active in the RAID 0/1 array, giving us one logical 34 GB drive. This is partitioned as 4GB system, and 30 GB data. The other two drives are "hot spares", ie if one of the four primary drives fails, one of the spares is automatically brought into the array, and the data reconstructed on it, restoring the full fault tolerance.
-
Sounds like a great system! I'll have to look into those Rev disks, our tape drive is starting to get pretty long in the tooth.
"A preoccupation with the next world pretty clearly signals an inability to cope credibly with this one."
I think they're great. As previously noted, not the cheapest solution, but then if the data's worth protecting, it's worth protecting properly. The appeal for us was 1) size: a 35GB disk fits in a shirt pocket 2) Recoverability - the offsite USB drive and data means that we can recover with any system running Windows with a USB port.
-
Yes, many years ago I was called in to do some work on a Lantastic network at a carpet retail store. I was more Novell and NT at the time, but Lantastic was pretty easy to work with. As I was finishing up I was chatting with the owner and asked the inevitable "how is your backup" question I always asked because 95% of every new business we took over didn't have any kind of backup at all. He said "oh it's great, really great, it takes only about 30 seconds" "WHAT?" I said. (there was no backup method at the time that could take anything less than an hour) "Sure, it's super fast, we've been using it for years" I take a look, it's an old Colorado tape drive and when I run the backup it indeed takes 30 seconds, 29 seconds to load the software then load the tape, then 1 second to report that there is no tape in the drive (even though there was). Apparently some dumbass technician had installed it for them years before and had not once tested it before leaving the place.
"A preoccupation with the next world pretty clearly signals an inability to cope credibly with this one."
-
??? Starting kit? This is a case where you should really call in a competent network technician with experience and ask them what they recommend even if you do it yourself. If your organization is really cheap then you can do the following (which I hated people doing to me when I was still doing that kind of work): Phone every networking company you can find, ask them to send a qualified tech to inspect your hardware and needs and and provide a detailed quote on a backup system including prices for hardware, installation etc. Then take the info and implement it yourself. It's very revealing what more than one place will quote and gives you an idea of what's out there etc. Backup is too critical to fart around with cheaply though, when the big earthquake happened in california there was a very good study done on the impacts of improper disaster recovery for business and a very high percentage of business that lost all their data actually never recovered and went bankrupt in the end. A lot of the others lost a *lot* of money and customers over lost data, it's really an area worth investing some real money and professional expertise in. If you don't do it now, chances are you will do it later. ;) I've been coding for a living for many years now and I'm not up on the latest backup technologies to recommend any hardware, but the process is the same regardless of the hardware involved.
"A preoccupation with the next world pretty clearly signals an inability to cope credibly with this one."
John Cardinal wrote: Phone every networking company you can find, ask them to... We are cheap, but we wouldn't do that :) It's almost impossible to find competent, reliable, affordable techies here. I'd love to have a on-demand guy who takes care of network / PC building / installation etc. Our technician and I share that stuff. Our current process works more or less - (we coped with 1.5 small failures) - we`re mainly looking for a process with less administration (switching media is OK), and plugging some of the non-critical "pain holes". I'll look into the Rev ZIP, it seems to be the right thing for us.
Pandoras Gift #44: Hope. The one that keeps you on suffering.
aber.. "Wie gesagt, der Scheiss is' Therapie"
boost your code || Fold With Us! || sighist | doxygen -
We are slowly growing out of our current semi-manual backup mechanism (problems being: time to restore, manual part, and incomplete coverage, and time to restore). We have to back up a file server (80G and growing) which is inactive during the night plus two system drives that rarely change (manual backup after install seems OK) So when do you back up what, how, and which media? ---- My thoughts: As far as I understand, a typical schedule would be (with the times choosen according to your data/requirements): - Monthly full backup - Weekly diff to the full backup - daily incremental backup How would you chose these times? Is this to much? Are there some rules of thumb for the sizes of the diff / incremental data? Where should I put the full backup on? Another Harddrive? Rotating two harddrives? 15 DVDs? Tape????
Pandoras Gift #44: Hope. The one that keeps you on suffering.
aber.. "Wie gesagt, der Scheiss is' Therapie"
boost your code || Fold With Us! || sighist | doxygenpeterchen wrote: We are slowly growing out of our current semi-manual backup mechanism (problems being: time to restore, manual part, and incomplete coverage, and time to restore). We have to back up a file server (80G and growing) which is inactive during the night plus two system drives that rarely change (manual backup after install seems OK) I highly recommend a low paid intern and lots and lots of floppy disks. Michael Martin Australia "I controlled my laughter and simple said "No,I am very busy,so I can't write any code for you". The moment they heard this all the smiling face turned into a sad looking face and one of them farted. So i had to leave the place as soon as possible." - Mr.Prakash 24/04/2004
-
peterchen wrote: We are slowly growing out of our current semi-manual backup mechanism (problems being: time to restore, manual part, and incomplete coverage, and time to restore). We have to back up a file server (80G and growing) which is inactive during the night plus two system drives that rarely change (manual backup after install seems OK) I highly recommend a low paid intern and lots and lots of floppy disks. Michael Martin Australia "I controlled my laughter and simple said "No,I am very busy,so I can't write any code for you". The moment they heard this all the smiling face turned into a sad looking face and one of them farted. So i had to leave the place as soon as possible." - Mr.Prakash 24/04/2004
:laugh: you are joking, right? :~
Pandoras Gift #44: Hope. The one that keeps you on suffering.
aber.. "Wie gesagt, der Scheiss is' Therapie"
boost your code || Fold With Us! || sighist | doxygen -
:laugh: you are joking, right? :~
Pandoras Gift #44: Hope. The one that keeps you on suffering.
aber.. "Wie gesagt, der Scheiss is' Therapie"
boost your code || Fold With Us! || sighist | doxygenIf he's got a beer in his hand, the answer is probably yes. :laugh: Seriously though...I've come across quite a few organisations whose IT strategy is about that organised. Scary. :~ Anna :rose: Riverblade Ltd - Software Consultancy Services Anna's Place | Tears and Laughter "Be yourself - not what others think you should be" - Marcia Graesch "Anna's just a sexy-looking lesbian tart" - A friend, trying to wind me up. It didn't work.
-
Yes, many years ago I was called in to do some work on a Lantastic network at a carpet retail store. I was more Novell and NT at the time, but Lantastic was pretty easy to work with. As I was finishing up I was chatting with the owner and asked the inevitable "how is your backup" question I always asked because 95% of every new business we took over didn't have any kind of backup at all. He said "oh it's great, really great, it takes only about 30 seconds" "WHAT?" I said. (there was no backup method at the time that could take anything less than an hour) "Sure, it's super fast, we've been using it for years" I take a look, it's an old Colorado tape drive and when I run the backup it indeed takes 30 seconds, 29 seconds to load the software then load the tape, then 1 second to report that there is no tape in the drive (even though there was). Apparently some dumbass technician had installed it for them years before and had not once tested it before leaving the place.
"A preoccupation with the next world pretty clearly signals an inability to cope credibly with this one."
That's just scary. :~ Anna :rose: Riverblade Ltd - Software Consultancy Services Anna's Place | Tears and Laughter "Be yourself - not what others think you should be" - Marcia Graesch "Anna's just a sexy-looking lesbian tart" - A friend, trying to wind me up. It didn't work.
-
If he's got a beer in his hand, the answer is probably yes. :laugh: Seriously though...I've come across quite a few organisations whose IT strategy is about that organised. Scary. :~ Anna :rose: Riverblade Ltd - Software Consultancy Services Anna's Place | Tears and Laughter "Be yourself - not what others think you should be" - Marcia Graesch "Anna's just a sexy-looking lesbian tart" - A friend, trying to wind me up. It didn't work.
Only problem - where to get a cheap intern. :-/ We'll look into the iomega REV drive, it seems to have the comfortable handling I'm looking for.
Pandoras Gift #44: Hope. The one that keeps you on suffering.
aber.. "Wie gesagt, der Scheiss is' Therapie"
boost your code || Fold With Us! || sighist | doxygen