Archival Methods (TAPE, IDE DISK DVD-R)...
-
Here is my problem... We do medical imaging research which generates several GB of data per day. We can not possibly store this online (which we would like to) but we currently archive everything to tape and store as much as we can on disk. We have several TB of IDE drives scattered in a few servers. The thing is that tape now costs more per GB than IDE drives cost. With our ADR drives tapes cost > $2 GB for 25 GB native, IDE drives run for less than $1 per GB for 120GB models... Tape is much slower and much harder to get your data off it. And also it is debatable on how reliable tapes are since we have lost more data on tapes than IDE drives... We are thinking of buying a bunch of 120GB or larger IDE drives and giving up on tape in the future. An other thought is to use DVD-R disks which can be purchased for $2 to $3 in bulk but 4.7GB would require a lot of DVD disks and would require more work to store and find the correct disk with the data on. Also we would probably not want cases to span disks so this will cause some waste and with the other two methods compression is easily available and transparent to the user but it seems it would be more difficult to use compression on a DVD disk unless we use packet writing and NTFS... My question is what are your thoughts on the matter? John
We ran into a similar problem a few months ago. A typical mapping project for us will generate between 1 and 4 GB of imagery and data files. We've been buying 120 GB IDE drives, but this obviously has its limitations. We decided to buy DVD writers and now burn DVD-R's for completed projects. I start the burning just before I'm done for the day. We've now turned our IDE drives into backup drives that are updated each night. First thing in the morning I burn a duplicate copy of each DVD-R for offsite storage. I resist the temptation to put nice labels on them - I've read some articles that suggest the glue can seriously reduce the lifespan of CD's and DVD's. I write on them with a Sharpie pen. Works great so far. We use an incremental year/month based project numbering system that keeps things simple but effective. Drew.
-
The data is medical images which can be anywhere from 36MB / case to almost 1 GB per case depending on what type of case it is digital mammograpgy, chest CT, MRI... Right now there are several databases that store the filenames associated with the images. A quick search on Veritas Backup Executive tells us what tape to insert... This system is in the process of change with new government regulations all of our images and data must be scrubbed to remove patient identification so at some point we must read all the tapes (over 100 now) anonymize them into a new honest broker database and write the data back out. I know this will be a pain... John
I did not make myself clear. Will your Veritas Backup Executive also handle which drive to insert if you swap them out? "For as long as I can remember, I have had memories. Colin Mochrie."
-
I did not make myself clear. Will your Veritas Backup Executive also handle which drive to insert if you swap them out? "For as long as I can remember, I have had memories. Colin Mochrie."
No it won't. John
-
We ran into a similar problem a few months ago. A typical mapping project for us will generate between 1 and 4 GB of imagery and data files. We've been buying 120 GB IDE drives, but this obviously has its limitations. We decided to buy DVD writers and now burn DVD-R's for completed projects. I start the burning just before I'm done for the day. We've now turned our IDE drives into backup drives that are updated each night. First thing in the morning I burn a duplicate copy of each DVD-R for offsite storage. I resist the temptation to put nice labels on them - I've read some articles that suggest the glue can seriously reduce the lifespan of CD's and DVD's. I write on them with a Sharpie pen. Works great so far. We use an incremental year/month based project numbering system that keeps things simple but effective. Drew.
I seriously thought of this method but if I can fit 24 DVDs on a single hard disk it seems that it would be much less work to use hard drives but your method makes the idea interesting again. The problem I have with this is our data comes from several different and unrealted sources. Mamographic images, chest CT, MRI ... John
-
Here is my problem... We do medical imaging research which generates several GB of data per day. We can not possibly store this online (which we would like to) but we currently archive everything to tape and store as much as we can on disk. We have several TB of IDE drives scattered in a few servers. The thing is that tape now costs more per GB than IDE drives cost. With our ADR drives tapes cost > $2 GB for 25 GB native, IDE drives run for less than $1 per GB for 120GB models... Tape is much slower and much harder to get your data off it. And also it is debatable on how reliable tapes are since we have lost more data on tapes than IDE drives... We are thinking of buying a bunch of 120GB or larger IDE drives and giving up on tape in the future. An other thought is to use DVD-R disks which can be purchased for $2 to $3 in bulk but 4.7GB would require a lot of DVD disks and would require more work to store and find the correct disk with the data on. Also we would probably not want cases to span disks so this will cause some waste and with the other two methods compression is easily available and transparent to the user but it seems it would be more difficult to use compression on a DVD disk unless we use packet writing and NTFS... My question is what are your thoughts on the matter? John
We are now thinking of adding a 1 TB RAID 5 system which we can put together for about $1000 US. I checked price watch and 120GB WD drives are about $100 US. Using our DC (Win2k Server) which has a 10 open bays and support for 12 HDs on the MOBO we should not need any additional hw or software because win2k has software raid. To me this is kind of a delay tactic but it will solve the immediate need. John
-
We are now thinking of adding a 1 TB RAID 5 system which we can put together for about $1000 US. I checked price watch and 120GB WD drives are about $100 US. Using our DC (Win2k Server) which has a 10 open bays and support for 12 HDs on the MOBO we should not need any additional hw or software because win2k has software raid. To me this is kind of a delay tactic but it will solve the immediate need. John
problem with lots of hard drives, is that you better be able to keep them cool.. otherwise they will kill each other.
-
problem with lots of hard drives, is that you better be able to keep them cool.. otherwise they will kill each other.
I make sure that the drives don't get too hot.. The server room is absolutely freezing and the pc that these are going into has 7 fans on it already... John
-
I make sure that the drives don't get too hot.. The server room is absolutely freezing and the pc that these are going into has 7 fans on it already... John
:cool:
-
Here is my problem... We do medical imaging research which generates several GB of data per day. We can not possibly store this online (which we would like to) but we currently archive everything to tape and store as much as we can on disk. We have several TB of IDE drives scattered in a few servers. The thing is that tape now costs more per GB than IDE drives cost. With our ADR drives tapes cost > $2 GB for 25 GB native, IDE drives run for less than $1 per GB for 120GB models... Tape is much slower and much harder to get your data off it. And also it is debatable on how reliable tapes are since we have lost more data on tapes than IDE drives... We are thinking of buying a bunch of 120GB or larger IDE drives and giving up on tape in the future. An other thought is to use DVD-R disks which can be purchased for $2 to $3 in bulk but 4.7GB would require a lot of DVD disks and would require more work to store and find the correct disk with the data on. Also we would probably not want cases to span disks so this will cause some waste and with the other two methods compression is easily available and transparent to the user but it seems it would be more difficult to use compression on a DVD disk unless we use packet writing and NTFS... My question is what are your thoughts on the matter? John
How about geting 200GB drives and putting them in removable carriers (many RAID controllers support hot swap now)? And yes, use the best compression you can ! Elaine :rose: The tigress is here :-D
-
We ran into a similar problem a few months ago. A typical mapping project for us will generate between 1 and 4 GB of imagery and data files. We've been buying 120 GB IDE drives, but this obviously has its limitations. We decided to buy DVD writers and now burn DVD-R's for completed projects. I start the burning just before I'm done for the day. We've now turned our IDE drives into backup drives that are updated each night. First thing in the morning I burn a duplicate copy of each DVD-R for offsite storage. I resist the temptation to put nice labels on them - I've read some articles that suggest the glue can seriously reduce the lifespan of CD's and DVD's. I write on them with a Sharpie pen. Works great so far. We use an incremental year/month based project numbering system that keeps things simple but effective. Drew.
We have now decided to do both. We purchased 1 TB of 120GB WD drives for < $1000 US which we are going to use with software RAID5 in our main server which has 12 open drive bays and we are also going to backup the data with inexpensive DVD+R ($1 to $3 per disk) disks using our Veritas Backup Executive program. The two solutions together will still cost less than $2 / GB so it is a win win situation... Thanks for your help. John