Data corruption during File Write
-
Hi All, I am having an application where a structure will be directly written into HDD in binary format at a particular time interval. But if PC gets turned off due to some abnormal problems, during file write, after the next restart the entire file has only 0's. it is getting corrupted. How can i avoid this problem (apart from taking backup and restore in case of corruption). Any help would be appreciated. Thanks.
Selva
-
Hi All, I am having an application where a structure will be directly written into HDD in binary format at a particular time interval. But if PC gets turned off due to some abnormal problems, during file write, after the next restart the entire file has only 0's. it is getting corrupted. How can i avoid this problem (apart from taking backup and restore in case of corruption). Any help would be appreciated. Thanks.
Selva
If the file is very big one way could be dividing the data to be written in small packets, opening file, saving a packet, closing file and repeating. It will be slower, but closing the file sometimes during the process should make to save the already sent content, you will lose only the packet you are sending in the moment the computer crashes. BTW It is just a guess, never tried it.
Greetings. -------- M.D.V. ;) If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about? Help me to understand what I'm saying, and I'll explain it better to you “The First Rule of Program Optimization: Don't do it. The Second Rule of Program Optimization (for experts only!): Don't do it yet.” - Michael A. Jackson
-
Hi All, I am having an application where a structure will be directly written into HDD in binary format at a particular time interval. But if PC gets turned off due to some abnormal problems, during file write, after the next restart the entire file has only 0's. it is getting corrupted. How can i avoid this problem (apart from taking backup and restore in case of corruption). Any help would be appreciated. Thanks.
Selva
An alternative to Nelek's suggestion of Open, Write, Close is to call a Flush after every write. There always is a window of opportunity for disk files to get trashed. Even using the suggestions you've received, there is a chance that the machine will power down before the disk cache is written to disk. The only way to make that window smaller is to use something like the
commitfunction, which is a pretty low level C function. It instructs the OS to issue the actual write commands to the disk. The one case you can't guard against is someone powering off a machine while the actual physical write is in progress, but from what little you said, it soundsflushshould give you what you need. Judy -
Hi All, I am having an application where a structure will be directly written into HDD in binary format at a particular time interval. But if PC gets turned off due to some abnormal problems, during file write, after the next restart the entire file has only 0's. it is getting corrupted. How can i avoid this problem (apart from taking backup and restore in case of corruption). Any help would be appreciated. Thanks.
Selva
SelvaKr wrote:
after the next restart the entire file has only 0's. it is getting corrupted.
I sense that you are after a way of writing mission critical data in an guaranteed safe way, so that integrity is maintained, even during a powerfailure halfway tru the write. I have done quite a bit of research on this,and came to the following design conclusions.- 1. It is Not possible to guarantee a successfull write under above circumstances 2. It is possible as Second Best, to maintain a consistent state. This means that under such circumstances, the transaction is either completed in full, or you are returned to the state prevailing before the transaction commenced. This is not a guaranteed write, but, atleast, you have not the corruption of a partially completed transaction. My system requires two identical file sets, marked Primary and Alternate. There is also a single third file, the State file, which holds only One Byte. This Byte assumes one of the following Values:- STATE_NORMAL, STATE_MOD_PRIMARY, STATE_MOD_ALTERNATE. When we start, Both Primary ad Alternate Sets are Identical. Step 1: The Write Operation commences by opening the State File, and Writing STATE_MOD_PRIMARY to it, after which it is closed. This indicates that the Primary Set is now entering into a potentially unstable State, but the Alternate holds a consistent backup. Step 2: Carry Out All your Writes to the Primary Set. When the Write is Completed, Close All Files. Step 3: Open the State File, and Write STATE_MOD_ALTERNATE to it, after which it is closed. This indicates that the Alternate Set is now entering into a potentially unstable State, but the Primary Set holds the New Dataset. Step 4: Delete All Alternate Files, and replace them with copies of the Primary Set. Step 5: Open the State File, and Write STATE_MOD_NORMAL to it, after which it is closed. This indicates that the Data Set is Once again in a Stable state. When your Excecutable starts, it must investigate the FileTime's of the State File, the Primary Set, and the Alternate Set. A Prima Facia Consistent state is indicated by FileTime(Primary) Another Consistency test is that the State File contains STATE_MOD_NORMAL If any of this fails, you can figure out from the above description, where things