I want to modify/replace some data from a binary file without loadig the file in memory.

Alex Manolescu

My quest is simple and I have some ideas, but I wonder if you have a better idea :) I have a binary file with some data in it, and I want to modify/replace data from position "x". I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size. Now I need a solution to do this. I taught at a simple solution: >read the binary file with a handler, and write to a new binary file with another handler >when reading arrive at my "x" position, I write my own binary data. My code is shown below:

public void MyWriteMethod()
{
int b;
int x = 153; // the position I want to insert binary data

        Stream str = File.Open("dT.man", FileMode.Open);
        Stream strIndex = File.Open("dT.index", FileMode.CreateNew);
        
        BinaryReader br = new BinaryReader(str);            
        BinaryWriter bw = new BinaryWriter(strIndex);
        
        while ((b = br.BaseStream.ReadByte()) != -1)
        {
            if (br.BaseStream.Position == x)
            {
                // below is the data that I want to write
                bw.Write(1);
                bw.Write("StringData");
                bw.Write("StringData");
                bw.Flush();
                continue;
            }
            bw.Write(b);
        }

        bw.Flush();
        bw.Close();
        br.Close();

        str.Close();
        strIndex.Close();           
    }

Hope there is a better solution, something like seek into the binary file and replace the data. :-D Cheer's, Alex Manolescu.

Lost User

The Stream class has an overridable method seek from MSDN:

_For an example of creating a file and writing text to a file, see How
to: Write Text to a File. For an example of reading text from a file, see
How to: Read Text from a File. For an example of reading from and writing
to a binary file, see How to: Read and Write to a Newly Created Data File.

Use the CanSeek property to determine whether the current instance supports seeking.

If offset is negative, the new position is required to precede the
position specified by origin by the number of bytes specified by
offset. If offset is zero (0), the new position is required to be the
position specified by origin. If offset is positive, the new position
is required to follow the position specified by origin by the number of
bytes specified by offset._

Stream.Seek() method[^]

Luc Pattyn

Hi, some coding comments: 1. that continue is ugly; what you want is an if (...) ... else ... 2. please drop the flushes, they don't make any sense just before a close. 3. you can replace the close() by using a using construct. FYI: BinaryWriter.Write(string) "Writes a length-prefixed string to this stream in the current encoding", which may or may not be what you want. And of course, inserting something in the middle of the file may break its format. Don't try it like that with an EXE file! :)

Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]

Merry Christmas and a Happy New Year to all.

Alex Manolescu

Thanks you verry much for answer! I know about the method Seek, but there is a problem! I want to modify/replace some binary data starting from the position "x", but when I try to modify data there is a problem with data integrity! The new data may be bigger (in size) that the replaced data!! I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size. To be more concret, supose the binary file is organised like this: 1 integer, and 2 string. (this is called, by me, a record). When I want to replace data from position "x", I have to replace with the same amount of bytes like the old data to not damage the other records! Now, what if I don't want to replace with the same amount of bytes? What if the amount of bytes is bigger/smaller than the old data bytes size? Can I do this without damage other data (records) ? What is the best solution to do this? Cheer's, Alex Manolescu.

Alex Manolescu

Thanks you verry much for answer! You're right about the flushes, but I am missing something at that

continue

? Is more efficient to use a if/else statement or why is ugly? I am just curious and I want to know :) I've wrote a replay above yours. Hope I've been more specific to what I want! Cheer's, Alex Manolescu.

Lost User

Hmm, Okay well an easy approach would be to add some padding to your file so it don't not matter, just use a greedy shuffling algorithm to move things around in the original. If you create a data structure in your app, fill it with data, and then serialize it to a file with padding you can add what ever the data structure will allow. You would have a set size file, once you get bigger then the file size you would have to recreate the file. Think of it as a loading problem, similar to a Hashtable. You create the Hashtable at approximately the correct working size. If the number falls within 70% of the total allocated space for the hashtable it 're-hashes' and increases to a larger size, usually 2x bigger.

realJSOP

Alex Manolescu wrote:

The new data may be bigger (in size) that the replaced data!!

Then you'll have to read the entire file into memory, change the part that needs changing, and write the entire thing back out to the file again.

.45 ACP - because shooting twice is just silly
-----
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"The staggering layers of obscenity in your statement make it a work of art on so many levels." - J. Jystad, 2001

Luc Pattyn

Hi, 1.

if (br.BaseStream.Position == x) {
// below is the data that I want to write
bw.Write(1);
bw.Write("StringData");
bw.Write("StringData");
} else {
bw.Write(b);
}

is much cleaner; performance is the same. 2. damage to file format depends on file type. You could not do that for a Word document, an EXE file, etc. You could do it for your own data files if your apps accept such changes. :)

Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]

Merry Christmas and a Happy New Year to all.

Alex Manolescu

Thanks you for answer! Happy New Year! :)

Alex Manolescu

Thanks you for answer! I'll give it a try and come back with news. Happy New Year!

Alex Manolescu

Thanks you for answer! I think my method, to read and write bytes from file to file without loading the memory, is working great for big files. Happy New Year! :)

petercrab

what you have written already looks ok to me. Like others have said, so long as binary file is not too large you can get a speed increase by loading entire file to memory in one read, modifying, then writing the output file. many thanks, petercrab