Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Code Project
CODE PROJECT For Those Who Code
  • Home
  • Articles
  • FAQ
Community
  1. Home
  2. General Programming
  3. C#
  4. I want to modify/replace some data from a binary file without loadig the file in memory.

I want to modify/replace some data from a binary file without loadig the file in memory.

Scheduled Pinned Locked Moved C#
databaseperformance
12 Posts 5 Posters 4 Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A Alex Manolescu

    My quest is simple and I have some ideas, but I wonder if you have a better idea :) I have a binary file with some data in it, and I want to modify/replace data from position "x". I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size. Now I need a solution to do this. I taught at a simple solution: >read the binary file with a handler, and write to a new binary file with another handler >when reading arrive at my "x" position, I write my own binary data. My code is shown below:

    public void MyWriteMethod()
    {
    int b;
    int x = 153; // the position I want to insert binary data

            Stream str = File.Open("dT.man", FileMode.Open);
            Stream strIndex = File.Open("dT.index", FileMode.CreateNew);
            
            BinaryReader br = new BinaryReader(str);            
            BinaryWriter bw = new BinaryWriter(strIndex);
            
            while ((b = br.BaseStream.ReadByte()) != -1)
            {
                if (br.BaseStream.Position == x)
                {
                    // below is the data that I want to write
                    bw.Write(1);
                    bw.Write("StringData");
                    bw.Write("StringData");
                    bw.Flush();
                    continue;
                }
                bw.Write(b);
            }
    
            bw.Flush();
            bw.Close();
            br.Close();
    
            str.Close();
            strIndex.Close();           
        }
    

    Hope there is a better solution, something like seek into the binary file and replace the data. :-D Cheer's, Alex Manolescu.

    L Offline
    L Offline
    Lost User
    wrote on last edited by
    #2

    The Stream class has an overridable method seek from MSDN:

    _For an example of creating a file and writing text to a file, see How
    to: Write Text to a File. For an example of reading text from a file, see
    How to: Read Text from a File. For an example of reading from and writing
    to a binary file, see How to: Read and Write to a Newly Created Data File.

    Use the CanSeek property to determine whether the current instance supports seeking.

    If offset is negative, the new position is required to precede the
    position specified by origin by the number of bytes specified by
    offset. If offset is zero (0), the new position is required to be the
    position specified by origin. If offset is positive, the new position
    is required to follow the position specified by origin by the number of
    bytes specified by offset._

    Stream.Seek() method[^]

    A 1 Reply Last reply
    0
    • A Alex Manolescu

      My quest is simple and I have some ideas, but I wonder if you have a better idea :) I have a binary file with some data in it, and I want to modify/replace data from position "x". I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size. Now I need a solution to do this. I taught at a simple solution: >read the binary file with a handler, and write to a new binary file with another handler >when reading arrive at my "x" position, I write my own binary data. My code is shown below:

      public void MyWriteMethod()
      {
      int b;
      int x = 153; // the position I want to insert binary data

              Stream str = File.Open("dT.man", FileMode.Open);
              Stream strIndex = File.Open("dT.index", FileMode.CreateNew);
              
              BinaryReader br = new BinaryReader(str);            
              BinaryWriter bw = new BinaryWriter(strIndex);
              
              while ((b = br.BaseStream.ReadByte()) != -1)
              {
                  if (br.BaseStream.Position == x)
                  {
                      // below is the data that I want to write
                      bw.Write(1);
                      bw.Write("StringData");
                      bw.Write("StringData");
                      bw.Flush();
                      continue;
                  }
                  bw.Write(b);
              }
      
              bw.Flush();
              bw.Close();
              br.Close();
      
              str.Close();
              strIndex.Close();           
          }
      

      Hope there is a better solution, something like seek into the binary file and replace the data. :-D Cheer's, Alex Manolescu.

      L Offline
      L Offline
      Luc Pattyn
      wrote on last edited by
      #3

      Hi, some coding comments: 1. that continue is ugly; what you want is an if (...) ... else ... 2. please drop the flushes, they don't make any sense just before a close. 3. you can replace the close() by using a using construct. FYI: BinaryWriter.Write(string) "Writes a length-prefixed string to this stream in the current encoding", which may or may not be what you want. And of course, inserting something in the middle of the file may break its format. Don't try it like that with an EXE file! :)

      Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]


      Merry Christmas and a Happy New Year to all.


      A 1 Reply Last reply
      0
      • L Lost User

        The Stream class has an overridable method seek from MSDN:

        _For an example of creating a file and writing text to a file, see How
        to: Write Text to a File. For an example of reading text from a file, see
        How to: Read Text from a File. For an example of reading from and writing
        to a binary file, see How to: Read and Write to a Newly Created Data File.

        Use the CanSeek property to determine whether the current instance supports seeking.

        If offset is negative, the new position is required to precede the
        position specified by origin by the number of bytes specified by
        offset. If offset is zero (0), the new position is required to be the
        position specified by origin. If offset is positive, the new position
        is required to follow the position specified by origin by the number of
        bytes specified by offset._

        Stream.Seek() method[^]

        A Offline
        A Offline
        Alex Manolescu
        wrote on last edited by
        #4

        Thanks you verry much for answer! I know about the method Seek, but there is a problem! I want to modify/replace some binary data starting from the position "x", but when I try to modify data there is a problem with data integrity! The new data may be bigger (in size) that the replaced data!! I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size. To be more concret, supose the binary file is organised like this: 1 integer, and 2 string. (this is called, by me, a record). When I want to replace data from position "x", I have to replace with the same amount of bytes like the old data to not damage the other records! Now, what if I don't want to replace with the same amount of bytes? What if the amount of bytes is bigger/smaller than the old data bytes size? Can I do this without damage other data (records) ? What is the best solution to do this? Cheer's, Alex Manolescu.

        L realJSOPR 2 Replies Last reply
        0
        • L Luc Pattyn

          Hi, some coding comments: 1. that continue is ugly; what you want is an if (...) ... else ... 2. please drop the flushes, they don't make any sense just before a close. 3. you can replace the close() by using a using construct. FYI: BinaryWriter.Write(string) "Writes a length-prefixed string to this stream in the current encoding", which may or may not be what you want. And of course, inserting something in the middle of the file may break its format. Don't try it like that with an EXE file! :)

          Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]


          Merry Christmas and a Happy New Year to all.


          A Offline
          A Offline
          Alex Manolescu
          wrote on last edited by
          #5

          Thanks you verry much for answer! You're right about the flushes, but I am missing something at that

          continue

          ? Is more efficient to use a if/else statement or why is ugly? I am just curious and I want to know :) I've wrote a replay above yours. Hope I've been more specific to what I want! Cheer's, Alex Manolescu.

          L 1 Reply Last reply
          0
          • A Alex Manolescu

            Thanks you verry much for answer! I know about the method Seek, but there is a problem! I want to modify/replace some binary data starting from the position "x", but when I try to modify data there is a problem with data integrity! The new data may be bigger (in size) that the replaced data!! I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size. To be more concret, supose the binary file is organised like this: 1 integer, and 2 string. (this is called, by me, a record). When I want to replace data from position "x", I have to replace with the same amount of bytes like the old data to not damage the other records! Now, what if I don't want to replace with the same amount of bytes? What if the amount of bytes is bigger/smaller than the old data bytes size? Can I do this without damage other data (records) ? What is the best solution to do this? Cheer's, Alex Manolescu.

            L Offline
            L Offline
            Lost User
            wrote on last edited by
            #6

            Hmm, Okay well an easy approach would be to add some padding to your file so it don't not matter, just use a greedy shuffling algorithm to move things around in the original. If you create a data structure in your app, fill it with data, and then serialize it to a file with padding you can add what ever the data structure will allow. You would have a set size file, once you get bigger then the file size you would have to recreate the file. Think of it as a loading problem, similar to a Hashtable. You create the Hashtable at approximately the correct working size. If the number falls within 70% of the total allocated space for the hashtable it 're-hashes' and increases to a larger size, usually 2x bigger.

            A 1 Reply Last reply
            0
            • A Alex Manolescu

              Thanks you verry much for answer! I know about the method Seek, but there is a problem! I want to modify/replace some binary data starting from the position "x", but when I try to modify data there is a problem with data integrity! The new data may be bigger (in size) that the replaced data!! I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size. To be more concret, supose the binary file is organised like this: 1 integer, and 2 string. (this is called, by me, a record). When I want to replace data from position "x", I have to replace with the same amount of bytes like the old data to not damage the other records! Now, what if I don't want to replace with the same amount of bytes? What if the amount of bytes is bigger/smaller than the old data bytes size? Can I do this without damage other data (records) ? What is the best solution to do this? Cheer's, Alex Manolescu.

              realJSOPR Offline
              realJSOPR Offline
              realJSOP
              wrote on last edited by
              #7

              Alex Manolescu wrote:

              The new data may be bigger (in size) that the replaced data!!

              Then you'll have to read the entire file into memory, change the part that needs changing, and write the entire thing back out to the file again.

              .45 ACP - because shooting twice is just silly
              -----
              "Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
              -----
              "The staggering layers of obscenity in your statement make it a work of art on so many levels." - J. Jystad, 2001

              A 1 Reply Last reply
              0
              • A Alex Manolescu

                Thanks you verry much for answer! You're right about the flushes, but I am missing something at that

                continue

                ? Is more efficient to use a if/else statement or why is ugly? I am just curious and I want to know :) I've wrote a replay above yours. Hope I've been more specific to what I want! Cheer's, Alex Manolescu.

                L Offline
                L Offline
                Luc Pattyn
                wrote on last edited by
                #8

                Hi, 1.

                if (br.BaseStream.Position == x) {
                // below is the data that I want to write
                bw.Write(1);
                bw.Write("StringData");
                bw.Write("StringData");
                } else {
                bw.Write(b);
                }

                is much cleaner; performance is the same. 2. damage to file format depends on file type. You could not do that for a Word document, an EXE file, etc. You could do it for your own data files if your apps accept such changes. :)

                Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]


                Merry Christmas and a Happy New Year to all.


                A 1 Reply Last reply
                0
                • L Luc Pattyn

                  Hi, 1.

                  if (br.BaseStream.Position == x) {
                  // below is the data that I want to write
                  bw.Write(1);
                  bw.Write("StringData");
                  bw.Write("StringData");
                  } else {
                  bw.Write(b);
                  }

                  is much cleaner; performance is the same. 2. damage to file format depends on file type. You could not do that for a Word document, an EXE file, etc. You could do it for your own data files if your apps accept such changes. :)

                  Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]


                  Merry Christmas and a Happy New Year to all.


                  A Offline
                  A Offline
                  Alex Manolescu
                  wrote on last edited by
                  #9

                  Thanks you for answer! Happy New Year! :)

                  1 Reply Last reply
                  0
                  • L Lost User

                    Hmm, Okay well an easy approach would be to add some padding to your file so it don't not matter, just use a greedy shuffling algorithm to move things around in the original. If you create a data structure in your app, fill it with data, and then serialize it to a file with padding you can add what ever the data structure will allow. You would have a set size file, once you get bigger then the file size you would have to recreate the file. Think of it as a loading problem, similar to a Hashtable. You create the Hashtable at approximately the correct working size. If the number falls within 70% of the total allocated space for the hashtable it 're-hashes' and increases to a larger size, usually 2x bigger.

                    A Offline
                    A Offline
                    Alex Manolescu
                    wrote on last edited by
                    #10

                    Thanks you for answer! I'll give it a try and come back with news. Happy New Year!

                    1 Reply Last reply
                    0
                    • realJSOPR realJSOP

                      Alex Manolescu wrote:

                      The new data may be bigger (in size) that the replaced data!!

                      Then you'll have to read the entire file into memory, change the part that needs changing, and write the entire thing back out to the file again.

                      .45 ACP - because shooting twice is just silly
                      -----
                      "Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
                      -----
                      "The staggering layers of obscenity in your statement make it a work of art on so many levels." - J. Jystad, 2001

                      A Offline
                      A Offline
                      Alex Manolescu
                      wrote on last edited by
                      #11

                      Thanks you for answer! I think my method, to read and write bytes from file to file without loading the memory, is working great for big files. Happy New Year! :)

                      1 Reply Last reply
                      0
                      • A Alex Manolescu

                        My quest is simple and I have some ideas, but I wonder if you have a better idea :) I have a binary file with some data in it, and I want to modify/replace data from position "x". I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size. Now I need a solution to do this. I taught at a simple solution: >read the binary file with a handler, and write to a new binary file with another handler >when reading arrive at my "x" position, I write my own binary data. My code is shown below:

                        public void MyWriteMethod()
                        {
                        int b;
                        int x = 153; // the position I want to insert binary data

                                Stream str = File.Open("dT.man", FileMode.Open);
                                Stream strIndex = File.Open("dT.index", FileMode.CreateNew);
                                
                                BinaryReader br = new BinaryReader(str);            
                                BinaryWriter bw = new BinaryWriter(strIndex);
                                
                                while ((b = br.BaseStream.ReadByte()) != -1)
                                {
                                    if (br.BaseStream.Position == x)
                                    {
                                        // below is the data that I want to write
                                        bw.Write(1);
                                        bw.Write("StringData");
                                        bw.Write("StringData");
                                        bw.Flush();
                                        continue;
                                    }
                                    bw.Write(b);
                                }
                        
                                bw.Flush();
                                bw.Close();
                                br.Close();
                        
                                str.Close();
                                strIndex.Close();           
                            }
                        

                        Hope there is a better solution, something like seek into the binary file and replace the data. :-D Cheer's, Alex Manolescu.

                        P Offline
                        P Offline
                        petercrab
                        wrote on last edited by
                        #12

                        what you have written already looks ok to me. Like others have said, so long as binary file is not too large you can get a speed increase by loading entire file to memory in one read, modifying, then writing the output file. many thanks, petercrab

                        1 Reply Last reply
                        0
                        Reply
                        • Reply as topic
                        Log in to reply
                        • Oldest to Newest
                        • Newest to Oldest
                        • Most Votes


                        • Login

                        • Don't have an account? Register

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • World
                        • Users
                        • Groups