working with binary files in C++
-
Hi guys I apologize for reposting this question but probably I didn't reach enough people last night and only got one response that didn't help me much (sorry Chris but thank you). I'm still strugling with the problem of reading a huge file in the most efficient way. I was suggested last week to "read the file into memory as one big chunk". I think that the advice refered to reading the data in binary format instead of text. Cool... so first I have to translate my text files into binary and then read them... Problem is that all examples that I've found refer to either read a single number or things such as a struct with phone directory... How about reading a 2 dimensional array of doubles as text, save it as binary and then load it into another array after reading the binary file??? examples??? Thanks a million.
-
Hi guys I apologize for reposting this question but probably I didn't reach enough people last night and only got one response that didn't help me much (sorry Chris but thank you). I'm still strugling with the problem of reading a huge file in the most efficient way. I was suggested last week to "read the file into memory as one big chunk". I think that the advice refered to reading the data in binary format instead of text. Cool... so first I have to translate my text files into binary and then read them... Problem is that all examples that I've found refer to either read a single number or things such as a struct with phone directory... How about reading a 2 dimensional array of doubles as text, save it as binary and then load it into another array after reading the binary file??? examples??? Thanks a million.
A file is neither binary nor text as it resides on disk. It all depends on how the file is processed that determines whether it is binary or text. You can open the file and read all of it into memory with something like:
HANDLE hFile = CreateFile("myfile", GENERIC_READ, ...);
LPBYTE lpBuffer = new BYTE[10240];
DWORD dwRead;
ReadFile(hFile, lpBuffer, sizeof(BYTE) * 10240, &dwRead, NULL);
CloseHandle(hFile);Now you can do whatever you want with
lpBuffer. You might also benefit from a memory-mapped file.
"Ideas are a dime a dozen. People who put them into action are priceless." - Unknown
-
A file is neither binary nor text as it resides on disk. It all depends on how the file is processed that determines whether it is binary or text. You can open the file and read all of it into memory with something like:
HANDLE hFile = CreateFile("myfile", GENERIC_READ, ...);
LPBYTE lpBuffer = new BYTE[10240];
DWORD dwRead;
ReadFile(hFile, lpBuffer, sizeof(BYTE) * 10240, &dwRead, NULL);
CloseHandle(hFile);Now you can do whatever you want with
lpBuffer. You might also benefit from a memory-mapped file.
"Ideas are a dime a dozen. People who put them into action are priceless." - Unknown
Hi Sorry if this sounds stupid, please bear with me since I'm a self instructed amateur. If files are neither text or binary, what's the point of using something like: To write a binary file
int MyNumber = 10; ofstream FileOut(FileName,ios::binary); FileOut.write((char*) &MyNumber, sizeof MyNumber); fout.close();and to readifstream FileIn(FileName, ios::binary) FileIn.read((char*) &MyNumber,sizeof MyNumber); FileIn.close();Thanks a million you for your help -
Hi Sorry if this sounds stupid, please bear with me since I'm a self instructed amateur. If files are neither text or binary, what's the point of using something like: To write a binary file
int MyNumber = 10; ofstream FileOut(FileName,ios::binary); FileOut.write((char*) &MyNumber, sizeof MyNumber); fout.close();and to readifstream FileIn(FileName, ios::binary) FileIn.read((char*) &MyNumber,sizeof MyNumber); FileIn.close();Thanks a million you for your helpios::binary is used to tell the ifstream and ofstream classes to treat the data as binary. If you don't specify ios::binary and try to read a binary file and if the binary file happens to have a non-text byte value, it'll stop there or throw an exception. You can open any file in binary mode though, because ifstream and ofstream will not try to interpret them as text characters. Regards Senthil _____________________________ My Blog | My Articles | WinMacro
-
ios::binary is used to tell the ifstream and ofstream classes to treat the data as binary. If you don't specify ios::binary and try to read a binary file and if the binary file happens to have a non-text byte value, it'll stop there or throw an exception. You can open any file in binary mode though, because ifstream and ofstream will not try to interpret them as text characters. Regards Senthil _____________________________ My Blog | My Articles | WinMacro
OK... so I wrote the following code and it writes and reads the file with a map. It compiles, runs and gives the requested output, but apparently there's a runtime problem since after execution a window pops indicating that there was a problem (one of those windows to send information to the creator of the program)... I can't see what's wrong but I know where the problem is since I commented different sections of the code and narrowed it down to the marked fragment below. Here's the code:
int main() { typedef map IMAP; IMAP Grid, NewGrid; int IntValue1, rows = 3; double DouValue2; for(int i=0; i < rows; i++) { IntValue1 = i + 1; DouValue2 = i * 2; Grid.insert(IMAP::value_type(IntValue1, DouValue2)); } IMAP::const_iterator IteratorG = Grid.begin(); cout << "Original Map" << endl; while (IteratorG != Grid.end() ) { cout << IteratorG->first << " " << IteratorG->second << endl; IteratorG ++; } ofstream FileOut("C:/MyBinary.bin" , ios::binary); FileOut.write((char*) &Grid, sizeof Grid); FileOut.close();THE ERROR IS FROM HEREifstream FileIn("C:/MyBinary.bin", ios::binary); FileIn.read((char*) &NewGrid,sizeof NewGrid); FileIn.close();TO HEREIMAP::const_iterator NewIteratorG = NewGrid.begin(); cout << " " << endl; cout << "New Map" << endl; while (NewIteratorG != NewGrid.end() ) { cout << NewIteratorG->first << " " << NewIteratorG->second << endl; NewIteratorG ++; } return 0; }Thanks a million! -
OK... so I wrote the following code and it writes and reads the file with a map. It compiles, runs and gives the requested output, but apparently there's a runtime problem since after execution a window pops indicating that there was a problem (one of those windows to send information to the creator of the program)... I can't see what's wrong but I know where the problem is since I commented different sections of the code and narrowed it down to the marked fragment below. Here's the code:
int main() { typedef map IMAP; IMAP Grid, NewGrid; int IntValue1, rows = 3; double DouValue2; for(int i=0; i < rows; i++) { IntValue1 = i + 1; DouValue2 = i * 2; Grid.insert(IMAP::value_type(IntValue1, DouValue2)); } IMAP::const_iterator IteratorG = Grid.begin(); cout << "Original Map" << endl; while (IteratorG != Grid.end() ) { cout << IteratorG->first << " " << IteratorG->second << endl; IteratorG ++; } ofstream FileOut("C:/MyBinary.bin" , ios::binary); FileOut.write((char*) &Grid, sizeof Grid); FileOut.close();THE ERROR IS FROM HEREifstream FileIn("C:/MyBinary.bin", ios::binary); FileIn.read((char*) &NewGrid,sizeof NewGrid); FileIn.close();TO HEREIMAP::const_iterator NewIteratorG = NewGrid.begin(); cout << " " << endl; cout << "New Map" << endl; while (NewIteratorG != NewGrid.end() ) { cout << NewIteratorG->first << " " << NewIteratorG->second << endl; NewIteratorG ++; } return 0; }Thanks a million!Writing out a std::map is not a good idea. It internally uses pointers and dynamically allocated memory, so writing the map will only write out the pointers and not their values. For eg
class Test
{
public:
int *p;
};int main()
{
Test test;
test.p = new int();
*(test.p) = 100;
}Writing out test to a file will write out p, which is just a address (and which will change for every program run). Because you're reading it back in the same program, you're alteast able to read back the values. Try reading the file after shutting down the program that wrote to it, you'll find that iteration itself crashes. I'd suggest writing out the values to file, manually and then reading them back, (again manually). Regards Senthil _____________________________ My Blog | My Articles | WinMacro
-
Writing out a std::map is not a good idea. It internally uses pointers and dynamically allocated memory, so writing the map will only write out the pointers and not their values. For eg
class Test
{
public:
int *p;
};int main()
{
Test test;
test.p = new int();
*(test.p) = 100;
}Writing out test to a file will write out p, which is just a address (and which will change for every program run). Because you're reading it back in the same program, you're alteast able to read back the values. Try reading the file after shutting down the program that wrote to it, you'll find that iteration itself crashes. I'd suggest writing out the values to file, manually and then reading them back, (again manually). Regards Senthil _____________________________ My Blog | My Articles | WinMacro
Hi Senthil Thanks a lot for your response. I've actually found that you are partly right. I can't send the whole map at once for the reasons that you point out. However, if you send the map element by element it does work like a charm. For me, the reason to work with this containers is because I need to read a humongous file and then find a record or a series of records associated to a tag or tags (in which case I'd use a multimap). These containers have algorithms that make the search process incredibly fast and easy. If you are interested in the new version of the code (the one that works) I'll be happy to post it. Cheers Carlos