Fastest way to read entire file into memory
-
Which is currently the fastest method to read a text file into memory. I am using ifstream and get and would like to know of anything better.
-
Which is currently the fastest method to read a text file into memory. I am using ifstream and get and would like to know of anything better.
Are you doing ifstream::read()? What is the length of the data block you are reading? Try increasing the size of the chunks you read if it's reading too slowly. Reading 1 byte at a time is incredibly slow but reading 1MB at a time will be really fast.
-
Which is currently the fastest method to read a text file into memory. I am using ifstream and get and would like to know of anything better.
What are you doing with the file? Memory mapped files deliver impressive performance and may be suitable for your task. Neville Franks, Author of ED for Windows www.getsoft.com and coming soon: Surfulater www.surfulater.com
-
Are you doing ifstream::read()? What is the length of the data block you are reading? Try increasing the size of the chunks you read if it's reading too slowly. Reading 1 byte at a time is incredibly slow but reading 1MB at a time will be really fast.
The length of the data block is the entire filesize (which varies)...in my case it is around 500K at the moment...the entire file is read w/ one get. It is not that slow, I just thought there may be a faster way.
-
The length of the data block is the entire filesize (which varies)...in my case it is around 500K at the moment...the entire file is read w/ one get. It is not that slow, I just thought there may be a faster way.
I have experimented with this now and again and always found that blocks of 2,4 and 8k ranges provide the best performance. I have always assumed that there must be variants across systems that might effect this. OS, File System, Disk Drive and probably more. It is relatively simple to create a console app that allows you to test variations.
"No matter where you go, there your are." - Buckaroo Banzai
-pete
-
Which is currently the fastest method to read a text file into memory. I am using ifstream and get and would like to know of anything better.
Hmmmm! 1) Open file. 2) Get size of file. 3) Allocate enough memory to hold entire file. 4) Read file into buffer. Of cource this only applies if file size if smaller than the maximum buffer size that you can allocate. Under Win32, you can allocate very very large buffers. INTP
-
Hmmmm! 1) Open file. 2) Get size of file. 3) Allocate enough memory to hold entire file. 4) Read file into buffer. Of cource this only applies if file size if smaller than the maximum buffer size that you can allocate. Under Win32, you can allocate very very large buffers. INTP
Yep, that's exactly what I did, just making sure it's the fastest way. Thanks for the help.