Out of Memory Exception while reading Text File

Revathij

Hi, My requirement is to read the file one by one and update the content in database. I'm getting Out of memory Exception while reading 32Mb text file. I ve used streamReader to read the File. StreamReader strReader = new StreamReader(filePath); return strReader.ReadToEnd(); I have tried with ReadLine() also. Same error comes. Is there any solutions for this? Thanks in advance

Manas Bhardwaj

String _result = String.Empty;
using(StreamReader strReader = new StreamReader(filePath))
{
_result = strReader.ReadToEnd();
}
return _result;

Manas Bhardwaj Please remember to rate helpful or unhelpful answers, it lets us and people reading the forums know if our answers are any good.

Keith Barrow

The problem is because your machine doesn't have enough memory to read the string, probably because the buffer required can't be created.See http://msdn.microsoft.com/en-us/library/system.io.streamreader.readtoend(VS.80).aspx[[^](http://msdn.microsoft.com/en-us/library/system.io.streamreader.readtoend
(VS.80).aspx "New Window")] You'll have more success reading each files in chunks, see http://msdn.microsoft.com/en-us/library/9kstw824(VS.80).aspx[^] Hope this helps!

Luc Pattyn

Hi, how many files (and how many megabytes of accumulated file data) does it take to get the OOM Exception? I suspect you are getting large-heap fragmentation. :)

Luc Pattyn [Forum Guidelines] [My Articles]

DISCLAIMER: this message may have been modified by others; it may no longer reflect what I intended, and may contain bad advice; use at your own risk and with extreme care.

Dan Neely

Luc is probably right about heap fragmentation being the problem. Instead of creating new strings with each read and flogging the heap to death like this:

string fileText ="";

while (strReader.EndOfFile())
{
fileText += strReader.ReadLine();
}

return fileText;

Use a stringBuilder initialized to the size of the file so that you only have a single memory allocation.

stringBuilder fileText = new stringBuilder(sizeOfFile);

while (strReader.EndOfFile())
{
fileText.AppendLine(strReader.ReadLine());
}

return fileText;

It is a truth universally acknowledged that a zombie in possession of brains must be in want of more brains. -- Pride and Prejudice and Zombies