Read large text files with c#
-
I know that i cannot read lets say 80mb big file into a textbox, first i takes too long and second it would take alot of memory. I need to find a approach to read sections from file to textbox and when user scrolls down or push page down it read another section... Is that a good approach to problem or do anyone have another approach to read large textfiles fast... <Im really impressed what Christan Ghisler done with his lister, its really fast.>) Im open for ideas how to solve this problem... -- modified at 12:13 Tuesday 7th March, 2006
-
I know that i cannot read lets say 80mb big file into a textbox, first i takes too long and second it would take alot of memory. I need to find a approach to read sections from file to textbox and when user scrolls down or push page down it read another section... Is that a good approach to problem or do anyone have another approach to read large textfiles fast... <Im really impressed what Christan Ghisler done with his lister, its really fast.>) Im open for ideas how to solve this problem... -- modified at 12:13 Tuesday 7th March, 2006
-
I know that i cannot read lets say 80mb big file into a textbox, first i takes too long and second it would take alot of memory. I need to find a approach to read sections from file to textbox and when user scrolls down or push page down it read another section... Is that a good approach to problem or do anyone have another approach to read large textfiles fast... <Im really impressed what Christan Ghisler done with his lister, its really fast.>) Im open for ideas how to solve this problem... -- modified at 12:13 Tuesday 7th March, 2006
You basically have two options: 1. Open the file as a textfile, but then you can only read it sequentially. 2. Open the file as a binary file, then you have to detect the line breaks yourself, and convert the bytes to text using the Encoding class, but you can read from any part of the file. --- b { font-weight: normal; }
-
I know that i cannot read lets say 80mb big file into a textbox, first i takes too long and second it would take alot of memory. I need to find a approach to read sections from file to textbox and when user scrolls down or push page down it read another section... Is that a good approach to problem or do anyone have another approach to read large textfiles fast... <Im really impressed what Christan Ghisler done with his lister, its really fast.>) Im open for ideas how to solve this problem... -- modified at 12:13 Tuesday 7th March, 2006
System.IO.File in .Net 2.0 is really fast and has some new methods like ReadAllText() and ReadAllLines(). Why not some custom methods for your app? Later, JoeSox "Football is a game of cliches, and I believe in every one of them." -Vincent Lombardi CPMCv1.0 ↔ humanaiproject.org ↔ Last.fm
-
I have created a simple program that does something similar. It will read a file in chunks using a reader and provides forward only navigation. If you send me an email I can send you the source code (C# vs2005) andrew at pmall dot com
-
System.IO.File in .Net 2.0 is really fast and has some new methods like ReadAllText() and ReadAllLines(). Why not some custom methods for your app? Later, JoeSox "Football is a game of cliches, and I believe in every one of them." -Vincent Lombardi CPMCv1.0 ↔ humanaiproject.org ↔ Last.fm
-
You basically have two options: 1. Open the file as a textfile, but then you can only read it sequentially. 2. Open the file as a binary file, then you have to detect the line breaks yourself, and convert the bytes to text using the Encoding class, but you can read from any part of the file. --- b { font-weight: normal; }
I dont quite get what you are saying here.. In this case it would be possible to read chunks from whereever you like in file...or did i dont understand what you saying?
FileStream fs=new FileStream(fileName, FileMode.Open, FileAccess.ReadWrite, FileShare.None, 8192 ); //we use the default buffer StringBuilder result=new StringBuilder(8192); byte[] buffer=new byte[8192]; int count=fs.Read(buffer,0,8192); //reading a block from the file, here we could start for example on 2000 instead of pos 0 //(fs.Read(buffer,2000,8192)) ASCIIEncoding encoding=new ASCIIEncoding(); while (count!=0) //stop when Read returns 0 { result.Append(encoding.GetString(buffer)); count=fs.Read(buffer,0,8192); } fs.Seek(0, SeekOrigin.Begin); -
I dont quite get what you are saying here.. In this case it would be possible to read chunks from whereever you like in file...or did i dont understand what you saying?
FileStream fs=new FileStream(fileName, FileMode.Open, FileAccess.ReadWrite, FileShare.None, 8192 ); //we use the default buffer StringBuilder result=new StringBuilder(8192); byte[] buffer=new byte[8192]; int count=fs.Read(buffer,0,8192); //reading a block from the file, here we could start for example on 2000 instead of pos 0 //(fs.Read(buffer,2000,8192)) ASCIIEncoding encoding=new ASCIIEncoding(); while (count!=0) //stop when Read returns 0 { result.Append(encoding.GetString(buffer)); count=fs.Read(buffer,0,8192); } fs.Seek(0, SeekOrigin.Begin);Yes, you can use the Seek method to go to any position in the file. Note: There is an error in this comment in the code:
//reading a block from the file, here we could start for example on 2000 instead of pos 0 //(fs.Read(buffer,2000,8192))
Using that call would result in an error message. As the size of the array is 8192 bytes, you can't read 8192 bytes and put them in the array from index 2000. Between index 2000 and the end of the array there is only room for 6192 bytes. --- b { font-weight: normal; }
-
System.IO.File in .Net 2.0 is really fast and has some new methods like ReadAllText() and ReadAllLines(). Why not some custom methods for your app? Later, JoeSox "Football is a game of cliches, and I believe in every one of them." -Vincent Lombardi CPMCv1.0 ↔ humanaiproject.org ↔ Last.fm
-
Yes, you can use the Seek method to go to any position in the file. Note: There is an error in this comment in the code:
//reading a block from the file, here we could start for example on 2000 instead of pos 0 //(fs.Read(buffer,2000,8192))
Using that call would result in an error message. As the size of the array is 8192 bytes, you can't read 8192 bytes and put them in the array from index 2000. Between index 2000 and the end of the array there is only room for 6192 bytes. --- b { font-weight: normal; }
-
ReadAllText() is synchronous method, when you try to read a large text file, your application will hang or slow down ... better to use asynchronous medthod !!!
hackerhcm wrote:
ReadAllText() is synchronous method, when you try to read a large text file, your application will hang or slow down ... better to use asynchronous medthod !!!
Of course, I was thinking of using ReadAllText() within the asynchronous medthod. Just read chunks of the 80mb from a temp file or something using ReadAllText(), when the user nears the end point create then read another chunk. Later, JoeSox "Football is a game of cliches, and I believe in every one of them." -Vincent Lombardi CPMCv1.0 ↔ humanaiproject.org ↔ Last.fm
-
I was thinking of using ReadAllText() within an asynchronous method. Just read chunks of the 80mb from a temp file using ReadAllText(), when the user nears the end point create, then read another chunk into the textbox.AppendText(tempfile.ReadAllText()). Later, JoeSox "Football is a game of cliches, and I believe in every one of them." -Vincent Lombardi CPMCv1.0 ↔ humanaiproject.org ↔ Last.fm