HELP! Reading Files...
-
Ok, in no way am I a N00b at reading files! However, I was debugging some code today in which I was using a
IO.StreamReaderto read the text from a file and using anIO.StreamWriterto write it into a different file. This file is about 3.2MB. TheStreamReaderDOES return the Correct Length of the File (IO.StreamReader.BaseStream.Length). HOWEVER, while reading the file, the Stream THINKS it has reached the end, and like a good little programmer, IFlushthe StreamWriter (BTW, I am flushing every 8096 Bytes - yes, that IS a lot of water!) but I am missing a HUGE chunk of data at the End of my file! I have noticed that theIO.StreamReader.BaseStream.Positionis usually TOTALLY different than the Sum of the Bytes returned for each re-iteration of retrieving data. For example, reading in 8096 Bytes at a time, when my Position variable thinks it's at 16,192, the BaseStream has it's internal buffer set to some other amount... Is this what's messing it up? Notice however, it I REALLY want to waste a lot of memory and processing time, I can just read it all in at once (IO.StreamReader.ReadToEnd) and then process it....GreeeEEEAAAaaaat!!! Just what I want! NOT!!! So...what the @%$* is going on!??? How do I read the rest of the data!??? :wtf: -
Ok, in no way am I a N00b at reading files! However, I was debugging some code today in which I was using a
IO.StreamReaderto read the text from a file and using anIO.StreamWriterto write it into a different file. This file is about 3.2MB. TheStreamReaderDOES return the Correct Length of the File (IO.StreamReader.BaseStream.Length). HOWEVER, while reading the file, the Stream THINKS it has reached the end, and like a good little programmer, IFlushthe StreamWriter (BTW, I am flushing every 8096 Bytes - yes, that IS a lot of water!) but I am missing a HUGE chunk of data at the End of my file! I have noticed that theIO.StreamReader.BaseStream.Positionis usually TOTALLY different than the Sum of the Bytes returned for each re-iteration of retrieving data. For example, reading in 8096 Bytes at a time, when my Position variable thinks it's at 16,192, the BaseStream has it's internal buffer set to some other amount... Is this what's messing it up? Notice however, it I REALLY want to waste a lot of memory and processing time, I can just read it all in at once (IO.StreamReader.ReadToEnd) and then process it....GreeeEEEAAAaaaat!!! Just what I want! NOT!!! So...what the @%$* is going on!??? How do I read the rest of the data!??? :wtf:Wait a tick, it gets even better! I just discovered that it's Reading past the end of the File and returning more data than it should! So, I know I need to use this handy-dandy Routine MS gave us:
IO.StreamReader.DiscardBufferedData(). Soo, anyone have a clue as to how this should be done?? Any examples? Becuase the Help doesn't...well, help! :confused: -
Ok, in no way am I a N00b at reading files! However, I was debugging some code today in which I was using a
IO.StreamReaderto read the text from a file and using anIO.StreamWriterto write it into a different file. This file is about 3.2MB. TheStreamReaderDOES return the Correct Length of the File (IO.StreamReader.BaseStream.Length). HOWEVER, while reading the file, the Stream THINKS it has reached the end, and like a good little programmer, IFlushthe StreamWriter (BTW, I am flushing every 8096 Bytes - yes, that IS a lot of water!) but I am missing a HUGE chunk of data at the End of my file! I have noticed that theIO.StreamReader.BaseStream.Positionis usually TOTALLY different than the Sum of the Bytes returned for each re-iteration of retrieving data. For example, reading in 8096 Bytes at a time, when my Position variable thinks it's at 16,192, the BaseStream has it's internal buffer set to some other amount... Is this what's messing it up? Notice however, it I REALLY want to waste a lot of memory and processing time, I can just read it all in at once (IO.StreamReader.ReadToEnd) and then process it....GreeeEEEAAAaaaat!!! Just what I want! NOT!!! So...what the @%$* is going on!??? How do I read the rest of the data!??? :wtf: -
Ok, I found the answer to this. I'm sure most of you do not need this functionality and only use "ReadLine"... However, try parsing ANSI 835 files ;P . (Yes, I'm aware there are 3rd party Libraries for this, but I hate them) - The End ;P
mikasa wrote: Ok, I found the answer So can you kindly share your solution with the CPians ? Thanks.
R.Bischoff
-
mikasa wrote: Ok, I found the answer So can you kindly share your solution with the CPians ? Thanks.
R.Bischoff
Sure thing, just curious if anyone would reply ;P
'Function to Read a File into Cache so RegEx can be Performed
Private Sub CacheFile(ByVal File As String)
Dim ioReader As IO.StreamReader
Dim vBuffer() As Char
Dim hResult As Integer'Exit if the File does not exist If (File = "") Then Return If (Not IO.File.Exists(File)) Then Return If (Me.\_File <> File) Then Me.\_File = File Else Return Try 'Initialize the Cache \_Cache = Nothing : \_Cache = New System.Text.StringBuilder(8096) 'Read the File ReDim vBuffer(8096) ioReader = New IO.StreamReader(File, System.Text.Encoding.UTF8, True, 8096) hResult = ioReader.Read(vBuffer, 0, vBuffer.Length) 'Read More Data While (hResult > 0) 'There is a bug with a StreamReader. Sometimes more data is read than is neccessary. 'If these are the Last Bytes of data, only read up to the hResult If (hResult < vBuffer.Length) Then \_Cache.Append(vBuffer, 0, hResult) Else \_Cache.Append(vBuffer, 0, vBuffer.Length) hResult = ioReader.Read(vBuffer, 0, vBuffer.Length) 'Read More Data End While Catch ex As Exception : MsgBox(ex.ToString) Finally Erase vBuffer If (Not IsNothing(ioReader)) Then ioReader.Close() : ioReader = Nothing End TryEnd Sub
-
Sure thing, just curious if anyone would reply ;P
'Function to Read a File into Cache so RegEx can be Performed
Private Sub CacheFile(ByVal File As String)
Dim ioReader As IO.StreamReader
Dim vBuffer() As Char
Dim hResult As Integer'Exit if the File does not exist If (File = "") Then Return If (Not IO.File.Exists(File)) Then Return If (Me.\_File <> File) Then Me.\_File = File Else Return Try 'Initialize the Cache \_Cache = Nothing : \_Cache = New System.Text.StringBuilder(8096) 'Read the File ReDim vBuffer(8096) ioReader = New IO.StreamReader(File, System.Text.Encoding.UTF8, True, 8096) hResult = ioReader.Read(vBuffer, 0, vBuffer.Length) 'Read More Data While (hResult > 0) 'There is a bug with a StreamReader. Sometimes more data is read than is neccessary. 'If these are the Last Bytes of data, only read up to the hResult If (hResult < vBuffer.Length) Then \_Cache.Append(vBuffer, 0, hResult) Else \_Cache.Append(vBuffer, 0, vBuffer.Length) hResult = ioReader.Read(vBuffer, 0, vBuffer.Length) 'Read More Data End While Catch ex As Exception : MsgBox(ex.ToString) Finally Erase vBuffer If (Not IsNothing(ioReader)) Then ioReader.Close() : ioReader = Nothing End TryEnd Sub
In your code you have:
_Cache = Nothing : _Cache = New System.Text.StringBuilder(8096)
Isn't the first statement somewhat redundant?
"You can have everything in life you want if you will just help enough other people get what they want." --Zig Ziglar The Second EuroCPian Event will be in Brussels on the 4th of September Can't manage to P/Invoke that Win32 API in .NET? Why not do interop the wiki way!
-
In your code you have:
_Cache = Nothing : _Cache = New System.Text.StringBuilder(8096)
Isn't the first statement somewhat redundant?
"You can have everything in life you want if you will just help enough other people get what they want." --Zig Ziglar The Second EuroCPian Event will be in Brussels on the 4th of September Can't manage to P/Invoke that Win32 API in .NET? Why not do interop the wiki way!