Read & Display a huge text file in textbox
-
Dengbo wrote:
reading and displaying a huge text file (>=300MB) in a textbox.
that sounds like extreme non-sense; who in his right mind is going to read one huge page of text like that? :(
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
All Toronto weekends should be extremely wet until we get it automated in regular forums, not just QA.
Luc Pattyn wrote:
Dengbo wrote: reading and displaying a huge text file (>=300MB) in a textbox. that sounds like extreme non-sense; who in his right mind is going to read one huge page of text like that?
you know, some routing reported files (shop order, job cost, etc) in ERP can be very huge...
--------------------------------- Believe what you saw!
-
Use your own scrollbars (HScrollBar, VScrollBar). Set the Maximum on the HScrollBar to the size of the textfile / 80 (or whatever average line length you think would be appropriate). When somebody scrolls, you can approximate where they are in the file and load up that text into a fixed-size TextBox. When you change the text as they scroll, you'll want to remember the position of the caret and the selected text so you can reapply them after you change the text. Also, intercept key presses for up/down/left/right/page down/page up/etc so you can implement those functions yourself. For the HScrollBar Maximum, default it to 80. Like the VScrollBar, you should use the position of the HScrollBar to scroll through the fixed-size textbox (which really means changing the text in the textbox. In a background thread, scan through the entire text file to find line breaks and actual line widths and the number of lines in the file. Store the positions of the line breaks in a file (text file, binary file, SQL Server CE file) so that you can get information about a given line by looking at that file (this helps you avoid using RAM). Once you have scanned the entire file, you can adjust the Maximum for the HScrollBar and VScrollBar to the actual values (i.e., the number of lines in the file and the number of characters in the widest line). Having the offset of the line beginnings stored in a file will allow you to instantly determine where in the text file the data is for a given line number. This will allow you to create perfect scrolling that is virtually instantaneous (the scrolling will be instantaneous and approximate up until you have loaded everything from the text file). It gets a little more complicated if you want to allow the user to make edits. In that case, you'd probably want to asynchronously load each line of the file into a SQL Server CE database. Whenever an edit is made, you'd have to intercept it (e.g., via keypress) and made the corresponding edit to that line in the database. If you want to save the file, you'd then store each of those lines in the database back to the text file. Or, if you want all edits to immediately be saved to the text file, you could just avoid the database altogether. There are other details you'll probably have to worry about, but I trust you can figure them out. Note that SQL Server CE has a maximum size of about 4GB. If you need larger than that, you can choose another database format or use your own file format.
Hi aspdotnetdev, Thank you very much for giving me so detailed tips. Yes, that's what I'm trying to implement. A good news is that the textbox will be readonly, user only has the read, find, print options for this huge text. dengbo.
--------------------------------- Believe what you saw!
-
Hey, I'm suffering an issue of reading and displaying a huge text file (>=300MB) in a textbox. I found that the maximum string I can read into memory is about 250MB because of the limitation of the RAM and also likely related to the limitation of the .net component. (when the text is very large to display in a textbox, out of memory exception will be thrown) So in this case, the only way we can do is to show the text file on demand, which is likely what the UltraEdit does. Could anyone give me some tips or solutions about this issue? Thanks in advance. dengbo
--------------------------------- Believe what you saw!
If it is a log file, I suggest saving it with xml node info. This can then be hooked to some database (file to xml) technique to enable search features. Also, it will be more organized data. If this log file is not created by you, then it sounds like a daunting task.
Dengbo wrote:
So you have 2 vertical scrollbars. One which scrolls the full height, one which scrolls 5000 lines at a time
This gives me an idea. What you can do is not to have text in the textbox. Show only the info that the user wants. Instead of adding ScrollBars, add buttons to scroll page up or down. This will enable a paging possibility and it will be easier to handle too.
-
If it is a log file, I suggest saving it with xml node info. This can then be hooked to some database (file to xml) technique to enable search features. Also, it will be more organized data. If this log file is not created by you, then it sounds like a daunting task.
Dengbo wrote:
So you have 2 vertical scrollbars. One which scrolls the full height, one which scrolls 5000 lines at a time
This gives me an idea. What you can do is not to have text in the textbox. Show only the info that the user wants. Instead of adding ScrollBars, add buttons to scroll page up or down. This will enable a paging possibility and it will be easier to handle too.
FYI, you misattributed that quote... I wrote that.
Som Shekhar wrote:
add buttons to scroll page up or down
That is what the PAGE UP/PAGE DOWN buttons would do. It would be, however, quite slow when paging 5,000 lines or so.
Som Shekhar wrote:
Instead of adding ScrollBars
Why not have both ScrollBars and PAGE UP/PAGE DOWN functionality?
-
FYI, you misattributed that quote... I wrote that.
Som Shekhar wrote:
add buttons to scroll page up or down
That is what the PAGE UP/PAGE DOWN buttons would do. It would be, however, quite slow when paging 5,000 lines or so.
Som Shekhar wrote:
Instead of adding ScrollBars
Why not have both ScrollBars and PAGE UP/PAGE DOWN functionality?
Well, I was thinking of having 5+ string builder objects. 1 for current text in textbox 1 for next page 1 for previous page 1 for 5000 lines ahead page 1 for 5000 lines behind page you can add more if you wish in the same way. You need to maintain few indexes in your program to read only those blocks using StringBuilder.ReadBlock method. Searching is something I am not sure of.
-
Well, I was thinking of having 5+ string builder objects. 1 for current text in textbox 1 for next page 1 for previous page 1 for 5000 lines ahead page 1 for 5000 lines behind page you can add more if you wish in the same way. You need to maintain few indexes in your program to read only those blocks using StringBuilder.ReadBlock method. Searching is something I am not sure of.
The problem is not the buffer... the speed issue is from the user... it will take many pages before they traverse 5,000 lines of text. The text can be loaded from the file on-demand without the user noticing. The trick is to reduce the amount the user must interact with the UI.
-
The problem is not the buffer... the speed issue is from the user... it will take many pages before they traverse 5,000 lines of text. The text can be loaded from the file on-demand without the user noticing. The trick is to reduce the amount the user must interact with the UI.
Now, this confuses me. What exactly is your question then? Reading your question, it sounded like displaying it in Textbox is the problem. Are you talking about Search? or some other thing?
-
Now, this confuses me. What exactly is your question then? Reading your question, it sounded like displaying it in Textbox is the problem. Are you talking about Search? or some other thing?
I am not the one with the question. Look at my username. I was merely correcting you.
-
I am not the one with the question. Look at my username. I was merely correcting you.
HA HA HA... This was funny... I am so sorry!!!!! HA HA HA... Now I see, I misquoted you, Misunderstood you and everything... just because you replied me first... Sorry Buddy.
-
Luc Pattyn wrote:
Dengbo wrote: reading and displaying a huge text file (>=300MB) in a textbox. that sounds like extreme non-sense; who in his right mind is going to read one huge page of text like that?
you know, some routing reported files (shop order, job cost, etc) in ERP can be very huge...
--------------------------------- Believe what you saw!
Dengbo wrote:
some routing reported files (shop order, job cost, etc) in ERP can be very huge...
I know, but that does not mean I want to see all their content, I want the app to present a decent user interface, not a truck load of data. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
All Toronto weekends should be extremely wet until we get it automated in regular forums, not just QA.
-
The problem is not the buffer... the speed issue is from the user... it will take many pages before they traverse 5,000 lines of text. The text can be loaded from the file on-demand without the user noticing. The trick is to reduce the amount the user must interact with the UI.
aspdotnetdev wrote:
The trick is to reduce the amount the user must interact with the UI.
this is where we agree. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
All Toronto weekends should be extremely wet until we get it automated in regular forums, not just QA.