How would you create a virtual TextEditor?
-
Hi folks, for some weeks now, I'm getting my head around an interesting problem: I'd like to display a text file of several gigs, and let it be editable. Now - how would you do that? All standard approaches fail, because they rely on the file being fully loaded into RAM, which is impossible with several gigabytes of data. So, I thought I'll create a CustomControl. Basically, I've derived a Control from the Control class, which embeds a FrameworkElement in it's Generic.xaml. This FrameworkElement has a VisualCollection. Now, I hand over a Stream to this control. The Stream must be seekable and writable, otherwise you will encounter an exception. So far, I do the following: I read a chunk of data from the Stream, draw that using FormattedText on a DrawingContext, retrieved by a DrawingVisual. For scrolling, I simply adjust the offset of the FormattedText-function. To achieve seamless scrolling (which is needed, so you can scroll from one buffer chunk to the next without a visible gap), I have a function which calculates the amount of text that fits precisely into a rectangle (e.g. the Window's viewport), so that the rest of the current buffer, together with the next buffer, can be printed on the next rectangle. But, for the sake of it, I can't find out how I would do standard editing functionality. Because, for the scrollbar and this "chunking" to work, one needs to precompute the needed amount of text to fill the rectangle, which changes when editing the text. tl;dr: Do you have a smart concept on hand, together with WPF and C#, to create a TextEditor, which is capable of editing gigabytes of text?
-
Hi folks, for some weeks now, I'm getting my head around an interesting problem: I'd like to display a text file of several gigs, and let it be editable. Now - how would you do that? All standard approaches fail, because they rely on the file being fully loaded into RAM, which is impossible with several gigabytes of data. So, I thought I'll create a CustomControl. Basically, I've derived a Control from the Control class, which embeds a FrameworkElement in it's Generic.xaml. This FrameworkElement has a VisualCollection. Now, I hand over a Stream to this control. The Stream must be seekable and writable, otherwise you will encounter an exception. So far, I do the following: I read a chunk of data from the Stream, draw that using FormattedText on a DrawingContext, retrieved by a DrawingVisual. For scrolling, I simply adjust the offset of the FormattedText-function. To achieve seamless scrolling (which is needed, so you can scroll from one buffer chunk to the next without a visible gap), I have a function which calculates the amount of text that fits precisely into a rectangle (e.g. the Window's viewport), so that the rest of the current buffer, together with the next buffer, can be printed on the next rectangle. But, for the sake of it, I can't find out how I would do standard editing functionality. Because, for the scrollbar and this "chunking" to work, one needs to precompute the needed amount of text to fill the rectangle, which changes when editing the text. tl;dr: Do you have a smart concept on hand, together with WPF and C#, to create a TextEditor, which is capable of editing gigabytes of text?
I'm curious, is there any text editor that will handle gb of data?
Never underestimate the power of human stupidity RAH
-
I'm curious, is there any text editor that will handle gb of data?
Never underestimate the power of human stupidity RAH
Not really, at least for Windows. On the one hand, there is Large Text File Viewer, but that is hard to find at the moment. EmEditor is commercial Software, but disables word wrapping and some other options, afaik. And I don't think editing will work. Then, there is HxD, a Hexeditor. It can even read your disk. That would be the last option, cause with a hex editor, you dont have to deal with line breaks etc. Everything can be set to fixed with. But that would be too easy.^^ A little bit easier to approach would be paging, I guess, instead of scrolling. But that is not so nice, but would be an Option, too.
-
Not really, at least for Windows. On the one hand, there is Large Text File Viewer, but that is hard to find at the moment. EmEditor is commercial Software, but disables word wrapping and some other options, afaik. And I don't think editing will work. Then, there is HxD, a Hexeditor. It can even read your disk. That would be the last option, cause with a hex editor, you dont have to deal with line breaks etc. Everything can be set to fixed with. But that would be too easy.^^ A little bit easier to approach would be paging, I guess, instead of scrolling. But that is not so nice, but would be an Option, too.
Personally I would go for chaptering or chunking the physical data on the drive, split it up into edible volumes, store it and when print, ghu forbid, put it back together. What sort of data is it that is gb big and requires editing? In a text editor!
Never underestimate the power of human stupidity RAH
-
Personally I would go for chaptering or chunking the physical data on the drive, split it up into edible volumes, store it and when print, ghu forbid, put it back together. What sort of data is it that is gb big and requires editing? In a text editor!
Never underestimate the power of human stupidity RAH
Hi, thanks for your thoughts on that. When splitting up into edible volumes, how would you handle selections across the volumes? To put things straight - it's not about solving an actual problem. Some time ago, I worked with databases. There is nothing more frustrating than dumping a large MySQL database, just to realize after some hours, that the "CREATE TABLE" statement has been added to the export. Just as an example I've in my mind now. But there is no customer request behind that. It's just an idea, stuck in my head for a few months now, which I'd like to solve. What I've got so far: If you use the text approach, chunking would be the best option. There is no need to split the files on the drive, but you would need some kind of controller which would manage the changes being made, map them to the chunks and save them if necessary. This would require paging for navigation, because scrolling would simply be to difficult if you need to take the window size and linebreaks etc. into account. If I want scrolling, I should take the Hex-Editor approach, that is: Set the textfield to a fixed amount of characters per line, and don't care for linebreaks, etc. I'm still open for more ideas, though. :)
-
Hi folks, for some weeks now, I'm getting my head around an interesting problem: I'd like to display a text file of several gigs, and let it be editable. Now - how would you do that? All standard approaches fail, because they rely on the file being fully loaded into RAM, which is impossible with several gigabytes of data. So, I thought I'll create a CustomControl. Basically, I've derived a Control from the Control class, which embeds a FrameworkElement in it's Generic.xaml. This FrameworkElement has a VisualCollection. Now, I hand over a Stream to this control. The Stream must be seekable and writable, otherwise you will encounter an exception. So far, I do the following: I read a chunk of data from the Stream, draw that using FormattedText on a DrawingContext, retrieved by a DrawingVisual. For scrolling, I simply adjust the offset of the FormattedText-function. To achieve seamless scrolling (which is needed, so you can scroll from one buffer chunk to the next without a visible gap), I have a function which calculates the amount of text that fits precisely into a rectangle (e.g. the Window's viewport), so that the rest of the current buffer, together with the next buffer, can be printed on the next rectangle. But, for the sake of it, I can't find out how I would do standard editing functionality. Because, for the scrollbar and this "chunking" to work, one needs to precompute the needed amount of text to fill the rectangle, which changes when editing the text. tl;dr: Do you have a smart concept on hand, together with WPF and C#, to create a TextEditor, which is capable of editing gigabytes of text?
It might be worth looking at Memory-Mapped files[^], which would allow you to map sections of the file into memory. No idea how well that would fit with your editor, though. Once you've finished, this sounds like a good topic for an article! :)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
-
It might be worth looking at Memory-Mapped files[^], which would allow you to map sections of the file into memory. No idea how well that would fit with your editor, though. Once you've finished, this sounds like a good topic for an article! :)
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
Hi Richard, thanks for your comment. I've heard of MMF, but haven't used it before. Sounds interesting, although that wouldn't solve the scrolling part. And thanks for the hint in terms of an article. I might consider that, why not. :)
-
Hi folks, for some weeks now, I'm getting my head around an interesting problem: I'd like to display a text file of several gigs, and let it be editable. Now - how would you do that? All standard approaches fail, because they rely on the file being fully loaded into RAM, which is impossible with several gigabytes of data. So, I thought I'll create a CustomControl. Basically, I've derived a Control from the Control class, which embeds a FrameworkElement in it's Generic.xaml. This FrameworkElement has a VisualCollection. Now, I hand over a Stream to this control. The Stream must be seekable and writable, otherwise you will encounter an exception. So far, I do the following: I read a chunk of data from the Stream, draw that using FormattedText on a DrawingContext, retrieved by a DrawingVisual. For scrolling, I simply adjust the offset of the FormattedText-function. To achieve seamless scrolling (which is needed, so you can scroll from one buffer chunk to the next without a visible gap), I have a function which calculates the amount of text that fits precisely into a rectangle (e.g. the Window's viewport), so that the rest of the current buffer, together with the next buffer, can be printed on the next rectangle. But, for the sake of it, I can't find out how I would do standard editing functionality. Because, for the scrollbar and this "chunking" to work, one needs to precompute the needed amount of text to fill the rectangle, which changes when editing the text. tl;dr: Do you have a smart concept on hand, together with WPF and C#, to create a TextEditor, which is capable of editing gigabytes of text?
For the scrolling, as soon as you have more than a thousand line or so, you won't be able to reach each line by using the scrollbar so making approximations might be enough... Or you can divide the file into blocks of a few KB and keep some information like the number of lines in that block or the actual size on screen once the information is known. In practice, I think that using approximations and make adjustment as required later could be more than enough. And you can probably buy already made editor component that works with big files. Finally, if files are so large, even if technical problems are solved, it would still be hard to use in many scenarios.
Philippe Mori
-
For the scrolling, as soon as you have more than a thousand line or so, you won't be able to reach each line by using the scrollbar so making approximations might be enough... Or you can divide the file into blocks of a few KB and keep some information like the number of lines in that block or the actual size on screen once the information is known. In practice, I think that using approximations and make adjustment as required later could be more than enough. And you can probably buy already made editor component that works with big files. Finally, if files are so large, even if technical problems are solved, it would still be hard to use in many scenarios.
Philippe Mori
I guess I go with the approximation approach, that sounds kind of reasonable.
