Plotting large amounts of data
-
Hello, I am half way through developing a graphing control. It includes the normal things like pan/zoom/multiple y axes. It is intended to be populated by reading data from a CSV file or something like that, but I can change that to whatever later if I need to - it's just a datasource. The trouble I am having is with the amount of data. Lets say I have 131200 data records to plot, and only 1000 pixels to plot them in. Now I have two options: 1. Plot the record for each pixel. 2. Plot every record, using PointF's and letting windows sort it out. The downsides are (respectively): 1. Data is missed, or stepped over. If we zoom in the data changes, this doesn't look good. 2. It takes ages drawing a 131200 points of data, and when zooming/panning it's slow. I have thought about plotting the data onto a offscreen bitmap so that panning is fast, but that doesn't help zooming. Please don't suggest using transformations - they are slow when using large data sets. A bitmap of the data size would also be unusable ( e.g. 131200x600x24 bit is about 250 Mb). Any help on how I could plot the image for both speed and accruacy. Many thanks,
-
Hello, I am half way through developing a graphing control. It includes the normal things like pan/zoom/multiple y axes. It is intended to be populated by reading data from a CSV file or something like that, but I can change that to whatever later if I need to - it's just a datasource. The trouble I am having is with the amount of data. Lets say I have 131200 data records to plot, and only 1000 pixels to plot them in. Now I have two options: 1. Plot the record for each pixel. 2. Plot every record, using PointF's and letting windows sort it out. The downsides are (respectively): 1. Data is missed, or stepped over. If we zoom in the data changes, this doesn't look good. 2. It takes ages drawing a 131200 points of data, and when zooming/panning it's slow. I have thought about plotting the data onto a offscreen bitmap so that panning is fast, but that doesn't help zooming. Please don't suggest using transformations - they are slow when using large data sets. A bitmap of the data size would also be unusable ( e.g. 131200x600x24 bit is about 250 Mb). Any help on how I could plot the image for both speed and accruacy. Many thanks,
Hi, I am curious to ask you if you have come up with a strategy for your solution yet, and, have you considered doing this in WPF rather than WinForms. I think a critical factor here would the latency of time-to-display you, or the end-user, would be willing to tolerate. Can the end-user manipulate the size of the view-port ? You mention having "1000 pixels to plot them in:" I'm trying to infer from that number a possible size of view-port, and ... assuming 3 bytes per pixel for color information, or 4 for color + alpha ... I come up with a very small view-port: unless each virtual-pixel is 'magnified' into a chunk of many screen pixels. A final question: can you assume that you are dealing with a 'static' data-set here, or, are you getting real-time changes ? The discussion here[^] may give you some ideas for strategy. My guess is you are going to need 'unsafe' access to the 'raw' bytes. ... edit 1 ... Can you assume the consumer of your application has a certain amount of memory available ? The discussion here[^] (particularly the comments by MusiGeneis and Hans Passant) may be useful to understand some 'misleading' aspects of the GDI+ "out of memory" Exception, and Passant's answer provides a useful Enum of GDI+ error codes you can use. ... end edit 1 ... best, Bill
"Reason is the natural order of truth; but imagination is the organ of meaning." C.S. Lewis
modified on Monday, July 18, 2011 3:05 AM