What queuing technique and GUI component should I choose for my multi-comport, receive-only terminal software?
-
I have a number of devices with Com ports (UART:s running at MBaud speeds) that I need to monitor and make sense of. The devices send ASCII-strings with information about what's going on and I need to log this information on the screen row-by-row as the strings arrive (one row per port_DataReceived event). I would also like to get a timestamp when each string arrives and log the timestamp in the leftmost part of the row and the time elapsed since previous row just to the right of the timestamp. The following requirements apply: a) The GUI must have a checkbox with the text "Auto-scroll". When unchecked, I should be able to scroll among the already written rows. b) The GUI must have a "Clear Log Window" button, which clears the log the window when pressed, even if data logging is ongoing (of course, the very next microsecond after the button is pressed, new data has arrived and will be put in the first row again). c) The bulk of the code must be written in C# (with no WPF). However, simple event handlers, such as button event and checkbox-changed handlers, with very little code inside them are ok in other langugages as well. d) Must show the data in columns that can easily be read, either in a table or something supporting equal width characters (e.g. Courier New font). e) If it's not Excel, then it should easy to copy-and-paste into Excel for further in-depth analysis. f) The timestamp should be as accurate as possible, i.e. I want to exit the interrupt routines asap in order to not block interrupts from other Com port. g) Must be able to show text in different colors. h) Must be able to show more than 65535 characters. My questions are: A. What queuing technique should I use to ensure that I stay as little times as possible in the port_DataReceived(object sender, SerialDataReceivedEventArgs e) event handler? Can I use a simple backgroundWorker (Threading in C# - Part 3 - Using Threads[^], basically queuing addRowToLogWindow-tasks) or is there a faster/better approach? B. What GUI component should I use to display the data in? I've considered the following: 1. Directly in the Visual Studio Output window: I don't know how to implement the "Auto-scroll" checkbox and "Clear Log Window" button. Also, there seems to come other unwanted data in this window as well (exceptions, threads are te
A Serialport as a part of the UI - so Threading isn't a good idea. But you can also work with the DataReceived-Event from the/each port. Why do you want to use Threading or a Backgroundworker for this approach ? Do you have considered that there are other Control than a Textbox ? For example a ListView ... But you should realize that each and every list has an End ... so an endless Form isn't really possible to realize. If you want to write to an Excel-Sheet it isn't necessary to know something about VBA - this work could be done complete inside your Application. What problem do you have with a DataGridView ? But I think it isn't the right approach (! glass-bowl ;-)) RichTextBox could also be a good idea ...
-
A Serialport as a part of the UI - so Threading isn't a good idea. But you can also work with the DataReceived-Event from the/each port. Why do you want to use Threading or a Backgroundworker for this approach ? Do you have considered that there are other Control than a Textbox ? For example a ListView ... But you should realize that each and every list has an End ... so an endless Form isn't really possible to realize. If you want to write to an Excel-Sheet it isn't necessary to know something about VBA - this work could be done complete inside your Application. What problem do you have with a DataGridView ? But I think it isn't the right approach (! glass-bowl ;-)) RichTextBox could also be a good idea ...
SerialPort isn't a UI control: it can be dropped on a form if you must, but it doesn't have any display component (that's why it appears at the bottom, off the form like a Timer control does). You can - and should - access it from a thread other than the UI: indeed the DataReceived event is never fired on the UI thread anyway!
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!
-
1. Since your application will not be running in Visual Studio, that is not an option. 2. Console applications do not lend themselves well to this type of problem. 3. Does not satisfy your requirements. 4. Same as 3. 5. More difficult than .NET's built in controls. 6. The best choice, but it does need to be managed properly. DatagridViews become slow if you overload them with too much data; you need to implement efficient paging. 7. Not the best option, as it is used for pure text, rather than tabulated data that you are looking for.
-
Richard MacCutchan wrote:
7. Not the best option, as it is used for pure text, rather than tabulated data that you are looking for.
No, but it can be easily solved by setting the font to Courier New and inserting tabs.
-
A Serialport as a part of the UI - so Threading isn't a good idea. But you can also work with the DataReceived-Event from the/each port. Why do you want to use Threading or a Backgroundworker for this approach ? Do you have considered that there are other Control than a Textbox ? For example a ListView ... But you should realize that each and every list has an End ... so an endless Form isn't really possible to realize. If you want to write to an Excel-Sheet it isn't necessary to know something about VBA - this work could be done complete inside your Application. What problem do you have with a DataGridView ? But I think it isn't the right approach (! glass-bowl ;-)) RichTextBox could also be a good idea ...
Ralf Meier wrote:
Why do you want to use Threading or a Backgroundworker for this approach ?
Let's assume updating the GUI component with an extra row takes 10 ms, but queueing a task takes 1 ms. Then, if I don't queue the task, the event handler will be blocked for 10 ms for each packet, but if I queue a task instead then my event handler will only block for 1 ms. Of course, with the latter approach my data will show up with a slight delay in the GUI component, but that's acceptable.
Ralf Meier wrote:
For example a ListView
I've never used a ListView. Would it be a better choice than RichTextBox and DataGridView, in your opinion?
Ralf Meier wrote:
But you should realize that each and every list has an End
That's fine, I don't have infinite RAM in my computer anyways, but for sure 65535 characters is too little.
Ralf Meier wrote:
If you want to write to an Excel-Sheet it isn't necessary to know something about VBA - this work could be done complete inside your Application.
But how do I know which row I should add the latest data to, without running a for-loop to find the first empty row? RichTextBox and DataGridBox have append methods that will figure out where the end is for you.
Ralf Meier wrote:
What problem do you have with a DataGridView ?
Today I have a logging window (that supports a single Com port) implemented as DataGridview and it seems to be lagging a lot, but I guess I must be doing something wrong. Also, I had to implement my own copy-to-clipboard function and I had some problems with it because every once in a while (e.g. during boot-up and power down) there would come strange characters from my device and if a null-character was present in the middle, then only half the text would appear in the clipboard, so I had to spend time making a workaround for that. With e.g. RichTextBox, I wouldn't have to do this.
-
First off, don't assume that a DataReceived event will contain a full message: COM ports aren't packet based - they are character based, and a DataReceived event will fire after each character arrives if necessary. What you need to do is read each COM port continuously in a separate thread and pass complete messages back to your UI thread for processing / logging / display. The simplest way to handle that would be to use a BackgroundWorker as they have two advantages in this scenario: 1) They provide an easy "status update" mechanism which allows you to ignore Invoke requirements and just pass a whole message back to your UI for display. In a Console app, that means that one task - teh original is responsible for queueing messages for the display, which is good - multiple tasks woudl mean that messages could easily get mixed together instead of separated by lines or similar. 2) They are automatically terminated when the app is: this is not the case with all thread types. This means your in-thread processing can be a lot cleaner as it doesn't have to check for termination at all. Auto scroll in a console - that's slightly complicated because it's all manual stuff: you'd need to buffer stuff up for display until the "autoscroll hold" is released, and then output the buffer in one long splurt. For a textbox, it's a similar procedure - add items with TextBox.AppendText and it will autoscroll to the end. Clearing the console is simple: Console.Clear Method (System) | Microsoft Docs[^] A DGV can be set to use any font you like, just like any other control. I wouldn;t faff with embedding Excel unless you really, really have to! DGV's aren't slow unless you mistreat them: any control with a lot of data will be sluggish but this may help: Walkthrough: Implement virtual mode in DataGridView control - Windows Forms .NET Framework | Microsoft Docs[^]
"I have no idea what I
OriginalGriff wrote:
First off, don't assume that a DataReceived event will contain a full message: COM ports aren't packet based - they are character based, and a DataReceived event will fire after each character arrives if necessary.
I have a Com port logging window (that supports a single Com port) today implemented as a DataGridView and it works reasonably well, it's not the end of the world if a logging message appears in 2 halves in 2 separate rows. What I do think is important is that the time stamps are accurate, that's why I don't want to stay in the event handler too long (the time stamp will be stored first thing in the port_DataReceived event handler). Is it possible make the application to be inside 2 (or more) event handlers (COM1port_DataReceived, COM2port_DataReceived, COM3port_DataReceived, etc) at the same time or will they always be called after each other even if the events happen at the same time?
OriginalGriff wrote:
add items with TextBox.AppendText and it will autoscroll to the end.
Is the same true for RichTextBox? In that case I guess I can delete RichTextBox off the list.
-
OriginalGriff wrote:
First off, don't assume that a DataReceived event will contain a full message: COM ports aren't packet based - they are character based, and a DataReceived event will fire after each character arrives if necessary.
I have a Com port logging window (that supports a single Com port) today implemented as a DataGridView and it works reasonably well, it's not the end of the world if a logging message appears in 2 halves in 2 separate rows. What I do think is important is that the time stamps are accurate, that's why I don't want to stay in the event handler too long (the time stamp will be stored first thing in the port_DataReceived event handler). Is it possible make the application to be inside 2 (or more) event handlers (COM1port_DataReceived, COM2port_DataReceived, COM3port_DataReceived, etc) at the same time or will they always be called after each other even if the events happen at the same time?
OriginalGriff wrote:
add items with TextBox.AppendText and it will autoscroll to the end.
Is the same true for RichTextBox? In that case I guess I can delete RichTextBox off the list.
That's why I wouldn't use the event at all, but put each SerialPort in into own thread, continually monitoring (or perhaps sleeping / checking depending on how accurate you need your timestamps). That way, your system uses as many cores as it has to receive the data, and you can queue the results into a (lower priority if necessary) UI thread to handle display. The timestamps are a message function, so you assemble that as a package with the whole message, and pass that on. BackgroundWorker is great for that as it includes the UserState object as part of the Progress reporting mechanism which is routed direct to the UI thread. This ensures independence of your serial ports, and if you have enough cores very low latency for the message / timestamp itself.
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!
-
That's why I wouldn't use the event at all, but put each SerialPort in into own thread, continually monitoring (or perhaps sleeping / checking depending on how accurate you need your timestamps). That way, your system uses as many cores as it has to receive the data, and you can queue the results into a (lower priority if necessary) UI thread to handle display. The timestamps are a message function, so you assemble that as a package with the whole message, and pass that on. BackgroundWorker is great for that as it includes the UserState object as part of the Progress reporting mechanism which is routed direct to the UI thread. This ensures independence of your serial ports, and if you have enough cores very low latency for the message / timestamp itself.
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!
I'm happy with the solution of using event driven processing, this is a quick-and-dirty piece of software I will need asap (or preferably, yesterday) so I think it's overkill to have separate threads polling SerialPort.BytesToRead. But are you saying that if I create the SerialPort objects in separate threads, then it will be possible for the PC to execute the following events handlers simultaneously: COM1port_DataReceived, COM2port_DataReceived, COM3port_DataReceived, etc?
-
I'm happy with the solution of using event driven processing, this is a quick-and-dirty piece of software I will need asap (or preferably, yesterday) so I think it's overkill to have separate threads polling SerialPort.BytesToRead. But are you saying that if I create the SerialPort objects in separate threads, then it will be possible for the PC to execute the following events handlers simultaneously: COM1port_DataReceived, COM2port_DataReceived, COM3port_DataReceived, etc?
I have no idea - I've never tried, and I'm really not sure how I would test it ... :laugh: I would doubt it: if all the DataReceived events are set up by the same thread, then the code to kick the actual event off will all be executing on that thread (though the handler itself will be executed on a new thread / pool thread) so there will be some delay introduced there. I also suspect that the actual serial port char received interrupt goes through many layers of software before it even gets to the SerialPort class and then be routed to the right thread to kick of the Event. When I need multiple sources, I prefer to use separate threads and poll instead of use Events as it's rather more "immediate".
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!
-
I have no idea - I've never tried, and I'm really not sure how I would test it ... :laugh: I would doubt it: if all the DataReceived events are set up by the same thread, then the code to kick the actual event off will all be executing on that thread (though the handler itself will be executed on a new thread / pool thread) so there will be some delay introduced there. I also suspect that the actual serial port char received interrupt goes through many layers of software before it even gets to the SerialPort class and then be routed to the right thread to kick of the Event. When I need multiple sources, I prefer to use separate threads and poll instead of use Events as it's rather more "immediate".
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!
-
OriginalGriff wrote:
I would doubt it: if all the DataReceived events are set up by the same thread
No, I wouldn't do that, I would create my SerialPort objects and hook up the DataReceived events in separate threads.
That's just adding a further layer of abstraction - the DataReceived event is fired on a pool thread, not the original starter thread, so ... :-D
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!
-
That's just adding a further layer of abstraction - the DataReceived event is fired on a pool thread, not the original starter thread, so ... :-D
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!
-
That's just adding a further layer of abstraction - the DataReceived event is fired on a pool thread, not the original starter thread, so ... :-D
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!
-
So you're saying it's not a good idea? How many pool threads does a PC have? Can I somehow dictate what pool thread the different Com ports should issue events onto?
arnold_w wrote:
How many pool threads does a PC have?
Depends: Thread Pools - Win32 apps | Microsoft Docs[^]
arnold_w wrote:
Can I somehow dictate what pool thread the different Com ports should issue events onto?
No. It's a pool. As an when a thread is needed, one is pulled from the pool and started.
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony "Common sense is so rare these days, it should be classified as a super power" - Random T-shirt AntiTwitter: @DalekDave is now a follower!
-
I need to peel potatoes but I don't want to use a potato peeler.
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it. ― Confucian Analects: Rules of Confucius about his food
You haven't got a swiss army knife?
Software rusts. Simon Stephenson, ca 1994. So does this signature. me, 2012
-
You haven't got a swiss army knife?
Software rusts. Simon Stephenson, ca 1994. So does this signature. me, 2012
-
I have a number of devices with Com ports (UART:s running at MBaud speeds) that I need to monitor and make sense of. The devices send ASCII-strings with information about what's going on and I need to log this information on the screen row-by-row as the strings arrive (one row per port_DataReceived event). I would also like to get a timestamp when each string arrives and log the timestamp in the leftmost part of the row and the time elapsed since previous row just to the right of the timestamp. The following requirements apply: a) The GUI must have a checkbox with the text "Auto-scroll". When unchecked, I should be able to scroll among the already written rows. b) The GUI must have a "Clear Log Window" button, which clears the log the window when pressed, even if data logging is ongoing (of course, the very next microsecond after the button is pressed, new data has arrived and will be put in the first row again). c) The bulk of the code must be written in C# (with no WPF). However, simple event handlers, such as button event and checkbox-changed handlers, with very little code inside them are ok in other langugages as well. d) Must show the data in columns that can easily be read, either in a table or something supporting equal width characters (e.g. Courier New font). e) If it's not Excel, then it should easy to copy-and-paste into Excel for further in-depth analysis. f) The timestamp should be as accurate as possible, i.e. I want to exit the interrupt routines asap in order to not block interrupts from other Com port. g) Must be able to show text in different colors. h) Must be able to show more than 65535 characters. My questions are: A. What queuing technique should I use to ensure that I stay as little times as possible in the port_DataReceived(object sender, SerialDataReceivedEventArgs e) event handler? Can I use a simple backgroundWorker (Threading in C# - Part 3 - Using Threads[^], basically queuing addRowToLogWindow-tasks) or is there a faster/better approach? B. What GUI component should I use to display the data in? I've considered the following: 1. Directly in the Visual Studio Output window: I don't know how to implement the "Auto-scroll" checkbox and "Clear Log Window" button. Also, there seems to come other unwanted data in this window as well (exceptions, threads are te
Hi, here are my suggestions: 1. use a single BackGroundWorker per COM port; use its progress reporting facility to report displayable results. 2. Use synchronous read operations, thus avoiding the tricky DataReceived event. If you are lucky and the peripheral reports text messages ending on some specific terminator, use ReadLine(); when necessary, adjust the NewLine property to equal that terminator. If not, just use Read and have the BGW discover the beginning and end of every message. 3. For the UI use the simplest List oriented Control that satisfies your interactive requirements; a ListView is good, however I prefer a ListBox. 4. Define a little class holding the raw data of one displayable result. That is one object, no more expensive than the array a ListView would need. Instances of this class, not strings, are to be added to the ListBox Items collection. 5. Make the ListBox ownerdrawn, giving it a fixed ItemHeight, and an appropriate DrawItem handler that takes care of horizontal positioning, coloring, and whatever fancy styling you may want. That's it; and it is bound to be simpler than the code you probably are experementing with right now... :)
Luc Pattyn [My Articles] If you can't find it on YouTube try TikTok...
