Ever have one of those days where you wonder why you even code?
-
I've been working like mad to get my Prang MIDI device working, and it does. But it lags. I can't stop the lag. It doesn't seem to matter what I do to it. If I run a loop between TX/RX on the breakout the lag is basically gone (more or less - it's still MIDI) As soon as I include my circuit, running my firmware in the loop the lag goes crazy. Even this yields lag:
// pass through RX-->TX
while(true) {
int i;
if(-1!=(i=MidiSerial.read())) {
MidiSerial.write(i);
}
}I think it might be the hardware UART buffers taking too long at 31250 baud. But at the end of the day it doesn't matter, because I've no way to fix it. My project is dead in the water. After already writing an article about it prior, and after writing an entire MIDI library that basically is effectively useless now since you can't get anything on or off the device without lag. *headdesk* I'm completely bummed out right now. I need a win.
To err is human. Fortune favors the monsters.
-
I've been working like mad to get my Prang MIDI device working, and it does. But it lags. I can't stop the lag. It doesn't seem to matter what I do to it. If I run a loop between TX/RX on the breakout the lag is basically gone (more or less - it's still MIDI) As soon as I include my circuit, running my firmware in the loop the lag goes crazy. Even this yields lag:
// pass through RX-->TX
while(true) {
int i;
if(-1!=(i=MidiSerial.read())) {
MidiSerial.write(i);
}
}I think it might be the hardware UART buffers taking too long at 31250 baud. But at the end of the day it doesn't matter, because I've no way to fix it. My project is dead in the water. After already writing an article about it prior, and after writing an entire MIDI library that basically is effectively useless now since you can't get anything on or off the device without lag. *headdesk* I'm completely bummed out right now. I need a win.
To err is human. Fortune favors the monsters.
Is the lag constant? This almost sounds like something in the hardware/software can't run TX and RX at the same time. We had that problem with a buggy serial chipset (I2C I think) once that used a single FIFO for both TX and RX. You could send bursts, and receive bursts, but you couldn't do both at the same time.
Software Zen:
delete this; -
Is the lag constant? This almost sounds like something in the hardware/software can't run TX and RX at the same time. We had that problem with a buggy serial chipset (I2C I think) once that used a single FIFO for both TX and RX. You could send bursts, and receive bursts, but you couldn't do both at the same time.
Software Zen:
delete this;It is constant and I've already run that down to the degree that I'm confident it's full duplex, but I'm thinking the hardware UART buffering has something to do with it. However, even calling flush all the time doesn't solve the issue. I may have to move the whole thing to USB. That was my original plan anyway, but I ran into technical challenges. Well, I'm throwing some different hardware at it this time and maybe that will change.
To err is human. Fortune favors the monsters.
-
I've been working like mad to get my Prang MIDI device working, and it does. But it lags. I can't stop the lag. It doesn't seem to matter what I do to it. If I run a loop between TX/RX on the breakout the lag is basically gone (more or less - it's still MIDI) As soon as I include my circuit, running my firmware in the loop the lag goes crazy. Even this yields lag:
// pass through RX-->TX
while(true) {
int i;
if(-1!=(i=MidiSerial.read())) {
MidiSerial.write(i);
}
}I think it might be the hardware UART buffers taking too long at 31250 baud. But at the end of the day it doesn't matter, because I've no way to fix it. My project is dead in the water. After already writing an article about it prior, and after writing an entire MIDI library that basically is effectively useless now since you can't get anything on or off the device without lag. *headdesk* I'm completely bummed out right now. I need a win.
To err is human. Fortune favors the monsters.
Don't worry, future codewitch will come to the rescue soon enough, I would wager! :)
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
-
I've been working like mad to get my Prang MIDI device working, and it does. But it lags. I can't stop the lag. It doesn't seem to matter what I do to it. If I run a loop between TX/RX on the breakout the lag is basically gone (more or less - it's still MIDI) As soon as I include my circuit, running my firmware in the loop the lag goes crazy. Even this yields lag:
// pass through RX-->TX
while(true) {
int i;
if(-1!=(i=MidiSerial.read())) {
MidiSerial.write(i);
}
}I think it might be the hardware UART buffers taking too long at 31250 baud. But at the end of the day it doesn't matter, because I've no way to fix it. My project is dead in the water. After already writing an article about it prior, and after writing an entire MIDI library that basically is effectively useless now since you can't get anything on or off the device without lag. *headdesk* I'm completely bummed out right now. I need a win.
To err is human. Fortune favors the monsters.
You'll fix it. :thumbsup:
-
I've been working like mad to get my Prang MIDI device working, and it does. But it lags. I can't stop the lag. It doesn't seem to matter what I do to it. If I run a loop between TX/RX on the breakout the lag is basically gone (more or less - it's still MIDI) As soon as I include my circuit, running my firmware in the loop the lag goes crazy. Even this yields lag:
// pass through RX-->TX
while(true) {
int i;
if(-1!=(i=MidiSerial.read())) {
MidiSerial.write(i);
}
}I think it might be the hardware UART buffers taking too long at 31250 baud. But at the end of the day it doesn't matter, because I've no way to fix it. My project is dead in the water. After already writing an article about it prior, and after writing an entire MIDI library that basically is effectively useless now since you can't get anything on or off the device without lag. *headdesk* I'm completely bummed out right now. I need a win.
To err is human. Fortune favors the monsters.
while(true) { int i; if(-1!=(i=MidiSerial.read())) { MidiSerial.write(i); } } Did you author these routines? if so, you might want to look at them in detail so be sure handshaking is not the source of lag. Something may be waiting on something else.
"A little time, a little trouble, your better day" Badfinger
-
while(true) { int i; if(-1!=(i=MidiSerial.read())) { MidiSerial.write(i); } } Did you author these routines? if so, you might want to look at them in detail so be sure handshaking is not the source of lag. Something may be waiting on something else.
"A little time, a little trouble, your better day" Badfinger
What handshaking? There are two lines. TX and RX There are no control lines. There is nothing to handshake. It's two wires.
To err is human. Fortune favors the monsters.
-
I've been working like mad to get my Prang MIDI device working, and it does. But it lags. I can't stop the lag. It doesn't seem to matter what I do to it. If I run a loop between TX/RX on the breakout the lag is basically gone (more or less - it's still MIDI) As soon as I include my circuit, running my firmware in the loop the lag goes crazy. Even this yields lag:
// pass through RX-->TX
while(true) {
int i;
if(-1!=(i=MidiSerial.read())) {
MidiSerial.write(i);
}
}I think it might be the hardware UART buffers taking too long at 31250 baud. But at the end of the day it doesn't matter, because I've no way to fix it. My project is dead in the water. After already writing an article about it prior, and after writing an entire MIDI library that basically is effectively useless now since you can't get anything on or off the device without lag. *headdesk* I'm completely bummed out right now. I need a win.
To err is human. Fortune favors the monsters.
Are these blocking reads/writes? Even if they are not I might be tempted to shunt the write on to another thread and have a send queue. Good luck - it sounds like you are within sight of the finish line here.
-
Are these blocking reads/writes? Even if they are not I might be tempted to shunt the write on to another thread and have a send queue. Good luck - it sounds like you are within sight of the finish line here.
They don't block. This is a hardware UART with internal send and receive buffers. I think that is my problem (maybe?). The issue with scheduling on this RTOS is it's too simplistic and easy to starve. The scheduler isn't bright. It's not CPU idling on I/O that is the problem, which punting to another thread would solve. It's simply latency between the time the data gets received and it gets sent. My CPU is running like a greased pig. Edit: I've got some hardware coming today which will hopefully allow me to switch all the MIDI stuff to USB. That may solve my problem by sidestepping the hardware UARTs altogether. My other option is to just bit bang the serial myself, but I don't like that.
To err is human. Fortune favors the monsters.
-
I've been working like mad to get my Prang MIDI device working, and it does. But it lags. I can't stop the lag. It doesn't seem to matter what I do to it. If I run a loop between TX/RX on the breakout the lag is basically gone (more or less - it's still MIDI) As soon as I include my circuit, running my firmware in the loop the lag goes crazy. Even this yields lag:
// pass through RX-->TX
while(true) {
int i;
if(-1!=(i=MidiSerial.read())) {
MidiSerial.write(i);
}
}I think it might be the hardware UART buffers taking too long at 31250 baud. But at the end of the day it doesn't matter, because I've no way to fix it. My project is dead in the water. After already writing an article about it prior, and after writing an entire MIDI library that basically is effectively useless now since you can't get anything on or off the device without lag. *headdesk* I'm completely bummed out right now. I need a win.
To err is human. Fortune favors the monsters.
I remember the UART buffering / lag problem! Sometimes these chips are too smart for their own good. Back in the day, there was a bit one could set somehow that would disable the buffering at least on some UART's, we had to actually order specific manufacturer UART's - worked like a charm, except then I had to tie in the interrupt so not to overwrite the byte being sent, and that was flaky because, well, it was an interrupt, and changing interrupt priorities didn't really fix the problem. Since we were sending small packets of data, I ended up doing a "write byte", "check for TX complete loop", "write the next byte". But that was for writing. For reading, I didn't much care if it was laggy.
Latest Article:
Create a Digital Ocean Droplet for .NET Core Web API with a real SSL Certificate on a Domain -
What handshaking? There are two lines. TX and RX There are no control lines. There is nothing to handshake. It's two wires.
To err is human. Fortune favors the monsters.
-
It is constant and I've already run that down to the degree that I'm confident it's full duplex, but I'm thinking the hardware UART buffering has something to do with it. However, even calling flush all the time doesn't solve the issue. I may have to move the whole thing to USB. That was my original plan anyway, but I ran into technical challenges. Well, I'm throwing some different hardware at it this time and maybe that will change.
To err is human. Fortune favors the monsters.
If the UART has a FIFO you may need to set the level down to 1 so you get an interrupt for each character received. Otherwise the UART will not interrupt until the FIFO is full or some time has passed.
-
If the UART has a FIFO you may need to set the level down to 1 so you get an interrupt for each character received. Otherwise the UART will not interrupt until the FIFO is full or some time has passed.
Yep. The problem is I can only set the buffer so small. Fortunately I set to its smallest setting and it improved things a lot. I didn't think it would, because 128 was the minimum and I thought it was bytes but it may be bits, which is much better for me.
To err is human. Fortune favors the monsters.
-
looking at all eight bits? any Control characters?
"A little time, a little trouble, your better day" Badfinger
Nah, it's just midi protocol over the wire. The receive buffer was too big by default. I turned it down and it improved things a lot. Unfortunately I can't eliminate it so I may have to switch to USB anyway.
To err is human. Fortune favors the monsters.
-
Yep. The problem is I can only set the buffer so small. Fortunately I set to its smallest setting and it improved things a lot. I didn't think it would, because 128 was the minimum and I thought it was bytes but it may be bits, which is much better for me.
To err is human. Fortune favors the monsters.
At 128 bytes, that is probably the operating system buffer for the UART. Inside some UART there is a FIFO buffer, usually 8 or 16 bytes. There is a threshold level you can set as to when the UART will interrupt the OS to read the buffer. And there is sometimes a way to set the timeout before the buffer is full. This would be set at the hardware level, which you may not have access to.
-
I've been working like mad to get my Prang MIDI device working, and it does. But it lags. I can't stop the lag. It doesn't seem to matter what I do to it. If I run a loop between TX/RX on the breakout the lag is basically gone (more or less - it's still MIDI) As soon as I include my circuit, running my firmware in the loop the lag goes crazy. Even this yields lag:
// pass through RX-->TX
while(true) {
int i;
if(-1!=(i=MidiSerial.read())) {
MidiSerial.write(i);
}
}I think it might be the hardware UART buffers taking too long at 31250 baud. But at the end of the day it doesn't matter, because I've no way to fix it. My project is dead in the water. After already writing an article about it prior, and after writing an entire MIDI library that basically is effectively useless now since you can't get anything on or off the device without lag. *headdesk* I'm completely bummed out right now. I need a win.
To err is human. Fortune favors the monsters.
My first optimization would be putting "int i" outside the loop. But since I don't know C++, it maybe makes no difference.
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
-
At 128 bytes, that is probably the operating system buffer for the UART. Inside some UART there is a FIFO buffer, usually 8 or 16 bytes. There is a threshold level you can set as to when the UART will interrupt the OS to read the buffer. And there is sometimes a way to set the timeout before the buffer is full. This would be set at the hardware level, which you may not have access to.
Actually that setting does exist, and you can set it if you use the native API. Unfortunately, doing so requires some fancy footwork you can't do under the Arduino framework, and that's what I'm targeting. Fortunately, I turned the buffer sizes down and that managed to nearly solve the problem - much better than I thought it would.
To err is human. Fortune favors the monsters.
-
My first optimization would be putting "int i" outside the loop. But since I don't know C++, it maybe makes no difference.
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
In this case it makes no difference to the compiler. It just basically gives the compiler a stack size and an offset, and it would have those no matter where it is in the code. "int i;" doesn't create any executable instructions.
To err is human. Fortune favors the monsters.
-
Nah, it's just midi protocol over the wire. The receive buffer was too big by default. I turned it down and it improved things a lot. Unfortunately I can't eliminate it so I may have to switch to USB anyway.
To err is human. Fortune favors the monsters.