Strolling off the edge
-
My last three major problems in my code involved buffer overruns. These are fun on the ESP32 or other platforms without memory protection because you don't crash when the Bad Thing(TM) happens. You crash in the aftermath as a consequence, by for example, trying to execute an illegal instruction because your data overwrote your code. Because of that, even *if* your stack dump isn't corrupt you still won't get much in the way of a hint as to where the actual error is. The bottom line is you get to debug it blind. I solved the last three times this happened, and it's making me a better troubleshooter. Damn it's difficult though.
To err is human. Fortune favors the monsters.
Age old problem. The best one can do is lot of bounds checking in one's code. As my old college CS. prof used to say about people coming to him with programming issues: "You are off by 1 somewhere."
"A little time, a little trouble, your better day" Badfinger
-
Age old problem. The best one can do is lot of bounds checking in one's code. As my old college CS. prof used to say about people coming to him with programming issues: "You are off by 1 somewhere."
"A little time, a little trouble, your better day" Badfinger
It wasn't a fencepost error/off-by-one error. It was something far more destructive. I ended up writing out twice the memory I intended to.
To err is human. Fortune favors the monsters.
-
It wasn't a fencepost error/off-by-one error. It was something far more destructive. I ended up writing out twice the memory I intended to.
To err is human. Fortune favors the monsters.
Hey, if you're going overrun the buffer at least make it worth your while! Double your fun, that's what I always say!
-
Hey, if you're going overrun the buffer at least make it worth your while! Double your fun, that's what I always say!
Specifically
static void wav_voice_16_1_to_16_2(const voice_func_info_t& info, void*state) {
wav_info_t* wi = (wav_info_t*)state;
if(!wi->loop&&wi->pos>=wi->length) {
return;
}
uint16_t* dst = (uint16_t*)info.buffer;
for(int i = 0;ipos>=wi->length) {
if(!wi->loop) {
break;
}
wi->on_seek_stream(wi->start,wi->on_seek_stream_state);
wi->pos = 0;
}
if(player_read16s(wi->on_read_stream,wi->on_read_stream_state,&i16)) {
wi->pos+=2;
} else {
break;
}
uint16_t u16 = (uint16_t)((i16+32768U)*wi->amplitude);
for(int j=0;j
To err is human. Fortune favors the monsters. -
Richard MacCutchan wrote:
Reminds me of the good old days
I was thinking it reminded me of the little less old days (1992) when I was learning C/C++ pointers on a machine running DOS 5.x and Win 3.1 It was the same then. The machine or app didn't immediately crash or anything. Just kept going until that memory was used by something else. A lot of the crashing now is caused because of DEP (Data Execution Prevention[^]) They're false crashes. The program probably would've kept on going. :laugh: Leave my code alone, Windows! :rolleyes:
I started to pick up C++ with Borland's compiler in the DOS days but it didn't crystalize for me. I actually picked it up out of necessity while working in the field. That was a trial by fire, and not without its missteps, like forgetting to mark my destructors for my COM objects virtual. :~ But man, now that I have GCC and Clang, I feel so much freer than I do with MS compilers. It use to be that their compilers were subpar, which is why I went to GCC, but as they improved, I've gotten so used to GCC's (and largely Clang's) flavor of standards compliance that the MSVC one throws me. I can write code in MSVC++ until it starts getting template heavy. Then I'm always having to massage the syntax of the code I write to get it to fit MSVC's way of doing things. The issue is coding windows stuff with it. You actually have to patch the direct x headers because of a nuanced difference in how GCC returns certain types from functions.
To err is human. Fortune favors the monsters.
-
Specifically
static void wav_voice_16_1_to_16_2(const voice_func_info_t& info, void*state) {
wav_info_t* wi = (wav_info_t*)state;
if(!wi->loop&&wi->pos>=wi->length) {
return;
}
uint16_t* dst = (uint16_t*)info.buffer;
for(int i = 0;ipos>=wi->length) {
if(!wi->loop) {
break;
}
wi->on_seek_stream(wi->start,wi->on_seek_stream_state);
wi->pos = 0;
}
if(player_read16s(wi->on_read_stream,wi->on_read_stream_state,&i16)) {
wi->pos+=2;
} else {
break;
}
uint16_t u16 = (uint16_t)((i16+32768U)*wi->amplitude);
for(int j=0;j
To err is human. Fortune favors the monsters.This reminds me of some programming I did once upon a time in C++ Borland. Did not like it one bit. Literally. :laugh:
-
honey the codewitch wrote:
other platforms without memory protection
In current OSes (desktop) and with current C and/or C++ compilers it is not possible to corrupt memory by creating code incorrectly? Far as I know neither ANSI C nor ANSI C++ mandates how memory is managed. So it would be a value add if a C/C++ compiler added something in that fully protected memory. Seems like it would require specific language code usage by the programmer as well.
I totally misread you when a responded before. Coming back to read the lounge and realized you weren't saying what I thought you were saying. Sorry. Yes, that would be cool. But honestly, I don't run into the problem enough that I'd want the necessary runtime checks that would come with preventing it. Unless maybe it only did it on debug builds. I think there are tools like this out there, but I'd have to dig.
To err is human. Fortune favors the monsters.
-
It wasn't a fencepost error/off-by-one error. It was something far more destructive. I ended up writing out twice the memory I intended to.
To err is human. Fortune favors the monsters.
"Off by one" example is not meant to be necessarily be literal. It means that somewhere a boundary check is not occurring or is not detected, a memory operation is corrupted, lots of things that may lead to a single point failure. Buffer overflows were and maybe still are reasons many virus attacks because they wander into uncontrolled parts of memory. Recall the infamous P = malloc( 0 ); not returning a null pointer. I know your system is quite complex, hence your explorations into the complex interactions in world of hardware and software at the lower levels, so my feedback may be naive. Keep at it.
"A little time, a little trouble, your better day" Badfinger
-
"Off by one" example is not meant to be necessarily be literal. It means that somewhere a boundary check is not occurring or is not detected, a memory operation is corrupted, lots of things that may lead to a single point failure. Buffer overflows were and maybe still are reasons many virus attacks because they wander into uncontrolled parts of memory. Recall the infamous P = malloc( 0 ); not returning a null pointer. I know your system is quite complex, hence your explorations into the complex interactions in world of hardware and software at the lower levels, so my feedback may be naive. Keep at it.
"A little time, a little trouble, your better day" Badfinger
jmaida wrote:
Buffer overflows were and maybe still are reasons many virus attacks
Yeah, I often joke when this happens that I wind up attacking my own code with an exploit. :-D
To err is human. Fortune favors the monsters.
-
jmaida wrote:
Buffer overflows were and maybe still are reasons many virus attacks
Yeah, I often joke when this happens that I wind up attacking my own code with an exploit. :-D
To err is human. Fortune favors the monsters.
-
Age old problem. The best one can do is lot of bounds checking in one's code. As my old college CS. prof used to say about people coming to him with programming issues: "You are off by 1 somewhere."
"A little time, a little trouble, your better day" Badfinger
jmaida wrote:
The best one can do is lot of bounds checking in one's code.
In embedded you really have to make each line count. When you have a handful of microseconds to compute the nes state of the transistors that are piloting a motor, your clock is 40 Mhz if you're lucky and you have 3kB of RAM and 64kB of flash bounds checking in the code is really on a if-needed basis. You just check the memory window while debugging step by step and infer from that.
GCS/GE d--(d) s-/+ a C+++ U+++ P-- L+@ E-- W+++ N+ o+ K- w+++ O? M-- V? PS+ PE Y+ PGP t+ 5? X R+++ tv-- b+(+++) DI+++ D++ G e++ h--- r+++ y+++* Weapons extension: ma- k++ F+2 X
-
My last three major problems in my code involved buffer overruns. These are fun on the ESP32 or other platforms without memory protection because you don't crash when the Bad Thing(TM) happens. You crash in the aftermath as a consequence, by for example, trying to execute an illegal instruction because your data overwrote your code. Because of that, even *if* your stack dump isn't corrupt you still won't get much in the way of a hint as to where the actual error is. The bottom line is you get to debug it blind. I solved the last three times this happened, and it's making me a better troubleshooter. Damn it's difficult though.
To err is human. Fortune favors the monsters.
buffer over/under write/read is probably the only bug i have never created as i have always been painstakingly careful about such . in fact at a job many years ago i wrote a quick little memory protection scheme . it quickly found the source of memory errors though the project failed miserably . as for your situation would it make sense to run your code on a desktop/laptop in a kind of simulator of your devices w/ memory protection code so as to find any such errors ? - Best
-
My last three major problems in my code involved buffer overruns. These are fun on the ESP32 or other platforms without memory protection because you don't crash when the Bad Thing(TM) happens. You crash in the aftermath as a consequence, by for example, trying to execute an illegal instruction because your data overwrote your code. Because of that, even *if* your stack dump isn't corrupt you still won't get much in the way of a hint as to where the actual error is. The bottom line is you get to debug it blind. I solved the last three times this happened, and it's making me a better troubleshooter. Damn it's difficult though.
To err is human. Fortune favors the monsters.
I feel your pain. Just located and fixed two buffer overruns yesterday where my code was writing I422 video to buffers sized for I420. :doh: It would run for hours, but just don't try to allocate/free anything! (... or try to make sense of the data that follows!)
-
I totally misread you when a responded before. Coming back to read the lounge and realized you weren't saying what I thought you were saying. Sorry. Yes, that would be cool. But honestly, I don't run into the problem enough that I'd want the necessary runtime checks that would come with preventing it. Unless maybe it only did it on debug builds. I think there are tools like this out there, but I'd have to dig.
To err is human. Fortune favors the monsters.
Ok. I believe data versus code protected status in OSes desktops have been around for at least 20 years so I didn't even think about that. And libraries to debug other types of memory problems, for example a C/C++ app that corrupts the stack by overwriting a boundary, have been around probably since the 80s since I remember using one in the early 90s. They still exist but with complexities and size of modern desktop apps using them effectively has gotten much harder.
-
jmaida wrote:
The best one can do is lot of bounds checking in one's code.
In embedded you really have to make each line count. When you have a handful of microseconds to compute the nes state of the transistors that are piloting a motor, your clock is 40 Mhz if you're lucky and you have 3kB of RAM and 64kB of flash bounds checking in the code is really on a if-needed basis. You just check the memory window while debugging step by step and infer from that.
GCS/GE d--(d) s-/+ a C+++ U+++ P-- L+@ E-- W+++ N+ o+ K- w+++ O? M-- V? PS+ PE Y+ PGP t+ 5? X R+++ tv-- b+(+++) DI+++ D++ G e++ h--- r+++ y+++* Weapons extension: ma- k++ F+2 X
-
Ok. I believe data versus code protected status in OSes desktops have been around for at least 20 years so I didn't even think about that. And libraries to debug other types of memory problems, for example a C/C++ app that corrupts the stack by overwriting a boundary, have been around probably since the 80s since I remember using one in the early 90s. They still exist but with complexities and size of modern desktop apps using them effectively has gotten much harder.
I've found with IoT, those are helpful in some cases, but I don't like instrumenting my code extensively for that kind of thing, or substituting a custom heap on these platforms because I haven't found a way to do it without impacting flash size, runtime performance, and/or maintainability/readability. Every solution I've found carries with it significant drawbacks, with IoT firmware usually doing sparse allocations, and generally on initialization, plus there's no teardown because there's no "prompt" to drop back to so it's easy to structure your code to avoid leaks and overruns for the most part. I've ran into a couple in my most complicated codebases that weren't stupid errors, but most of the time it's me having an "oh duh" moment :doh: so while somewhat embarrassing, they're easy to catch if I stare at it sideways or hard enough. It's such that those tools don't really benefit me so much (I think? I haven't done any actual real world testing on it, it's just my hunch) I would use this stuff in a complicated desktop app, but honestly? I'd just as soon use something like Boehm's collector if I was even writing a desktop app in C++.
To err is human. Fortune favors the monsters.
-
I feel your pain. Just located and fixed two buffer overruns yesterday where my code was writing I422 video to buffers sized for I420. :doh: It would run for hours, but just don't try to allocate/free anything! (... or try to make sense of the data that follows!)
So much nicer when it crashes immediately!
To err is human. Fortune favors the monsters.
-
buffer over/under write/read is probably the only bug i have never created as i have always been painstakingly careful about such . in fact at a job many years ago i wrote a quick little memory protection scheme . it quickly found the source of memory errors though the project failed miserably . as for your situation would it make sense to run your code on a desktop/laptop in a kind of simulator of your devices w/ memory protection code so as to find any such errors ? - Best
as for your situation would it make sense to run your code on a desktop/laptop in a kind of simulator of your devices w/ memory protection code so as to find any such errors ?
In some cases - well many cases - yes, because I write my code to be cross platform wherever possible, but in this case, it wouldn't have made sense because it's I2S driver level stuff where there's no corollary on a PC. I'd have to write (and debug!) an emulator for it, so it just adds to the test matrix rather than solving anything. My errors were dumb ones, TBH. The kind that are somewhat embarrassing but easy to find and fix once you step away and come back to it. Pointer ops are nothing special to me. I take to them pretty readily and don't often make mistakes. When I do, they're usually kind of typo or memory lapse varieties.
To err is human. Fortune favors the monsters.
-
as for your situation would it make sense to run your code on a desktop/laptop in a kind of simulator of your devices w/ memory protection code so as to find any such errors ?
In some cases - well many cases - yes, because I write my code to be cross platform wherever possible, but in this case, it wouldn't have made sense because it's I2S driver level stuff where there's no corollary on a PC. I'd have to write (and debug!) an emulator for it, so it just adds to the test matrix rather than solving anything. My errors were dumb ones, TBH. The kind that are somewhat embarrassing but easy to find and fix once you step away and come back to it. Pointer ops are nothing special to me. I take to them pretty readily and don't often make mistakes. When I do, they're usually kind of typo or memory lapse varieties.
To err is human. Fortune favors the monsters.
regarding the cause of errors , many years ago at a small firm it occurred to me to investigate the ultimate presumably psychological cause of each of the many errors in our then project . i never began the investigation but it would not have succeeded anyway had i attempted as it would have required considerable cooperation from my 2 co-workers who each held me in contempt not to mention the 3 owners who disliked me as they were aware i believed each of them to be insane .
-
regarding the cause of errors , many years ago at a small firm it occurred to me to investigate the ultimate presumably psychological cause of each of the many errors in our then project . i never began the investigation but it would not have succeeded anyway had i attempted as it would have required considerable cooperation from my 2 co-workers who each held me in contempt not to mention the 3 owners who disliked me as they were aware i believed each of them to be insane .
Boy that sounds like a fun place to work. :~
"the debugger doesn't tell me anything because this code compiles just fine" - random QA comment "Facebook is where you tell lies to your friends. Twitter is where you tell the truth to strangers." - chriselst "I don't drink any more... then again, I don't drink any less." - Mike Mullikins uncle
