Designing a system test for error handling functions
-
I'm not even sure if I can ask this question properly (shows how unclear I am about it myself). So I developed a small part of a rather complex software. I am to run a system test on this software (the whole thing) to test its error handling procedures. But it's not going so smoothly. It's alright for those errors that I can actually cause (e.g. run the software without some essential hardware). But of course I can't generate errors such as "Hardware broken" or "Hardware is connected but not responding" as there is a chance of actually doing harm to the hardware. So I need a cheat. Putting break points in the software and overwriting values is not an option because this is a system test. So I duplicated a small part of my software and put some error testing mechanism in the second copy (let's call it the tester version). The software launches with one or the other of these versions depending on the registry value I set. The problem is that I've limited the duplicated portion of the code, which I thought and still think is a good idea, but this means that the dummy errors can only be emitted from one place. In reality there are many paths errors can take. Depending on the path, the final output to the user can be different. I want to map the various error paths to help decide how best to generate the dummy errors. It appears I need to do something like: "Generate error A as if function B caused it" and "Generate error A as if function C caused it" etc So I need a mapping from error A to functions B and C, and so on. Is there a clever way of doing this? Or maybe a better question would be, what is the best way to run a system test on error handling functions? Thanks for any input.
Almost, but not quite, entirely unlike... me...
-
I'm not even sure if I can ask this question properly (shows how unclear I am about it myself). So I developed a small part of a rather complex software. I am to run a system test on this software (the whole thing) to test its error handling procedures. But it's not going so smoothly. It's alright for those errors that I can actually cause (e.g. run the software without some essential hardware). But of course I can't generate errors such as "Hardware broken" or "Hardware is connected but not responding" as there is a chance of actually doing harm to the hardware. So I need a cheat. Putting break points in the software and overwriting values is not an option because this is a system test. So I duplicated a small part of my software and put some error testing mechanism in the second copy (let's call it the tester version). The software launches with one or the other of these versions depending on the registry value I set. The problem is that I've limited the duplicated portion of the code, which I thought and still think is a good idea, but this means that the dummy errors can only be emitted from one place. In reality there are many paths errors can take. Depending on the path, the final output to the user can be different. I want to map the various error paths to help decide how best to generate the dummy errors. It appears I need to do something like: "Generate error A as if function B caused it" and "Generate error A as if function C caused it" etc So I need a mapping from error A to functions B and C, and so on. Is there a clever way of doing this? Or maybe a better question would be, what is the best way to run a system test on error handling functions? Thanks for any input.
Almost, but not quite, entirely unlike... me...
There is no general purpose easy way to do everything. It is possible, but difficult, to use code insertion techniques to simulate any error. At best, and at the most complicated, you actually modify the code at run time to force an error. Some people suggest using interfaces. These are outside those required by the design but are put in place solely to support testing. The problem with that is that not everything is solved with that and it might require a lot of interfaces. One can also be creative about testing. For example if I need to test connectivity problems I can use a system call, in the test code, to drop my IP (of course I better restore it as well.) Or I can stop SQL Server pro grammatically when doing database error tests (again make sure to restart it.)