Introducing tests into a legacy application?
-
Our team has the task of maintaining and developing new features for a very old application. There is no budget (and no hope we can ever convince management) to rewrite it from scratch, so the only feasible way to keep this beast running is to gradually rewrite only parts of it, just as I have done before, occasionally. It isn't quite that easy though, because, as I learned the hard way, even when I thought I completely understood a part of the program, errors kept creeping up. Often much, much later than when I initially wrote the replacement, making it much harder for me to understand my original thought process, and why it didn't work. So I need tests, and the only valid method of testing under these circumstances that I can think of are Unit Tests. The problem is - I never worked with Unit Tests, I only know of them through articles that mention them, and therefore I only have a vague understanding of what they are and how they are supposed to be done. Now I need some guidance where to start: I did search for articles and information on the web, but unfortunately didn't find anything useful that doesn't already require a thorough understanding of Unit Tests to start with. Somehow there don't seem to be any introductory articles - or I'm too stupid to find them. If anyone knows a link, could you please share? As for books, I know there are literally thousands out there - but that is also my problem: which of them would be the right one? Could you advise some good picks to help out an old, experienced programmer, who for some reason never picked up the art of unit-testing?
-
Our team has the task of maintaining and developing new features for a very old application. There is no budget (and no hope we can ever convince management) to rewrite it from scratch, so the only feasible way to keep this beast running is to gradually rewrite only parts of it, just as I have done before, occasionally. It isn't quite that easy though, because, as I learned the hard way, even when I thought I completely understood a part of the program, errors kept creeping up. Often much, much later than when I initially wrote the replacement, making it much harder for me to understand my original thought process, and why it didn't work. So I need tests, and the only valid method of testing under these circumstances that I can think of are Unit Tests. The problem is - I never worked with Unit Tests, I only know of them through articles that mention them, and therefore I only have a vague understanding of what they are and how they are supposed to be done. Now I need some guidance where to start: I did search for articles and information on the web, but unfortunately didn't find anything useful that doesn't already require a thorough understanding of Unit Tests to start with. Somehow there don't seem to be any introductory articles - or I'm too stupid to find them. If anyone knows a link, could you please share? As for books, I know there are literally thousands out there - but that is also my problem: which of them would be the right one? Could you advise some good picks to help out an old, experienced programmer, who for some reason never picked up the art of unit-testing?
The idea of unit tests is to test the functionality of units, as opposed to whole systems. (I assume you've read the relevant Wikipedia page.) It's not as widely talked about in relation to C++ as some other languages (notably Java, but also C# and 'new' dynamic languages like Python, Ruby and that ilk) because a) there isn't an established practically-standard unit testing framework, and b) the language does not support reflection. You can roll your own little framework, or grab one of the many free ones available. See Exploring the C++ Unit Testing Framework Jungle for a decent run-down. At my current workplace we use Boost.Test, but I've used CppUnit in the past. They both work, but are clunky when compared with, say, NUnit for C#. Now, the idea is to test a unit, so you have a separate test application that links in the unit(s) you want to test, and work to their public interfaces. It's usual to have a setup-test-teardown structure, as you might need to create mock objects to pass in, or otherwise set up an environment to run the test. So next time you need to make a change to this old legacy application, you start with identifying what units/modules/classes you need to change, and then sit down and make a series of tests that test them. For it to be any use, you need to have a comprehensive test suite - if you test only the behaviour on one sanitised input your test has poor coverage. You want to be sure that you exercise the whole module, and all execution paths. This means testing for failures, to make sure that bad inputs or invalid environment setups generate appropriate failures (exceptions or error return codes). Test the edge cases, test "obviously" bad calls (for instance all functions that take int parameters that are assumed to be always positive - what happens when you pass in -1?) and test unlikely (but possible) combinations of inputs. So you make your tests, and you check that they pass. If they don't pass, you need to figure out if the test is poorly written, or if there is a bug in the unit. If you were fixing a bug in the unit, write a test that exercises that bug. In other words, write a test that will fail with the existing code. That way, you can verify that you've fixed the bug, since the test should pass after you're done. Oh, and somethi
-
The idea of unit tests is to test the functionality of units, as opposed to whole systems. (I assume you've read the relevant Wikipedia page.) It's not as widely talked about in relation to C++ as some other languages (notably Java, but also C# and 'new' dynamic languages like Python, Ruby and that ilk) because a) there isn't an established practically-standard unit testing framework, and b) the language does not support reflection. You can roll your own little framework, or grab one of the many free ones available. See Exploring the C++ Unit Testing Framework Jungle for a decent run-down. At my current workplace we use Boost.Test, but I've used CppUnit in the past. They both work, but are clunky when compared with, say, NUnit for C#. Now, the idea is to test a unit, so you have a separate test application that links in the unit(s) you want to test, and work to their public interfaces. It's usual to have a setup-test-teardown structure, as you might need to create mock objects to pass in, or otherwise set up an environment to run the test. So next time you need to make a change to this old legacy application, you start with identifying what units/modules/classes you need to change, and then sit down and make a series of tests that test them. For it to be any use, you need to have a comprehensive test suite - if you test only the behaviour on one sanitised input your test has poor coverage. You want to be sure that you exercise the whole module, and all execution paths. This means testing for failures, to make sure that bad inputs or invalid environment setups generate appropriate failures (exceptions or error return codes). Test the edge cases, test "obviously" bad calls (for instance all functions that take int parameters that are assumed to be always positive - what happens when you pass in -1?) and test unlikely (but possible) combinations of inputs. So you make your tests, and you check that they pass. If they don't pass, you need to figure out if the test is poorly written, or if there is a bug in the unit. If you were fixing a bug in the unit, write a test that exercises that bug. In other words, write a test that will fail with the existing code. That way, you can verify that you've fixed the bug, since the test should pass after you're done. Oh, and somethi
Thanks a ton for your time. I was aware of most of what you said, but as I picked up each individual piece of information from all kinds of different sources it's a great help for me to see it all in one cohesive writeup! I like the title of the article you linked to and will definitely try to get a look at it, but first I'll have to convince the admin to unblock the site :doh: I've heard about the 'clunkiness' of CppUnit and other tools, and that is one of the reasons I posted here: I didn't want to invest a lot of time to learn working with a tool as long as I don't even know what to expect and what everything is about - that kind of would be like learning to use a hammer without knowing what a nail is ;) I've already set up a little project to get a feel for it: it has two cpp files for each header file, one as a mockup and one for the real thing. That part was easy, but I already wonder if I could somehow automate it. :) Also I'm asking myself how to set up a framework that calls tests for more than one unit: If I replace one cpp file in the mock project, then I can only test that one unit; I need to create a new project for each test! Is that right? :confused:
-
Thanks a ton for your time. I was aware of most of what you said, but as I picked up each individual piece of information from all kinds of different sources it's a great help for me to see it all in one cohesive writeup! I like the title of the article you linked to and will definitely try to get a look at it, but first I'll have to convince the admin to unblock the site :doh: I've heard about the 'clunkiness' of CppUnit and other tools, and that is one of the reasons I posted here: I didn't want to invest a lot of time to learn working with a tool as long as I don't even know what to expect and what everything is about - that kind of would be like learning to use a hammer without knowing what a nail is ;) I've already set up a little project to get a feel for it: it has two cpp files for each header file, one as a mockup and one for the real thing. That part was easy, but I already wonder if I could somehow automate it. :) Also I'm asking myself how to set up a framework that calls tests for more than one unit: If I replace one cpp file in the mock project, then I can only test that one unit; I need to create a new project for each test! Is that right? :confused:
Mocking isn't always necessary - it depends on the units you are testing. If they conform to the classic design ideals - low coupling, small interface, high cohesion - the lion's share of their function should be testable with no mock objects. But seeing such designs in real life, especially in old production apps, is quite rare. :( A mockup is only needed if the unit you want to test is dependent on something that is expensive or impossible to provide in a simple test. For instance, a self-contained unit that calculates income tax probably wouldn't need any mock objects.
// Header file with interface of unit to test
namespace taxCalc
{
double calcIncomeTax(double annualIncome);
double calcIncomeTax(double annualIncome, double pensionDeduction);
...
}// Hand-rolled unit test file
void test_taxCalc()
{
std::cout << "Testing taxCalc::calcIncomeTax" << std::endl;
int passed = 0;
int tests = 4;
if (15.3 == calcIncomeTax(60.0))
++passed;
if (0 == calcIncomeTax(5.0))
++passed;
if (0 == calcIncomeTax(0.0))
++passed;
if (14.6 == calcIncomeTax(60.0, 4.0))
++passed;
std::cout << passed << " out of " << tests << " passed" << std::endl;
if (passed != tests)
std::cout << "There were errors" << std::endl;
}int main()
{
test_taxCalc();
return 0;
}There you go, a very simple unit test. As you see, there can be multiple tests in one function. If you had a second unit in the system, to calculate national insurance or something, you can test that in the same test file, though it makes sense to write a separate test function for it. Now, it could be that for hysterical reasons a simple function is dependent on something hugely expensive, like access to a corporate database with lots of security and whatnot. That's when you make a mock. Not of what you're testing, but of what the tested unit is dependent on.
// Expensive object interface
class DataBaseTableFromFortKnox
{
public:
DataBaseTableFromFortKnox();
// Lots of fancy stuff and set-up and keys and blood of firstborn
...
// The one function the unit we're testing actually uses
double GetCentsPerDollar() const;
...
};// Header file with interface of unit to test
namespace taxCalc
{
double calcIncomeTax(double annualIncome);
double calcIncomeTax(double annualIncome, double pensionDeduction);
double calcIncomeTaxInCents(double annualIncome, const DataBaseTableFromFortKnox& dbTab);
...
}// Ou
-
Thanks a ton for your time. I was aware of most of what you said, but as I picked up each individual piece of information from all kinds of different sources it's a great help for me to see it all in one cohesive writeup! I like the title of the article you linked to and will definitely try to get a look at it, but first I'll have to convince the admin to unblock the site :doh: I've heard about the 'clunkiness' of CppUnit and other tools, and that is one of the reasons I posted here: I didn't want to invest a lot of time to learn working with a tool as long as I don't even know what to expect and what everything is about - that kind of would be like learning to use a hammer without knowing what a nail is ;) I've already set up a little project to get a feel for it: it has two cpp files for each header file, one as a mockup and one for the real thing. That part was easy, but I already wonder if I could somehow automate it. :) Also I'm asking myself how to set up a framework that calls tests for more than one unit: If I replace one cpp file in the mock project, then I can only test that one unit; I need to create a new project for each test! Is that right? :confused:
Also: one purpose of unit test frameworks is to aid with reporting and result checking, keeping track of how may tests you have, trapping expected exceptions and so on. I'd recommend Boost, if only because it's got the best documentation of the frameworks I've looked at. It's using lots of macros, which isn't to everybody's taste, but it's quite straightforward to use.
-
Mocking isn't always necessary - it depends on the units you are testing. If they conform to the classic design ideals - low coupling, small interface, high cohesion - the lion's share of their function should be testable with no mock objects. But seeing such designs in real life, especially in old production apps, is quite rare. :( A mockup is only needed if the unit you want to test is dependent on something that is expensive or impossible to provide in a simple test. For instance, a self-contained unit that calculates income tax probably wouldn't need any mock objects.
// Header file with interface of unit to test
namespace taxCalc
{
double calcIncomeTax(double annualIncome);
double calcIncomeTax(double annualIncome, double pensionDeduction);
...
}// Hand-rolled unit test file
void test_taxCalc()
{
std::cout << "Testing taxCalc::calcIncomeTax" << std::endl;
int passed = 0;
int tests = 4;
if (15.3 == calcIncomeTax(60.0))
++passed;
if (0 == calcIncomeTax(5.0))
++passed;
if (0 == calcIncomeTax(0.0))
++passed;
if (14.6 == calcIncomeTax(60.0, 4.0))
++passed;
std::cout << passed << " out of " << tests << " passed" << std::endl;
if (passed != tests)
std::cout << "There were errors" << std::endl;
}int main()
{
test_taxCalc();
return 0;
}There you go, a very simple unit test. As you see, there can be multiple tests in one function. If you had a second unit in the system, to calculate national insurance or something, you can test that in the same test file, though it makes sense to write a separate test function for it. Now, it could be that for hysterical reasons a simple function is dependent on something hugely expensive, like access to a corporate database with lots of security and whatnot. That's when you make a mock. Not of what you're testing, but of what the tested unit is dependent on.
// Expensive object interface
class DataBaseTableFromFortKnox
{
public:
DataBaseTableFromFortKnox();
// Lots of fancy stuff and set-up and keys and blood of firstborn
...
// The one function the unit we're testing actually uses
double GetCentsPerDollar() const;
...
};// Header file with interface of unit to test
namespace taxCalc
{
double calcIncomeTax(double annualIncome);
double calcIncomeTax(double annualIncome, double pensionDeduction);
double calcIncomeTaxInCents(double annualIncome, const DataBaseTableFromFortKnox& dbTab);
...
}// Ou
Cool Cow Orjan wrote:
low coupling, small interface, high cohesion
You guessed it: yes, 95% of our application is everything but that. Much of it is 'objectified C', stuff like structs renamed to class, but with everything staying on the public interface because it's being referenced in thousands of places. And that, exactly is the problem: everyone references everything, or in other words, calling our 'interfaces' huge might be an understatement. :( I know I will eventually have to face that, but before I do I wish to start small to get a hang of Unit Tests. What I'll start with is the libraries I wrote myself, beginning with one I've just started implementing. At least here I know exactly where it's referenced (nowhere so far), and what it is supposed to achieve. I've just seen your followup post. Thanks for the tip, yes I will be in need of good documentation, so I might just as well look into boost.