How would one go about doing this? (Code Coverage)

gantww

I've noted with interest that there seems to be a lack of open source tools for .NET test coverage analysis. I'm pretty comfortable with .NET, but off the top of my head, I can't figure out how someone could do it without making their own version of the runtime or tampering with the IL emitted when code is compiled (which sounds like either great resume material, insanity, or possibly both). I'm just kind of curious, but I'm not sure even how this sort of thing is typically done.

Jimmanuel

Easier than tampering with the IL is tampering with the source code. I used to do formal unit testing and I don't know if it's a formal term or not but we called it "instrumenting a file" and it's a bit of tedium that automated processes are made for. The trick is to modify the source without actually changing the control flow of the code. Take the following function:

private void MyMethod()
{
int x = SomeFunction();
int y = SomeOtherFunction();

if (x > y)
{
    // do something
}
else
{
    // do something else
}

}

There are 5 bits of code you need to ensure are executed, so for each place insert a boolean:

static bool flag1 = false;
static bool flag2 = false;
static bool flag3 = false;
static bool flag4 = false;
static bool flag5 = false;

private void MyMethod()
{
flag1 = true; // verify that the method even gets called

int x = SomeFunction();

flag2 = true; // if SomeFunction() throws an unhandled exception this won't be executed

int y = SomeOtherFunction();

flag3 = true; // same thing with SomeOtherFunction()

if (x > y)
{
    flag4 = true; // verify the true case
    // do something
}
else
{
    flag5 = true; // verify the false case
    // do something else
}

}

Then you can run your test cases and check the value of your bools to make sure that all of your code was executed. Note that the control flow of the code hasn't changed, we've just added stuff to verify that all bits are executed. The difficulty comes in when you start taking into account the endless variations of source code. For example:

private void MyMethod()
{
int x = SomeFunction();
int y = SomeOtherFunction();

if (x > y)
{
    // do something
}

}

looks easier, but you still need to insert the else to the if when you instrument that method to prove that the if statement can be failed. If the if can't be failed then it's unnecessary (dead code) and shouldn't be there. The biggest PITA to instrument (and test) is full MCDC coverage (google it). Take the following sample:

private void MyMethod(bool a, bool b)
{
if (a || b)
{
// do something
}
}

There's a multi-conditional if statement there, and you need to prove that both conditions are necessary. Why? Well, suppose hypothetically that a is always true. Then the statement will always evalua

Curtis Schlak

The .NET Framework has a profiling API available for just such a thing. Here's a nice introductory article.[^].

"we must lose precision to make significant statements about complex systems." -deKorvin on uncertainty

S Senthil Kumar

The CLR profiler allows you to rewrite IL at JIT time. That would be one place to put instrumentation code to track what code is getting executed.

Regards Senthil _____________________________ My Home Page |My Blog | My Articles | My Flickr | WinMacro