VS Test Tools & Live coverage - Am I overreacting?
-
Hello all. I'm kinda ranting with Microsoft and I wanted to ask CPian community about it, because I may be overreacting. For a new project I'm using unit testing, with vanilla unit tests. Many of these tests include throwing exceptions, such as ArgumentNullException, etc. Then I use VS 2017 Live Coverage to track how many code blocks are being tested and how many remain untested. I think this is a really useful tool, but I'm having this issue: when you run a test method with ExpectedExceptionAttribute, the Live Coverage marks the closing brace as not covered. Weird. This adds a small percentage of non-covered code blocks. So, as the project grows in size, and so do the unit tests, the % of non-covered blocks increase. In my project, I get a 8% of non-covered blocks just because of this scenario! I filed a bug report[^] with Microsoft, and they replied that this was by design and thus they won't fix it. So, what do you think, am I overreacting? I don't see the usefulness of a test coverage tool that will nevr yield a 100% coverage score unless I stop throwing exceptions altogether. Thoughts?
-
Hello all. I'm kinda ranting with Microsoft and I wanted to ask CPian community about it, because I may be overreacting. For a new project I'm using unit testing, with vanilla unit tests. Many of these tests include throwing exceptions, such as ArgumentNullException, etc. Then I use VS 2017 Live Coverage to track how many code blocks are being tested and how many remain untested. I think this is a really useful tool, but I'm having this issue: when you run a test method with ExpectedExceptionAttribute, the Live Coverage marks the closing brace as not covered. Weird. This adds a small percentage of non-covered code blocks. So, as the project grows in size, and so do the unit tests, the % of non-covered blocks increase. In my project, I get a 8% of non-covered blocks just because of this scenario! I filed a bug report[^] with Microsoft, and they replied that this was by design and thus they won't fix it. So, what do you think, am I overreacting? I don't see the usefulness of a test coverage tool that will nevr yield a 100% coverage score unless I stop throwing exceptions altogether. Thoughts?
On my behalf of understanding i think you overreact and misunderstood the purpose of Expected exceptions! Your Test is expecting that an exception is thrown at some place because (probably test data is, input parameters are forcing this). So you can only test until the exception is thrown, which results in non full coverage of the method, but the test is correct! If you want to do it completely right you might test a method way more than 10 times. Then you get 100% coverage on that method and yeah, obviously are running fully through it multiple times :) SO microsoft is completly right on this behaviour because you broke your method somewhere (Exception). I for my self write minimum 1 happy path, 1 total failure, and for each input param one test, plus if there is more than one input param several more where 1 param is good the other bad and vice versa. I learned this thinking at DWX on a Software Testing session and this guy was totally right. 16 Tests for one method? Wow, overload, but no, you find the bugs, kill them and your method is 100% unbreakable :)
if(this.signature != "")
{
MessageBox.Show("This is my signature: " + Environment.NewLine + signature);
}
else
{
MessageBox.Show("404-Signature not found");
} -
Hello all. I'm kinda ranting with Microsoft and I wanted to ask CPian community about it, because I may be overreacting. For a new project I'm using unit testing, with vanilla unit tests. Many of these tests include throwing exceptions, such as ArgumentNullException, etc. Then I use VS 2017 Live Coverage to track how many code blocks are being tested and how many remain untested. I think this is a really useful tool, but I'm having this issue: when you run a test method with ExpectedExceptionAttribute, the Live Coverage marks the closing brace as not covered. Weird. This adds a small percentage of non-covered code blocks. So, as the project grows in size, and so do the unit tests, the % of non-covered blocks increase. In my project, I get a 8% of non-covered blocks just because of this scenario! I filed a bug report[^] with Microsoft, and they replied that this was by design and thus they won't fix it. So, what do you think, am I overreacting? I don't see the usefulness of a test coverage tool that will nevr yield a 100% coverage score unless I stop throwing exceptions altogether. Thoughts?
You're unlikely to get 100% coverage on all methods from a single unit test, you generally need a unit test per path through your code, and probably more on top. As well as the test that tests null arguments and exceptions you need tests that exercise the "happy path" and all other paths until you get 100% coverage. That's just what unit testing is.