Intellitest - Automated Test Generation.
-
Then all you're testing is that the code you have does what it currently does....but you know it does, that's a tautology.
public int Add(int x, int y)
{
return x + x;
}The above code has a typo, it should be x + y, but my auto-generated tests are going to test;
public void TestAdd()
{
int addResult = Instance.Add(5, 8);
Assert.AreEqual(10, addResult);
}I see no value in this at all, it takes a human being who understands the *intent* of the code to write valid tests.
F-ES Sitecore wrote:
Then all you're testing is that the code you have does what it currently does
Exactly. I called them regression tests for a reason. Knowing I didn't change any other behavior unintentionally is the entire point. The application is very well covered via a manual pound on the keyboard test procedure. It also takes 1-2 weeks for a person to run through because it's so exhaustive. With the application currently only getting a trickle of keep alive funding, not having to spend a third or more on testing would be a major cost efficiency boost; but we can't cut the test procedure down without something to replace it. And we don't have anything approaching the amount of money that would be needed to do the massive refactor that would be needed to change the code into something that would place nicely with unit tests and then find all the regression bugs we created in the process.
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, waging all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
-
None of those are issues. Your unit tests shouldn't touch UI (winforms UI is inherently non unit testable, as is the session\cookies\etc\etc in a website), or the filesystem, or database connections. Unit tests are *not* integration tests, end-to-end tests, UI tests or anything else...they are for unit testing small functions of code, and not all code is unit testable. If your code is behind a click event it is untestable, if it references a textbox it is untestable, if it touches *any* resource not in its immediate environment such as file system\database\asp session\etc it is untestable. If you want your code unit-testable it normally takes a large amount of time and effort and specific architectural patterns such as abstracting away file systems etc, for it to be achieved. The reason the tool is ignoring your code is actually correct...that code can't be unit tested.
Automated tests are automated tests. What level they run at is just arguing over how many angels can dance on the head of a pin. Insisting that because 5 angels can dance on this pin, 9 on that one, and 23 on some other pin; means that you need to use three different tools with three different syntaxes to test them is even more assinine. If some of the high level tests take significantly longer to run, split them off and normally only run them on the CI server (at checkin or even just nightly is really huge); but there's no reason not to use the same tools to write and run them.
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, waging all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
-
Automated tests are automated tests. What level they run at is just arguing over how many angels can dance on the head of a pin. Insisting that because 5 angels can dance on this pin, 9 on that one, and 23 on some other pin; means that you need to use three different tools with three different syntaxes to test them is even more assinine. If some of the high level tests take significantly longer to run, split them off and normally only run them on the CI server (at checkin or even just nightly is really huge); but there's no reason not to use the same tools to write and run them.
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, waging all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
This thread is about unit tests, though. I think people are getting confused with the "automated" part in the thread title. It is the *generation* that is automated, not the execution. Also there are many reasons to use different suites to run different kinds of tests, but that's not particularly relevant to this thread.