It's OK Not to Write Unit Tests
-
I once spent 3 months writing a reporting API in C# that spat out Excel docs sucking data from a localized data cube. Now the sales guys and gals could wow potential clients and it was generally well received. 'Twas a thing of surprising beauty considering it featured the words "Excel" and "reporting" in it. Then a Java guy plodded over and wanted to see the Unit Tests. I had none. He pointed out it was policy, and demanded to know how I could qualify a generated report. I jokingly suggested that I build an Excel document parser and run it against different templates based on desired output. He was not amused. Then I ran the app, set it for one year for one product variety and ran the report. I handed the printout to an Argentinean sales rep who happened to be in the office to see a demo. He said it looked good. Unit test complete.
10110011001111101010101000001000001101001010001010100000100000101000001000111100010110001011001011
This is a very roundabout way of saying that there's a competency issue here and that you don't know how to write unit tests. If you did, you'd do it.
-
From the article: Unit testing is no substitute for adversarial testing. A man after my own heart. Too many companies think that testers should be chums with engineering. That's dumb. Testers should have an adversarial relationship with engineering; their job isn't to prove a product works, but that it doesn't. Another way to put it is that testers should think engineers are arrogant assholes and set out to prove it. Engineering should be attempting prove otherwise. That said, I do think "unit" testing has it's place. When I write libraries and classes, I want to make sure they work the way I designed them to work. Writing them in a slightly more organized way helps me to debug them as well. (One of the beefs I do have with many testing suites is that they are awful in helping you debug.) However, I never for a moment believe that I'm doing anything but testing low level functionality and that it's any substitute for system/integration testing. (I'd even say that if my company were big enough and we had a separate testing department, they wouldn't use any of my tests. They'd write their own!) (I should point out that I use "unit" testing quite loosely here. When testing classes, I test them as a class. I just rewrote a Trie class; my test put stuff in and looked it up--I tested only the public interfaces, which tests the protected/private interfaces. But what about those private interfaces that aren't used by the public interfaces? There aren't any.)
There's a lot of truth here.
-
This is a very roundabout way of saying that there's a competency issue here and that you don't know how to write unit tests. If you did, you'd do it.
So what you're saying is that it's worth spending the company's time and money for a few weeks to develop parsing algorithms to determine whether objects embedded in an Excel document correspond to the given API call and/or data instead of using human visual acuity for an hour in printing and verifying 20 odd reports? You are technically correct, it could be done, however we are not in the business of doing what could be done. We are in the business of doing what is required by our clients so that they improve efficiency and make money. Asinine unit testing, especially wrt UI and reporting where functional testing is vastly better at catching subtlties, is an obsessive scourge brought about by software engineers who found a new hammer and saw everything as nails.
10110011001111101010101000001000001101001010001010100000100000101000001000111100010110001011001011
-
I personally really like unit testing and I am quite suprised by some of the replies here. It's an interesting comment about college students liking it. I left university a while back, and I think it's unfair to call me incompetent. Unit testing was never mentioned when I was at university though. What do people do instead of unit testing? Do you manually start the whole application and test every feature manually? With unit tests you can test a whole application very quickly. Even the best programmers make mistakes. What happens when you take over another project from somebody else eg. when joining a new company. You may not know what it does, how it does it, or even how obscure the code may be. If the code came with unit tests then you would have some confidence in changes that you make. Some code is very fragile, and like a mine field when making changes. Making developers write tests also forces them to write testable code. Although testable code isn't necessarily good, I think it will improve bad code in most cases. I also think it is very good for a developer to be writing the tests themselves. Only a developer will really understand the potential problems in the code. I find it useful to look at the code and think about what may happen eg. what if the parameters are NULL? It's very easy to write tests that make sure that special values like NULL are handled correctly, and these are the areas where we usually get problems. The tests also depend on how well they are written. Bad tests are almost useless, but good ones will highlight problems before they reach the customer. I'm not saying that unit tests are right or wrong, but I do think they have their place. I like them and want to use them more, but after seeing this thread I am starting to wonder whether I'm heading in the right direction. Any thoughts?
-
So what you're saying is that it's worth spending the company's time and money for a few weeks to develop parsing algorithms to determine whether objects embedded in an Excel document correspond to the given API call and/or data instead of using human visual acuity for an hour in printing and verifying 20 odd reports? You are technically correct, it could be done, however we are not in the business of doing what could be done. We are in the business of doing what is required by our clients so that they improve efficiency and make money. Asinine unit testing, especially wrt UI and reporting where functional testing is vastly better at catching subtlties, is an obsessive scourge brought about by software engineers who found a new hammer and saw everything as nails.
10110011001111101010101000001000001101001010001010100000100000101000001000111100010110001011001011
Okay, so it can be done. But are you capable of it? Again, a question of competency.
-
Okay, so it can be done. But are you capable of it? Again, a question of competency.
Yes, but irrelevant. As much as you're trying to portray me as an incompetent boob, you're at risk of seeming to be an expensive liability. Unit testing has it's place, typically away from human interactive aspects of the application where, in my experience, most exceptions occur as a consequence of multiple interactions in unusual sequences. These are seldom caught by unit tests which are by design more atomic in their testing.
10110011001111101010101000001000001101001010001010100000100000101000001000111100010110001011001011
-
You're so controversial! I agree with the article. Some people test for the sake of testing, as if that's more important than actually getting something done. I've seen tests that ensures that a property getter returns the last value set by the setter. :wtf: I also don't buy the design methodology that tests should be written first and then the code. This will affect the overall design, in such a way that you might not find holes in the design until it's time to program the damn thing. What do you do then? Rewrite the old tests and/or write new tests. I've found that tests that can be automated, and that actually tests a fair amount of complexity, are more helpful than tests like the stupid getter/setter example above. Tests like "does this collection of objects sent through the DB-layer generate this set of rows in the tables?". I guess I'm swearing in the TDD church, but who cares. I want to deliver correct software on time.
-
And a response article from Justin Etheredge: It's OK to write unit tests[^].
Religiously blogging on the intarwebs since the early 21st century: Kineti L'Tziyon Judah Himango
-
Yes, but irrelevant. As much as you're trying to portray me as an incompetent boob, you're at risk of seeming to be an expensive liability. Unit testing has it's place, typically away from human interactive aspects of the application where, in my experience, most exceptions occur as a consequence of multiple interactions in unusual sequences. These are seldom caught by unit tests which are by design more atomic in their testing.
10110011001111101010101000001000001101001010001010100000100000101000001000111100010110001011001011
Well, maybe it's not a competency issue. You can do it. But you don't want to. Is it because you don't find it valuable? Also, even if this software isn't a great candidate to be unit tested, there would still be value in having a suite of automated functional tests to validate it's functionality. This would help you if you need to test your product with different versions of Office, including future ones. It's also easier to transfer your code to another programmer, even an intern. You can trust that they probably won't screw things up too bad as long as your tests are run with whatever changes they make (within reason. tests can't catch everything.) If you were to quit the company, it would certainly be easier for someone else to understand and take over your work if there were a great set of automated tests. Is the product moving to a maintenance phase where others have to support it for years to come? Think of those maintenance programmers, who may be outsourced. However, even without functional testing, there's logic in your code that is independent of the user interface which could benefit from being unit tested. It's a lot of work, without direct benefit to that sales guy that you showed the report to, and harder to justify your time doing. A lot of great reasons not to do the right thing, which is why so many people don't do it. In the end, this is a management issue. If the managers don't require programmers to deliver on automated tests, then they won't get it, and they're the ones that will ultimately suffer. If they do require it, the programmer really won't have much of a choice, unless he doesn't want to get a paycheck.
-
Wow, I thought unit testing was all about focusing on the functionality of a quite logical independent unit, "not (a) test (that) cannot talk to a database, communicate across the network, touch the filesystem, run concurrently with another test, or require extensive setup. If they are any dependencies, they are mocked away". Dogmatic approaches are pretty useless for me. :)
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler. -- Alfonso the Wise, 13th Century King of Castile.
This is going on my arrogant assumptions. You may have a superb reason why I'm completely wrong. -- Iain Clarke
[My articles]I think you're missing the point. A unit test ultimately tries to validate the functionality of a single piece of code. Imagine a method being a math function. You simply want to validate that *that* code is correct, not the rest of the system. So you do that by mocking everything around it. It's a bad argument to say that by doing a unit test it doesn't test how the code works integrated with the rest of the system, therefore a unit test is not useful. No one is saying that the unit test is the only test for code. In fact, some argue that's not even a "test", but a way to verify the the design. You still need integration, functional, exploratory tests, etc.
-
Unit tests like so much methodology these days are simply ways to commoditize software development. Nothing more, nothing less and like cows to slaughter most developers eat up these kinds of methodologies like candy all the while ignoring the fact that these were designed primarily to make us replaceable worthless cogs in a big machine of corporate development. Unit tests are on most levels counterproductive and worthless to any decent software developer who can recognize bad code before or while they are writing it and nip it in the bud. They're great for accounting and managerial types though so .... yay for them! :) Software developers used to be gods in their domain, now they're just easily replaced cogs in a giant machine of mediocrity and have no one to blame but themselves when their work can be easily outsourced or indeed they themselves can be easily replaced with someone cheaper and less experienced.
"Creating your own blog is about as easy as creating your own urine, and you're about as likely to find someone else interested in it." -- Lore Sjöberg
You're right. Unit testing makes it easier to replace developers. Is that a bad things? So are you saying you won't write unit tests so that you're not as easily replaceable? The more you're difficult to replace, the more you're like to be stuck at the same job.
-
I don't understand article, or author is writing about writing unit test to his own code? If yes then, something like this is like leaving thief to judge himself. When you write app alone unit tests are pointless. "Let's be honest. Your tests mostly follow the “happy path”." Get a tester and You won't follow the "happy path" in tests.
You're absolutely right. A unit test can only verify the programmer's assumptions, which is why it's never going to be enough, and you need a separate person to run the functional and exploratory tests. Don't downplay unit-testing because of this fact. No one is saying unit tests are the only tests. You're using a staw-man argument. If you write the code perfectly the first time, without unit tests, you still need unit tests because someone else may need to change your code to add functionality. And it'd be much better if they had your assumptions codified as much as possible so that they don't break your code.
-
And a response article from Justin Etheredge: It's OK to write unit tests[^].
Religiously blogging on the intarwebs since the early 21st century: Kineti L'Tziyon Judah Himango
Eventually, computer languages will be designed such that you're encouraged, if not required, to write a test first for a piece of code before you write the code.
-
You're absolutely right. A unit test can only verify the programmer's assumptions, which is why it's never going to be enough, and you need a separate person to run the functional and exploratory tests. Don't downplay unit-testing because of this fact. No one is saying unit tests are the only tests. You're using a staw-man argument. If you write the code perfectly the first time, without unit tests, you still need unit tests because someone else may need to change your code to add functionality. And it'd be much better if they had your assumptions codified as much as possible so that they don't break your code.
I don't want to downplay writing unit tests at all but only writing unit tests to your code, if someone else writes it, then problem stated in article, as I understand doesn't exist.
-
Well, maybe it's not a competency issue. You can do it. But you don't want to. Is it because you don't find it valuable? Also, even if this software isn't a great candidate to be unit tested, there would still be value in having a suite of automated functional tests to validate it's functionality. This would help you if you need to test your product with different versions of Office, including future ones. It's also easier to transfer your code to another programmer, even an intern. You can trust that they probably won't screw things up too bad as long as your tests are run with whatever changes they make (within reason. tests can't catch everything.) If you were to quit the company, it would certainly be easier for someone else to understand and take over your work if there were a great set of automated tests. Is the product moving to a maintenance phase where others have to support it for years to come? Think of those maintenance programmers, who may be outsourced. However, even without functional testing, there's logic in your code that is independent of the user interface which could benefit from being unit tested. It's a lot of work, without direct benefit to that sales guy that you showed the report to, and harder to justify your time doing. A lot of great reasons not to do the right thing, which is why so many people don't do it. In the end, this is a management issue. If the managers don't require programmers to deliver on automated tests, then they won't get it, and they're the ones that will ultimately suffer. If they do require it, the programmer really won't have much of a choice, unless he doesn't want to get a paycheck.
You're still thinking solely along engineering lines. This is irrelevant to the situation I described. What MAY happen in the future is a probability, it is a calculated risk. Also you're weighing time spent developing now as equal to time spent developing in the future. This is wrong. Time spent now is more expensive then time spent in the future. Time spent now solving or mitigating theoretical future upgrades is much much much more expensive then time spent in the future fixing it. This is perhaps because you're seeing the software project as the be all and end all. It's not, the product helps others perform their tasks more effectively thereby improving their productivity and reducing costs. Cost savings net overtime and can be allocated into more productive tasks netting an even higher ROI. Delaying a product with the aim of producing perfect testing reduces the potential ROI by constraining the time the productivity boost would otherwise have to work. This ultimately will cost more than fixing issues in the future. In the end it's up to the project sponsers to weigh these competing factors and decide the most reasonable path. Creating unit tests, though feasible, for a reporting engine was simply not cost effective. Unit testing seems to have become the new Gospel of testing over the last 5 or 6 years, but like all IT holy crusades it was led solely by an Engineering mindset whilst ignoring that ever annoying bogeyman we like to call the real world.
10110011001111101010101000001000001101001010001010100000100000101000001000111100010110001011001011
-
You're right. Unit testing makes it easier to replace developers. Is that a bad things? So are you saying you won't write unit tests so that you're not as easily replaceable? The more you're difficult to replace, the more you're like to be stuck at the same job.
Kenneth Kasajian wrote:
So are you saying you won't write unit tests so that you're not as easily replaceable?
Nope, personally I don't write unit tests because I work in a small shop with very experienced developers and we have a very good and thorough testing team and we can't afford the overhead because the benefit does not outweigh the cost. We tested that method of development and it turned out to be redundant for us, too slow and time consuming and caught none of the typical bugs that we normally always see anyway which are highly complex interactions between things caused by users doing the unexpected and unanticipated. My primary problem with many modern development methodologies is not that they make developers easily replaceable, that's just a side effect of my primary concern which is that they commoditize developers. In other words they reduce what used to be the job of a skilled and experienced craftsman to a job that can be done by any idiot with minimal experience. This drives down the value of all developers everywhere, i.e. salaries, job security and it often results in software that is less than exceptional. It's a method of risk aversion and risk aversion is pretty much everything that is wrong with every creativity based industry these days. It's why you predominantly get bland remakes of movies with the same small set of actors as every other movie instead of new compelling movies with new talent. In the end I suspect it's really not saving software houses any money in the long run commoditizing development. One highly skilled and talented and experienced developer is worth a whole team of cheap developers. Ask any experienced restaurateur whether they can put out more dishes of high quality with a huge brigade of average cooks or a very small highly skilled team and there's no comparison. Something the hospitality industry has known for centuries but the software development industry seems determined to ignore. The problem for software houses is they are afraid to have so much riding on so few people, i.e. risk averse. And no one is ever stuck at the same job, we do have free will after all. :) A highly skilled developer can get work they truly enjoy in any circumstances.
"Creating your own blog is about as easy as creating your own urine, and you're about as likely to find someone else interested in it." -- Lore Sjöberg
-
You're still thinking solely along engineering lines. This is irrelevant to the situation I described. What MAY happen in the future is a probability, it is a calculated risk. Also you're weighing time spent developing now as equal to time spent developing in the future. This is wrong. Time spent now is more expensive then time spent in the future. Time spent now solving or mitigating theoretical future upgrades is much much much more expensive then time spent in the future fixing it. This is perhaps because you're seeing the software project as the be all and end all. It's not, the product helps others perform their tasks more effectively thereby improving their productivity and reducing costs. Cost savings net overtime and can be allocated into more productive tasks netting an even higher ROI. Delaying a product with the aim of producing perfect testing reduces the potential ROI by constraining the time the productivity boost would otherwise have to work. This ultimately will cost more than fixing issues in the future. In the end it's up to the project sponsers to weigh these competing factors and decide the most reasonable path. Creating unit tests, though feasible, for a reporting engine was simply not cost effective. Unit testing seems to have become the new Gospel of testing over the last 5 or 6 years, but like all IT holy crusades it was led solely by an Engineering mindset whilst ignoring that ever annoying bogeyman we like to call the real world.
10110011001111101010101000001000001101001010001010100000100000101000001000111100010110001011001011
Time spent now is a lot cheaper than time spent in the future. This is not about designing for features that don't yet exist. This is about developing a complete solution for the software you're delivering now. Maintainability is built-in, not added on, so are pretty much every other non-functional requirement.
-
I think you're missing the point. A unit test ultimately tries to validate the functionality of a single piece of code. Imagine a method being a math function. You simply want to validate that *that* code is correct, not the rest of the system. So you do that by mocking everything around it. It's a bad argument to say that by doing a unit test it doesn't test how the code works integrated with the rest of the system, therefore a unit test is not useful. No one is saying that the unit test is the only test for code. In fact, some argue that's not even a "test", but a way to verify the the design. You still need integration, functional, exploratory tests, etc.
Kenneth Kasajian wrote:
So you do that by mocking everything around it.
That's very arguable.
Kenneth Kasajian wrote:
In fact, some argue that's not even a "test", but a way to verify the the design. You still need integration, functional, exploratory tests, etc.
I consider it a test.
Kenneth Kasajian wrote:
You still need integration, functional, exploratory tests, etc.
I know that. My point was against 'academic' approach to unit testing in real systems. :)
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler. -- Alfonso the Wise, 13th Century King of Castile.
This is going on my arrogant assumptions. You may have a superb reason why I'm completely wrong. -- Iain Clarke
[My articles] -
Kenneth Kasajian wrote:
So are you saying you won't write unit tests so that you're not as easily replaceable?
Nope, personally I don't write unit tests because I work in a small shop with very experienced developers and we have a very good and thorough testing team and we can't afford the overhead because the benefit does not outweigh the cost. We tested that method of development and it turned out to be redundant for us, too slow and time consuming and caught none of the typical bugs that we normally always see anyway which are highly complex interactions between things caused by users doing the unexpected and unanticipated. My primary problem with many modern development methodologies is not that they make developers easily replaceable, that's just a side effect of my primary concern which is that they commoditize developers. In other words they reduce what used to be the job of a skilled and experienced craftsman to a job that can be done by any idiot with minimal experience. This drives down the value of all developers everywhere, i.e. salaries, job security and it often results in software that is less than exceptional. It's a method of risk aversion and risk aversion is pretty much everything that is wrong with every creativity based industry these days. It's why you predominantly get bland remakes of movies with the same small set of actors as every other movie instead of new compelling movies with new talent. In the end I suspect it's really not saving software houses any money in the long run commoditizing development. One highly skilled and talented and experienced developer is worth a whole team of cheap developers. Ask any experienced restaurateur whether they can put out more dishes of high quality with a huge brigade of average cooks or a very small highly skilled team and there's no comparison. Something the hospitality industry has known for centuries but the software development industry seems determined to ignore. The problem for software houses is they are afraid to have so much riding on so few people, i.e. risk averse. And no one is ever stuck at the same job, we do have free will after all. :) A highly skilled developer can get work they truly enjoy in any circumstances.
"Creating your own blog is about as easy as creating your own urine, and you're about as likely to find someone else interested in it." -- Lore Sjöberg
It's not risk aversion, it's risk mitigation, which is accepting whatever that would introduce the risk, but you mitigate against it. The cost of developers finding a bug and fixing it is always less expensive than software testers doing the same. Let them find the problems you cannot find. If there's a bug you can find by writing an isolated unit test, it would be more expensive if that test were to pass through to software test, and have them have to find it, reproduce it, write down the repeat steps, create an incident in a bug tracking system, then you fixing it, then they have to verify that you fixed it. When you increase the length of development by adding more time for developers to write unit tests, it's a much more predictable % increase, then if you were to deliver more buggy code software test. It's hard for them to predict how long their test cycle is going to be at the start of the project, when no code has been written. They have to predict how many additional cycles are required, which is based on how well the previous cycles went. Cycles 2 and on are extremely difficult to predict. In the end the cost of the project may be the same, but spending it in the more predictable stage of the project enables you to have more predictable release dates, which in the end, saves more money for the company.
-
Time spent now is a lot cheaper than time spent in the future. This is not about designing for features that don't yet exist. This is about developing a complete solution for the software you're delivering now. Maintainability is built-in, not added on, so are pretty much every other non-functional requirement.
Not if it delays deployment, and not if deployment is delayed to mitigate what *might* happen. Seesh, please read the replies. Time spent building a decent architecture may reduce future costs if you have a good idea of what will be required. If not it won't since you're designing against what might be there, and so far as I know there isn't a recognized profession of Clairvoyent Software Engineer. Similarly spending significant extra time developing unit tests of no immediate value to the client costs more than future fixes if it forces delays in deployment. Sales Persons had no reliable way of dynamically generating reports in the field. That means lost sales. That means lost revenue. That means lost profit. It's nice that you want to reduce the work load on future developers, but ultimately it's *probable* future work thus not real work or therein a real cost. It's considered a fractional cost as it's a risk and not an absolute which again makes work now more expensive. I feel supremely confident that I'm a good source to verify that CBA decision was a good one since I was actually there and helped make it. Apparently you were there too and disagreed? I'm sorry I don't remember you. All code dealing directly with the automation objects were abstracted to well documented classes. This was deemed sufficient to mitigate future risk. If you can recall what your viewpoint was at the meeting, please remind me ...
10110011001111101010101000001000001101001010001010100000100000101000001000111100010110001011001011