The importance of testers
-
Hi All, Please could anyone suggest a link, or some convincing arguments, or examples from your own experiences, on the need for independent (i.e. not the same people that wrote the code) software/hardware testers. I've had a look with Google, which comes up with lots of companies offering testing services, but what I'm after at this point is some reasons that I can use in a memo to management about why testing is: a) not a menial task, and b) why R&D should be supported by independent testers and not do the whole product testing process entirely themselves. Joel On Software has a great article http://www.joelonsoftware.com/articles/fog0000000067.html[^], and so I'm looking for more of the same type of thing please? Many thanks, Sam W
-
Hi All, Please could anyone suggest a link, or some convincing arguments, or examples from your own experiences, on the need for independent (i.e. not the same people that wrote the code) software/hardware testers. I've had a look with Google, which comes up with lots of companies offering testing services, but what I'm after at this point is some reasons that I can use in a memo to management about why testing is: a) not a menial task, and b) why R&D should be supported by independent testers and not do the whole product testing process entirely themselves. Joel On Software has a great article http://www.joelonsoftware.com/articles/fog0000000067.html[^], and so I'm looking for more of the same type of thing please? Many thanks, Sam W
Well the most crucial reason is that the architect of the system knows what it's supposed to do, and why, so will often miss incredibly glaring bugs, simply because they know not to do something, so they never try. A user who's just given a bit of software and is asked to use it, will not be aware of the assumptions the developer made, so even though it's obvious to you that you shouldn't press the "process data" button until an input file is selected, it might not be to the end user, so they'll probably try it. Basically, if it can go wrong, they'll find a way, and will often manage to break software in many bizarre and inexpicable ways. -- Help me! I'm turning into a grapefruit! Phoenix Paint - back from DPaint's ashes!
-
Hi All, Please could anyone suggest a link, or some convincing arguments, or examples from your own experiences, on the need for independent (i.e. not the same people that wrote the code) software/hardware testers. I've had a look with Google, which comes up with lots of companies offering testing services, but what I'm after at this point is some reasons that I can use in a memo to management about why testing is: a) not a menial task, and b) why R&D should be supported by independent testers and not do the whole product testing process entirely themselves. Joel On Software has a great article http://www.joelonsoftware.com/articles/fog0000000067.html[^], and so I'm looking for more of the same type of thing please? Many thanks, Sam W
Well, I can think of a few from my own personal experience and my experince with other developers and dedicated testers (or test teams). Now, understand that I am not trying to argue that a developer should do no testing at all... They should test their own stuff to a point beyond compile-and-link, but short of full 2-day regression tests. 1: Concurrent Work! A developer can complete one part of something and while the test team is executing the test harness on it, (s)he can begin implementation on the other part. Additionally, a developer does not have to "switch gears", which might cause them to lose productivity. 2: Costs. I do not know about you, but given my salary and the salary of most of the testers (QA staff) that I have every worked with, my time costs a bit more. Ask management if they want to spend $50/hr to have a developer test software or $25/hr to have a tester do it. 3: Expertise. Developers are good at developing software - at least, they should be! :) Developers are educated and trained for that purpose. Testers, on the other hand, are educated and trained for the purpose of QA-ing software. 4: Impartiality (most of the time!). Developers often have a lot of pride in their work. They have to; if they did not care much, the code quality would likely suffer as a result. However, when a developer tests their own code, they might be more likely to disregard a possible issue, or try to hide it completely. Testers, acting as impartial staff, should call 'em as they see 'em. 5: A Second Opinion. Having (and using) testing staff guarantees that the product at least gets a "second opinion" on it before release. Note that I firmly believe that ALL developers should spend at least 6-12 months early in their career working in QA as a tester. This will allow them to gain a much better respect for the hard work and process required to correctly test software. Peace! -=- James
If you think it costs a lot to do it right, just wait until you find out how much it costs to do it wrong!
Tip for new SUV drivers: Professional Driver on Closed Course does not mean your Dumb Ass on a Public Road! -
Well the most crucial reason is that the architect of the system knows what it's supposed to do, and why, so will often miss incredibly glaring bugs, simply because they know not to do something, so they never try. A user who's just given a bit of software and is asked to use it, will not be aware of the assumptions the developer made, so even though it's obvious to you that you shouldn't press the "process data" button until an input file is selected, it might not be to the end user, so they'll probably try it. Basically, if it can go wrong, they'll find a way, and will often manage to break software in many bizarre and inexpicable ways. -- Help me! I'm turning into a grapefruit! Phoenix Paint - back from DPaint's ashes!
I agree. I have found out in recent years that I am the worst possible tester of my own code because I know too much how it operates and generally always hit the buttons in the proper sequence... I also believe for testing purposes you need to employ a variety of different testers of your code each with different skill levels. The reason for this has come up recently in one of my applications. I had an application with a property page that the user was supposed to enter all the patient information then click ok for the property page to be prompted for a yes/no question that would activate a second property page on yes. In days of testing my application not a single problem with this property sheet was found by about 5 different windows users. They each filled out the box about 100 times as part of the data entry process. A few weeks later one of the end users of my application was complaining that the yes/no question was popping up before they entered the patient information. How could this have happened? We tested this property page several hundred times. At the same time there was a hardware problem with the system so I decided to drive out to check on it. I fixed the hardware problem in a few seconds (the input tray of a film loader was misaligned). So after fixing the hardware problem I had the user send a case via my system while I was watching. She proceeded to click on each edit box with the mouse and hit enter after typing each entry. :sigh: I can't believe it. How long has windows had the enter button to mean the dialog is finished? Now I see the problem with my testers. They were all are experienced windows users and would have never hit the enter button before entering all the data. John
-
Well, I can think of a few from my own personal experience and my experince with other developers and dedicated testers (or test teams). Now, understand that I am not trying to argue that a developer should do no testing at all... They should test their own stuff to a point beyond compile-and-link, but short of full 2-day regression tests. 1: Concurrent Work! A developer can complete one part of something and while the test team is executing the test harness on it, (s)he can begin implementation on the other part. Additionally, a developer does not have to "switch gears", which might cause them to lose productivity. 2: Costs. I do not know about you, but given my salary and the salary of most of the testers (QA staff) that I have every worked with, my time costs a bit more. Ask management if they want to spend $50/hr to have a developer test software or $25/hr to have a tester do it. 3: Expertise. Developers are good at developing software - at least, they should be! :) Developers are educated and trained for that purpose. Testers, on the other hand, are educated and trained for the purpose of QA-ing software. 4: Impartiality (most of the time!). Developers often have a lot of pride in their work. They have to; if they did not care much, the code quality would likely suffer as a result. However, when a developer tests their own code, they might be more likely to disregard a possible issue, or try to hide it completely. Testers, acting as impartial staff, should call 'em as they see 'em. 5: A Second Opinion. Having (and using) testing staff guarantees that the product at least gets a "second opinion" on it before release. Note that I firmly believe that ALL developers should spend at least 6-12 months early in their career working in QA as a tester. This will allow them to gain a much better respect for the hard work and process required to correctly test software. Peace! -=- James
If you think it costs a lot to do it right, just wait until you find out how much it costs to do it wrong!
Tip for new SUV drivers: Professional Driver on Closed Course does not mean your Dumb Ass on a Public Road!Thank you to all for your input. Much appreciated, Sam
-
Well, I can think of a few from my own personal experience and my experince with other developers and dedicated testers (or test teams). Now, understand that I am not trying to argue that a developer should do no testing at all... They should test their own stuff to a point beyond compile-and-link, but short of full 2-day regression tests. 1: Concurrent Work! A developer can complete one part of something and while the test team is executing the test harness on it, (s)he can begin implementation on the other part. Additionally, a developer does not have to "switch gears", which might cause them to lose productivity. 2: Costs. I do not know about you, but given my salary and the salary of most of the testers (QA staff) that I have every worked with, my time costs a bit more. Ask management if they want to spend $50/hr to have a developer test software or $25/hr to have a tester do it. 3: Expertise. Developers are good at developing software - at least, they should be! :) Developers are educated and trained for that purpose. Testers, on the other hand, are educated and trained for the purpose of QA-ing software. 4: Impartiality (most of the time!). Developers often have a lot of pride in their work. They have to; if they did not care much, the code quality would likely suffer as a result. However, when a developer tests their own code, they might be more likely to disregard a possible issue, or try to hide it completely. Testers, acting as impartial staff, should call 'em as they see 'em. 5: A Second Opinion. Having (and using) testing staff guarantees that the product at least gets a "second opinion" on it before release. Note that I firmly believe that ALL developers should spend at least 6-12 months early in their career working in QA as a tester. This will allow them to gain a much better respect for the hard work and process required to correctly test software. Peace! -=- James
If you think it costs a lot to do it right, just wait until you find out how much it costs to do it wrong!
Tip for new SUV drivers: Professional Driver on Closed Course does not mean your Dumb Ass on a Public Road!I would also add that when you test your own code, you tend to test specific areas based on your own prejudice that comes from knowing (or thinking you know) the areas that you've affected with your code. Good testers know not to be swayed by these factors and therefore, perform both unit testing for the new code and regression-testing against any modules that could be affected. Cheers, Tom Archer - Archer Consulting Group Programmer Trainer and Mentor and Project Management Consultant
-
Hi All, Please could anyone suggest a link, or some convincing arguments, or examples from your own experiences, on the need for independent (i.e. not the same people that wrote the code) software/hardware testers. I've had a look with Google, which comes up with lots of companies offering testing services, but what I'm after at this point is some reasons that I can use in a memo to management about why testing is: a) not a menial task, and b) why R&D should be supported by independent testers and not do the whole product testing process entirely themselves. Joel On Software has a great article http://www.joelonsoftware.com/articles/fog0000000067.html[^], and so I'm looking for more of the same type of thing please? Many thanks, Sam W
One of the most famous instances of a problem slipping through because the author/designer tested his own work was the Pentium FDIV bug. The guy who designed the microcode for the FDIV instruction also wrote his own test vectors and as we saw that particular bug was missed in testing and simulation. Intel then changed their methodology such that the original designers now NEVER write the test vectors. At least, that's what they claimed shortly after the bug's discovery and the chip recall.
-
Well, I can think of a few from my own personal experience and my experince with other developers and dedicated testers (or test teams). Now, understand that I am not trying to argue that a developer should do no testing at all... They should test their own stuff to a point beyond compile-and-link, but short of full 2-day regression tests. 1: Concurrent Work! A developer can complete one part of something and while the test team is executing the test harness on it, (s)he can begin implementation on the other part. Additionally, a developer does not have to "switch gears", which might cause them to lose productivity. 2: Costs. I do not know about you, but given my salary and the salary of most of the testers (QA staff) that I have every worked with, my time costs a bit more. Ask management if they want to spend $50/hr to have a developer test software or $25/hr to have a tester do it. 3: Expertise. Developers are good at developing software - at least, they should be! :) Developers are educated and trained for that purpose. Testers, on the other hand, are educated and trained for the purpose of QA-ing software. 4: Impartiality (most of the time!). Developers often have a lot of pride in their work. They have to; if they did not care much, the code quality would likely suffer as a result. However, when a developer tests their own code, they might be more likely to disregard a possible issue, or try to hide it completely. Testers, acting as impartial staff, should call 'em as they see 'em. 5: A Second Opinion. Having (and using) testing staff guarantees that the product at least gets a "second opinion" on it before release. Note that I firmly believe that ALL developers should spend at least 6-12 months early in their career working in QA as a tester. This will allow them to gain a much better respect for the hard work and process required to correctly test software. Peace! -=- James
If you think it costs a lot to do it right, just wait until you find out how much it costs to do it wrong!
Tip for new SUV drivers: Professional Driver on Closed Course does not mean your Dumb Ass on a Public Road!4: Impartiality (most of the time!). Developers often have a lot of pride in their work. They have to; if they did not care much, the code quality would likely suffer as a result. However, when a developer tests their own code, they might be more likely to disregard a possible issue, or try to hide it completely. Testers, acting as impartial staff, should call 'em as they see 'em. I disagree with this. QA people are not impartial and it's a good thing. They are motivated to break the software as well as to contribute to the improvement of its quality. So, they usually are not impartial, but they don't share the same partialities as the developer. Note that I firmly believe that ALL developers should spend at least 6-12 months early in their career working in QA as a tester. This will allow them to gain a much better respect for the hard work and process required to correctly test software. I would caution against this if a software development career is your real goal, because once you have QA on your resume (or cv), it can be a challenge getting into development. Every place you interview will tell you that they're looking for just a little bit more development experience than you have, but there's this greate QA opening for which you are a perfect fit. Matt Gerrans
-
4: Impartiality (most of the time!). Developers often have a lot of pride in their work. They have to; if they did not care much, the code quality would likely suffer as a result. However, when a developer tests their own code, they might be more likely to disregard a possible issue, or try to hide it completely. Testers, acting as impartial staff, should call 'em as they see 'em. I disagree with this. QA people are not impartial and it's a good thing. They are motivated to break the software as well as to contribute to the improvement of its quality. So, they usually are not impartial, but they don't share the same partialities as the developer. Note that I firmly believe that ALL developers should spend at least 6-12 months early in their career working in QA as a tester. This will allow them to gain a much better respect for the hard work and process required to correctly test software. I would caution against this if a software development career is your real goal, because once you have QA on your resume (or cv), it can be a challenge getting into development. Every place you interview will tell you that they're looking for just a little bit more development experience than you have, but there's this greate QA opening for which you are a perfect fit. Matt Gerrans
Matt Gerrans wrote: -> Impartiality [...] I disagree with this. [...] A good and valid point. However, if fostering this kind of atmosphere, I would suggest that you be careful not to create a "us vs. them"-kind of relationship between developers and QA. We are all supposed to work together, not be "out to get each other". That kind of situation can degenerate rather quickly. (I have seen it happen more than once.) Matt Gerrans wrote: -> Note that I firmly believe that ALL developers should spend at least 6-12 months early in their career working in QA as a tester. [...] I would caution against this if a software development career is your real goal, [...] Another good point. Revise my statement to add: "-But perhaps leave this experience off of your resume. Just say that due to an NDA, you cannot discuss the terms of your employment and/or responsibilities! :) I maintain that "walking a mile in his mocs" is still a good thing to do in principal, however. Peace! -=- James
If you think it costs a lot to do it right, just wait until you find out how much it costs to do it wrong!
Tip for new SUV drivers: Professional Driver on Closed Course does not mean your Dumb Ass on a Public Road!
DeleteFXPFiles & CheckFavorites -
Matt Gerrans wrote: -> Impartiality [...] I disagree with this. [...] A good and valid point. However, if fostering this kind of atmosphere, I would suggest that you be careful not to create a "us vs. them"-kind of relationship between developers and QA. We are all supposed to work together, not be "out to get each other". That kind of situation can degenerate rather quickly. (I have seen it happen more than once.) Matt Gerrans wrote: -> Note that I firmly believe that ALL developers should spend at least 6-12 months early in their career working in QA as a tester. [...] I would caution against this if a software development career is your real goal, [...] Another good point. Revise my statement to add: "-But perhaps leave this experience off of your resume. Just say that due to an NDA, you cannot discuss the terms of your employment and/or responsibilities! :) I maintain that "walking a mile in his mocs" is still a good thing to do in principal, however. Peace! -=- James
If you think it costs a lot to do it right, just wait until you find out how much it costs to do it wrong!
Tip for new SUV drivers: Professional Driver on Closed Course does not mean your Dumb Ass on a Public Road!
DeleteFXPFiles & CheckFavoritesNot to rub it in (honestly! :)), but Joel's take on the second one, from the linked article was this: **Here's one way not to deal with it:
- Don't even think of trying to tell college CS graduates that they can come work for you, but "everyone has to do a stint in QA for a while before moving on to code". I've seen a lot of this. Programmers do not make good testers, and you'll lose a good programmer, who is a lot harder to replace.
**Matt Gerrans