Automated testing...
-
Joergen Sigvardsson wrote:
a POS application
Aw. Don't be so hard on yourself, Joergen. It can't be that bad. ;P
And I get on my knees and pray We don't get fooled again
:laugh: :laugh:
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
-
what you're describing is functional testing, not unit testing. -- Rules of thumb should not be taken for the whole hand.
So, unit testing doesn't test for correct functionality..? :~ I honestly never understood the difference between functional testing and unit testing, other than that unit testing often means testing very small pieces of code, while functional testing means testing a larger pieces of code.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
-
Joergen Sigvardsson wrote:
nd other types of "hard copy" output, which I unfortunately have to test by hand
One of my main apps is a form printing engine, and I wrote some custom test software that involves outputting the forms to PDFs (via a special PDF print driver), and then comparing the PDF against a known good version. Works like a charm and has saved my bacon on more than one occasion.
You just gave me an idea! In the debug build, I actually output printouts as bitmaps. Maybe if I stream it to disc... thanks for the idea man! Much appreciated!
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
-
I've been thinking "Ah, that's nice. In theory!" about software unit testing. Until now. :) I've implemented a VBScript scripting host in my app, which allows me to automate input to the app. The app (a POS application) does a series of operations based on the input, and outputs information to database (not counting receipts, and other types of "hard copy" output, which I unfortunately have to test by hand). I predict the output by calculating the output in beforehand in my VBScripts. I then compare the output in the database with my predictions. If there are any deviations from my predictions, I sound the alarm. I've just completed a couple of thousands of tests - a task which would normally have taken me many weeks to do by hand. Admittedly, this is not very fine grained module testing. It's basically seeing the entire app as a black box. But it's better than nothing, and I have already nailed two "once in a blue moon" bugs. Other than that, I think I'm very close to a release. :cool: ps. It's also a great way to stress test your app. After 2000 transactions - no memory leak. Take that, you janitor dependent programmers! :rolleyes: ds.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
-
So, unit testing doesn't test for correct functionality..? :~ I honestly never understood the difference between functional testing and unit testing, other than that unit testing often means testing very small pieces of code, while functional testing means testing a larger pieces of code.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
Joergen Sigvardsson wrote:
I honestly never understood the difference between functional testing and unit testing, other than that unit testing often means testing very small pieces of code, while functional testing means testing a larger pieces of code.
well...... close. Functional testing simulates user interaction or external influence, I/O with the outside world. Although you can look at unit testing as the same thing, and some people do. Basically unit testing should test a "unit" a piece of stand-alone code for correctness. Does the compression routine compress, does the decompression routine decompress based on a variety of samples of input data. Does the network class send, does it receive, does it calculate a CRC32, etc. A functional test is similar except it is usually applied to larger class groupings or full applications. If you are simulating mouse clicks, data entry, data transmission/receipt to an entire program to stimulate final-functionality then you are doing a functional test. The problem comes with noun-verb use of function. A functional test refers to the verb use of function, not the noun. Unit tests referse to the noun version of function. Since programmers love analogies: using a car as an example, a functionality test is test-driving a car after building it, a unit test would be making sure a wheel or gear moves, tubes hold heat/pressure to spec and pistons fire at correct timing, etc.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
So, unit testing doesn't test for correct functionality..? :~ I honestly never understood the difference between functional testing and unit testing, other than that unit testing often means testing very small pieces of code, while functional testing means testing a larger pieces of code.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
Unit testing, if a maner of testing functionalities. (At least it's my concept :p). It is the phylosophy to test a functionality in little portions. So basically it is not the same, but Unit testing is a way to test functionnalities. But if you were not testing func what are you testing?
-
I've been thinking "Ah, that's nice. In theory!" about software unit testing. Until now. :) I've implemented a VBScript scripting host in my app, which allows me to automate input to the app. The app (a POS application) does a series of operations based on the input, and outputs information to database (not counting receipts, and other types of "hard copy" output, which I unfortunately have to test by hand). I predict the output by calculating the output in beforehand in my VBScripts. I then compare the output in the database with my predictions. If there are any deviations from my predictions, I sound the alarm. I've just completed a couple of thousands of tests - a task which would normally have taken me many weeks to do by hand. Admittedly, this is not very fine grained module testing. It's basically seeing the entire app as a black box. But it's better than nothing, and I have already nailed two "once in a blue moon" bugs. Other than that, I think I'm very close to a release. :cool: ps. It's also a great way to stress test your app. After 2000 transactions - no memory leak. Take that, you janitor dependent programmers! :rolleyes: ds.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
Joergen Sigvardsson wrote:
The app (a POS application)...
This is unrelated (as most of my posts are) but around these parts, POS stands for "piece of sh*t" so whenever I see that I have to reread it! :laugh: Cheers! Mark
-
Joergen Sigvardsson wrote:
The app (a POS application)...
This is unrelated (as most of my posts are) but around these parts, POS stands for "piece of sh*t" so whenever I see that I have to reread it! :laugh: Cheers! Mark
Best thing I had was a job I was looking into that included support and deployment of POS systems to end users:laugh: I too had to reread it at first. I was thinking, "Boy, they sure do have a high opinion of their product";P
------------------------------------- Do not do what has already been done. Absolute power corrupts absolutely.. but it ROCKS absolutely, too.
-
So, unit testing doesn't test for correct functionality..? :~ I honestly never understood the difference between functional testing and unit testing, other than that unit testing often means testing very small pieces of code, while functional testing means testing a larger pieces of code.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
<kidding> unit test -> the programmer's tests integration test -> the test guys job :) </kidding> ... Well, of course if this is a project of one...
-
I really like unit testing. It also covers you from other programmers breaking your code, and blaming it on you. You have proof that you code passed the unit test before they touched it.
ednrgc wrote:
You have proof that you code passed the unit test before they touched it.
Don't you have your code in a revision controls system? Other dev, or you break the code, roll it back. :)
My Blog A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects. - -Lazarus Long
-
ednrgc wrote:
You have proof that you code passed the unit test before they touched it.
Don't you have your code in a revision controls system? Other dev, or you break the code, roll it back. :)
My Blog A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects. - -Lazarus Long
-
dan neely wrote:
what you're describing is functional testing, not unit testing.
True, but at least he is testing!!! :-D
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
what you're describing is functional testing, not unit testing. -- Rules of thumb should not be taken for the whole hand.
dan neely wrote:
what you're describing is functional testing, not unit testing.
That's an interesting point. When does unit testing cross over into functional testing? Or does it ever? In other words, if I test anything but my lowest level classes and methods, does that mean I'm now doing functional testing? Marc
People are just notoriously impossible. --DavidCrow
There's NO excuse for not commenting your code. -- John Simmons / outlaw programmer
People who say that they will refactor their code later to make it "good" don't understand refactoring, nor the art and craft of programming. -- Josh Smith -
Joergen Sigvardsson wrote:
The app (a POS application)...
This is unrelated (as most of my posts are) but around these parts, POS stands for "piece of sh*t" so whenever I see that I have to reread it! :laugh: Cheers! Mark
Mark Salsbery wrote:
but around these parts, POS stands for "piece of sh*t"
yup, in the business community it refers to Point-Of-Sale, which is a generalized term for capturing information at a remote sale location (gas station for instance, I was in the POS gasoline sales before transferring to Engineering with another company). Ironically, either definition works with POS even in the business community. ;) which was why I jumped ship and went to engineering instead of staying in accounting.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
Jeffry J. Brickley wrote:
True, but at least he is testing!!!
Which is why I ed him, instead ofing. -- Rules of thumb should not be taken for the whole hand.
dan neely wrote:
Which is why I ed him, instead ofing.
true... but the worst thing about nit picking... is you end up with a handfull of nits.... :rolleyes:
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
Mark Salsbery wrote:
but around these parts, POS stands for "piece of sh*t"
yup, in the business community it refers to Point-Of-Sale, which is a generalized term for capturing information at a remote sale location (gas station for instance, I was in the POS gasoline sales before transferring to Engineering with another company). Ironically, either definition works with POS even in the business community. ;) which was why I jumped ship and went to engineering instead of staying in accounting.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
Jeffry J. Brickley wrote:
Ironically, either definition works with POS even in the business community.
:laugh:
-
I've been thinking "Ah, that's nice. In theory!" about software unit testing. Until now. :) I've implemented a VBScript scripting host in my app, which allows me to automate input to the app. The app (a POS application) does a series of operations based on the input, and outputs information to database (not counting receipts, and other types of "hard copy" output, which I unfortunately have to test by hand). I predict the output by calculating the output in beforehand in my VBScripts. I then compare the output in the database with my predictions. If there are any deviations from my predictions, I sound the alarm. I've just completed a couple of thousands of tests - a task which would normally have taken me many weeks to do by hand. Admittedly, this is not very fine grained module testing. It's basically seeing the entire app as a black box. But it's better than nothing, and I have already nailed two "once in a blue moon" bugs. Other than that, I think I'm very close to a release. :cool: ps. It's also a great way to stress test your app. After 2000 transactions - no memory leak. Take that, you janitor dependent programmers! :rolleyes: ds.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
We've used automated testing for various specific situations (for example, determining the worst case execution times of function - this was with real-time embedded systems, where worst case execution times are important). But this[^] is my favourite automatic testing system (albeit for a non-mainstream language). Define properties you believe should hold true, and they'll be tested automatically.
-
I've been thinking "Ah, that's nice. In theory!" about software unit testing. Until now. :) I've implemented a VBScript scripting host in my app, which allows me to automate input to the app. The app (a POS application) does a series of operations based on the input, and outputs information to database (not counting receipts, and other types of "hard copy" output, which I unfortunately have to test by hand). I predict the output by calculating the output in beforehand in my VBScripts. I then compare the output in the database with my predictions. If there are any deviations from my predictions, I sound the alarm. I've just completed a couple of thousands of tests - a task which would normally have taken me many weeks to do by hand. Admittedly, this is not very fine grained module testing. It's basically seeing the entire app as a black box. But it's better than nothing, and I have already nailed two "once in a blue moon" bugs. Other than that, I think I'm very close to a release. :cool: ps. It's also a great way to stress test your app. After 2000 transactions - no memory leak. Take that, you janitor dependent programmers! :rolleyes: ds.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
Over the weekend, i set up NUnit and CruiseControl.NET to allow periodic, automated building and testing of a certain library that has, unfortunately, grown far too large to test by hand (not, i suspect, that it was ever properly tested even when it was small and simple). At present, i'm doing pretty much the same as you, treating the library as a black box. As time permits (and future bugs demand), i hope to add more fine-grained tests into the mix. My only regret is that i didn't set this up a year ago. It would have saved a lot of grief (and manual testing on my part). I've been thinking for a while now about building in a scripting engine for use in testing a larger, unmanaged app. The only thing really holding me back is the time needed to implement an Automation interface that properly captures the functionality of the application... These things would be sooo much easier up-front. :sigh: (Marc, you around? See, i don't listen to your advice either...)
---- I just want you to be happy; That's my only little wish...
-
Over the weekend, i set up NUnit and CruiseControl.NET to allow periodic, automated building and testing of a certain library that has, unfortunately, grown far too large to test by hand (not, i suspect, that it was ever properly tested even when it was small and simple). At present, i'm doing pretty much the same as you, treating the library as a black box. As time permits (and future bugs demand), i hope to add more fine-grained tests into the mix. My only regret is that i didn't set this up a year ago. It would have saved a lot of grief (and manual testing on my part). I've been thinking for a while now about building in a scripting engine for use in testing a larger, unmanaged app. The only thing really holding me back is the time needed to implement an Automation interface that properly captures the functionality of the application... These things would be sooo much easier up-front. :sigh: (Marc, you around? See, i don't listen to your advice either...)
---- I just want you to be happy; That's my only little wish...
Shog9 wrote:
The only thing really holding me back is the time needed to implement an Automation interface that properly captures the functionality of the application... These things would be sooo much easier up-front.
:nod: It took me some swearing, shedding of blood, sweating and god awful hacks to get away with the automation (it's near Daily WTF class). Last time I reimplemented an app framework for the application (I do dumb stuff like that once in a while, hoping that it'll ignite a total rewrite of the released app), I did it properly. All operations are executed using a generalized command abstraction. Not only would it have been easy to implement stuff like redo/undo, it would've made automation sooo much easier. One day...
-- No humans were probed in the making of this episode
-
So, unit testing doesn't test for correct functionality..? :~ I honestly never understood the difference between functional testing and unit testing, other than that unit testing often means testing very small pieces of code, while functional testing means testing a larger pieces of code.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
No it's more a matter of scale. In Unit testing you're checking that MyClass.DoFoo() MyClass.DoBar() does Foo and Bar to MyClass in every possible case including all the boundary conditions. It's the bottom up approach. Functional testing is topdown, you're looking at the overall behavior, so when you test the File->Baz menu item you're only worrying about the final result, not the details of the implementation (ie calls MyClass.DoFoo(), MyClass.DoBar() and MyOtherClass.DoQux()). In theory a good functional test suite should cover all the same areas that the unit tests do. The biggest differences is that unit tests make isolating a specific problem easiest (ie the problem is MyClass.DoBar(), not something called by File->Baz), while functional testing's big picture makes catching boundaries/interations you didn't know existed easier because you're working multiple parts of the app at the same time.
-- Rules of thumb should not be taken for the whole hand.