Automated testing...
-
dan neely wrote:
what you're describing is functional testing, not unit testing.
True, but at least he is testing!!! :-D
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
what you're describing is functional testing, not unit testing. -- Rules of thumb should not be taken for the whole hand.
dan neely wrote:
what you're describing is functional testing, not unit testing.
That's an interesting point. When does unit testing cross over into functional testing? Or does it ever? In other words, if I test anything but my lowest level classes and methods, does that mean I'm now doing functional testing? Marc
People are just notoriously impossible. --DavidCrow
There's NO excuse for not commenting your code. -- John Simmons / outlaw programmer
People who say that they will refactor their code later to make it "good" don't understand refactoring, nor the art and craft of programming. -- Josh Smith -
Joergen Sigvardsson wrote:
The app (a POS application)...
This is unrelated (as most of my posts are) but around these parts, POS stands for "piece of sh*t" so whenever I see that I have to reread it! :laugh: Cheers! Mark
Mark Salsbery wrote:
but around these parts, POS stands for "piece of sh*t"
yup, in the business community it refers to Point-Of-Sale, which is a generalized term for capturing information at a remote sale location (gas station for instance, I was in the POS gasoline sales before transferring to Engineering with another company). Ironically, either definition works with POS even in the business community. ;) which was why I jumped ship and went to engineering instead of staying in accounting.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
Jeffry J. Brickley wrote:
True, but at least he is testing!!!
Which is why I ed him, instead ofing. -- Rules of thumb should not be taken for the whole hand.
dan neely wrote:
Which is why I ed him, instead ofing.
true... but the worst thing about nit picking... is you end up with a handfull of nits.... :rolleyes:
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
Mark Salsbery wrote:
but around these parts, POS stands for "piece of sh*t"
yup, in the business community it refers to Point-Of-Sale, which is a generalized term for capturing information at a remote sale location (gas station for instance, I was in the POS gasoline sales before transferring to Engineering with another company). Ironically, either definition works with POS even in the business community. ;) which was why I jumped ship and went to engineering instead of staying in accounting.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
Jeffry J. Brickley wrote:
Ironically, either definition works with POS even in the business community.
:laugh:
-
I've been thinking "Ah, that's nice. In theory!" about software unit testing. Until now. :) I've implemented a VBScript scripting host in my app, which allows me to automate input to the app. The app (a POS application) does a series of operations based on the input, and outputs information to database (not counting receipts, and other types of "hard copy" output, which I unfortunately have to test by hand). I predict the output by calculating the output in beforehand in my VBScripts. I then compare the output in the database with my predictions. If there are any deviations from my predictions, I sound the alarm. I've just completed a couple of thousands of tests - a task which would normally have taken me many weeks to do by hand. Admittedly, this is not very fine grained module testing. It's basically seeing the entire app as a black box. But it's better than nothing, and I have already nailed two "once in a blue moon" bugs. Other than that, I think I'm very close to a release. :cool: ps. It's also a great way to stress test your app. After 2000 transactions - no memory leak. Take that, you janitor dependent programmers! :rolleyes: ds.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
We've used automated testing for various specific situations (for example, determining the worst case execution times of function - this was with real-time embedded systems, where worst case execution times are important). But this[^] is my favourite automatic testing system (albeit for a non-mainstream language). Define properties you believe should hold true, and they'll be tested automatically.
-
I've been thinking "Ah, that's nice. In theory!" about software unit testing. Until now. :) I've implemented a VBScript scripting host in my app, which allows me to automate input to the app. The app (a POS application) does a series of operations based on the input, and outputs information to database (not counting receipts, and other types of "hard copy" output, which I unfortunately have to test by hand). I predict the output by calculating the output in beforehand in my VBScripts. I then compare the output in the database with my predictions. If there are any deviations from my predictions, I sound the alarm. I've just completed a couple of thousands of tests - a task which would normally have taken me many weeks to do by hand. Admittedly, this is not very fine grained module testing. It's basically seeing the entire app as a black box. But it's better than nothing, and I have already nailed two "once in a blue moon" bugs. Other than that, I think I'm very close to a release. :cool: ps. It's also a great way to stress test your app. After 2000 transactions - no memory leak. Take that, you janitor dependent programmers! :rolleyes: ds.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
Over the weekend, i set up NUnit and CruiseControl.NET to allow periodic, automated building and testing of a certain library that has, unfortunately, grown far too large to test by hand (not, i suspect, that it was ever properly tested even when it was small and simple). At present, i'm doing pretty much the same as you, treating the library as a black box. As time permits (and future bugs demand), i hope to add more fine-grained tests into the mix. My only regret is that i didn't set this up a year ago. It would have saved a lot of grief (and manual testing on my part). I've been thinking for a while now about building in a scripting engine for use in testing a larger, unmanaged app. The only thing really holding me back is the time needed to implement an Automation interface that properly captures the functionality of the application... These things would be sooo much easier up-front. :sigh: (Marc, you around? See, i don't listen to your advice either...)
---- I just want you to be happy; That's my only little wish...
-
Over the weekend, i set up NUnit and CruiseControl.NET to allow periodic, automated building and testing of a certain library that has, unfortunately, grown far too large to test by hand (not, i suspect, that it was ever properly tested even when it was small and simple). At present, i'm doing pretty much the same as you, treating the library as a black box. As time permits (and future bugs demand), i hope to add more fine-grained tests into the mix. My only regret is that i didn't set this up a year ago. It would have saved a lot of grief (and manual testing on my part). I've been thinking for a while now about building in a scripting engine for use in testing a larger, unmanaged app. The only thing really holding me back is the time needed to implement an Automation interface that properly captures the functionality of the application... These things would be sooo much easier up-front. :sigh: (Marc, you around? See, i don't listen to your advice either...)
---- I just want you to be happy; That's my only little wish...
Shog9 wrote:
The only thing really holding me back is the time needed to implement an Automation interface that properly captures the functionality of the application... These things would be sooo much easier up-front.
:nod: It took me some swearing, shedding of blood, sweating and god awful hacks to get away with the automation (it's near Daily WTF class). Last time I reimplemented an app framework for the application (I do dumb stuff like that once in a while, hoping that it'll ignite a total rewrite of the released app), I did it properly. All operations are executed using a generalized command abstraction. Not only would it have been easy to implement stuff like redo/undo, it would've made automation sooo much easier. One day...
-- No humans were probed in the making of this episode
-
So, unit testing doesn't test for correct functionality..? :~ I honestly never understood the difference between functional testing and unit testing, other than that unit testing often means testing very small pieces of code, while functional testing means testing a larger pieces of code.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
No it's more a matter of scale. In Unit testing you're checking that MyClass.DoFoo() MyClass.DoBar() does Foo and Bar to MyClass in every possible case including all the boundary conditions. It's the bottom up approach. Functional testing is topdown, you're looking at the overall behavior, so when you test the File->Baz menu item you're only worrying about the final result, not the details of the implementation (ie calls MyClass.DoFoo(), MyClass.DoBar() and MyOtherClass.DoQux()). In theory a good functional test suite should cover all the same areas that the unit tests do. The biggest differences is that unit tests make isolating a specific problem easiest (ie the problem is MyClass.DoBar(), not something called by File->Baz), while functional testing's big picture makes catching boundaries/interations you didn't know existed easier because you're working multiple parts of the app at the same time.
-- Rules of thumb should not be taken for the whole hand.
-
I've been thinking "Ah, that's nice. In theory!" about software unit testing. Until now. :) I've implemented a VBScript scripting host in my app, which allows me to automate input to the app. The app (a POS application) does a series of operations based on the input, and outputs information to database (not counting receipts, and other types of "hard copy" output, which I unfortunately have to test by hand). I predict the output by calculating the output in beforehand in my VBScripts. I then compare the output in the database with my predictions. If there are any deviations from my predictions, I sound the alarm. I've just completed a couple of thousands of tests - a task which would normally have taken me many weeks to do by hand. Admittedly, this is not very fine grained module testing. It's basically seeing the entire app as a black box. But it's better than nothing, and I have already nailed two "once in a blue moon" bugs. Other than that, I think I'm very close to a release. :cool: ps. It's also a great way to stress test your app. After 2000 transactions - no memory leak. Take that, you janitor dependent programmers! :rolleyes: ds.
-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören
We use SilkTest and it runs thousands of automated tests against our software.