Unit Testing... yay or nay?
-
So I got to thinking... dangerous I know. But curious to know how many peeps unit test their code. IMO _some_ arguments can be made for not doing BDD/functional testing, but unit testing is hard to say "that's a bad thing" for. I know for me, I used to loathe the concept of unit testing. It was like just as boring and tedious as documentation (that nobody ever reads). That was right up until it saved my bacon a few times. Prior to that experience, I've only ever seen devs write crappy tests that were useless and thus considered it a feel-good exercise for a green checkmark. Didn't really think about the dev just being lousy at writing tests. Still don't do TDD though, but fo sho do unit tests after development. Anyone here big into unit testing? Yay? Nay? Has cooties?
Jeremy Falcon
Started writing some Unit Testing year ago and found that with the services I develop, unit testing is futile. That said, I have my own extremely broad testing infrastructure that is constantly running JScript and PowerShell generated client requests against my servers, some of those requests intentionally contain client request errors that we've seen come from specific client types. If you are dealing with library functions and methods that have fairly simple input parameters, Unit Testing can be useful. When dealing with a client server model that takes a wide variety of complicated XML HTTP posts, from various vendors, all of whom implement specifications differently, not so much. Most of the problems would be caught somewhere else farther down the stack. That said, I have a variety of iOS clients to test with since Apple's developers excel at not following specifications, especially when it comes to return codes.
-
I always write tests for the small components in the code (aka unit tests) for two reasons: 1. 1 day of writing unit tests saves me a week of looking for bugs in the small crevices of a larger project 2. unit tests describe the behaviour of the component, so they double as documentation Also, since I have mostly worked at small companies there is usually nobody to double check my code. So testing is fundamental to avoid big mistakes.
Nelson Goncalves Oct2022 wrote:
Also, since I have mostly worked at small companies there is usually nobody to double check my code.
That's a good point. I've found some of my own silly bugs that way too.
Jeremy Falcon
-
It's a "yay" from me! However I'm a bigger fan of integration testing, whereby one can test the full functionality of a system or part of it. Not a believer in TDD.
Fo sho, both integration testing and unit testing should happen. Usually integration testing is done by QA though.
Jeremy Falcon
-
The best use of unit-testing I've seen (ie. admired, admittedly from a distance thus far) is to create a test that breaks in a meaningful way (when fixing a bug, it tickles the bug and fails ... or when adding a feature, it tries to perform the actions that are not yet implemented). Then, 'fixing the bug' or 'implementing the feature' is 'done' when your test passes. The test lingers on ... because it continues to pass, you know that your latest changes didn't take other parts of your code backward. A great example of this discipline in action is the main dev of jOOQ (Github link)[^] ... he pretty much doesn't start a bit of new code without an issue and a failing test. Unit testing should absolutely not be used for things like double-checking that code does what the complier pretty much says it will. Less is more.
DT Bullock wrote:
Unit testing should absolutely not be used for things like double-checking that code does what the complier pretty much says it will. Less is more.
Compilers can't check logic errors. Not sure if that's what you meant or not.
Jeremy Falcon
-
I Test, but I don't "TDD Unit Test". While I develop a piece of functionality, I repeatedly exercise the code I'm working on, as I write it. If at any point, it fails to compile, or shows signs of not "processing" some inputs correctly, I'll stop and fully debug everything, until it is working correctly once again. My testing can take many forms, but often, if it's a runnable app, then I'll just make sure that "the app" is runnable at all times. If it's a stand alone library, or isolated bit of functionality, then I'll often build a small command line program along side of it, that I can use to "test run" the code, allowing me to do things in my regular debug loop way. Once I'm happy the code is good, I then move up to building some test code, that integrates the system with the larger project (Should that be required), or set up some kind of testing harness (If it's a stand alone system) that exercises it using real test inputs and data. I do not, mock out things like databases, external API's and all that jazz. If I have to connect to an external API, then I connect to an external API, and if that API is not yet available, then that bit of work simply does not get started until it is. I simply will not write test code that "pretends" to be something it is not. My final step is usually one of setting up, large scale integration testing if required, or some smaller integration style unit test if code has to be independently testable. The key here, is I will create these unit tests only AFTER I'm satisfied I have done everything possible in other ways to produce good code that does the job required of it. I'll then use the integration testing, to A) ensure that the code stays working as it should with it's dependents & B) ensure that data & input changes don't screw anything up.
Peter Shaw wrote:
I Test, but I don't "TDD Unit Test".
Same. :thumbsup:
Peter Shaw wrote:
I do not, mock out things like databases, external API's and all that jazz. If I have to connect to an external API, then I connect to an external API, and if that API is not yet available, then that bit of work simply does not get started until it is. I simply will not write test code that "pretends" to be something it is not.
Technically, if you needed fake DB data that would be a fixture. But, a unit test shouldn't call a live resource. You can't do gated check-ins that way as it would take too long to run thousands of tests.
Peter Shaw wrote:
My final step is usually one of setting up, large scale integration testing if required, or some smaller integration style unit test if code has to be independently testable.
Fo sho man. It's a very important step. QA usually does that though unless it's a small team. For unit testing in particular that's all dev though.
Jeremy Falcon
-
Yes, but nothing formalized. None of this automated stuff the kids do these days. Frequently, when developing an SQL function (in SSMS), I'll add some calls to the function in a commented-out area so I can test it and remember what some of the known edge cases are. I wish there were a way to do that for C# in VS.
PIEBALDconsult wrote:
I wish there were a way to do that for C# in VS.
Doesn't VS offer some sorta doxygen style comments? That's a great idea and I do the same in Node with jsdoc style comments. For VSCode at least, it has the bonus of also showing the example uses or edge cases via intellisense too.
Jeremy Falcon
-
Jeremy you are spot on; as the learned say: the more you know, the more you realise how much you have yet to learn. And the converse, of course.
haughtonomous wrote:
And the converse, of course.
100% man. :laugh:
Jeremy Falcon
-
Jeremy Falcon wrote:
Thanks for being honest, buddy. :laugh: :laugh: :laugh: This is why we get along.
This was a bit joke and a bit truth. You can read the reality a couple of messages below in my answer to haughtonomous[^].
Jeremy Falcon wrote:
Yes!!!! :OMG:
Win+"." = 🤯
M.D.V. ;) If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about? Help me to understand what I'm saying, and I'll explain it better to you Rating helpful answers is nice, but saying thanks can be even nicer.
Nelek wrote:
You can read the reality a couple of messages below in my answer to haughtonomous[^].
:thumbsup:
Nelek wrote:
Win+"." = 🤯
Holy crap. Never knew that. 💯
Jeremy Falcon
-
That's not an argument against writing tests, it's merely pointing out that some functions need to be tested exhaustively to be completely confident in their correctness, which may be impractical.
That was exactly my point.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
Peter Shaw wrote:
I Test, but I don't "TDD Unit Test".
Same. :thumbsup:
Peter Shaw wrote:
I do not, mock out things like databases, external API's and all that jazz. If I have to connect to an external API, then I connect to an external API, and if that API is not yet available, then that bit of work simply does not get started until it is. I simply will not write test code that "pretends" to be something it is not.
Technically, if you needed fake DB data that would be a fixture. But, a unit test shouldn't call a live resource. You can't do gated check-ins that way as it would take too long to run thousands of tests.
Peter Shaw wrote:
My final step is usually one of setting up, large scale integration testing if required, or some smaller integration style unit test if code has to be independently testable.
Fo sho man. It's a very important step. QA usually does that though unless it's a small team. For unit testing in particular that's all dev though.
Jeremy Falcon
Quote:
Technically, if you needed fake DB data that would be a fixture. But, a unit test shouldn't call a live resource. You can't do gated check-ins that way as it would take too long to run thousands of tests.
This is why I always, always, always advocate a dev/stage/prod setup, esp for web applications. Dev has the "same server software", but may have data quality issues, maybe the odd broken dependency here and there, but usually nothing that the development team in general can't fix. It irritates the hell out of me, when corp/internal I.T. and the business, mandate that the same "I.T. security policy's" regarding admin access should be applied to developer only instances, as if they where prod. Staging, should always be a "clean" dev copy. Software should be as close to prod as possible, deployments should ONLY be to staging after seniors on the dev teams have verified that the code is sound, working and potentially ready for prod. Prod, well I don't need to state anything about this one :-) My point here is that, it should be perfectly acceptable to use "Live" resources, if you have a proper dev/stage/prod set-up. If data quality is a necessity, then there are ways to easy mirror a live DB to the dev & stage environments, while maintaining PII security, such as redacting information with stars as it's copied across, that way the data "format" is preserved well enough to work in testing. In many of the projects I work, I go in, and build the dev team myself, usually a very tight knit bunch who've all worked together before, and who bounce off each other very well. If it's not a large project, or a simple desktop app that one dev can handle, I'll run the entire project myself, so I don't often find myself in a situation where I have a very large team to co-ordinate with. The last time I found myself in that environment was back when I worked FT for a single corp, and as a corp I had to follow corp policy's, if they mandated TDD down to the bone, then it was TDD down to the bone. These days I much prefer the consultancy life style, where I go in, advise, build, test after it's built then move on to the next exciting project :-)
-
haughtonomous wrote:
In my experience it was always the less experienced, less diligent, over hasty developers who rebelled against it (not to mention the few who thought they were too clever for their work to need testing, too sexy for their shirt, in fact).
I have been more or less half my working life programing PLCs and Robots, you couldn't program tests like this in them, additionally every few projects (timeslot between a couple of weeks and some months) there were something new that I needed to learn quick to make the project, so I never cared to go out of scope with my time as I already had enough new staff to keep me busy / happy. That's why I got used to test in other ways, and believe me, I can be really nitpicky while testing. When I came back to high level languages, I had to learn C#, WPF and the style of my senior, plus full responsibility on the PLCs interacting with the software. I know about the different Testing Trends, but being honest, I didn't feel like needing them that much. It might give me a bad surprise somewhen? for sure. Has it until now? Not once Will I learn it after a bad situation? very probably. Am I going to learn it right now? no, I have better things to do with my time.
M.D.V. ;) If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about? Help me to understand what I'm saying, and I'll explain it better to you Rating helpful answers is nice, but saying thanks can be even nicer.
Nelek wrote:
I have been more or less half my working life programing PLCs and Robots, you couldn't program tests like this in them, additionally every few projects (timeslot between a couple of weeks and some months) there were something new that I needed to learn quick to make the project, so I never cared to go out of scope with my time as I already had enough new staff to keep me busy / happy. That's why I got used to test in other ways, and believe me, I can be really nitpicky while testing.
Keep in mind, I don't know PLC programming, so there's a chance I'm talking out of my arse here... but would it be possible to at least treat the parts that interact with the hardware like a UI layer of sorts? If so, you can still test a logic layer irrespective of it's IO, etc.
Nelek wrote:
I didn't feel like needing them that much.
Can't blame ya man. I felt that way too for years. Didn't realize at the time though it's only because I've ever seen lousy tests written prior to that. It's more the fault of the dev though IMO. Like lousy tests are a waste of time that accomplish nothing. Good ones.. are... well... good. :laugh:
Nelek wrote:
Will I learn it after a bad situation? very probably.
That's what it took for me. :laugh: :laugh: :laugh:
Jeremy Falcon
-
Quote:
Technically, if you needed fake DB data that would be a fixture. But, a unit test shouldn't call a live resource. You can't do gated check-ins that way as it would take too long to run thousands of tests.
This is why I always, always, always advocate a dev/stage/prod setup, esp for web applications. Dev has the "same server software", but may have data quality issues, maybe the odd broken dependency here and there, but usually nothing that the development team in general can't fix. It irritates the hell out of me, when corp/internal I.T. and the business, mandate that the same "I.T. security policy's" regarding admin access should be applied to developer only instances, as if they where prod. Staging, should always be a "clean" dev copy. Software should be as close to prod as possible, deployments should ONLY be to staging after seniors on the dev teams have verified that the code is sound, working and potentially ready for prod. Prod, well I don't need to state anything about this one :-) My point here is that, it should be perfectly acceptable to use "Live" resources, if you have a proper dev/stage/prod set-up. If data quality is a necessity, then there are ways to easy mirror a live DB to the dev & stage environments, while maintaining PII security, such as redacting information with stars as it's copied across, that way the data "format" is preserved well enough to work in testing. In many of the projects I work, I go in, and build the dev team myself, usually a very tight knit bunch who've all worked together before, and who bounce off each other very well. If it's not a large project, or a simple desktop app that one dev can handle, I'll run the entire project myself, so I don't often find myself in a situation where I have a very large team to co-ordinate with. The last time I found myself in that environment was back when I worked FT for a single corp, and as a corp I had to follow corp policy's, if they mandated TDD down to the bone, then it was TDD down to the bone. These days I much prefer the consultancy life style, where I go in, advise, build, test after it's built then move on to the next exciting project :-)
Peter Shaw wrote:
This is why I always, always, always advocate a dev/stage/prod setup, esp for web applications.
Fo sho man. Totally agree on the 4 environments that should be set up. You can get away with 3 if you're a solo dev in the company, but otherwise 4. My point was more about calling a live resource for a unit test makes them no longer pure or deterministic and very slow to run. By live that could be a dev environment as well, as in an actual API call.
Peter Shaw wrote:
Prod, well I don't need to state anything about this one
:laugh:
Peter Shaw wrote:
In many of the projects I work, I go in, and build the dev team myself, usually a very tight knit bunch who've all worked together before, and who bounce off each other very well.
It's so hard to find that too. Real hard. But when you have that camaraderie it's gold. Usually it seems everyone is unhappy and hates life and has an agenda rather than the love of tech.
Peter Shaw wrote:
These days I much prefer the consultancy life style, where I go in, advise, build, test after it's built then move on to the next exciting project
IMO a lot can be learned from that. Like, if you have a team that refuses to modernize, you're stuck in one spot. Also a lot can be learned from sticking with a project for years, usually about supporting it, but a lot can be learned. I choose the former too though, if given a choice. I wouldn't want to be beholden to people who stopped learning and are content with that.
Jeremy Falcon
-
So I got to thinking... dangerous I know. But curious to know how many peeps unit test their code. IMO _some_ arguments can be made for not doing BDD/functional testing, but unit testing is hard to say "that's a bad thing" for. I know for me, I used to loathe the concept of unit testing. It was like just as boring and tedious as documentation (that nobody ever reads). That was right up until it saved my bacon a few times. Prior to that experience, I've only ever seen devs write crappy tests that were useless and thus considered it a feel-good exercise for a green checkmark. Didn't really think about the dev just being lousy at writing tests. Still don't do TDD though, but fo sho do unit tests after development. Anyone here big into unit testing? Yay? Nay? Has cooties?
Jeremy Falcon
I really like using unit tests especially when I am working on some new algorithms. I can test instantly without firing up the GUI and entering the data manually. Helps me find the inverted logic and poorly managed edge cases (hey, sometimes I rush it a bit when I get excited!) Other people's unit tests have saved my bacon many times. "Well, this is an obvious bug that needs fixing" followed by failed unit tests has led me to learn a lot more about some seemingly innocuous code. I usually add comments so future devs don't make the same mistake, btw.
-
PIEBALDconsult wrote:
I wish there were a way to do that for C# in VS.
Doesn't VS offer some sorta doxygen style comments? That's a great idea and I do the same in Node with jsdoc style comments. For VSCode at least, it has the bonus of also showing the example uses or edge cases via intellisense too.
Jeremy Falcon
I think it's things which get executed at compile time, but I would want to have the ability to execute ad-hoc tests whenever I like.
-
So I got to thinking... dangerous I know. But curious to know how many peeps unit test their code. IMO _some_ arguments can be made for not doing BDD/functional testing, but unit testing is hard to say "that's a bad thing" for. I know for me, I used to loathe the concept of unit testing. It was like just as boring and tedious as documentation (that nobody ever reads). That was right up until it saved my bacon a few times. Prior to that experience, I've only ever seen devs write crappy tests that were useless and thus considered it a feel-good exercise for a green checkmark. Didn't really think about the dev just being lousy at writing tests. Still don't do TDD though, but fo sho do unit tests after development. Anyone here big into unit testing? Yay? Nay? Has cooties?
Jeremy Falcon
Unit testing: very much yay. I've been doing unit testing steadily since about 1999. I have my own simple unit test driver. Tests are static member functions. It can all be statically linked with an executable. No tests enabled equals no overhead in size or time. The two places I've worked that did unit testing also had the highest code quality of the places I've worked. I've used a couple of open source frameworks for unit tests, but they seemed unnecessarily complicated to me, and it's annoying to have to separately compile test executables. Writing unit tests helps me wring out my designs and of course avoid regressions when I change things (which happens all the time).
-
Nelek wrote:
I have been more or less half my working life programing PLCs and Robots, you couldn't program tests like this in them, additionally every few projects (timeslot between a couple of weeks and some months) there were something new that I needed to learn quick to make the project, so I never cared to go out of scope with my time as I already had enough new staff to keep me busy / happy. That's why I got used to test in other ways, and believe me, I can be really nitpicky while testing.
Keep in mind, I don't know PLC programming, so there's a chance I'm talking out of my arse here... but would it be possible to at least treat the parts that interact with the hardware like a UI layer of sorts? If so, you can still test a logic layer irrespective of it's IO, etc.
Nelek wrote:
I didn't feel like needing them that much.
Can't blame ya man. I felt that way too for years. Didn't realize at the time though it's only because I've ever seen lousy tests written prior to that. It's more the fault of the dev though IMO. Like lousy tests are a waste of time that accomplish nothing. Good ones.. are... well... good. :laugh:
Nelek wrote:
Will I learn it after a bad situation? very probably.
That's what it took for me. :laugh: :laugh: :laugh:
Jeremy Falcon
In PLC Programming you have a limited set of instructions, some more than in assembly. And you almost speak directly to the hardware. For instance:
If (a == true)
{
return;
}if ((b == true) && (c==false))
{
x = true;
}In PLC that would be: ("i" for input register and "o" for 24 V / 0.5 A output register)
a = i0.0
b = i0.1
c = i0.2
x = o0.0A a
BEBA b
AN c
= xBelieve me when I say, you can't test like in high level. Checking out boundaries or plausibility is something I have done too though, but the logic behind is pretty diffrerent as in Unit testing.
M.D.V. ;) If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about? Help me to understand what I'm saying, and I'll explain it better to you Rating helpful answers is nice, but saying thanks can be even nicer.
-
Nelek wrote:
You can read the reality a couple of messages below in my answer to haughtonomous[^].
:thumbsup:
Nelek wrote:
Win+"." = 🤯
Holy crap. Never knew that. 💯
Jeremy Falcon
Jeremy Falcon wrote:
Holy crap. Never knew that. 💯
The only wasted day is the day where you learn nothing new ;)
M.D.V. ;) If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about? Help me to understand what I'm saying, and I'll explain it better to you Rating helpful answers is nice, but saying thanks can be even nicer.
-
Unit testing: very much yay. I've been doing unit testing steadily since about 1999. I have my own simple unit test driver. Tests are static member functions. It can all be statically linked with an executable. No tests enabled equals no overhead in size or time. The two places I've worked that did unit testing also had the highest code quality of the places I've worked. I've used a couple of open source frameworks for unit tests, but they seemed unnecessarily complicated to me, and it's annoying to have to separately compile test executables. Writing unit tests helps me wring out my designs and of course avoid regressions when I change things (which happens all the time).
SeattleC++ wrote:
I've been doing unit testing steadily since about 1999.
Noice. For me it's only been a few years, but the more I do it and the better I get at it, the less of a chance of ever turning back ya know.
SeattleC++ wrote:
and it's annoying to have to separately compile test executables
Oh yeah, that is one one of the caveats I faced in a C project once. The way I handled it was to have my overall build command just compile both. Probably harder to get away with that for C++, so cool idea.
Jeremy Falcon
-
In PLC Programming you have a limited set of instructions, some more than in assembly. And you almost speak directly to the hardware. For instance:
If (a == true)
{
return;
}if ((b == true) && (c==false))
{
x = true;
}In PLC that would be: ("i" for input register and "o" for 24 V / 0.5 A output register)
a = i0.0
b = i0.1
c = i0.2
x = o0.0A a
BEBA b
AN c
= xBelieve me when I say, you can't test like in high level. Checking out boundaries or plausibility is something I have done too though, but the logic behind is pretty diffrerent as in Unit testing.
M.D.V. ;) If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about? Help me to understand what I'm saying, and I'll explain it better to you Rating helpful answers is nice, but saying thanks can be even nicer.
Nelek wrote:
Believe me when I say, you can't test like in high level.
I believe ya man. On the upside, you get cool points for going low level.
Nelek wrote:
Checking out boundaries or plausibility is something I have done too though
Noice
Jeremy Falcon
-
DT Bullock wrote:
Unit testing should absolutely not be used for things like double-checking that code does what the complier pretty much says it will. Less is more.
Compilers can't check logic errors. Not sure if that's what you meant or not.
Jeremy Falcon
OK, I was a little vague about that. I've seen people write tests that exercise getters/setters, behaviour from missing arguments, etc. In Java at least, a few good annotations takes care of all that rigmarole and you don't need to write tests for that stuff. But let's talk about tests which 'confirm expected behaviour'. I feel like this kind of test is a waste of time until we've encountered a non-expected behaviour that we want to squash and know that it stays squashed. Because 'the expected behaviour' is already a path we have trodden while developing/debugging, and obviously we wouldn't think we're done until it's behaving as expected already. But our oversights are the things we need to come back for and scaffold with some tests, because we're prone to overlooking some aspects of the state-space and need that support. It's about benefit vs bother in the end. You have to cherry-pick your testing opportunities and get on with making the code. IMHO.