A lesson in bug testing.
-
So..... I've got an Android app going live on Google Play soon. I've been testing this app six ways to Sunday, trying to find any and all bugs before release. I have hammered this app, beat it up in every way possible....test, bug fixes, test bug fixes ... for weeks! Saturday night, my 5 year old daughter and I went to have pizza, mommy was working. I was showing the app to her and telling her how proud I am of the app, how it took a lot of work, how I've bug tested it exhaustively for weeks and that I think it's finally bug free and ready for release. My daughter asks "daddy can I play with it?" I said sure and explained how to use it. She has no idea what the app does, it's just numbers and bright colors in her eyes. She loved it! As I chomped on my pizza and watched her, in the back of my mind, a thought ...... hmmm ... a 5 year old randomly pushing buttons could be a good QA test. Quickly that thought faded, there was more pizza to eat! So, I chomped some more pizza and listened to her giggle. A couple of minutes later, I look down and she's got fingers on both entry buttons...... You ever get that "cosmic swirl" feeling? Like time and space is swirling in a vortex around you at a significant moment in time? That sensation, is usually coupled with the feeling, like you are in a dream, falling from the sky...... Right then, the planets changed their orbits, space and time started to swirl around us. I feel like I'm in a dream falling out of the sky..... She looks up at me and says.... "Daddy.....what happens if I press BOTH buttons at the same time?" I could leave the story at that. You programmers know what happened and have probably snorted coffee out your nose, laughing. But for the rest of the story.....yep, my 5 year old daughter, while eating pizza, found her first bug. A bug that daddy, despite herculean effort, missed after weeks and weeks of testing! The future is bright for this one!
That's basically the old adage of "devs shall not test themselves and call it a day". As someone who's built that thing, certain assumptions on what it's supposed to do and how are thoroughly burned into our synapses rendering us impossible to see beyond.
-
So..... I've got an Android app going live on Google Play soon. I've been testing this app six ways to Sunday, trying to find any and all bugs before release. I have hammered this app, beat it up in every way possible....test, bug fixes, test bug fixes ... for weeks! Saturday night, my 5 year old daughter and I went to have pizza, mommy was working. I was showing the app to her and telling her how proud I am of the app, how it took a lot of work, how I've bug tested it exhaustively for weeks and that I think it's finally bug free and ready for release. My daughter asks "daddy can I play with it?" I said sure and explained how to use it. She has no idea what the app does, it's just numbers and bright colors in her eyes. She loved it! As I chomped on my pizza and watched her, in the back of my mind, a thought ...... hmmm ... a 5 year old randomly pushing buttons could be a good QA test. Quickly that thought faded, there was more pizza to eat! So, I chomped some more pizza and listened to her giggle. A couple of minutes later, I look down and she's got fingers on both entry buttons...... You ever get that "cosmic swirl" feeling? Like time and space is swirling in a vortex around you at a significant moment in time? That sensation, is usually coupled with the feeling, like you are in a dream, falling from the sky...... Right then, the planets changed their orbits, space and time started to swirl around us. I feel like I'm in a dream falling out of the sky..... She looks up at me and says.... "Daddy.....what happens if I press BOTH buttons at the same time?" I could leave the story at that. You programmers know what happened and have probably snorted coffee out your nose, laughing. But for the rest of the story.....yep, my 5 year old daughter, while eating pizza, found her first bug. A bug that daddy, despite herculean effort, missed after weeks and weeks of testing! The future is bright for this one!
@HappyDotNet I'll give you a better test that Lord Murphy always gives me. First block some time with someone really high up that you are possibly trying to impress. Then go on your routine demo. Then ask him to use it in front of you. Don't forget to record the screen if its possible to do so. Nine times out of ten, you will get a big glaring bug, thrown in your face. It will happen for each thoroughly tested application that is not a POC or low ticket sort of development effort. And that is the lesson, no matter what you do, Lord Murphy always gets the last laugh. -Paras
-
So..... I've got an Android app going live on Google Play soon. I've been testing this app six ways to Sunday, trying to find any and all bugs before release. I have hammered this app, beat it up in every way possible....test, bug fixes, test bug fixes ... for weeks! Saturday night, my 5 year old daughter and I went to have pizza, mommy was working. I was showing the app to her and telling her how proud I am of the app, how it took a lot of work, how I've bug tested it exhaustively for weeks and that I think it's finally bug free and ready for release. My daughter asks "daddy can I play with it?" I said sure and explained how to use it. She has no idea what the app does, it's just numbers and bright colors in her eyes. She loved it! As I chomped on my pizza and watched her, in the back of my mind, a thought ...... hmmm ... a 5 year old randomly pushing buttons could be a good QA test. Quickly that thought faded, there was more pizza to eat! So, I chomped some more pizza and listened to her giggle. A couple of minutes later, I look down and she's got fingers on both entry buttons...... You ever get that "cosmic swirl" feeling? Like time and space is swirling in a vortex around you at a significant moment in time? That sensation, is usually coupled with the feeling, like you are in a dream, falling from the sky...... Right then, the planets changed their orbits, space and time started to swirl around us. I feel like I'm in a dream falling out of the sky..... She looks up at me and says.... "Daddy.....what happens if I press BOTH buttons at the same time?" I could leave the story at that. You programmers know what happened and have probably snorted coffee out your nose, laughing. But for the rest of the story.....yep, my 5 year old daughter, while eating pizza, found her first bug. A bug that daddy, despite herculean effort, missed after weeks and weeks of testing! The future is bright for this one!
I used to develop some apps for the PalmOS, remember those? You could run an emulator on the desktop, and there was a auto-test feature that would literally just press the emulated screen randomly for as long as you like. I think it was called something like Monkey testing or something like that. Leave it running overnight and check in on it in the morning!
-
So..... I've got an Android app going live on Google Play soon. I've been testing this app six ways to Sunday, trying to find any and all bugs before release. I have hammered this app, beat it up in every way possible....test, bug fixes, test bug fixes ... for weeks! Saturday night, my 5 year old daughter and I went to have pizza, mommy was working. I was showing the app to her and telling her how proud I am of the app, how it took a lot of work, how I've bug tested it exhaustively for weeks and that I think it's finally bug free and ready for release. My daughter asks "daddy can I play with it?" I said sure and explained how to use it. She has no idea what the app does, it's just numbers and bright colors in her eyes. She loved it! As I chomped on my pizza and watched her, in the back of my mind, a thought ...... hmmm ... a 5 year old randomly pushing buttons could be a good QA test. Quickly that thought faded, there was more pizza to eat! So, I chomped some more pizza and listened to her giggle. A couple of minutes later, I look down and she's got fingers on both entry buttons...... You ever get that "cosmic swirl" feeling? Like time and space is swirling in a vortex around you at a significant moment in time? That sensation, is usually coupled with the feeling, like you are in a dream, falling from the sky...... Right then, the planets changed their orbits, space and time started to swirl around us. I feel like I'm in a dream falling out of the sky..... She looks up at me and says.... "Daddy.....what happens if I press BOTH buttons at the same time?" I could leave the story at that. You programmers know what happened and have probably snorted coffee out your nose, laughing. But for the rest of the story.....yep, my 5 year old daughter, while eating pizza, found her first bug. A bug that daddy, despite herculean effort, missed after weeks and weeks of testing! The future is bright for this one!
I lead a small dev team. We had a old mainframe coder here years back, refused to learn anything new (it's a govt job...), so he had zero work. I enlisted him to test our applications before fielding them. Being completely out of the loop on things, he was a valuable tester. He'd come back with things like "Well, if I press shift, F6, K, and Enter, this happens" "Why would you do that?" "I dunno, but this happens" Found things we'd never test for. Annoying, but hey, it happens.
-
So..... I've got an Android app going live on Google Play soon. I've been testing this app six ways to Sunday, trying to find any and all bugs before release. I have hammered this app, beat it up in every way possible....test, bug fixes, test bug fixes ... for weeks! Saturday night, my 5 year old daughter and I went to have pizza, mommy was working. I was showing the app to her and telling her how proud I am of the app, how it took a lot of work, how I've bug tested it exhaustively for weeks and that I think it's finally bug free and ready for release. My daughter asks "daddy can I play with it?" I said sure and explained how to use it. She has no idea what the app does, it's just numbers and bright colors in her eyes. She loved it! As I chomped on my pizza and watched her, in the back of my mind, a thought ...... hmmm ... a 5 year old randomly pushing buttons could be a good QA test. Quickly that thought faded, there was more pizza to eat! So, I chomped some more pizza and listened to her giggle. A couple of minutes later, I look down and she's got fingers on both entry buttons...... You ever get that "cosmic swirl" feeling? Like time and space is swirling in a vortex around you at a significant moment in time? That sensation, is usually coupled with the feeling, like you are in a dream, falling from the sky...... Right then, the planets changed their orbits, space and time started to swirl around us. I feel like I'm in a dream falling out of the sky..... She looks up at me and says.... "Daddy.....what happens if I press BOTH buttons at the same time?" I could leave the story at that. You programmers know what happened and have probably snorted coffee out your nose, laughing. But for the rest of the story.....yep, my 5 year old daughter, while eating pizza, found her first bug. A bug that daddy, despite herculean effort, missed after weeks and weeks of testing! The future is bright for this one!
Your 5 year-old deserves some sort of treat for teaching you something new about QA. If she were (much) older, I'd say that she deserves a :beer: :)
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
So..... I've got an Android app going live on Google Play soon. I've been testing this app six ways to Sunday, trying to find any and all bugs before release. I have hammered this app, beat it up in every way possible....test, bug fixes, test bug fixes ... for weeks! Saturday night, my 5 year old daughter and I went to have pizza, mommy was working. I was showing the app to her and telling her how proud I am of the app, how it took a lot of work, how I've bug tested it exhaustively for weeks and that I think it's finally bug free and ready for release. My daughter asks "daddy can I play with it?" I said sure and explained how to use it. She has no idea what the app does, it's just numbers and bright colors in her eyes. She loved it! As I chomped on my pizza and watched her, in the back of my mind, a thought ...... hmmm ... a 5 year old randomly pushing buttons could be a good QA test. Quickly that thought faded, there was more pizza to eat! So, I chomped some more pizza and listened to her giggle. A couple of minutes later, I look down and she's got fingers on both entry buttons...... You ever get that "cosmic swirl" feeling? Like time and space is swirling in a vortex around you at a significant moment in time? That sensation, is usually coupled with the feeling, like you are in a dream, falling from the sky...... Right then, the planets changed their orbits, space and time started to swirl around us. I feel like I'm in a dream falling out of the sky..... She looks up at me and says.... "Daddy.....what happens if I press BOTH buttons at the same time?" I could leave the story at that. You programmers know what happened and have probably snorted coffee out your nose, laughing. But for the rest of the story.....yep, my 5 year old daughter, while eating pizza, found her first bug. A bug that daddy, despite herculean effort, missed after weeks and weeks of testing! The future is bright for this one!
This reminds me: We had a user, that despite taking the online training for our software, could not get various features to work. (All of them involved F-Keys, in hindsight, but she referred to them by their names, like Refresh, etc). On a visit, I stopped by her desk, and I asked her to show me. When she was required to press F3 to search, she was PRESSING: F, and then 3 I was ASTOUNDED, until I realized she had never pulled her keyboard drawer fully open. I pulled it open for her, and she excitedly exclaimed "Oh, do you think THEY meant that F3 key?" I said "I don't know! Try it!", and, of course, it worked... The manager overheard this, and as I walked to the other part of the office, TRYING TO PREVENT MY SKULL From exploding... He said "I know you will tell this story in the future, just promise me to NEVER mention the company!"... ROTFLMAO. Users... They come in all varieties. And it's why engineers/testers fail to find the problems.
-
HappyDotNet wrote:
a 5 year old randomly pushing buttons could be a good QA test.
Many users probably have less brain than your 5 years daughter, so yes, she might be a really important QA Tester. Back then, my first day in new company, I went into the office and a guy was staring at the roof, just randomly moving his fingers through a touch panel of Siemens... when I looked at him with an "what the hell is he doing?" expression in my face, my mentor said: "He is simulating a bored worker" And it is like this, we unconsciously do things in more or less the way they are designed, we don't think about the order of small processes because for us they are "extremely logical" and mostly "self explanatory" but there are people that will play with the GUI with malice just due to boredom or plain stupidity. Fast forward 3 years in my career, I inherited an industrial line automation at what was going to be my main customer. 4 years later and a lot of months improving the processes and adding new stuff... I knew almost everything by heart and could solve almost everything just on the phone, without having to look at the code. I got a call, line was blocked. Aha... Do this, nothing. Do that, nothing. Mmmmhhh... I'll be right there. Once at the factory, I try this... nothing. I try that... nothing. Mmmmhhhh... I better have a look to the code. I pack my laptop out, nothing. I connect my laptop for online diagnose... 1 hour later... bug found. How the hell did this happen? I had no explanation. I had to reset the PLC, overwrite some blocks with my fixed versions and manually change the value of some parameters, then I had to move back the robot to home manually and do a restart of some slaves... The girl / woman operating the line that day was a new one and it was her second day at that position. 2 years without any issue that couldn't be solve with a simple "back to home position" button click and she had found a bug that blocked the line beyond any recoverable point in less than 2 work-shifts. :doh: :doh: X| X|
M.D.V. ;) If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about? Help me to understand what I'm saying, and I'll explain it better to you Rating helpful answers is nice, but saying thanks can be even nicer.
-
So..... I've got an Android app going live on Google Play soon. I've been testing this app six ways to Sunday, trying to find any and all bugs before release. I have hammered this app, beat it up in every way possible....test, bug fixes, test bug fixes ... for weeks! Saturday night, my 5 year old daughter and I went to have pizza, mommy was working. I was showing the app to her and telling her how proud I am of the app, how it took a lot of work, how I've bug tested it exhaustively for weeks and that I think it's finally bug free and ready for release. My daughter asks "daddy can I play with it?" I said sure and explained how to use it. She has no idea what the app does, it's just numbers and bright colors in her eyes. She loved it! As I chomped on my pizza and watched her, in the back of my mind, a thought ...... hmmm ... a 5 year old randomly pushing buttons could be a good QA test. Quickly that thought faded, there was more pizza to eat! So, I chomped some more pizza and listened to her giggle. A couple of minutes later, I look down and she's got fingers on both entry buttons...... You ever get that "cosmic swirl" feeling? Like time and space is swirling in a vortex around you at a significant moment in time? That sensation, is usually coupled with the feeling, like you are in a dream, falling from the sky...... Right then, the planets changed their orbits, space and time started to swirl around us. I feel like I'm in a dream falling out of the sky..... She looks up at me and says.... "Daddy.....what happens if I press BOTH buttons at the same time?" I could leave the story at that. You programmers know what happened and have probably snorted coffee out your nose, laughing. But for the rest of the story.....yep, my 5 year old daughter, while eating pizza, found her first bug. A bug that daddy, despite herculean effort, missed after weeks and weeks of testing! The future is bright for this one!
When my daughter was about the same age (2 decades ago), she inserted a linux install disc into our windows computer and by random key presses, not only installed linux, but forced a login for either partition with a root password of God only knows. Sigh. After that we put a screenlock on with the failed password alert saying aloud, "No, no, Katie!" LOL
-
Kids and Cats right? LOL Best QA testers!
-
"The 5yo test", or 'If you can make the program stop, and show daddy how you did it, you'll have an ice cream cone' has been well known among my co-workers for at least twenty years - but mostly as a joke. It is great to see that someone is actually using the method in real life!
Hi, when i was designing security equipment in the 80-90s (assembler, naturally). I would give a working prototype to a salesman to take home for his kids to play with. It quite often came back with a verbal bug report. (he would ask them how they made it do THAT and reproduce it himself) If his kids could not get it to I knew I was right for initial production If you cannot find any bugs, give it to a kid. They will find one. Fun.
-
So..... I've got an Android app going live on Google Play soon. I've been testing this app six ways to Sunday, trying to find any and all bugs before release. I have hammered this app, beat it up in every way possible....test, bug fixes, test bug fixes ... for weeks! Saturday night, my 5 year old daughter and I went to have pizza, mommy was working. I was showing the app to her and telling her how proud I am of the app, how it took a lot of work, how I've bug tested it exhaustively for weeks and that I think it's finally bug free and ready for release. My daughter asks "daddy can I play with it?" I said sure and explained how to use it. She has no idea what the app does, it's just numbers and bright colors in her eyes. She loved it! As I chomped on my pizza and watched her, in the back of my mind, a thought ...... hmmm ... a 5 year old randomly pushing buttons could be a good QA test. Quickly that thought faded, there was more pizza to eat! So, I chomped some more pizza and listened to her giggle. A couple of minutes later, I look down and she's got fingers on both entry buttons...... You ever get that "cosmic swirl" feeling? Like time and space is swirling in a vortex around you at a significant moment in time? That sensation, is usually coupled with the feeling, like you are in a dream, falling from the sky...... Right then, the planets changed their orbits, space and time started to swirl around us. I feel like I'm in a dream falling out of the sky..... She looks up at me and says.... "Daddy.....what happens if I press BOTH buttons at the same time?" I could leave the story at that. You programmers know what happened and have probably snorted coffee out your nose, laughing. But for the rest of the story.....yep, my 5 year old daughter, while eating pizza, found her first bug. A bug that daddy, despite herculean effort, missed after weeks and weeks of testing! The future is bright for this one!
It reminds me of the story of Margaret Hamilton letting her daughter play with the Apollo simulations : Margaret Hamilton on Her Daughter's Simulation | Hack the Moon[^] Kids are awesome :-D
-
It reminds me of the story of Margaret Hamilton letting her daughter play with the Apollo simulations : Margaret Hamilton on Her Daughter's Simulation | Hack the Moon[^] Kids are awesome :-D