Strange Nunit test failure result in C# projects.
-
Hi I couldn't find any TESTING forum so using this to post. I am using Nunit GUI test runner (version 2.4.5) for running almost 1000 tests written using C#. It runs completely and give me results in the end. Now, there are some failed tests which if I run them in Visual studio directly, they work fine. And GUI log window doesn't say much about why tests failed. Why could these tests be failing? Is there a way to find out? More logging, maybe. But what should I be logging? Please advise. Thanks Pankaj
-
Hi I couldn't find any TESTING forum so using this to post. I am using Nunit GUI test runner (version 2.4.5) for running almost 1000 tests written using C#. It runs completely and give me results in the end. Now, there are some failed tests which if I run them in Visual studio directly, they work fine. And GUI log window doesn't say much about why tests failed. Why could these tests be failing? Is there a way to find out? More logging, maybe. But what should I be logging? Please advise. Thanks Pankaj
-
too fast!
Panic, Chaos, Destruction. My work here is done.
-
too fast!
Panic, Chaos, Destruction. My work here is done.
yeah, strange that... you see I was just writing a template reply in case anyone was posting here as they couldn't find a testing forum then just as I wrote it <BANG!> someone has that very problem... Spooky :laugh:
Life goes very fast. Tomorrow, today is already yesterday.
-
Hi I couldn't find any TESTING forum so using this to post. I am using Nunit GUI test runner (version 2.4.5) for running almost 1000 tests written using C#. It runs completely and give me results in the end. Now, there are some failed tests which if I run them in Visual studio directly, they work fine. And GUI log window doesn't say much about why tests failed. Why could these tests be failing? Is there a way to find out? More logging, maybe. But what should I be logging? Please advise. Thanks Pankaj
pankazmittal wrote:
Now, there are some failed tests which if I run them in Visual studio directly
Sometimes you can have hidden dependencies between tests such that you can run them all you get some failures but if you run them in isolation they pass. What happens if you run those tests individually in the NUnit GUI? Do they pass or do they still fail? How are you running them when inside VS? Are you using TestDriven.net or similar? You might want to examine the state of your test fixture before and after running each test.
Kevin
-
It's not in the drop down list and the last option "All Message Boards" probably skipped from my eyes. I should have looked deeper. Thanks
-
pankazmittal wrote:
Now, there are some failed tests which if I run them in Visual studio directly
Sometimes you can have hidden dependencies between tests such that you can run them all you get some failures but if you run them in isolation they pass. What happens if you run those tests individually in the NUnit GUI? Do they pass or do they still fail? How are you running them when inside VS? Are you using TestDriven.net or similar? You might want to examine the state of your test fixture before and after running each test.
Kevin
I am using Resharper plugin inside Visual Studio. I think I have all dependenies copied over. Not sure about the TextFixture state. Will try to find something on that. Thanks ..
-
I am using Resharper plugin inside Visual Studio. I think I have all dependenies copied over. Not sure about the TextFixture state. Will try to find something on that. Thanks ..
I too have run into the problem you describe in the past. It turned out to be my own mistakes. It's likely that one or more tests are not leaving the test fixture class in a clean state, such that a subsequent test(s) fails. But if you run a test in isolation you're always starting from a clean state so it may pass. Btw, that ReSharper unit test plugin is pretty good. Unfortunately, I'm not a ReSharper user so I have to make use of their old free Unit Test Run 1.0 which they've now discontinued. :( It's a bit flaky but it works.
Kevin
-
I too have run into the problem you describe in the past. It turned out to be my own mistakes. It's likely that one or more tests are not leaving the test fixture class in a clean state, such that a subsequent test(s) fails. But if you run a test in isolation you're always starting from a clean state so it may pass. Btw, that ReSharper unit test plugin is pretty good. Unfortunately, I'm not a ReSharper user so I have to make use of their old free Unit Test Run 1.0 which they've now discontinued. :( It's a bit flaky but it works.
Kevin
yes! it does run in isolation. must be something I am missing. gotta find it. and yes! resharper is very very cool. Before we got the license I use to run it's 30 day trial. Uninstall it and run it again. Not sure if that still works though. It becomes pain though after a while. But this tool really makes loads of difference.
-
yes! it does run in isolation. must be something I am missing. gotta find it. and yes! resharper is very very cool. Before we got the license I use to run it's 30 day trial. Uninstall it and run it again. Not sure if that still works though. It becomes pain though after a while. But this tool really makes loads of difference.
I've used ReSharper briefly in the past but I'm currently in the CodeRush/Refactor! camp (actually really just Refactor!). ReSharper is more widely used though. The impression I get is that ReSharper is better at general code analysis and unit testing (CodeRush has only recently started doing this). CodeRush/Refactor! is stronger on refactoring (I think). But anyway both tools are good, though both periodically run into performance problems.
Kevin
-
I've used ReSharper briefly in the past but I'm currently in the CodeRush/Refactor! camp (actually really just Refactor!). ReSharper is more widely used though. The impression I get is that ReSharper is better at general code analysis and unit testing (CodeRush has only recently started doing this). CodeRush/Refactor! is stronger on refactoring (I think). But anyway both tools are good, though both periodically run into performance problems.
Kevin
After reading your views about codeRush, I wanted to check it out. Guess what I found... A free license of CodeRush/Refactor. I still have to form my opinion on this product for thought you may like free stuff. Get it at http://msdn.microsoft.com/en-us/vcsharp/dd218053.aspx and Enjoy! :)
-
After reading your views about codeRush, I wanted to check it out. Guess what I found... A free license of CodeRush/Refactor. I still have to form my opinion on this product for thought you may like free stuff. Get it at http://msdn.microsoft.com/en-us/vcsharp/dd218053.aspx and Enjoy! :)
Yes, in fact it's what I use. I have Refactor! Pro but not the full CodeRush (long-term umemployed contractor :( ) So I use Refactor! Pro + CodeRush Xpress. I think in the DevExpress forums there is someone who is a ReSharper user who also uses CodeRush Xpress at the same time and reports that the combination is OK. The feature I like most is quick navigation but I assume you must already have this in ReSharper? I have a blog post on this here: Developer Productivity Tools[^]
Kevin
-
Yes, in fact it's what I use. I have Refactor! Pro but not the full CodeRush (long-term umemployed contractor :( ) So I use Refactor! Pro + CodeRush Xpress. I think in the DevExpress forums there is someone who is a ReSharper user who also uses CodeRush Xpress at the same time and reports that the combination is OK. The feature I like most is quick navigation but I assume you must already have this in ReSharper? I have a blog post on this here: Developer Productivity Tools[^]
Kevin
cool .. i will check this out too. Thanks
-
Yes, in fact it's what I use. I have Refactor! Pro but not the full CodeRush (long-term umemployed contractor :( ) So I use Refactor! Pro + CodeRush Xpress. I think in the DevExpress forums there is someone who is a ReSharper user who also uses CodeRush Xpress at the same time and reports that the combination is OK. The feature I like most is quick navigation but I assume you must already have this in ReSharper? I have a blog post on this here: Developer Productivity Tools[^]
Kevin
Can't see DevExpress menu in VStudio. How do you bring it up? Did it happen to you too?
-
Can't see DevExpress menu in VStudio. How do you bring it up? Did it happen to you too?
For the free versions of these products the menu has been removed at the request of Microsoft apparently. I don't have this issue because I get the mnu through having the paid for Refactor! Pro. However, there is a way round this. See near the bottom of this thread (it's not long) http://community.devexpress.com/forums/t/72689.aspx[^] There is a registry key you can set to display the menu. Also there is a keyboard shortcut for invoking a refactoring(s) that you can customise. if you go to VS Tools/Options/Keyboard and search for CodeRush.Refactor you will see a keyboard binding. I set mine to Ctrl+# (UK keyboard). You also get the refactorings via a smart tag - three grey dots that appear next to a code element when a refactoring is available and the cursor or mouse is at that location. If you have any other issues try searching or posting in the DevExpress forums. There are lots of very helpful guys there.
Kevin
-
For the free versions of these products the menu has been removed at the request of Microsoft apparently. I don't have this issue because I get the mnu through having the paid for Refactor! Pro. However, there is a way round this. See near the bottom of this thread (it's not long) http://community.devexpress.com/forums/t/72689.aspx[^] There is a registry key you can set to display the menu. Also there is a keyboard shortcut for invoking a refactoring(s) that you can customise. if you go to VS Tools/Options/Keyboard and search for CodeRush.Refactor you will see a keyboard binding. I set mine to Ctrl+# (UK keyboard). You also get the refactorings via a smart tag - three grey dots that appear next to a code element when a refactoring is available and the cursor or mouse is at that location. If you have any other issues try searching or posting in the DevExpress forums. There are lots of very helpful guys there.
Kevin
Awesome! Registry change did the trick. Asking you proved to be much faster than sending it to DevExpress team. thanks a lot :)
-
Awesome! Registry change did the trick. Asking you proved to be much faster than sending it to DevExpress team. thanks a lot :)
Have you managed to solve your unit test error btw?
Kevin
-
Have you managed to solve your unit test error btw?
Kevin
Funny :) Yes! Some of them are working now. When I run the tests together from outside the user credentials are not getting refreshed. But somehow they do inside VStudio. Maybe Resharper helping it out somehow. I don't know that. I am still finding reason for not so lucky tests. Thanks for all of your help and introducing me to new tool. I really appreciate that. Pankaj
-
Funny :) Yes! Some of them are working now. When I run the tests together from outside the user credentials are not getting refreshed. But somehow they do inside VStudio. Maybe Resharper helping it out somehow. I don't know that. I am still finding reason for not so lucky tests. Thanks for all of your help and introducing me to new tool. I really appreciate that. Pankaj
Last year at work I created a suite of ASP.NET UI tests using WatiN (very cool open source app. btw) and NUnit. One set of tests involved logging in and logging out in various roles and verifying the appropriate UI elements were present or not. I had to be careful to make sure I carefully returned things to a clean slate after each test. For other tests I also had a test database that I was writing to, so I had to return that to a clean slate as well. Sounds like it could be quite tough trying to track down the problem in your case. What I do when I'm at a dead end is comment out all the code and then gradually uncomment until things start going wrong. It can be tedious but sometimes you have no choice.
Kevin
-
Last year at work I created a suite of ASP.NET UI tests using WatiN (very cool open source app. btw) and NUnit. One set of tests involved logging in and logging out in various roles and verifying the appropriate UI elements were present or not. I had to be careful to make sure I carefully returned things to a clean slate after each test. For other tests I also had a test database that I was writing to, so I had to return that to a clean slate as well. Sounds like it could be quite tough trying to track down the problem in your case. What I do when I'm at a dead end is comment out all the code and then gradually uncomment until things start going wrong. It can be tedious but sometimes you have no choice.
Kevin
Well! The way I am going to nail this down is keep skipping the ones not working and keep them for last. And then take one test at a time. its gonna be a longggggggggg walk ...