What do you do?
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Any app that I write will deploy with code to write an error log that gives me a stack track and error information if the app crashes. I do sometimes write code that adds to this log file, for deployment to machines where a bug is hard to track down, but, usually, I look and see what the problem is. Writing small functions is generally a good thing, but it really helps here - if there's a way to log the line number of a crash in C#, I don't know it. So, a 100 line function would be hard to debug, a 10 line function, less so :-)
Christian Graus - Microsoft MVP - C++ Metal Musings - Rex and my new metal blog "I am working on a project that will convert a FORTRAN code to corresponding C++ code.I am not aware of FORTRAN syntax" ( spotted in the C++/CLI forum )
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
I'm a big fan of crash dumps. Here's a good article to get started: http://www.codeproject.com/debug/postmortemdebug_standalone1.asp[^]
-
I'm a big fan of crash dumps. Here's a good article to get started: http://www.codeproject.com/debug/postmortemdebug_standalone1.asp[^]
-
Richie308 wrote:
what on Earth is there to do about it?
In my application is a multithreaded debugging logger as well as an execution startup log. The startup log does not affect real-time only because it is there only during startup. It logs steps in the process of starting the application such that I know where something crashes. But that is only for startup, as I said. I have a multi-threaded UDP transmission thread that when enabled monitors application activity. Think of it as a debugger/profiler internal to the system. When enabled I know what is running, what was last operated on, and what is taking longer than it should. The client has a copy of my debugging monitoring application, though they usually don't know it. When something goes wrong, I walk them through enabling the debug-thread, and starting up the client to monitor it. When it dies I ask for the last few lines. This on top of what others have said.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
Did you roll your own debugging monitor, or is it a commercial variety?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
Thank you very much for this information! I think this is the course I will follow. Does the binary that I ship to the client need to have symbols for the dump to work correctly, or do I just need the symbols on my machine?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
To add to what Rama's said: do the work to build a symbol repository, and modify your build process to automatically add the symbols and executables for each released build to the repository. Being able to rely on your debugger pulling the correct .exes, .dlls, and .pdbs for whatever dump you're opening is... just really nice. '
----
It appears that everybody is under the impression that I approve of the documentation. You probably also blame Ken Burns for supporting slavery.
--Raymond Chen on MSDN
-
Did you roll your own debugging monitor, or is it a commercial variety?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Richie308 wrote:
Did you roll your own debugging monitor, or is it a commercial variety?
I rolled my own but there are similar ones commercially available. If work doesn't pay for it, and I think I need it, I write it. :)
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
Thanks for your advice. I will see about implementing some logging.
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
log4net has saved my ass on several occasions. It's easy and reliable and just about the best logger for .NET I've ever seen so that's what I'd suggest. http://logging.apache.org/log4net/[^]
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Hi, Your application works on a range of Operating Systems ,that if you support a specific OS, should have been tested on that OS. When it works but crashes at your client the question is what caused your application to crash, what's different? I think there are no 'mysterious crashes', only faults made that need to be corrected. It's better to ask what causes this crash, and what you can do about it' technicaly, not mysteriously:)
With friendly greetings,:) Eric Goedhart Interbritt
-
Hi, Your application works on a range of Operating Systems ,that if you support a specific OS, should have been tested on that OS. When it works but crashes at your client the question is what caused your application to crash, what's different? I think there are no 'mysterious crashes', only faults made that need to be corrected. It's better to ask what causes this crash, and what you can do about it' technicaly, not mysteriously:)
With friendly greetings,:) Eric Goedhart Interbritt
My question was what strategy to follow to uncover the fault. I did not mean to imply that I thought the fault was due to the positions of the planets and stars.
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
Logging, logging, and more logging. In my main product (see .sig) I put in lots of logging just in case some critical part blows up. And since the app deals with IE and network communication - two areas that are susceptible to errors - it's really come in handy many times. We can ask the customer to send us a log (the log is encrypted, but they can click a button and it's zipped up into a CAB file on their desktop that they mail us) and many times that gives us enough info to provide a fix, or at least an explanation and a workaround. It does take extra work to do the logging, but it's like comments - if you write the logging at the same time as the "real" code, it ain't bad.
--Mike-- Visual C++ MVP :cool: LINKS~! Ericahist | PimpFish | CP SearchBar v3.0 | C++ Forum FAQ Ford, what's this fish doing in my ear?
Yeah, I agree! One of the first steps in a major application, logging.. Too many people are spoiled by debuggers, I still remember Printf as being my debugging tool :)
Rocky <>< Latest Code Blog Post: OpenID - C# project! Latest Tech Blog Post: Want to test Joost (video on demand) - I have invites!
-
Thanks for your excellent post. The client and I do make extensive use of Remote Assistance, but the crashes are intermittent, and we can't be connected all day long.
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Yeah, that is where logging first in so good. Even if the crash itself does not generate an error in the log, if you have enough tracing points in the program, you will know the last things that were happening prior to the crash by looking in the log and seeing when the log was restarted. I always used a method of logging that allowed me to enable it from either a command line option or in a config file for the program. It would be setup so that I could specify a level of trace so that it could program simple overview or a level of great detail. Normally, the apps would have it turned off, but if a problem appeared, would have the client set the value to overview and if that did get the information I needed, I would have the keep increasing the level of detail until it did. This can cost a slight overhead in performance when it is turned off, but the logging is critical when things go wrong!
Rocky <>< Latest Code Blog Post: OpenID - C# project! Latest Tech Blog Post: Want to test Joost (video on demand) - I have invites!
-
Richie308 wrote:
Do you happen to know if the VS remote debugger works across the internet? It doesn't specifically say in the docs. It says only "Across domains" which I'm not sure is the same thing.
I think you'd have to ask the client to allow you to VPN in to their network. I don't think it would work over the Internet - specially since I assume your client would have appropriate proxies and firewalls in place.
Regards, Nish
Nish’s thoughts on MFC, C++/CLI and .NET (my blog)
C++/CLI in ActionFly on your way like an eagle Fly as high as the sun On your wings like an eagle Fly and touch the sun
Right - it will almost surely not work over the internet - and you will have to do a cross domain debuging session which can be a real pain (you know you have to be in the "debuging-role" on the remote computer). Bye the way a domain is some kind of "networking-namespace" ;)
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
It really depends on your application as there are many methods of identifying such problems. Usually this can occur for the following reasons ... 1. Older DB engine ... you have SQL2005 he's got MSDE. 2. You use local DB acces (SQL2005Express), he uses remote (SQL2005Server). 3. ODBC driver issue. 4. Different OS Service Pack. 5. Networking is more strict .... he's got firewalls, proxys, ... etc.:~ 6. Wrong Framework version ... (silly, i know).:doh: 7. You use DLLs from other apps (Ms-Office, ... etc) that are different than the ones on your clinet. 8. G.A.C issue (God help us all !!!!:omg:) 9. Click once issue / App needs admistrative privileges.:confused: 10. Our own (or the customers for that mater) stupidity .... e.x: I had a firewall that i set to prevent system wide hooks for applications (others than IE, .. etc), thus crashing several of my .NET assemblies. It took me 1 week to figure that out. X| 11. a million other tiny issues that can occur. :~ Things to do ... 1. IMPORTANT: Define your error trapping and logging approach, BEFORE starting coding. It WILL make the whole logging proccess A LOT faster and easier. 2. Log procedure execution (parameters, results, stack, .. .etc). 3. Log SQL statements. 4. Log object creation, memory usage (It's hard, i know). 5. Log file activity. 6. Log assembly activity. 7. Log errors. 8. Provide "Send error report" functionality to your application. Many were the times where the so called BUG, wasn't a bug at all. The above logging method has helped me pinpoint the problem and identify it as a 'Bug' and issue a patch to fix it or as a 'Customer Lack of IT knowhow' and help the customer pinpoint it and fix it.
-
Give your machine to the client.
But you have to convince him it's the only option he has ;-) I would suggest taking his machine instead (maybe for a week or so). And start local debugging. I think it would be best. Or, if your customer can't do without his PC, make a complete image of his system, and install it in a virtual machine.
-
Ok, that get's a 5...
-
It really depends on your application as there are many methods of identifying such problems. Usually this can occur for the following reasons ... 1. Older DB engine ... you have SQL2005 he's got MSDE. 2. You use local DB acces (SQL2005Express), he uses remote (SQL2005Server). 3. ODBC driver issue. 4. Different OS Service Pack. 5. Networking is more strict .... he's got firewalls, proxys, ... etc.:~ 6. Wrong Framework version ... (silly, i know).:doh: 7. You use DLLs from other apps (Ms-Office, ... etc) that are different than the ones on your clinet. 8. G.A.C issue (God help us all !!!!:omg:) 9. Click once issue / App needs admistrative privileges.:confused: 10. Our own (or the customers for that mater) stupidity .... e.x: I had a firewall that i set to prevent system wide hooks for applications (others than IE, .. etc), thus crashing several of my .NET assemblies. It took me 1 week to figure that out. X| 11. a million other tiny issues that can occur. :~ Things to do ... 1. IMPORTANT: Define your error trapping and logging approach, BEFORE starting coding. It WILL make the whole logging proccess A LOT faster and easier. 2. Log procedure execution (parameters, results, stack, .. .etc). 3. Log SQL statements. 4. Log object creation, memory usage (It's hard, i know). 5. Log file activity. 6. Log assembly activity. 7. Log errors. 8. Provide "Send error report" functionality to your application. Many were the times where the so called BUG, wasn't a bug at all. The above logging method has helped me pinpoint the problem and identify it as a 'Bug' and issue a patch to fix it or as a 'Customer Lack of IT knowhow' and help the customer pinpoint it and fix it.
Thank you for your good post. I dread hand-coding log function calls for every little thing going on in the app. But I understand that that is what has to be done. However, if you log too much, the log file becomes too huge, and they can't email it! :doh:
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
Yeah, I agree! One of the first steps in a major application, logging.. Too many people are spoiled by debuggers, I still remember Printf as being my debugging tool :)
Rocky <>< Latest Code Blog Post: OpenID - C# project! Latest Tech Blog Post: Want to test Joost (video on demand) - I have invites!
Rocky Moore wrote:
I still remember Printf as being my debugging tool
Printf has saved my life several times. :) It is the best 'debugging tool'!
-
Give your machine to the client.
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Activate logging;) Once I run into a crash. The responsible developer - "It works here. Send your machine to me.". And he was serious... rgds /Jonas
