What do you do?
-
Logging, logging, and more logging. In my main product (see .sig) I put in lots of logging just in case some critical part blows up. And since the app deals with IE and network communication - two areas that are susceptible to errors - it's really come in handy many times. We can ask the customer to send us a log (the log is encrypted, but they can click a button and it's zipped up into a CAB file on their desktop that they mail us) and many times that gives us enough info to provide a fix, or at least an explanation and a workaround. It does take extra work to do the logging, but it's like comments - if you write the logging at the same time as the "real" code, it ain't bad.
--Mike-- Visual C++ MVP :cool: LINKS~! Ericahist | PimpFish | CP SearchBar v3.0 | C++ Forum FAQ Ford, what's this fish doing in my ear?
Thanks for your advice. I will see about implementing some logging.
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
Thank you very much for this information! I think this is the course I will follow. Does the binary that I ship to the client need to have symbols for the dump to work correctly, or do I just need the symbols on my machine?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
You just need to have symbols on your machine. You don't need to ship the symbols. Using symbols people can reverse engineer your application.:)
-
Yeah, as Michael said it's pretty much something you have to plan for in advance. Our app has a debug mode that can be set by the end user in the config file and it will log everything that goes on of any consequence. I remember going through the phase your going through many years ago, once you have a very large user base these kinds of things are much easier to diagnose and after a few years inevitably they are something oddball on the computer that is failing. In a very serious case where it's important and you really need to get it going I would not at all hesitate to arrange with the end user to have a courier pick up the computer and bring it back to your shop, that's not out of the question in some situations. If you don't have serious logging in place I would not trust at all what the client is telling you, not because they are malicious necessarily, they just don't have an eye for detail like a programmer does and often miss out on critical things when relating to you what's happening. This is where you should consider something like CoPilot[^] It's dead easy to set up a session, nothing special is needed and you can walk through it with them and see exactly what's happening as well you can examine the machine in detail for yourself. We use that product all the time for sticky situations where it's difficult to communicate with the end user or they require extra help to set something up.
"110%" - it's the new 70%
Thanks for your excellent post. The client and I do make extensive use of Remote Assistance, but the crashes are intermittent, and we can't be connected all day long.
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Simple. You need to utilize your inner psychic debugging skills. ;)
Tech, life, family, faith: Give me a visit. I'm currently blogging about: Virginia Tech Shootings, Guns, and Politics The apostle Paul, modernly speaking: Epistles of Paul Judah Himango
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
I've seen sometimes that the users are not telling the truth, in our programs we log almost everything, and even doing that sometimes the users can catch us by not telling all the truth or by missing some important things because they don't know that they were important. You can: - Remote debug through internet (this will imply modifying the internal VPN's of the customer so this can become a problem if you must do it yourself (I'd never do that by myself)). - Remote connect to that computer in order to be able to see what has happened exactly once the app has crashed or even having an eye on it working in order to see what is the customer doing. In order to do that I've started using the "logmein" software and it works like charm because it goes through the 80 port and in this way you must not modify anything in the routers... - Log everything, it is very important when you deploy an application to be able to control what is happening. - Buy a tent and start living in front of that computer... ;) Hope this helps, and tons of luck finding the cause of your problems! :rose:
-
I've seen sometimes that the users are not telling the truth, in our programs we log almost everything, and even doing that sometimes the users can catch us by not telling all the truth or by missing some important things because they don't know that they were important. You can: - Remote debug through internet (this will imply modifying the internal VPN's of the customer so this can become a problem if you must do it yourself (I'd never do that by myself)). - Remote connect to that computer in order to be able to see what has happened exactly once the app has crashed or even having an eye on it working in order to see what is the customer doing. In order to do that I've started using the "logmein" software and it works like charm because it goes through the 80 port and in this way you must not modify anything in the routers... - Log everything, it is very important when you deploy an application to be able to control what is happening. - Buy a tent and start living in front of that computer... ;) Hope this helps, and tons of luck finding the cause of your problems! :rose:
Thanks, Joan. My customer does not have a VPN, he's just a little guy, so that pretty much rules out remote debugging. Thanks for your knowledge and encouragement!
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Richie308 wrote:
what on Earth is there to do about it?
In my application is a multithreaded debugging logger as well as an execution startup log. The startup log does not affect real-time only because it is there only during startup. It logs steps in the process of starting the application such that I know where something crashes. But that is only for startup, as I said. I have a multi-threaded UDP transmission thread that when enabled monitors application activity. Think of it as a debugger/profiler internal to the system. When enabled I know what is running, what was last operated on, and what is taking longer than it should. The client has a copy of my debugging monitoring application, though they usually don't know it. When something goes wrong, I walk them through enabling the debug-thread, and starting up the client to monitor it. When it dies I ask for the last few lines. This on top of what others have said.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Any app that I write will deploy with code to write an error log that gives me a stack track and error information if the app crashes. I do sometimes write code that adds to this log file, for deployment to machines where a bug is hard to track down, but, usually, I look and see what the problem is. Writing small functions is generally a good thing, but it really helps here - if there's a way to log the line number of a crash in C#, I don't know it. So, a 100 line function would be hard to debug, a 10 line function, less so :-)
Christian Graus - Microsoft MVP - C++ Metal Musings - Rex and my new metal blog "I am working on a project that will convert a FORTRAN code to corresponding C++ code.I am not aware of FORTRAN syntax" ( spotted in the C++/CLI forum )
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
I'm a big fan of crash dumps. Here's a good article to get started: http://www.codeproject.com/debug/postmortemdebug_standalone1.asp[^]
-
I'm a big fan of crash dumps. Here's a good article to get started: http://www.codeproject.com/debug/postmortemdebug_standalone1.asp[^]
-
Richie308 wrote:
what on Earth is there to do about it?
In my application is a multithreaded debugging logger as well as an execution startup log. The startup log does not affect real-time only because it is there only during startup. It logs steps in the process of starting the application such that I know where something crashes. But that is only for startup, as I said. I have a multi-threaded UDP transmission thread that when enabled monitors application activity. Think of it as a debugger/profiler internal to the system. When enabled I know what is running, what was last operated on, and what is taking longer than it should. The client has a copy of my debugging monitoring application, though they usually don't know it. When something goes wrong, I walk them through enabling the debug-thread, and starting up the client to monitor it. When it dies I ask for the last few lines. This on top of what others have said.
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
Did you roll your own debugging monitor, or is it a commercial variety?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
Thank you very much for this information! I think this is the course I will follow. Does the binary that I ship to the client need to have symbols for the dump to work correctly, or do I just need the symbols on my machine?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
To add to what Rama's said: do the work to build a symbol repository, and modify your build process to automatically add the symbols and executables for each released build to the repository. Being able to rely on your debugger pulling the correct .exes, .dlls, and .pdbs for whatever dump you're opening is... just really nice. '
----
It appears that everybody is under the impression that I approve of the documentation. You probably also blame Ken Burns for supporting slavery.
--Raymond Chen on MSDN
-
Did you roll your own debugging monitor, or is it a commercial variety?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Richie308 wrote:
Did you roll your own debugging monitor, or is it a commercial variety?
I rolled my own but there are similar ones commercially available. If work doesn't pay for it, and I think I need it, I write it. :)
_________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
-
Thanks for your advice. I will see about implementing some logging.
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
log4net has saved my ass on several occasions. It's easy and reliable and just about the best logger for .NET I've ever seen so that's what I'd suggest. http://logging.apache.org/log4net/[^]
-
I'm only looking for general advice with this question, that's why I posted it in the lounge. Please accept my groveling apologies if this is not appropriate for the lounge. When your app is having mysterious crashes on the client's machine, and you cannot replicate them on your development machine, what on Earth is there to do about it? Assuming it's not an option to install Visual Studio onto the client machine, and the fact that the crashes are random and infrequent enough to make a remote debug session impractical, what strategy can be employed?
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Hi, Your application works on a range of Operating Systems ,that if you support a specific OS, should have been tested on that OS. When it works but crashes at your client the question is what caused your application to crash, what's different? I think there are no 'mysterious crashes', only faults made that need to be corrected. It's better to ask what causes this crash, and what you can do about it' technicaly, not mysteriously:)
With friendly greetings,:) Eric Goedhart Interbritt
-
Hi, Your application works on a range of Operating Systems ,that if you support a specific OS, should have been tested on that OS. When it works but crashes at your client the question is what caused your application to crash, what's different? I think there are no 'mysterious crashes', only faults made that need to be corrected. It's better to ask what causes this crash, and what you can do about it' technicaly, not mysteriously:)
With friendly greetings,:) Eric Goedhart Interbritt
My question was what strategy to follow to uncover the fault. I did not mean to imply that I thought the fault was due to the positions of the planets and stars.
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
-
Logging, logging, and more logging. In my main product (see .sig) I put in lots of logging just in case some critical part blows up. And since the app deals with IE and network communication - two areas that are susceptible to errors - it's really come in handy many times. We can ask the customer to send us a log (the log is encrypted, but they can click a button and it's zipped up into a CAB file on their desktop that they mail us) and many times that gives us enough info to provide a fix, or at least an explanation and a workaround. It does take extra work to do the logging, but it's like comments - if you write the logging at the same time as the "real" code, it ain't bad.
--Mike-- Visual C++ MVP :cool: LINKS~! Ericahist | PimpFish | CP SearchBar v3.0 | C++ Forum FAQ Ford, what's this fish doing in my ear?
Yeah, I agree! One of the first steps in a major application, logging.. Too many people are spoiled by debuggers, I still remember Printf as being my debugging tool :)
Rocky <>< Latest Code Blog Post: OpenID - C# project! Latest Tech Blog Post: Want to test Joost (video on demand) - I have invites!
-
Thanks for your excellent post. The client and I do make extensive use of Remote Assistance, but the crashes are intermittent, and we can't be connected all day long.
-------------------------------- "All that is necessary for the forces of evil to win in the world is for enough good men to do nothing" -- Edmund Burke
Yeah, that is where logging first in so good. Even if the crash itself does not generate an error in the log, if you have enough tracing points in the program, you will know the last things that were happening prior to the crash by looking in the log and seeing when the log was restarted. I always used a method of logging that allowed me to enable it from either a command line option or in a config file for the program. It would be setup so that I could specify a level of trace so that it could program simple overview or a level of great detail. Normally, the apps would have it turned off, but if a problem appeared, would have the client set the value to overview and if that did get the information I needed, I would have the keep increasing the level of detail until it did. This can cost a slight overhead in performance when it is turned off, but the logging is critical when things go wrong!
Rocky <>< Latest Code Blog Post: OpenID - C# project! Latest Tech Blog Post: Want to test Joost (video on demand) - I have invites!
-
Richie308 wrote:
Do you happen to know if the VS remote debugger works across the internet? It doesn't specifically say in the docs. It says only "Across domains" which I'm not sure is the same thing.
I think you'd have to ask the client to allow you to VPN in to their network. I don't think it would work over the Internet - specially since I assume your client would have appropriate proxies and firewalls in place.
Regards, Nish
Nish’s thoughts on MFC, C++/CLI and .NET (my blog)
C++/CLI in ActionFly on your way like an eagle Fly as high as the sun On your wings like an eagle Fly and touch the sun
Right - it will almost surely not work over the internet - and you will have to do a cross domain debuging session which can be a real pain (you know you have to be in the "debuging-role" on the remote computer). Bye the way a domain is some kind of "networking-namespace" ;)
