In Search Of A Tool
-
The usual recommendation would be to use [Wireshark](https://www.wireshark.org/) Problem one is that it can only analyze the traffic on the computer it sits on. Problem 2 is that a switch is passing the traffic only between the nodes that are talking, if you would plug a PC with Wireshark installed to the switch it would only see broadcast messages on the network. Solution one, use a hub instead of a switch. Sorry, going back to the stone age is hardly a solution. So, the solutions are either to get a switch where you can monitor traffic directly, or program it to mirror a port to create a data tap. What switch is it you have? Sometimes "unmanaged" switches aren't totally stupid, or rather sitting somewhere in between smart and stupid switches.
Wrong is evil and must be defeated. - Jeff Ello
Thanks, Jorgan... I rather thought that would be the answer, but I was hopeful that there might be a way around that switch limitation. This switch is quite limited, I'm afraid. It's Hirschmann, and meant for industrial solutions, and therefor dumb but reliable. Sigh...
Will Rogers never met me.
-
The 840 kW Solar system, along with the 2.5 MW generator system that supports it, went teats up Friday. I spent 12 hours Saturday, and 16 hours yesterday, trying to get it back running again, powering a million visitors per year resort. The first notice we had was a message that the generators had lost all communication with the solar controller that rules them. The way we're configured is a fiber link from the solar (plus battery) yard to the generators about 2 miles away. The message actually originated from the generators - I have no such alert function in the solar/battery installation. Within the solar + battery installation I have an unmanaged switch that allows all of the components of the installation to work together and keep the lights on. During this event I asked the Battery supplier (Tesla) for a tool to allow me to view network traffic or log files for commands sent to the generators during this event. They told me that no such tool exists. So I'm asking for a tool that let's me monitor and record network activity on a particular group of IP addresses. I'm not asking for any fancy tools, just a simple tool to allow creating a log of messages from one IP address to another. Thanks, in advance, if you have anything to offer.
Will Rogers never met me.
I'm not sure how you're going to monitor for an event that has already happened; and for which there is no log. All you will get up to a point, is "normal" traffic; in anticipation of an event that may never happen again under the same circumstances. One should be focusing on future "recovery" plan(s); instead of diagnostics; at this point (IMO).
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
-
I'm not sure how you're going to monitor for an event that has already happened; and for which there is no log. All you will get up to a point, is "normal" traffic; in anticipation of an event that may never happen again under the same circumstances. One should be focusing on future "recovery" plan(s); instead of diagnostics; at this point (IMO).
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
A large part of diagnosing a problem is having visibility into what has recently happened - this system (by Tesla design) completely lacks that visibility. The situation cannot be improved until I have some means of seeing inside the black box and reading its history. If this happens again tonight, we are back to square one with no insight as to what should be done, or even where to investigate.
Will Rogers never met me.
-
Thanks, Jorgan... I rather thought that would be the answer, but I was hopeful that there might be a way around that switch limitation. This switch is quite limited, I'm afraid. It's Hirschmann, and meant for industrial solutions, and therefor dumb but reliable. Sigh...
Will Rogers never met me.
Maybe find a router to replace the switch. Many routers will have some sort of configurable logging built into them so you'd just need one that was robust in that respect. Alternatively, it's inelegant, but what is probably possible would be to use a basic proxy server through which the traffic routes bidirectionally between these things. It would require a different configuration of things though (to talk to the proxy server and let it handle the routing). Basically, use port forwarding to forward not just to a different internal port but also to a different internal IP. Wiresharking the proxy then should be able to give you what you want.
-
I'm not sure how you're going to monitor for an event that has already happened; and for which there is no log. All you will get up to a point, is "normal" traffic; in anticipation of an event that may never happen again under the same circumstances. One should be focusing on future "recovery" plan(s); instead of diagnostics; at this point (IMO).
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
-
A large part of diagnosing a problem is having visibility into what has recently happened - this system (by Tesla design) completely lacks that visibility. The situation cannot be improved until I have some means of seeing inside the black box and reading its history. If this happens again tonight, we are back to square one with no insight as to what should be done, or even where to investigate.
Will Rogers never met me.
I have to believe Tesla "has the tools"; but they're "company tools". I would try leaning on them. Talk about all your "connections". (On the other hand, some hardware vendors have used my tools to debug the new product my tool will use).
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
-
If you know something should do something every N-minutes/seconds or whatever, the lack of evidence is evidence of something lacking.
From the post, it is a "black box". There was an "event". And there is no "log". Where is the "evidence"? Psycho-babble about "no evidence" being evidence, is nonsense. I expect a "chance" of evidence before I go looking for it. Regardless of "orders". This is like exception handling without "try blocks" ... or messages. Try and post this in Q&A under: "It's working now, but it stopped working and I don't know why" ... and see what you get.
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
-
The 840 kW Solar system, along with the 2.5 MW generator system that supports it, went teats up Friday. I spent 12 hours Saturday, and 16 hours yesterday, trying to get it back running again, powering a million visitors per year resort. The first notice we had was a message that the generators had lost all communication with the solar controller that rules them. The way we're configured is a fiber link from the solar (plus battery) yard to the generators about 2 miles away. The message actually originated from the generators - I have no such alert function in the solar/battery installation. Within the solar + battery installation I have an unmanaged switch that allows all of the components of the installation to work together and keep the lights on. During this event I asked the Battery supplier (Tesla) for a tool to allow me to view network traffic or log files for commands sent to the generators during this event. They told me that no such tool exists. So I'm asking for a tool that let's me monitor and record network activity on a particular group of IP addresses. I'm not asking for any fancy tools, just a simple tool to allow creating a log of messages from one IP address to another. Thanks, in advance, if you have anything to offer.
Will Rogers never met me.
Interesting problem. Ignoring cost I figured it should be easy to find a solution. But I didn't find anything at all. Most suggestions point out that the device has to be configured to generate logs. So in your case the 'switch'. That of course depends on the exact vendor, product, version, etc. Presuming that there is a way to do that you are also going to need somewhere to store it. And quite possible there is going to be a lot of data. You probably also want to set up a round robin. So it only stores, for example, last 7 days before reusing the storage space. If the device doesn't do that you will need to put a server in place to proxy that. Then you will need to figure out how to analyze the data that you do get.
-
Interesting problem. Ignoring cost I figured it should be easy to find a solution. But I didn't find anything at all. Most suggestions point out that the device has to be configured to generate logs. So in your case the 'switch'. That of course depends on the exact vendor, product, version, etc. Presuming that there is a way to do that you are also going to need somewhere to store it. And quite possible there is going to be a lot of data. You probably also want to set up a round robin. So it only stores, for example, last 7 days before reusing the storage space. If the device doesn't do that you will need to put a server in place to proxy that. Then you will need to figure out how to analyze the data that you do get.
Surprisingly, I actually did find a tool for this. Port sniffers have been around in software for years, bit I discovered a number of physical traffic monitors out the on the interwebs. Since the Site Controller runs through a simple switch, then through a media converter to interface with the fiber running to the generators, I think I can install one in the path between the media converter and the switch. It won't help with the recent event, but it might be a huge help if this kind of failure ever occurs again. As you mentioned, too, I only need a few days' of recording depth. If nobody visits in that time, the Grand Canyon West resort has been nuked, or nothing bad happened.
Will Rogers never met me.
-
From the post, it is a "black box". There was an "event". And there is no "log". Where is the "evidence"? Psycho-babble about "no evidence" being evidence, is nonsense. I expect a "chance" of evidence before I go looking for it. Regardless of "orders". This is like exception handling without "try blocks" ... or messages. Try and post this in Q&A under: "It's working now, but it stopped working and I don't know why" ... and see what you get.
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I