I've got an oddball issue I'm troubleshooting. I have a heavy duty data exchange service (windows service hosted in a data center) currently routing data requests from about 120 clients. Any individual client has about 5-15 other clients able to query them. At any given time around 100 or so will be connected, each with a single socket to the exchange service. Most client connection come in from the internet (a few are also hosted at the same data center). The behaviour I'm observing is that frequently the service will carry out a formal disconnect (exchange terminate messages, and then close the socket) from the client after being connected for less than 10 seconds. Tracking it down I've identified the problem to the EndRead method below. I have the socket managed by custom implementation of Stream.
public override IAsyncResult BeginRead(byte[] buffer, int offset, int count, AsyncCallback callback, object state) {
if (null != _socket && _socket.Connected)
return _socket.BeginReceive(buffer, offset, count, SocketFlags.None, callback, state);
else
return null;
}
public override int EndRead(IAsyncResult asyncResult) {
int result = -1;
if (null != _socket && _socket.Connected) {
try { result = _socket.EndReceive(asyncResult); } catch (ObjectDisposedException) { }
}
return result;
}
One of two things are happening (my logging didn't track which). Either _socket.Connected is false, or _socket.EndReceive is throwing ObjectDisposedException, since -1 is returned. Wireshark captures from both the server and client sides show no FIN or RST was ever sent by the client prior to the formal disconnect, instead the disconnect is incidentally caused by Dispose logic once the read fails. Since the connection is in fact still active the terminate message is sent and responded to. If any other exception was thrown it would have been captured and logged. The read that fails is the first of 3 reads for a complete message (being Header, payload length and payload), nor does the Wireshark trace show any message attempted immediately before the disconnect. The last client message was appropriately responded to by the service. Any one have any hints on where to go hunting for answers? Addendum: This is TCP.
062142174041062102