A problem while transferring data over the NetworkStream
-
Hi, I am trying to send a big file over a TCP connection using NetworkStream and TCPClient. Both the client and the server is written in C#. There is no problem while transferring small files around the size of 300KB. My buffer size is 2KB and here is how the protocol works. -Server requests the file. -Client tells the server the size of the file, and then sends the first 2KB of it. -The server asks for another chunk of the file -like, send 2KB data, starting from 42649- -Client sends the chunk -Server asks for more, and client sends it... -When the end of file is reached, the client acknowledges the server that transfer is complete. Without looking at the code, since it is too long to be posted here, can you see any possible problems that may be causing this problem? I even tried sleeping the thread while receiving and processing the incoming chunks of data. Regards
Since it works for small files, and appears to be deterministic (does it always fail at the same point? or is it just more likely that larger files will fail?), it must be to do with filling an array, running out of space in a TCP buffer or something similar. If it is not so simple (i.e. it fails more on larger files, but it is not deterministic) then it is probably to do with the handshake to request more data. It is impossible to go into any more detail than that with the information available. I can only echo the advice to sprinkle your code liberally with logging statements to find out what is actually happening. Why are you doing that, anyway? TCP guarantees the ordering and transmission so you can just send the whole file on the sender end. What you are doing adds latency (2 way ping) each chunk.
-
You have no mechanism in your protocol to force either the client or server to force another request for the same data. If, after a timeout expires, the client doesn't get an ACK from the server, it should be able to try and reestablish communication, telling the server it's going to resend a block of data.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak -
TCP guarantees that the sent packets will arrive, unless the connection drops entirely (which the OP would have noticed), so I doubt that this is the problem.
The connection is still intact, it doesn't drop. The only problem is that the server stops requesting chunks.
-
Since it works for small files, and appears to be deterministic (does it always fail at the same point? or is it just more likely that larger files will fail?), it must be to do with filling an array, running out of space in a TCP buffer or something similar. If it is not so simple (i.e. it fails more on larger files, but it is not deterministic) then it is probably to do with the handshake to request more data. It is impossible to go into any more detail than that with the information available. I can only echo the advice to sprinkle your code liberally with logging statements to find out what is actually happening. Why are you doing that, anyway? TCP guarantees the ordering and transmission so you can just send the whole file on the sender end. What you are doing adds latency (2 way ping) each chunk.
I can't say that it is deterministic since if I pause the program at certain intervals, it can complete the transmission. If I don't pause the program at all, it stops after transferring 200-300KB. I am aware that what I am doing is just slowing down the transfer rate, but I am new to this are of programming and currently trying to develop this project to increase my experience. I will now add a timeout system to the server and request the rest of the data if nothing is incoming.
-
TCP guarantees that the sent packets will arrive, unless the connection drops entirely (which the OP would have noticed), so I doubt that this is the problem.
I know. It works great too, on paper. In actual practice, it's not 100% accurate.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak -
I can't say that it is deterministic since if I pause the program at certain intervals, it can complete the transmission. If I don't pause the program at all, it stops after transferring 200-300KB. I am aware that what I am doing is just slowing down the transfer rate, but I am new to this are of programming and currently trying to develop this project to increase my experience. I will now add a timeout system to the server and request the rest of the data if nothing is incoming.
SimpleData wrote:
I can't say that it is deterministic since if I pause the program at certain intervals, it can complete the transmission. If I don't pause the program at all, it stops after transferring 200-300KB.
As another thought - a firewall. It is throttling your throughput. When you pause it you are throttling yourself so it doesn't get involved. The same behavior could be due to some oddity in your server code though. Such as attempting to be clever by using threads for reading and writing.
-
SimpleData wrote:
I can't say that it is deterministic since if I pause the program at certain intervals, it can complete the transmission. If I don't pause the program at all, it stops after transferring 200-300KB.
As another thought - a firewall. It is throttling your throughput. When you pause it you are throttling yourself so it doesn't get involved. The same behavior could be due to some oddity in your server code though. Such as attempting to be clever by using threads for reading and writing.
Can you elaborate on "attempting to be clever by using threads for reading and writing"? Because it sounds like something I might have done. :)
-
Can you elaborate on "attempting to be clever by using threads for reading and writing"? Because it sounds like something I might have done. :)
-
To use two threads something needs to control access to the shared data. If you mess that control up then you can end up blocking or taking a large amount of time.
Wouldn't using the "lock" directive be enough?
-
Wouldn't using the "lock" directive be enough?