TCP client/server data exchange
-
TCP is a stream based protocol - it only understands a stream of bytes. It's up to you to parse the bytes into meaningful sequences. You need to design a protocol on top of TCP so the receiving end knows how many bytes to expect. This could be a fixed-length "packet" scheme. A simple, more effiicient scheme is to pass the data length first (for example, a 4-byte int - don't forget byte order issues!) before passing the data. The receiving end first receives the count, then it knows how many bytes follow. Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
Thanks Mark. I thought so... What I am currently doing is receiving text seperated by ";" and parse them appropriately..
-
Thanks Mark. I thought so... What I am currently doing is receiving text seperated by ";" and parse them appropriately..
How do you do that efficiently without knowing how many bytes to receive? Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
In addition to what Mark said, don't expect your Receive call (whatever your particular API is - recv in plain old sockets) to receive all the bytes sent in a single call by the Send. You might get "hel" then "lohelloa" then "gain" in consecutive calls. You must code your receiving logic to handle rebuilding the entire packet that Mark is talking about. Judy
Hi JudyL_FL, I am not sure if I understand what do you mean, by: "You must code your receiving logic to handle rebuilding the entire packet that Mark is talking about." I am just using one
recv(), so you mean that it is possible that I just get: "hell" and nothing else? Note, I am callingrecv()only once. Thanks. -
How do you do that efficiently without knowing how many bytes to receive? Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
I just assume that the maximum length of text should be 512 bytes, for example. Not sure if it is very efficient :)
-
I just assume that the maximum length of text should be 512 bytes, for example. Not sure if it is very efficient :)
bigdenny200 wrote:
Not sure if it is very efficient
I would imagine it is not :) At some point, the TCP protocol has to decide it's waited long enough for 512 bytes and just return what it's received so far. I'm not sure what that timeout value is (or how it decides) but it's certainly less efficient than knowing the exact count to recv. Also, make sure you follow Judy's advice... it is theoretically possible for recv() to receive one byte at a time! Your recv() logic MUST be in a loop, or allow for partial receives some other way. Another reason knowing the exact length to receive is good :) Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
Hi JudyL_FL, I am not sure if I understand what do you mean, by: "You must code your receiving logic to handle rebuilding the entire packet that Mark is talking about." I am just using one
recv(), so you mean that it is possible that I just get: "hell" and nothing else? Note, I am callingrecv()only once. Thanks.bigdenny200 wrote:
I am just using one recv(), so you mean that it is possible that I just get: "hell" and nothing else? Note, I am calling recv() only once.
Exactly. There is no guarantee that a single
recvcall on one end of a connection will get all the data sent with a singlesendcall on the other end. Looking at the reply you sent to Mark, you need to keep callingrecvuntil you get your "end of packet" marker ;. Handle that packet somewhere somewhen and repeat the process. Judy -
I just assume that the maximum length of text should be 512 bytes, for example. Not sure if it is very efficient :)
You are best off by buffering the incoming data, and parse from it. I recommend a ring buffer, preferable one which can hold the biggest possible command (otherwise you won't be able to parse a complete sentence). Whatever you do, don't read one byte at a time. It is horrendously inefficient.
-- Kein Mitleid Für Die Mehrheit
-
bigdenny200 wrote:
Not sure if it is very efficient
I would imagine it is not :) At some point, the TCP protocol has to decide it's waited long enough for 512 bytes and just return what it's received so far. I'm not sure what that timeout value is (or how it decides) but it's certainly less efficient than knowing the exact count to recv. Also, make sure you follow Judy's advice... it is theoretically possible for recv() to receive one byte at a time! Your recv() logic MUST be in a loop, or allow for partial receives some other way. Another reason knowing the exact length to receive is good :) Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
Dear Mark, I got a bit confused. Let me ask you one thing. In previous replies you mention, "a fixed-length "packet" scheme". For example I want to send text: "hiu","bye","guy" seperately. Ans assume I call them using three respective
sendcalls. The server knows it should get 3 bytes at a time. So I havewhile ( i = recv(buff, 3,..) )
{...}My question is, does this way guarantee me that first time
bufferwill contain "hiu", second time "bye", etc. ? Also, what do you mean by: "a 4-byte int - don't forget byte order issues!" In your previous reply :) ? As I can see I am not of appropriate knowledge :( :) Thanks -
bigdenny200 wrote:
I am just using one recv(), so you mean that it is possible that I just get: "hell" and nothing else? Note, I am calling recv() only once.
Exactly. There is no guarantee that a single
recvcall on one end of a connection will get all the data sent with a singlesendcall on the other end. Looking at the reply you sent to Mark, you need to keep callingrecvuntil you get your "end of packet" marker ;. Handle that packet somewhere somewhen and repeat the process. JudyJudyL_FL wrote:
There is no guarantee that a single recv call on one end of a connection will get all the data sent with a single send call on the other end.
It is possible with fixed sized messages, and blocking sockets. Although, I don't think the message sizes are fixed, because he mentions parsing messages, separated with a ;
-- Kein Mitleid Für Die Mehrheit
-
You are best off by buffering the incoming data, and parse from it. I recommend a ring buffer, preferable one which can hold the biggest possible command (otherwise you won't be able to parse a complete sentence). Whatever you do, don't read one byte at a time. It is horrendously inefficient.
-- Kein Mitleid Für Die Mehrheit
Thats what I am doing, but as Mark suggested, the TCP timeout can stop receiving the data, if the buffer is too big? Did I misundertand something ?
-
JudyL_FL wrote:
There is no guarantee that a single recv call on one end of a connection will get all the data sent with a single send call on the other end.
It is possible with fixed sized messages, and blocking sockets. Although, I don't think the message sizes are fixed, because he mentions parsing messages, separated with a ;
-- Kein Mitleid Für Die Mehrheit
I could fix their lengths Jorgen, i.e. if I have "bye" I could remake it to "bye**" and remove two stars later. here block size would be 5. Is that what you mean ? Thanks
-
Dear Mark, I got a bit confused. Let me ask you one thing. In previous replies you mention, "a fixed-length "packet" scheme". For example I want to send text: "hiu","bye","guy" seperately. Ans assume I call them using three respective
sendcalls. The server knows it should get 3 bytes at a time. So I havewhile ( i = recv(buff, 3,..) )
{...}My question is, does this way guarantee me that first time
bufferwill contain "hiu", second time "bye", etc. ? Also, what do you mean by: "a 4-byte int - don't forget byte order issues!" In your previous reply :) ? As I can see I am not of appropriate knowledge :( :) Thanksbigdenny200 wrote:
you mention, "a fixed-length "packet" scheme"
By that, I meant, for example, a structure (which is fixed length) - since the sender sends a structure, the receiver knows to receive sizeof(thestructure) bytes. This is inefficient if the structure isn't always fully used, since there will be needless extra bytes sent across the network.
bigdenny200 wrote:
The server knows it should get 3 bytes at a time. So I have while ( i = recv(buff, 3,..) ) {...}
The problem here (that Judy was referring to) is that a recv() call is successful even if only ONE byte is received. Even though the sender sent 3 bytes, that doesn't mean the receiver receives 3 bytes. TCP guarantees you'll eventually get the 3 bytes, in order, but there's no guarantee they will come in one call to recv(). Remember, TCP only knows bytes. Here's an example, with some error checking, of a way to receive a known number of bytes into a BYTE buffer...
int BytesRemaining = # of expected bytes;
BYTE *pCurBuf = start address of buffer to recv bytes into;while (BytesRemaining > 0)
{
int CurBytesReceived = ::recv(hSocket, (char*)pCurBuf, BytesRemaining, 0);if (CurBytesReceived == 0)
{
// connection was gracefully closedbreak;
}
else if (nCurBytesReceived == SOCKET_ERROR)
{
// Some error occurred!int rc = WSAGetLastError();
if (rc != WSAEWOULDBLOCK)
{
break;
}
}
else
{
// Some bytes were successfully received&nb
-
JudyL_FL wrote:
There is no guarantee that a single recv call on one end of a connection will get all the data sent with a single send call on the other end.
It is possible with fixed sized messages, and blocking sockets. Although, I don't think the message sizes are fixed, because he mentions parsing messages, separated with a ;
-- Kein Mitleid Für Die Mehrheit
Joergen Sigvardsson wrote:
It is possible with fixed sized messages
Not guaranteed. Even with fixed sized messages, there's NO guarantee the TCP protocol won't deliver it in different size chunks. Only a datagram protocol guarantees this. :) Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
bigdenny200 wrote:
you mention, "a fixed-length "packet" scheme"
By that, I meant, for example, a structure (which is fixed length) - since the sender sends a structure, the receiver knows to receive sizeof(thestructure) bytes. This is inefficient if the structure isn't always fully used, since there will be needless extra bytes sent across the network.
bigdenny200 wrote:
The server knows it should get 3 bytes at a time. So I have while ( i = recv(buff, 3,..) ) {...}
The problem here (that Judy was referring to) is that a recv() call is successful even if only ONE byte is received. Even though the sender sent 3 bytes, that doesn't mean the receiver receives 3 bytes. TCP guarantees you'll eventually get the 3 bytes, in order, but there's no guarantee they will come in one call to recv(). Remember, TCP only knows bytes. Here's an example, with some error checking, of a way to receive a known number of bytes into a BYTE buffer...
int BytesRemaining = # of expected bytes;
BYTE *pCurBuf = start address of buffer to recv bytes into;while (BytesRemaining > 0)
{
int CurBytesReceived = ::recv(hSocket, (char*)pCurBuf, BytesRemaining, 0);if (CurBytesReceived == 0)
{
// connection was gracefully closedbreak;
}
else if (nCurBytesReceived == SOCKET_ERROR)
{
// Some error occurred!int rc = WSAGetLastError();
if (rc != WSAEWOULDBLOCK)
{
break;
}
}
else
{
// Some bytes were successfully received&nb
Dear Mark, thank you for your (very) helpful replies. I will try to run the code tomorrow, and see what I get. Thanks for the other hint as well, but believe me I just prey this thing to work only for Windows :):)
-
Dear Mark, thank you for your (very) helpful replies. I will try to run the code tomorrow, and see what I get. Thanks for the other hint as well, but believe me I just prey this thing to work only for Windows :):)
bigdenny200 wrote:
I just prey
I would pray instead :)
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
Thats what I am doing, but as Mark suggested, the TCP timeout can stop receiving the data, if the buffer is too big? Did I misundertand something ?
With a buffer I mean that you should read in fixed sized chunks. Say 512 bytes per chunk (or less, if your sockets are non-blocking, in which you could end up with 0 bytes, should no bytes have been delivered to the reader). After you read data from the socket, parse/continue parsing the buffer. Once you find the packet you are looking for, remove it from the buffer, and keep the rest of the buffer (might be a partial packet).
-
I could fix their lengths Jorgen, i.e. if I have "bye" I could remake it to "bye**" and remove two stars later. here block size would be 5. Is that what you mean ? Thanks
Yes, but looking at it that way (with text based commands and all), it's smarter to take the buffered approach. That way you don't lock yourself down in case you need to change the protocol.
-
JudyL_FL wrote:
There is no guarantee that a single recv call on one end of a connection will get all the data sent with a single send call on the other end.
It is possible with fixed sized messages, and blocking sockets. Although, I don't think the message sizes are fixed, because he mentions parsing messages, separated with a ;
-- Kein Mitleid Für Die Mehrheit
-
Joergen Sigvardsson wrote:
It is possible with fixed sized messages
Not guaranteed. Even with fixed sized messages, there's NO guarantee the TCP protocol won't deliver it in different size chunks. Only a datagram protocol guarantees this. :) Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
He is right if it is a blocking socket. If you ask for x bytes in the
recvfunction, it will either deliver x bytes or return a timeout indication. JudyWhat's a blocking socket? ;P Thanks Judy!! Mark
Mark Salsbery Microsoft MVP - Visual C++ :java: