TCP client/server data exchange
-
JudyL_FL wrote:
There is no guarantee that a single recv call on one end of a connection will get all the data sent with a single send call on the other end.
It is possible with fixed sized messages, and blocking sockets. Although, I don't think the message sizes are fixed, because he mentions parsing messages, separated with a ;
-- Kein Mitleid Für Die Mehrheit
I could fix their lengths Jorgen, i.e. if I have "bye" I could remake it to "bye**" and remove two stars later. here block size would be 5. Is that what you mean ? Thanks
-
Dear Mark, I got a bit confused. Let me ask you one thing. In previous replies you mention, "a fixed-length "packet" scheme". For example I want to send text: "hiu","bye","guy" seperately. Ans assume I call them using three respective
sendcalls. The server knows it should get 3 bytes at a time. So I havewhile ( i = recv(buff, 3,..) )
{...}My question is, does this way guarantee me that first time
bufferwill contain "hiu", second time "bye", etc. ? Also, what do you mean by: "a 4-byte int - don't forget byte order issues!" In your previous reply :) ? As I can see I am not of appropriate knowledge :( :) Thanksbigdenny200 wrote:
you mention, "a fixed-length "packet" scheme"
By that, I meant, for example, a structure (which is fixed length) - since the sender sends a structure, the receiver knows to receive sizeof(thestructure) bytes. This is inefficient if the structure isn't always fully used, since there will be needless extra bytes sent across the network.
bigdenny200 wrote:
The server knows it should get 3 bytes at a time. So I have while ( i = recv(buff, 3,..) ) {...}
The problem here (that Judy was referring to) is that a recv() call is successful even if only ONE byte is received. Even though the sender sent 3 bytes, that doesn't mean the receiver receives 3 bytes. TCP guarantees you'll eventually get the 3 bytes, in order, but there's no guarantee they will come in one call to recv(). Remember, TCP only knows bytes. Here's an example, with some error checking, of a way to receive a known number of bytes into a BYTE buffer...
int BytesRemaining = # of expected bytes;
BYTE *pCurBuf = start address of buffer to recv bytes into;while (BytesRemaining > 0)
{
int CurBytesReceived = ::recv(hSocket, (char*)pCurBuf, BytesRemaining, 0);if (CurBytesReceived == 0)
{
// connection was gracefully closedbreak;
}
else if (nCurBytesReceived == SOCKET_ERROR)
{
// Some error occurred!int rc = WSAGetLastError();
if (rc != WSAEWOULDBLOCK)
{
break;
}
}
else
{
// Some bytes were successfully received&nb
-
JudyL_FL wrote:
There is no guarantee that a single recv call on one end of a connection will get all the data sent with a single send call on the other end.
It is possible with fixed sized messages, and blocking sockets. Although, I don't think the message sizes are fixed, because he mentions parsing messages, separated with a ;
-- Kein Mitleid Für Die Mehrheit
Joergen Sigvardsson wrote:
It is possible with fixed sized messages
Not guaranteed. Even with fixed sized messages, there's NO guarantee the TCP protocol won't deliver it in different size chunks. Only a datagram protocol guarantees this. :) Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
bigdenny200 wrote:
you mention, "a fixed-length "packet" scheme"
By that, I meant, for example, a structure (which is fixed length) - since the sender sends a structure, the receiver knows to receive sizeof(thestructure) bytes. This is inefficient if the structure isn't always fully used, since there will be needless extra bytes sent across the network.
bigdenny200 wrote:
The server knows it should get 3 bytes at a time. So I have while ( i = recv(buff, 3,..) ) {...}
The problem here (that Judy was referring to) is that a recv() call is successful even if only ONE byte is received. Even though the sender sent 3 bytes, that doesn't mean the receiver receives 3 bytes. TCP guarantees you'll eventually get the 3 bytes, in order, but there's no guarantee they will come in one call to recv(). Remember, TCP only knows bytes. Here's an example, with some error checking, of a way to receive a known number of bytes into a BYTE buffer...
int BytesRemaining = # of expected bytes;
BYTE *pCurBuf = start address of buffer to recv bytes into;while (BytesRemaining > 0)
{
int CurBytesReceived = ::recv(hSocket, (char*)pCurBuf, BytesRemaining, 0);if (CurBytesReceived == 0)
{
// connection was gracefully closedbreak;
}
else if (nCurBytesReceived == SOCKET_ERROR)
{
// Some error occurred!int rc = WSAGetLastError();
if (rc != WSAEWOULDBLOCK)
{
break;
}
}
else
{
// Some bytes were successfully received&nb
Dear Mark, thank you for your (very) helpful replies. I will try to run the code tomorrow, and see what I get. Thanks for the other hint as well, but believe me I just prey this thing to work only for Windows :):)
-
Dear Mark, thank you for your (very) helpful replies. I will try to run the code tomorrow, and see what I get. Thanks for the other hint as well, but believe me I just prey this thing to work only for Windows :):)
bigdenny200 wrote:
I just prey
I would pray instead :)
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
Thats what I am doing, but as Mark suggested, the TCP timeout can stop receiving the data, if the buffer is too big? Did I misundertand something ?
With a buffer I mean that you should read in fixed sized chunks. Say 512 bytes per chunk (or less, if your sockets are non-blocking, in which you could end up with 0 bytes, should no bytes have been delivered to the reader). After you read data from the socket, parse/continue parsing the buffer. Once you find the packet you are looking for, remove it from the buffer, and keep the rest of the buffer (might be a partial packet).
-
I could fix their lengths Jorgen, i.e. if I have "bye" I could remake it to "bye**" and remove two stars later. here block size would be 5. Is that what you mean ? Thanks
Yes, but looking at it that way (with text based commands and all), it's smarter to take the buffered approach. That way you don't lock yourself down in case you need to change the protocol.
-
JudyL_FL wrote:
There is no guarantee that a single recv call on one end of a connection will get all the data sent with a single send call on the other end.
It is possible with fixed sized messages, and blocking sockets. Although, I don't think the message sizes are fixed, because he mentions parsing messages, separated with a ;
-- Kein Mitleid Für Die Mehrheit
-
Joergen Sigvardsson wrote:
It is possible with fixed sized messages
Not guaranteed. Even with fixed sized messages, there's NO guarantee the TCP protocol won't deliver it in different size chunks. Only a datagram protocol guarantees this. :) Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
He is right if it is a blocking socket. If you ask for x bytes in the
recvfunction, it will either deliver x bytes or return a timeout indication. JudyWhat's a blocking socket? ;P Thanks Judy!! Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
What's a blocking socket? ;P Thanks Judy!! Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
He is right if it is a blocking socket. If you ask for x bytes in the
recvfunction, it will either deliver x bytes or return a timeout indication. JudyWell Judy, I had to test this before I respectfully disagreed with you :) I respectfully disagree... A blocking Recv() WILL return before all bytes requested are received. Here's the test I did - note I try to recv 512 bytes but only send 256...
//----------------------------------------------
// Listener/ receiver thread (started in CMYTestDlg::OnOK() below)
//----------------------------------------------
UINT __cdecl TestThreadProc( LPVOID pParam )
{
SOCKET ListenSocket;
ListenSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (ListenSocket == INVALID_SOCKET)
{
return 1;
}sockaddr_in service;
service.sin_family = AF_INET;
service.sin_addr.s_addr = inet_addr("127.0.0.1");
service.sin_port = htons(2460);if (bind( ListenSocket, (SOCKADDR*) &service, sizeof(service)) == SOCKET_ERROR)
{
closesocket(ListenSocket);
return 1;
}if (listen( ListenSocket, 1 ) == SOCKET_ERROR)
{
closesocket(ListenSocket);
return 1;
}SOCKET AcceptSocket;
AcceptSocket = accept( ListenSocket, NULL, NULL );
if (AcceptSocket == INVALID_SOCKET)
{
closesocket(ListenSocket);
return 1;
}BYTE *pBuf = new BYTE[512];
BYTE *pCurBuf = pBuf;
int BytesRemaining = 512;while (BytesRemaining > 0)
{
int CurBytesReceived = ::recv(AcceptSocket, (char*)pCurBuf, BytesRemaining, 0);if (CurBytesReceived == 0) //<-- put a breakpoint here - 256 bytes will be received
{
// connection was gracefully closedbreak;
}
e -
Well Judy, I had to test this before I respectfully disagreed with you :) I respectfully disagree... A blocking Recv() WILL return before all bytes requested are received. Here's the test I did - note I try to recv 512 bytes but only send 256...
//----------------------------------------------
// Listener/ receiver thread (started in CMYTestDlg::OnOK() below)
//----------------------------------------------
UINT __cdecl TestThreadProc( LPVOID pParam )
{
SOCKET ListenSocket;
ListenSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (ListenSocket == INVALID_SOCKET)
{
return 1;
}sockaddr_in service;
service.sin_family = AF_INET;
service.sin_addr.s_addr = inet_addr("127.0.0.1");
service.sin_port = htons(2460);if (bind( ListenSocket, (SOCKADDR*) &service, sizeof(service)) == SOCKET_ERROR)
{
closesocket(ListenSocket);
return 1;
}if (listen( ListenSocket, 1 ) == SOCKET_ERROR)
{
closesocket(ListenSocket);
return 1;
}SOCKET AcceptSocket;
AcceptSocket = accept( ListenSocket, NULL, NULL );
if (AcceptSocket == INVALID_SOCKET)
{
closesocket(ListenSocket);
return 1;
}BYTE *pBuf = new BYTE[512];
BYTE *pCurBuf = pBuf;
int BytesRemaining = 512;while (BytesRemaining > 0)
{
int CurBytesReceived = ::recv(AcceptSocket, (char*)pCurBuf, BytesRemaining, 0);if (CurBytesReceived == 0) //<-- put a breakpoint here - 256 bytes will be received
{
// connection was gracefully closedbreak;
}
eCan you try calling WSAGetLastError after you receive the partial bytes? I'm curious if it is set to show the timeout. According to the documentation,
recvshould either read all or timeout. The docs do say: Windows Sockets 2 does not define any standard blocking time-out mechanism for this function I think we're hitting this case - the receive has timed out and who knows how the function is reporting the timeout. I think we can safely say the following: For a blocking socket, you need to check the result from the recv and if you don't get the requested number of bytes, you have hit the timeout condition and need to handle it accordingly. Judy -
Can you try calling WSAGetLastError after you receive the partial bytes? I'm curious if it is set to show the timeout. According to the documentation,
recvshould either read all or timeout. The docs do say: Windows Sockets 2 does not define any standard blocking time-out mechanism for this function I think we're hitting this case - the receive has timed out and who knows how the function is reporting the timeout. I think we can safely say the following: For a blocking socket, you need to check the result from the recv and if you don't get the requested number of bytes, you have hit the timeout condition and need to handle it accordingly. JudyJudyL_FL wrote:
Can you try calling WSAGetLastError after you receive the partial bytes?
Sure :) WSAGetLastError() returns 0; A long time ago, while writing some of my first communication code, I got bit by this (using pipes actually, but it made me look at my sockets code) and it taught me to read the documentation thoroughly (which I wasn't doing apparently :)). That's when I found this: "For connection-oriented sockets (type SOCK_STREAM for example), calling recv will return as much data as is currently available—up to the size of the buffer specified." FWIW, This also applies to pipes. So, any amount of bytes returned by recv() indicates success. This can be seen especially when receiving amounts larger than the socket buffer, where the protocol MUST return a buffer full of data to make room to receive more. This seems to be the number one thing overlooked by beginner socket programmers. I studied MS's implementation of TCP a while back, and there is a timeout involved internally, but it's not an error of any kind. I believe by default it's 200ms (that's the number that comes to mind) before a recv() returns what it has currently buffered. That's what makes it inefficient to always request more bytes than you're expecting. Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
Can you try calling WSAGetLastError after you receive the partial bytes? I'm curious if it is set to show the timeout. According to the documentation,
recvshould either read all or timeout. The docs do say: Windows Sockets 2 does not define any standard blocking time-out mechanism for this function I think we're hitting this case - the receive has timed out and who knows how the function is reporting the timeout. I think we can safely say the following: For a blocking socket, you need to check the result from the recv and if you don't get the requested number of bytes, you have hit the timeout condition and need to handle it accordingly. JudyMark Salsbery wrote:
and it taught me to read the documentation thoroughly
BTW, I wasn't implying that you don't read the docs - I meant that was a turning point for me, which is why I remember it vividly :) Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
JudyL_FL wrote:
Can you try calling WSAGetLastError after you receive the partial bytes?
Sure :) WSAGetLastError() returns 0; A long time ago, while writing some of my first communication code, I got bit by this (using pipes actually, but it made me look at my sockets code) and it taught me to read the documentation thoroughly (which I wasn't doing apparently :)). That's when I found this: "For connection-oriented sockets (type SOCK_STREAM for example), calling recv will return as much data as is currently available—up to the size of the buffer specified." FWIW, This also applies to pipes. So, any amount of bytes returned by recv() indicates success. This can be seen especially when receiving amounts larger than the socket buffer, where the protocol MUST return a buffer full of data to make room to receive more. This seems to be the number one thing overlooked by beginner socket programmers. I studied MS's implementation of TCP a while back, and there is a timeout involved internally, but it's not an error of any kind. I believe by default it's 200ms (that's the number that comes to mind) before a recv() returns what it has currently buffered. That's what makes it inefficient to always request more bytes than you're expecting. Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
Mark Salsbery wrote:
WSAGetLastError() returns 0;
I was curious - it was easier to have you check since you had working code. How long does your recv sit before it returns? Never mind, it's not important. I vaguely remember some socket I did a looooong while back that (I think) waited a whole lot longer than your 200ms. Too far in the past to remember the details or if I had set a timeout value or it was using the built-in one. This may have been on CE, now that I think about it and that's a whole 'nother kettle of sockets. If it wasn't CE, it was long enough ago that it was at least as old as NT4. I think we have our answer - always check the number of bytes received :-D Judy
-
Mark Salsbery wrote:
and it taught me to read the documentation thoroughly
BTW, I wasn't implying that you don't read the docs - I meant that was a turning point for me, which is why I remember it vividly :) Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
Mark Salsbery wrote:
WSAGetLastError() returns 0;
I was curious - it was easier to have you check since you had working code. How long does your recv sit before it returns? Never mind, it's not important. I vaguely remember some socket I did a looooong while back that (I think) waited a whole lot longer than your 200ms. Too far in the past to remember the details or if I had set a timeout value or it was using the built-in one. This may have been on CE, now that I think about it and that's a whole 'nother kettle of sockets. If it wasn't CE, it was long enough ago that it was at least as old as NT4. I think we have our answer - always check the number of bytes received :-D Judy
JudyL_FL wrote:
was curious - it was easier to have you check since you had working code.
I knew that hehe.
JudyL_FL wrote:
How long does your recv sit before it returns?
About a second (accurate, huh?) from when I hit the OK button - that includes the 1000ms Sleep() in my fancy thread synchronization. I agree it doesn't matter - the internal "timeout" is implementation specific. Different protocol stack vendors are free to implement the internals the way they want. That's why I only have MS specific info (and even that is vague now). recv() API behavior, however, is well defined.
JudyL_FL wrote:
I think we have our answer - always check the number of bytes received
Agreed! :beer: Mark
Mark Salsbery Microsoft MVP - Visual C++ :java:
-
no offense taken (especially since I quoted the docs a reply or so back), have a .... wait, where's the beer emoticon??? aaaargh Judy
:beer: :beer: :beer: :)
Mark Salsbery Microsoft MVP - Visual C++ :java: