Winsock send FiFo Size

ScotDolan

I am developing Socket application that is going to send packet across network. The data packets contain, STX, Lenght, Opcode, Type, Qualifier, Data[128], CRC, ETX. I was hoping to be able to send the packet via send command. However, I might run into a problem where the que is full and only half my packet goes out. This would cause a sync problem. Is there any way, i can verify how much space is available in the outgoing write buffer. if( TxQueue.Pop( &temp_packet ) == true ) { iResults = send( client_socket, (char *)&temp_packet, sizeof(temp_packet), NULL ); if( iResults != sizeof(temp_packet) ) Error = -1; }

Scott Dolan Jernie Corporation Engineering & Manufacturing Software, Hardware, & Enclosures

Mark Salsbery

ScotDolan wrote:

Is there any way, i can verify how much space is available in the outgoing write buffer

Not that I know of. It looks like your data is already queued for sending which is good IMO. I'm assuming you are using non-blocking socket (or it wouldn't be an issue). Maybe changing the send logic a bit will help...

if( TxQueue.Pop( &temp_packet ) == true )
{
BYTE *pCurSendByte = (BYTE *)&temp_packet;
int iBytesRemaining = sizeof(temp_packet);

while (iBytesRemaining > 0)
{
iResults = send( client_socket, (char *)pCurSendByte, iBytesRemaining, NULL );
if( iResults != SOCKET_ERROR )
{
iBytesRemaining -= iResults;
if (iBytesRemaining > 0)
{
pCurSendByte += iResults;
}
}
else
{
int nWSAErrorCode = WSAGetLastError();
if (nWSAErrorCode != WSAEWOULDBLOCK)
{
Error = -1;
break;
}
}
}
}

This is fairly ok on a separate thread but more efficient would be to use an event object with WSAEventSelect() to wait on FD_WRITE whenever send() doesn't write all the bytes (or returns WSAEWOULDBLOCK error) in one shot. Takes a few extra lines of code :) -- modified at 16:37 Thursday 8th February, 2007 *EDIT* Added WSAEWOULDBLOCK error handling :doh:

Mark Salsbery

Also, if using TCP, it's important to receive data efficiently on the other end. send() depends on the receiving end having buffer room as well! :)

Jorgen Sigvardsson

Assuming it's TCP, why not just resend whatever send() couldn't send? send() returns the number of bytes it managed to send... :~

-- Verletzen zerfetzen zersetzen zerstören Doch es darf nicht mir gehören Ich muss zerstören

Mike ONeill

If you want to use TCP, then you must write your code to accommodate partial sends and recvs, and also to accommodate multiple sends and recvs. There is no way around this. TCP is reliable but it's stream-based, not message-based. This means that TCP views the data as a simple stream of singular bytes, and it's up to the appication to decide what the bytes mean. Even if you could somehow check for availability of room in the send buffer, that information is completely unrelated to how the stack will ultimately send data out over the wire, and of course is totally unrelated to how the recipient stack will re-assemble data at the receiving side. UDP, on the other hand, is message-based. It will definitely preserve "messages" or "packets". However, unlike TCP, UDP is unreliable, in the sense that either the entire message will be delivered, or nothing at all will be delivered. You need to decide on your requirements. Mike

ScotDolan

Mark, That code basiclly makes the send function into a blocking function. I am try to develop a real-time socket applciation(As best as possible). Right now, the send and recv are in a single worker thread. The worker thread also maintains the status of the connection. If disconnects occures, thread will automaticlly reattempt to connect and update status byte within the class notifying calling function the socket is disconnected. The Blocking function does not work in my application because, i will be waiting on write, when i could receiving data. This all said, I thinking I might have to create two or three worker threads within this socket class I am developing. One thread to recv packet and place in the safeque. One Thread to send packet from safeque. One Thread to check for FD_CLOSE and FD_CONNECTION events. Do you think the three thread class would work.

Scott Dolan Jernie Corporation Engineering & Manufacturing Software, Hardware, & Enclosures

ScotDolan

Mike, From my understanding, all the UDP data grams are recieved into a single recieve FIFO. There is no way within winsock to pull packets out of the buffers. And even if you could pull packets out of buffer, there is no garantee that all the packets are of the same size as application/someone might placed a unexpected packet..

Scott Dolan Jernie Corporation Engineering & Manufacturing Software, Hardware, & Enclosures

Mark Salsbery

ScotDolan wrote:

This all said, I thinking I might have to create two or three worker threads within this socket class I am developing. One thread to recv packet and place in the safeque. One Thread to send packet from safeque. One Thread to check for FD_CLOSE and FD_CONNECTION events.

That works but for even more throughput with minimal threads - one thread for sends and recvs, one for queueing data for send, and one for handling queued received data works well. The problem with multiple threads sending and receiving is that it's not guaranteed to be safe on the same socket. You end up blocking to wait for the other anyway so having more than one thread for recv and send is IMO unnecessary. In my socket I/O thread loops, I have an array of events - 1 for termination, 1 for queued send data, and one for FD_READ/FD_CLOSE events from the socket. Using WaitForMultipleObjects() I wait on those events (outer loop). When an event is signalled check for terminate (exiting the thread if signalled), check if FD_READ and if so loop until WSAWOULDBLOCK error, if queued send data then send like my sample code. After a send check for more queued send packets and set the event if necessary so next loop will catch it immediately. It works well for real time streaming data. :)

ScotDolan

I think i understand you. You basicly wait for events Once you get a read event you start a thread to handle that event and read the fifo. But, before you start that thread you make sure the past current read thead has terminate. I like the sounds of this socket. Do you have it posted up here. Send me a copy. Please.... Right now, I am going to try simpler versiion of that, I am going to do all my reads and write in the same thread. It will attempt to write one packet, than read one packet and it will never block.. But, I am betting this will not working my application. My new code below.. Let case LOOP_CLIENT: WSAEnumNetworkEvents(client_socket, g_hClientEvent, &NetworkClientEvents); if( NetworkClientEvents.lNetworkEvents == FD_CLOSE) { printf("Client Socket Closed Message: \n\r"); WSACloseEvent(g_hClientEvent); closesocket(client_socket); WSACleanup(); state = CONNECT_CLIENT; } if( NetworkClientEvents.lNetworkEvents == FD_CONNECT) { if (0 != NetworkClientEvents.iErrorCode[FD_CONNECT_BIT]) { printf("Client Connection Failed: \n\r"); state = CONNECT_CLIENT; }else{ printf("Client Connected Message: \n\r"); state = LOOP_CLIENT; } } // Transmitts Packets Across the Newtork if( txbytesremaining == 0 ) { if( TxQueue.Pop( &temp_txpacket) == true) { temp_txbuffer = (char *)&temp_txpacket; int txbytesremaining = sizeof(temp_txbuffer); iResults = send( client_socket, (char *)temp_txbuffer, txbytesremaining, NULL ); if( iResults != SOCKET_ERROR ) { txbytesremaining -= iResults; if (txbytesremaining > 0) temp_txbuffer += iResults; //Increments the Byte Pointer }else{ //ERROR break; } } }else{ iResults = send( client_socket, (char *)temp_txbuffer, txbytesremaining, NULL ); if( iResults != SOCKET_ERROR ) { txbytesremaining -= iResults; if (txbytesremaining > 0) temp_txbuffer += iResults; //Increments the Byte Pointer }else{ break;//ERRROR } } // Receive Packets Across the Newtork if( rxbytesremaining == 0 ) rxbytesremaining = sizeof(temp_rxbuffer); iResults = recv( client_socket, (char *)temp_rxbuffer, rxbytesremaining, NULL ); switch( iResults ) { case 0: break; case SOCKET_ERROR: break; default: rxbytesremaining -= iResul

Mark Salsbery

ScotDolan wrote:

You basicly wait for events Once you get a read event you start a thread to handle that event and read the fifo. But, before you start that thread you make sure the past current read thead has terminate.

No actually I'm talking about waiting on events when already in the send/recv thread. For good throughput using TCP it's important to keep the socket send buffers full and the recv buffers empty. The socket should never have to wait for the app to receive data from its buffers otherwise communication stops. Of course, when there's no data to send that's fine - that means everything's caught up. Using the method below, the thread deals solely with sending and receiving on the socket. Data to send is queued by another thread, which sets the datatosendevent. Received data is immediately queued and an event is set indicating to another thread that data has been received for processing. Those other threads can take as much time processing as they want - the point is the send/recv thread keeps the data flowing with minimal waiting. Also, being completely event driven, there's little wasted CPU cycles.

// Socket I/O thread proc pseudocode

while (1)
{
WaitMultipleObjects() on terminateevent, datatosendevent, socketevent(FD_READ/FD_CLOSE)

if (terminateevent)
break;

if (socketevent)
{
if (FD_CLOSE event)
{
notify app that remote socket closed
break;
}
else //must be FD_READ
{
loop calling recv() (queueing complete packets)
until WSAWOULDBLOCK error
Notify recv handler thread using an event object when received data available
}
}

if (datatosendevent)
{
dequeue a send packet
loop calling send()
until WSAWOULDBLOCK error or no more queued send data
if WSAWOULDBLOCK and still data on send queue then set datatosendevent
}
}

return (exit thread)