One last IOCP question :)
-
Alright, school is up back to the IOCP server :) Everything works near to fine now, thanks to the good help i got from this site. Anyways, there is one small problem/bug/design flaw left. I currently found a way around it but i dont like having ugly code with a way uglier solution. Ok again im still using the IOCP with UDP. Anyways, The problem is that i get trash in my completion ports. for example, i send something which goes fine by the way. After that i do a WSARecvFrom. Then i retrieve a IOCP notification pointing to the right extended overlapped. Now, the data inside is somethimes the right data i expected to recieve with the right size like 600 bytes, but somethimes it returns a size of 1 and no data unless i give it some time. So what do i do, i just continue and recieve a next notificication when recieving 1. this next notification contains the right data + size, so my first question, why did i get notified about my recv request before he comepletely recieved the data. a second problem, whenever i had the right data, it notifies me for like 2 extra times about it before nocking it off. Anyone knows a NICE way to prevent this because its really ugly. Thanks already!
-
Alright, school is up back to the IOCP server :) Everything works near to fine now, thanks to the good help i got from this site. Anyways, there is one small problem/bug/design flaw left. I currently found a way around it but i dont like having ugly code with a way uglier solution. Ok again im still using the IOCP with UDP. Anyways, The problem is that i get trash in my completion ports. for example, i send something which goes fine by the way. After that i do a WSARecvFrom. Then i retrieve a IOCP notification pointing to the right extended overlapped. Now, the data inside is somethimes the right data i expected to recieve with the right size like 600 bytes, but somethimes it returns a size of 1 and no data unless i give it some time. So what do i do, i just continue and recieve a next notificication when recieving 1. this next notification contains the right data + size, so my first question, why did i get notified about my recv request before he comepletely recieved the data. a second problem, whenever i had the right data, it notifies me for like 2 extra times about it before nocking it off. Anyone knows a NICE way to prevent this because its really ugly. Thanks already!
Can you show the code you are using to look at the send completion packets - from the GetQueuedCompletionStatus() call on? If it truly returned indicating one byte received then you'd have to keep that one byte and repost the recv for the remaining bytes. This doesn't make sense for UDP though, unless your buffer sizes are messed up when you call WSARecvFrom().
-
Can you show the code you are using to look at the send completion packets - from the GetQueuedCompletionStatus() call on? If it truly returned indicating one byte received then you'd have to keep that one byte and repost the recv for the remaining bytes. This doesn't make sense for UDP though, unless your buffer sizes are messed up when you call WSARecvFrom().
i stripped the functions in the main code so its easier to read. this is what happends. i got a list of virtual sessions, every virtual session can be used once at a time (logical) as soon as a virtual session is released, at will be reset to another host and a post the the IOCP will be made with the task set to Begin. I've checked and counted. it one virtual session, the enum will be set to Begin and a post will be made, then he calls the member function RetrieveServerInfo which sets the buffers associated with the virtual session only, set the enum to RequestSend which sends the data. this happends one time, then it goes to after send and then the request send. till now all those steps have been held 1 time per session. After that he goes to afterrecieve. this step he repeates for a few time, even if the session is released. it might be that i use the same session AND OVERLAPPED structure for the new server query.
while (pThis->m_bKeepAlive){ BOOL bIORet = GetQueuedCompletionStatus ( // Get a completed IO request pThis->m_hIOCompiltionPort, &dwIoSize, reinterpret_cast(pThis), &ptrOverlapped, INFINITE); ptrOverlappedPlus = reinterpret_cast<_OVERLAPPEDVirtualSession*>(ptrOverlapped); ptrSession = ptrOverlappedPlus->pClientContext; if (!bIORet){ __REPORT("Winsock error code: %i\n", GetLastError()); Gameye::Library::Thread::CMPThread::Sleep(0); } else{ CAbstractServerClass* ptrGameClass = (*pThis->m_ptrGameManager)[ptrSession->getNDServer()->GameID]; if (ptrGameClass == NULL){ __REPORT("Game is unknown, probably removed"); } else{ objToolset.Session = ptrSession; switch(ptrSession->getTask()){ case Begin: //Starts the procedure pThis->m_objAddGameToolset.Session = ptrSession; ptrGameClass->RequestServerInfo(ptrSession->getNDServer(), &pThis->m_objAddGameToolset); break; case RequestSend: //Request to send a packet dwIoSize = 0; iTemp = 0; ptrSession->setStatus(AfterSend); if (WSASendTo(ptrSession->getSocket(), ptrSession->getWSABuffer(), 1, &dwIoSize, 0,reinterpret_cast(ptrSession->getAddress()), sizeof(sockaddr_in), ptrSession->getOverlapped(), NULL) == -1) { int iErrorCode = WSAGetLastError(); if (iErrorCode == ERROR_IO_PENDING){ break; -
i stripped the functions in the main code so its easier to read. this is what happends. i got a list of virtual sessions, every virtual session can be used once at a time (logical) as soon as a virtual session is released, at will be reset to another host and a post the the IOCP will be made with the task set to Begin. I've checked and counted. it one virtual session, the enum will be set to Begin and a post will be made, then he calls the member function RetrieveServerInfo which sets the buffers associated with the virtual session only, set the enum to RequestSend which sends the data. this happends one time, then it goes to after send and then the request send. till now all those steps have been held 1 time per session. After that he goes to afterrecieve. this step he repeates for a few time, even if the session is released. it might be that i use the same session AND OVERLAPPED structure for the new server query.
while (pThis->m_bKeepAlive){ BOOL bIORet = GetQueuedCompletionStatus ( // Get a completed IO request pThis->m_hIOCompiltionPort, &dwIoSize, reinterpret_cast(pThis), &ptrOverlapped, INFINITE); ptrOverlappedPlus = reinterpret_cast<_OVERLAPPEDVirtualSession*>(ptrOverlapped); ptrSession = ptrOverlappedPlus->pClientContext; if (!bIORet){ __REPORT("Winsock error code: %i\n", GetLastError()); Gameye::Library::Thread::CMPThread::Sleep(0); } else{ CAbstractServerClass* ptrGameClass = (*pThis->m_ptrGameManager)[ptrSession->getNDServer()->GameID]; if (ptrGameClass == NULL){ __REPORT("Game is unknown, probably removed"); } else{ objToolset.Session = ptrSession; switch(ptrSession->getTask()){ case Begin: //Starts the procedure pThis->m_objAddGameToolset.Session = ptrSession; ptrGameClass->RequestServerInfo(ptrSession->getNDServer(), &pThis->m_objAddGameToolset); break; case RequestSend: //Request to send a packet dwIoSize = 0; iTemp = 0; ptrSession->setStatus(AfterSend); if (WSASendTo(ptrSession->getSocket(), ptrSession->getWSABuffer(), 1, &dwIoSize, 0,reinterpret_cast(ptrSession->getAddress()), sizeof(sockaddr_in), ptrSession->getOverlapped(), NULL) == -1) { int iErrorCode = WSAGetLastError(); if (iErrorCode == ERROR_IO_PENDING){ break;Reusing the overlapped struct is no problem. You just can't use it in more than one pending (uncompleted) overlapped operation at a time. I suggest carefully checking error conditions here. The documented results for GetQueuedCompletionStatus() should be handled something like this:
BOOL bIORet = GetQueuedCompletionStatus ( // Get a completed IO request
pThis->m_hIOCompiltionPort,
&dwIoSize,
reinterpret_cast(pThis),
&ptrOverlapped, INFINITE);(pseudocode)
if (bIORet)
{
if (was overlapped socket operation)
{
if (dwIoSize == 0)
{
// socket closed
}
else
{
if (AfterRecieve)
{
if (dwIoSize < numberofbytesrequested)
receive (numberofbytesrequested - dwIoSize) more bytes
else
// all bytes received
}
}
}
}
else
{
if (ptrOverlapped)
{
// I/O failed on the operation associated with this completion packet
// GetLastError() for extended error info
}
else
{
//GetQueuedCompletionStatus() failed
// GetLastError() for extended error info
}
}Can you post the code that calls WSARecvFrom, including the parameters passed and how you are checking for errors? Please use the "pre" tags instead of the "code" tags as well :)
-
Reusing the overlapped struct is no problem. You just can't use it in more than one pending (uncompleted) overlapped operation at a time. I suggest carefully checking error conditions here. The documented results for GetQueuedCompletionStatus() should be handled something like this:
BOOL bIORet = GetQueuedCompletionStatus ( // Get a completed IO request
pThis->m_hIOCompiltionPort,
&dwIoSize,
reinterpret_cast(pThis),
&ptrOverlapped, INFINITE);(pseudocode)
if (bIORet)
{
if (was overlapped socket operation)
{
if (dwIoSize == 0)
{
// socket closed
}
else
{
if (AfterRecieve)
{
if (dwIoSize < numberofbytesrequested)
receive (numberofbytesrequested - dwIoSize) more bytes
else
// all bytes received
}
}
}
}
else
{
if (ptrOverlapped)
{
// I/O failed on the operation associated with this completion packet
// GetLastError() for extended error info
}
else
{
//GetQueuedCompletionStatus() failed
// GetLastError() for extended error info
}
}Can you post the code that calls WSARecvFrom, including the parameters passed and how you are checking for errors? Please use the "pre" tags instead of the "code" tags as well :)
sure:
int Recieve(char\* buffer, size\_t buflen){ if (m\_iLastRecieved > 0){ //Were expecting a second packed m\_wsaBuf.len = static\_cast(buflen) - m\_iTotalRecieved; m\_wsaBuf.buf = buffer + m\_iTotalRecieved; m\_wsaBuf.buf\[0\] = 0; } else{ buffer\[0\] = 0; m\_wsaBuf.len = static\_cast(buflen); m\_wsaBuf.buf = buffer; } static int sockaddr\_len = sizeof(sockaddr\_in); DWORD temp(0); setStatus(RequestRecieve); //returning 0 is ok return (PostQueuedCompletionStatus(m\_hIOCP, 1, reinterpret\_cast(this), getOverlapped()) == FALSE); }Thats where i make my request to post. You might wonder if it aint dangerous for the buffer to overflow or something but most of the checking goes outside this function and i can assure that there is no overflow since i'm only testing with 1 game which is secured in any way i know off. The requested bytes is unknown, it can variate depending on the used game from like 100 to a few packets of 1500 kb however mostly its just something like 1 packet with 600 bytes (i know because my previous version of the application worked perfectly, only it wasnt using IOCP just the lame one thread per server way ) anyways, with this topic still being open and all, there is one other ugly thing i cant find a good solution for. Is there any way to close a overlapped operation because i need to reuse the overlapped structures, and that means that if there is a timeout check function somewhere in my application just running once in a while over all the overlapped operations looking if they arent timed out because it could hapen that one of the 1000 servers wont give a response for some or other reason and else the overlapped structure would be in use all the time waiting for nothing to come.
-
sure:
int Recieve(char\* buffer, size\_t buflen){ if (m\_iLastRecieved > 0){ //Were expecting a second packed m\_wsaBuf.len = static\_cast(buflen) - m\_iTotalRecieved; m\_wsaBuf.buf = buffer + m\_iTotalRecieved; m\_wsaBuf.buf\[0\] = 0; } else{ buffer\[0\] = 0; m\_wsaBuf.len = static\_cast(buflen); m\_wsaBuf.buf = buffer; } static int sockaddr\_len = sizeof(sockaddr\_in); DWORD temp(0); setStatus(RequestRecieve); //returning 0 is ok return (PostQueuedCompletionStatus(m\_hIOCP, 1, reinterpret\_cast(this), getOverlapped()) == FALSE); }Thats where i make my request to post. You might wonder if it aint dangerous for the buffer to overflow or something but most of the checking goes outside this function and i can assure that there is no overflow since i'm only testing with 1 game which is secured in any way i know off. The requested bytes is unknown, it can variate depending on the used game from like 100 to a few packets of 1500 kb however mostly its just something like 1 packet with 600 bytes (i know because my previous version of the application worked perfectly, only it wasnt using IOCP just the lame one thread per server way ) anyways, with this topic still being open and all, there is one other ugly thing i cant find a good solution for. Is there any way to close a overlapped operation because i need to reuse the overlapped structures, and that means that if there is a timeout check function somewhere in my application just running once in a while over all the overlapped operations looking if they arent timed out because it could hapen that one of the 1000 servers wont give a response for some or other reason and else the overlapped structure would be in use all the time waiting for nothing to come.
Polity4h wrote:
to a few packets of 1500 kb
1500KB? That's too big for a UDP datagram :) All I can think of is check all your send and receive counts on both ends in the debugger. With UDP you should be receiving complete datagrams the size that were sent (providing they arrive successfully - UDP doesn't guarantee that). Why do you use UDP? Have you allowed for messages that don't make it?
Polity4h wrote:
Is there any way to close a overlapped operation because i need to reuse the overlapped structures, and that means that if there is a timeout check function somewhere in my application just running once in a while over all the overlapped operations looking if they arent timed out...
The only way I know is to close the socket. Any pending queued overlapped operations on the socket will be released by GetQueuedCompletionStatus(). If you want to check periodically then you could use a timer, the dwMilliseconds parameter to GetQueuedCompletionStatus(), or some other method. Close the socket after the appropriate period of inactivity.
-
sure:
int Recieve(char\* buffer, size\_t buflen){ if (m\_iLastRecieved > 0){ //Were expecting a second packed m\_wsaBuf.len = static\_cast(buflen) - m\_iTotalRecieved; m\_wsaBuf.buf = buffer + m\_iTotalRecieved; m\_wsaBuf.buf\[0\] = 0; } else{ buffer\[0\] = 0; m\_wsaBuf.len = static\_cast(buflen); m\_wsaBuf.buf = buffer; } static int sockaddr\_len = sizeof(sockaddr\_in); DWORD temp(0); setStatus(RequestRecieve); //returning 0 is ok return (PostQueuedCompletionStatus(m\_hIOCP, 1, reinterpret\_cast(this), getOverlapped()) == FALSE); }Thats where i make my request to post. You might wonder if it aint dangerous for the buffer to overflow or something but most of the checking goes outside this function and i can assure that there is no overflow since i'm only testing with 1 game which is secured in any way i know off. The requested bytes is unknown, it can variate depending on the used game from like 100 to a few packets of 1500 kb however mostly its just something like 1 packet with 600 bytes (i know because my previous version of the application worked perfectly, only it wasnt using IOCP just the lame one thread per server way ) anyways, with this topic still being open and all, there is one other ugly thing i cant find a good solution for. Is there any way to close a overlapped operation because i need to reuse the overlapped structures, and that means that if there is a timeout check function somewhere in my application just running once in a while over all the overlapped operations looking if they arent timed out because it could hapen that one of the 1000 servers wont give a response for some or other reason and else the overlapped structure would be in use all the time waiting for nothing to come.
Cancel Overlapped Socket Op Update: From a Larry Osterman (Microsoft engineer) blog, apparently you can use CancelIo(), casting your socket handle to a "HANDLE". Give it a try - worst that can happen is it fails :) Mark
-
Cancel Overlapped Socket Op Update: From a Larry Osterman (Microsoft engineer) blog, apparently you can use CancelIo(), casting your socket handle to a "HANDLE". Give it a try - worst that can happen is it fails :) Mark
CancelIo doesnt work, the thing is that it needs to be ran from the same thread where Overlapped operations were posted. I could however close the socket and open it again, but what are the drawbacks of this? this means that i need to close and open like 20 sockets per second in the future. looks really hard to me, but i cant really figure out what might happen when i do this.
-
CancelIo doesnt work, the thing is that it needs to be ran from the same thread where Overlapped operations were posted. I could however close the socket and open it again, but what are the drawbacks of this? this means that i need to close and open like 20 sockets per second in the future. looks really hard to me, but i cant really figure out what might happen when i do this.
Polity4h wrote:
CancelIo doesnt work, the thing is that it needs to be ran from the same thread where Overlapped operations were posted.
Yeah I thought about that since you are posting overlapped ops from your worker thread(s) :) 20 sockets per second? How long is the time you are willing to wait before the remote node is considered inactive??
-
Polity4h wrote:
CancelIo doesnt work, the thing is that it needs to be ran from the same thread where Overlapped operations were posted.
Yeah I thought about that since you are posting overlapped ops from your worker thread(s) :) 20 sockets per second? How long is the time you are willing to wait before the remote node is considered inactive??
alright, i might have thought a bid to far in the future were i got myself a mainframe ;) << -_- whatever, anyways, lets say 2 sockets needs to be created per second. wont this kill my system after doing this a long time ( my app needs to run constantly for, count them up a few years ) << thats the perfect vision ofcourse.
-
alright, i might have thought a bid to far in the future were i got myself a mainframe ;) << -_- whatever, anyways, lets say 2 sockets needs to be created per second. wont this kill my system after doing this a long time ( my app needs to run constantly for, count them up a few years ) << thats the perfect vision ofcourse.
I'm not sure what you are trying to do. Why do you need to keep creating sockets but never destroy them? Surely you'll run out of sockets if you do that :)
-
I'm not sure what you are trying to do. Why do you need to keep creating sockets but never destroy them? Surely you'll run out of sockets if you do that :)
...I could however close the socket and open it again, but what are the drawbacks of this? this means that i need to close and open like 20 sockets per second in the future... hehe, anyways your saying that your server wont suffer to hard when creating AND DESTROYING lets say 2 sockets per second?
-
...I could however close the socket and open it again, but what are the drawbacks of this? this means that i need to close and open like 20 sockets per second in the future... hehe, anyways your saying that your server wont suffer to hard when creating AND DESTROYING lets say 2 sockets per second?
No. As long as that's really what your app needs to do. :) Think about how many TCP sockets a busy web (HTTP) server must open and close per second. 20 per second is nothing :) Mark
-
No. As long as that's really what your app needs to do. :) Think about how many TCP sockets a busy web (HTTP) server must open and close per second. 20 per second is nothing :) Mark
-
...I could however close the socket and open it again, but what are the drawbacks of this? this means that i need to close and open like 20 sockets per second in the future... hehe, anyways your saying that your server wont suffer to hard when creating AND DESTROYING lets say 2 sockets per second?
*EDIT* Having a pool of sockets could improve performance - that way only ConnectEx/DisconnectEx needs to be called on a socket instead of recreating/destoying the entire socket every time. Mark
-
*EDIT* Having a pool of sockets could improve performance - that way only ConnectEx/DisconnectEx needs to be called on a socket instead of recreating/destoying the entire socket every time. Mark
src=msdn:The Windows Sockets ConnectEx function establishes a connection to a specified socket, and optionally sends data (called connect data) once the connection is established. The ConnectEx function is only supported on connection-oriented sockets. remember, i'm forced to use connectionless protocols :) thanks anyway
-
src=msdn:The Windows Sockets ConnectEx function establishes a connection to a specified socket, and optionally sends data (called connect data) once the connection is established. The ConnectEx function is only supported on connection-oriented sockets. remember, i'm forced to use connectionless protocols :) thanks anyway
Gotcha man :) I was just extending my comment about TCP sockets on an HTTP server :)