Socket listen backlog woes
-
OK, I think I might know what the answer to this is going to be, put I'll post my problem anyway, as you never know, there may be a simpler solution... I have a service app that listems for incoming socket connections using good old Berkeley style socket code - I have code that creates the socket to listen on and then a thread that uses "select" to wait for the connections, spawning a new thread per-connection to handle the data, e.g.:
for ( ; ; ) { ... fd_set read; FD_ZERO(&read); FD_SET(m_hSocket, &read); int nResult = ::select(-1, &read, NULL, NULL, &tv); if (nResult == 1) { // Accept the connection SOCKET s = ::accept(read.fd_array[i], NULL, NULL); // Create a thread to process the incoming connection AfxBeginThread(...); } }The problem is that if I hammer the app using a test program that creates lots of simultaneous connections to the server (using a separate thread for each all started at once), before long some of these connections will fail with WSAECONNREFUSED. This seems to be because creating the thread to handle the accepted socket in the code above is expensive in terms of CPU and the incoming connections piling up whilst I am doing this are causing the backlog (5 sockets?) to be exceeded. I *think* the answer will be to use IO completion ports but this is going to be a big job. The threads I create to process each socket are short-lived, and I know IOCP can ensure fewer are needed but will it also help with this backlog problem or is there another more generic way to handle this? At some point in the distant future I may want to port the code to Linux (yikes!) so I wanted to avoif anything MS specific. Comments?
-
OK, I think I might know what the answer to this is going to be, put I'll post my problem anyway, as you never know, there may be a simpler solution... I have a service app that listems for incoming socket connections using good old Berkeley style socket code - I have code that creates the socket to listen on and then a thread that uses "select" to wait for the connections, spawning a new thread per-connection to handle the data, e.g.:
for ( ; ; ) { ... fd_set read; FD_ZERO(&read); FD_SET(m_hSocket, &read); int nResult = ::select(-1, &read, NULL, NULL, &tv); if (nResult == 1) { // Accept the connection SOCKET s = ::accept(read.fd_array[i], NULL, NULL); // Create a thread to process the incoming connection AfxBeginThread(...); } }The problem is that if I hammer the app using a test program that creates lots of simultaneous connections to the server (using a separate thread for each all started at once), before long some of these connections will fail with WSAECONNREFUSED. This seems to be because creating the thread to handle the accepted socket in the code above is expensive in terms of CPU and the incoming connections piling up whilst I am doing this are causing the backlog (5 sockets?) to be exceeded. I *think* the answer will be to use IO completion ports but this is going to be a big job. The threads I create to process each socket are short-lived, and I know IOCP can ensure fewer are needed but will it also help with this backlog problem or is there another more generic way to handle this? At some point in the distant future I may want to port the code to Linux (yikes!) so I wanted to avoif anything MS specific. Comments?
Why spawn a thread *AFTER* a connection attempt? Prepare a pool of worker threads all waiting on a single semaphore would be the easiest fix I believe(least effort on your part)... after all, triggering a single semaphore event should cost a fraction of AfxBeginThread(...) try Recycle all threads instead of respawn... and dont harm any animals. :~ -Dan
-
OK, I think I might know what the answer to this is going to be, put I'll post my problem anyway, as you never know, there may be a simpler solution... I have a service app that listems for incoming socket connections using good old Berkeley style socket code - I have code that creates the socket to listen on and then a thread that uses "select" to wait for the connections, spawning a new thread per-connection to handle the data, e.g.:
for ( ; ; ) { ... fd_set read; FD_ZERO(&read); FD_SET(m_hSocket, &read); int nResult = ::select(-1, &read, NULL, NULL, &tv); if (nResult == 1) { // Accept the connection SOCKET s = ::accept(read.fd_array[i], NULL, NULL); // Create a thread to process the incoming connection AfxBeginThread(...); } }The problem is that if I hammer the app using a test program that creates lots of simultaneous connections to the server (using a separate thread for each all started at once), before long some of these connections will fail with WSAECONNREFUSED. This seems to be because creating the thread to handle the accepted socket in the code above is expensive in terms of CPU and the incoming connections piling up whilst I am doing this are causing the backlog (5 sockets?) to be exceeded. I *think* the answer will be to use IO completion ports but this is going to be a big job. The threads I create to process each socket are short-lived, and I know IOCP can ensure fewer are needed but will it also help with this backlog problem or is there another more generic way to handle this? At some point in the distant future I may want to port the code to Linux (yikes!) so I wanted to avoif anything MS specific. Comments?
First, you should be able to set the backlog value to much higher than 5. This will ease most problems immediatley. You've not shown the listen section so I will assume you've used a large value. Second, you could split the code into two threads:* Thread 1 sits waiting for new connections, accepts them, and puts the socket handle into a queue. Thats all it does - it then immediatley resumes accepting other connections.
- Thread 2 sits on the queue, and creates a thread for each socket.
This detachs the (slow) thread creation process from the (fast) accept process, and you should be able to pretty much reach the maximum accept rate you can.
- Thread 2 sits on the queue, and creates a thread for each socket.
-
First, you should be able to set the backlog value to much higher than 5. This will ease most problems immediatley. You've not shown the listen section so I will assume you've used a large value. Second, you could split the code into two threads:* Thread 1 sits waiting for new connections, accepts them, and puts the socket handle into a queue. Thats all it does - it then immediatley resumes accepting other connections.
- Thread 2 sits on the queue, and creates a thread for each socket.
This detachs the (slow) thread creation process from the (fast) accept process, and you should be able to pretty much reach the maximum accept rate you can.
Using winsock2, the maximum backlog value is apparently now 0x7fffffff (SOMAXCONN) but I tried this without any success. I will experiment with new values. I also tried two threads as you suggested, using a shared array of SOCKETs - the accept thread adds the accepted socket to the list and then signals the second thread via a semaphore, but still I had problems. I used a critical section to protect the shared socket array, but perhaps I need to revisit this. Thanks for the advice - switching to IOCP and AcceptEx is a major task that itself looks fraught with problems if my search of Google Groups is anything to go by! ;)
- Thread 2 sits on the queue, and creates a thread for each socket.
-
Why spawn a thread *AFTER* a connection attempt? Prepare a pool of worker threads all waiting on a single semaphore would be the easiest fix I believe(least effort on your part)... after all, triggering a single semaphore event should cost a fraction of AfxBeginThread(...) try Recycle all threads instead of respawn... and dont harm any animals. :~ -Dan
-
Using winsock2, the maximum backlog value is apparently now 0x7fffffff (SOMAXCONN) but I tried this without any success. I will experiment with new values. I also tried two threads as you suggested, using a shared array of SOCKETs - the accept thread adds the accepted socket to the list and then signals the second thread via a semaphore, but still I had problems. I used a critical section to protect the shared socket array, but perhaps I need to revisit this. Thanks for the advice - switching to IOCP and AcceptEx is a major task that itself looks fraught with problems if my search of Google Groups is anything to go by! ;)
I have a framework that does IOCP/AcceptEx/System Thread Pool stuff for Windows based network servers. It's (n)ill documented at present but you can take a look here. The intent is to be able to support multiple listening sockets on multiple adaptors using multiple network protocols (primarily IPv4 and IPv6 I suppose) and exploit the Windows good stuff for networking. The idea was to impose a minimum of dependency between the server framework and the rest of the application, you have to provide a 'per connection' class that must implement a minimal interface, but after that it's up to you. No inheritance required, namespaces are used so that my idea of a CRITICAL_SECTION wrapper doesn't collide with yours, etc etc. I believe it would be possible to construct a framework for Unix type systems that would be compatible with such a 'per connection' class and be reasonably efficient in that environment. Time will tell. Paul