Maximum file open problem with fopen
-
We have an application on Solaris and Windows that uses the fopen family of calls to perform file I/O operations. However it was discovered that fopen has a limit of 255 open file handles for a process. We therefore need to replace fopen with C++ fstreams to circumvent this problem. However the MSDN reveals that a fstream object internally uses a basic_filebuf object for stream buffering which in turn uses the fopen family of calls for file I/O. We therefore wrote a test program to validate this fact. Surprisingly however the test program, which uses fstream objects to open files does NOT fail at the limit of 255 but instead goes on to open more than 2000 files. This apparently contradicts with what MSDN says about fstream objects internally using fopen family of calls for file manipulation. Can someone provide me some pointers in this regard?
-
We have an application on Solaris and Windows that uses the fopen family of calls to perform file I/O operations. However it was discovered that fopen has a limit of 255 open file handles for a process. We therefore need to replace fopen with C++ fstreams to circumvent this problem. However the MSDN reveals that a fstream object internally uses a basic_filebuf object for stream buffering which in turn uses the fopen family of calls for file I/O. We therefore wrote a test program to validate this fact. Surprisingly however the test program, which uses fstream objects to open files does NOT fail at the limit of 255 but instead goes on to open more than 2000 files. This apparently contradicts with what MSDN says about fstream objects internally using fopen family of calls for file manipulation. Can someone provide me some pointers in this regard?
What is it that you are wanting to know? If you needed more than 255 files opened simultaneously and your test application proved that more than 2,000 files could be, isn't your problem solved?
"When I was born I was so surprised that I didn't talk for a year and a half." - Gracie Allen
-
We have an application on Solaris and Windows that uses the fopen family of calls to perform file I/O operations. However it was discovered that fopen has a limit of 255 open file handles for a process. We therefore need to replace fopen with C++ fstreams to circumvent this problem. However the MSDN reveals that a fstream object internally uses a basic_filebuf object for stream buffering which in turn uses the fopen family of calls for file I/O. We therefore wrote a test program to validate this fact. Surprisingly however the test program, which uses fstream objects to open files does NOT fail at the limit of 255 but instead goes on to open more than 2000 files. This apparently contradicts with what MSDN says about fstream objects internally using fopen family of calls for file manipulation. Can someone provide me some pointers in this regard?
I can not directly answer your question, but I have a suggestion. You can load the visual studio and the source code to the C runtime library and maybe see what the basic_filebuf calls are doing. The fopen calls eventually use the CreateFile API, but because of using the pseudo file handles, you run into the problem with a file open limitation. Maybe the other calls using the basic_filebuf do something different to avoid the limitation. After examining the source code, you can assure yourself that the MSDN is indeed wrong.
-
We have an application on Solaris and Windows that uses the fopen family of calls to perform file I/O operations. However it was discovered that fopen has a limit of 255 open file handles for a process. We therefore need to replace fopen with C++ fstreams to circumvent this problem. However the MSDN reveals that a fstream object internally uses a basic_filebuf object for stream buffering which in turn uses the fopen family of calls for file I/O. We therefore wrote a test program to validate this fact. Surprisingly however the test program, which uses fstream objects to open files does NOT fail at the limit of 255 but instead goes on to open more than 2000 files. This apparently contradicts with what MSDN says about fstream objects internally using fopen family of calls for file manipulation. Can someone provide me some pointers in this regard?
What OS are you on? The MSDN infomation may give you the maximum number for a windows 9X machine. MSDN seems to always state the lowest common denomionator when giving information. The only way to know if your solution is viable is to try it out on all the OS's it will be deployed on. Noting Service Pack Level and Version of IE installed. Best of Luck Forever Developing
-
We have an application on Solaris and Windows that uses the fopen family of calls to perform file I/O operations. However it was discovered that fopen has a limit of 255 open file handles for a process. We therefore need to replace fopen with C++ fstreams to circumvent this problem. However the MSDN reveals that a fstream object internally uses a basic_filebuf object for stream buffering which in turn uses the fopen family of calls for file I/O. We therefore wrote a test program to validate this fact. Surprisingly however the test program, which uses fstream objects to open files does NOT fail at the limit of 255 but instead goes on to open more than 2000 files. This apparently contradicts with what MSDN says about fstream objects internally using fopen family of calls for file manipulation. Can someone provide me some pointers in this regard?
int _setmaxstdio(int newmax) is your friend :). On .NET 2003 fopen defaults to 512 simultaneously open files which can be increased to 2,048 using this method. fopen() etc. are low-level C routines that C++ fstream is built on top of, therefore it probably just sets the default to the upper limit when it initialises 'for maximum simplicity'. Hope that helps ya, Al. "When I left you I was but the learner, now I am the master" - Darth Vader
-
int _setmaxstdio(int newmax) is your friend :). On .NET 2003 fopen defaults to 512 simultaneously open files which can be increased to 2,048 using this method. fopen() etc. are low-level C routines that C++ fstream is built on top of, therefore it probably just sets the default to the upper limit when it initialises 'for maximum simplicity'. Hope that helps ya, Al. "When I left you I was but the learner, now I am the master" - Darth Vader