Guys, I've been astonished by the help I've received with this problem - thank you very much everyone. I'm afraid to say that the problem was as stupid as me consuming the exception on a particular record. Best practice point to self, and I'm really sorry for wasting peoples time on such a simple thing. I normally go out of my way to Google for problems, but since 726301 is not a magic number/ceiling (as it was just an exception) I was really scratching my head over this one without seeing the obvious. Anyway, thanks again for everyone's time, and for the suggestions. Ben
Ben Cocker
Posts
-
Is the a max row count for SQLDataReader? -
Is the a max row count for SQLDataReader?Hi Jon, Thanks for your reply, and the suggestions. Could I point you to my reply to Luc for a discussion of this out-of-memory limitation? And, I have been very careful to limit the number of fields I use and don't have any string columns, so although I take your point that 8 million rows is a beast, in SQL management studio the size of comma delimited output file is less than 1.5gig - I've got a 32 bit machine with maximum memory so I am skating on thin ice, but still I expected it to work out memory wise. I'll look into LAPACK - thanks for the suggestion. Ben
-
Is the a max row count for SQLDataReader?Hi Luc, Thanks for your reply. In answer to your questions: I don't get an exception - it just terminates the while(Reader.Read()) loop at 7263001 records. I have actually slightly increased the field count of the query (which makes it also take much longer) - but this doesn't affect the number of records it tops out at, which made me think that timeout and oom might not be the reason. I used a sp in order to be able to test and maintain the query on the server rather than just having a long text string select statement in my C# code. Cheers, Ben
-
Is the a max row count for SQLDataReader?Hi there, I am reading a (rather large) MS-SQL stored procedure into C# using the SqlDataReader. I expect this to return slightly over 8,000,000 records, but it consistently maxes out at 726,301. I cannot for the life in me see why - when running the stored procedure (which has no input parameters) in SQL Server Management Studio it returns the correct number of records. I am running VS 2008 Professional with SQL Server 2008 Developer Ed. The code is paraphrased as follows: // Set up a list of data type Record (my own) to hold the output of the sp List records = new List(); System.Data.SqlClient.SqlConnection Connection = new System.Data.SqlClient.SqlConnection(ConnectionString); System.Data.SqlClient.SqlCommand Command; try { Connection.Open(); { Command = new System.Data.SqlClient.SqlCommand("StoredProcedure", Connection); Command.CommandType = CommandType.StoredProcedure; System.Data.SqlClient.SqlDataReader Reader = Command.ExecuteReader(); while (Reader.Read()) { Record NewRecord = new Record(); NewRecord.PAR1 = Reader.GetInt32(Reader.GetOrdinal("PAR1")); NewRecord.PAR2 = Reader.GetInt32(Reader.GetOrdinal("PAR2"));... etc records.Add(NewRecord); } } } catch { } finally { Connection.Close(); } Thanks a million for any help! Ben