Huge file [modified]
-
I want to read a very large files such az log files in IIS line by line with an especial format for each line (by c#),Which way or function do you recommand? -- modified at 11:37 Wednesday 19th July, 2006
StreamReader reader = new StreamReader("MyFile"); string strLine; while( (strLine = reader.ReadLine() ) != null ) { // do something } -
I want to read a very large files such az log files in IIS line by line with an especial format for each line (by c#),Which way or function do you recommand? -- modified at 11:37 Wednesday 19th July, 2006
tail ? (download cygwin) or perhaps // warning, pseudo-code, if it doesn't compile => fix the code yourself static void Main(string[] args) { string line; using(StreamReader sr = new StreamReader(args[0])) while((line = sr.ReadLine()) != null) { Console.WriteLine(line); Console.ReadLine(); } }
-
tail ? (download cygwin) or perhaps // warning, pseudo-code, if it doesn't compile => fix the code yourself static void Main(string[] args) { string line; using(StreamReader sr = new StreamReader(args[0])) while((line = sr.ReadLine()) != null) { Console.WriteLine(line); Console.ReadLine(); } }
-
Hello first of all thanks for answer,So Fast! Look the problem is that these are huge file,Maybe 1G... The solution that you recommanded ,have problem of speed,I want a solution that read file faster. Thanks in advance
It is two parts, one you have to use a stream with a file so big because you do not have the memory to do otherwise. Secondly, if the file is not indexed your SOL. If it is indexed you can use any traversal algorithm to get the data you need. ( I did this technique with the national do not call list and it works fairly fast) Also, you may consider c++ if you just need speed. A man said to the universe: "Sir I exist!" "However," replied the Universe, "The fact has not created in me A sense of obligation." -- Stephen Crane
-
It is two parts, one you have to use a stream with a file so big because you do not have the memory to do otherwise. Secondly, if the file is not indexed your SOL. If it is indexed you can use any traversal algorithm to get the data you need. ( I did this technique with the national do not call list and it works fairly fast) Also, you may consider c++ if you just need speed. A man said to the universe: "Sir I exist!" "However," replied the Universe, "The fact has not created in me A sense of obligation." -- Stephen Crane
-
Ennis Ray Lynch, Jr. wrote:
Also, you may consider c++ if you just need speed.
I'm not sure how much that would help though. The filesystem would still be the bottleneck in performance.
You can use memory mapping in C++ which will reduce the bottle neck for speed. A man said to the universe: "Sir I exist!" "However," replied the Universe, "The fact has not created in me A sense of obligation." -- Stephen Crane
-
Hello first of all thanks for answer,So Fast! Look the problem is that these are huge file,Maybe 1G... The solution that you recommanded ,have problem of speed,I want a solution that read file faster. Thanks in advance
what is it you want to do? the question was: "I want to read a very large files" I told you how to read without memory problem. Now it can't read faster than you! Had the question be: "I am looking for a particular pattern" I might have added an automated search function..... But if you want to read it all, you've got to do it man!