Read until "- -" ???
-
Hello.... I'm stuck on a piece of logic that i'm sure is pretty simple but i've been thinking about it so long now that my mind has literally stopped working.... With that said... I am writing events to a log file that i then need to turn around and once a day backup-or move to the db. Once i begin moving to the db i need to parse the separate fields (date/time, user, event message, priority, etc) into separate columns. Yes, I am fully aware that it would be easier to just write it to the db from the begining...but every action we call is already going to the db so it doesn't need to be contending with event logs. So, due to the format of my log files i need to read until "
- -" for instance:{username}- -{Event Message}- -{etc...}i then need to send that data to the db to the independant columns. This of course poses two problems, at least that i can see..first being how do i safely parse the string until i hit the "- -"(i am willing to change that to something more unique if necessary). and Second... how do i quickly deposit this to the db? I am doing this at night when 90% of our clients will be closed, but there is that small amount of sites that run 24/7, so it doesn't need to interupt their operations!!! (oh, by the way i've got about 5 logs to send over and they normaly are 100+k each daily) string Beautiful; Beautiful = "ignorant"; label1.Text = "The world is full of " + Beautiful +" people."; Why is common sense such an un-common comodity? -
Hello.... I'm stuck on a piece of logic that i'm sure is pretty simple but i've been thinking about it so long now that my mind has literally stopped working.... With that said... I am writing events to a log file that i then need to turn around and once a day backup-or move to the db. Once i begin moving to the db i need to parse the separate fields (date/time, user, event message, priority, etc) into separate columns. Yes, I am fully aware that it would be easier to just write it to the db from the begining...but every action we call is already going to the db so it doesn't need to be contending with event logs. So, due to the format of my log files i need to read until "
- -" for instance:{username}- -{Event Message}- -{etc...}i then need to send that data to the db to the independant columns. This of course poses two problems, at least that i can see..first being how do i safely parse the string until i hit the "- -"(i am willing to change that to something more unique if necessary). and Second... how do i quickly deposit this to the db? I am doing this at night when 90% of our clients will be closed, but there is that small amount of sites that run 24/7, so it doesn't need to interupt their operations!!! (oh, by the way i've got about 5 logs to send over and they normaly are 100+k each daily) string Beautiful; Beautiful = "ignorant"; label1.Text = "The world is full of " + Beautiful +" people."; Why is common sense such an un-common comodity?1. If you would reduce it to one special chcracter you can spplit the whole string via string.Split('|') into smaller pieces without having to parse anything. 2. The fastest way greatly depends on your database type. But rethink if you really want it fast: Is it probably better to make it a bit slower, so that the database or any other processes can also answer other requests in the meanwhile? For this a seperate thread with low priority could be usefull.
-
Hello.... I'm stuck on a piece of logic that i'm sure is pretty simple but i've been thinking about it so long now that my mind has literally stopped working.... With that said... I am writing events to a log file that i then need to turn around and once a day backup-or move to the db. Once i begin moving to the db i need to parse the separate fields (date/time, user, event message, priority, etc) into separate columns. Yes, I am fully aware that it would be easier to just write it to the db from the begining...but every action we call is already going to the db so it doesn't need to be contending with event logs. So, due to the format of my log files i need to read until "
- -" for instance:{username}- -{Event Message}- -{etc...}i then need to send that data to the db to the independant columns. This of course poses two problems, at least that i can see..first being how do i safely parse the string until i hit the "- -"(i am willing to change that to something more unique if necessary). and Second... how do i quickly deposit this to the db? I am doing this at night when 90% of our clients will be closed, but there is that small amount of sites that run 24/7, so it doesn't need to interupt their operations!!! (oh, by the way i've got about 5 logs to send over and they normaly are 100+k each daily) string Beautiful; Beautiful = "ignorant"; label1.Text = "The world is full of " + Beautiful +" people."; Why is common sense such an un-common comodity?I wouldn't worry if you mean 100+KB per day, because that is quite small. If you mean 100+K records, then it could take a while. Just read each line, split it with split('- -'), and shove it into the database using a stored procedure. On a moderate single processor machine running sql server, you should easily get 1000 records per second. A very slow server may take up to 5 times longer. Try the simple approach first, then optimize later. If you DO need to optimize, look to sending batches of inserts in each command, or passing your data as strings into a stored proc and parsing them out in there. Usually, these extremes are not necessary. Simple approach: StreamReader sr; String str; String[] arr; String delim = "- -"; SQLCommand cmd; SQLConnection cn; //init connetion and command here and open connection //open streamreader here str = sr.ReadLine; while (sr.Peek() >= 0) { arr = str.Split(delim.ToCharArray()); cmd.Parameters("@Param1").Value = arr[0]; ... cmd.ExecuteNonQuery(); } cn.close; sr.close;