Continued Values Collection/List/Dictionary [modified]
-
I am learning further in C# and came across a special requirement of collections. I wonder if such implementations exists. I need a list with a key and multiple values. After the first entry, next entries can be partial only to denote a change. Something like this:
index value1 value2 value3 Result value1 value2 value3
1 100 200 300 - 100 200 300
2 200 - 100 200 200
3 150 300 - 150 200 300
4 250 - 150 250 300
5 100 200 - 100 200 300This can be achieved using DataTable and using select statements/compute etc. But, is there a simpler option available? Speed is the key here. ----------------------------Question Modified---------------------------------- Please understand that creating a list/dictionary/array/hashtable is not the problem here. Storing the data is easy. This could be done using numeric Key/Index (I can manage to keep indexes of the keys also). The idea is to be able to look for the last fetched value against each key. The problem comes when I want to retrieve he value without having to loop again. Look at the same example with modified index values.
index value1 value2 value3 Result value1 value2 value3
100 100 200 300 - 100 200 300
200 200 - 100 200 200
300 150 300 - 150 200 300
400 250 - 150 250 300
500 100 200 - 100 200 300Consider the next step. if I want to find out the value for key 250, I would want to look for 200, and then find results for it. Is it possible without going through loops again? Current Solution: I created a class as:
class DataClass
{
int Value1; int Value2; int Value3
int ValueIndex1; int ValueIndex2; int ValueIndex3;
}Then for each entry done, I need to run the loop once to find out the last index for which entry was done against the value and store it.
Dictionary entries = new Dictionary();
entries.Add(100, new DataClass(100,200,300, 100,100,100);
entries.Add(200, new DataClass( 0,150, 0, 100,200,100);
entries.Add(300, new DataClass( 0, 0,250, 100,200,300);
entries.Add(400, new DataClass(150, 0, 0, 400,200,300);
entries.Add(500, new DataClass( 0,200, 0, 400,500,300);Now, whenever i need to look for 250, i search for closest smaller key (here that is 200). find values. whatever is available is taken as it is. For all others, find the key in the same record and get the new entries. This solution works for the first time entries. However it creates problems when an en
-
I am learning further in C# and came across a special requirement of collections. I wonder if such implementations exists. I need a list with a key and multiple values. After the first entry, next entries can be partial only to denote a change. Something like this:
index value1 value2 value3 Result value1 value2 value3
1 100 200 300 - 100 200 300
2 200 - 100 200 200
3 150 300 - 150 200 300
4 250 - 150 250 300
5 100 200 - 100 200 300This can be achieved using DataTable and using select statements/compute etc. But, is there a simpler option available? Speed is the key here. ----------------------------Question Modified---------------------------------- Please understand that creating a list/dictionary/array/hashtable is not the problem here. Storing the data is easy. This could be done using numeric Key/Index (I can manage to keep indexes of the keys also). The idea is to be able to look for the last fetched value against each key. The problem comes when I want to retrieve he value without having to loop again. Look at the same example with modified index values.
index value1 value2 value3 Result value1 value2 value3
100 100 200 300 - 100 200 300
200 200 - 100 200 200
300 150 300 - 150 200 300
400 250 - 150 250 300
500 100 200 - 100 200 300Consider the next step. if I want to find out the value for key 250, I would want to look for 200, and then find results for it. Is it possible without going through loops again? Current Solution: I created a class as:
class DataClass
{
int Value1; int Value2; int Value3
int ValueIndex1; int ValueIndex2; int ValueIndex3;
}Then for each entry done, I need to run the loop once to find out the last index for which entry was done against the value and store it.
Dictionary entries = new Dictionary();
entries.Add(100, new DataClass(100,200,300, 100,100,100);
entries.Add(200, new DataClass( 0,150, 0, 100,200,100);
entries.Add(300, new DataClass( 0, 0,250, 100,200,300);
entries.Add(400, new DataClass(150, 0, 0, 400,200,300);
entries.Add(500, new DataClass( 0,200, 0, 400,500,300);Now, whenever i need to look for 250, i search for closest smaller key (here that is 200). find values. whatever is available is taken as it is. For all others, find the key in the same record and get the new entries. This solution works for the first time entries. However it creates problems when an en
Have each object maintain a dictionary of values. In the samples you rpovided above: Index 1 would contain 1,100 2,200 3,300 Index 2 would contain 3, 200 Index 3 would contain 1, 150 3, 300 Index 4 would contain 2,250 Index 5 would contain 1,100 2,200 Use google to learn how to use dictionary collections.
.45 ACP - because shooting twice is just silly
-----
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"The staggering layers of obscenity in your statement make it a work of art on so many levels." - J. Jystad, 2001 -
I am learning further in C# and came across a special requirement of collections. I wonder if such implementations exists. I need a list with a key and multiple values. After the first entry, next entries can be partial only to denote a change. Something like this:
index value1 value2 value3 Result value1 value2 value3
1 100 200 300 - 100 200 300
2 200 - 100 200 200
3 150 300 - 150 200 300
4 250 - 150 250 300
5 100 200 - 100 200 300This can be achieved using DataTable and using select statements/compute etc. But, is there a simpler option available? Speed is the key here. ----------------------------Question Modified---------------------------------- Please understand that creating a list/dictionary/array/hashtable is not the problem here. Storing the data is easy. This could be done using numeric Key/Index (I can manage to keep indexes of the keys also). The idea is to be able to look for the last fetched value against each key. The problem comes when I want to retrieve he value without having to loop again. Look at the same example with modified index values.
index value1 value2 value3 Result value1 value2 value3
100 100 200 300 - 100 200 300
200 200 - 100 200 200
300 150 300 - 150 200 300
400 250 - 150 250 300
500 100 200 - 100 200 300Consider the next step. if I want to find out the value for key 250, I would want to look for 200, and then find results for it. Is it possible without going through loops again? Current Solution: I created a class as:
class DataClass
{
int Value1; int Value2; int Value3
int ValueIndex1; int ValueIndex2; int ValueIndex3;
}Then for each entry done, I need to run the loop once to find out the last index for which entry was done against the value and store it.
Dictionary entries = new Dictionary();
entries.Add(100, new DataClass(100,200,300, 100,100,100);
entries.Add(200, new DataClass( 0,150, 0, 100,200,100);
entries.Add(300, new DataClass( 0, 0,250, 100,200,300);
entries.Add(400, new DataClass(150, 0, 0, 400,200,300);
entries.Add(500, new DataClass( 0,200, 0, 400,500,300);Now, whenever i need to look for 250, i search for closest smaller key (here that is 200). find values. whatever is available is taken as it is. For all others, find the key in the same record and get the new entries. This solution works for the first time entries. However it creates problems when an en
Value1, value2 and value3 are distinct entities that you want to track? A dictionary consists of a value and a key. The key would be your index-column, the value could be a struct that holds the three columns;
System.Collections.Generic.Dictionary<int, MyValues> myListOfValues;
struct MyValues
{
int value1;
int value2;
int value3;
}You could consider the generic list as your datatable, and the struct as a single row in that table. Now you can add values to your dictionary;
MyValues oneRow = new MyValues();
oneRow.value1 = 100;
oneRow.value2 = 200;
oneRow.value3 = 300;
myListOfValues.Add(1, oneRow);Now, if you want the recreate those values for index 3, you would have to search in the list for the first non-empty value for the column value2. If speed is important, then I'd create one row outside of that list, just to mirror the current values. That wouldn't help if you need the values from the middle of the index, you'd still have to walk the list for that. How many records will be in that list? It might just be worth to store copies of the latest value, instead of looking for them.
I are Troll :suss:
-
I am learning further in C# and came across a special requirement of collections. I wonder if such implementations exists. I need a list with a key and multiple values. After the first entry, next entries can be partial only to denote a change. Something like this:
index value1 value2 value3 Result value1 value2 value3
1 100 200 300 - 100 200 300
2 200 - 100 200 200
3 150 300 - 150 200 300
4 250 - 150 250 300
5 100 200 - 100 200 300This can be achieved using DataTable and using select statements/compute etc. But, is there a simpler option available? Speed is the key here. ----------------------------Question Modified---------------------------------- Please understand that creating a list/dictionary/array/hashtable is not the problem here. Storing the data is easy. This could be done using numeric Key/Index (I can manage to keep indexes of the keys also). The idea is to be able to look for the last fetched value against each key. The problem comes when I want to retrieve he value without having to loop again. Look at the same example with modified index values.
index value1 value2 value3 Result value1 value2 value3
100 100 200 300 - 100 200 300
200 200 - 100 200 200
300 150 300 - 150 200 300
400 250 - 150 250 300
500 100 200 - 100 200 300Consider the next step. if I want to find out the value for key 250, I would want to look for 200, and then find results for it. Is it possible without going through loops again? Current Solution: I created a class as:
class DataClass
{
int Value1; int Value2; int Value3
int ValueIndex1; int ValueIndex2; int ValueIndex3;
}Then for each entry done, I need to run the loop once to find out the last index for which entry was done against the value and store it.
Dictionary entries = new Dictionary();
entries.Add(100, new DataClass(100,200,300, 100,100,100);
entries.Add(200, new DataClass( 0,150, 0, 100,200,100);
entries.Add(300, new DataClass( 0, 0,250, 100,200,300);
entries.Add(400, new DataClass(150, 0, 0, 400,200,300);
entries.Add(500, new DataClass( 0,200, 0, 400,500,300);Now, whenever i need to look for 250, i search for closest smaller key (here that is 200). find values. whatever is available is taken as it is. For all others, find the key in the same record and get the new entries. This solution works for the first time entries. However it creates problems when an en
I do not believe that there is something built into the framework. There may be something in the big wide internet, but I'm not sure what I'd use to look it up. Are the items all indexed concsecutively? If so, you will probably want to look for something that implements
IListinstead ofIDictionary. The reason other people suggested Dictionary is probably because you said your list had "keys" and "values" implying possible non-consecutive indexes. Making such a collection yourself is not that hard. I put together a sample of how I would do it. It turned out to ~400 lines, most of which were boilerplate. The Tuple class from .NET 4.0 would be helpful here, but you could create one yourself pretty easily. My sample class definition was:public class DeltaCollection<T1, T2, T3> :
IList,
IList<Tuple<T1, T2, T3>>,
IList<Tuple<T1?, T2?, T3?>>
where T1 : struct, T2 : struct, T3 : struct(I did it in VB, so I may have messed up the generic constraint syntax.) Update I played around some more and was able to get it down to ~200 lines +15-30 lines per tuple size (number of values per row) you want to support. Again, this would be benefited greatly by .NET 4.0's Tuple class, which had to be reimplemented to use in my class now. The main difference between this version and the original is that the Item property and
foreachenumerator will now returnTuple<T1?, T2?, T3?>instead ofTuple<T1, T2, T3>.modified on Wednesday, December 16, 2009 11:04 PM
-
I am learning further in C# and came across a special requirement of collections. I wonder if such implementations exists. I need a list with a key and multiple values. After the first entry, next entries can be partial only to denote a change. Something like this:
index value1 value2 value3 Result value1 value2 value3
1 100 200 300 - 100 200 300
2 200 - 100 200 200
3 150 300 - 150 200 300
4 250 - 150 250 300
5 100 200 - 100 200 300This can be achieved using DataTable and using select statements/compute etc. But, is there a simpler option available? Speed is the key here. ----------------------------Question Modified---------------------------------- Please understand that creating a list/dictionary/array/hashtable is not the problem here. Storing the data is easy. This could be done using numeric Key/Index (I can manage to keep indexes of the keys also). The idea is to be able to look for the last fetched value against each key. The problem comes when I want to retrieve he value without having to loop again. Look at the same example with modified index values.
index value1 value2 value3 Result value1 value2 value3
100 100 200 300 - 100 200 300
200 200 - 100 200 200
300 150 300 - 150 200 300
400 250 - 150 250 300
500 100 200 - 100 200 300Consider the next step. if I want to find out the value for key 250, I would want to look for 200, and then find results for it. Is it possible without going through loops again? Current Solution: I created a class as:
class DataClass
{
int Value1; int Value2; int Value3
int ValueIndex1; int ValueIndex2; int ValueIndex3;
}Then for each entry done, I need to run the loop once to find out the last index for which entry was done against the value and store it.
Dictionary entries = new Dictionary();
entries.Add(100, new DataClass(100,200,300, 100,100,100);
entries.Add(200, new DataClass( 0,150, 0, 100,200,100);
entries.Add(300, new DataClass( 0, 0,250, 100,200,300);
entries.Add(400, new DataClass(150, 0, 0, 400,200,300);
entries.Add(500, new DataClass( 0,200, 0, 400,500,300);Now, whenever i need to look for 250, i search for closest smaller key (here that is 200). find values. whatever is available is taken as it is. For all others, find the key in the same record and get the new entries. This solution works for the first time entries. However it creates problems when an en
Namaste Sri Som, [edit : made array of "current value" nullable integers private and non-static after realizing that by making the array static you could only have one usable instance of the class : please remember this code was written in less than five minutes "off top of my head" : welcome any suggestions to improve it ! ] Here's an idea off the "top of my head" : 1. use a class inheriting from a generic dictionary (we'll use an int [non-nullable] here for a key, but, of course, you could use something else as a key) 2. for the value part of each dictionary entry use a List of nullable ints List<int?> were going to make use of nulllable here to simulate "missing" data entries. 3. use an array of nullable ints inside the dictionary to track the current values you may need to replace "missing values" with : that gets us around the problem that a .NET Dictionary has no inherent "ordinality" you can rely on.
public class keyedNullableIntTable : Dictionary<int, List<int?>> { private int?\[\] referenceValues = new int?\[3\]; public void Add(int theKey, List<int?> theValues) { for (int i = 0; i < 3; i++) { if (theValues\[i\] == null) { if (referenceValues\[i\] != null) { theValues\[i\] = referenceValues\[i\]; } } else { referenceValues\[i\] = theValues\[i\]; } } base.Add(theKey, theValues); } }Here's a sample test : assumes you have a form with a 'button1 on it to call the eventhandler :
private void button1\_Click(object sender, EventArgs e) { keyedNullableIntTable t1 = new keyedNullableIntTable(); t1.Add(1, new List<int?> { 100,null,300 }); t1.Add(2, new List<int?> { null, null, 300 }); t1.Add(3, new List<int?> { 222, 333, 444 }); t1.Add(4, new List<int?> { 111, null, null }); t1.Add(5, new List<int?> { null, null, null }); foreach (var theEntry in t1) { Console.Write("key = " + theEntry.Key + "\\tlist = "); foreach (var listValue in theEntry.Value) { -
I do not believe that there is something built into the framework. There may be something in the big wide internet, but I'm not sure what I'd use to look it up. Are the items all indexed concsecutively? If so, you will probably want to look for something that implements
IListinstead ofIDictionary. The reason other people suggested Dictionary is probably because you said your list had "keys" and "values" implying possible non-consecutive indexes. Making such a collection yourself is not that hard. I put together a sample of how I would do it. It turned out to ~400 lines, most of which were boilerplate. The Tuple class from .NET 4.0 would be helpful here, but you could create one yourself pretty easily. My sample class definition was:public class DeltaCollection<T1, T2, T3> :
IList,
IList<Tuple<T1, T2, T3>>,
IList<Tuple<T1?, T2?, T3?>>
where T1 : struct, T2 : struct, T3 : struct(I did it in VB, so I may have messed up the generic constraint syntax.) Update I played around some more and was able to get it down to ~200 lines +15-30 lines per tuple size (number of values per row) you want to support. Again, this would be benefited greatly by .NET 4.0's Tuple class, which had to be reimplemented to use in my class now. The main difference between this version and the original is that the Item property and
foreachenumerator will now returnTuple<T1?, T2?, T3?>instead ofTuple<T1, T2, T3>.modified on Wednesday, December 16, 2009 11:04 PM
I am working on 3.5 sp1. So, can't use Tuple for now. Please read the comments in the modified question.
-
Value1, value2 and value3 are distinct entities that you want to track? A dictionary consists of a value and a key. The key would be your index-column, the value could be a struct that holds the three columns;
System.Collections.Generic.Dictionary<int, MyValues> myListOfValues;
struct MyValues
{
int value1;
int value2;
int value3;
}You could consider the generic list as your datatable, and the struct as a single row in that table. Now you can add values to your dictionary;
MyValues oneRow = new MyValues();
oneRow.value1 = 100;
oneRow.value2 = 200;
oneRow.value3 = 300;
myListOfValues.Add(1, oneRow);Now, if you want the recreate those values for index 3, you would have to search in the list for the first non-empty value for the column value2. If speed is important, then I'd create one row outside of that list, just to mirror the current values. That wouldn't help if you need the values from the middle of the index, you'd still have to walk the list for that. How many records will be in that list? It might just be worth to store copies of the latest value, instead of looking for them.
I are Troll :suss:
I modified the question to answer all the people who replied. Please read that. And yes. thanks for the attempt.
-
Have each object maintain a dictionary of values. In the samples you rpovided above: Index 1 would contain 1,100 2,200 3,300 Index 2 would contain 3, 200 Index 3 would contain 1, 150 3, 300 Index 4 would contain 2,250 Index 5 would contain 1,100 2,200 Use google to learn how to use dictionary collections.
.45 ACP - because shooting twice is just silly
-----
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"The staggering layers of obscenity in your statement make it a work of art on so many levels." - J. Jystad, 2001Thanks for the attempt. I have modified the question for all the responses. Answering the same answer to all replies was not a good idea. Please give it a read.
-
Namaste Sri Som, [edit : made array of "current value" nullable integers private and non-static after realizing that by making the array static you could only have one usable instance of the class : please remember this code was written in less than five minutes "off top of my head" : welcome any suggestions to improve it ! ] Here's an idea off the "top of my head" : 1. use a class inheriting from a generic dictionary (we'll use an int [non-nullable] here for a key, but, of course, you could use something else as a key) 2. for the value part of each dictionary entry use a List of nullable ints List<int?> were going to make use of nulllable here to simulate "missing" data entries. 3. use an array of nullable ints inside the dictionary to track the current values you may need to replace "missing values" with : that gets us around the problem that a .NET Dictionary has no inherent "ordinality" you can rely on.
public class keyedNullableIntTable : Dictionary<int, List<int?>> { private int?\[\] referenceValues = new int?\[3\]; public void Add(int theKey, List<int?> theValues) { for (int i = 0; i < 3; i++) { if (theValues\[i\] == null) { if (referenceValues\[i\] != null) { theValues\[i\] = referenceValues\[i\]; } } else { referenceValues\[i\] = theValues\[i\]; } } base.Add(theKey, theValues); } }Here's a sample test : assumes you have a form with a 'button1 on it to call the eventhandler :
private void button1\_Click(object sender, EventArgs e) { keyedNullableIntTable t1 = new keyedNullableIntTable(); t1.Add(1, new List<int?> { 100,null,300 }); t1.Add(2, new List<int?> { null, null, 300 }); t1.Add(3, new List<int?> { 222, 333, 444 }); t1.Add(4, new List<int?> { 111, null, null }); t1.Add(5, new List<int?> { null, null, null }); foreach (var theEntry in t1) { Console.Write("key = " + theEntry.Key + "\\tlist = "); foreach (var listValue in theEntry.Value) {This only solves the problem of creating the dictionary. Question is for retrieving them back. I have added my comments to the question itself. There are some more areas to look at. Please give it a read. And yes. thanks for the attempt.
-
This only solves the problem of creating the dictionary. Question is for retrieving them back. I have added my comments to the question itself. There are some more areas to look at. Please give it a read. And yes. thanks for the attempt.
Namaste Suits just fine. However, you could do with a Hello too. (We are too used to "Hello World" anyways) :laugh:
-
This only solves the problem of creating the dictionary. Question is for retrieving them back. I have added my comments to the question itself. There are some more areas to look at. Please give it a read. And yes. thanks for the attempt.
Som Shekhar wrote: "I have added my comments to the question itself. There are some more areas to look at." Hi Som, Have edited the first code example to make the internal list of "current values to replace with if incoming item# is null" private and not static. Will be able to review your comments later tonight (I live at GMT +7 by the way) to try to understand what you mean by "retrieving them back" : isn't the test example I show in the code ... where the keys and list values are being read out in a foreach loop ... and printed to the console ... an example of retrieving back the values ? If I want the 3rd. item in the List in the dictionary t1 which is accessed by the key #4 : and I access it via : t1[4][2] Isn't that retrieving ? Namaste, Bill
"Many : not conversant with mathematical studies, imagine that because it [the Analytical Engine] is to give results in numerical notation, its processes must consequently be arithmetical, numerical, rather than algebraical and analytical. This is an error. The engine can arrange and combine numerical quantities as if they were letters or any other general symbols; and it fact it might bring out its results in algebraical notation, were provisions made accordingly." Ada, Countess Lovelace, 1844
-
I modified the question to answer all the people who replied. Please read that. And yes. thanks for the attempt.
Som Shekhar wrote:
Can there be something faster than this?
Yup; don't store the empty values, but do the looping on insert. Meaning that when you insert the values (null, 100, null), that you'll loop the list *at that moment* to fetch the null-values and store them at that index. It's either a lookup on insert (fast, consumes more memory) or a lookup when you're reading (slower, less memory-pressure). AFAIK, you can't have both (yet)
I are Troll :suss:
-
Som Shekhar wrote:
Can there be something faster than this?
Yup; don't store the empty values, but do the looping on insert. Meaning that when you insert the values (null, 100, null), that you'll loop the list *at that moment* to fetch the null-values and store them at that index. It's either a lookup on insert (fast, consumes more memory) or a lookup when you're reading (slower, less memory-pressure). AFAIK, you can't have both (yet)
I are Troll :suss:
I prefer anything to be done at the insert stage. So, that's taken. Storing null values depends if it is an object or a list. In case of an class object constructor may demand all values and hence the null value. It can be skipped if possible. Doesn't any implementation exist in this format? A very practical example will be Configuration Plans which is time bound. We may only want to save the planned changes and not at every stage.
-
I prefer anything to be done at the insert stage. So, that's taken. Storing null values depends if it is an object or a list. In case of an class object constructor may demand all values and hence the null value. It can be skipped if possible. Doesn't any implementation exist in this format? A very practical example will be Configuration Plans which is time bound. We may only want to save the planned changes and not at every stage.
Som Shekhar wrote:
Doesn't any implementation exist in this format?
I don't know of any in the Framework, but then again, I don't know the entire framework by heart yet.
Som Shekhar wrote:
We may only want to save the planned changes and not at every stage.
Like I said, memory is cheap these days - and I doubt that you'll save a lot by inserting null values for a non-changed field. These kind of constructs are very common in SQL Server, where one would use a query to recreate the full list. The query does the lookups (over a speedy index) and you can read it as if it were the original table. ..but in-memory? Short lists should just store the values in a redundant way, and long lists shouldn't be kept in memory at all.
I are Troll :suss:
-
Som Shekhar wrote:
Doesn't any implementation exist in this format?
I don't know of any in the Framework, but then again, I don't know the entire framework by heart yet.
Som Shekhar wrote:
We may only want to save the planned changes and not at every stage.
Like I said, memory is cheap these days - and I doubt that you'll save a lot by inserting null values for a non-changed field. These kind of constructs are very common in SQL Server, where one would use a query to recreate the full list. The query does the lookups (over a speedy index) and you can read it as if it were the original table. ..but in-memory? Short lists should just store the values in a redundant way, and long lists shouldn't be kept in memory at all.
I are Troll :suss:
Eddy Vluggen wrote:
Like I said, memory is cheap these days
Memory is not a trouble. Speed is. This is for a single such list. When this kind of list exist in 100s and each has to be going through a for loop with multiple calculations in between, the calculations go in 0.2-0.3 sec range. this is not too much to look at but with drag and drop, 0.2-0.3 lag is not acceptable. I am just trying to go to 0.05-0.06 sec range. These kind of change cannot come by simply tweaking a line or two. It can only come by changing the whole method of searching records. By the way, this whole thing is not even happening in the server/sql. its happening in the memory... hence the speed trouble.
-
Eddy Vluggen wrote:
Like I said, memory is cheap these days
Memory is not a trouble. Speed is. This is for a single such list. When this kind of list exist in 100s and each has to be going through a for loop with multiple calculations in between, the calculations go in 0.2-0.3 sec range. this is not too much to look at but with drag and drop, 0.2-0.3 lag is not acceptable. I am just trying to go to 0.05-0.06 sec range. These kind of change cannot come by simply tweaking a line or two. It can only come by changing the whole method of searching records. By the way, this whole thing is not even happening in the server/sql. its happening in the memory... hence the speed trouble.
Som Shekhar wrote:
By the way, this whole thing is not even happening in the server/sql. its happening in the memory... hence the speed trouble.
The good news is that memory is usually faster to access then a server :)
Som Shekhar wrote:
each has to be going through a for loop with multiple calculations in between
The only option that I'm aware of is to precalculate the missing values. That way you only have a lookup in a list, which is quite fast. So, instead of inserting the nulls for the unchanged values, insert the value. That would move the cost for inserting from the "read from the list"-part to the "add to the list"-part of the code.
I are Troll :suss:
-
Som Shekhar wrote:
By the way, this whole thing is not even happening in the server/sql. its happening in the memory... hence the speed trouble.
The good news is that memory is usually faster to access then a server :)
Som Shekhar wrote:
each has to be going through a for loop with multiple calculations in between
The only option that I'm aware of is to precalculate the missing values. That way you only have a lookup in a list, which is quite fast. So, instead of inserting the nulls for the unchanged values, insert the value. That would move the cost for inserting from the "read from the list"-part to the "add to the list"-part of the code.
I are Troll :suss:
If you read the modified question, i am saying an index along with as where to lookup again next time. Saving values instead of null may have problems later one when more values are inserted or removed. The whole purpose of the list is to save the value where changed.
-
If you read the modified question, i am saying an index along with as where to lookup again next time. Saving values instead of null may have problems later one when more values are inserted or removed. The whole purpose of the list is to save the value where changed.
Som Shekhar wrote:
Saving values instead of null may have problems later one when more values are inserted or removed. The whole purpose of the list is to save the value where changed.
It sounded like the main issue was to have a list with values that you can query at a high speed. If you save all the values, then you won't have the problems you mentioned when removing an item;
Col1 Col2 Col3
100 200 300
100 200 100
100 100 100Remove the second line, all data will still be correct. On the other hand, if you store the null-values (to indicate a non-change), then you might run into trouble;
Col1 Col2 Col3
100 200 300
? ? 100
? 100 ?If you remove the second line now, you'll get this;
Col1 Col2 Col3
100 200 300
? 100 ?Which decodes to this when you try to track back the changes;
Col1 Col2 Col3
100 200 300
100 100 300As you can see, the last item has changed. I don't see the advantage of storing only the changes in this particular structure, only more challenges :( It seems that I don't understand the question well enough to provide you with an answer.
I are Troll :suss:
-
Som Shekhar wrote:
Saving values instead of null may have problems later one when more values are inserted or removed. The whole purpose of the list is to save the value where changed.
It sounded like the main issue was to have a list with values that you can query at a high speed. If you save all the values, then you won't have the problems you mentioned when removing an item;
Col1 Col2 Col3
100 200 300
100 200 100
100 100 100Remove the second line, all data will still be correct. On the other hand, if you store the null-values (to indicate a non-change), then you might run into trouble;
Col1 Col2 Col3
100 200 300
? ? 100
? 100 ?If you remove the second line now, you'll get this;
Col1 Col2 Col3
100 200 300
? 100 ?Which decodes to this when you try to track back the changes;
Col1 Col2 Col3
100 200 300
100 100 300As you can see, the last item has changed. I don't see the advantage of storing only the changes in this particular structure, only more challenges :( It seems that I don't understand the question well enough to provide you with an answer.
I are Troll :suss:
Hey there!!! I really appreciate your efforts. But the question is to know if I am missing something fundamentally? If there is an implementation already present? Like we have Hashtables, List, Dictionaries for various purposes. Is there any other tool that I missed which can handle such a case? Or, may be there could be a need to develop such a list which could only record changes, do all the calculations internally, fast enough to match those of dictionary/indexed methods. In any case, thanks once again.
-
I am working on 3.5 sp1. So, can't use Tuple for now. Please read the comments in the modified question.
Well, you could make your own Tuple if you really wanted, but it looks like you already have a similar class to use instead. Like Eddy said, for your situation, you will have to loop somewhere. Either you have to keep a continuous list with the calculated values for all indexes up to the highest one, which implies a loop on every item insert/change, or you have to perform a search for the next lowest item and calculate its values on item access. It sounds like you want array-like access performance, so you will probably want to take the first option of calculating the values on insert (as Eddy suggested). If you will be accessing the items in sequence and not randomly indexing, you could get the best of both worlds by writing an iterator that keeps a "current value" and "index" to keep the calculation time minimal. A quick search did not find any existing implementation for such a collection, but it should not be that difficult to roll your own (especially since you know exactly what features you want).
