Generic collections
-
Often when you have a problem that looks like "I need a data structure that has the properties of A and of B" where A and B are some data structure, there's an obvious way to just combine A and B and wrap them a little thing that keeps them synchronized. Such as here, it looks like a List and like a Dictionary.. what can you do? Well you can make a List<Tvalue> and a Dictionary<Tkey, int> and let the dictionary map keys to indexes in the list. If you also want removal, you can use a linked list instead of a list, and let the dictionary map keys to nodes in the linked list.
I did the same as a workaround solution... thanks a lot
-
I really like (and upvoted) your question; it was fun to see what I could implement quickly, and I'm going to post the code here before I read the other responses on the thread, since I feel I can learn something from knowing whether or not the code I post actually meets your requirements, and, possibly, learning from other people's responses. I wonder if I have not just duplicated the functionality of a Dictionary<K, V> ? Also, I suspect Linq queries could be used in some way, efficiently here, rather than, as I did, creating two Lists in the class (memory cost ?). Disclaimer: I've tested only with KeyValuePair<string, double> since that's what you used. Testing with instances of a custom class as K, or V: that's on my list :) Here's the revised class:
public class IndexedKVPList<K, V> : List<KeyValuePair<K, V>>
{
private readonly List<K> keyList = new List<K>();
private readonly List<V> valueList = new List<V>();// modified to use 'new new public void Add(KeyValuePair<K, V> newKVP) { keyList.Add(newKVP.Key); valueList.Add(newKVP.Value); base.Add(newKVP); } // new function to Remove KeyValue pair new public void Remove(KeyValuePair<K, V> killKVP) { if(! this.Contains(killKVP)) throw new Exception("error in Remove in IndexedKVPPair"); keyList.Remove(killKVP.Key); valueList.Remove(killKVP.Value); base.Remove(killKVP); } public KeyValuePair<K, V> GetKVPByValue(V theValue) { return this\[valueList.IndexOf(theValue)\]; } public K GetKeyByValue(V theValue) { return this\[valueList.IndexOf(theValue)\].Key; } // new function to get the index by KeyValue pair public int GetIndexByKVP(KeyValuePair<K, V> theKVPPair) { return this.IndexOf(theKVPPair); } // new function to get the index by Value // what happens if there are multiple identical values ? public int GetIndexByValue(V theValue) { return (valueList.IndexOf(theValue)); } public KeyValuePair<K, V> GetKVPByKey(K theKey) { return this\[keyList.IndexOf(theKey)\]; } public V GetValueByKey(K theKey) { return this\[keyList.IndexOf(theKey)\].Value; } // new function to get the index by Key public int GetIndexByKey(K theKey) { reThanks a lot, I did a workaround using a syncronized List.. Then I found the following solution
Dictionary d = new Dictionary();
d.Add("z", 1); d.Add("a", 2); d.Add("c", 3); d.Add("b", 4); foreach (KeyValuePair k in d) { Console.WriteLine(k.Key + " " + k.Value); } KeyValuePair i\_item = d.ElementAt(2); int myIndex = Array.IndexOf(d.Keys.ToArray(), "a"); -
Often when you have a problem that looks like "I need a data structure that has the properties of A and of B" where A and B are some data structure, there's an obvious way to just combine A and B and wrap them a little thing that keeps them synchronized. Such as here, it looks like a List and like a Dictionary.. what can you do? Well you can make a List<Tvalue> and a Dictionary<Tkey, int> and let the dictionary map keys to indexes in the list. If you also want removal, you can use a linked list instead of a list, and let the dictionary map keys to nodes in the linked list.
Hi Harold, Upvoted, as usual: why is it that when I read your comments, I often feel like I am listening to Yoda trying to tell me something through metaphor that's just beyond my ability to grasp ? :) yours, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
-
Thanks a lot, I did a workaround using a syncronized List.. Then I found the following solution
Dictionary d = new Dictionary();
d.Add("z", 1); d.Add("a", 2); d.Add("c", 3); d.Add("b", 4); foreach (KeyValuePair k in d) { Console.WriteLine(k.Key + " " + k.Value); } KeyValuePair i\_item = d.ElementAt(2); int myIndex = Array.IndexOf(d.Keys.ToArray(), "a");It may be a little faster to get the index using:
private int theIndex;
//
theIndex = dictionaryStringInt.Keys.ToList<string>().IndexOf("KeyString");However, note that in using both 'ElementAt, and 'ToArray ... or 'ToList ... you are using Linq functions that are, internally, going to do an enumeration. If the entries in the dictionary are not being removed, moved around, etc., then you could create a List once from the Keys collection, however, I doubt that's the case you want here. I will add some extra functions that return an index to my first answer above, with the hope some of our members here who have more formal computer science training (than I do) will comment on the relative cost / lookup-times, of the example I've given. good luck, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
-
Thanks a lot, I did a workaround using a syncronized List.. Then I found the following solution
Dictionary d = new Dictionary();
d.Add("z", 1); d.Add("a", 2); d.Add("c", 3); d.Add("b", 4); foreach (KeyValuePair k in d) { Console.WriteLine(k.Key + " " + k.Value); } KeyValuePair i\_item = d.ElementAt(2); int myIndex = Array.IndexOf(d.Keys.ToArray(), "a");That doesn't work. Or rather, it may work - accidentally. But see http://msdn.microsoft.com/en-us/library/yt2fy5zk.aspx[^]:
The order of the keys in the Dictionary.KeyCollection is unspecified
It is not (necessarily) in the order that you added the items. It often works. But it isn't guaranteed to. edit: it turns out that, in the current implementation (which may change!) it will indeed work as long as you never delete anything. That's just an implementation detail, it would be foolish to rely on it. It could change in any update. Also, of course, you're doing a linear search through a list. That is not ideal.
-
It may be a little faster to get the index using:
private int theIndex;
//
theIndex = dictionaryStringInt.Keys.ToList<string>().IndexOf("KeyString");However, note that in using both 'ElementAt, and 'ToArray ... or 'ToList ... you are using Linq functions that are, internally, going to do an enumeration. If the entries in the dictionary are not being removed, moved around, etc., then you could create a List once from the Keys collection, however, I doubt that's the case you want here. I will add some extra functions that return an index to my first answer above, with the hope some of our members here who have more formal computer science training (than I do) will comment on the relative cost / lookup-times, of the example I've given. good luck, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
thanks very useful
-
Hi Harold, Upvoted, as usual: why is it that when I read your comments, I often feel like I am listening to Yoda trying to tell me something through metaphor that's just beyond my ability to grasp ? :) yours, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
:)
-
That doesn't work. Or rather, it may work - accidentally. But see http://msdn.microsoft.com/en-us/library/yt2fy5zk.aspx[^]:
The order of the keys in the Dictionary.KeyCollection is unspecified
It is not (necessarily) in the order that you added the items. It often works. But it isn't guaranteed to. edit: it turns out that, in the current implementation (which may change!) it will indeed work as long as you never delete anything. That's just an implementation detail, it would be foolish to rely on it. It could change in any update. Also, of course, you're doing a linear search through a list. That is not ideal.
oky thank
-
That doesn't work. Or rather, it may work - accidentally. But see http://msdn.microsoft.com/en-us/library/yt2fy5zk.aspx[^]:
The order of the keys in the Dictionary.KeyCollection is unspecified
It is not (necessarily) in the order that you added the items. It often works. But it isn't guaranteed to. edit: it turns out that, in the current implementation (which may change!) it will indeed work as long as you never delete anything. That's just an implementation detail, it would be foolish to rely on it. It could change in any update. Also, of course, you're doing a linear search through a list. That is not ideal.
harold aptroot wrote:
you're doing a linear search through a list.
Hi Harold, I am curious: do you know for a fact that using Linq 'ElementAt, or using the (not Linq) 'IndexOf on an Array, or List, internally performs a linear search ? In the case of 'ElementAt on a Dictionary, I am assuming there's some cost of some kind of internal conversion since Dictionary doesn't inherently support "sequential" indexing by integer. thanks, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
-
harold aptroot wrote:
you're doing a linear search through a list.
Hi Harold, I am curious: do you know for a fact that using Linq 'ElementAt, or using the (not Linq) 'IndexOf on an Array, or List, internally performs a linear search ? In the case of 'ElementAt on a Dictionary, I am assuming there's some cost of some kind of internal conversion since Dictionary doesn't inherently support "sequential" indexing by integer. thanks, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
The IndexOf of a List uses the IndexOf of Array, and that ones's a linear search (I checked it just be sure, but really there's no other choice anyway). What ElementAt does depends on the thing you're applying it to, it will use IList<T>'s indexer if that interface is implemented (Dictionary does not implement it, so ElementAt would scan through the enumerator). So in conclusion, that code was pretty inefficient.
-
The IndexOf of a List uses the IndexOf of Array, and that ones's a linear search (I checked it just be sure, but really there's no other choice anyway). What ElementAt does depends on the thing you're applying it to, it will use IList<T>'s indexer if that interface is implemented (Dictionary does not implement it, so ElementAt would scan through the enumerator). So in conclusion, that code was pretty inefficient.
Thanks for your reply, Harold, that is very interesting to know. It makes me wonder: if you were dealing with a really big generic List, or a huge Dictionary, if you could implement a much more efficient solution than the current operators, and, if so, why MS has not done that. I would have thought that Linq operators, particularly, were highly optimized with the latest algorithms: reminds me I should never assume anything about software tools. yours, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
-
Thanks for your reply, Harold, that is very interesting to know. It makes me wonder: if you were dealing with a really big generic List, or a huge Dictionary, if you could implement a much more efficient solution than the current operators, and, if so, why MS has not done that. I would have thought that Linq operators, particularly, were highly optimized with the latest algorithms: reminds me I should never assume anything about software tools. yours, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
There aren't many options there.. (there are some, discussed below) I mean, say you have an array and you want to know the index of some value and you have an algorithm that does not potentially look at all elements, how can it know the thing it's trying to find isn't in the part that it hasn't looked at? Clearly it can't, so in the worst case you're always looking at all elements anyway. Leaving the "standard complexity model" behind for a moment, you can do a lot better than O(n): divide the array in m parts, search each part in parallel, then do a Min-Reduction phase on all the results. That's O(log m) in PRAM (where the assumption m = n is allowed), but PRAM is not a realistic model for present-day CPU computing. In the CPU world that same algorithm is still O(n) (because it's really O(n/m + log m) and m is a constant now), which doesn't necessarily have to be a bad thing, not all O(n)'s are created equally after all. But this isn't a particularly awesome algorithm: 1) it spends a lot of time just fetching things from memory, and doesn't do a whole lot with them. (of course the normal algorithm does that too, that it will be a worse bottleneck when multiple threads are doing it) 2) even if you abort the threads working on the higher ranges (or if you want any index, not just the lowest: all other threads) when you find the item, you will have wasted work, so even when it's faster it takes more resources. 3) it has a lot of overhead from all the thread business. In a quick test, this algorithms start pulling ahead on average for an array of a million integers, and then only when the item is not actually in the array (the worst case for the simple linear search). It almost reaches a speedup of 4x on my machine (which isn't that bad considering it's a quad core + hyperthreading) but each time an item actually is in the array, the multithreaded algorithm loses or almost loses, no matter how big I make the array (tested up to an array of 228 ints). So it's not really suitable as a default algorithm for libraries: it's terrible in the common case that the item is actually in the array.
-
There aren't many options there.. (there are some, discussed below) I mean, say you have an array and you want to know the index of some value and you have an algorithm that does not potentially look at all elements, how can it know the thing it's trying to find isn't in the part that it hasn't looked at? Clearly it can't, so in the worst case you're always looking at all elements anyway. Leaving the "standard complexity model" behind for a moment, you can do a lot better than O(n): divide the array in m parts, search each part in parallel, then do a Min-Reduction phase on all the results. That's O(log m) in PRAM (where the assumption m = n is allowed), but PRAM is not a realistic model for present-day CPU computing. In the CPU world that same algorithm is still O(n) (because it's really O(n/m + log m) and m is a constant now), which doesn't necessarily have to be a bad thing, not all O(n)'s are created equally after all. But this isn't a particularly awesome algorithm: 1) it spends a lot of time just fetching things from memory, and doesn't do a whole lot with them. (of course the normal algorithm does that too, that it will be a worse bottleneck when multiple threads are doing it) 2) even if you abort the threads working on the higher ranges (or if you want any index, not just the lowest: all other threads) when you find the item, you will have wasted work, so even when it's faster it takes more resources. 3) it has a lot of overhead from all the thread business. In a quick test, this algorithms start pulling ahead on average for an array of a million integers, and then only when the item is not actually in the array (the worst case for the simple linear search). It almost reaches a speedup of 4x on my machine (which isn't that bad considering it's a quad core + hyperthreading) but each time an item actually is in the array, the multithreaded algorithm loses or almost loses, no matter how big I make the array (tested up to an array of 228 ints). So it's not really suitable as a default algorithm for libraries: it's terrible in the common case that the item is actually in the array.
Once again, thanks, for that very illuminating reply ! I'd say if you tested up to 228, that would cover most non-astronomical level scenarios ! Of more interest, and something I actually intend to explore, is what happens when have compound structures with KeyValue pairs, and both Key and Value Types are complex classes, not just ValueTypes. ... edit ... on second thought: if K and V are complex classes, I think it's just a case of comparing pointers in any IndexOf, or ElementAt, type operation, so: no difference than if K and V were .NET ReferenceTypes (?) ... Please come to northern Thailand (in late-October~early-January is best), and let me take you, and your family, elephant riding in the jungle :) yours, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
-
Once again, thanks, for that very illuminating reply ! I'd say if you tested up to 228, that would cover most non-astronomical level scenarios ! Of more interest, and something I actually intend to explore, is what happens when have compound structures with KeyValue pairs, and both Key and Value Types are complex classes, not just ValueTypes. ... edit ... on second thought: if K and V are complex classes, I think it's just a case of comparing pointers in any IndexOf, or ElementAt, type operation, so: no difference than if K and V were .NET ReferenceTypes (?) ... Please come to northern Thailand (in late-October~early-January is best), and let me take you, and your family, elephant riding in the jungle :) yours, Bill
~ “This isn't right; this isn't even wrong." Wolfgang Pauli, commenting on a physics paper submitted for a journal
BillWoodruff wrote:
Of more interest, and something I actually intend to explore, is what happens when have compound structures with KeyValue pairs, and both Key and Value Types are complex classes, not just ValueTypes.
It depends, it calls Equals(T other) when IEquatable<T> is implemented (in that type or a base class), regular Equals otherwise which might default to Object.Equals, in that case it uses Reference Equality for reference types and Bitwise Equality for value types (that's how the official documentation puts it), which could be restated as "always bitwise equality, so if you give it a reference it's bitwise equality on the reference itself" (that might help a person who thinks in pointers, it would probably confuse someone who thinks in objects). The code where it tests to see which case it's in and then makes an EqualityComparer uses a lot of reflection, by the way. Thanks for inviting me by the way, but I've found that jungles aren't my thing, so I don't think I'll do anything with that..