Most efficient Collection?
-
Whats the most efficient way of storing and referecing a list of ~2000 unsorted key value pairs in C#? I understand that there is not really any definite answer here since it will depend on the data, however, I'm not familiar enough with the .NET framework to know which ones to even test. I'm an MFC C++ developer who is migrating to C#. In MFC C++ I would use a CMap or a muiltidimensional array to store the data and it would be efficient enough for my needs. I'm finding in C#, neither (a series of arrays or a Hashtable) is working out very well. Both seem to be far less efficient than my implementations in C++ but I'm assuming this is mostly because of my inability to code C# as well as I do in C++. For reference, I need to store 5-6 values for ~2000 keys (3-30 character strings) and receive between 200-10,000 updates a second (averaging about 750) total. Recomendations? Should I look at another data storage method or should I work on improving my existing attempts?
-
Whats the most efficient way of storing and referecing a list of ~2000 unsorted key value pairs in C#? I understand that there is not really any definite answer here since it will depend on the data, however, I'm not familiar enough with the .NET framework to know which ones to even test. I'm an MFC C++ developer who is migrating to C#. In MFC C++ I would use a CMap or a muiltidimensional array to store the data and it would be efficient enough for my needs. I'm finding in C#, neither (a series of arrays or a Hashtable) is working out very well. Both seem to be far less efficient than my implementations in C++ but I'm assuming this is mostly because of my inability to code C# as well as I do in C++. For reference, I need to store 5-6 values for ~2000 keys (3-30 character strings) and receive between 200-10,000 updates a second (averaging about 750) total. Recomendations? Should I look at another data storage method or should I work on improving my existing attempts?
-
Whats the most efficient way of storing and referecing a list of ~2000 unsorted key value pairs in C#? I understand that there is not really any definite answer here since it will depend on the data, however, I'm not familiar enough with the .NET framework to know which ones to even test. I'm an MFC C++ developer who is migrating to C#. In MFC C++ I would use a CMap or a muiltidimensional array to store the data and it would be efficient enough for my needs. I'm finding in C#, neither (a series of arrays or a Hashtable) is working out very well. Both seem to be far less efficient than my implementations in C++ but I'm assuming this is mostly because of my inability to code C# as well as I do in C++. For reference, I need to store 5-6 values for ~2000 keys (3-30 character strings) and receive between 200-10,000 updates a second (averaging about 750) total. Recomendations? Should I look at another data storage method or should I work on improving my existing attempts?
if you want to remember only one value (the most recent one) for each key you encounter, then yes a Dictionary is what you want, and
myDictionary[myKey]=myValue;
is all it takes. Dictionaries are hashed; they are built on top of arrays, which when running out of their capacity are copied to larger ones. Therefore it may make sense to use a constructor overload that takes an initial capacity parameter. :)
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
-
Whats the most efficient way of storing and referecing a list of ~2000 unsorted key value pairs in C#? I understand that there is not really any definite answer here since it will depend on the data, however, I'm not familiar enough with the .NET framework to know which ones to even test. I'm an MFC C++ developer who is migrating to C#. In MFC C++ I would use a CMap or a muiltidimensional array to store the data and it would be efficient enough for my needs. I'm finding in C#, neither (a series of arrays or a Hashtable) is working out very well. Both seem to be far less efficient than my implementations in C++ but I'm assuming this is mostly because of my inability to code C# as well as I do in C++. For reference, I need to store 5-6 values for ~2000 keys (3-30 character strings) and receive between 200-10,000 updates a second (averaging about 750) total. Recomendations? Should I look at another data storage method or should I work on improving my existing attempts?
Interested in what kind of 'performance' you are seeing and if you are running these tests in debug or release and if you are timing this in the IDE or not. I have seen some interesting differences in speed and performance between these configurations before but when doing a test in a full production build outside the IDE they have ended up as very close to a wash. Also you may want to look at allocating new additions to the collections in groups rather than single records at a time. I have also seen that help performance wise.
-
Whats the most efficient way of storing and referecing a list of ~2000 unsorted key value pairs in C#? I understand that there is not really any definite answer here since it will depend on the data, however, I'm not familiar enough with the .NET framework to know which ones to even test. I'm an MFC C++ developer who is migrating to C#. In MFC C++ I would use a CMap or a muiltidimensional array to store the data and it would be efficient enough for my needs. I'm finding in C#, neither (a series of arrays or a Hashtable) is working out very well. Both seem to be far less efficient than my implementations in C++ but I'm assuming this is mostly because of my inability to code C# as well as I do in C++. For reference, I need to store 5-6 values for ~2000 keys (3-30 character strings) and receive between 200-10,000 updates a second (averaging about 750) total. Recomendations? Should I look at another data storage method or should I work on improving my existing attempts?
Like others said, Dictionary<>() is a good bet. However, I wouldn't recommend using 3 - 30 character strings as the key. Performance will be better if you use something lighter like an int or something. You really haven't given us enough information to make a more informed design.
-
Whats the most efficient way of storing and referecing a list of ~2000 unsorted key value pairs in C#? I understand that there is not really any definite answer here since it will depend on the data, however, I'm not familiar enough with the .NET framework to know which ones to even test. I'm an MFC C++ developer who is migrating to C#. In MFC C++ I would use a CMap or a muiltidimensional array to store the data and it would be efficient enough for my needs. I'm finding in C#, neither (a series of arrays or a Hashtable) is working out very well. Both seem to be far less efficient than my implementations in C++ but I'm assuming this is mostly because of my inability to code C# as well as I do in C++. For reference, I need to store 5-6 values for ~2000 keys (3-30 character strings) and receive between 200-10,000 updates a second (averaging about 750) total. Recomendations? Should I look at another data storage method or should I work on improving my existing attempts?
WTF? This is not what I typed, and I can vote on it. I can't delete or edit it. Whats is wrong with this? Also the other Answers do not appear to relate to the Question.
------------------------------------ I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave CCC League Table Link CCC Link[^]
-
WTF? This is not what I typed, and I can vote on it. I can't delete or edit it. Whats is wrong with this? Also the other Answers do not appear to relate to the Question.
------------------------------------ I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave CCC League Table Link CCC Link[^]
Dave, whatever it is you are doing in the C# forum, it is disruptive: the top thread is cut short, the next one is decapitated. You may want to get into your topmost message there and fix something. Thanks. If you could note what the content is, and how things get fixed, it probably would be material for a sucks&bugs thread. :)
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
modified on Thursday, January 13, 2011 10:11 PM
-
Dave, whatever it is you are doing in the C# forum, it is disruptive: the top thread is cut short, the next one is decapitated. You may want to get into your topmost message there and fix something. Thanks. If you could note what the content is, and how things get fixed, it probably would be material for a sucks&bugs thread. :)
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
modified on Thursday, January 13, 2011 10:11 PM
I replied showing some code to give a total for the cells in his DGV. There was nothing like what appears in my post up there. Something is amiss! It wasn't me, it was a big boy who did it and then he ran away!
------------------------------------ I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave CCC League Table Link CCC Link[^]
-
I replied showing some code to give a total for the cells in his DGV. There was nothing like what appears in my post up there. Something is amiss! It wasn't me, it was a big boy who did it and then he ran away!
------------------------------------ I will never again mention that I was the poster of the One Millionth Lounge Post, nor that it was complete drivel. Dalek Dave CCC League Table Link CCC Link[^]
OK, I'll take it to S&B authorities then. :)
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
-
Whats the most efficient way of storing and referecing a list of ~2000 unsorted key value pairs in C#? I understand that there is not really any definite answer here since it will depend on the data, however, I'm not familiar enough with the .NET framework to know which ones to even test. I'm an MFC C++ developer who is migrating to C#. In MFC C++ I would use a CMap or a muiltidimensional array to store the data and it would be efficient enough for my needs. I'm finding in C#, neither (a series of arrays or a Hashtable) is working out very well. Both seem to be far less efficient than my implementations in C++ but I'm assuming this is mostly because of my inability to code C# as well as I do in C++. For reference, I need to store 5-6 values for ~2000 keys (3-30 character strings) and receive between 200-10,000 updates a second (averaging about 750) total. Recomendations? Should I look at another data storage method or should I work on improving my existing attempts?