Deserializing Generic List<object> taking a lot of time</object>
-
Hello Guys, Need some help from you guys. I have Generic List having a Class object. I have added around 3000000 items in the list
List <CTempClass> lsTemp = new List <CTempClass> ();
when i tried to Serialize the with binary formater it is working fine but when I deserialize the List it takes a lot of time around 30 Seconds. But when i fill the Generic List with any datatype likeList<ulong> lsTemp = new List<ulong>();
and follow the same procedure the speed difference is drastic the while thing deserializes in 1 -2 Seconds My Class object is Declared Serializable. I dont understand why it is serialzer is behaving like this. Am i Missing something. Here is the Sample Code :List<CTempClass> lsTemp = new List<CTempClass>(); for(int i=0; i < 5000000;i++) { CTempClass oTemp = new CTempClass(); oTemp.ID = i; lsTemp.Add(oTemp); oTemp=null; } using (Stream stream = File.Open("c:\\data.bin", FileMode.CreateNew)) { BinaryFormatter bin = new BinaryFormatter(); bin.Serialize(stream, lsTemp ); } //this takes a lot of time using (Stream stream = File.Open("c:\\data.bin", FileMode.Open)) { BinaryFormatter bin = new BinaryFormatter(); var lsTemp1 = bin.Deserialize(stream); }
here is the code of temp class[Serializable] public class CTempClass : ISerializable { // public int nId; private ulong m_uCrc64; public CTempClass () { m_uCrc64 = 0; } //Deserialization constructor. public CTempClass (SerializationInfo info, StreamingContext ctxt) { //Get the values from info and assign them to the appropriate properties m_uCrc64 = info.GetUInt64("m_uCrc64"); } //Serialization function. public void GetObjectData(SerializationInfo info, StreamingContext ctxt) { info.AddValue("m_uCrc64", m_uCrc64); } }
Can any one please tell me why it is taking so much of time. Any suggestion will be a great helpabhinav
-
Hello Guys, Need some help from you guys. I have Generic List having a Class object. I have added around 3000000 items in the list
List <CTempClass> lsTemp = new List <CTempClass> ();
when i tried to Serialize the with binary formater it is working fine but when I deserialize the List it takes a lot of time around 30 Seconds. But when i fill the Generic List with any datatype likeList<ulong> lsTemp = new List<ulong>();
and follow the same procedure the speed difference is drastic the while thing deserializes in 1 -2 Seconds My Class object is Declared Serializable. I dont understand why it is serialzer is behaving like this. Am i Missing something. Here is the Sample Code :List<CTempClass> lsTemp = new List<CTempClass>(); for(int i=0; i < 5000000;i++) { CTempClass oTemp = new CTempClass(); oTemp.ID = i; lsTemp.Add(oTemp); oTemp=null; } using (Stream stream = File.Open("c:\\data.bin", FileMode.CreateNew)) { BinaryFormatter bin = new BinaryFormatter(); bin.Serialize(stream, lsTemp ); } //this takes a lot of time using (Stream stream = File.Open("c:\\data.bin", FileMode.Open)) { BinaryFormatter bin = new BinaryFormatter(); var lsTemp1 = bin.Deserialize(stream); }
here is the code of temp class[Serializable] public class CTempClass : ISerializable { // public int nId; private ulong m_uCrc64; public CTempClass () { m_uCrc64 = 0; } //Deserialization constructor. public CTempClass (SerializationInfo info, StreamingContext ctxt) { //Get the values from info and assign them to the appropriate properties m_uCrc64 = info.GetUInt64("m_uCrc64"); } //Serialization function. public void GetObjectData(SerializationInfo info, StreamingContext ctxt) { info.AddValue("m_uCrc64", m_uCrc64); } }
Can any one please tell me why it is taking so much of time. Any suggestion will be a great helpabhinav
One of the most obvious differences is between value types and references. It takes time to allocate memory for 3 million objects.
I know the language. I've read a book. - _Madmatt
-
One of the most obvious differences is between value types and references. It takes time to allocate memory for 3 million objects.
I know the language. I've read a book. - _Madmatt
-
Hello Guys, Need some help from you guys. I have Generic List having a Class object. I have added around 3000000 items in the list
List <CTempClass> lsTemp = new List <CTempClass> ();
when i tried to Serialize the with binary formater it is working fine but when I deserialize the List it takes a lot of time around 30 Seconds. But when i fill the Generic List with any datatype likeList<ulong> lsTemp = new List<ulong>();
and follow the same procedure the speed difference is drastic the while thing deserializes in 1 -2 Seconds My Class object is Declared Serializable. I dont understand why it is serialzer is behaving like this. Am i Missing something. Here is the Sample Code :List<CTempClass> lsTemp = new List<CTempClass>(); for(int i=0; i < 5000000;i++) { CTempClass oTemp = new CTempClass(); oTemp.ID = i; lsTemp.Add(oTemp); oTemp=null; } using (Stream stream = File.Open("c:\\data.bin", FileMode.CreateNew)) { BinaryFormatter bin = new BinaryFormatter(); bin.Serialize(stream, lsTemp ); } //this takes a lot of time using (Stream stream = File.Open("c:\\data.bin", FileMode.Open)) { BinaryFormatter bin = new BinaryFormatter(); var lsTemp1 = bin.Deserialize(stream); }
here is the code of temp class[Serializable] public class CTempClass : ISerializable { // public int nId; private ulong m_uCrc64; public CTempClass () { m_uCrc64 = 0; } //Deserialization constructor. public CTempClass (SerializationInfo info, StreamingContext ctxt) { //Get the values from info and assign them to the appropriate properties m_uCrc64 = info.GetUInt64("m_uCrc64"); } //Serialization function. public void GetObjectData(SerializationInfo info, StreamingContext ctxt) { info.AddValue("m_uCrc64", m_uCrc64); } }
Can any one please tell me why it is taking so much of time. Any suggestion will be a great helpabhinav
You won’t be able to improve significantly performance while using .Net built in serializers. They are inheritably slow. I bumped into this many times. If performance is an issue I would recommend considering implementing custom destabilization (another word for “read you file and parse data in you code”).
-
You won’t be able to improve significantly performance while using .Net built in serializers. They are inheritably slow. I bumped into this many times. If performance is an issue I would recommend considering implementing custom destabilization (another word for “read you file and parse data in you code”).
-
Hello Ilia, can you please give me any example or some link for cusuom deserialization to start of with it would be a great help
abhinav
Well, what I basically recommend using StreamWriter, StreamReader class to write to file /read from the file data stored in your objects. The approach requires specifying format of the file and how pieces of the data stored / retrieved from the file. When applied to your example is might look something like this <code> using System; using System.Collections.Generic; using System.Text; using System.Runtime.Serialization; using System.IO; using System.Runtime.Serialization.Formatters.Binary; using System.Diagnostics; namespace ConsoleApplication1 { public class CTempClass { public int m_uCrc64; public int M_uCrc64 { get { return m_uCrc64; } set { m_uCrc64 = value; } } public CTempClass() { m_uCrc64 = 0; } } class Program { static void Main(string[] args) { List<CTempClass> lsTemp = new List<CTempClass>(); for (int i = 0; i < 1000000; i++) {