BinaryFormatter - Serialize OutOfMemory Exception
-
Hello guys, I'm having a problem and, although I understand the problem, I can't find a solution to the problem. I'm using NamedPipes and I have created this method to Serialise on object T to a byte[].
private byte\[\] Serialize(T obj) { using (var memoryStream = new MemoryStream()) { oBinaryFormatter.Serialize(memoryStream, obj); return memoryStream.ToArray(); } }I have a test that sends a object with 300Mb and the Serialization fails with a OutOfmemory.
oBinaryFormatter.Serialize(memoryStream, 300MbObject);
I understand that the problem is related to memory. Can you guys suggest another solution for this problem? Thanks Guys :)
-
Hello guys, I'm having a problem and, although I understand the problem, I can't find a solution to the problem. I'm using NamedPipes and I have created this method to Serialise on object T to a byte[].
private byte\[\] Serialize(T obj) { using (var memoryStream = new MemoryStream()) { oBinaryFormatter.Serialize(memoryStream, obj); return memoryStream.ToArray(); } }I have a test that sends a object with 300Mb and the Serialization fails with a OutOfmemory.
oBinaryFormatter.Serialize(memoryStream, 300MbObject);
I understand that the problem is related to memory. Can you guys suggest another solution for this problem? Thanks Guys :)
Don't use the BinaryFormatter for anything that size. I would recommend either using protobuf.net[^] or manual serializing: BinaryFormatter vs. Manual Serializing[^]. Both will be faster and more compact.
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
-
Don't use the BinaryFormatter for anything that size. I would recommend either using protobuf.net[^] or manual serializing: BinaryFormatter vs. Manual Serializing[^]. Both will be faster and more compact.
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
Since Google Code is shutting down, you may want to direct people to the correct homepage of protobuf.net[^] on Github.
What do you get when you cross a joke with a rhetorical question? The metaphorical solid rear-end expulsions have impacted the metaphorical motorized bladed rotating air movement mechanism. Do questions with multiple question marks annoy you???
-
Hello guys, I'm having a problem and, although I understand the problem, I can't find a solution to the problem. I'm using NamedPipes and I have created this method to Serialise on object T to a byte[].
private byte\[\] Serialize(T obj) { using (var memoryStream = new MemoryStream()) { oBinaryFormatter.Serialize(memoryStream, obj); return memoryStream.ToArray(); } }I have a test that sends a object with 300Mb and the Serialization fails with a OutOfmemory.
oBinaryFormatter.Serialize(memoryStream, 300MbObject);
I understand that the problem is related to memory. Can you guys suggest another solution for this problem? Thanks Guys :)
A typical object is not 300Mb; that already counts as a rather large transfer of data. Do you need to have it completely in memory? If it represents "data", would it not be more logical to stream the data straight to file, as opposed to creating a very large object in memory? The memory manager really does not like large objects. I'd recommend to cut your block into ten chuncks. If you have to stream it to a memorystream, then at least allocate it once and with the correct size - otherwise it may need be reallocated and cause fragmentation.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^][](X-Clacks-Overhead: GNU Terry Pratchett)
-
Since Google Code is shutting down, you may want to direct people to the correct homepage of protobuf.net[^] on Github.
What do you get when you cross a joke with a rhetorical question? The metaphorical solid rear-end expulsions have impacted the metaphorical motorized bladed rotating air movement mechanism. Do questions with multiple question marks annoy you???
Right - forgot about that. Thank you for the heads-up.
If the brain were so simple we could understand it, we would be so simple we couldn't. — Lyall Watson
-
A typical object is not 300Mb; that already counts as a rather large transfer of data. Do you need to have it completely in memory? If it represents "data", would it not be more logical to stream the data straight to file, as opposed to creating a very large object in memory? The memory manager really does not like large objects. I'd recommend to cut your block into ten chuncks. If you have to stream it to a memorystream, then at least allocate it once and with the correct size - otherwise it may need be reallocated and cause fragmentation.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^][](X-Clacks-Overhead: GNU Terry Pratchett)
-
Thx guys for your replys. I went with the solution of cutting the objects into chuncks ;)