Tada! A new serializer is coming!
-
For (my extremely slowly) upcoming take over the world project I created a new (C#) serializer! Why? Well this take over the world project, without disclosing too much, is a document editor. I need to save those documents. Writing a Save() and Read() method is quite cumbersome... Serializer, XML Serializer and DataContractSerializer don't make the cut for obvious reason. JsonSerializer is too bloated (think a list of 1000 points, endlessly repeating "x =" or "{" or "}") plus it doesn't work very well with property of type object or List Enter my serializer, it's version tolerant, strongly typed, have some kind of header with all type info, then it's a stream of value and I think it's output quite a compact stream of data. And it works with PCL (i.e. desktop, UWP, IOS, Android, Linux with .NET core) And it just finally worked! this morning at 8:35AM! ;P (well, not completely.. I just tweaked a test which then failed.. apparently there is still a known bug...) Well all of that to say... I might make a CodeProject article if there is some interest for it! ;) As a side note, Not really a GitHub project since it's smack in the middle of my utility library and removing it is .. tedious since I shared plenty of reflection extension method....
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
One question pops out though: why not protobuf.net? Protocol buffers provide extremely space efficient binary files, with versioning support, and .NET port can use reflection to configure the serializer without attributes. It's also among fastest serializers out there.
-
For (my extremely slowly) upcoming take over the world project I created a new (C#) serializer! Why? Well this take over the world project, without disclosing too much, is a document editor. I need to save those documents. Writing a Save() and Read() method is quite cumbersome... Serializer, XML Serializer and DataContractSerializer don't make the cut for obvious reason. JsonSerializer is too bloated (think a list of 1000 points, endlessly repeating "x =" or "{" or "}") plus it doesn't work very well with property of type object or List Enter my serializer, it's version tolerant, strongly typed, have some kind of header with all type info, then it's a stream of value and I think it's output quite a compact stream of data. And it works with PCL (i.e. desktop, UWP, IOS, Android, Linux with .NET core) And it just finally worked! this morning at 8:35AM! ;P (well, not completely.. I just tweaked a test which then failed.. apparently there is still a known bug...) Well all of that to say... I might make a CodeProject article if there is some interest for it! ;) As a side note, Not really a GitHub project since it's smack in the middle of my utility library and removing it is .. tedious since I shared plenty of reflection extension method....
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
-
Yes, yes I did! One the problem with
KnownTypeAttributeis that it disallow plugin, code that are not know when the base class is written. Alright then, what aboutDataContractResolverthen? Well it helps... but one still need to putDataContractandDataMemberattribute on everything that need be serialized otherwise it will be (sometimes) omitted. Further as we used it at work, we kept stumbling on bug where some (new) subclass where not serialized properly for lack of proper DataContract and DataMember annotation. To summarise the second problem withDataContractSerializer, it is far from idiot proof! On thatNewtonsoft.Jsonseems like a better deal. The development constraints it imposes are minimal and, with some settings, it supports circular reference and strongly typed serialization. However it is way too chatty! Finally, using easy to define surrogate classes I can even serialize such things as Stream, Bitmap, etc... (I did define DateTime, TimeSpan, Tuple<1,2...> surrogates) And it also supports IList, IList<T>. IDictionary, IDictionary<K, V> Contrast that with one of my test method:public class AAA { public int ID { get; set; } public string Name { get; set; } public List Heights { get; set; } public List PreviousHeights { get; set; } public DayOfWeek Day { get; set; } public AAA Child { get; set; } public Tuple Tuple { get; set; } public List<object> Objects { get; set; } = new List<object>(); } \[Fact\] public void SimpleSerialTest() { // create object to be serialized var aaa = new AAA() { ID = 73, Name = "hello", Heights = new List { 1, 2, 4, 8, 16 }, Day = DayOfWeek.Sunday, Tuple = Tuple.Create(42, "42"), }; aaa.Child = aaa; aaa.Objects.Add(aaa); aaa.Objects.Add("hello"); // write it to a string build (MemoryStream alternative exists) var sb = new StringBuilder(); var pw = new PrimitiveTextWriter(new StringWriter(sb)); var ow = new ObjectWriter(pw); ow.Write(aaa); // read it var s = sb.ToString(); var pr = nYour serializer looks quite easy to use. The output is human readable, but does not look to be human 'intuitive'. To get the data even more compact, maybe nice to have an option to compress (GZipStream?) the output since it does not look like anyone would ever 'hand edit' the serialized output, correct?? What am I missing? Look forward to your CP article! Curtis
-
Nah... While I be happy to be popular with it, I won't deny it! :cool: Its primary aim is to provide a much needed help with super secret project (where JSON just won't do)...
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
-
One question pops out though: why not protobuf.net? Protocol buffers provide extremely space efficient binary files, with versioning support, and .NET port can use reflection to configure the serializer without attributes. It's also among fastest serializers out there.
Well... ignorance perhaps? I am only proficient with the .NET Serializer, DataContractSerializer, XmlSerializer and JsonSerializer. None of them satisfied me... So a few question about protobuf: 1. can it save IList, IList<T>, IDIctionary, IDictionary<K, V> ? 2. can it provide a simple mechanism (like surrogate) to serialize opaque type like Bitmap? 3. can it cope with a class which have different property altogether after serialization? 3.a. how does it do (3)? My serializer do 3 by saving class meta data once so that it knows what to read... does it do the same or does it save like JSON: ('property name' 'property value'), one my main contention with JSON... my serializer save type metada once, then only ('property value' 'property value' 'property value'......)
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
-
Haha thanks... I had a quick look at Transit.. but it's not C#! All my super secret take over the world project are in C#...
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
-
Your serializer looks quite easy to use. The output is human readable, but does not look to be human 'intuitive'. To get the data even more compact, maybe nice to have an option to compress (GZipStream?) the output since it does not look like anyone would ever 'hand edit' the serialized output, correct?? What am I missing? Look forward to your CP article! Curtis
The Serializer write to an interface I created call
IPrimitiveWriterI provide 2 implementation of IPrimitiveWriterpublic class PrimitiveBinaryWriter : IPrimitiveWriter
{
public PrimitiveBinaryWriter(Stream stream)
}
public class PrimitiveTextWriter : IPrimitiveWriter
{
public PrimitiveTextWriter(TextWriter writer)
}So... binary output is covered. And it's also forward only (unlike first version, haha), so the stream can be a GZipStream. The text output is there mostly for fun. It also provide a tiny help for debugging and help me check there is no repetition (all reference, including string should be there only once)
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
-
The Serializer write to an interface I created call
IPrimitiveWriterI provide 2 implementation of IPrimitiveWriterpublic class PrimitiveBinaryWriter : IPrimitiveWriter
{
public PrimitiveBinaryWriter(Stream stream)
}
public class PrimitiveTextWriter : IPrimitiveWriter
{
public PrimitiveTextWriter(TextWriter writer)
}So... binary output is covered. And it's also forward only (unlike first version, haha), so the stream can be a GZipStream. The text output is there mostly for fun. It also provide a tiny help for debugging and help me check there is no repetition (all reference, including string should be there only once)
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
Great! Looking forward to trying it after your CP Article.....
-
Hey, The intent of this serializer is to replace
JsonSerializer(NOT the .NET Serializer) (i.e it's a document serialize, won't work on WPF/WinForm class, unfortunately...) while being way more compact, supportIList, IDictionary(normal and generic version), be strongly typed yet type error tolerant. I also dropped human readability format. Text output is just for (painful) debugging purpose. It also DOES NOT support delegates. *It is, indeed, not perfect....* Also it doesn't supportTypeConverter / ValueConverter / ISerializable, but I am thinking to add support for them... So to address each point in detail: - by default it only serialize public fields and property. This can be modified with attribute on the class and/or field/property. - what about versioning? It doesn't care if the property is missing while deserializing it is ignored - class invariant... Right now one can enumerate all **reference** object deserialized with the ObjectContext of the deserializer. **I was thinking / maybe** to go one step further and create anIDeserializedinterface to automatically "awake / complete" objects after deserialization - also difficult type to serialize can be helped with an "easy" to implementISurrogate<>class, define this class to replace problematic type when serializing / deserializingpublic interface ISurrogate {
void Initialize(T value); // called when serializing
T Instance(); // called when deserializing
}- thinking of WPF.. I just realized that some readonly property (such as
ChildrenUI component) while being readonly, can still be serialized as anIList, should be fixed by publication time! ;) Forget that, it cause too much ambiguity about the serizalizer, since it can also (optionally) serialize private field.All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
Soudns good to me :) `IDeserialized` would allow to cope with many common problems I've encountered. `ISurrogate` could nicely isolate some complexity from the class / serializer itself. Your striked-out part reminds me of another thing: read only Collections (they are managed internally) where the elements are mutable and should be serialized. This could be handled in IDeserialized if I could construct thecollection there and then request serialization for the collection members.
-
Soudns good to me :) `IDeserialized` would allow to cope with many common problems I've encountered. `ISurrogate` could nicely isolate some complexity from the class / serializer itself. Your striked-out part reminds me of another thing: read only Collections (they are managed internally) where the elements are mutable and should be serialized. This could be handled in IDeserialized if I could construct thecollection there and then request serialization for the collection members.
Great to know IDeserialized works out for you! :) I needed such concept for my IoC+IServiceProvider to completely initialize object tree (it's called IRegistryDelegate for that). Came with that there first! :-D After all the comments I decided to add yet more functionalities and now it's all broken.... Plus, exceptionally, I work this Saturday! :(( (this my home project, not the work one) So it's gonna be delayed some more... But the current (broken) version does support ISerializable (but not [SerializableAttribute] which assume private field reading), TypeConverter (but not ValueSerializer which are not PLC / .NET Core) and it also support restoring public readonly reference property, such as:
public List Children { get; } = new List();
public Configuration Configuration { get { return existingConfig; } }I was misleaded, proved easy and helpful! :) ETA should be end of next week now...
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
-
Great to know IDeserialized works out for you! :) I needed such concept for my IoC+IServiceProvider to completely initialize object tree (it's called IRegistryDelegate for that). Came with that there first! :-D After all the comments I decided to add yet more functionalities and now it's all broken.... Plus, exceptionally, I work this Saturday! :(( (this my home project, not the work one) So it's gonna be delayed some more... But the current (broken) version does support ISerializable (but not [SerializableAttribute] which assume private field reading), TypeConverter (but not ValueSerializer which are not PLC / .NET Core) and it also support restoring public readonly reference property, such as:
public List Children { get; } = new List();
public Configuration Configuration { get { return existingConfig; } }I was misleaded, proved easy and helpful! :) ETA should be end of next week now...
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
-
Super Lloyd wrote:
So it's gonna be delayed some more...
No worries, I'll be on vacation for a week. :rolleyes: (I expect it on my desk when I return, 9am point ;P )
9am sharp! alright! :laugh:
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
-
For (my extremely slowly) upcoming take over the world project I created a new (C#) serializer! Why? Well this take over the world project, without disclosing too much, is a document editor. I need to save those documents. Writing a Save() and Read() method is quite cumbersome... Serializer, XML Serializer and DataContractSerializer don't make the cut for obvious reason. JsonSerializer is too bloated (think a list of 1000 points, endlessly repeating "x =" or "{" or "}") plus it doesn't work very well with property of type object or List Enter my serializer, it's version tolerant, strongly typed, have some kind of header with all type info, then it's a stream of value and I think it's output quite a compact stream of data. And it works with PCL (i.e. desktop, UWP, IOS, Android, Linux with .NET core) And it just finally worked! this morning at 8:35AM! ;P (well, not completely.. I just tweaked a test which then failed.. apparently there is still a known bug...) Well all of that to say... I might make a CodeProject article if there is some interest for it! ;) As a side note, Not really a GitHub project since it's smack in the middle of my utility library and removing it is .. tedious since I shared plenty of reflection extension method....
All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!
Just curious, why not something like the Avro serializer (Serializing data with the Microsoft .NET Library for Avro[^]) or Protocol Buffers (https://developers.google.com/protocol-buffers/docs/csharptutorial[^])? Avro supports schemas and versioning, and both produce a fairly compact output and are cross platform compatible.
-
Just curious, why not something like the Avro serializer (Serializing data with the Microsoft .NET Library for Avro[^]) or Protocol Buffers (https://developers.google.com/protocol-buffers/docs/csharptutorial[^])? Avro supports schemas and versioning, and both produce a fairly compact output and are cross platform compatible.
I was about to replay (once again, like 2 thread above above protocol buffer): Ignorance. But then, following your link (for Avro serializer at least) and going down the memory lane on protocol buffer with a CP Article, the answer is obvious: I want something as simple as NewtonSoft JsonConverter. It should just work! And also it should work with
objector subclass. And while you can use some attribute with it, it should work fine without any! In a word I want to serialize that (without any more hidden work!):class Data {
public object Something;
public List MoreThings { get; } = new List();
}All in one Menu-Ribbon Bar
DirectX for WinRT/C# since 2013!
Taking over the world since 1371! -
Super Lloyd wrote:
might make a CodeProject article if there is some interest for it!
Yes please!
cheers Chris Maunder
No article just yet... But code is on GitHub! GitHub - superlloyd/Galador.Reflection[^]
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
-
I would read it, so yes please.
Wrong is evil and must be defeated. - Jeff Ello
No article just yet... But code is on GitHub! GitHub - superlloyd/Galador.Reflection[^]
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
-
Super Lloyd wrote:
since I shared plenty of reflection extension method....
An article in itself. I've been having fun with reflection and LINQ, but I'd really like to see some cool reflection extension methods, as well as that serializer that you describe. Marc
Imperative to Functional Programming Succinctly Contributors Wanted for Higher Order Programming Project! Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny
No article just yet... But code is on GitHub! And I have 2 other reflection goodie inside! ;) GitHub - superlloyd/Galador.Reflection[^]
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
-
Super Lloyd wrote:
So it's gonna be delayed some more...
No worries, I'll be on vacation for a week. :rolleyes: (I expect it on my desk when I return, 9am point ;P )
phew, just before your vacation ends! GitHub - superlloyd/Galador.Reflection[^] That's just the GitHub version... CP article still coming....
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
-
phew, just before your vacation ends! GitHub - superlloyd/Galador.Reflection[^] That's just the GitHub version... CP article still coming....
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
-
I'm impressed! The introduction on GitHub already looks good - haven't dug deeper yet, though.
Thanks! :) Just committed some performance improvements! CodeProject article is next... might take a while...
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!