Tada! A new serializer is coming!

Super Lloyd

For (my extremely slowly) upcoming take over the world project I created a new (C#) serializer! Why? Well this take over the world project, without disclosing too much, is a document editor. I need to save those documents. Writing a Save() and Read() method is quite cumbersome... Serializer, XML Serializer and DataContractSerializer don't make the cut for obvious reason. JsonSerializer is too bloated (think a list of 1000 points, endlessly repeating "x =" or "{" or "}") plus it doesn't work very well with property of type object or List Enter my serializer, it's version tolerant, strongly typed, have some kind of header with all type info, then it's a stream of value and I think it's output quite a compact stream of data. And it works with PCL (i.e. desktop, UWP, IOS, Android, Linux with .NET core) And it just finally worked! this morning at 8:35AM! ;P (well, not completely.. I just tweaked a test which then failed.. apparently there is still a known bug...) Well all of that to say... I might make a CodeProject article if there is some interest for it! ;) As a side note, Not really a GitHub project since it's smack in the middle of my utility library and removing it is .. tedious since I shared plenty of reflection extension method....

All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!

Chris Maunder

Super Lloyd wrote:

might make a CodeProject article if there is some interest for it!

Yes please!

cheers Chris Maunder

Super Lloyd

Alright then! :) Plus I got some silent upvote... CP Article I guess it is! ^_^ Hopefully next week! ;)

All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!

Jorgen Andersson

I would read it, so yes please.

Wrong is evil and must be defeated. - Jeff Ello

Super Lloyd

Thanks! :) Will try to reply again when it's done! ;) Or maybe announce it in the lounge as well! ;P

All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!

DaveAuld

Super Lloyd wrote:

I might make a CodeProject article

no not might - you must write an article! :-D

Dave Find Me On:Web|Facebook|Twitter|LinkedIn Folding Stats: Team CodeProject

BillWoodruff

Congratulations on your breakthrough ! I do wonder if you have really explored systematically what the WCF Serialization/Deserialization tools (DataContract, DataMember, etc.) can do. For example, I can serialize/deserialize a WinForm TreeView TreeNodeCollection object by providing the necessary Type declarations and 'KnownType Attributes. I intend to publish this technique soon, since I have not seen it before (which doesn't mean it isn't "out there"). There is also a remarkable serializer presented in an article here on CodeProject, by Christophe Bertrand (2015), that is the only serializer I've seen that can serialize WinForm Controls as is [1] : [^]. [1] "as is:" i.e., not by creating a kind of "shadow class" that serializes only certain Properties, Fields, etc.

«There is a spectrum, from "clearly desirable behaviour," to "possibly dodgy behavior that still makes some sense," to "clearly undesirable behavior." We try to make the latter into warnings or, better, errors. But stuff that is in the middle category you don’t want to restrict unless there is a clear way to work around it.» Eric Lippert, May 14, 2008

Super Lloyd

Alright then! :)

All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!

Super Lloyd

Yes, yes I did! One the problem with KnownTypeAttribute is that it disallow plugin, code that are not know when the base class is written. Alright then, what about DataContractResolver then? Well it helps... but one still need to put DataContract and DataMember attribute on everything that need be serialized otherwise it will be (sometimes) omitted. Further as we used it at work, we kept stumbling on bug where some (new) subclass where not serialized properly for lack of proper DataContract and DataMember annotation. To summarise the second problem with DataContractSerializer, it is far from idiot proof! On that Newtonsoft.Json seems like a better deal. The development constraints it imposes are minimal and, with some settings, it supports circular reference and strongly typed serialization. However it is way too chatty! Finally, using easy to define surrogate classes I can even serialize such things as Stream, Bitmap, etc... (I did define DateTime, TimeSpan, Tuple<1,2...> surrogates) And it also supports IList, IList<T>. IDictionary, IDictionary<K, V> Contrast that with one of my test method:

    public class AAA
    {
        public int ID { get; set; }
        public string Name { get; set; }
        public List Heights { get; set; }
        public List PreviousHeights { get; set; }
        public DayOfWeek Day { get; set; }
        public AAA Child { get; set; }
        public Tuple Tuple { get; set; }
        public List<object> Objects { get; set; } = new List<object>();
    }
    \[Fact\]
    public void SimpleSerialTest()
    {
        // create object to be serialized
        var aaa = new AAA()
        {
            ID = 73,
            Name = "hello",
            Heights = new List { 1, 2, 4, 8, 16 },
            Day = DayOfWeek.Sunday,
            Tuple = Tuple.Create(42, "42"),
        };
        aaa.Child = aaa;
        aaa.Objects.Add(aaa);
        aaa.Objects.Add("hello");

        // write it to a string build (MemoryStream alternative exists)
        var sb = new StringBuilder();
        var pw = new PrimitiveTextWriter(new StringWriter(sb));
        var ow = new ObjectWriter(pw);
        ow.Write(aaa);

        // read it
        var s = sb.ToString();
        var pr = n

Super Lloyd

Thanks for the link, btw! Will see if I can get some idea from it! :D Must try to serialize a WPF app hey! good idea! Also.. My default setting for now is to save public property and fields. I wonder if it's the right default settings.. might find some insight in that other serializer... This serializer seems damn good too! Will have to check mine compare to his... But.. I can aleady see I have more work to do to deserialize a windows form app! :~ :laugh:

All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!

BillWoodruff

I really appreciate your lengthy reply, and your comments on your experience with WCF DataContract. I'd like to join in with the other comments here, and express my hope you will write a CP article on your work ! fyi: the XML output of the WCF DataContract serializer is "bloated" with, imho, bizarrely long internal field/tag-names: that's why I have gotten in the habit of GZiping it going and coming ... typical reduction in file size greater than 70% in my experience. imho, one of the best "acid tests" for a serializer is self-referential types like a TreeNode that contains a pointer to its Parent Node, a list of pointers to its child nodes ... and, in the case of WinForms TreeView/TreeNodes, a pointer to the TreeView Control itself. cheers, Bill

«There is a spectrum, from "clearly desirable behaviour," to "possibly dodgy behavior that still makes some sense," to "clearly undesirable behavior." We try to make the latter into warnings or, better, errors. But stuff that is in the middle category you don’t want to restrict unless there is a clear way to work around it.» Eric Lippert, May 14, 2008

Marc Clifton

Super Lloyd wrote:

since I shared plenty of reflection extension method....

An article in itself. I've been having fun with reflection and LINQ, but I'd really like to see some cool reflection extension methods, as well as that serializer that you describe. Marc

Imperative to Functional Programming Succinctly Contributors Wanted for Higher Order Programming Project! Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny

Super Lloyd

Not something which struck my mind initially... But now that you mention I have handy methods to try to call a constructor or a method... using a list of parameters... and they match using default value as well! Which I haven't seen very often! (using LINQ magic it's just one BIG statement! haha!) Might do a tips / trick for those instead.. less work! :laugh: Other than those astute methods (TryConstruct() TryInvoke() TryConvert()) the rest is various very simple method to plug the hole in the PCL version of Reflection.

All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!

peterchen

Versioning support: Can it read a stream written by an older version, with well-defined and useful behavior for the properties not present in the stream? Can it read a stream written by a newer version that contains properties it does not understand? Another problem I've commonly ran into is class invariants: they must be guaranteed when the class is properly constructed, but during serialization, it is not. Typically, they are enforced through getters/setters - but if a serializer initializes an object through those, it can encounter incorrect intermediate states. If constraints are not checked during read, then you need a separate function validating the object, duplicating the code in the getters / setters. I haven't encountered a enjoyable solution to that problem that did not end with "well, then you have to change how your class is implemented". Or, to snowclone a wonderful quote: The amount of serializers available is a sure sign that none of them really works.

Super Lloyd

In fact I also have another serialization utility. I didn't think of it since it has nothing to do with the serializer. But it let you write something this handy!

doNotCollect = PropertyPath.Watch(aValue, x => x.A.B.C.D, newD => { /* do something */ });

All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!

Super Lloyd

Hey, The intent of this serializer is to replace JsonSerializer (NOT the .NET Serializer) (i.e it's a document serialize, won't work on WPF/WinForm class, unfortunately...) while being way more compact, support IList, IDictionary (normal and generic version), be strongly typed yet type error tolerant. I also dropped human readability format. Text output is just for (painful) debugging purpose. It also DOES NOT support delegates. *It is, indeed, not perfect....* Also it doesn't support TypeConverter / ValueConverter / ISerializable, but I am thinking to add support for them... So to address each point in detail: - by default it only serialize public fields and property. This can be modified with attribute on the class and/or field/property. - what about versioning? It doesn't care if the property is missing while deserializing it is ignored - class invariant... Right now one can enumerate all **reference** object deserialized with the ObjectContext of the deserializer. **I was thinking / maybe** to go one step further and create an IDeserialized interface to automatically "awake / complete" objects after deserialization - also difficult type to serialize can be helped with an "easy" to implement ISurrogate<> class, define this class to replace problematic type when serializing / deserializing

public interface ISurrogate {
void Initialize(T value); // called when serializing
T Instance(); // called when deserializing
}

- thinking of WPF.. I just realized that some readonly property (such as Children UI component) while being readonly, can still be serialized as an IList, should be fixed by publication time! ;) Forget that, it cause too much ambiguity about the serizalizer, since it can also (optionally) serialize private field.

All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!

Kent Bolton

Really nice SL !

Super Lloyd

Haha, thanks! ;P

All in one Menu-Ribbon Bar DirectX for WinRT/C# since 2013! Taking over the world since 1371!

Thornik

After JSON was popularized, making own serializer is just one more try to blush.

_groo_

One question pops out though: why not protobuf.net? Protocol buffers provide extremely space efficient binary files, with versioning support, and .NET port can use reflection to configure the serializer without attributes. It's also among fastest serializers out there.