Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[Free] MessagePack (binary) and Json (text) serialization

Discussion in 'Assets and Asset Store' started by DenisZykov, Jun 3, 2016.

  1. DenisZykov

    DenisZykov

    Joined:
    Mar 31, 2016
    Posts:
    19
    MessagePack and Json serialization package allow you to save/load your data into/from file or network stream.

    [Source Code]
    [Documentation] [Package In Asset Store]

    Supported Platforms:

    • PC/Mac/Linux
    • IOS
    • Android
    • WebGL

    Serialized types:

    • Primitives: Boolean, Byte, Double, Int16, Int32, Int64, SBytes, Single, UInt16, UInt32, UInt64, String
    • Standart Types: Decimal, DateTimeOffset, DateTime, TimeSpan, Guid, Uri, Version, DictionaryEntry
    Unity3D Types: Bounds, Vector, Matrix4x4, Quaternion, Rect, Color
    • Binary: Stream, byte[]
    • Lists: Array, ArrayList, List[T], HashSet[T] and any other IEnumerable types with Add method
    • Maps: Hashtable, Dictionary[K,V], and other IDictionary types
    • Nullable types
    • Enumeration
    • Objects
     
    yty and rakkarage like this.
  2. IanNeufeld

    IanNeufeld

    Joined:
    Nov 23, 2012
    Posts:
    1
    There's a bug in the current version (1.0.10) related to serializing null values in arrays because it doesn't call WriteFormatting() in the WriteNull() function.
     
  3. DenisZykov

    DenisZykov

    Joined:
    Mar 31, 2016
    Posts:
    19
    I fixed this bug. The new version on GitHub and will soon be in Asset Store.
    Thank you for such a detailed error report.
     
  4. kennel_

    kennel_

    Joined:
    Feb 2, 2016
    Posts:
    2
    Great job!

    May i post here some feedback?
    1) it would be nice to have some Exception in case of trying to deserialize stream which Position != 0. First of all i tried to serialize and deserialize some object just from that MemoryStream and it took me some time to get that stream must be in zero position to deserialization work.
    2) Example in documentation is simple but not actually helpful. RectSerializer code seems more useful.
    3) I can't get how it is designed to serialize MemoryStream properly.
    I have a code like this:
    Code (CSharp):
    1. public MemoryStream gameRoomData;
    2. ......      
    3. writer.WriteMember("g");
    4. writer.WriteValue(src.gameRoomData, typeof(MemoryStream));
    5. writer.WriteObjectEnd();
    Trying to execute it generates Exception:
    The only way i could find to make it work was this hack in JsonWriterExtentions.cs:
    Code (CSharp):
    1.             var actualValueType = value.GetType();
    2. //dirty hack start
    3.             if (valueType == typeof(System.IO.MemoryStream))
    4.             {
    5.                 actualValueType = typeof(System.IO.Stream);
    6.             }
    7. //dirty hack end
    8.             var serializer = writer.Context.GetSerializerForType(actualValueType);
    What do i do wrong?
     
  5. DenisZykov

    DenisZykov

    Joined:
    Mar 31, 2016
    Posts:
    19
    1) Currently it's throwing "Unexpected token 'EndOfStream'" it's quite descriptive.
    2) Yep, description is quite short. It's hard for me writing documentation in english.
    3) I make few changed related to Stream serialization in source code on GitHub, but your code still will not work without changes. It's throws exception because it's don't know how to serialize MemoryStream, only Stream(base class of MemoryStream) and you should:
    a) use WriteValue(memoryStream, typeof(Stream))
    b) use class with Stream property/field
    c) make hint to serializer how to serialize MemoryStream

    Code (CSharp):
    1.     class Program
    2.     {
    3.         public class MyClass
    4.         {
    5.             public MemoryStream mySteam;
    6.         }
    7.         public static void Main()
    8.         {
    9.             // prepare context
    10.             var context = new SerializationContext
    11.             {
    12.                 Serializers = { { typeof(MemoryStream), StreamSerializer.Instance } },
    13.                 // or
    14.                 SerializerFactory = type => (type == typeof(MemoryStream)) ? StreamSerializer.Instance : null,
    15.                 // this will suppress _type member
    16.                 Options = SerializationOptions.SuppressTypeInformation
    17.             };
    18.  
    19.             var myObject = new MyClass
    20.             {
    21.                 mySteam = new MemoryStream(...)
    22.             };
    23.             var tmpStream = new MemoryStream();
    24.             MsgPack.Serialize(myObject, tmpStream, context);
    25.  
    26.             // reading MyClass
    27.             tmpStream.Position = 0;
    28.             myObject = MsgPack.Deserialize<MyClass>(tmpStream, context);
    29.  
    30.             // reading MyClass manually
    31.             tmpStream.Position = 0;
    32.             var reader = new MsgPackReader(tmpStream, context);
    33.  
    34.             reader.ReadObjectBegin();
    35.             if (reader.ReadMember() != "mySteam") throw JsonSerializationException.UnexpectedMemberName(reader.Value.AsString, "mySteam", reader);
    36.             var readedStream = (MemoryStream)reader.ReadValue(typeof(MemoryStream));
    37.             reader.ReadObjectEnd(nextToken: false);
    38.         }
    39.     }
    And you should't write custom TypeSerializers for your classes if they too complex for serialization. It's better to declare "data storage" class and put all data into it. Writing custom TypeSerializer is quite advanced solution.
     
    kennel_ likes this.
  6. kennel_

    kennel_

    Joined:
    Feb 2, 2016
    Posts:
    2
    Thank you for reply!
     
  7. Deleted User

    Deleted User

    Guest

    Is there any way to get serialisation working with generics? Can I attach a TypeSerializer to a generic class?
     
  8. DenisZykov

    DenisZykov

    Joined:
    Mar 31, 2016
    Posts:
    19
    Could you provide some example of class (c# code) which you want to serialize?
     
    ibps13 likes this.
  9. artengineer07

    artengineer07

    Joined:
    Aug 19, 2018
    Posts:
    1
    Hi, i'm new to unity, i used binary formater to do save and load before. then after learned a bit about other method to save and load, i found option like bf (ofcourse), json (the readable ones), and msgpack (the smallest ones). if you don't mind, can you give me example (complete ones) on save() and load() function in unity using persistentdatapath and msgpack please... thanks a lot.
     
  10. DenisZykov

    DenisZykov

    Joined:
    Mar 31, 2016
    Posts:
    19
    Hi!

    You could 'save' and 'load' any custom class with following code:
    Code (CSharp):
    1.         [DataContract]
    2.         public class MySaveClass
    3.         {
    4.             [DataMember]
    5.             public int MyField;
    6.         }
    7.  
    8.         public static void Save(MySaveClass mySaveClass)
    9.         {
    10.             using (var saveFile = System.IO.File.Create(System.IO.Path.Combine(Application.persistentDataPath, "my.save")))
    11.             {
    12.                 MsgPack.Serialize(mySaveClass, saveFile, SerializationOptions.None);
    13.             }
    14.         }
    15.         public static MySaveClass Load()
    16.         {
    17.             using (var saveFile = System.IO.File.OpenRead(System.IO.Path.Combine(Application.persistentDataPath, "my.save")))
    18.             {
    19.                 return MsgPack.Deserialize<MySaveClass>(saveFile, SerializationOptions.None);
    20.             }
    21.         }
    Of course checks for arguments, paths and file existence are required. This is scaffold code.
     
  11. CoderPro

    CoderPro

    Joined:
    Feb 21, 2014
    Posts:
    327
    Hi @DenisZykov
    When import to Unity 2018.3.0f2, its throw below error:
    Code (CSharp):
    1. Assets\Plugins\GameDevWare.Serialization\Metadata\ReflectionUtils.cs(86,34): error CS0246: The type or namespace name 'DynamicMethod' could not be found (are you missing a using directive or an assembly reference?)
    Unity 2018.3.0f2 which default script runtime version is .NET 4.x, and SRV .NET 3.5.x was deprecated, so will be remove in the future. Please fix this ! Thanks in advanced
     
    deednait likes this.
  12. PushoN

    PushoN

    Joined:
    Jul 5, 2012
    Posts:
    16
    @DenisZykov Please provide a updated sample how to use your library with data comes from e.g. WebSocket ?
    Thank you very much.