MessagePack - An efficient binary serialization format
Download as PPTX, PDF0 likes987 views
The document discusses MessagePack, an efficient binary serialization format that is like JSON but faster and more compact. It describes MessagePack's format which uses a single byte to identify the type of data, such as integers, floats, booleans, strings, arrays and maps. It also demonstrates how to serialize and deserialize objects to and from MessagePack format using the MsgPack.Cli library in C#. Finally, it compares the performance of MessagePack to Protocol Buffers by measuring the size and speed of serializing and deserializing different amounts of data.
![MSGPACK.CLI
using MsgPack.Serialization;
...
public static byte[] Serialize<T>(T thisObj) {
var serializer = SerializationContext.Default.GetSerializer<T>();
using (var ms = new MemoryStream()) {
serializer.Pack(ms, thisObj);
return ms.ToArray();
}
}
public static T Deserialize<T>(byte[] bytes) {
var serializer = SerializationContext.Default.GetSerializer<T>();
using (var byteStream = new MemoryStream(bytes)) {
return serializer.Unpack(byteStream);
}
} 18](https://image.slidesharecdn.com/messagepack-anefficientbinaryserializationformat-170608024101/85/MessagePack-An-efficient-binary-serialization-format-18-320.jpg)
![MSGPACK.CLI
public class OldPerson {
[MessagePackMember(0)]
public String Name { get; set; }
[MessagePackMember(1)]
public String NickName { get; set; }
[MessagePackIgnore]
public Object Tag { get; set; }
}
public class NewPerson {
[MessagePackMember(2)]
public String ID { get; set; }
[MessagePackMember(0)]
public String Name { get; set; }
[MessagePackMember(1)]
public String NickName { get; set; }
}
19](https://image.slidesharecdn.com/messagepack-anefficientbinaryserializationformat-170608024101/85/MessagePack-An-efficient-binary-serialization-format-19-320.jpg)
![MESSAGEPACK VS PROTOBUF
var counts = new int[] { 100, 1000, 10000, 100000, 1000000, 10000000, 100000000 };
...
Serialize(ms, larry);
Console.WriteLine("MsgPack Size: {0}", ms.Length);
SerializeToStream(ms, larry);
Console.WriteLine("ProtoBuf Size: {0}", ms.Length);
...
foreach (var count in counts)
{
…
Console.WriteLine("MsgPack Serialize: {0} ms", DoTest(count, () => { Serialize(ms,
larry); }));
Console.WriteLine("ProtoBuf Serialize: {0} ms", DoTest(count, () =>
{ SerializeToStream(ms, larry); }));
Console.WriteLine("MsgPack DeSerialize: {0} ms", DoTest(count, () =>
{ Deserialize<Person>(ms); }));
Console.WriteLine("ProtoBuf DeSerialize: {0} ms", DoTest(count, () =>
{ DeSerializeFromStream<Person>(ms); }));
}
23](https://image.slidesharecdn.com/messagepack-anefficientbinaryserializationformat-170608024101/85/MessagePack-An-efficient-binary-serialization-format-23-320.jpg)
MessagePack - An efficient binary serialization format
- 1. MESSAGEPACK - AN EFFICIENT BINARY SERIALIZATION FORMAT Larry Nung
- 4. INTRODUCTION An efficient binary serialization format Like JSON. but fast and small. 4
- 5. FORMAT 5
- 6. FORMAT 6
- 7. FORMAT 7
- 8. FORMAT format name first byte (in binary) first byte (in hex) positive fixint 0xxxxxxx 0x00 - 0x7f fixmap 1000xxxx 0x80 - 0x8f fixarray 1001xxxx 0x90 - 0x9f fixstr 101xxxxx 0xa0 - 0xbf nil 11000000 0xc0 (never used) 11000001 0xc1 false 11000010 0xc2 true 11000011 0xc3 bin 8 11000100 0xc4 bin 16 11000101 0xc5 bin 32 11000110 0xc6 8
- 9. FORMAT format name first byte (in binary) first byte (in hex) ext 8 11000111 0xc7 ext 16 11001000 0xc8 ext 32 11001001 0xc9 float 32 11001010 0xca float 64 11001011 0xcb uint 8 11001100 0xcc uint 16 11001101 0xcd uint 32 11001110 0xce uint 64 11001111 0xcf int 8 11010000 0xd0 int 16 11010001 0xd1 9
- 10. FORMAT format name first byte (in binary) first byte (in hex) int 32 11010010 0xd2 int 64 11010011 0xd3 fixext 1 11010100 0xd4 fixext 2 11010101 0xd5 fixext 4 11010110 0xd6 fixext 8 11010111 0xd7 fixext 16 11011000 0xd8 str 8 11011001 0xd9 str 16 11011010 0xda str 32 11011011 0xdb array 16 11011100 0xdc 10
- 11. FORMAT format name first byte (in binary) first byte (in hex) array 32 11011101 0xdd map 16 11011110 0xde map 32 11011111 0xdf negative fixint 111xxxxx 0xe0 - 0xff 11
- 12. FORMAT 12 Fixed length types Integer Floating point Boolean Nil
- 13. FORMAT 13 Nil => 11000000 => 0xc0 Boolean False => 11000010 => 0xc2 True => 11000011 => 0xc3 Positive fixint 0 => 00000000 => 0x00 1 => 00000001 => 0x01
- 14. FORMAT 14 Variable length types Raw bytes Array Map
- 15. FORMAT Fixstr Compact => 10100007 01100011 01101111 01101101 01110000 01100001 01100011 01110100 => a7 63 6f 6d 70 61 63 74 15
- 16. MSGPACK.CLI 16
- 18. MSGPACK.CLI using MsgPack.Serialization; ... public static byte[] Serialize<T>(T thisObj) { var serializer = SerializationContext.Default.GetSerializer<T>(); using (var ms = new MemoryStream()) { serializer.Pack(ms, thisObj); return ms.ToArray(); } } public static T Deserialize<T>(byte[] bytes) { var serializer = SerializationContext.Default.GetSerializer<T>(); using (var byteStream = new MemoryStream(bytes)) { return serializer.Unpack(byteStream); } } 18
- 19. MSGPACK.CLI public class OldPerson { [MessagePackMember(0)] public String Name { get; set; } [MessagePackMember(1)] public String NickName { get; set; } [MessagePackIgnore] public Object Tag { get; set; } } public class NewPerson { [MessagePackMember(2)] public String ID { get; set; } [MessagePackMember(0)] public String Name { get; set; } [MessagePackMember(1)] public String NickName { get; set; } } 19
- 20. MSGPACK.CLI … var larry = new OldPerson { Name = "Larry Nung", NickName = "Larry" }; var bytes = Serialize(larry); var person = Deserialize<NewPerson>(bytes); Console.WriteLine("{0} ({1})", person.NickName, person.Name); … 20
- 22. MESSAGEPACK VS PROTOBUF private static void SerializeToStream<T>(Stream stream, T obj) { stream.Seek(0, SeekOrigin.Begin); Serializer.Serialize(stream, obj); } private static T DeSerializeFromStream<T>(Stream stream) { stream.Seek(0, SeekOrigin.Begin); return Serializer.Deserialize<T>(stream); } public static void Serialize<T>(Stream stream, T thisObj) { stream.Seek(0, SeekOrigin.Begin); var serializer = SerializationContext.Default.GetSerializer<T>(); serializer.Pack(stream, thisObj); } public static T Deserialize<T>(Stream stream) { stream.Seek(0, SeekOrigin.Begin); var serializer = SerializationContext.Default.GetSerializer<T>(); return serializer.Unpack(stream); } 22
- 23. MESSAGEPACK VS PROTOBUF var counts = new int[] { 100, 1000, 10000, 100000, 1000000, 10000000, 100000000 }; ... Serialize(ms, larry); Console.WriteLine("MsgPack Size: {0}", ms.Length); SerializeToStream(ms, larry); Console.WriteLine("ProtoBuf Size: {0}", ms.Length); ... foreach (var count in counts) { … Console.WriteLine("MsgPack Serialize: {0} ms", DoTest(count, () => { Serialize(ms, larry); })); Console.WriteLine("ProtoBuf Serialize: {0} ms", DoTest(count, () => { SerializeToStream(ms, larry); })); Console.WriteLine("MsgPack DeSerialize: {0} ms", DoTest(count, () => { Deserialize<Person>(ms); })); Console.WriteLine("ProtoBuf DeSerialize: {0} ms", DoTest(count, () => { DeSerializeFromStream<Person>(ms); })); } 23
- 25. MESSAGEPACK VS PROTOBUF Serialize 25 0 5000 10000 15000 20000 25000 30000 35000 40000 45000 100 1000 10000 100000 1000000 10000000 100000000 MsgPack Protobuf
- 26. MESSAGEPACK VS PROTOBUF Deserialize 26 0 10000 20000 30000 40000 50000 60000 70000 80000 100 1000 10000 100000 1000000 10000000 100000000 MsgPack Protobuf
- 27. REFERENCE 27
- 28. REFERENCE MessagePack: It's like JSON. but fast and small. http://msgpack.org/ 28
- 29. Q&A 29