LeapSerial

The Leap Motion cross-format, cross-platform declarative serialization library

Description

LeapSerial is a cross-format, declarative, serialization and deserialization library written and maintained by Leap Motion. This library is built with CMake, and makes heavy use of C++11.

LeapSerial is mainly intended to provide users with a way to describe how their types should be serialized without being too concerned about the wire format serialization should take. Users should be able to directly annotate and serialize/deserialize their business objects rather than having to convert to and from DTOs generated by tools like protoc.

Code Quality Rank: L4

Programming language: C++

License: Apache License 2.0

Tags: Serialization     Portable     Meta Programming    

Latest version: v0.5.2

README

Introduction to LeapSerial

LeapSerial is a cross-format, declarative, serialization and deserialization library written and maintained by Leap Motion. This library is built with CMake, and makes heavy use of C++11.

LeapSerial is mainly intended to provide users with a way to describe how their types should be serialized without being too concerned about the wire format serialization should take. Users should be able to directly annotate and serialize/deserialize their business objects rather than having to convert to and from DTOs generated by tools like protoc.

The following four output formats are currently supported:

• JSON
• Protobuf
• Flatbuffers
• LeapSerial Proprietary

LeapSerial also provides a simple stream concept, some wrapper classes, and a few standard implementations. These streams may be composed with one another similarly to how it's done with Protobuf streams.

• Wrapper for std::istream and std::ostream
• AES-256 encryption
• Zlib compression
• BZip2 compression
• Memory stream

Users may also write their own.

Quick Start

Here's how you mark the fields to be serialized:

#include <LeapSerial/LeapSerial.h>

class MyClass {
  int my_member;

  static leap::descriptor GetDescriptor() {
    return {
      &MyClass::my_member
    };
  }
};

Serialization one-liner:

MyClass myClass;
std::stringstream os;
leap::Serialize(os, myClass);

Deserialization is also a one-liner:

std::shared_ptr<MyClass> a = leap::Deserialize<MyClass>(ss, myClass);

If your type doesn't use native pointers (either directly or transitively), you can also deserialize in-place.

MyClass b;
leap::Deserialize(ss, b);

Alternative Archivers

LeapSerial has a few output formats that are well supported. The leap::Serialize call, by default, will use the internal LeapSerial archiver, which formats data in a custom bitstream format. You can use other formats, though, such as Protobuf, but this requires that your fields are numbered or named. The following sections all use the following numbered and named data structure:

class MyProtobufObject {
public:
  int a = 949;
  std::string b = "Hello world!";
  std::vector<int> c {4, 5, 6};

  static leap::descriptor GetDescriptor(void) {
    return{
      "MyProtobufObject",
      {
        { 424, "a", &MyProtobufObject::a },
        { 425, "b", &MyProtobufObject::b },
        { 426, "c", &MyProtobufObject::c }
      }
    };
  }
};

Protobuf

Protobuf serialization can be done with OArchiveProtobuf:

#include <LeapSerial/IArchiveProtobuf.h>
#include <LeapSerial/OArchiveProtobuf.h>

void Foo() {
  MyProtobufObject myobj;
  std::stringstream ss;

  leap::Serialize<leap::OArchiveProtobuf>(ss, defaultPerson);
}

The resulting object can be parsed by Protobuf, if you have a schema for MyProtobufObject. You can also create the corresponding proto file by serializing the descriptor, like this:

std::stringstream ss;
leap::Serialize<leap::protobuf_v1>(
  ss,
  MyProtobufObject::GetDescriptor()
);

This is what you get:

message MyProtobufObject {
  required sint32 a = 424;
  optional string b = 425;
  repeated sint32 c = 426;
}

Right now, leap::protobuf_v1 and leap::protobuf_v2 are supported to ensure the generated proto file can be parsed by your chosen version of protoc.

JSON

JSON serialization is also supported. The protov3 specification's JSON mapping is used wherever possible. Currently, deserialization is not supported, but there is a leap::IArchiveJSON type which will provide deserialization once it's implemented.

#include <LeapSerial/ArchiveJSON.h>

void Foo() {
  MyProtobufObject obj;
  std::stringstream ss;
  leap::Serialize<leap::OArchiveJSON>(ss, obj);
}

Here's the resulting JSON:

{
  "a": 949,
  "b": "Hello world!",
  "c": [ 4, 5, 6 ]
}

Alternative Streams

You can also serialize to a memory buffer, if you don't like std::stringstream:

leap::MemoryStream ms;
MyClass b;
leap::Serialize(ms, b);

// In case you want to do something with the data
std::vector<uint8_t>& data = ms.GetDatadata();

// Or you can just round trip right from here
std::shared_ptr<MyClass> c = leap::Deserialize(ms);

Maybe you have a buffer already that you want to deserialize

char buf[1024];
FillMyBuffer(buf, 1024);

leap::BufferedStream bs{buf, sizeof(buf), sizeof(buf)};
std::shared_ptr<MyClass> mc = leap::Deserialize<MyClass>(bs);

Encryption and compression are supported, too. Encrypting and compressing streams are declared as a filter. Protobuf's zero copy streams are similarly implemented. Here's how you might encrypt a file directly to disk with an std::ofstream. Make sure you fill key with a cryptographic function or you might be vulnerable to a related key attack.

std::array<uint8_t, 32> myKey;

std::ofstream of("myfile.dat");
leap::OutputStreamAdapter osa(of);
leap::CompressionStream<Zlib> cs { osa };
leap::AESEncryptionStream ecs { cs, myKey };
MyObject obj;
leap::Serialize(ecs, obj);

If you use encryption and compression, make sure the compression step happens before the encryption step, otherwise you will wind up making the file size larger.