Packable is an effective data interchange format.

Overview

Packbele

中文文档

1. Overview

Packable is a well designed data interchange format.
It has been impleamented multiple languages , which is effective and easy to use.
It can be used for object serialization/deserialization and message pack up, to storage or RPC.

Packable has the following advantages:

  1. Fast encoding/decoding
  2. Compact coding and small size
  3. Easy to use, flexiblely
  4. The code is light
  5. Support multiple types
  6. Support multiple compression strategies
  7. Supports multiple languages and cross-platforms

Packable currently implements version of Java, C++, C#, Objective-C, Go.

2. Exmaple

Here is usage of Java version, and APIs on other plaform are similar to Java.

Suppose there are two objects like this:

class Data {
    String msg;
    Item[] items;
}

class Item {
    int a;
    long b;
}

2.1 General usage

static class Data implements Packable {
    String msg;
    Item[] items;

    @Override
    public void encode(PackEncoder encoder) {
        encoder.putString(0, msg)
                .putPackableArray(1, items);
    }

    public static final PackCreator<Data> CREATOR = decoder -> {
        Data data = new Data();
        data.msg = decoder.getString(0);
        data.items = decoder.getPackableArray(1, Item.CREATOR);
        return data;
    };
}

static class Item implements Packable {
    int a;
    long b;

    Item(int a, long b) {
        this.a = a;
        this.b = b;
    }

    @Override
    public void encode(PackEncoder encoder) {
        encoder.putInt(0, a);
        encoder.putLong(1, b);
    }

    static final PackArrayCreator<Item> CREATOR = new PackArrayCreator<Item>() {
        @Override
        public Item[] newArray(int size) {
            return new Item[size];
        }

        @Override
        public Item decode(PackDecoder decoder) {
            return new Item(
                    decoder.getInt(0),
                    decoder.getLong(1)
            );
        }
    };
}

static void test() {
    Data data = new Data();
    
    byte[] bytes = PackEncoder.marshal(data);
   
    Data data_2 = PackDecoder.unmarshal(bytes, Data.CREATOR);
}

Serialization

  1. Implements the interface Packable
  2. Implement the encode() method, to encode each field (PackEncoder provides various types of API)
  3. Call the PackEncoder.marshal() method, tranfer the object, and get the byte array.

Deserialization

  1. Create a static object, which is an instance of PackCreator
  2. Implement the decode() method, decode each field, and assign it to the object;
  3. Call PackDecoder.unmarshal(), tranfer the byte array and PackCreator instance,.

If you need to deserialize an array of objects, you need to create an instance of PackArrayCreator (Only Java version need to do this, other platform no need to do this).

2.2 Coded Directly

The example above is only one of the examples. It can be used flexibly.

  1. PackCreator does not have to be created in the class which impleaments Packable. It can also be created anywhere else.
  2. If you only need serialization (sender), you just need to implement Packable.
  3. You can also directly encode / decode the message.
static void test2() {
    String msg = "message";
    int a = 100;
    int b = 200;

    PackEncoder encoder = new PackEncoder();
    encoder.putString(0, msg)
                .putInt(1, a)
                .putInt(2, b);
    byte[] bytes = encoder.getBytes();

    PackDecoder decoder = PackDecoder.newInstance(bytes);
    String dMsg = decoder.getString(0);
    int dA = decoder.getInt(1);
    int dB = decoder.getInt(2);
    decoder.recycle();

    boolean equal = msg.equals(dMsg) && (a == dA) && (b == dB);
    Assert.assertTrue(equal);
}

2.3 Custom Coding

For example, there is a Class like this:

class Info  {
    public long id;
    public String name;
    public Rectangle rect;
}

Rectangle has four fields:

There's a effetive way supplied by packable:

public static class Info implements Packable {
    public long id;
    public String name;
    public Rectangle rect;

    @Override
    public void encode(PackEncoder encoder) {
        encoder.putLong(0, id)
                .putString(1, name);
        EncodeBuffer buf = encoder.putCustom(2, 16);
        buf.writeInt(rect.x);
        buf.writeInt(rect.y);
        buf.writeInt(rect.width);
        buf.writeInt(rect.height);
    }

    public static final PackCreator<Info> CREATOR = decoder -> {
        Info info = new Info();
        info.id = decoder.getLong(0);
        info.name = decoder.getString(1);
        DecodeBuffer buf = decoder.getCustom(2);
        if (buf != null) {
            info.rect = new Rectangle(
                    buf.readInt(),
                    buf.readInt(),
                    buf.readInt(),
                    buf.readInt());
        }
        return info;
    };
}

3. Benchmark

Space of serialization bytes:

Space (byte)
packable 2537191 (57%)
protobuf 2614001 (59%)
gson 4407901 (100%)

Process time:

  1. Macbook Pro
Serialization (ms) Deserialization (ms)
packable 9 8
protobuf 19 11
gson 67 46
  1. Huawei Honor 20S
Serialization (ms) Deserialization (ms)
packable 32 21
protobuf 81 38
gson 190 128

It should be noted that different test data will get different results. Certainty, packable is usually faster than protobuf and gson.

License

See the LICENSE file for license rights and limitations.

You might also like...

Simple yet effective password manager.

Password Manager By Edric Antoine This application provides a convenient way to store usernames and passwords for sites you visit. It will include fun

Jan 5, 2022

The Download Manager uses a simple yet effective GUI interface built with java’s Swing libraries

The Download Manager uses a simple yet effective GUI interface built with java’s Swing libraries.The use of Swing gives the interface a crisp, modern look and feel. The GUI maintains a list of downloads that are currently being managed.

Jan 2, 2022

Efficient yet Effective plugin to catch X-rayers.

Efficient yet Effective plugin to catch X-rayers.

XCatch Efficient yet effective plugin to catch X-rayers for Minecraft 1.13-1.18. XCatch analizes how a player is mining by looking it how many turns i

Nov 10, 2022

The code examples of the "Effective Software Testing: A Developer's Guide" book

Effective software testing This repository contains the code examples of the Software Testing: A Developer's Guide book, by Maurício Aniche. Each fold

Dec 29, 2022

A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

Apache Gobblin Apache Gobblin is a highly scalable data management solution for structured and byte-oriented data in heterogeneous data ecosystems. Ca

Jan 4, 2023

Convert Journeymap image data into Xaero format for Xaero's minecraft map mod

Convert Journeymap image data into Xaero format for Xaero's minecraft map mod

JMtoXaero JMtoXaero is a tool to convert JourneyMap tiles to regions used by Xaero's World Map Description Reads images from the Journeymap folder Wri

Dec 21, 2022

A scientific charting library focused on performance optimised real-time data visualisation at 25 Hz update rates for data sets with a few 10 thousand up to 5 million data points.

A scientific charting library focused on performance optimised real-time data visualisation at 25 Hz update rates for data sets with a few 10 thousand up to 5 million data points.

ChartFx ChartFx is a scientific charting library developed at GSI for FAIR with focus on performance optimised real-time data visualisation at 25 Hz u

Jan 2, 2023

A scientific charting library focused on performance optimised real-time data visualisation at 25 Hz update rates for data sets with a few 10 thousand up to 5 million data points.

A scientific charting library focused on performance optimised real-time data visualisation at 25 Hz update rates for data sets with a few 10 thousand up to 5 million data points.

ChartFx ChartFx is a scientific charting library developed at GSI for FAIR with focus on performance optimised real-time data visualisation at 25 Hz u

Dec 30, 2022

Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems.

Firehose - Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems.

Dec 22, 2022

Jansi is a small java library that allows you to use ANSI escape sequences to format your console output which works even on windows.

Description Jansi is a small java library that allows you to use ANSI escape codes to format your console output which works even on Windows. It also

Dec 28, 2022

A Java library for serializing objects as PHP serialization format.

Java PHP Serializer Latest release: A Java library for serializing objects as PHP serialization format. The library fully implements the PHP serializa

Jun 13, 2022

binary serialization format

Colfer Colfer is a binary serialization format optimized for speed and size. The project's compiler colf(1) generates source code from schema definiti

Dec 25, 2022

Turn -XX:+TraceBytecodes output into a FlameGraph compatible stack format

Turn -XX:+TraceBytecodes output into a FlameGraph compatible stack format

A tool to turn the output of -XX:+TraceBytecodes (a JDK debug-only feature to print every bytecode executed by the interpreter) into a simple stack fo

Dec 11, 2022

Get Method Sampling from Java Flight Recorder Dump and convert to FlameGraph compatible format.

Note: Travis has removed the support for Oracle JDK 8. Therefore the build status is removed temporarily. Converting JFR Method Profiling Samples to F

Dec 16, 2022

Maven plugin to help creating CHANGELOG by keeping one format and solving merge request conflicts problem by extraction of new CHANGELOG entries to seperate files.

keep-changelog-maven-plugin CHANGELOG.md is one of the most important files in a repository. It allows others to find out about the most important cha

Aug 28, 2022

Highly experimental and unstable alternative world format for Minecraft

Radon for Fabric An alternative world storage format that tries to be more performant and durable than vanilla's own. This is hardly more than a day's

Dec 9, 2022

HMFF - A Hierarchical Mapping File Format

HMFF - A Hierarchical Mapping File Format This library provides a recursive key-value interpretation for your configuration files, with comment suppor

Oct 23, 2021

Json Entity Models for Minecraft (Not MCPatcher Format)

JsonEM (Json Entity Models) Library for modders, resource pack makers, and modpack makers to create and edit entity models WITH JSON Does not work wit

Jul 23, 2022
Owner
Billy Wei
The journey of a thousand miles begins with a single step.
Billy Wei
A compiler built in Java that allows to translate xml format to json.

A compiler built in Java that allows to translate xml format to json. Getting Started Welcome to the VS Code Java world. Here is a guideline to help y

Víctor Andrés Rojas 1 Jan 6, 2022
Uber-project for (some) standard Jackson textual format backends: csv, properties, yaml (xml to be added in future)

Overview This is a multi-module umbrella project for Jackson standard text-format dataformat backends. Dataformat backends are used to support format

FasterXML, LLC 351 Dec 22, 2022
Generate Java types from JSON or JSON Schema and annotates those types for data-binding with Jackson, Gson, etc

jsonschema2pojo jsonschema2pojo generates Java types from JSON Schema (or example JSON) and can annotate those types for data-binding with Jackson 2.x

Joe Littlejohn 5.9k Jan 5, 2023
null 5 Jan 11, 2022
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. https://developers.google.com/protocol-buffers/ Overview Protocol Buffe

Protocol Buffers 57.6k Jan 1, 2023
Split into data blocks,In this format, efficient reading can be realized,Avoid unnecessary data reading operations.

dataTear 切换至:中文文档 knowledge base dataTear Split into data fragments for data management. In this format, efficient reading can be achieved to avoid un

LingYuZhao 24 Dec 15, 2022
Log sourcing is method of trying to map all the ERROR and WARN logs you have in your system in a cost effective way.

log-sourcing Log sourcing is method of trying to map all the ERROR and WARN logs you have in your system in a cost effective way. The basic idea is th

Shimon Magal 12 Apr 19, 2021
'Effective Java 3/E' 스터디 저장소입니다.

Effective Java 스터디 Effective Java 3/E 스터디 저장소입니다. 0. 스터디 정보 기간 : 2021/08/19 ~ 참여 인원 : 강동민(@riyenas0925), 김하늬(@kimhanui) 도서 : 이펙티브 자바 Effective Java 3/

null 3 Feb 26, 2022