Java binary serialization and cloning: fast, efficient, automatic

Related tags

Serialization kryo
Overview

KryoNet

Build Status Maven Central Join the chat at https://gitter.im/EsotericSoftware/kryo

Kryo is a fast and efficient binary object graph serialization framework for Java. The goals of the project are high speed, low size, and an easy to use API. The project is useful any time objects need to be persisted, whether to a file, database, or over the network.

Kryo can also perform automatic deep and shallow copying/cloning. This is direct copying from object to object, not object to bytes to object.

Contact / Mailing list

Please use the Kryo mailing list for questions, discussions, and support. Please limit use of the Kryo issue tracker to bugs and enhancements, not questions, discussions, or support.

Table of contents

Recent releases

  • 5.0.4 - brings an important fix for Pool
  • 5.0.3 - brings an important fix for generics optimization.
  • 5.0.2 - brings an important fix for CompatibleFieldSerializer.
  • 5.0.1 - brings several incremental fixes and improvements.
  • 5.0.0 - the final Kryo 5 release fixing many issues and making many long awaited improvements over Kryo 4. Note: For libraries (not applications) using Kryo, there's now a completely self-contained, versioned artifact (for details see installation). For migration from Kryo 4.x see also Migration to v5.
  • 4.0.2 - brings several incremental fixes and improvements.

Installation

Kryo publishes two kinds of artifacts/jars:

  • the default jar (with the usual library dependencies) which is meant for direct usage in applications (not libraries)
  • a dependency-free, "versioned" jar which should be used by other libraries. Different libraries shall be able to use different major versions of Kryo.

Kryo JARs are available on the releases page and at Maven Central. The latest snapshots of Kryo, including snapshot builds of master, are in the Sonatype Repository.

With Maven

To use the latest Kryo release in your application, use this dependency entry in your pom.xml:

<dependency>
   <groupId>com.esotericsoftware</groupId>
   <artifactId>kryo</artifactId>
   <version>5.0.4</version>
</dependency>

To use the latest Kryo release in a library you want to publish, use this dependency entry in your pom.xml:

<dependency>
   <groupId>com.esotericsoftware.kryo</groupId>
   <artifactId>kryo5</artifactId>
   <version>5.0.4</version>
</dependency>

To use the latest Kryo snapshot, use:

<repository>
   <id>sonatype-snapshots</id>
   <name>sonatype snapshots repo</name>
   <url>https://oss.sonatype.org/content/repositories/snapshots</url>
</repository>

<!-- for usage in an application: -->
<dependency>
   <groupId>com.esotericsoftware</groupId>
   <artifactId>kryo</artifactId>
   <version>5.0.5-SNAPSHOT</version>
</dependency>
<!-- for usage in a library that should be published: -->
<dependency>
   <groupId>com.esotericsoftware.kryo</groupId>
   <artifactId>kryo5</artifactId>
   <version>5.0.5-SNAPSHOT</version>
</dependency>

Without Maven

Not everyone is a Maven fan. Using Kryo without Maven requires placing the Kryo JAR on your classpath along with the dependency JARs found in lib.

On Android

Kryo 5 ships with Objenesis 3.1 which currently supports Android API >= 26. If you want to use Kryo with older Android APIs, you need to explicitely depend on Objensis 2.6.

implementation ('com.esotericsoftware:kryo:5.0.4') {
  exclude group: "org.objenesis"
}
implementation 'org.objenesis:objenesis:2.6'

Quickstart

Jumping ahead to show how the library can be used:

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import java.io.*;

public class HelloKryo {
   static public void main (String[] args) throws Exception {
      Kryo kryo = new Kryo();
      kryo.register(SomeClass.class);

      SomeClass object = new SomeClass();
      object.value = "Hello Kryo!";

      Output output = new Output(new FileOutputStream("file.bin"));
      kryo.writeObject(output, object);
      output.close();

      Input input = new Input(new FileInputStream("file.bin"));
      SomeClass object2 = kryo.readObject(input, SomeClass.class);
      input.close();   
   }
   static public class SomeClass {
      String value;
   }
}

The Kryo class performs the serialization automatically. The Output and Input classes handle buffering bytes and optionally flushing to a stream.

The rest of this document details how this works and advanced usage of the library.

IO

Getting data in and out of Kryo is done using the Input and Output classes. These classes are not thread safe.

Output

The Output class is an OutputStream that writes data to a byte array buffer. This buffer can be obtained and used directly, if a byte array is desired. If the Output is given an OutputStream, it will flush the bytes to the stream when the buffer becomes full, otherwise Output can grow its buffer automatically. Output has many methods for efficiently writing primitives and strings to bytes. It provides functionality similar to DataOutputStream, BufferedOutputStream, FilterOutputStream, and ByteArrayOutputStream, all in one class.

Tip: Output and Input provide all the functionality of ByteArrayOutputStream. There is seldom a reason to have Output flush to a ByteArrayOutputStream.

Output buffers the bytes when writing to an OutputStream, so flush or close must be called after writing is complete to cause the buffered bytes to be written to the OutputStream. If the Output has not been provided an OutputStream, calling flush or close is unnecessary. Unlike many streams, an Output instance can be reused by setting the position, or setting a new byte array or stream.

Tip: Since Output buffers already, there is no reason to have Output flush to a BufferedOutputStream.

The zero argument Output constructor creates an uninitialized Output. Output setBuffer must be called before the Output can be used.

Input

The Input class is an InputStream that reads data from a byte array buffer. This buffer can be set directly, if reading from a byte array is desired. If the Input is given an InputStream, it will fill the buffer from the stream when all the data in the buffer has been read. Input has many methods for efficiently reading primitives and strings from bytes. It provides functionality similar to DataInputStream, BufferedInputStream, FilterInputStream, and ByteArrayInputStream, all in one class.

Tip: Input provides all the functionality of ByteArrayInputStream. There is seldom a reason to have Input read from a ByteArrayInputStream.

If the Input close is called, the Input's InputStream is closed, if any. If not reading from an InputStream then it is not necessary to call close. Unlike many streams, an Input instance can be reused by setting the position and limit, or setting a new byte array or InputStream.

The zero argument Input constructor creates an uninitialized Input. Input setBuffer must be called before the Input can be used.

ByteBuffers

The ByteBufferOutput and ByteBufferInput classes work exactly like Output and Input, except they use a ByteBuffer rather than a byte array.

Unsafe buffers

The UnsafeOutput, UnsafeInput, UnsafeByteBufferOutput, and UnsafeByteBufferInput classes work exactly like their non-unsafe counterparts, except they use sun.misc.Unsafe for higher performance in many cases. To use these classes Util.unsafe must be true.

The downside to using unsafe buffers is that the native endianness and representation of numeric types of the system performing the serialization affects the serialized data. For example, deserialization will fail if the data is written on X86 and read on SPARC. Also, if data is written with an unsafe buffer, it must be read with an unsafe buffer.

The biggest performance difference with unsafe buffers is with large primitive arrays when variable length encoding is not used. Variable length encoding can be disabled for the unsafe buffers or only for specific fields (when using FieldSerializer).

Variable length encoding

The IO classes provide methods to read and write variable length int (varint) and long (varlong) values. This is done by using the 8th bit of each byte to indicate if more bytes follow, which means a varint uses 1-5 bytes and a varlong uses 1-9 bytes. Using variable length encoding is more expensive but makes the serialized data much smaller.

When writing a variable length value, the value can be optimized either for positive values or for both negative and positive values. For example, when optimized for positive values, 0 to 127 is written in one byte, 128 to 16383 in two bytes, etc. However, small negative numbers are the worst case at 5 bytes. When not optimized for positive, these ranges are shifted down by half. For example, -64 to 63 is written in one byte, 64 to 8191 and -65 to -8192 in two bytes, etc.

Input and Output buffers provides methods to read and write fixed sized or variable length values. There are also methods to allow the buffer to decide whether a fixed size or variable length value is written. This allows serialization code to ensure variable length encoding is used for very common values that would bloat the output if a fixed size were used, while still allowing the buffer configuration to decide for all other values.

Method Description
writeInt(int) Writes a 4 byte int.
writeVarInt(int, boolean) Writes a 1-5 byte int.
writeInt(int, boolean) Writes either a 4 or 1-5 byte int (the buffer decides).
writeLong(long) Writes an 8 byte long.
writeVarLong(long, boolean) Writes an 1-9 byte long.
writeLong(long, boolean) Writes either an 8 or 1-9 byte long (the buffer decides).

To disable variable length encoding for all values, the writeVarInt, writeVarLong, readVarInt, and readVarLong methods would need to be overridden.

Chunked encoding

It can be useful to write the length of some data, then the data. When the length of the data is not known ahead of time, all the data needs to be buffered to determine its length, then the length can be written, then the data. using a single, large buffer for this would prevent streaming and may require an unreasonably large buffer, which is not ideal.

Chunked encoding solves this problem by using a small buffer. When the buffer is full, its length is written, then the data. This is one chunk of data. The buffer is cleared and this continues until there is no more data to write. A chunk with a length of zero denotes the end of the chunks.

Kryo provides classes to maked chunked encoding. OutputChunked is used to write chunked data. It extends Output, so has all the convenient methods to write data. When the OutputChunked buffer is full, it flushes the chunk to another OutputStream. The endChunk method is used to mark the end of a set of chunks.

OutputStream outputStream = new FileOutputStream("file.bin");
OutputChunked output = new OutputChunked(outputStream, 1024);
// Write data to output...
output.endChunk();
// Write more data to output...
output.endChunk();
// Write even more data to output...
output.endChunk();
output.close();

To read the chunked data, InputChunked is used. It extends Input, so has all the convenient methods to read data. When reading, InputChunked will appear to hit the end of the data when it reaches the end of a set of chunks. The nextChunks method advances to the next set of chunks, even if not all the data has been read from the current set of chunks.

InputStream outputStream = new FileInputStream("file.bin");
InputChunked input = new InputChunked(inputStream, 1024);
// Read data from first set of chunks...
input.nextChunks();
// Read data from second set of chunks...
input.nextChunks();
// Read data from third set of chunks...
input.close();

Buffer performance

Generally Output and Input provide good performance. Unsafe buffers perform as well or better, especially for primitive arrays, if their crossplatform incompatibilities are acceptable. ByteBufferOutput and ByteBufferInput provide slightly worse performance, but this may be acceptable if the final destination of the bytes must be a ByteBuffer.

Variable length encoding is slower than fixed values, especially when there is a lot of data using it.

Chunked encoding uses an intermediary buffer so it adds one additional copy of all the bytes. This alone may be acceptable, however when used in a reentrant serializer, the serializer must create an OutputChunked or InputChunked for each object. Allocating and garbage collecting those buffers during serialization can have a negative impact on performance.

Reading and writing objects

Kryo has three sets of methods for reading and writing objects. If the concrete class of the object is not known and the object could be null:

kryo.writeClassAndObject(output, object);

Object object = kryo.readClassAndObject(input);
if (object instanceof SomeClass) {
   // ...
}

If the class is known and the object could be null:

kryo.writeObjectOrNull(output, object);

SomeClass object = kryo.readObjectOrNull(input, SomeClass.class);

If the class is known and the object cannot be null:

kryo.writeObject(output, object);

SomeClass object = kryo.readObject(input, SomeClass.class);

All of these methods first find the appropriate serializer to use, then use that to serialize or deserialize the object. Serializers can call these methods for recursive serialization. Multiple references to the same object and circular references are handled by Kryo automatically.

Besides methods to read and write objects, the Kryo class provides a way to register serializers, reads and writes class identifiers efficiently, handles null objects for serializers that can't accept nulls, and handles reading and writing object references (if enabled). This allows serializers to focus on their serialization tasks.

Round trip

While testing and exploring Kryo APIs, it can be useful to write an object to bytes, then read those bytes back to an object.

Kryo kryo = new Kryo();

// Register all classes to be serialized.
kryo.register(SomeClass.class);

SomeClass object1 = new SomeClass();

Output output = new Output(1024, -1);
kryo.writeObject(output, object1);

Input input = new Input(output.getBuffer(), 0, output.position());
SomeClass object2 = kryo.readObject(input, SomeClass.class);

In this example the Output starts with a buffer that has a capacity of 1024 bytes. If more bytes are written to the Output, the buffer will grow in size without limit. The Output does not need to be closed because it has not been given an OutputStream. The Input reads directly from the Output's byte[] buffer.

Deep and shallow copies

Kryo supports making deep and shallow copies of objects using direct assignment from one object to another. This is more efficient than serializing to bytes and back to objects.

Kryo kryo = new Kryo();
SomeClass object = ...
SomeClass copy1 = kryo.copy(object);
SomeClass copy2 = kryo.copyShallow(object);

All the serializers being used need to support copying. All serializers provided with Kryo support copying.

Like with serialization, when copying, multiple references to the same object and circular references are handled by Kryo automatically if references are enabled.

If using Kryo only for copying, registration can be safely disabled.

Kryo getOriginalToCopyMap can be used after an object graph is copied to obtain a map of old to new objects. The map is cleared automatically by Kryo reset, so is only useful when Kryo setAutoReset is false.

References

By default references are not enabled. This means if an object appears in an object graph multiple times, it will be written multiple times and will be deserialized as multiple, different objects. When references are disabled, circular references will cause serialization to fail. References are enabled or disabled with Kryo setReferences for serialization and setCopyReferences for copying.

When references are enabled, a varint is written before each object the first time it appears in the object graph. For subsequent appearances of that class within the same object graph, only a varint is written. After deserialization the object references are restored, including any circular references. The serializers in use must support references by calling Kryo reference in Serializer read.

Enabling references impacts performance because every object that is read or written needs to be tracked.

ReferenceResolver

Under the covers, a ReferenceResolver handles tracking objects that have been read or written and provides int reference IDs. Multiple implementations are provided:

  1. MapReferenceResolver is used by default if a reference resolver is not specified. It uses Kryo's IdentityObjectIntMap (a cuckoo hashmap) to track written objects. This kind of map has very fast gets and does not allocate for put, but puts for very large numbers of objects can be somewhat slow.
  2. HashMapReferenceResolver uses a HashMap to track written objects. This kind of map allocates for put but may provide better performance for object graphs with a very high number of objects.
  3. ListReferenceResolver uses an ArrayList to track written objects. For object graphs with relatively few objects, this can be faster than using a map (~15% faster in some tests). This should not be used for graphs with many objects because it has a linear look up to find objects that have already been written.

ReferenceResolver useReferences(Class) can be overridden. It returns a boolean to decide if references are supported for a class. If a class doesn't support references, the varint reference ID is not written before objects of that type. If a class does not need references and objects of that type appear in the object graph many times, the serialized size can be greatly reduced by disabling references for that class. The default reference resolver returns false for all primitive wrappers and enums. It is common to also return false for String and other classes, depending on the object graphs being serialized.

public boolean useReferences (Class type) {
   return !Util.isWrapperClass(type) && !Util.isEnum(type) && type != String.class;
}

Reference limits

The reference resolver determines the maximum number of references in a single object graph. Java array indices are limited to Integer.MAX_VALUE, so reference resolvers that use data structures based on arrays may result in a java.lang.NegativeArraySizeException when serializing more than ~2 billion objects. Kryo uses int class IDs, so the maximum number of references in a single object graph is limited to the full range of positive and negative numbers in an int (~4 billion).

Context

Kryo getContext returns a map for storing user data. The Kryo instance is available to all serializers, so this data is easily accessible to all serializers.

Kryo getGraphContext is similar, but is cleared after each object graph is serialized or deserialized. This makes it easy to manage state that is only relevant for the current object graph. For example, this can be used to write some schema data the first time a class is encountered in an object graph. See CompatibleFieldSerializer for an example.

Reset

By default, Kryo reset is called after each entire object graph is serialized. This resets unregistered class names in the class resolver, references to previously serialized or deserialized objects in the reference resolver, and clears the graph context. Kryo setAutoReset(false) can be used to disable calling reset automatically, allowing that state to span multiple object graphs.

Serializer framework

Kryo is a framework to facilitate serialization. The framework itself doesn't enforce a schema or care what or how data is written or read. Serializers are pluggable and make the decisions about what to read and write. Many serializers are provided out of the box to read and write data in various ways. While the provided serializers can read and write most objects, they can easily be replaced partially or completely with your own serializers.

Registration

When Kryo goes to write an instance of an object, first it may need to write something that identifies the object's class. By default, all classes that Kryo will read or write must be registered beforehand. Registration provides an int class ID, the serializer to use for the class, and the object instantiator used to create instances of the class.

Kryo kryo = new Kryo();
kryo.register(SomeClass.class);
Output output = ...
SomeClass object = ...
kryo.writeObject(output, object);

During deserialization, the registered classes must have the exact same IDs they had during serialization. When registered, a class is assigned the next available, lowest integer ID, which means the order classes are registered is important. The class ID can optionally be specified explicitly to make order unimportant:

Kryo kryo = new Kryo();
kryo.register(SomeClass.class, 9);
kryo.register(AnotherClass.class, 10);
kryo.register(YetAnotherClass.class, 11);

Class IDs -1 and -2 are reserved. Class IDs 0-8 are used by default for primitive types and String, though these IDs can be repurposed. The IDs are written as positive optimized varints, so are most efficient when they are small, positive integers. Negative IDs are not serialized efficiently.

ClassResolver

Under the covers, a ClassResolver handles actually reading and writing bytes to represent a class. The default implementation is sufficient in most cases, but it can be replaced to customize what happens when a class is registered, what an unregistered class is encountered during serialization, and what is read and written to represent a class.

Optional registration

Kryo can be configured to allow serialization without registering classes up front.

Kryo kryo = new Kryo();
kryo.setRegistrationRequired(false);
Output output = ...
SomeClass object = ...
kryo.writeObject(output, object);

Use of registered and unregistered classes can be mixed. Unregistered classes have two major drawbacks:

  1. There are security implications because it allows deserialization to create instances of any class. Classes with side effects during construction or finalization could be used for malicious purposes.
  2. Instead of writing a varint class ID (often 1-2 bytes), the fully qualified class name is written the first time an unregistered class appears in the object graph. Subsequent appearances of that class within the same object graph are written using a varint. Short package names could be considered to reduce the serialized size.

If using Kryo only for copying, registration can be safely disabled.

When registration is not required, Kryo setWarnUnregisteredClasses can be enabled to log a message when an unregistered class is encountered. This can be used to easily obtain a list of all unregistered classes. Kryo unregisteredClassMessage can be overridden to customize the log message or take other actions.

Default serializers

When a class is registered, a serializer instance can optionally be specified. During deserialization, the registered classes must have the exact same serializers and serializer configurations they had during serialization.

Kryo kryo = new Kryo();
kryo.register(SomeClass.class, new SomeSerializer());
kryo.register(AnotherClass.class, new AnotherSerializer());

If a serializer is not specified or when an unregistered class is encountered, a serializer is chosen automatically from a list of "default serializers" that maps a class to a serializer. Having many default serializers doesn't affect serialization performance, so by default Kryo has 50+ default serializers for various JRE classes. Additional default serializers can be added:

Kryo kryo = new Kryo();
kryo.setRegistrationRequired(false);
kryo.addDefaultSerializer(SomeClass.class, SomeSerializer.class);

Output output = ...
SomeClass object = ...
kryo.writeObject(output, object);

This will cause a SomeSerializer instance to be created when SomeClass or any class which extends or implements SomeClass is registered.

Default serializers are sorted so more specific classes are matched first, but are otherwise matched in the order they are added. The order they are added can be relevant for interfaces.

If no default serializers match a class, then the global default serializer is used. The global default serializer is set to FieldSerializer by default, but can be changed. Usually the global serializer is one that can handle many different types.

Kryo kryo = new Kryo();
kryo.setDefaultSerializer(TaggedFieldSerializer.class);
kryo.register(SomeClass.class);

With this code, assuming no default serializers match SomeClass, TaggedFieldSerializer will be used.

A class can also use the DefaultSerializer annotation, which will be used instead of choosing one of Kryo's default serializers:

@DefaultSerializer(SomeClassSerializer.class)
public class SomeClass {
   // ...
}

For maximum flexibility, Kryo getDefaultSerializer can be overridden to implement custom logic for choosing and instantiating a serializer.

Serializer factories

The addDefaultSerializer(Class, Class) method does not allow for configuration of the serializer. A serializer factory can be set instead of a serializer class, allowing the factory to create and configure each serializer instance. Factories are provided for common serializers, often with a getConfig method to configure the serializers that are created.

Kryo kryo = new Kryo();
 
TaggedFieldSerializerFactory defaultFactory = new TaggedFieldSerializerFactory();
defaultFactory.getConfig().setReadUnknownTagData(true);
kryo.setDefaultSerializer(defaultFactory);

FieldSerializerFactory someClassFactory = new FieldSerializerFactory();
someClassFactory.getConfig().setFieldsCanBeNull(false);
kryo.register(SomeClass.class, someClassFactory);

The serializer factory has an isSupported(Class) method which allows it to decline to handle a class, even if it otherwise matches the class. This allows a factory to check for multiple interfaces or implement other logic.

Object creation

While some serializers are for a specific class, others can serialize many different classes. Serializers can use Kryo newInstance(Class) to create an instance of any class. This is done by looking up the registration for the class, then using the registration's ObjectInstantiator. The instantiator can be specified on the registration.

Registration registration = kryo.register(SomeClass.class);
registration.setInstantiator(new ObjectInstantiator<SomeClass>() {
   public SomeClass newInstance () {
      return new SomeClass("some constructor arguments", 1234);
   }
});

If the registration doesn't have an instantiator, one is provided by Kryo newInstantiator. To customize how objects are created, Kryo newInstantiator can be overridden or an InstantiatorStrategy provided.

InstantiatorStrategy

Kryo provides DefaultInstantiatorStrategy which creates objects using ReflectASM to call a zero argument constructor. If that is not possible, it uses reflection to call a zero argument constructor. If that also fails, then it either throws an exception or tries a fallback InstantiatorStrategy. Reflection uses setAccessible, so a private zero argument constructor can be a good way to allow Kryo to create instances of a class without affecting the public API.

DefaultInstantiatorStrategy is the recommended way of creating objects with Kryo. It runs constructors just like would be done with Java code. Alternative, extralinguistic mechanisms can also be used to create objects. The Objenesis StdInstantiatorStrategy uses JVM specific APIs to create an instance of a class without calling any constructor at all. Using this is dangerous because most classes expect their constructors to be called. Creating the object by bypassing its constructors may leave the object in an uninitialized or invalid state. Classes must be designed to be created in this way.

Kryo can be configured to try DefaultInstantiatorStrategy first, then fallback to StdInstantiatorStrategy if necessary.

kryo.setInstantiatorStrategy(new DefaultInstantiatorStrategy(new StdInstantiatorStrategy()));

Another option is SerializingInstantiatorStrategy, which uses Java's built-in serialization mechanism to create an instance. Using this, the class must implement java.io.Serializable and the first zero argument constructor in a super class is invoked. This also bypasses constructors and so is dangerous for the same reasons as StdInstantiatorStrategy.

kryo.setInstantiatorStrategy(new DefaultInstantiatorStrategy(new SerializingInstantiatorStrategy()));

Overriding create

Alternatively, some generic serializers provide methods that can be overridden to customize object creation for a specific type, instead of calling Kryo newInstance.

kryo.register(SomeClass.class, new FieldSerializer(kryo, SomeClass.class) {
   protected T create (Kryo kryo, Input input, Class<? extends T> type) {
      return new SomeClass("some constructor arguments", 1234);
   }
});

Some serializers provide a writeHeader method that can be overridden to write data that is needed in create at the right time.

static public class TreeMapSerializer extends MapSerializer<TreeMap> {
   protected void writeHeader (Kryo kryo, Output output, TreeMap map) {
      kryo.writeClassAndObject(output, map.comparator());
   }

   protected TreeMap create (Kryo kryo, Input input, Class<? extends TreeMap> type, int size) {
      return new TreeMap((Comparator)kryo.readClassAndObject(input));
   }
}

If a serializer doesn't provide writeHeader, writing data for create can be done in write.

static public class SomeClassSerializer extends FieldSerializer<SomeClass> {
   public SomeClassSerializer (Kryo kryo) {
      super(kryo, SomeClass.class);
   }
   public void write (Kryo kryo, Output output, SomeClass object) {
      output.writeInt(object.value);
   }
   protected SomeClass create (Kryo kryo, Input input, Class<? extends SomeClass> type) {
      return new SomeClass(input.readInt());
   }
}

Final classes

Even when a serializer knows the expected class for a value (eg a field's class), if the value's concrete class is not final then the serializer needs to first write the class ID, then the value. Final classes can be serialized more efficiently because they are non-polymorphic.

Kryo isFinal is used to determine if a class is final. This method can be overridden to return true even for types which are not final. For example, if an application uses ArrayList extensively but never uses an ArrayList subclass, treating ArrayList as final could allow FieldSerializer to save 1-2 bytes per ArrayList field.

Closures

Kryo can serialize Java 8+ closures that implement java.io.Serializable, with some caveats. Closures serialized on one JVM may fail to be deserialized on a different JVM.

Kryo isClosure is used to determine if a class is a closure. If so, then ClosureSerializer.Closure is used to find the class registration instead of the closure's class. To serialize closures, the following classes must be registered: ClosureSerializer.Closure, SerializedLambda, Object[], and Class. Additionally, the closure's capturing class must be registered.

kryo.register(Object[].class);
kryo.register(Class.class);
kryo.register(SerializedLambda.class);
kryo.register(ClosureSerializer.Closure.class, new ClosureSerializer());
kryo.register(CapturingClass.class);

Callable<Integer> closure1 = (Callable<Integer> & java.io.Serializable)( () -> 72363 );

Output output = new Output(1024, -1);
kryo.writeObject(output, closure1);

Input input = new Input(output.getBuffer(), 0, output.position());
Callable<Integer> closure2 = (Callable<Integer>)kryo.readObject(input, ClosureSerializer.Closure.class);

Serializing closures which do not implement Serializable is possible with some effort.

Compression and encryption

Kryo supports streams, so it is trivial to use compression or encryption on all of the serialized bytes:

OutputStream outputStream = new DeflaterOutputStream(new FileOutputStream("file.bin"));
Output output = new Output(outputStream);
Kryo kryo = new Kryo();
kryo.writeObject(output, object);
output.close();

If needed, a serializer can be used to compress or encrypt the bytes for only a subset of the bytes for an object graph. For example, see DeflateSerializer or BlowfishSerializer. These serializers wrap another serializer to encode and decode the bytes.

Implementing a serializer

The Serializer abstract class defines methods to go from objects to bytes and bytes to objects.

public class ColorSerializer extends Serializer<Color> {
   public void write (Kryo kryo, Output output, Color color) {
      output.writeInt(color.getRGB());
   }

   public Color read (Kryo kryo, Input input, Class<? extends Color> type) {
      return new Color(input.readInt());
   }
}

Serializer has only two methods that must be implemented. write writes the object as bytes to the Output. read creates a new instance of the object and reads from the Input to populate it.

Serializer references

When Kryo is used to read a nested object in Serializer read then Kryo reference must first be called with the parent object if it is possible for the nested object to reference the parent object. It is unnecessary to call Kryo reference if the nested objects can't possibly reference the parent object, if Kryo is not being used for nested objects, or if references are not being used. If nested objects can use the same serializer, the serializer must be reentrant.

public SomeClass read (Kryo kryo, Input input, Class<? extends SomeClass> type) {
   SomeClass object = new SomeClass();
   kryo.reference(object);
   // Read objects that may reference the SomeClass instance.
   object.someField = kryo.readClassAndObject(input);
   return object;
}

Nested serializers

Serializers should not usually make direct use of other serializers, instead the Kryo read and write methods should be used. This allows Kryo to orchestrate serialization and handle features such as references and null objects. Sometimes a serializer knows which serializer to use for a nested object. In that case, it should use Kryo's read and write methods which accept a serializer.

If the object could be null:

Serializer serializer = ...
kryo.writeObjectOrNull(output, object, serializer);

SomeClass object = kryo.readObjectOrNull(input, SomeClass.class, serializer);

If the object cannot be null:

Serializer serializer = ...
kryo.writeObject(output, object, serializer);

SomeClass object = kryo.readObject(input, SomeClass.class, serializer);

During serialization Kryo getDepth provides the current depth of the object graph.

KryoException

When a serialization fails, a KryoException can be thrown with serialization trace information about where in the object graph the exception occurred. When using nested serializers, KryoException can be caught to add serialization trace information.

Object object = ...
Field[] fields = ...
for (Field field : fields) {
   try {
      // Use other serializers to serialize each field.
   } catch (KryoException ex) {
      ex.addTrace(field.getName() + " (" + object.getClass().getName() + ")");
      throw ex;
   } catch (Throwable t) {
      KryoException ex = new KryoException(t);
      ex.addTrace(field.getName() + " (" + object.getClass().getName() + ")");
      throw ex;
   }
}

Stack size

The serializers Kryo provides use the call stack when serializing nested objects. Kryo minimizes stack calls, but a stack overflow can occur for extremely deep object graphs. This is a common issue for most serialization libraries, including the built-in Java serialization. The stack size can be increased using -Xss, but note that this applies to all threads. Large stack sizes in a JVM with many threads may use a large amount of memory.

Kryo setMaxDepth can be used to limit the maximum depth of an object graph. This can prevent malicious data from causing a stack overflow.

Accepting null

By default, serializers will never receive a null, instead Kryo will write a byte as needed to denote null or not null. If a serializer can be more efficient by handling nulls itself, it can call Serializer setAcceptsNull(true). This can also be used to avoid writing the null denoting byte when it is known that all instances the serializer will handle will never be null.

Generics

Kryo getGenerics provides generic type information so serializers can be more efficient. This is most commonly used to avoid writing the class when the type parameter class is final.

Generic type inference is enabled by default and can be disabled with Kryo setOptimizedGenerics(false). Disabling generics optimization can increase performance at the cost of a larger serialized size.

If the class has a single type parameter, nextGenericClass returns the type parameter class, or null if none. After reading or writing any nested objects, popGenericType must be called. See CollectionSerializer for an example.

public class SomeClass<T> {
   public T value;
}
public class SomeClassSerializer extends Serializer<SomeClass> {
   public void write (Kryo kryo, Output output, SomeClass object) {
      Class valueClass = kryo.getGenerics().nextGenericClass();

      if (valueClass != null && kryo.isFinal(valueClass)) {
         Serializer serializer = kryo.getSerializer(valueClass);
         kryo.writeObjectOrNull(output, object.value, serializer);
      } else
         kryo.writeClassAndObject(output, object.value);

      kryo.getGenerics().popGenericType();
   }

   public SomeClass read (Kryo kryo, Input input, Class<? extends SomeClass> type) {
      Class valueClass = kryo.getGenerics().nextGenericClass();

      SomeClass object = new SomeClass();
      kryo.reference(object);

      if (valueClass != null && kryo.isFinal(valueClass)) {
         Serializer serializer = kryo.getSerializer(valueClass);
         object.value = kryo.readObjectOrNull(input, valueClass, serializer);
      } else
         object.value = kryo.readClassAndObject(input);

      kryo.getGenerics().popGenericType();
      return object;
   }
}

For a class with multiple type parameters, nextGenericTypes returns an array of GenericType instances and resolve is used to obtain the class for each GenericType. After reading or writing any nested objects, popGenericType must be called. See MapSerializer for an example.

public class SomeClass<K, V> {
   public K key;
   public V value;
}
public class SomeClassSerializer extends Serializer<SomeClass> {
   public void write (Kryo kryo, Output output, SomeClass object) {
      Class keyClass = null, valueClass = null;
      GenericType[] genericTypes = kryo.getGenerics().nextGenericTypes();
      if (genericTypes != null) {
         keyClass = genericTypes[0].resolve(kryo.getGenerics());
         valueClass = genericTypes[1].resolve(kryo.getGenerics());
      }

      if (keyClass != null && kryo.isFinal(keyClass)) {
         Serializer serializer = kryo.getSerializer(keyClass);
         kryo.writeObjectOrNull(output, object.key, serializer);
      } else
         kryo.writeClassAndObject(output, object.key);

      if (valueClass != null && kryo.isFinal(valueClass)) {
         Serializer serializer = kryo.getSerializer(valueClass);
         kryo.writeObjectOrNull(output, object.value, serializer);
      } else
         kryo.writeClassAndObject(output, object.value);

      kryo.getGenerics().popGenericType();
   }

   public SomeClass read (Kryo kryo, Input input, Class<? extends SomeClass> type) {
      Class keyClass = null, valueClass = null;
      GenericType[] genericTypes = kryo.getGenerics().nextGenericTypes();
      if (genericTypes != null) {
         keyClass = genericTypes[0].resolve(kryo.getGenerics());
         valueClass = genericTypes[1].resolve(kryo.getGenerics());
      }

      SomeClass object = new SomeClass();
      kryo.reference(object);

      if (keyClass != null && kryo.isFinal(keyClass)) {
         Serializer serializer = kryo.getSerializer(keyClass);
         object.key = kryo.readObjectOrNull(input, keyClass, serializer);
      } else
         object.key = kryo.readClassAndObject(input);

      if (valueClass != null && kryo.isFinal(valueClass)) {
         Serializer serializer = kryo.getSerializer(valueClass);
         object.value = kryo.readObjectOrNull(input, valueClass, serializer);
      } else
         object.value = kryo.readClassAndObject(input);

      kryo.getGenerics().popGenericType();
      return object;
   }
}

For serializers which pass type parameter information for nested objects in the object graph (somewhat advanced usage), first GenericsHierarchy is used to store the type parameters for a class. During serialization, Generics pushTypeVariables is called before generic types are resolved (if any). If >0 is returned, this must be followed by Generics popTypeVariables. See FieldSerializer for an example.

public class SomeClass<T> {
   T value;
   List<T> list;
}
public class SomeClassSerializer extends Serializer<SomeClass> {
   private final GenericsHierarchy genericsHierarchy;

   public SomeClassSerializer () {
      genericsHierarchy = new GenericsHierarchy(SomeClass.class);
   }

   public void write (Kryo kryo, Output output, SomeClass object) {
      Class valueClass = null;
      Generics generics = kryo.getGenerics();
      int pop = 0;
      GenericType[] genericTypes = generics.nextGenericTypes();
      if (genericTypes != null) {
         pop = generics.pushTypeVariables(genericsHierarchy, genericTypes);
         valueClass = genericTypes[0].resolve(generics);
      }

      if (valueClass != null && kryo.isFinal(valueClass)) {
         Serializer serializer = kryo.getSerializer(valueClass);
         kryo.writeObjectOrNull(output, object.value, serializer);
      } else
         kryo.writeClassAndObject(output, object.value);

      kryo.writeClassAndObject(output, object.list);

      if (pop > 0) generics.popTypeVariables(pop);
      generics.popGenericType();
   }

   public SomeClass read (Kryo kryo, Input input, Class<? extends SomeClass> type) {
      Class valueClass = null;
      Generics generics = kryo.getGenerics();
      int pop = 0;
      GenericType[] genericTypes = generics.nextGenericTypes();
      if (genericTypes != null) {
         pop = generics.pushTypeVariables(genericsHierarchy, genericTypes);
         valueClass = genericTypes[0].resolve(generics);
      }

      SomeClass object = new SomeClass();
      kryo.reference(object);

      if (valueClass != null && kryo.isFinal(valueClass)) {
         Serializer serializer = kryo.getSerializer(valueClass);
         object.value = kryo.readObjectOrNull(input, valueClass, serializer);
      } else
         object.value = kryo.readClassAndObject(input);

      object.list = (List)kryo.readClassAndObject(input);

      if (pop > 0) generics.popTypeVariables(pop);
      generics.popGenericType();
      return object;
   }
}

KryoSerializable

Instead of using a serializer, a class can choose to do its own serialization by implementing KryoSerializable (similar to java.io.Externalizable).

public class SomeClass implements KryoSerializable {
   private int value;
   public void write (Kryo kryo, Output output) {
      output.writeInt(value, false);
   }
   public void read (Kryo kryo, Input input) {
      value = input.readInt(false);
   }
}

Obviously the instance must already be created before read can be called, so the class isn't able to control its own creation. A KryoSerializable class will use the default serializer KryoSerializableSerializer, which uses Kryo newInstance to create a new instance. It is trivial to write your own serializer to customize the process, call methods before or after serialiation, etc.

Serializer copying

Serializers only support copying if copy is overridden. Similar to Serializer read, this method contains the logic to create and configure the copy. Just like read, Kryo reference must be called before Kryo is used to copy child objects, if any of the child objects could reference the parent object.

class SomeClassSerializer extends Serializer<SomeClass> {
   public SomeClass copy (Kryo kryo, SomeClass original) {
      SomeClass copy = new SomeClass();
      kryo.reference(copy);
      copy.intValue = original.intValue;
      copy.object = kryo.copy(original.object);
      return copy;
   }
}

KryoCopyable

Instead of using a serializer, classes can implement KryoCopyable to do their own copying:

public class SomeClass implements KryoCopyable<SomeClass> {
   public SomeClass copy (Kryo kryo) {
      SomeClass copy = new SomeClass();
      kryo.reference(copy);
      copy.intValue = intValue;
      copy.object = kryo.copy(object);
      return copy;
   }
}

Immutable serializers

Serializer setImmutable(true) can be used when the type is immutable. In that case, Serializer copy does not need to be implemented -- the default copy implementation will return the original object.

Kryo versioning and upgrading

The following rules of thumb are applied to Kryo's version numbering:

  1. The major version is increased if serialization compatibility is broken. This means data serialized with a previous version may not be deserialized with the new version.
  2. The minor version is increased if binary or source compatibility of the documented public API is broken. To avoid increasing the version when very few users are affected, some minor breakage is allowed if it occurs in public classes that are seldom used or not intended for general usage.

Upgrading any dependency is a significant event, but a serialization library is more prone to breakage than most dependencies. When upgrading Kryo check the version differences and test the new version thoroughly in your own applications. We try to make it as safe and easy as possible.

  • At development time serialization compatibility is tested for the different binary formats and default serializers.
  • At development time binary and source compatibility is tracked with clirr.
  • For each release a changelog is provided that also contains a section reporting the serialization, binary, and source compatibilities.
  • For reporting binary and source compatibility japi-compliance-checker is used.

Interoperability

The Kryo serializers provided by default assume that Java will be used for deserialization, so they do not explicitly define the format that is written. Serializers could be written using a standardized format that is more easily read by other languages, but this is not provided by default.

Compatibility

For some needs, such as long term storage of serialized bytes, it can be important how serialization handles changes to classes. This is known as forward compatibility (reading bytes serialized by newer classes) and backward compatibility (reading bytes serialized by older classes). Kryo provides a few generic serializers which take different approaches to handling compatibility. Additional serializers can easily be developed for forward and backward compatibility, such as a serializer that uses an external, hand written schema.

Serializers

Kryo provides many serializers with various configuration options and levels of compatibility. Additional serializers can be found in the kryo-serializers sister project, which hosts serializers that access private APIs or are otherwise not perfectly safe on all JVMs. More serializers can be found in the links section.

FieldSerializer

FieldSerializer works by serializing each non-transient field. It can serialize POJOs and many other classes without any configuration. All non-public fields are written and read by default, so it is important to evaluate each class that will be serialized. If fields are public, serialization may be faster.

FieldSerializer is efficient by writing only the field data, without any schema information, using the Java class files as the schema. It does not support adding, removing, or changing the type of fields without invalidating previously serialized bytes. Renaming fields is allowed only if it doesn't change the alphabetical order of the fields.

FieldSerializer's compatibility drawbacks can be acceptable in many situations, such as when sending data over a network, but may not be a good choice for long term data storage because the Java classes cannot evolve.

FieldSerializer settings

Setting Description Default value
fieldsCanBeNull When false it is assumed that no field values are null, which can save 0-1 byte per field. true
setFieldsAsAccessible When true, all non-transient fields (inlcuding private fields) will be serialized and setAccessible if necessary. If false, only fields in the public API will be serialized. true
ignoreSyntheticFields If true, synthetic fields (generated by the compiler for scoping) are serialized. false
fixedFieldTypes If true, it is assumed every field value's concrete type matches the field's type. This removes the need to write the class ID for field values. false
copyTransient If true, all transient fields will be copied. true
serializeTransient If true, transient fields will be serialized. false
variableLengthEncoding If true, variable length values are used for int and long fields. true
extendedFieldNames If true, field names are prefixed by their declaring class. This can avoid conflicts when a subclass has a field with the same name as a super class. false

CachedField settings

FieldSerializer provides the fields that will be serialized. Fields can be removed, so they won't be serialized. Fields can be configured to make serialiation more efficient.

FieldSerializer fieldSerializer = ...

fieldSerializer.removeField("id"); // Won't be serialized.

CachedField nameField = fieldSerializer.getField("name");
nameField.setCanBeNull(false);

CachedField someClassField = fieldSerializer.getField("someClass");
someClassField.setClass(SomeClass.class, new SomeClassSerializer());
Setting Description Default value
canBeNull When false it is assumed the field value is never null, which can save 0-1 byte. true
valueClass Sets the concrete class and serializer to use for the field value. This removes the need to write the class ID for the value. If the field value's class is a primitive, primitive wrapper, or final, this setting defaults to the field's class. null
serializer Sets the serializer to use for the field value. If the serializer is set, some serializers required the value class to also be set. If null, the serializer registered with Kryo for the field value's class will be used. null
variableLengthEncoding If true, variable length values are used. This only applies to int or long fields. true
optimizePositive If true, positive values are optimized for variable length values. This only applies to int or long fields when variable length encoding is used. true

FieldSerializer annotations

Annotations can be used to configure the serializers for each field.

Annotation Description
@Bind Sets the CachedField settings for any field.
@CollectionBind Sets the CollectionSerializer settings for Collection fields.
@MapBind Sets the MapSerializer settings for Map fields.
@NotNull Marks a field as never being null.
public class SomeClass {
   @NotNull
   @Bind(serializer = StringSerializer.class, valueClass = String.class, canBeNull = false) 
   Object stringField;

   @Bind(variableLengthEncoding = false)
   int intField;

   @BindMap(
      keySerializer = StringSerializer.class, 
      valueSerializer = IntArraySerializer.class, 
      keyClass = String.class, 
      valueClass = int[].class, 
      keysCanBeNull = false)
   Map map;
   
   @BindCollection(
      elementSerializer = LongArraySerializer.class,
      elementClass = long[].class, 
      elementsCanBeNull = false) 
   Collection collection;
}

VersionFieldSerializer

VersionFieldSerializer extends FieldSerializer and provides backward compatibility. This means fields can be added without invalidating previously serialized bytes. Removing, renaming, or changing the type of a field is not supported.

When a field is added, it must have the @Since(int) annotation to indicate the version it was added in order to be compatible with previously serialized bytes. The annotation value must never change.

VersionFieldSerializer adds very little overhead to FieldSerializer: a single additional varint.

VersionFieldSerializer settings

Setting Description Default value
compatible When false, an exception is thrown when reading an object with a different version. The version of an object is the maximum version of any field. true

VersionFieldSerializer also inherits all the settings of FieldSerializer.

TaggedFieldSerializer

TaggedFieldSerializer extends FieldSerializer to provide backward compatibility and optional forward compatibility. This means fields can be added or renamed and optionally removed without invalidating previously serialized bytes. Changing the type of a field is not supported.

Only fields that have a @Tag(int) annotation are serialized. Field tag values must be unique, both within a class and all its super classes. An exception is thrown if duplicate tag values are encountered.

The forward and backward compatibility and serialization performance depends on the readUnknownTagData and chunkedEncoding settings. Additionally, a varint is written before each field for the tag value.

When readUnknownTagData and chunkedEncoding are false, fields must not be removed but the @Deprecated annotation can be applied. Deprecated fields are read when reading old bytes but aren't written to new bytes. Classes can evolve by reading the values of deprecated fields and writing them elsewhere. Fields can be renamed and/or made private to reduce clutter in the class (eg, ignored1, ignored2).

TaggedFieldSerializer settings

Setting Description Default value
readUnknownTagData When false and an unknown tag is encountered, an exception is thrown or, if chunkedEncoding is true, the data is skipped.

When true, the class for each field value is written before the value. When an unknown tag is encountered, an attempt to read the data is made. This is used to skip the data and, if references are enabled, any other values in the object graph referencing that data can still be deserialized. If reading the data fails (eg the class is unknown or has been removed) then an exception is thrown or, if chunkedEncoding is true, the data is skipped.

In either case, if the data is skipped and references are enabled, then any references in the skipped data are not read and further deserialization may receive the wrong references and fail.
chunkedEncoding When true, fields are written with chunked encoding to allow unknown field data to be skipped. This impacts performance. false
chunkSize The maximum size of each chunk for chunked encoding. 1024

TaggedFieldSerializer also inherits all the settings of FieldSerializer.

CompatibleFieldSerializer

CompatibleFieldSerializer extends FieldSerializer to provided both forward and backward compatibility. This means fields can be added or removed without invalidating previously serialized bytes. Renaming or changing the type of a field is not supported. Like FieldSerializer, it can serialize most classes without needing annotations.

The forward and backward compatibility and serialization performance depends on the readUnknownFieldData and chunkedEncoding settings. Additionally, the first time the class is encountered in the serialized bytes, a simple schema is written containing the field name strings. Because field data is identified by name, if a super class has a field with the same name as a subclass, extendedFieldNames must be true.

CompatibleFieldSerializer settings

Setting Description Default value
readUnknownFieldData When false and an unknown field is encountered, an exception is thrown or, if chunkedEncoding is true, the data is skipped.

When true, the class for each field value is written before the value. When an unknown field is encountered, an attempt to read the data is made. This is used to skip the data and, if references are enabled, any other values in the object graph referencing that data can still be deserialized. If reading the data fails (eg the class is unknown or has been removed) then an exception is thrown or, if chunkedEncoding is true, the data is skipped.

In either case, if the data is skipped and references are enabled, then any references in the skipped data are not read and further deserialization may receive the wrong references and fail.
chunkedEncoding When true, fields are written with chunked encoding to allow unknown field data to be skipped. This impacts performance. false
chunkSize The maximum size of each chunk for chunked encoding. 1024

CompatibleFieldSerializer also inherits all the settings of FieldSerializer.

BeanSerializer

BeanSerializer is very similar to FieldSerializer, except it uses bean getter and setter methods rather than direct field access. This slightly slower, but may be safer because it uses the public API to configure the object. Like FieldSerializer, it provides no forward or backward compatibility.

CollectionSerializer

CollectionSerializer serializes objects that implement the java.util.Collection interface.

CollectionSerializer settings

Setting Description Default value
elementsCanBeNull When false it is assumed that no elements in the collection are null, which can save 0-1 byte per element. true
elementClass Sets the concrete class to use for each element in the collection. This removes the need to write the class ID for each element. If the element class is known (eg through generics) and a primitive, primitive wrapper, or final, then CollectionSerializer won't write the class ID even when this setting is null. null
elementSerializer Sets the serializer to use for every element in the collection. If the serializer is set, some serializers required the value class to also be set. If null, the serializer registered with Kryo for each element's class will be used. null

MapSerializer

MapSerializer serializes objects that implement the java.util.Map interface.

MapSerializer settings

Setting Description Default value
keysCanBeNull When false it is assumed that no keys in the map are null, which can save 0-1 byte per entry. true
valuesCanBeNull When false it is assumed that no values in the map are null, which can save 0-1 byte per entry. true
keyClass Sets the concrete class to use for every key in the map. This removes the need to write the class ID for each key. null
valueClass Sets the concrete class to use for every value in the map. This removes the need to write the class ID for each value. null
keySerializer Sets the serializer to use for every key in the map. If the value serializer is set, some serializers required the value class to also be set. If null, the serializer registered with Kryo for each key's class will be used. null
valueSerializer Sets the serializer to use for every value in the map. If the key serializer is set, some serializers required the value class to also be set. If null, the serializer registered with Kryo for each value's class will be used. null

JavaSerializer and ExternalizableSerializer

JavaSerializer and ExternalizableSerializer are Kryo serializers which uses Java's built-in serialization. This is as slow as usual Java serialization, but may be necessary for legacy classes.

java.io.Externalizable and java.io.Serializable do not have default serializers set by default, so the default serializers must be set manually or the serializers set when the class is registered.

class SomeClass implements Externalizable { /* ... */ }
kryo.addDefaultSerializer(Externalizable.class, ExternalizableSerializer.class);
kryo.register(SomeClass.class);
kryo.register(SomeClass.class, new JavaSerializer());
kryo.register(SomeClass.class, new ExternalizableSerializer());

Logging

Kryo makes use of the low overhead, lightweight MinLog logging library. The logging level can be set by one of the following methods:

Log.ERROR();
Log.WARN();
Log.INFO();
Log.DEBUG();
Log.TRACE();

Kryo does no logging at INFO (the default) and above levels. DEBUG is convenient to use during development. TRACE is good to use when debugging a specific problem, but generally outputs too much information to leave on.

MinLog supports a fixed logging level, which causes the Java compiler to remove logging statements below that level at compile time. Kryo must be compiled with a fixed logging level MinLog JAR.

Thread safety

Kryo is not thread safe. Each thread should have its own Kryo, Input, and Output instances.

Pooling

Because Kryo is not thread safe and constructing and configuring a Kryo instance is relatively expensive, in a multithreaded environment ThreadLocal or pooling might be considered.

static private final ThreadLocal<Kryo> kryos = new ThreadLocal<Kryo>() {
   protected Kryo initialValue() {
      Kryo kryo = new Kryo();
      // Configure the Kryo instance.
      return kryo;
   };
};

Kryo kryo = kryos.get();

For pooling, Kryo provides the Pool class which can pool Kryo, Input, Output, or instances of any other class.

// Pool constructor arguments: thread safe, soft references, maximum capacity
Pool<Kryo> kryoPool = new Pool<Kryo>(true, false, 8) {
   protected Kryo create () {
      Kryo kryo = new Kryo();
      // Configure the Kryo instance.
      return kryo;
   }
};

Kryo kryo = kryoPool.obtain();
// Use the Kryo instance here.
kryoPool.free(kryo);
Pool<Output> outputPool = new Pool<Output>(true, false, 16) {
   protected Output create () {
      return new Output(1024, -1);
   }
};

Output output = outputPool.obtain();
// Use the Output instance here.
outputPool.free(output);
Pool<Input> inputPool = new Pool<Input>(true, false, 16) {
   protected Input create () {
      return new Input(1024, -1);
   }
};

Input input = inputPool.obtain();
// Use the Input instance here.
inputPool.free(input);

If true is passed as the first argument to the Pool constructor, the Pool uses synchronization internally and can be accessed by multiple threads concurrently.

If true is passed as the second argument to the Pool constructor, the Pool stores objects using java.lang.ref.SoftReference. This allows objects in the pool to be garbage collected when memory pressure on the JVM is high. Pool clean removes all soft references whose object has been garbage collected. This can reduce the size of the pool when no maximum capacity has been set. When the pool has a maximum capacity, it is not necessary to call clean because Pool free will try to remove an empty reference if the maximum capacity has been reached.

The third Pool parameter is the maximum capacity. If an object is freed and the pool already contains the maximum number of free objects, the specified object is reset but not added to the pool. The maximum capacity may be omitted for no limit.

If an object implements Pool.Poolable then Poolable reset is called when the object is freed. This gives the object a chance to reset its state for reuse in the future. Alternatively, Pool reset can be overridden to reset objects. Input and Output implement Poolable to set their position and total to 0. Kryo does not implement Poolable because its object graph state is typically reset automatically after each serialization (see Reset). If you disable automatic reset via setAutoReset(false), make sure that you call Kryo.reset() before returning the instance to the pool.

Pool getFree returns the number of objects available to be obtained. If using soft references, this number may include objects that have been garbage collected. clean may be used first to remove empty soft references.

Pool getPeak returns the all-time highest number of free objects. This can help determine if a pool's maximum capacity is set appropriately. It can be reset any time with resetPeak.

Benchmarks

Kryo provides a number of JMH-based benchmarks and R/ggplot2 files.

Kryo can be compared to many other serialization libraries in the JVM Serializers project. The benchmarks are small, dated, and homegrown rather than using JMH, so are less trustworthy. Also, it is very difficult to thoroughly compare serialization libraries using a benchmark. Libraries have many different features and often have different goals, so they may excel at solving completely different problems. To understand these benchmarks, the code being run and data being serialized should be analyzed and contrasted with your specific needs. Some serializers are highly optimized and use pages of code, others use only a few lines. This is good to show what is possible, but may not be a relevant comparison for many situations.

Links

Projects using Kryo

There are a number of projects using Kryo. A few are listed below. Please submit a pull request if you'd like your project included here.

Scala

Clojure

Objective-C

Comments
  • Seeing java.lang.IllegalAccessError when upgrading to 2.24.0

    Seeing java.lang.IllegalAccessError when upgrading to 2.24.0

    Wen trying to upgrade from 2.23.0 to 2.24.0, I am seeing following issue:

    java.lang.IllegalAccessError: tried to access method mypkg.MyClazz$MyInnerClazz.()V from class mypkg.MyClazz$MyInnerClazzConstructorAccess at mypkg.MyClazz$MyInnerClazzConstructorAccess.newInstance(Unknown Source) at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy$1.newInstance(Kryo.java:1193) at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1061) at com.esotericsoftware.kryo.serializers.FieldSerializer.createCopy(FieldSerializer.java:620) at com.esotericsoftware.kryo.serializers.FieldSerializer.copy(FieldSerializer.java:624) at com.esotericsoftware.kryo.Kryo.copy(Kryo.java:862) at com.esotericsoftware.kryo.serializers.MapSerializer.copy(MapSerializer.java:157) at com.esotericsoftware.kryo.serializers.MapSerializer.copy(MapSerializer.java:21) at com.esotericsoftware.kryo.Kryo.copy(Kryo.java:862) at com.esotericsoftware.kryo.serializers.UnsafeCacheFields$UnsafeObjectField.copy(UnsafeCacheFields.java:297) at com.esotericsoftware.kryo.serializers.FieldSerializer.copy(FieldSerializer.java:634) at com.esotericsoftware.kryo.Kryo.copy(Kryo.java:862)

    Anyone know what the problem could be?

    opened by lichtin 47
  • Shaded dependencies should be removed or have provided scope

    Shaded dependencies should be removed or have provided scope

    Kryo shaded artifact (with all dependencies, like minlog) replaces main Kryo Maven artifact (jar without dependencies) and becomes the one that gets published to repositories and used. Problem is that the Kryo pom artifact published on repositories does not reflect that dependencies are shaded, it still lists them as compile scoped. So shading a project which depends on Kryo produces merge conflicts, same classes provided from Kryo, and from its dependencies.

    Please adjust Kryo build script to have published pom list shaded dependencies as with provided scope, or removed from dependencies list. See maven-shade-plugin configuration

    opened by sslavic 40
  • Provide sun.misc.Unsafe-based implementation of FieldSerializer

    Provide sun.misc.Unsafe-based implementation of FieldSerializer

    From romixlev on June 28, 2012 08:37:32

    It could be interesting to provide sun.misc.Unsafe based implementation of FieldSerializer. It would be able to read object's memory, write into memory buffers directly using readXYZ/writeXYZ methods of the Unsafe class.

    One potential advantage could be speed.

    More over, and probably more important, it could be seen as an alternative to using ASM, because some people may want to reduce external dependencies to a minimum.

    The implementation should pretty easy and straight forward. Many frameworks use a similar trick already (protostuff, etc).

    Original issue: http://code.google.com/p/kryo/issues/detail?id=75

    bug imported priority medium 
    opened by ghost 39
  • Add simple, queue based kryo pool

    Add simple, queue based kryo pool

    I'm submitting this as a proposal for a simple kryo pool, didn't want to push it directly into master. Some words on the suggested pool...

    Kryo instances are created using a KryoFactory that's passed to the pool. The pool uses a ConcurrentLinkedQueue to manage kryo instances. The pool also allows to run callbacks by passing a kryo instance.

    The included KryoPoolBenchmarkTest (with ITER_CNT = 100000) shows the following output for me (excerpt):

    >>> With pool (average): 8 ms
    >>> Without pool (average): 2,105 ms
    

    KryoPool usage example:

    KryoFactory factory = new KryoFactory() {
      public Kryo create () {
        Kryo kryo = new Kryo();
        // configure kryo
        return kryo;
      }
    };
    KryoPool pool = new KryoPool(factory);
    Kryo kryo = pool.borrow();
    // do s.th. with kryo here, and afterwards release it
    pool.release(kryo);
    
    // or use a callback to work with kryo
    String value = pool.run(new KryoCallback() {
      public String execute(Kryo kryo) {
        return kryo.readObject(input, String.class);
      }
    });
    
    opened by magro 38
  • Java8 Lambdas cannot be deserialized

    Java8 Lambdas cannot be deserialized

    Deserializing Java8 lambdas fails with ClassNotFoundException: Test$$Lambda$1/1279149968

    Example:

      private <T> T kryoSerDeser(T r) throws IOException, ClassNotFoundException {
        Kryo kryo = new Kryo();
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        try (Output output = new Output(baos)) {
          kryo.writeClassAndObject(output, r);
        }
    
        try (Input input = new Input((new ByteArrayInputStream(baos.toByteArray())))) {
          return (T) kryo.readClassAndObject(input);
        }
      }
    
      @Test
      public void testLambdaInKryo() throws IOException, ClassNotFoundException {
        Runnable r = () -> System.out.println("works");
        kryoSerDeser(r).run();
      }
    
    

    Using Java's standard ObjectOutput/Input streams on lambdas implementing Serializable interface works fine.

    opened by ikabiljo 34
  • ArrayIndexOutOfBoundsException in IdentityMap.clear()

    ArrayIndexOutOfBoundsException in IdentityMap.clear()

    From [email protected] on June 24, 2013 22:27:59

    What steps will reproduce the problem?

    1. Instantiate Kryo instance.
    2. Instantiate fairly complex composite Java class (my project uses one that I can't share)
    3. Create 100+ copies of the complex instance in a loop What is the expected output? What do you see instead? Expect all 100+ copies to be made, but instead, I'm getting an ArrayIndexOutOfBoundsException at some point. What version of the Kryo are you using? 2.19, 2.20, 2.21 Please provide any additional information below. This problem does NOT exhibit itself in 2.17 or lower. It pops up only in 2.19 and above. I recognize the problem is probably very specific to the nature of my complex, composite Java object that I'm cloning (copying) and therefore may be difficult for you guys to replicate. If time permits, I may try to troubleshoot this for you, however, I can clearly see something must have changed between 2.17 & 2.19 that affects IdentityMap.clear() to cause this. Perhaps you guys will be able to see the difference and arrive at a conclusion fairly easily. If not, let me know and I'll try to have a deeper look myself when I get an opportunity. In the meantime, I'll go just use 2.17. Thanks

    Original issue: http://code.google.com/p/kryo/issues/detail?id=115

    bug imported priority medium 
    opened by ghost 34
  • Add support for Records in JDK 14

    Add support for Records in JDK 14

    Here is the initial version of a record serializer. A few things to note:

    1. The serializer uses method handles instead of core reflection, i.e to invoke the record's constructor
    2. The serial form of the record is a stream of the record components in the same order as in the class file. I see FieldSerialiser orders by name, this could be done here as well.
    3. I kept plenty of debugging output in RecordSerializerTest, this can be cleaned up before integration.
    4. RecordSerializerTest was added to the default serializers. Do we needs to add a test case to SerializerCompatTest.TestData?
    5. Additional config was added to the pom files to only compile and run the new test if JDK 14+ is available. In this case all tests are compiled and run with JDK 14+ --enable-preview. I am not very familiar with Maven and ended up with this solution after asking a few questions on their mailing list. There might well be a better way of handling test compilation and test runs.
    6. The only test that acts up when run with JDK 14 is UnsafeByteBufferInputOutputTest

    [INFO] Running com.esotericsoftware.kryo.io.UnsafeByteBufferInputOutputTest Streams with preallocated direct memory are not supported on this JVM java.lang.UnsupportedOperationException: No direct ByteBuffer constructor is available. at com.esotericsoftware.kryo.unsafe.UnsafeUtil.newDirectBuffer(UnsafeUtil.java:125) at com.esotericsoftware.kryo.io.UnsafeByteBufferInputOutputTest.testByteBufferOutputWithPreallocatedMemory(UnsafeByteBufferInputOutputTest.java:43)

    I'm sure there are plenty of things that can be improved. Any suggestions welcome!

    opened by FrauBoes 31
  • kryo Encountered unregistered class ID, even though I've registered all classes

    kryo Encountered unregistered class ID, even though I've registered all classes

    I have a spark job that serializes an object with setRegistrationRequired(true). I have added registrations for all the classes. I have java code that deserializes the serialized object, also with setRegistrationRequired(true) and all appropriate registrations, making sure that the order of registrations is the same.

    While deserializing, I get com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 113

    At first, I thought I was missing some class, but then realized if that was the case, I wouldn't be able to serialize the object in the first place, since requiredRegistration is set to true.

    so I'm not sure what I'm doing wrong here.

    opened by mklosi 27
  • Major breaking changes while keeping the same package

    Major breaking changes while keeping the same package

    reading migration guide.

    the best approach is to load the old data somehow, then write it with the newer version. For example, by juggling classloaders the data could be loaded with v2, then written with v5.

    I fail to see how this is the best or even a good approach. Could you please elaborate?

    As far as I see it's best practice for java libs in general to simply move to a new package for major / breaking updates like this eg. com.esotericsoftware.kryo.v5, so everybody is happy with smooth migrations.

    opened by masc3d 26
  • implement the fastGetObjectMap to get better read performance

    implement the fastGetObjectMap to get better read performance

    1. unroll the locate to get method

    • Unroll version have better performance: I test the version of roll and unroll (about 2%)
        SimpleMapBenchmark.read          0.5f    unroll           2048       1000  thrpt    9  360077.662 ± 7646.572  ops/s
        SimpleMapBenchmark.read          0.5f    unroll           2048       3000  thrpt    9  352933.493 ± 5690.624  ops/s
        SimpleMapBenchmark.read          0.5f    roll           2048          1000  thrpt    9  368682.803 ± 7567.719  ops/s
        SimpleMapBenchmark.read          0.5f    roll           2048          3000  thrpt    9  358348.669 ± 7085.066  ops/s
    

    There are two possible reasons: JIT doesn't inline the method locateKey, there is little overhead get method need to compare the result of locateKey and then decide to return the real value

    2. Remove the magic number in place

    • this algorithm will make the low probability of hash collision and take some cost
    • with low loadFactor, we don't need to consider the hash collision

    3. Automatically adjust load factor

    • according to previous benchmark, different size have different best loadFactor
    • I guess the reason is that small map can utilize the cache locality if we get high loadFactor.
    Other little optimization (maybe optimized automatically , no obvious improvement )
    -   remove the key not null check(cuckooObjectMap also not check it, check it in put is enough)
    -   switch the order of check other is null and check other is equal to key(the second is more likely to be true, and key 
            would not be null so we can use key.equals(other))
    

    Benchmark Result

    num = 100

    Benchmark (mapType) (maxCapacity) (numClasses) Mode Cnt Score Error Units SimpleMapBenchmark.read fastGet 2048 100 thrpt 9 1767666.150 ± 7451.419 ops/s SimpleMapBenchmark.read cuckoo 2048 100 thrpt 9 1458135.074 ± 373999.983 ops/s

    num=1000

    Benchmark (mapType) (maxCapacity) (numClasses) Mode Cnt Score Error Units SimpleMapBenchmark.read fastGet 2048 1000 thrpt 9 370069.484 ± 20877.873 ops/s SimpleMapBenchmark.read cuckoo 2048 1000 thrpt 9 363330.568 ± 18756.168 ops/s

    num=3000

    Benchmark (mapType) (maxCapacity) (numClasses) Mode Cnt Score Error Units SimpleMapBenchmark.read fastGet 2048 3000 thrpt 9 355674.722 ± 18533.691 ops/s SimpleMapBenchmark.read cuckoo 2048 3000 thrpt 9 333397.797 ± 78791.696 ops/s

    num=10000

    Benchmark (mapType) (maxCapacity) (numClasses) Mode Cnt Score Error Units SimpleMapBenchmark.read fastGet 2048 10000 thrpt 9 319202.343 ± 11023.639 ops/s SimpleMapBenchmark.read cuckoo 2048 10000 thrpt 9 316253.575 ± 68273.397 ops/s

    enhancement 
    opened by pyb1993 25
  • ArrayIndexOutOfBoundsException when serializing generic classes

    ArrayIndexOutOfBoundsException when serializing generic classes

    Using Kryo 5.0.0-RC3-SNAPSHOT, I get the following Stacktrace when running the test:

    com.esotericsoftware.kryo.KryoException: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
    Serialization trace:
    input (SerializeTest$StringSupplierContainer)
    	at com.esotericsoftware.kryo.serializers.ReflectField.write(ReflectField.java:92)
    	at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:107)
    	at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:630)
    
    import com.esotericsoftware.kryo.Kryo;
    import com.esotericsoftware.kryo.io.Output;
    import org.junit.jupiter.api.Test;
    
    import java.io.Serializable;
    import java.util.function.Supplier;
    
    import static org.assertj.core.api.Assertions.assertThat;
    
    public class SerializeTest {
    
    	@Test
    	void serialize() {
    		final Kryo kryo = new Kryo();
    		kryo.setRegistrationRequired(false);
    
    		final Output output = new Output(1024);
    		kryo.writeClassAndObject(output, new StringSupplierContainer());
    		final byte[] result = output.toBytes();
    
    		assertThat(result).isNotNull();
    	}
    
    	static class EmptyStringSupplier implements Supplier<String>, Serializable {
    
    		@Override
    		public String get() {
    			return "";
    		}
    	}
    
    	static class StringSupplierContainer extends SupplierContainer<String> {
    
    		StringSupplierContainer() {
    			super(new EmptyStringSupplier());
    		}
    	}
    
    	static class SupplierContainer<T> {
    
    		private final Supplier<T> input;
    
    		SupplierContainer(Supplier<T> input) {
    			this.input = input;
    		}
    	}
    }
    

    Possibly related to #648

    opened by theigl 25
Releases(kryo-parent-5.4.0)
  • kryo-parent-5.4.0(Dec 30, 2022)

    This is a maintenance release coming with bug fixes and performance improvements.

    The most notable change is performance improvements for record serialization (#927) by caching access to components, getters and constructors. The new implementation is roughly 20x faster.

    Several PRs improve Kryo's compatibility with JDK 17+ (#930, #932, #933).

    #923 Add helper method to register serializers for java.util.ImmutableCollections (#933) #885 Delay access to constructor and methods for DirectBuffers until first use (#932) #885 Add additional safe serializers for commonly used JDK classes (#930) #884 Cache components, getters and constructors in RecordSerializer (#927) #922 Fall back to default class resolution if class cannot be loaded with provided class loader (#926) #920 Align ByteBufferOutput.writeAscii with implementation in Output (#921) #889 Serializer for java.sql.Timestamp which preserves nanoseconds (#890) #884 Fix eclipse project setup: Add JUnit 5, ByteBuddy (#887)

    Other Tasks:

    • Improve tests for JDK 17 (#886)
    • Upgrade GitHub actions (#925)
    • Upgrade Objenesis to 3.3

    The full list of changes can be found here.

    Many thanks to all contributors!

    Upgrade Notes

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.4.0-all.zip(2.12 MB)
  • kryo-parent-5.3.0(Feb 11, 2022)

    This is a maintenance release coming with bug fixes and performance improvements:

    #872 Custom exception for buffer over/underflow (#874) #873 Set record component getter accessible (#875) #845 Performance improvements for maps (#876) #845 Use IdentityMap instead of CuckooObjectMap in class resolver (#877) #882 Catch LinkageError in addition to RTE when accessing fields via ASM (#883)

    Other Tasks:

    • Enforce minimum maven version and update plugins and test dependencies

    The full list of changes can be found here.

    Many thanks to all contributors!

    Upgrade Notes

    This release brings performance improvements for Kryo's custom map implementations (#876). These improvements allowed us to change the map for resolving registrations in DefaultClassResolver from a CuckooObjectMap to an IdentityMap with 3-5% faster throughput (#877). CuckooObjectMap is now deprecated and will be removed in Kryo 6.

    Kryo now throws dedicated exceptions in case of buffer under- or overflows (#872). If you currently parse the error message to check for these conditions, you can catch KryoBufferUnderflowException and KryoBufferOverflowException instead.

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - No (Details)
    • Source compatible - No (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.3.0-all.zip(3.25 MB)
  • kryo-parent-5.2.1(Dec 11, 2021)

    This is a maintenance release coming with bug fixes and improvements:

    #834 Support skipping input chunks after a buffer underflow (#850) #865 Ensure empty PriorityQueue can be deserialized (#866) #870 Shade contents of source jar for versioned artifact

    Other Tasks:

    • Migrate from Travis CI to GitHub Actions
    • Build and test with JDK 17

    The full list of changes can be found here.

    Many thanks to all contributors!

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.2.1-all.zip(2.17 MB)
  • kryo-parent-5.2.0(Jul 31, 2021)

    This is a maintenance release coming with bug fixes and improvements.

    #849 Fall back to getDeclaredConstructor for non-public records #848 Fix #847 Ensure that RecordSerializer can deal with subtypes #841 Fix #840 Ensure primitive types are assignable to Comparable/Serializables fields #839 Fix #838 Avoid flush repeatedly when has finished flushing #829 OSS-Fuzz Integration

    The full list of changes can be found here.

    Many thanks to all contributors!

    Important Upgrade Information

    This release fixes two critical issues with the serializer for java.util.Record. One of the issues (#847) seriously limits the serializers practical usefulness, so we decided to make an exception and break serialization compatibility. If you have serialized records with non-final field types that you need to read with Kryo 5.2.0, you can enable backwards compatibility globally or for individual types (recommended):

    1. Register global default serializer:
        final RecordSerializer<?> rs = new RecordSerializer<>();
        rs.setFixedFieldTypes(true);
        kryo.addDefaultSerializer(Record.class, rs);
    
    1. Register serializer per type:
        final RecordSerializer<?> rs = new RecordSerializer<>();
        rs.setFixedFieldTypes(true);
        kryo.register(MyRecord.class, rs);
    

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes (except for java.util.Record)
      • Unsafe-based IO: Yes (except for java.util.Record)
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.2.0-all.zip(2.18 MB)
  • kryo-parent-5.1.1(May 2, 2021)

    This is a maintenance release coming with bug fixes for CompatibleFieldSerializer and the versioned artifact.

    In Kryo 5.1.0, a direct dependency on the unversioned JAR accidentally made it into the POM for the versioned artifact (see #825). If you are using the versioned artifact, upgrading to 5.1.1 is recommended.

    #822: Fix #821 Support closures when validating types in CompatibleFieldSerializer #824: Fix #823 Fix ArrayIndexOutOfBoundsException in binary search logic #825: Remove direct dependency from versioned artifact

    The full list of changes can be found here.

    Many thanks to all contributors!

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.1.1-all.zip(2.16 MB)
  • kryo-parent-5.1.0(Apr 8, 2021)

    This release comes with support for java.util.Record, improved compatibility with Android API <26, and OSGI support for the versioned artifact.

    #766: Fix #735 Support for java.util.Record #814: Fix #691 Bump Objenesis to 3.2 #818: Fix #811 OSGI support for versioned artifact

    The full list of changes can be found here.

    Many thanks to all contributors!

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.1.0-all.zip(6.38 MB)
  • kryo-parent-5.0.4(Mar 12, 2021)

    This is a maintenance release coming with a bugfix for Pool with soft references.

    #805 : Fix #804 Prevent IllegalStateException: Queue full

    The full list of changes can be found here.

    Many thanks to all contributors!

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.4-all.zip(3.18 MB)
  • kryo-parent-5.0.3(Dec 15, 2020)

    This is a maintenance release coming with a bugfix for generics optimization in highly concurrent environments.

    #799: Fix #798 Use equals to compare type variables

    The full list of changes can be found here.

    Many thanks to all contributors!

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.3-all.zip(7.50 MB)
  • kryo-parent-5.0.2(Dec 1, 2020)

    This is a maintenance release coming with a bugfix for CompatibleFieldSerializer.

    #794: Fix #774 Validate read types against field type instead of valueClass

    The full list of changes can be found here.

    Many thanks to all contributors!

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.2-all.zip(3.18 MB)
  • kryo-parent-5.0.1(Nov 22, 2020)

    This is a maintenance release coming with fixes and improvements.

    #777: Fix #776 Automatic module name #779: Dependency Upgrades #780: Fix #778 Use canonical class name for registration hint if available #787: Fix #786 Ensure field serializers balance pushGenericType and popGenericType calls #788: Fix #774, #784 Do not re-use serializers for CompatibleFieldSerializer and TaggedFieldSerializer #790: Fix #789 Catch StackOverflowError and add hint for enabling references

    The full list of changes can be found here.

    Many thanks to all contributors!

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.1-all.zip(4.57 MB)
  • kryo-parent-5.0.0(Oct 18, 2020)

    This is the final release of the new major version 5.0.0 of Kryo (see also the RC1 release notes for major changes of v5).

    This version comes with only minor changes on top of RC9 (the list of changes from RC9 can be found here). For a detailed list of changes to 4.x please go though the release notes for the release candidates for kryo 5.

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    Due to big changes 5.0.0 is both source/binary incompatible and serialization incompatible to previous major versions - for details see the RC1 compatibility report and those of the subsequent RCs.

    Compatibility of 5.0.0 to RC9:

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.0-all.zip(16.99 MB)
  • kryo-parent-5.0.0-RC9(Aug 14, 2020)

    This is the 9th release candidate of the new major version of Kryo (see also the RC1 release notes for major changes of v5).

    This RC comes with significant performance improvements. The exact list of changes from RC8 to RC9 can be found here.

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    Due to big changes 5.0.0 is both source/binary incompatible and serialization incompatible to previous major versions - for details see the RC1 compatibility report.

    Compatibility of 5.0.0-RC9 to RC8:

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - No (Details)
    • Source compatible - No (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.0-RC9-all.zip(11.69 MB)
  • kryo-parent-5.0.0-RC8(Aug 3, 2020)

    This is the 8th release candidate of the new major version of Kryo (see also the RC1 release notes for major changes of v5).

    This RC comes with minor fixes and performance improvements. The exact list of changes from RC7 to RC8 can be found here.

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    Due to big changes 5.0.0 is both source/binary incompatible and serialization incompatible to previous major versions - for details see the RC1 compatibility report.

    Compatibility of 5.0.0-RC8 to RC7:

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.0-RC8-all.zip(7.51 MB)
  • kryo-parent-5.0.0-RC7(Jul 11, 2020)

    This is the 7th release candidate of the new major version of Kryo (see also the RC1 release notes for major changes of v5).

    This RC comes with support for JDK9+ immutable collections, more improvements of generics handling and some code cleanup. The exact list of changes from RC6 to RC7 can be found here.

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    Due to big changes 5.0.0 is both source/binary incompatible and serialization incompatible to previous major versions - for details see the RC1 compatibility report.

    Compatibility of 5.0.0-RC7 to RC6:

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - No (Details)
    • Source compatible - No (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.0-RC7-all.zip(4.44 MB)
  • kryo-parent-5.0.0-RC6(May 16, 2020)

    This is the 6th release candidate of the new major version of Kryo (see also the RC1 release notes for major changes of v5).

    This RC comes with improvements of generics handling compared to RC5. The exact list of changes from RC5 to RC6 can be found here.

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    Due to big changes 5.0.0 is both source/binary incompatible and serialization incompatible to previous major versions - for details see the RC1 compatibility report.

    Compatibility of 5.0.0-RC6 to RC5:

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.0-RC6-all.zip(8.07 MB)
  • kryo-parent-5.0.0-RC5(Mar 8, 2020)

    This is the 5th release candidate of the new major version of Kryo (see also the RC1 release notes for major changes of v5).

    This RC comes with several improvements over the previous RCs, e.g. there's now a completely self-contained, versioned artifact with zero dependencies for usage of Kryo in libraries. The full list of changes from RC4 to RC5 can be found here.

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    Compatibility

    Due to big changes 5.0.0 is both source/binary incompatible and serialization incompatible to previous major versions - for details see the RC1 compatibility report.

    Compatibility of 5.0.0-RC5 to RC4:

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - No (Details)
    • Source compatible - No (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.0-RC5-all.zip(6.42 MB)
  • kryo-parent-5.0.0-RC4(Apr 14, 2019)

  • kryo-parent-5.0.0-RC3(Apr 8, 2019)

    This is the third release candidate of the new major version of Kryo, which fixes many issues and makes many long awaited improvements (see also the RC1 release notes).

    This RC comes with fixes and approvements over the previous RCs: here's what changed from RC2 to RC3

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    This RC may be the last one asking for feedback before 5.0.0. If you think you found a bug, please submit an issue. If you think something should be changed before 5.0.0 is released, please post on the mailing list.

    Compatibility

    Due to big changes 5.0.0 is both source/binary incompatible and serialization incompatible to previous major versions - for details see the RC1 compatibility report.

    Compatibility of 5.0.0-RC3 to RC2:

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - No (Details)
    • Source compatible - No (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.0-RC3-all.zip(1.60 MB)
  • kryo-parent-5.0.0-RC2(Feb 5, 2019)

    This is the second release candidate of the new major version of Kryo, which fixes many issues and makes many long awaited improvements (see also the RC1 release notes).

    The second RC comes with fixes and approvements over the first RC based on your valuable feedback: here's what changed from RC1 to RC2

    For migration from previous major versions please check out the migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    This RC may be the last one asking for feedback before 5.0.0. If you think you found a bug, please submit an issue. If you think something should be changed before 5.0.0 is released, please post on the mailing list.

    Compatibility

    Due to big changes 5.0.0 is both source/binary incompatible and serialization incompatible to previous major versions - for details see the RC1 compatibility report.

    Compatibility of 5.0.0-RC2 to RC1:

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.0-RC2-all.zip(1.59 MB)
  • kryo-parent-5.0.0-RC1(Jun 19, 2018)

    This is the first release candidate of the new major version of Kryo, which fixes many issues and makes many long awaited improvements.

    For details see this mailing list thread or check out the changes from 4.0.2 to 5.0.0-RC1.

    So far there is only a minimal migration guide. We're asking the community to help and contribute this part: please edit the migration wiki page as you encounter any information or issues that might help others.

    With the release candidate we're looking for feedback. If you think you found a bug, please submit an issue. If you think something should be changed before 5.0.0 is released, please post on the mailing list.

    Compatibility

    Due to big changes it is both source/binary incompatible and serialization incompatible.

    • Serialization compatible
      • Standard IO: No
      • Unsafe-based IO: No
    • Binary compatible - No (Details)
    • Source compatible - No (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-5.0.0-RC1-all.zip(1.72 MB)
  • kryo-parent-4.0.2(Mar 20, 2018)

    This is a maintenance release coming with fixes and improvements.

    • #567: Use public member instead of private field to resolve immutability for serializer (bceef26)
    • Fix #558, #549: IdentityObjectIntMap.clear taking more time - Realloc instead of clearing large maps (77935c6)
    • Fix #539, #554: Fix crash when removing multiple fields with CompatibleFieldSerializer (56fb1a1)
    • #530: Fix #529 Support serializing the Enum class object (033d659)

    Many thanks to all contributors!

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-4.0.2-all.zip(1.67 MB)
  • kryo-parent-4.0.1(Jul 23, 2017)

    This is a maintenance release coming with fixes and improvements.

    • #527: Add documentation on very large object graphs (67a3499)
    • #516: Use relatively safe max size of java array (389d33a)
    • #521: Use chunked encoding for TaggedFieldSerializer forward compatibility (fixes #442), replaces ignoreUnknownTags with skipUnknownTags, use chunked encoding (6dc6aa5)
    • Fix #500: Push java source/target version to 1.7 (c1ff83c)
    • Deprecate Util.isAndroid, superseded by the final Util.IS_ANDROID (64d7784)
    • #514: Fix warning on instantiate object on Android N+ - update Objenesis to the latest version 2.5.1 (4050fc9, 6040efd)
    • #520: Correctly deserialize objects when fields are added to a Class with enough fields to trigger binary search in CompatibleFieldSerializer (d31e2bf)
    • #518: error message for problems with anonymous classes (0eb7b71)
    • #509: Clarify default used registrations (75a41a6)
    • #441: Check for overlapping tags. (763ce88)
    • Fix #503: writeAscii_slow should be able to write at least one byte into the output buffer. (c3ed14c)
    • #486: Fix IdentityMap constructor (968c240)
    • #483: Override ObjectInputStream ClassLoader (19a6b5e)
    • #465: Fix growing ByteBufferOutput while writing varint (e721a44)
    • Fix #450: Update to the latest clirr-maven-plugin (6b8cb36)

    Many thanks to all contributors!

    Compatibility

    • Serialization compatible
      • Standard IO: Yes
      • Unsafe-based IO: Yes
    • Binary compatible - Yes (Details)
    • Source compatible - Yes (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-4.0.1-all.zip(1.67 MB)
  • kryo-parent-4.0.0(Jul 5, 2016)

    • [BREAKING] Generics handling is more robust now, the former optimization for smaller size (but increased serialization time) is now optional and disabled by default.
      Important: This change breaks the serialization format of the FieldSerializer for generic fields, therefore generic classes serialized with Kryo 3 and FieldSerializer cannot be deserialized with Kryo 4 by default. To deserialize such Kryo 3 serialized generic classes you have to set kryo.getFieldSerializerConfig().setOptimizedGenerics(true);!
      For details see #433: Disable the optimization of generics serialization and provide an API for enabling/disabling it (9923d05).
      This improves/fixes #377 "Kryo does not correctly support parameterized type hierarchy", #384 "ClassCastException When Writing a Map After Previously Reading a Map with Different Generic Type"
    • #424 Fixed (Compatible)FieldSerializer inheritance issue (5a7b7c5)
    • Update reflectasm to latest version, reduction of synchronization (132456e, d77c752)
    • #352 Add forward compatibility to TaggedFieldSerializer (93bff2d)
    • #418 Add checks to setting output buffer and throw IllegalArgumentException when setting a buffer size greater than the max buffer size (d57e00c)
    • #417 Make FieldSerializer.serializeTransient usable (8aae38d)
    • [BREAKING] #415 Make Closure public, moving it to ClosureSerializer (bf80397, 6c402d9) Fixes #299 "Can't register ClosureSerializer"
    • #414 Introduce FieldSerializerConfig to encapsulate config in Kryo (4a47981)
    • Fix #412: Add default serializer for java.net.URL (c10fd76)
    • #404 Use system property to detect Android (c238f97)
    • #396 Adds Charset serializer, also set as default serializer (d7e924b) Resolves #364 "Few of the java Charsets classes from rt.jar can not be deserialized"
    • #395 Adds support for java 8 java.time.* classes (aff0689)
    • [BREAKING] #392 Move Generics{,Resolver} to serializers, reduce public api (e22ffb8)
    • #400 Modified utility hash maps to gracefully handle very large number of entries. (2b8e6fd) Resolves #382 "Kryo breaks down while serializing genomic data"
    • Adds serializers for java8 Optional{Int,Long,Double} (fb65ee9)
    • #362 Add java8 support and serializer for java.util.Optional (12229ca)
    • #393 Fix #389 revert ByteOrder on overflow in require during writeVar_() during writeVar_() (7d840a0)
    • #394 Fix positioning and order updates in ByteBufferInput and ByteBufferOutput (88f5875)
    • #388 Resolve #386: Automatically test serialization compatibility (c904c7e)
    • #375 Fix #370 copyTransient is not global (8414444, 481ddf4, 1ada47d)
    • #368) Try to load classes with fallback that uses current ClassLoader. (8dee484)
    • Update objenesis to 2.2 (ba67801)
    • #359 Use binary search to find field in CompatibleFieldSerializer for object with lots of fields (8678e50)
    • Fixed #346 Issue resizing UnsafeMemoryOutput on OSX 10.10.5 (50d1a6f)
    • Fixed bug #340 KryoException: Encountered unregistered class ID: <negative_number> (2fdf64e)
    • #344 Propagate flush to underlying stream (b3d6bda)
    • #342 Prevent creation of cyclic Generics stack when serializing specific generics object trees
    • #337/#338 ByteBufferInputStream violates InputStream contract (cf06241)
    • #297 Warn unregistered classes option (fd7d1b2)

    Here's the list of all commits since 3.0.3

    And most importantly; many thanks to all contributors!

    Compatibility

    • Serialization compatible
      • Standard IO: No (format for generic fields written by FieldSerializer changed, compatibility can be achieved with kryo.getFieldSerializerConfig().setOptimizedGenerics(true);)
      • Unsafe-based IO: No (format for generic fields written by FieldSerializer changed, compatibility can be achieved with kryo.getFieldSerializerConfig().setOptimizedGenerics(true);)
    • Binary compatible - No (Details)
    • Source compatible - No (Details)
    Source code(tar.gz)
    Source code(zip)
    kryo-4.0.0-all.zip(1.65 MB)
Owner
Esoteric Software
Esoteric Software
FST: fast java serialization drop in-replacement

fast-serialization up to 10 times faster 100% JDK Serialization compatible drop-in replacement (Ok, might be 99% ..). As an example: Lambda Serializat

moru0011 1.5k Dec 15, 2022
A Java library for serializing objects as PHP serialization format.

Java PHP Serializer Latest release: A Java library for serializing objects as PHP serialization format. The library fully implements the PHP serializa

Marcos Passos 14 Jun 13, 2022
Library for manually creating Java serialization data.

Library for creating Java serialization data; mainly intended for research purposes. It is not recommended to use it in production as alternative for ObjectOutputStream.

null 20 Aug 23, 2022
this project is a checker for virus's and token loggers in java apps

Rat checker this project is a checker for virus's and token loggers in java apps this project is not finished and when it is it will never be perfect.

max! 40 Sep 30, 2021
MessagePack serializer implementation for Java / msgpack.org[Java]

MessagePack for Java MessagePack is a binary serialization format. If you need a fast and compact alternative of JSON, MessagePack is your friend. For

MessagePack 1.3k Dec 31, 2022
Classpy is a GUI tool for investigating Java class file, Lua binary chunk, Wasm binary code, and other binary file formats.

Classpy Classpy is a GUI tool for investigating Java class file, Lua binary chunk, Wasm binary code, and other binary file formats. Inspiration This t

null 1k Dec 17, 2022
binary serialization format

Colfer Colfer is a binary serialization format optimized for speed and size. The project's compiler colf(1) generates source code from schema definiti

Pascal S. de Kloe 680 Dec 25, 2022
FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

Google 19.6k Dec 31, 2022
Screaming fast JSON parsing and serialization library for Android.

#LoganSquare The fastest JSON parsing and serializing library available for Android. Based on Jackson's streaming API, LoganSquare is able to consiste

BlueLine Labs 3.2k Dec 18, 2022
FST: fast java serialization drop in-replacement

fast-serialization up to 10 times faster 100% JDK Serialization compatible drop-in replacement (Ok, might be 99% ..). As an example: Lambda Serializat

moru0011 1.5k Dec 15, 2022
Immutable key/value store with efficient space utilization and fast reads. They are ideal for the use-case of tables built by batch processes and shipped to multiple servers.

Minimal Perfect Hash Tables About Minimal Perfect Hash Tables are an immutable key/value store with efficient space utilization and fast reads. They a

Indeed Engineering 92 Nov 22, 2022
Tank - a beginner-friendly, fast, and efficient FTC robot framework

Tank beta a beginner-friendly, fast, and efficient FTC robot framework Overview tank is a FTC robot framework designed to be beginner-friendly, fast,

Aarush Gupta 1 Jan 8, 2022
Netflix, Inc. 809 Dec 28, 2022
Fast and Easy mapping from database and csv to POJO. A java micro ORM, lightweight alternative to iBatis and Hibernate. Fast Csv Parser and Csv Mapper

Simple Flat Mapper Release Notes Getting Started Docs Building it The build is using Maven. git clone https://github.com/arnaudroger/SimpleFlatMapper.

Arnaud Roger 418 Dec 17, 2022
Apache POI - A Java library for reading and writing Microsoft Office binary and OOXML file formats.

Apache POI A Java library for reading and writing Microsoft Office binary and OOXML file formats. The Apache POI Project's mission is to create and ma

The Apache Software Foundation 1.5k Jan 1, 2023
Automatic creation of simple CRUD API of Spring boot and JPA project.

fast-crud Automatic creation of simple CRUD API of Spring boot and JPA project.

JinHwanKim 18 Oct 23, 2022
Automatic generated Jump and Run

Project GenJumpAndRun An automatic generating Jump And Run ?? I’m currently working on Nothing ?? I’m currently learning Games developer (Java) ??‍??

quodix 0 Jul 13, 2022
Automatic generation of the Builder pattern for Java

FreeBuilder Automatic generation of the Builder pattern for Java 1.8+ The Builder pattern is a good choice when designing classes whose constructors o

inferred.org 797 Dec 19, 2022
OAUTHScan is a Burp Suite Extension written in Java with the aim to provide some automatic security checks

OAUTHScan is a Burp Suite Extension written in Java with the aim to provide some automatic security checks, which could be useful during penetration testing on applications implementing OAUTHv2 and OpenID standards.

Maurizio S 163 Nov 29, 2022