High performance I/O library for Java using io_uring under the hood

Related tags

Utility nio_uring
Overview

nio_uring

nio_uring is an I/O library for Java that uses io_uring under the hood, which aims to be:

  • A simple and flexible API
  • Super fast and efficient
  • Truly zero-copy (the kernel addresses direct ByteBuffers for I/O operations)
  • Slightly opinionated

Feedback, suggestions, and contributions are most welcome!

Requirements

  • Linux >= 5.1
  • Java >= 8

For both of these, the higher the version the better - free performance!

Maven Usage

<dependency>
    <groupId>sh.blake.niouringgroupId>
    <artifactId>nio_uringartifactId>
    <version>0.1.1version>
dependency>

TCP Server Example

Here's a basic HTTP server from sh.blake.niouring.examples.HttpHelloWorldServer. There are a few other examples in the same package.

{ // queue another accept request for the next client ring.queueAccept(serverSocket); // set up the read handler and queue a read operation socket.onRead(in -> ring.queueWrite(socket, responseBuffer.slice())); ring.queueRead(socket, ByteBuffer.allocateDirect(1024)); // HTTP spec says the server should close when done socket.onWrite(out -> socket.close()); // and some really basic error handling socket.onException(ex -> { ex.printStackTrace(); socket.close(); }); }); new IoUring() .onException(Exception::printStackTrace) .queueAccept(serverSocket) // queue an accept request, onAccept will be called when a socket connects .loop(); // process I/O events until interrupted }">
public static void main(String[] args) {
    String response = "HTTP/1.1 200 OK\r\n\r\nHello, world!";
    ByteBuffer responseBuffer = ByteBufferUtil.wrapDirect(response);

    IoUringServerSocket serverSocket = new IoUringServerSocket(8080);
    serverSocket.onAccept((ring, socket) -> {
        // queue another accept request for the next client
        ring.queueAccept(serverSocket);

        // set up the read handler and queue a read operation
        socket.onRead(in -> ring.queueWrite(socket, responseBuffer.slice()));
        ring.queueRead(socket, ByteBuffer.allocateDirect(1024));

        // HTTP spec says the server should close when done
        socket.onWrite(out -> socket.close());

        // and some really basic error handling
        socket.onException(ex -> {
            ex.printStackTrace();
            socket.close();
        });
    });

    new IoUring()
        .onException(Exception::printStackTrace)
        .queueAccept(serverSocket) // queue an accept request, onAccept will be called when a socket connects
        .loop(); // process I/O events until interrupted
}

File Support

A barebones cat implementation from sh.blake.niouring.examples.CatExample:

public static void main(String[] args) {
    IoUringFile file = new IoUringFile(args[0]);
    file.onRead(in -> {
        System.out.println(StandardCharsets.UTF_8.decode(in));
        file.close();
    });
    new IoUring()
        .queueRead(file, ByteBuffer.allocateDirect(8192))
        .execute(); // process at least one I/O event (blocking until complete)
}

There's also an example HTTP server that will respond with this README in the examples package!

Performance Tuning

Pretty much all performance tuning is done with one knob - the ringSize argument to the IoUring constructor, which has a default value of 512 if not provided. This value controls the number of outstanding I/O events (accepts, reads, and writes) at any given time. It is constrained by memlock limits (ulimit -l) which can be increased as necessary. Don't forget about file descriptor limits (ulimit -n) too!

Beyond this, you will have to run multiple rings across multiple threads. See sh.blake.niouring.examples.ParallelHttpEchoServer for a simple starter using java.util.concurrent APIs.

Caveats / Warnings

Thread safety

As of now, you should only call read/write/close operations from an IoUring handler (onAccept, onRead, onWrite, etc). This is because nio_uring uses liburing under the hood and its internal submission/completion system is shared and not thread safe. We understand this is an important feature and are working on an efficient way to wake up the main ring loop and have it wait for external threads to perform modifications before resuming.

Runtime Exceptions

nio_uring holds the opinion that checked exceptions, as a concept, were probably a mistake. Therefore, all exceptions produced by nio_uring will extend java.lang.RuntimeException and will always be forwarded to the appropriate exception handler if generated intentionally (see socket.onException).

Direct buffers

All ByteBuffer instances used with this library must be direct (e.g. allocated with allocateDirect). This is a hard requirement and is what enables zero-copy functionality.

Multiple reads/writes

Queuing multiple operations is fine, but try not to queue a read/write operation for the same buffer to the same socket more than once per ring execution, because the internal mapping system is not designed to handle this. The data will be read/written, but your handler will only be called for the first operation, and an java.lang.IllegalStateException exception with message "Buffer already removed" will be sent to the exception handler for the second.

Building

Set the LIBURING_PATH environment variable to the root of a fully compiled liburing directory.

Then ./gradlew build and you're off!

License

MIT. Have fun and make cool things!

Comments
  • Benchmarks

    Benchmarks

    Need to do some distributed benchmarking, all local tools I've tried will overwhelm the client program far before the server sees any measurable load. As performance is a feature, it might be a good idea to set up a pipeline that measures performance and flags serious regressions.

    documentation 
    opened by bbeaupain 1
  • Test Coverage

    Test Coverage

    While publishing to Maven central is a huge goal for this project, I'm not comfortable doing it until we have some serious test coverage. Let's get that moving.

    enhancement 
    opened by bbeaupain 0
  • Fix errnos generated for buffer error conditions

    Fix errnos generated for buffer error conditions

    16 means "Device or resource busy" and that's lying in these situations:

    https://github.com/bbeaupain/nio_uring/blob/main/src/main/c/liburing_provider.c#L152 https://github.com/bbeaupain/nio_uring/blob/main/src/main/c/liburing_provider.c#L183

    bug 
    opened by bbeaupain 0
  • Alternatives for primitive maps

    Alternatives for primitive maps

    Thanks for the amazing work in this project. The narrow focus and attention to detail is much appreciated.

    I was curious to understand the decision to take the dependency on Colt for primitive maps, given alternatives like Eclipse Collection's LongObjectMap or the open addressing maps from Agrona.

    Do you have any interest in dropping the Colt dependency in favor of a smaller/slimmer dep focused only on the primitive collection?

    opened by ohpauleez 0
  • Polled submission mode & pre-registered buffers

    Polled submission mode & pre-registered buffers

    Polled submission mode would be awesome to support for performance reasons. Integration with nio_uring, however, will not be quite straightforward due to polled mode requiring pre-registration of buffers. Ideally, we make the user experience simple and graceful with a managed buffer pool.

    enhancement 
    opened by bbeaupain 0
  • External thread support

    External thread support

    We need an efficient (if possible) and safe way to queue I/O operations from external threads. Possibly just a simple synchronization mechanism and a ring wakeup call will be enough.

    enhancement 
    opened by bbeaupain 0
Releases(0.1.2)
Owner
Blake Beaupain
Blake Beaupain
Discord4J is a fast, powerful, unopinionated, reactive library to enable quick and easy development of Discord bots for Java, Kotlin, and other JVM languages using the official Discord Bot API.

Discord4J is a fast, powerful, unopinionated, reactive library to enable quick and easy development of Discord bots for Java, Kotlin, and other JVM languages using the official Discord Bot API.

null 1.5k Jan 4, 2023
AWS Lambda Performance comparison

aws-lambda-runtimes-performance AWS Lambda Performance comparison The full analyze is here https://filia-aleks.medium.com/benchmarking-all-aws-lambda-

Aleksandr Filichkin 87 Dec 11, 2022
Diff Utils library is an OpenSource library for performing the comparison / diff operations between texts or some kind of data: computing diffs

Diff Utils library is an OpenSource library for performing the comparison / diff operations between texts or some kind of data: computing diffs, applying patches, generating unified diffs or parsing them, generating diff output for easy future displaying (like side-by-side view) and so on.

null 951 Jan 5, 2023
"Pathfinder" - a small demo app made in Java, using Swing which shows an example implementation of a pathfinding algorithm based on BFS

"Pathfinder" is a small demo app made in Java, using Swing which shows an example implementation of a pathfinding algorithm based on BFS.

Dan Sirbu 2 Mar 9, 2022
An open-source Java library for Constraint Programming

Documentation, Support and Issues Contributing Download and installation Choco-solver is an open-source Java library for Constraint Programming. Curre

null 607 Jan 3, 2023
Java rate limiting library based on token/leaky-bucket algorithm.

Java rate-limiting library based on token-bucket algorithm. Advantages of Bucket4j Implemented on top of ideas of well known algorithm, which are by d

Vladimir Bukhtoyarov 1.7k Jan 8, 2023
A Java library for designing good error messages

JDoctor, a Java library for good error messages Designing good error messages is hard. In Java, most often, developers just rely on throwing exception

Cédric Champeau 125 Oct 24, 2022
Ta4j is an open source Java library for technical analysis

Ta4j is an open source Java library for technical analysis. It provides the basic components for creation, evaluation and execution of trading strategies.

null 1.7k Dec 31, 2022
hella-html is a library that makes it hella easy to generate dynamic HTML in vanilla Java.

Hella easy HTML in Java hella-html is a library that makes it hella easy to generate dynamic HTML in vanilla Java. Very lightweight and fast, the prim

null 1 Nov 23, 2022
documents4j is a Java library for converting documents into another document format

documents4j is a Java library for converting documents into another document format. This is achieved by delegating the conversion to any

documents4j 455 Dec 23, 2022
java common utils library

java-common-utils java common utils library 一个简单的Java通用工具类,目前的设想,包括简化异常处理工具、简易限流处理工具等 ExceptionHandler, 目标简化try catch的代码冗余度

xuangy 2 Jan 21, 2022
A Java API for checking if text contains profanity via the alt-profanity-checker Python library.

ProfanityCheckerAPI A Java API for checking if text contains profanity via the alt-profanity-checker Python library. It uses jep to run and interpret

William 2 Feb 19, 2022
archifacts is a library to extract your architectural concepts out of your application's code

archifacts is a free (Apache 2.0 license) library for describing and detecting architectural building blocks and their relationships in your Java appl

null 45 Nov 29, 2022
Java lib for monitoring directories or individual files via java.nio.file.WatchService

ch.vorburger.fswatch Java lib for monitoring directories or individual files based on the java.nio.file.WatchService. Usage Get it from Maven Central

Michael Vorburger ⛑️ 21 Jan 7, 2022
Tencent Kona JDK11 is a no-cost, production-ready distribution of the Open Java Development Kit (OpenJDK), Long-Term Support(LTS) with quarterly updates. Tencent Kona JDK11 is certified as compatible with the Java SE standard.

Tencent Kona JDK11 Tencent Kona JDK11 is a no-cost, production-ready distribution of the Open Java Development Kit (OpenJDK), Long-Term Support(LTS) w

Tencent 268 Dec 16, 2022
This repository contains Java programs to become zero to hero in Java.

This repository contains Java programs to become zero to hero in Java. Data Structure programs topic wise are also present to learn data structure problem solving in Java. Programs related to each and every concep are present from easy to intermidiate level

Sahil Batra 15 Oct 9, 2022
Java Constraint Programming solver

https://maven-badges.herokuapp.com/maven-central/org.jacop/jacop/badge.svg [] (https://maven-badges.herokuapp.com/maven-central/org.jacop/jacop/) JaCo

null 202 Dec 30, 2022
Java Constraint Solver to solve vehicle routing, employee rostering, task assignment, conference scheduling and other planning problems.

OptaPlanner www.optaplanner.org Looking for Quickstarts? OptaPlanner’s quickstarts have moved to optaplanner-quickstarts repository. Quick development

KIE (Drools, OptaPlanner and jBPM) 2.8k Jan 2, 2023
Alibaba Java Diagnostic Tool Arthas/Alibaba Java诊断利器Arthas

Arthas Arthas is a Java Diagnostic tool open sourced by Alibaba. Arthas allows developers to troubleshoot production issues for Java applications with

Alibaba 31.5k Jan 4, 2023