Unconventional I/O library for Java

Overview

one-nio Build Status Javadocs

one-nio

one-nio is a library for building high performance Java servers. It features OS capabilities and JDK internal APIs essential for making your high load applications get maximum of the underlying system.

one-nio highlights

  • Own native socket library.
  • APIs for managing gigabytes of RAM beyond Java Heap.
  • Fast and compact serialization mechanism.
  • RPC client/server technology which is an order of magnitude faster than Java RMI.

More info on the wiki page

https://github.com/odnoklassniki/one-nio/wiki

Comments
  • Register fields types for loading type

    Register fields types for loading type

    For more convinient to use the serial package will be nise when library generate code for some type to generate code for all types of fields of this class.

    Example: I have class

    class MyDto {
            private BigDecimal decimal;
        }
    

    and 2 JVMs between which will be data exchange. (First - serialize, second - deserialize)

    Even if I preregister this class on both JVM like

    Repository.preload(MyDto.class);
    

    second jvm will not be able to deserialize a data because there is no a serializer for BigDecimal. I know that we can request a schema but it's no needed here we have one already.

    opened by QIvan 15
  • HttpClient extra whitespace break Transfer-Encoding chunked header

    HttpClient extra whitespace break Transfer-Encoding chunked header

    As in the title. We noticed that if the server returns some extra whitespace for transfer-encoding header this condition isn't met.

      if (method != Request.METHOD_HEAD && mayHaveBody(response.getStatus())) {
                    if ("chunked".equalsIgnoreCase(response.getHeader("Transfer-Encoding: "))) {
                        response.setBody(readChunkedBody());
                    } else { 
    
    opened by avrecko 7
  • Emit checkcast to avoid crash on fastdebug vm

    Emit checkcast to avoid crash on fastdebug vm

    At first I must say that there was an unrelated bug I inspected in VM but also found this one. If you run this minimal example on fastdebug vm (any vm will go, not just jdk8u232) you will get a crash like this. As far as I understand this has no impact on release VM and will never happen there. However I suppose this is important to fix to reduce amount of VM crashes you get when you debug it.

    opened by genuss 7
  • undefined symbol: OPENSSL_init_ssl

    undefined symbol: OPENSSL_init_ssl

    Hi! I built my app with one-nio 1.2.0 and it doesn't start. In logs I can see: /usr/bin/java: symbol lookup error: /tmp/libonenio.97307c0a.so: undefined symbol: OPENSSL_init_ssl Both servers: where I built and where I tryed to start, have openssl version 1.0.2k-fips What I should do?

    opened by asurovenko 5
  • Enhanced http response with native buffer support and completion listener

    Enhanced http response with native buffer support and completion listener

    Basically what I am missing from the library:

    • Put ByteBuffer or native memory body into the http response
    • Have a completion listener

    Let me know if you would do anything differently and I'll change it. Don't want to maintain my own fork of this. Thank you for making this library public. It is a charm to work with it and easy to extend.

    I refined the code some more.

    The only breaking change is this:

    • QueueItem#release takes Throwable to signal problems. No-arg release method removed. It would be to much confusion which gets called. I think it is nice to have this. But don't mind if we don't. I only care to know if written == totalCount meaning everything is sent.
    opened by avrecko 5
  • Serialization of java.time.* classes doesn't work out of the box

    Serialization of java.time.* classes doesn't work out of the box

    At the moment the read method of for example LocalDate is generated to something like this

    public final Object read(DataStream var1) throws IOException, ClassNotFoundException {
        LocalDate var2;
        var1.register(var2 = new LocalDate);
        JavaInternals.unsafe.putInt(var2, 12L, var1.readInt());
        JavaInternals.unsafe.putShort(var2, 16L, var1.readShort());
        JavaInternals.unsafe.putShort(var2, 18L, var1.readShort());
        var2.readObject(NullObjectInputStream.INSTANCE);
        return var2;
    }
    

    But the LocalDate#readObject will throw InvalidObjectException because it uses serialization proxy java.time.Ser. The simple solution is to exclude calling of this method from generated code like this:

    Repository.setOptions(LocalDate.class, Repository.SKIP_READ_OBJECT);
    // ... same for all java.time.* classes
    

    However it could be made in a general way if we check for presence of writeReplace method and exclude readObject automatically. Do you have plans for this? I suppose I can make a PR, it looks like a simple condition in DelegateGenerator#generateRead.

    opened by genuss 5
  • Возможность давать объектам свои имена для сериализации в JSON

    Возможность давать объектам свои имена для сериализации в JSON

    При использовании библиотеки наткнулся на небольшую проблему. Нельзя было задать обьектам свои имена. Теперь проблему решил добавлением аннотации, теперь не нужно делать названия переменных в виде - "error_text" и т.п.

    opened by mmityushkin 5
  • java.lang.IllegalAccessError: java/util/concurrent/locks/ReentrantLock$Sync

    java.lang.IllegalAccessError: java/util/concurrent/locks/ReentrantLock$Sync

    Привет!

    Получаю ошибку при попытке десериализовать класс у которого есть поле типа ReentrantLock. Следующий код

    DeserializeStream stream = new DeserializeStream( bytes );
    return ( T )stream.readObject();
    
    

    выбрасывает exception java.lang.IllegalAccessError: java/util/concurrent/locks/ReentrantLock$Sync

    opened by renatn 4
  • Minor HttpClient improvement. Mutable bufferSize.

    Minor HttpClient improvement. Mutable bufferSize.

    Hi! Please tell us more about BUFFER_SIZE=8000 meaning. What if response bigger than 8000 bytes? If I'm sure that almost all my responses have less than 6000 bytes, does it mean that I should set BUFFER_SIZE=6000?

    opened by asurovenko 4
  • Little get started for serializer

    Little get started for serializer

    Is there any tutorial 'hot to use serializer'?

    I have serveral questions:

    1. Did I right understand that a work do through
    • one.nio.serial.Serializer#serialize
    • one.nio.serial.Serializer#deserialize Is it the best (for perfomance) way?
    1. What is the difference between persist and serialize?
    2. You told that there is a feature to request a schema. I didn't find it =(
    3. Can I add a custom Serializer for my type?

    Thanks!

    opened by QIvan 4
  • Socket multicast support

    Socket multicast support

    I find the Socket is missing join/leave multicast support. Any plans to add it? Why was this left out? Would it be fine if this gets included?

    I made a quick try without native support here 65bc2aa8e577f9a06a195ebfa8cd867c1bbf744e. Would the API changed be acceptable?

    opened by avrecko 3
  • Enhancements for Json support.

    Enhancements for Json support.

    • on linux unicode problems without + 10 (JsonReader#readHexChar)
    • call no-arg constructor if annotated with @Before (needed for e.g. field init)
    • if long value outside javascript safe integer range write it as String
    • support reading long field as String
    • support reading boolean filed as String
    • support reading String field as boolean or number
    opened by avrecko 1
  • BigDecimal with new Java does not work any more

    BigDecimal with new Java does not work any more

    Deserialization BigDecimal with Java 1.8.0_351 does not work any more:

    java.io.IOException: readObject() is not fully supported. See implementation notes.
    	at one.nio.serial.gen.NullObjectInputStream.unsupported(NullObjectInputStream.java:170)
    	at one.nio.serial.gen.NullObjectInputStream.readFields(NullObjectInputStream.java:50)
    	at java.math.BigDecimal.readObject(BigDecimal.java:3798)
    	at sun.reflect.Delegate14_BigDecimal.read(Unknown Source)
    	at one.nio.serial.GeneratedSerializer.read(GeneratedSerializer.java:127)
    	at one.nio.serial.DeserializeStream.readObject(DeserializeStream.java:75)
    ...
    
    opened by MacX19 1
  • BigInteger problem

    BigInteger problem

    Serialization BigInteger with different versions of Java produces different result

        byte[] res;
        try (PersistStream stream = new PersistStream()) {
          stream.writeObject(new BigInteger("123"));
          res = stream.toByteArray();
        }
        System.out.println(res.length + " bytes, hash=" + Arrays.hashCode(res));
    

    Java 8 output: 141 bytes, hash=-1637736666 Java 11 and 17 output: 169 bytes, hash=1514769574

    If BigInteger serialized with Java 8 and deserialized with Java 11

        try (DeserializeStream des = new DeserializeStream(res)) {
          System.out.println(des.readObject());
        }
    

    Output:

    Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.bitCount
    Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.bitLength
    Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.lowestSetBit
    Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.firstNonzeroIntNum
    Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.bitCountPlusOne
    Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.bitLengthPlusOne
    Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.lowestSetBitPlusTwo
    Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.firstNonzeroIntNumPlusTwo
    123
    

    If vice versa Output:

    Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.bitCountPlusOne
    Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.bitLengthPlusOne
    Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.lowestSetBitPlusTwo
    Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.firstNonzeroIntNumPlusTwo
    Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.bitCount
    Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.bitLength
    Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.lowestSetBit
    Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
    WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.firstNonzeroIntNum
    123
    
    opened by MacX19 0
  • How to implement a request filter in HttpServer

    How to implement a request filter in HttpServer

    I want to add a user login check handler in my application when using the HttpServer, but I can't find a graceful way to do this in one.nio.http package.

    opened by forchid 4
Releases(1.6.0)
  • 1.6.0(Nov 27, 2022)

    • java.time serialization
    • Arena Allocator
    • PaddedRWLock to avoid false sharing
    • Handle EINTR in socket read/write
    • Enum serialization fixes
    • Fixed global perf counters in presence of offline CPUs
    • HttpClient should support responses without Content-Length / Transfer-Encoding
    • Gracefully close sessions when stopping Server
    Source code(tar.gz)
    Source code(zip)
    one-nio-1.6.0.jar(512.18 KB)
  • 1.5.0(Feb 18, 2022)

    • JDK 17 support. Workaround setAccessible restrictions
    • New BPF map types
    • SOCK_SEQPACKET support
    • sendMsg/recvMsg for passing descriptors over a UNIX domain socket
    • Serializer generator now tolerates missing class files
    • Fixed serialization of private Records
    • Optimize Utf8.read for Compact Strings
    • Customizable strategy for reusing OffheapMap entries
    • Allow custom reference tags in serialization streams
    • Fixed overflow bugs in ObjectInput(Output)Channel
    • A method to open all JDK modules
    • Json.fromJson with a given Class or Type
    • A few new methods in HTTP API
    • Migrated to ASM 9
    • Reduced size of libonenio binary
    Source code(tar.gz)
    Source code(zip)
    one-nio-1.5.0.jar(510.84 KB)
  • 1.4.0(Jul 26, 2021)

    • eBPF and extended perf_events support
    • HttpClient fixes
    • Reflection and serialization now works for classes with missing field types
    • Improved serialization performance
    • OffheapMap API enhancements; compatibility with newer JDKs
    • Configure scheduling policy for thread pools
    • Fixed byte[] <-> long conversion
    Source code(tar.gz)
    Source code(zip)
    one-nio-1.4.0.jar(661.42 KB)
  • 1.3.1(Sep 7, 2020)

  • 1.3.0(Aug 16, 2020)

    • SSL improvements: RDRAND, PEER_CERTIFICATE_CHAIN, autoupdate certificates
    • Extended Perf Events API, including PEBS
    • Garbage free thread safe date utilities; high performance date formatter
    • Serialization: collection type evolution; better JSON support
    • PROXY protocol support for RPC server
    • Configurable DigestStream
    • Thread.onSpinWait bridge
    • Custom collection converters for ConfigParser
    • HttpClient fixes
    • Utility class to invoke HotSpot diagnostic commands
    • CustomThreadFactory for creating named/daemon/batch threads
    • RpcStream memory optimizations
    • setAffinity/getAffinity for machines with more than 64 CPUs
    • AF_UNIX socket support. Refactored InetAddress handling
    • systemd-notify API
    Source code(tar.gz)
    Source code(zip)
    one-nio-1.3.0.jar(603.11 KB)
  • 1.2.0(Sep 4, 2019)

    • Updated SSL API. Client certificate inspection support
    • SerializedWrapper for transfering pre-serialized objects
    • Http server returns "400 Bad Request" if fails to parse parameters
    • Getters for all socket options
    • Synchronous Server.start
    • Java wrappers for setpriority/getpriority (thread niceness)
    • HttpClient invoke with timeout
    • Support for connected UDP Java socket
    • Java API for perf events on Linux
    • Serialization bugfixes
    • ConnectionString recognizes well-known ports
    • ConfigParser enhancements
    • Fixed compatibility issues with JDK 11. Get rid of tools.jar dependency
    Source code(tar.gz)
    Source code(zip)
    one-nio-1.2.0.jar(468.15 KB)
  • 1.1.0(Jan 9, 2019)

    • ConfigParser better handles scalars, collections and generics
    • Malloc compatibility with older format
    • SerializeWith(getter, setter) and SerialOptions annotations
    • SSL key passphrase and SOL_SSL socket options
    • Socket.read() with flags
    • Fixed EINTR during connect
    • Fixed readFully/writeFully on JavaSocket
    • RpcClient invoke with timeout
    • Server threads now extend PayloadThread
    • JDK 9,10,11 support
    • Iterator over all nonempty HTTP Request query parameters
    • Return query parameters as an iterable
    • Handle Throwable instead of Exception in AsyncExecutor
    • YAML parser fixes
    • Java 8 minimum requirement
    • HTTP and SOCKS proxy support
    • Bridge to setns() syscall
    • Batch/Idle thread scheduling priorities
    • FIFO/LIFO pools
    • Socket I/O improvements: ByteChannel API, accept4() support
    • JSON serialization/deserialization
    • HTTP @RequestMethod annotation
    • Object Streaming API
    • RPC streaming and HTTP-RPC support
    Source code(tar.gz)
    Source code(zip)
Owner
OK.ru
Most famous Russian social network
OK.ru
A Java library that implements a ByteChannel interface over SSLEngine, enabling easy-to-use (socket-like) TLS for Java applications.

TLS Channel TLS Channel is a library that implements a ByteChannel interface over a TLS (Transport Layer Security) connection. It delegates all crypto

Mariano Barrios 149 Dec 31, 2022
TCP/UDP client/server library for Java, based on Kryo

KryoNet can be downloaded on the releases page. Please use the KryoNet discussion group for support. Overview KryoNet is a Java library that provides

Esoteric Software 1.7k Jan 2, 2023
An annotation-based Java library for creating Thrift serializable types and services.

Drift Drift is an easy-to-use, annotation-based Java library for creating Thrift clients and serializable types. The client library is similar to JAX-

null 225 Dec 24, 2022
Java library for representing, parsing and encoding URNs as in RFC2141 and RFC8141

urnlib Java library for representing, parsing and encoding URNs as specified in RFC 2141 and RFC 8141. The initial URN RFC 2141 of May 1997 was supers

SLUB 24 May 10, 2022
A Java library for capturing, crafting, and sending packets.

Japanese Logos Pcap4J Pcap4J is a Java library for capturing, crafting and sending packets. Pcap4J wraps a native packet capture library (libpcap, Win

Kaito Yamada 1k Dec 30, 2022
Full-featured Socket.IO Client Library for Java, which is compatible with Socket.IO v1.0 and later.

Socket.IO-client Java This is the Socket.IO Client Library for Java, which is simply ported from the JavaScript client. See also: Android chat demo en

Socket.IO 5k Jan 4, 2023
Asynchronous Http and WebSocket Client library for Java

Async Http Client Follow @AsyncHttpClient on Twitter. The AsyncHttpClient (AHC) library allows Java applications to easily execute HTTP requests and a

AsyncHttpClient 6k Dec 31, 2022
Telegram API Client and Telegram BOT API Library and Framework in Pure java.

Javagram Telegram API Client and Telegram Bot API library and framework in pure Java. Hello Telegram You can use Javagram for both Telegram API Client

Java For Everything 3 Oct 17, 2021
🧚‍♀️ Java library to interact with YouTrack's REST API.

YouTrack API for Java ??‍ Java library to interact with YouTrack's REST API.

Noel 2 Oct 1, 2021
An netty based asynchronous socket library for benchion java applications

Benchion Sockets Library An netty based asynchronous socket library for benchion java applications ?? Documents ?? Report Bug · Request Feature Conten

Fitchle 3 Dec 25, 2022
A High Performance Network ( TCP/IP ) Library

Chronicle-Network About A High Performance Network library Purpose This library is designed to be lower latency and support higher throughputs by empl

Chronicle Software : Open Source 231 Dec 31, 2022
Simple & Lightweight Netty packet library + event system

Minimalistic Netty-Packet library Create packets with ease Bind events to packets Example Packet: public class TestPacket extends Packet { privat

Pierre Maurice Schwang 17 Dec 7, 2022
A Minecraft library for working with minecraft packets on various platforms, using MCProtocolLib

BetterProtocol A Minecraft library for working with minecraft packets on various platforms, using MCProtocolLib This library is still based on the pro

John 8 Jul 2, 2022
Library for composability of interdependent non-blocking I/O tasks

Composer Composer helps you to organize and execute multiple interdependent asynchronous input/output tasks such as webservice calls, database read/wr

krupal 19 Oct 8, 2021
This is library that look like Scarlet Wrapper Socket.io

This is library that look like Scarlet Wrapper Socket.io

Adkhambek 8 Jan 2, 2023
Socket.IO server implemented on Java. Realtime java framework

Netty-socketio Overview This project is an open-source Java implementation of Socket.IO server. Based on Netty server framework. Checkout Demo project

Nikita Koksharov 6k Dec 30, 2022
Apache Dubbo is a high-performance, java based, open source RPC framework.

Apache Dubbo Project Apache Dubbo is a high-performance, Java-based open-source RPC framework. Please visit official site for quick start and document

The Apache Software Foundation 38.2k Dec 31, 2022
The Java gRPC implementation. HTTP/2 based RPC

gRPC-Java - An RPC library and framework gRPC-Java works with JDK 7. gRPC-Java clients are supported on Android API levels 16 and up (Jelly Bean and l

grpc 10.2k Jan 1, 2023
ssh, scp and sftp for java

sshj - SSHv2 library for Java To get started, have a look at one of the examples. Hopefully you will find the API pleasant to work with :) Getting SSH

Jeroen van Erp 2.2k Jan 8, 2023