Microhttp - a fast, scalable, event-driven, self-contained Java web server

Related tags

Networking microhttp
Overview

Microhttp

Microhttp is a fast, scalable, event-driven, self-contained Java web server that is small enough for a programmer to understand and reason about.

Comprehensibility is the highest priority. This library is intended to be an alternative to commonly used frameworks with overwhelming complexity. Implementation decisions aim to strike a balance between simplicity and efficiency.

Microhttp discretizes all requests and responses. Streaming is not supported. This aligns well with transactional web services that exchange small payloads. Limiting request body size has the added benefit of overflow protection. This is frequently overlooked in web services that consume request bodies in a stream-oriented fashion.

TLS is not supported. Edge proxies and load balancers provide this capability. The last hop to Microhttp typically does not require TLS.

HTTP 2 is not supported for a similar reason. Edge proxies can support HTTP 2 while using HTTP 1.1 on the last hop to Microhttp.

Principles:

  • No dependencies
  • Small, targeted codebase (~500 LOC)
  • Highly concurrent
  • Single threaded
  • Event-driven non-blocking NIO
  • No TLS support
  • No streaming support
  • Traceability via log events

Includes:

  • HTTP 1.0 and 1.1
  • Chunked transfer encoding
  • Persistent connections
  • Pipelining

Excludes:

  • HTTP 2
  • Range requests
  • Caching
  • Compression

Dependency

Microhttp is available in the Maven Central repository with group org.microhttp and artifact microhttp.

<dependency>
    <groupId>org.microhttpgroupId>
    <artifactId>microhttpartifactId>
    <version>0.4version>
dependency>

Getting Started

The snippet below represents a minimal starting point. Default options and debug logging.

The application consists of an event loop running in the main thread. There are no additional application threads.

Responses are handled immediately in the Handler.handle method.

callback.accept(response); EventLoop eventLoop = new EventLoop(handler); eventLoop.start();">
Response response = new Response(
        200,
        "OK",
        List.of(new Header("Content-Type", "text/plain")),
        "hello world\n".getBytes());
Handler handler = (req, callback) -> callback.accept(response);
EventLoop eventLoop = new EventLoop(handler);
eventLoop.start();

The following example demonstrates the full range of configuration options.

callback.accept(response); EventLoop eventLoop = new EventLoop(options, logger, handler); eventLoop.start();">
Response response = new Response(
        200,
        "OK",
        List.of(new Header("Content-Type", "text/plain")),
        "hello world\n".getBytes());
Options options = new Options()
        .withHost("localhost")
        .withPort(8080)
        .withSocketTimeout(Duration.ofSeconds(60))
        .withResolution(Duration.ofMillis(100))
        .withReadBufferSize(1_024 * 64)
        .withMaxRequestSize(1_024 * 1_024)
        .withAcceptLength(0);
Logger logger = new DebugLogger();
Handler handler = (req, callback) -> callback.accept(response);
EventLoop eventLoop = new EventLoop(options, logger, handler);
eventLoop.start();

The example below demonstrates asynchronous request handling.

Responses are handled in a separate background thread after an artificial one-second delay.

executorService.schedule(() -> callback.accept(response), 1, TimeUnit.SECONDS); EventLoop eventLoop = new EventLoop(handler); eventLoop.start();">
Response response = new Response(
        200,
        "OK",
        List.of(new Header("Content-Type", "text/plain")),
        "hello world\n".getBytes());
ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
Handler handler = (req, callback) -> executorService.schedule(() -> callback.accept(response), 1, TimeUnit.SECONDS);
EventLoop eventLoop = new EventLoop(handler);
eventLoop.start();

This example demonstrates the use of a separate thread for the event loop.

executorService.schedule(() -> callback.accept(response), 1, TimeUnit.SECONDS); EventLoop eventLoop = new EventLoop(handler); Thread thread = new Thread(eventLoop::start); thread.start(); // ... eventLoop.stop(); thread.join();">
Response response = new Response(
        200,
        "OK",
        List.of(new Header("Content-Type", "text/plain")),
        "hello world\n".getBytes());
ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
Handler handler = (req, callback) -> executorService.schedule(() -> callback.accept(response), 1, TimeUnit.SECONDS);
EventLoop eventLoop = new EventLoop(handler);
Thread thread = new Thread(eventLoop::start);
thread.start();
// ...
eventLoop.stop();
thread.join();

Benchmarks

The experiments detailed below were conducted on a pair of EC2 instances in AWS, one running the server and another running the client.

  • Region: us-west-2
  • Instance type: c5.2xlarge compute optimized instance 8 vCPU and 16 GB of memory
  • OS: Amazon Linux 2 with Linux Kernel 5.10, AMI ami-00f7e5c52c0f43726
  • OpenJDK 17.0.2 from https://jdk.java.net/17/

In order to facilitate the rapid creation of 50,000 connections, the following sysctl kernel parameter changes were committed on both hosts prior to the start of the experiment:

sysctl net.ipv4.ip_local_port_range="2000 64000"
sysctl net.ipv4.tcp_fin_timeout=30
sysctl net.core.somaxconn=8192
sysctl net.core.netdev_max_backlog=8000
sysctl net.ipv4.tcp_max_syn_backlog=8192

Throughput

The goal of throughput benchmarks is to gauge the maximum request-per-second rate that can be supported by Microhttp. These experiments are intended to surface the costs and limitations of Microhttp alone. They are not intended to provide a real world estimate of throughput in an integrated system with many components and dependencies.

Server

ThroughputServer.java was used for throughput tests.

It simply returns "hello world" in a tiny, plain-text response to every request. Requests are handled in the context of the main application thread, directly within the Handler.handle method.

Apache Bench

The first throughput test was conducted with Apache Bench.

A throughput of 100,000 requests per second was easily reproducible.

[ec2-user@ip-10-39-196-99 ~]$ ab -k -c 100 -n 1000000 http://10.39.196.164:8080/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.39.196.164 (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
Completed 1000000 requests
Finished 1000000 requests


Server Software:        
Server Hostname:        10.39.196.164
Server Port:            8080

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      100
Time taken for tests:   9.964 seconds
Complete requests:      1000000
Failed requests:        0
Keep-Alive requests:    1000000
Total transferred:      101000000 bytes
HTML transferred:       12000000 bytes
Requests per second:    100364.03 [#/sec] (mean)
Time per request:       0.996 [ms] (mean)
Time per request:       0.010 [ms] (mean, across all concurrent requests)
Transfer rate:          9899.19 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       1
Processing:     1    1   0.0      1       3
Waiting:        0    1   0.0      1       3
Total:          1    1   0.0      1       3

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      3 (longest request)

NIO Client

A second throughput client, Client.java, was implemented with Java NIO and tailored specifically for Microhttp throughput testing.

Again, 100,000+ requests per second was readily available.

[ec2-user@ip-10-39-196-99 ~]$ ./jdk-17.0.2/bin/java -cp microhttp-0.1-SNAPSHOT.jar test.Client 10.39.196.164 8080 100 30000
Args[host=10.39.196.164, port=8080, numConnections=100, duration=30000]
barrier opened!
duration: 30001 ms, messages: 3250567, throughput: 108348.621713 msg/sec

Concurrency

The goal of concurrency benchmarks is to gauge the number of concurrent connections and clients that can be supported by Microhttp.

Server

ConcurrencyServer.java was used for concurrency tests.

"hello world" responses are handled in a separate background thread after an injected one-second delay. The one-second delay dramatically reduces the resource footprint since requests and responses aren't speeding over each connection continuously. This leaves room to scale up connections, which is the metric of interest.

Apache Bench

Apache Bench only supports a maximum of 20,000 concurrency connections.

Microhttp holds up well.

[ec2-user@ip-10-39-196-99 ~]$ ab -k -c 20000 -n 100000 http://10.39.196.164:8080/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.39.196.164 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        10.39.196.164
Server Port:            8080

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      20000
Time taken for tests:   6.825 seconds
Complete requests:      100000
Failed requests:        0
Keep-Alive requests:    100000
Total transferred:      10100000 bytes
HTML transferred:       1200000 bytes
Requests per second:    14652.72 [#/sec] (mean)
Time per request:       1364.934 [ms] (mean)
Time per request:       0.068 [ms] (mean, across all concurrent requests)
Transfer rate:          1445.24 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   42  84.6      0     266
Processing:  1000 1048  54.2   1013    1256
Waiting:     1000 1048  54.2   1013    1174
Total:       1000 1090 117.3   1022    1349

Percentage of the requests served within a certain time (ms)
  50%   1022
  66%   1091
  75%   1137
  80%   1256
  90%   1313
  95%   1322
  98%   1328
  99%   1332
 100%   1349 (longest request)

NIO Client

A concurrency level of 50,000 connections was achievable using Client.java.

[ec2-user@ip-10-39-196-99 ~]$ ./jdk-17.0.2/bin/java -cp microhttp-0.1-SNAPSHOT.jar test.Client 10.39.196.164 8080 50000 60000
Args[host=10.39.196.164, port=8080, numConnections=50000, duration=60000]
barrier opened!
duration: 61001 ms, messages: 2968864, throughput: 48669.103785 msg/sec
Comments
  • Refactor Options to be immutable using a builder

    Refactor Options to be immutable using a builder

    Why this change is required? Currently Options.java is effectively mutable and can be misused in the following way The EventLoop constructor uses the options by reference. So while an options is passed to this constructor, it could also be updated later with any of the with methods such as withHost resulting in unpredictable behavior.

    How does this change solves the above problem? This change aims to make the Options immutable using a builder OptionsBuilder. All the fields of Options are marked as final and an instance is only created after invoking the build method on the builder. Post this, none of the fields in Options are mutable. This prevents the code from ending up in any of the unpredictable scenarios where an option is updated post being passed to the event loop.

    A getDefaultInstance method is also added for use cases where we don't want to set any of the properties of Options and just use the default properties.

    opened by varunu28 13
  • Discussion point - any features of jdk.httpserver desired?  managing idle connections perhaps?

    Discussion point - any features of jdk.httpserver desired? managing idle connections perhaps?

    I presume that we are aware of jdk.httpserver so I am having a quick look at the features of jdk.httpserver and just opening this as a discussion point to see / confirm if there are any features in there that might be considered desirable for microhttp.

    Glancing at the jdk.httpserver code and the thing that popped out was the management of idle connections. It has a background timertask to manage idle connections (plus an optional one to manage request exceeding MAX_REQ_TIME and MAX_RSP_TIME).

    Are there any features in jdk.httpserver that are desired to be added to microhttp?

    opened by rbygrave 5
  • Discussion - style option rather than using new EventLoop directly/explicitly for usual expected use?

    Discussion - style option rather than using new EventLoop directly/explicitly for usual expected use?

    So instead of having this style ...

        public static void main(String[] args) throws IOException, InterruptedException {
    
            Options options = new Options()
                    .withPort(8080)
                    .withSocketTimeout(Duration.ofSeconds(60))
                    .withReadBufferSize(1_024 * 64)
                    .withMaxRequestSize(1_024 * 1_024);
    
      
            Handler handler = ...;
    
            EventLoop eventLoop = new EventLoop(options, new DisabledLogger(), handler);
            Thread eventLoopThread = new Thread(() -> {
                try {
                    eventLoop.start();
                } catch (IOException e) {
                    // thread terminates
                }
            });
            eventLoopThread.start();
    
           ...
            
            eventLoopThread.join();
        }
    
    

    Have a MicroHttp.Builder where we with the optional config options, pass the mandatory params to a start() ... under the hood create the event loop etc.

    Return a MicroHttp with a stop() method?

        public static void main(String[] args) throws IOException, InterruptedException {
    
            Handler handler = ...;
    
            var microHttp = new MicroHttp.newBuilder()
                    .withPort(8080)
                    .withSocketTimeout(Duration.ofSeconds(60))
                    .withReadBufferSize(1_024 * 64)
                    .withMaxRequestSize(1_024 * 1_024)
    
                   .withLogger(...)
    
                   .start(handler);  // creates event loop, starts
      
    
           microHttp.stop();
    
        }
    
    

    Just using "defaults" would be like:

    public static void main(String[] args) {
    
      Handler handler = ...;
    
      var microHttp = new MicroHttp
          .newBuilder()
          .start(handler);
      
    
        // programmatically stop ..
        microHttp.stop();
    
    }
    
    

    The thinking would be to make the "usual use" a little bit more obvious (in theory).

    opened by rbygrave 3
  • Create tagged GitHub releases

    Create tagged GitHub releases

    There are currently two releases of Microhttp on Maven Central:

    https://search.maven.org/artifact/org.microhttp/microhttp

    v0.1 and v0.2 appear to have no corresponding Git tags:

    https://github.com/ebarlas/microhttp/tags

    There are also no GitHub releases:

    https://github.com/ebarlas/microhttp/releases

    Here are some release examples from a personal project:

    https://github.com/beatngu13/pdf-zoom-wizard/releases

    PR #6 will add a basic GitHub Actions build. Based on this, another workflow could be created to automatically set the project's version, tag the commit, and create a GitHub release. It is also possible to use something like JReleaser.

    What do you think? I'm happy to help if you are interested.

    opened by beatngu13 3
  • Change source baseline to be Java 8

    Change source baseline to be Java 8

    Change the source baseline to be Java 8.

    The IntelliJ support for using different source versions in source and test is still not available in their latest 2021.3 release. However, there is an open issue IDEA-85478, which seems to have gained some traction and there is something working in the 2022.1 EAP (I tested it and it works good).

    Almost all changes are due changing the convenience of the usage. However, the change in the ByteTokenizer might affect the performance, a multi release jar could be used here to in order to not lose any performance on Java 9+.

    opened by filiphr 3
  • Address warnings

    Address warnings

    This PR addresses some (IMHO justified) IntelliJ IDEA warnings. Feel free to throw away specific commits if you don't the like the corresponding changes.

    Side question: Is there a reason why you use CRLF instead of LF in Response and RequestParser?

    opened by beatngu13 2
  • 'SO_REUSEPORT' not supported

    'SO_REUSEPORT' not supported

    Exception in thread "main" java.lang.UnsupportedOperationException: 'SO_REUSEPORT' not supported at java.base/sun.nio.ch.ServerSocketChannelImpl.setOption(ServerSocketChannelImpl.java:212) at org.microhttp.EventLoop.(EventLoop.java:80)

    opened by brucemelo 1
  • Add simple concurrency to Microhttp

    Add simple concurrency to Microhttp

    Split event loop into (1) single passive-server-socket event loop and (2) many active-socket events loops.

    Each event loop is single-threaded and independent. Still no locking or synchronization.

    This design is facilitated by Selector, which permits select on one thread and register on another.

    opened by ebarlas 0
  • Discussion point - Buffer recycling on readBuffer but not writeBuffer (maybe)

    Discussion point - Buffer recycling on readBuffer but not writeBuffer (maybe)

    I could be wrong or low on caffeine but it looks to me like there is effective buffer recycling with readBuffer but not with writeBuffer at the moment? Am I reading that incorrectly or maybe it doesn't matter?

    That is, it looks like with writeBuffer there is a decent amount of copying in Response.merge() and then wrap(). I get a sense we could avoid the byte[] result = new byte[size]; and look to recycle writeBuffer instead?

    opened by rbygrave 14
  • Demo using todobackend

    Demo using todobackend

    https://todobackend.com/ is a an initiative to show that a bunch of REST services can pass a known frontend test suite.

    That test suite is a Jasmine suite and runs in the browser. There' a series of requests performed in order. GET, POST and others. It is very cool. Can microhttp stand in the middle and delegate requests to one of the other backends, and still pass the test suite.

    Or maybe a native TodoBackend using microhttp. Easiest to fork one of the existing Java ones perhaps - and there's no DB requirement - a HashMap would do.

    opened by paul-hammant 2
Releases(v0.9)
  • v0.9(Nov 18, 2022)

    • Include org.microhttp module-info
    • Fix bug related to improper handling of response ByteBuffer when it was not fully flushed to SocketChannel on write
    • Use System.nanoTime() in DebugLogger to determine uptime rather than RuntimeMXBean
    Source code(tar.gz)
    Source code(zip)
  • v0.8(Jul 12, 2022)

    • Introduce parallel processing by separating singular event loop into single passive socket parent event loop and multiple active socket children event loops
    • Introduce concurrency configuration option
    • Use direct off-heap buffer for both reading and writing
    • Rename associated configuration option from readBufferSize to bufferSize
    Source code(tar.gz)
    Source code(zip)
  • v0.7(Apr 12, 2022)

    • Replace socket timeout with request timeout
    • Initiate write immediately rather than waiting for write-ready event
    • Schedule deferred write processing tasks on dedicated queue rather than on scheduler
    Source code(tar.gz)
    Source code(zip)
  • v0.6(Mar 18, 2022)

  • v0.5(Mar 18, 2022)

  • v0.4(Feb 21, 2022)

    • Minor profile-guided performance optimizations in EventLoop and RequestParser
    • Emit log event when event loop exits due to exception rather than propagating
    Source code(tar.gz)
    Source code(zip)
  • v0.3(Feb 17, 2022)

  • v0.2(Feb 17, 2022)

  • v0.1(Feb 17, 2022)

Owner
Elliot Barlas
Elliot Barlas
A Java event based WebSocket and HTTP server

Webbit - A Java event based WebSocket and HTTP server Getting it Prebuilt JARs are available from the central Maven repository or the Sonatype Maven r

null 808 Dec 23, 2022
Simple & Lightweight Netty packet library + event system

Minimalistic Netty-Packet library Create packets with ease Bind events to packets Example Packet: public class TestPacket extends Packet { privat

Pierre Maurice Schwang 17 Dec 7, 2022
A simple Discord bot, which shows the server status of the Lost Ark server Beatrice

Beatrice A simple Discord bot, which shows the server status of the Lost Ark server Beatrice. Example Usage Clone the repository. Edit the property fi

Leon 3 Mar 9, 2022
A simple, fast integration of WebSocket spring-stater

websocket-spring-boot-starter readme 介绍 一个简单,快速,低配的spring-boot-starter,是对spring-boot-starter-websocket的扩展与二次封装,简化了springboot应用对websocket的操作 特点 简单,低配 支

Jack 4 Dec 24, 2021
Experimental Netty-based Java 16 application/web framework

Experimental Netty-based application/web framework. An example application can be seen here. Should I use this? Probably not! It's still incredibly ea

amy null 8 Feb 17, 2022
Unconventional Java code for building web servers / services without a framework.

Unconventional Java code for building web servers / services without a framework. Think dropwizard but as a seed project instead of a framework. If this project had a theme it would be break the rules but be mindful of your decisions.

StubbornJava 227 Nov 15, 2022
Socket.IO server implemented on Java. Realtime java framework

Netty-socketio Overview This project is an open-source Java implementation of Socket.IO server. Based on Netty server framework. Checkout Demo project

Nikita Koksharov 6k Dec 30, 2022
Fibers and actors for web development

COMSAT Scalable, Concurrent Web Apps Getting started Add the following Maven/Gradle dependencies: Feature Artifact Servlet integration for defining fi

Parallel Universe 600 Dec 23, 2022
SCG used as as proxy to connect gRPC-Web and back end gRPC services

gRPC-Web Spring Cloud Gateway Spring Cloud Gateway 3.1.1 supports for gRPC and HTTP/2. It is possible to use Spring Cloud Gateway to connect gRPC-Web

null 1 Apr 4, 2022
TCP/UDP client/server library for Java, based on Kryo

KryoNet can be downloaded on the releases page. Please use the KryoNet discussion group for support. Overview KryoNet is a Java library that provides

Esoteric Software 1.7k Jan 2, 2023
A barebones WebSocket client and server implementation written in 100% Java.

Java WebSockets This repository contains a barebones WebSocket server and client implementation written in 100% Java. The underlying classes are imple

Nathan Rajlich 9.5k Dec 30, 2022
HTTP Server Model made in java

SimplyJServer HTTP Server Model made in java Features Fast : SimplyJServer is 40%-60% faster than Apache, due to it's simplicity. Simple to implement

Praudyogikee for Advanced Technology 2 Sep 25, 2021
A small java project consisting of Client and Server, that communicate via TCP/UDP protocols.

Ninja Battle A small java project consisting of Client and Server, that communicate via TCP/UDP protocols. Client The client is equipped with a menu i

Steliyan Dobrev 2 Jan 14, 2022
FileServer - A multithreaded client-server program that uses Java Sockets to establish TCP/IP connection

A multithreaded client-server program that uses Java Sockets to establish TCP/IP connection. The server allows multiple clients to upload, retrieve and delete files on/from the server.

Lokesh Bisht 3 Nov 13, 2022
BAIN Social is a Fully Decentralized Server/client system that utilizes Concepts pioneered by I2P, ToR, and PGP to create a system which bypasses singular hosts for data while keeping that data secure.

SYNOPSIS ---------------------------------------------------------------------------------------------------- Welcome to B.A.I.N - Barren's A.I. Natio

Barren A.I. Wolfsbane 14 Jan 11, 2022
Book Finder application is a client-server application (gRPC) for educational purposes.

Book-Finder Book Finder application is a client-server application (gRPC) for educational purposes. Instalation These projects (Client/Server) are Mav

Mihai-Lucian Rîtan 21 Oct 27, 2022
Realtime Client Server Framework for the JVM, supporting WebSockets with Cross-Browser Fallbacks

Welcome to Atmosphere: The Event Driven Framework supporting WebSocket and HTTP The Atmosphere Framework contains client and server side components fo

Atmosphere Framework 3.6k Jan 3, 2023
Check the connectivity of a server with this API.

ServerStatusAPI Presentation : This is a java API with which can test the conectivity of server (all server but you can also use for minecraft). The f

Gabriel MERCIER 1 Mar 16, 2022
WebSocket server with creatable/joinable channels.

bytesocks ?? bytesocks is a WebSocket server which allows clients to create "channels" and send messages in them. It's effectively an add-on for byteb

lucko 6 Nov 29, 2022