Event capture and querying framework for Java

Overview

Build Status Maven Central Download Join the chat at https://gitter.im/eventsourcing/eventsourcing

logo Eventsourcing for Java

Enabling plurality and evolution of domain models

Instead of mutating data in a database, Eventsourcing stores all changes (events) and what caused them (commands). To make this data useful, Eventsourcing builds indices over it.

This helps developing applications faster because there is no need to worry about designing the right domain models upfront (or as close to right as possible). By keeping all the commands and events, we can enrich or change our domain models over time with very little friction. Furthermore, this approach removes a need to have a one and only domain model for every entity. We experience the world and reality in different ways, depending on circumstances and points of view, and our programs should be able to reflect that.

To learn more about what kind of problems ES4J addresses, please read Why Use Eventsourcing Database

Lazy event sourcing: living in the now

Key benefits

  • Domain model flexibility
  • Late domain model binding
  • Persistence of causal information
  • Serializable conflict resolution
  • Audit trail logging
  • Mapping application functionality

Key features

  • Strongly typed schemas
  • Event migrations
  • Domain protocols
  • Batteries included (shared event languages)
  • Basic support for Kotlin
  • Causality-preserving Hybrid Logical Clocks
  • In-memory and server (PostgreSQL) storage
  • Locking synchronization primitive
  • JMX-based introspection and management

Presentation

You can find our current slide deck at https://eventsourcing.com/presentation

Downloading and installing

To start using ES4J, please follow the installation instructions.

Documentation

Documentation can be found at es4j.eventsourcing.com

We strive to specify the building blocks behind Eventsourcing and its ecosystem as succinct specifications, you can find the current list of them at rfc.eventsourcing.com

Roadmap

As this project is striving to be a decentralized, contributors-driven project governed by the C4 process, there is no central roadmap per se. However, there's a centralized list of reported issues. These do not imply an actual roadmap, just what has been reported, ranging from bugs to longer-term design issues.

Contributing

Contributions of all kinds (code, documentation, testing, artwork, etc.) are highly encouraged. Please open a GitHub issue if you want to suggest an idea or ask a question.

We use Unprotocols C4 process. In a nutshell, this means:

  • We merge pull requests rapidly (try!)
  • We are open to diverse ideas
  • We prefer code now over consensus later

For more details, please refer to CONTRIBUTING

Related projects

  • es4j-graphql A Relay.js/GraphQL adaptor for ES4J-based applications.
Comments
  • Bintray: eventsourcing-inmem is missing

    Bintray: eventsourcing-inmem is missing

    I can't use the dependency "com.eventsourcing:eventsourcing-inmem:0.3.1" because it is missing on the Bintray Maven repository [1].

    Work-around: Deploy the dependency to a local maven repository.

    [1] http://dl.bintray.com/eventsourcing/maven/com/eventsourcing/

    opened by hastebrot 4
  • Implement schema mismatch reporter

    Implement schema mismatch reporter

    In any system the schema of commands and events will naturally evolve. While this is typically not a problem when using MemoryJournal (the data will not survive through restarts), it is an issue with persistent journals (MVStoreJournal and others).

    Every persistent journal will need to print out the discrepancies found between entities available in the journal and in runtime. Using layout's hash we can easily detect which entities in the journal are no longer covered by the code at hand.

    The report should look like this:

    Found 1 unrecognized entity:
    
        d395f099078879b005797edf7ef37f83a673bb3582056490655984d3d5b18cf8 (234 records)
    
    

    This reporter should not try to attempt facilitating schema migrations in any way, it should be just a reporter.

    It would be great if some meta information can be stored every time the journal is used. This way, the report can be much more readable:

    Found 1 unrecognized entity:
    
        com.foo.bar.events.UserLoggedIn (234 records) # d395f099078879b005797edf7ef37f83a673bb3582056490655984d3d5b18cf8
    
    

    Even better if the layout can be saved, too:

    Found 1 unrecognized entity:
    
        com.foo.bar.events.UserLoggedIn (234 records) # d395f099078879b005797edf7ef37f83a673bb3582056490655984d3d5b18cf8
         login: String
    
    
    opened by yrashk 3
  • Problem: reindexing large datasets is very time consuming

    Problem: reindexing large datasets is very time consuming

    Upon index initialization, index has no idea if the object store's data has been indexed.

    Solution: in PostgreSQL-backed journal, assign sequential IDs to entities and introduce the use of NotSeenBy query option to avoid reindexing

    opened by yrashk 2
  • Problem: postgresql navigable index over a timestamp has a composite key

    Problem: postgresql navigable index over a timestamp has a composite key

    (Since the timestamp is not really comparable as is, it needs to be transparently re-encoded to a comparable value in order to work)

    eventsourcing=# \d+ index_v1_9c22ea3e2733ba23f1e23c8577698827575a4457_navigable;
                       Table "public.index_v1_9c22ea3e2733ba23f1e23c8577698827575a4457_navigable"
     Column |                        Type                        | Modifiers | Storage  | Stats target | Description
    --------+----------------------------------------------------+-----------+----------+--------------+-------------
     key    | layout_v1_c47d416193cba63554e5a69ca701973bc3e44172 |           | extended |              |
     object | uuid                                               |           | plain    |              |
    Indexes:
        "index_v1_9c22ea3e2733ba23f1e23c8577698827575a4457_navigable_key" btree (key)
        "index_v1_9c22ea3e2733ba23f1e23c8577698827575a4457_navigable_obj" btree (object)
    
    eventsourcing=# \d+ layout_v1_c47d416193cba63554e5a69ca701973bc3e44172
    Composite type "public.layout_v1_c47d416193cba63554e5a69ca701973bc3e44172"
         Column     |  Type  | Modifiers | Storage | Description
    ----------------+--------+-----------+---------+-------------
     logicalCounter | bigint |           | plain   |
     logicalTime    | bigint |           | plain   |
    
    opened by yrashk 2
  • Problem: it is impossible to know when query results change

    Problem: it is impossible to know when query results change

    Once a query is executed, it is impossible to know when the data queried against has changed, leaving us to requery at some time intervals.

    Solution: implement a QuerySubscriber that would listen to all entities and notify its own subscribers that a particular query's set has changed.

    It is rather simplistic for now, and I am not quite sure it is suitable for all scenarios, but we can address this later, and if it doesn't work out, remove it. One of the areas for improvement is to implement query-and-subscribe.

    opened by yrashk 1
  • Proposal: use SonarQube for code analysis

    Proposal: use SonarQube for code analysis

    I would like to propose to use http://docs.sonarqube.org/display/HOME/SonarQube+Platform for code analysis. For IntelliJ - Sonar Lint http://www.sonarlint.org/intellij/. It can be done as part of CI pipeline and will improve quality of the code.

    opened by AlexMikhalev 1
  • Problem: Command#events can block the repository

    Problem: Command#events can block the repository

    If Command#events takes long time to complete (for example, making external API calls), the entire queue of commands will be blocked, because command processing is a single-thread process (originally, to avoid the complexities associated with multithread access to journal, indexing, etc.). This also holds true for massive event streams.

    Solution: remove Disruptor and use a worker pool

    Closes #145

    opened by yrashk 1
  • Problem: repository doesn't store the layout metainformation

    Problem: repository doesn't store the layout metainformation

    Solution: make repository generate EntityLayoutIntroduced on demand

    This change is fairly massive for a number of reasons. Besides adding generation of EntityLayoutIntroduced (RFC 8/EMT), the changes are:

    1. Repository implementation has been split into eventsourcing-repository module to allow it to have a non-circular dependency on eventsourcing-migrations
    2. RepositoryImpl has been renamed to StandardRepository for consistency and Repository.create() has been replaced with new StandardRepository()
    3. Fixed PostgreSQL connection leaks, closes #90
    4. Fixed PostgreSQL Optional serialization and deserialization
    5. Fixed composite type treatment with lists inside
    opened by yrashk 1
  • Problem: PostgreSQLJournal leaking connections

    Problem: PostgreSQLJournal leaking connections

    This is most likely happening due to https://github.com/npgall/cqengine/issues/74 — that said, we do have a workaround for this — if the iterator was actually finished, it should close.

    bug 
    opened by yrashk 1
  • Problem: Command#onCompletion method name is confusing

    Problem: Command#onCompletion method name is confusing

    It sounds like a callback, while in fact it is a function that takes a state from an EventStream and produces a "result"

    Solution: rename it to #result()

    Also, change the order of type parameters for Command from Result, State to State, Result to follow the order of their dependency.

    opened by yrashk 1
  • Problem: entities are forced to be mutable

    Problem: entities are forced to be mutable

    Also, the requirement for entities and objects to have a default empty constructors is onerous.

    Solution: make layouts constructor+getters driven

    Instead of defining layouts through a full set of accessors, a single constructor drives the layout. Therefore a use of the builder pattern and immutable (final) properties is now advised.

    This solution changes some APIs:

    1. new Layout<>(klass) is deprecated in favour of Layout.forClass(klass)
    opened by yrashk 1
  • Problem: Min/Max queries are global to the attribute

    Problem: Min/Max queries are global to the attribute

    At this moment, Min/Max queries are limited to finding the smallest or largest value of a particular attribute globally across an entity. Which means you can tell "what's the latest order" but not "what's the latest order for this person". IsLatestEntity/LatestAssociatedEntryQuery are currently being used to address this type of need, but they are very slow and cumbersome. Min/Max has been already optimized for PostgreSQL storage.

    It would have been great if it was possible to fit this type query within Min/Max as an option:

    // Find the last order for user with id=<id>
    max(Order.TIMESTAMP, equal(Order.USER_ID, id))
    // Find all last orders for all users
    max(Order.TIMESTAMP, Order.USER_ID)
    

    (Or something similar)

    enhancement help wanted 
    opened by yrashk 0
  • Problem: PostgreSQL database provisioning contention

    Problem: PostgreSQL database provisioning contention

    PostgreSQL-backed journal & indices will create necessary tables and types without consideration for others — meaning there might be a conflict and two nodes might attempt to do the same thing at the same time.

    Proposed solution: ensure these storages are locking these initialization routines. PostgreSQL advisory locks can be used to easily accomplish this.

    bug help wanted 
    opened by yrashk 0
  • Problem: *Index.as interferes with Kotlin

    Problem: *Index.as interferes with Kotlin

    In Kotlin, as is a reserved keyword, and leads to this:

    @JvmField var X = SimpleIndex.`as` { o: TestEntity -> o.x }
    

    as opposed to

    @JvmField var X = SimpleIndex.as { o: TestEntity -> o.x }
    

    which is far from great

    opened by yrashk 0
  • Problem: eventsourcing is the whole organization

    Problem: eventsourcing is the whole organization

    And we're using the same name for es4j for naming maven artifacts and package names, which might make it more difficult to assign proper naming to other solutions.

    Proposed solution 1: rename com.eventsourcing:eventsourcing-X maven artifacts to com.eventsourcing:es4j-X and packages com.eventsourcing.* to com.eventsourcing.es4j.*

    Proposed solution 2: rename com.eventsourcing:eventsourcing-X maven artifacts to org.es4j:es4j-X and packages com.eventsourcing.* to org.es4j.*

    Proposed solution 3: rename com.eventsourcing:eventsourcing-X maven artifacts to com.eventsourcing:es4j-X and packages com.eventsourcing.* to org.es4j.*

    Proposed solution 4: find a completely different name for es4j

    opened by yrashk 0
  • Problem: sometimes, -args aren't or can't be enabled

    Problem: sometimes, -args aren't or can't be enabled

    It is, however, still very desirable to detect parameter names. @ParameterName is one possible workaround, but it is a bit verbose.

    Proposed solution: In some cases, it might be possible to use @java.beans.ConstructorProperties. Nice thing is that (for example) Lombok generates it automatically.

    opened by yrashk 0
Owner
Eventsourcing, Inc.
Non-profit organization
Eventsourcing, Inc.
Samples showing practical aspect of EventStoreDB, Event Sourcing

EventStoreDB samples EventStoreDB is an industrial-strength database technology used as the central data store for event-sourced systems. It is availa

Event Store 47 Oct 26, 2022
Apache Aurora - A Mesos framework for long-running services, cron jobs, and ad-hoc jobs

NOTE: The Apache Aurora project has been moved into the Apache Attic. A fork led by members of the former Project Management Committee (PMC) can be fo

The Apache Software Foundation 627 Nov 28, 2022
Reladomo is an enterprise grade object-relational mapping framework for Java.

Reladomo What is it? Reladomo is an object-relational mapping (ORM) framework for Java with the following enterprise features: Strongly typed compile-

Goldman Sachs 360 Nov 2, 2022
Multi-DBMS SQL Benchmarking Framework via JDBC

BenchBase BenchBase (formerly OLTPBench) is a Multi-DBMS SQL Benchmarking Framework via JDBC. Table of Contents Quickstart Description Usage Guide Con

CMU Database Group 213 Dec 29, 2022
LSPatch: A non-root Xposed framework fork from Xpatch

Introduction LSPatch fork from Xpatch. LSPatch provides a way to insert dex and so into the target APK by repackaging. The following changes have been

LSPosed 1.9k Jan 2, 2023
Ja-netfilter - A javaagent framework

ja-netfilter v2.0.1 A javaagent framework Usage download from the releases page add -javaagent:/absolute/path/to/ja-netfilter.jar argument (Change to

null 7.3k May 26, 2022
Apache Cayenne is an open source persistence framework licensed under the Apache License

Apache Cayenne is an open source persistence framework licensed under the Apache License, providing object-relational mapping (ORM) and remoting services.

The Apache Software Foundation 284 Dec 31, 2022
A javaagent framework

ja-netfilter 2022.2.0 A javaagent framework Usage download from the releases page add -javaagent:/absolute/path/to/ja-netfilter.jar argument (Change t

null 35 Jan 2, 2023
MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap-memory. It is a fast and easy to use embedded Java database engine.

MapDB: database engine MapDB combines embedded database engine and Java collections. It is free under Apache 2 license. MapDB is flexible and can be u

Jan Kotek 4.6k Dec 30, 2022
MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap-memory. It is a fast and easy to use embedded Java database engine.

MapDB: database engine MapDB combines embedded database engine and Java collections. It is free under Apache 2 license. MapDB is flexible and can be u

Jan Kotek 4.6k Jan 1, 2023
jdbi is designed to provide convenient tabular data access in Java; including templated SQL, parameterized and strongly typed queries, and Streams integration

The Jdbi library provides convenient, idiomatic access to relational databases in Java. Jdbi is built on top of JDBC. If your database has a JDBC driv

null 1.7k Dec 27, 2022
esProc SPL is a scripting language for data processing, with well-designed rich library functions and powerful syntax, which can be executed in a Java program through JDBC interface and computing independently.

esProc esProc is the unique name for esProc SPL package. esProc SPL is an open-source programming language for data processing, which can perform comp

null 990 Dec 27, 2022
A RatingBar library for android, you can customize size, spacing, color and image easily, and support right to left.

AndRatingBar A RatingBar library for android, you can customize size, spacing, color and image easily, and support right to left. 安卓RatingBar终极方案,继承自原

dqq 271 Aug 14, 2021
A blazingly small and sane redis java client

Jedis Jedis is a blazingly small and sane Redis java client. Jedis was conceived to be EASY to use. Jedis is fully compatible with redis 2.8.x, 3.x.x

Redis 10.8k Dec 31, 2022
Speedment is a Stream ORM Java Toolkit and Runtime

Java Stream ORM Speedment is an open source Java Stream ORM toolkit and runtime. The toolkit analyzes the metadata of an existing SQL database and aut

Speedment 2k Dec 21, 2022