Zero-allocation hashing for Java

Overview

Zero-Allocation Hashing

Version

badge
javadoc

Overview

This project provides a Java API for hashing any sequence of bytes in Java, including all kinds of primitive arrays, buffers, `CharSequence`s and more.

Written for Java 7+ under Apache 2.0 license.

The key difference compared to other similar projects, e.g. Guava hashing, is that this has no object allocation during the hash computation and does not use ThreadLocal.

The implementation utilises native access where possible, but is also platform-endianness-agnostic. This provides consistent results whatever the byte order, while only moderately affecting performance.

Currently long-valued hash function interface is defined for 64-bit hash, and long[]-valued hash function interface for more than 64-bit hash, with the following implementations (in alphabetical order):

  • CityHash, version 1.1 (latest; 1.1.1 is a C++ language-specific maintenance release).

  • Two algorithms from FarmHash: farmhashna (introduced in FarmHash 1.0) and farmhashuo (introduced in FarmHash 1.1).

  • MetroHash (using the metrohash64_2 initialization vector).

  • MurmurHash3 128-bit and low 64-bit.

  • wyHash, version 3.

  • xxHash.

These are thoroughly tested with LTS JDKs 7, 8, and 11, the latest non-LTS JDKs 15 on both little- and big- endian platforms. Other non-LTS JDKs from 9 should also work, but they will not be tested from half year after EOL.

Performance

Tested on Intel Core i7-4870HQ CPU @ 2.50GHz

Algorithm Speed, GB/s Bootstrap, ns

xxHash

9.5

6

FarmHash na

9.0

6

FarmHash uo

7.2

7

CityHash

7.0

7

MurmurHash

5.3

12

MetroHash

??

??

WyHash

??

??

To sum up,

When to use Zero-Allocation Hashing

  • You need to hash plain byte sequences, memory blocks or "flat" objects.

  • You want zero-allocation and good performance (at Java scale).

  • You need hashing to be agile with regards to byte ordering.

When not to use Zero-Allocation Hashing

  • You need to hash POJOs whose actual data is scattered in memory between managed objects. There is no simple way to hash these using this project, for example, classes such as:

        class Person {
            String givenName, surName;
            int salary;
        }
  • You need to hash byte sequences of unknown length, for the simpliest example, Iterator<Byte>.

  • You need to transform the byte sequence (e.g. encode or decode it with a specific coding), and hash the resulting byte sequence on the way without dumping it to memory.

Quick start

Gradle:

dependencies {
    implementation 'net.openhft:zero-allocation-hashing:0.11'
}

Or Maven:

<dependency>
  <groupId>net.openhft</groupId>
  <artifactId>zero-allocation-hashing</artifactId>
  <version>0.11</version>
</dependency>

In Java:

long hash = LongHashFunction.wy_3().hashChars("hello");

See JavaDocs for more information.

Contributions are most welcome!

See the list of open issues.

Comments
  • Add new api for more than 64-bit hash

    Add new api for more than 64-bit hash

    Add an interface LongTupleHashFunction which extends the 64-bit LongHashFunction. The new interface add two sets of APIs while keeping compatible to the original long API:

    1. long hash(..., long[] result);
    2. long[] hashTuple(...);

    Among three sets of the hash API, the real work will appear in the long hash(..., long[] result) style functions which should returns result[0] as the direct result. The other sets can be deduced from this style ones.

    In the mean time, the implementation of LongTupleHashFunction must override bits() function providing the real bits of this function. The result of bits() should be larger than 64, but don't have to be divided by 64.

    I choose long[] for the result container since it is very simple and can be optimized by JIT. When developing a long hash(..., long[] result), we just need to check if the result is null, which means omitting the full result, and don't have to checkresult.length(). LongTupleHashFunction provides a newLongTuple() function to alloc a correct array based on bits().

    Since a result container can be easily reused within one thread, so the cost of allocation could trend to zero.

    Murmur3F 128-bit has been migrated to `LongTupleHashFunction'.

    enhancement 
    opened by gzm55 15
  • NoSuchMethodError for android 6

    NoSuchMethodError for android 6

    I cannot run xxHash on devices running android 6. Stacktrace:

    java.lang.NoSuchMethodError: No virtual method getByte(Ljava/lang/Object;J)B in class Lsun/misc/Unsafe; or its super classes (declaration of 'sun.misc.Unsafe' appears in /system/framework/core-libart.jar)
            at net.openhft.hashing.UnsafeAccess.getByte(UnsafeAccess.java:87)
            at net.openhft.hashing.UnsafeAccess.getUnsignedByte(UnsafeAccess.java:82)
            at net.openhft.hashing.XxHash.fetch8(XxHash.java:51)
            at net.openhft.hashing.XxHash.xxHash64(XxHash.java:150)
            at net.openhft.hashing.XxHash$AsLongHashFunction.hash(XxHash.java:272)
            at net.openhft.hashing.LongHashFunction.unsafeHash(LongHashFunction.java:453)
            at net.openhft.hashing.LongHashFunction.hashBytes(LongHashFunction.java:494)
    
    opened by jakoss 10
  • Release latest version to Maven central

    Release latest version to Maven central

    See:

    • https://search.maven.org/artifact/net.openhft/zero-allocation-hashing
    • https://mvnrepository.com/artifact/net.openhft/zero-allocation-hashing

    Also, a changelog would be welcome since https://github.com/OpenHFT/Zero-Allocation-Hashing/releases doesn't include the actual releases and the readme contains old refrences to versions:

    image and image and image

    opened by paplorinc 8
  • Class loading deadlock when using LongTupleHashFunction & LongHashFunction

    Class loading deadlock when using LongTupleHashFunction & LongHashFunction

    zero-allocation-hashing version: 0.12 java version: 14 jvm: openjdk

    We are experiencing class loading deadlock when using concurrently LongTupleHashFunction and LongHashFunction, here is the thread dump of the app when the deadlock occurs:

    "default.work-dispatcher-39" #98 prio=5 os_prio=0 cpu=1399.65ms elapsed=747.54s tid=0x00007f54fc07b000 nid=0x7a in Object.wait()  [0x00007f540f4f2000]
       java.lang.Thread.State: RUNNABLE
            at net.openhft.hashing.LongHashFunction.murmur_3(LongHashFunction.java:223)
            - waiting on the Class initialization monitor for net.openhft.hashing.MurmurHash_3
            (...)
    "default.work-dispatcher-48" #108 prio=5 os_prio=0 cpu=2235.06ms elapsed=747.42s tid=0x00007f5410038000 nid=0x84 in Object.wait()  [0x00007f540e4b5000]
       java.lang.Thread.State: RUNNABLE
            at net.openhft.hashing.LongHashFunction.murmur_3(LongHashFunction.java:223)
            - waiting on the Class initialization monitor for net.openhft.hashing.MurmurHash_3
            (...)
    "default.work-dispatcher-50" #109 prio=5 os_prio=0 cpu=211.69ms elapsed=747.42s tid=0x00007f541003c000 nid=0x85 in Object.wait()  [0x00007f540e3b2000]
       java.lang.Thread.State: RUNNABLE
            at net.openhft.hashing.CompactLatin1CharSequenceAccess.<clinit>(CompactLatin1CharSequenceAccess.java:83)
            - waiting on the Class initialization monitor for net.openhft.hashing.UnsafeAccess
            at net.openhft.hashing.ModernCompactStringHash.<clinit>(ModernCompactStringHash.java:14)
            at net.openhft.hashing.Util.<clinit>(Util.java:41)
            at net.openhft.hashing.MurmurHash_3.<clinit>(MurmurHash_3.java:38)
            at net.openhft.hashing.LongHashFunction.murmur_3(LongHashFunction.java:223)
            (...)
    "default.work-dispatcher-103" #195 prio=5 os_prio=0 cpu=378.90ms elapsed=727.08s tid=0x00007f54c8009800 nid=0xe1 in Object.wait()  [0x00007f539fdc6000]
       java.lang.Thread.State: RUNNABLE
            at net.openhft.hashing.UnsafeAccess.<clinit>(UnsafeAccess.java:30)
            - waiting on the Class initialization monitor for net.openhft.hashing.Util
            at net.openhft.hashing.LongTupleHashFunction.<clinit>(LongTupleHashFunction.java:948)
            (...)
    

    NB: I have sanitised the stack traces to remove mention of our own classes.

    opened by blemale 7
  • Hashing a CharSequence directly

    Hashing a CharSequence directly

    Hello! I'd like to be able to hash parts of a long string without having to copy them into a separate StringBuilder / String etc.. For this I'd like to be able to implement CharSequence in order to present the chars I would like to hash.

    The public methods for LongHashFunction only accept String and StringBuilder which are final. However there is the package private method: long hashNativeChars(CharSequence input) in LongHashFunction.java which would provide that. Is there a reason why it cannot be made public to allow access from other packages to it? (without ofc resorting to accessing it through the private-package scope). That would be really great. Thanks!

    opened by talwgx 6
  • Refactor to simplify implementing hash functions

    Refactor to simplify implementing hash functions

    1. move the byte order converting logic into Primitives and Access class
    2. replace common fetch* methods in hash class by the Access short api

    After refactoring, the hashing functions could only care about the calculations in static methods and create suitable LongHashFunction/LongTupleHashFunction instances.

    opened by gzm55 6
  • NoSuchMethodError on Android: Unsafe.getByte

    NoSuchMethodError on Android: Unsafe.getByte

    Using the library on some Android versions results in following Error:

    java.lang.NoSuchMethodError: No virtual method getByte(Ljava/lang/Object;J)B in class Lsun/misc/Unsafe; or its super classes (declaration of 'sun.misc.Unsafe' appears in /system/framework/core-libart.jar)
                      at net.openhft.hashing.LongHashFunction.<clinit>(LongHashFunction.java:73)
    

    It does work on Android 7. It seems, older Android versions do not implement full functionality of Unsafe.

    Considering that Unsafe is likely to be replaced/killed off in future versions of mainline Java, you should probably move all related static constants into the UnsafeAccess and add some handling for situations when it is not available.

    opened by Alexander-- 6
  • release failed

    release failed

    [INFO] --- maven-release-plugin:3.0.0-M1:perform (default-cli) @ zero-allocation-hashing ---
    [INFO] phase verify-release-configuration
    [INFO] starting perform goal, composed of 3 phases: verify-completed-prepare-phases, checkout-project-from-scm, run-perform-goals
    [INFO] [perform] 1/3 verify-completed-prepare-phases
    [INFO] [perform] 2/3 checkout-project-from-scm
    [INFO] Checking out the project to perform the release ...
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD FAILURE
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time:  20.289 s
    [INFO] Finished at: 2021-06-19T16:27:05+02:00
    [INFO] ------------------------------------------------------------------------
    [WARNING] The requested profile "guarded" could not be activated because it does not exist.
    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:3.0.0-M1:perform (default-cli) on project zero-allocation-hashing: Execution default-cli of goal org.apache.maven.plugins:maven-release-plugin:3.0.0-M1:perform failed: An API incompatibility was encountered while executing org.apache.maven.plugins:maven-release-plugin:3.0.0-M1:perform: java.lang.AbstractMethodError: org.apache.maven.scm.command.checkout.AbstractCheckOutCommand.executeCheckOutCommand(Lorg/apache/maven/scm/provider/ScmProviderRepository;Lorg/apache/maven/scm/ScmFileSet;Lorg/apache/maven/scm/ScmVersion;ZZ)Lorg/apache/maven/scm/command/checkout/CheckOutScmResult;
    [ERROR] -----------------------------------------------------
    [ERROR] realm =    plugin>org.apache.maven.plugins:maven-release-plugin:3.0.0-M1
    [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
    [ERROR] urls[0] = file:/home/teamcity/.m2/repository/org/apache/maven/plugins/maven-release-plugin/3.0.0-M1/maven-release-plugin-3.0.0-M1.jar
    [ERROR] urls[1] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-gitexe/1.8.1/maven-scm-provider-gitexe-1.8.1.jar
    [ERROR] urls[2] = file:/home/teamcity/.m2/repository/regexp/regexp/1.3/regexp-1.3.jar
    [ERROR] urls[3] = file:/home/teamcity/.m2/repository/commons-io/commons-io/2.2/commons-io-2.2.jar
    [ERROR] urls[4] = file:/home/teamcity/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar
    [ERROR] urls[5] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-git-commons/1.8.1/maven-scm-provider-git-commons-1.8.1.jar
    [ERROR] urls[6] = file:/home/teamcity/.m2/repository/org/sonatype/sisu/sisu-inject-bean/1.4.2/sisu-inject-bean-1.4.2.jar
    [ERROR] urls[7] = file:/home/teamcity/.m2/repository/org/sonatype/sisu/sisu-guice/2.1.7/sisu-guice-2.1.7-noaop.jar
    [ERROR] urls[8] = file:/home/teamcity/.m2/repository/org/apache/maven/release/maven-release-manager/3.0.0-M1/maven-release-manager-3.0.0-M1.jar
    [ERROR] urls[9] = file:/home/teamcity/.m2/repository/org/apache/maven/release/maven-release-api/3.0.0-M1/maven-release-api-3.0.0-M1.jar
    [ERROR] urls[10] = file:/home/teamcity/.m2/repository/org/codehaus/plexus/plexus-interactivity-api/1.0-alpha-6/plexus-interactivity-api-1.0-alpha-6.jar
    [ERROR] urls[11] = file:/home/teamcity/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.14/plexus-interpolation-1.14.jar
    [ERROR] urls[12] = file:/home/teamcity/.m2/repository/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar
    [ERROR] urls[13] = file:/home/teamcity/.m2/repository/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.jar
    [ERROR] urls[14] = file:/home/teamcity/.m2/repository/org/apache/maven/shared/maven-invoker/2.2/maven-invoker-2.2.jar
    [ERROR] urls[15] = file:/home/teamcity/.m2/repository/org/apache/commons/commons-lang3/3.7/commons-lang3-3.7.jar
    [ERROR] urls[16] = file:/home/teamcity/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar
    [ERROR] urls[17] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-providers-standard/1.11.2/maven-scm-providers-standard-1.11.2.pom
    [ERROR] urls[18] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-accurev/1.11.2/maven-scm-provider-accurev-1.11.2.jar
    [ERROR] urls[19] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-bazaar/1.11.2/maven-scm-provider-bazaar-1.11.2.jar
    [ERROR] urls[20] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-clearcase/1.11.2/maven-scm-provider-clearcase-1.11.2.jar
    [ERROR] urls[21] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-cvsexe/1.11.2/maven-scm-provider-cvsexe-1.11.2.jar
    [ERROR] urls[22] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-cvs-commons/1.11.2/maven-scm-provider-cvs-commons-1.11.2.jar
    [ERROR] urls[23] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-cvsjava/1.11.2/maven-scm-provider-cvsjava-1.11.2.jar
    [ERROR] urls[24] = file:/home/teamcity/.m2/repository/org/netbeans/lib/cvsclient/20060125/cvsclient-20060125.jar
    [ERROR] urls[25] = file:/home/teamcity/.m2/repository/ch/ethz/ganymed/ganymed-ssh2/build210/ganymed-ssh2-build210.jar
    [ERROR] urls[26] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-jgit/1.11.2/maven-scm-provider-jgit-1.11.2.jar
    [ERROR] urls[27] = file:/home/teamcity/.m2/repository/org/eclipse/jgit/org.eclipse.jgit/4.5.4.201711221230-r/org.eclipse.jgit-4.5.4.201711221230-r.jar
    [ERROR] urls[28] = file:/home/teamcity/.m2/repository/com/jcraft/jsch/0.1.53/jsch-0.1.53.jar
    [ERROR] urls[29] = file:/home/teamcity/.m2/repository/com/googlecode/javaewah/JavaEWAH/0.7.9/JavaEWAH-0.7.9.jar
    [ERROR] urls[30] = file:/home/teamcity/.m2/repository/org/apache/httpcomponents/httpclient/4.3.6/httpclient-4.3.6.jar
    [ERROR] urls[31] = file:/home/teamcity/.m2/repository/org/apache/httpcomponents/httpcore/4.3.3/httpcore-4.3.3.jar
    [ERROR] urls[32] = file:/home/teamcity/.m2/repository/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar
    [ERROR] urls[33] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-hg/1.11.2/maven-scm-provider-hg-1.11.2.jar
    [ERROR] urls[34] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-perforce/1.11.2/maven-scm-provider-perforce-1.11.2.jar
    [ERROR] urls[35] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-starteam/1.11.2/maven-scm-provider-starteam-1.11.2.jar
    [ERROR] urls[36] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-svnexe/1.11.2/maven-scm-provider-svnexe-1.11.2.jar
    [ERROR] urls[37] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-synergy/1.11.2/maven-scm-provider-synergy-1.11.2.jar
    [ERROR] urls[38] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-vss/1.11.2/maven-scm-provider-vss-1.11.2.jar
    [ERROR] urls[39] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-tfs/1.11.2/maven-scm-provider-tfs-1.11.2.jar
    [ERROR] urls[40] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-integrity/1.11.2/maven-scm-provider-integrity-1.11.2.jar
    [ERROR] urls[41] = file:/home/teamcity/.m2/repository/com/mks/api/mksapi-jar/4.10.9049/mksapi-jar-4.10.9049.jar
    [ERROR] urls[42] = file:/home/teamcity/.m2/repository/org/codehaus/groovy/groovy-all/1.7.6/groovy-all-1.7.6.jar
    [ERROR] urls[43] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-jazz/1.11.2/maven-scm-provider-jazz-1.11.2.jar
    [ERROR] urls[44] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-manager-plexus/1.8/maven-scm-manager-plexus-1.8.jar
    [ERROR] urls[45] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-provider-svn-commons/1.11.2/maven-scm-provider-svn-commons-1.11.2.jar
    [ERROR] urls[46] = file:/home/teamcity/.m2/repository/org/apache/maven/shared/maven-artifact-transfer/0.9.1/maven-artifact-transfer-0.9.1.jar
    [ERROR] urls[47] = file:/home/teamcity/.m2/repository/org/apache/maven/shared/maven-common-artifact-filters/3.0.1/maven-common-artifact-filters-3.0.1.jar
    [ERROR] urls[48] = file:/home/teamcity/.m2/repository/org/apache/maven/shared/maven-shared-utils/3.1.0/maven-shared-utils-3.1.0.jar
    [ERROR] urls[49] = file:/home/teamcity/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar
    [ERROR] urls[50] = file:/home/teamcity/.m2/repository/org/sonatype/aether/aether-util/1.7/aether-util-1.7.jar
    [ERROR] urls[51] = file:/home/teamcity/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.7.1/plexus-component-annotations-1.7.1.jar
    [ERROR] urls[52] = file:/home/teamcity/.m2/repository/org/apache/maven/scm/maven-scm-api/1.11.2/maven-scm-api-1.11.2.jar
    [ERROR] urls[53] = file:/home/teamcity/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.15/plexus-utils-3.0.15.jar
    [ERROR] urls[54] = file:/home/teamcity/.m2/repository/org/jdom/jdom/1.1/jdom-1.1.jar
    [ERROR] Number of foreign imports: 1
    [ERROR] import: Entry[import  from realm ClassRealm[project>net.openhft:zero-allocation-hashing:0.15-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]]
    [ERROR] 
    [ERROR] -----------------------------------------------------
    [ERROR] 
    [ERROR] -> [Help 1]
    [ERROR] 
    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
    [ERROR] Re-run Maven using the -X switch to enable full debug logging.
    [ERROR] 
    [ERROR] For more information about the errors and possible solutions, please read the following articles:
    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException
    [teamcity@teamcity Zero-Allocation-Hashing]$ 
    
    
    opened by RobAustin 5
  • Can't build with JDK8 in IDEA

    Can't build with JDK8 in IDEA

    After Clone the repository I try to build this project in IDEA with JDK8. I receive a error in file src/main/java-stub/java/lang/Math.java because JDK8 don't have Math.multiplyHigh and try-catch can't handle this.

    opened by csm-dev-hub 4
  • CharSequenceAccess supports unalign offsets

    CharSequenceAccess supports unalign offsets

    detect unalign offsets in runtime. this may take in a little performance cost, but should work. all ut cases are designed for little endian machines.

    Close #36

    bug 
    opened by gzm55 4
  • Question: the least collision rate for UUID like input

    Question: the least collision rate for UUID like input

    Hey,

    I'm trying to avoid using off-heap Map/Set implementations for performance reasons (serialization/deserialization) and a good way of doing that is using Long instead of String for UUID like keys and using some Map implementation that doesn't autobox primitives...

    I'm just wondering what is the most suitable 64bit hash function? I'm mostly going for the lowest collision rate rather than for performance. I'd just like to have a HashMap/HashSet with up to 30-40M records with the least memory footprint possible without GC overhead and test it for presence of 400M distinct UUIDs. I have no idea what collision rate could be there if I use say murmur3 for 64bit hashing of those UUIDs.

    I'm very grateful for your library, it is the only source of 64bit hash functions. I'm kind of terrified of using Guava's 128bit hash functions and then calling : http://google.github.io/guava/releases/snapshot/api/docs/com/google/common/hash/HashCode.html#padToLong--

    opened by l15k4 4
  • Replace Unsafe with VarHandle

    Replace Unsafe with VarHandle

    I tested my implementation for Java 9+ with VarHandles instead of Unsafe. In my test case (hashing multiple files with a total size of ~ 1 GB) I noticed a huge speed improvement

    • XxHash: Unsafe 1220 ms vs VarHandle 630 ms
    • XxHash3: Unsafe 1280 ms vs VarHandle 720 ms

    Is it possible to publish separate version for modern JDKs without dependency on Unsafe?

    opened by qweek 6
  • migrating CI to travis-ci.com

    migrating CI to travis-ci.com

    according to travis notice:

    Please be aware travis-ci.org will be shutting down in several weeks, with all accounts migrating to travis-ci.com. Please stay tuned here for more information.

    opened by gzm55 0
  • Fatal crash on Samsung Galaxy J5 (SM-J530F)

    Fatal crash on Samsung Galaxy J5 (SM-J530F)

    Hi,

    I have it only on android device: Samsung Galaxy J5 (SM-J530F), but possibly there are other devices affected.

    We have fatal crash that always happens when trying to hash byte array (contents does not seem to matter) using xxHash:

    LongHashFunction
            .xx().hashBytes(value)
    

    Taken from LogCat:

        --------- beginning of crash
    2020-03-12 12:07:08.966 16010-16280/? A/libc: Fatal signal 7 (SIGBUS), code 1, fault addr 0x1338520c in tid 16280 (.pl/...), pid 16010 ()
    2020-03-12 12:07:09.051 16283-16283/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    2020-03-12 12:07:09.051 16283-16283/? A/DEBUG: Build fingerprint: 'samsung/j5y17ltexx/j5y17lte:8.1.0/M1AJQ/J530FXXU3BRJ2:user/release-keys'
    2020-03-12 12:07:09.052 16283-16283/? A/DEBUG: Revision: '7'
    2020-03-12 12:07:09.052 16283-16283/? A/DEBUG: ABI: 'arm'
    2020-03-12 12:07:09.052 16283-16283/? A/DEBUG: pid: 16010, tid: 16280, name: .pl/...  >>> com.erfg.music <<<
    2020-03-12 12:07:09.052 16283-16283/? A/DEBUG: signal 7 (SIGBUS), code 1 (BUS_ADRALN), fault addr 0x1338520c
    2020-03-12 12:07:09.052 16283-16283/? A/DEBUG:     r0 1338520c  r1 0000000c  r2 ca9e95cc  r3 0000000c
    2020-03-12 12:07:09.052 16283-16283/? A/DEBUG:     r4 6f31be58  r5 00000004  r6 00000000  r7 ca9e98c8
    2020-03-12 12:07:09.052 16283-16283/? A/DEBUG:     r8 00000000  r9 cb5f2c00  sl ca9e96c8  fp ca9e9654
    2020-03-12 12:07:09.052 16283-16283/? A/DEBUG:     ip eae9fced  sp ca9e95a8  lr eae9fcf7  pc eae9fcfa  cpsr 600d0030
    2020-03-12 12:07:09.185 16283-16283/? A/DEBUG: backtrace:
    2020-03-12 12:07:09.185 16283-16283/? A/DEBUG:     #00 pc 00310cfa  /system/lib/libart.so (art::Unsafe_getLong(_JNIEnv*, _jobject*, _jobject*, long long)+13)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #01 pc 005db08f  /system/framework/arm/boot.oat (offset 0x1cb000) (sun.misc.Unsafe.getLong [DEDUPED]+110)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #02 pc 0040c575  /system/lib/libart.so (art_quick_invoke_stub_internal+68)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #03 pc 004116e5  /system/lib/libart.so (art_quick_invoke_stub+228)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #04 pc 000b0227  /system/lib/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+138)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #05 pc 00204005  /system/lib/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*)+224)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #06 pc 001ff54d  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+588)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #07 pc 003f8c87  /system/lib/libart.so (MterpInvokeVirtualQuick+598)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #08 pc 00402714  /system/lib/libart.so (ExecuteMterpImpl+29972)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #09 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #10 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #11 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #12 pc 003f8c87  /system/lib/libart.so (MterpInvokeVirtualQuick+598)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #13 pc 00402714  /system/lib/libart.so (ExecuteMterpImpl+29972)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #14 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #15 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #16 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #17 pc 003f8c87  /system/lib/libart.so (MterpInvokeVirtualQuick+598)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #18 pc 00402714  /system/lib/libart.so (ExecuteMterpImpl+29972)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #19 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #20 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #21 pc 00200159  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb1ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+444)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #22 pc 003f8fa5  /system/lib/libart.so (MterpInvokeVirtualQuickRange+472)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #23 pc 00402794  /system/lib/libart.so (ExecuteMterpImpl+30100)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #24 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #25 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #26 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #27 pc 003f8c87  /system/lib/libart.so (MterpInvokeVirtualQuick+598)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #28 pc 00402714  /system/lib/libart.so (ExecuteMterpImpl+29972)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #29 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.186 16283-16283/? A/DEBUG:     #30 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #31 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #32 pc 003f77b9  /system/lib/libart.so (MterpInvokeStatic+184)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #33 pc 003feb14  /system/lib/libart.so (ExecuteMterpImpl+14612)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #34 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #35 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #36 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #37 pc 003f8c87  /system/lib/libart.so (MterpInvokeVirtualQuick+598)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #38 pc 00402714  /system/lib/libart.so (ExecuteMterpImpl+29972)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #39 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #40 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #41 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #42 pc 003f8c87  /system/lib/libart.so (MterpInvokeVirtualQuick+598)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #43 pc 00402714  /system/lib/libart.so (ExecuteMterpImpl+29972)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #44 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #45 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #46 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #47 pc 003f7391  /system/lib/libart.so (MterpInvokeInterface+1080)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #48 pc 003feb94  /system/lib/libart.so (ExecuteMterpImpl+14740)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #49 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #50 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #51 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #52 pc 003f7391  /system/lib/libart.so (MterpInvokeInterface+1080)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #53 pc 003feb94  /system/lib/libart.so (ExecuteMterpImpl+14740)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #54 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #55 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.187 16283-16283/? A/DEBUG:     #56 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.188 16283-16283/? A/DEBUG:     #57 pc 003f7391  /system/lib/libart.so (MterpInvokeInterface+1080)
    2020-03-12 12:07:09.188 16283-16283/? A/DEBUG:     #58 pc 003feb94  /system/lib/libart.so (ExecuteMterpImpl+14740)
    2020-03-12 12:07:09.188 16283-16283/? A/DEBUG:     #59 pc 001e6bc1  /system/lib/libart.so (art::interpreter::Execute(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame&, art::JValue, bool)+340)
    2020-03-12 12:07:09.188 16283-16283/? A/DEBUG:     #60 pc 001eb36f  /system/lib/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::DexFile::CodeItem const*, art::ShadowFrame*, art::JValue*)+142)
    2020-03-12 12:07:09.188 16283-16283/? A/DEBUG:     #61 pc 001ff535  /system/lib/libart.so (_ZN3art11interpreter6DoCallILb0ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+564)
    2020-03-12 12:07:09.188 16283-16283/? A/DEBUG:     #62 pc 003f7391  /system/lib/libart.so (MterpInvokeInterface+1080)
    2020-03-12 12:07:09.188 16283-16283/? A/DEBUG:     #63 pc 003feb94  /system/lib/libart.so (ExecuteMterpImpl+14740)
    2020-03-12 12:07:11.258 2748-2748/? E//system/bin/tombstoned: Tombstone written to: /data/tombstones/tombstone_07
    2020-03-12 12:07:11.266 2693-2693/? E/audit: type=1701 audit(1584011231.251:1220): auid=4294967295 uid=10219 gid=10219 ses=4294967295 subj=u:r:untrusted_app:s0:c512,c768 pid=16280 comm=".pl/..." exe="/system/bin/app_process32" sig=7
    2020-03-12 12:07:11.301 2962-16284/? E/ActivityManager: Found activity ActivityRecord{dc4c01c u0 com.efgd.music/.MainActivity t-1 f} in proc activity list using null instead of expected ProcessRecord{ca03534 16010:com.efgd.music/u0a219}
    2020-03-12 12:07:11.398 3338-3338/? E/SKBD: bbw getInstance start
    2020-03-12 12:07:11.398 3338-3338/? E/SKBD: bbw sendSIPInformation state: 6   isAbstractKeyboardView :  true
    2020-03-12 12:07:11.404 3338-16293/? E/SKBD: bbw sending null keyboardInfo as SIP is closed
    2020-03-12 12:07:11.419 5224-5254/? E/PBSessionCacheImpl: sessionId[22976978907188413] not persisted.
    
    
    opened by jakoss 8
  • Why is hashing stateless?

    Why is hashing stateless?

    Hello.

    I'm trying to understand the design decision behind requiring hashing to be stateless.

    Each call to hashByte/hashLong etc. starts with the state equal to the original seed and calls finalize at the end. As far as I can tell, this means that you can't chain these hash calls (like you can with Guava, for example) unless you keep passing finalized() hashes around.

    Am I missing something? It seems like we should require the user to call finalize and to have a stateful long hash variable that can be reset() in each implementation. This allows the user to replicate the current behavior by calling reset()-hashByte()-finalize() but also to chain calls to each of the hash functions (reset()-hashByte()-hashLong()...finalize()).

    opened by JeffGreen 3
Releases(zero-allocation-hashing-0.15)
  • zero-allocation-hashing-0.15(Jun 20, 2021)

    • new API set: LongTupleHashFunction, support 128 bit hash
    • improve performance on JDK9+
    • add hash: xxh3 64bits hash, xxh128 128bits hash, xxh128low 64bits hash
    • add hash: murmur_3 128bits hash
    • public all implementations of Access
    Source code(tar.gz)
    Source code(zip)
  • zero-allocation-hashing-0.11(Feb 25, 2020)

  • zero-allocation-hashing-0.10.1(Dec 22, 2019)

    • Added WyHash implementation (thanks to @firestrand!)
    • The library is tested on OpenJDK from version 7 to 13.

    (Compared to 0.10 release, an unwanted class is hidden from the public library API.)

    Download JAR: http://oss.sonatype.org/content/repositories/releases/net/openhft/zero-allocation-hashing/0.10.1/zero-allocation-hashing-0.10.1.jar

    Source code(tar.gz)
    Source code(zip)
  • zero-allocation-hashing-0.9(Feb 23, 2019)

    • Added MetroHash implementation (thanks to @Derongan!)
    • Minor improvements
    • Dropped Java 6 support
    • Minor improvements to serialization (serialization format of some functions has changed; Functions serialized with previous versions of the library may not be deserialized with this version of the library).

    Download JAR: http://oss.sonatype.org/content/repositories/releases/net/openhft/zero-allocation-hashing/0.9/zero-allocation-hashing-0.9.jar

    Source code(tar.gz)
    Source code(zip)
    zero-allocation-hashing-0.9.jar(52.61 KB)
  • zero-allocation-hashing-0.8(Feb 27, 2017)

    • Fixed #18 (issue on pre-Nougat Android)
    • Renamed LongHashFunction.xx_r39() to simply xx()

    Download jar: https://oss.sonatype.org/content/repositories/releases/net/openhft/zero-allocation-hashing/0.8/zero-allocation-hashing-0.8.jar

    Source code(tar.gz)
    Source code(zip)
  • zero-allocation-hashing-0.7(Feb 18, 2017)

    • Fixed a bug in xxHash implementation of LongHashFunction.hashInt() and hashByte()

    Download jar: https://oss.sonatype.org/content/repositories/releases/net/openhft/zero-allocation-hashing/0.7/zero-allocation-hashing-0.7.jar

    Source code(tar.gz)
    Source code(zip)
  • zero-allocation-hashing-0.6(Mar 2, 2016)

    • OSGi support

    Download jar: https://oss.sonatype.org/content/repositories/releases/net/openhft/zero-allocation-hashing/0.6/zero-allocation-hashing-0.6.jar

    Source code(tar.gz)
    Source code(zip)
  • zero-allocation-hashing-0.5(Dec 2, 2015)

    • Optimized performance
    • Made com.intellij:annotations dependency optional

    Download jar: https://oss.sonatype.org/content/repositories/releases/net/openhft/zero-allocation-hashing/0.5/zero-allocation-hashing-0.5.jar

    Source code(tar.gz)
    Source code(zip)
  • zero-allocation-hashing-0.4(Sep 11, 2015)

    • Added FarmHash: farmhashna and farmhashuo algorithms

    Download jar: https://oss.sonatype.org/content/repositories/releases/net/openhft/zero-allocation-hashing/0.4/zero-allocation-hashing-0.4.jar

    Source code(tar.gz)
    Source code(zip)
  • zero-allocation-hashing-0.3(Jul 18, 2015)

    • Added xxHash

    Download jar: https://oss.sonatype.org/content/repositories/releases/net/openhft/zero-allocation-hashing/0.3/zero-allocation-hashing-0.3.jar

    Source code(tar.gz)
    Source code(zip)
  • zero-allocation-hashing-0.2(Jan 25, 2015)

    • Added MurmurHash3
    • Minor fixes

    Download jar: https://oss.sonatype.org/content/groups/public/net/openhft/zero-allocation-hashing/0.2/zero-allocation-hashing-0.2.jar

    Source code(tar.gz)
    Source code(zip)
  • zero-allocation-hashing-0.1(Jan 25, 2015)

Owner
Chronicle Software : Open Source
Open Source components of Chronicle Software
Chronicle Software : Open Source
Zero is a core test automation project that can be used as a basis for any kind of test automation project (API, Browser, Mobile)

Zero Zero is our feature rich, core test automation framework, that can be used as an underlying automation framework for any/and all kind of test aut

Pramod Kumar Yadav 10 Dec 16, 2022
High Performance data structures and utility methods for Java

Agrona Agrona provides a library of data structures and utility methods that are a common need when building high-performance applications in Java. Ma

Real Logic 2.5k Jan 5, 2023
Bloofi: A java implementation of multidimensional Bloom filters

Bloofi: A java implementation of multidimensional Bloom filters Bloom filters are probabilistic data structures commonly used for approximate membersh

Daniel Lemire 71 Nov 2, 2022
A high performance caching library for Java

Caffeine is a high performance, near optimal caching library. For more details, see our user's guide and browse the API docs for the latest release. C

Ben Manes 13k Jan 5, 2023
Chronicle Bytes has a similar purpose to Java NIO's ByteBuffer with many extensions

Chronicle-Bytes Chronicle-Bytes Chronicle Bytes contains all the low level memory access wrappers. It is built on Chronicle Core’s direct memory and O

Chronicle Software : Open Source 334 Jan 1, 2023
High performance Java implementation of a Cuckoo filter - Apache Licensed

Cuckoo Filter For Java This library offers a similar interface to Guava's Bloom filters. In most cases it can be used interchangeably and has addition

Mark Gunlogson 161 Dec 30, 2022
An advanced, but easy to use, platform for writing functional applications in Java 8.

Getting Cyclops X (10) The latest version is cyclops:10.4.0 Stackoverflow tag cyclops-react Documentation (work in progress for Cyclops X) Integration

AOL 1.3k Dec 29, 2022
Eclipse Collections is a collections framework for Java with optimized data structures and a rich, functional and fluent API.

English | 中文 | Deutsch | Español | Ελληνικά | Français | 日本語 | Norsk (bokmål) | Português-Brasil | Русский | हिंदी Eclipse Collections is a comprehens

Eclipse Foundation 2.1k Dec 29, 2022
External-Memory Sorting in Java

Externalsortinginjava External-Memory Sorting in Java: useful to sort very large files using multiple cores and an external-memory algorithm. The vers

Daniel Lemire 235 Dec 29, 2022
A Java library for quickly and efficiently parsing and writing UUIDs

fast-uuid fast-uuid is a Java library for quickly and efficiently parsing and writing UUIDs. It yields the most dramatic performance gains when compar

Jon Chambers 142 Jan 1, 2023
Geohash utitlies in java

geo Java utility methods for geohashing. Status: production, available on Maven Central Maven site reports are here including javadoc. Add this to you

Dave Moten 386 Jan 1, 2023
Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-only access.

Hollow Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-on

Netflix, Inc. 1.1k Dec 25, 2022
High Performance Primitive Collections for Java

HPPC: High Performance Primitive Collections Collections of primitive types (maps, sets, stacks, lists) with open internals and an API twist (no java.

Carrot Search 890 Dec 28, 2022
Java port of a concurrent trie hash map implementation from the Scala collections library

About This is a Java port of a concurrent trie hash map implementation from the Scala collections library. It is almost a line-by-line conversion from

null 147 Oct 31, 2022
Java library for the HyperLogLog algorithm

java-hll A Java implementation of HyperLogLog whose goal is to be storage-compatible with other similar offerings from Aggregate Knowledge. NOTE: This

Aggregate Knowledge (a Neustar service) 296 Dec 30, 2022
A simple integer compression library in Java

JavaFastPFOR: A simple integer compression library in Java License This code is released under the Apache License Version 2.0 http://www.apache.org/li

Daniel Lemire 487 Dec 30, 2022
Java Collections till the last breadcrumb of memory and performance

Koloboke A family of projects around collections in Java (so far). The Koloboke Collections API A carefully designed extension of the Java Collections

Roman Leventov 967 Nov 14, 2022
Port of LevelDB to Java

LevelDB in Java This is a rewrite (port) of LevelDB in Java. This goal is to have a feature complete implementation that is within 10% of the performa

Dain Sundstrom 1.4k Dec 30, 2022