Java Collections till the last breadcrumb of memory and performance

Overview

Koloboke

A family of projects around collections in Java (so far).

The Koloboke Collections API

koloboke-api maven central

A carefully designed extension of the Java Collections Framework with primitive specializations and more. Java 6+. Apache 2.0 license.

Compatibility with the Java Collections Framework

  • All primitive specialization collections extend basic interfaces (Collection, Set, Map), hence could be used as drop-in replacements of slow collections of boxed values
  • API for Java 6 and 7 is forward-compatible with all methods new in Java 8

JavaDoc: Java 6 | Java 7 | Java 8

Koloboke Compile

koloboke-compile maven central

An annotation processor, generates implementations for collection-like abstract classes or interfaces. API agnostic, may be used to generate implementation for classes or interfaces, extending interfaces from the Koloboke Collections API, or, for example, interfaces from the Trove collections library. "Embeddable version" of the Koloboke implementation library.

Quick start

Add the following dependencies in your Maven pom.xml:

  <dependency>
    <groupId>com.koloboke</groupId>
    <artifactId>koloboke-compile</artifactId>
    <version>0.5.1</version>
    <scope>provided</scope>
  </dependency>
  <dependency>
    <groupId>com.koloboke</groupId>
    <!-- `jdk6-7` instead of `jdk8` if you use Java 6 or 7 -->
    <artifactId>koloboke-impl-common-jdk8</artifactId>
    <version>1.0.0</version>
  </dependency>

Or in your Gradle build script, you should first apply the propdeps Gradle plugin to enable provided dependencies, and then configure the dependencies block:

dependencies {
    provided 'com.koloboke:koloboke-compile:0.5.1'
    // `jdk6-7` instead of `jdk8` if you use Java 6 or 7
    compile 'com.koloboke:koloboke-impl-common-jdk8:1.0.0'
}

Next step: read the Koloboke Compile tutorial.

Javadocs

The Koloboke implementation library

koloboke-impl maven central

An efficient implementation of the Koloboke Collections API.

Compatibility with the Java Collections Framework

  • Fail-fast semantics everywhere
  • null keys are (optionally) supported, just like in java.util.HashMap
  • Float.NaN and Double.NaN keys are treated consistently with boxed version (all NaNs are considered equal)

Quick start

Add the following dependencies in your Maven pom.xml:

  <dependencies>
    <dependency>
      <groupId>com.koloboke</groupId>
      <artifactId>koloboke-api-jdk8</artifactId>
      <version>1.0.0</version>
    </dependency>
    <dependency>
      <groupId>com.koloboke</groupId>
      <artifactId>koloboke-impl-jdk8</artifactId>
      <version>1.0.0</version>
      <scope>runtime</scope>
    </dependency>
  <dependencies>

Or to your Gradle build script:

dependencies {
    // `jdk6-7` instead of `jdk8` if you use Java 7 or older
    compile 'com.koloboke:koloboke-api-jdk8:1.0.0'
    runtime 'com.koloboke:koloboke-impl-jdk8:1.0.0'
}

Or similarly for your favourite build system.

Then you can start using collections. Replace all lines like

Map<Integer, Integer> map = new HashMap<>();

with

Map<Integer, Integer> map = HashIntIntMaps.newMutableMap();

Next step: see [the table of equivalents of JDK collection patterns] (http://leventov.github.io/Koloboke/api/1.0/java8/index.html#jdk-equivalents).


Releases (with changelog)


Contributing, Feedback & Support

Use issues or ask a question on StackOverflow.


How to build and develop

Gradle build requires Java 8 compiler, set JAVA_HOME environment variable to the JDK 8 location. Next to your JDK 8 location (i. e. a jdk1.8 directory), JDK 9 installation has to be present in a jdk-9 directory. For meta projects development, JDK 6 and JDK 7 also have to be present in jdk1.6 and jdk1.7 directories sibling to the jdk1.8 directory.

Then

$ git clone [email protected]:leventov/Koloboke.git
$ cd Koloboke
$ ./gradlew :buildMeta
$ ./gradlew buildMain -x test -x findbugsMain -x findbugsTest
$ ./gradlew idea

Then you can open the project in IntelliJ IDEA.

To rebuild meta projects (code generators), run from the project root dir:

$ ./gradlew :cleanMeta :buildMeta

To rebuild either the lib, benchmarks or both, run

$ ./gradlew cleanMain buildMain

from the lib, benchmarks subdir or the root project dir respectively.

To build the lib for Java 8, run

$ ../gradlew cleanMain buildMain -PlibTargetJava=8

from the lib subdir.

If you want to generate proper Javadocs, especially for Java 6 or 7, you should specify javadocExecutable and jdkSrc build properties (see Gradle docs for how to do that). Typical javadocExecutable value is JAVA_HOME/bin/javadoc[.exe], jdkSrc should point to a directory which contain uncompressed JDK sources, i. e. package structure starting from java, javax, sun, etc. subdirs.


Project name history

  • Trove (This project was started as a Trove fork, but has nothing in common with Trove for already very long time.)
  • UntitledCollectionsProject, UCP
  • Higher Frequency Trading Collections, OpenHFT Collections, HFT Collections, HFTC
  • Koloboke (Collections) -- current name!
Comments
  • new map creation in multiple threads throws java.util.NoSuchElementException

    new map creation in multiple threads throws java.util.NoSuchElementException

    Exception in thread "pool-1-thread-1" java.util.NoSuchElementException at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:340) at java.util.ServiceLoader$1.next(ServiceLoader.java:428) at net.openhft.koloboke.collect.map.hash.HashIntDoubleMaps.getDefaultFactory(HashIntDoubleMaps.java:56) at net.openhft.koloboke.collect.map.hash.HashIntDoubleMaps.newMutableMap(HashIntDoubleMaps.java:81) at Test.run(Test.java:18) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:695)

    bug 
    opened by amirkamran 10
  • provide a small core distribution without type-specialization

    provide a small core distribution without type-specialization

    Many projects could benefit from drop-in replacement Set/Map implementations (Object -> Object). Please provide a small distribution of koloboke (in Maven central), that only contains this core, instead of the 20Mb version with all the primitive specializations.

    Build 
    opened by phraktle 10
  • Cannot build with Java 1.8.0_11

    Cannot build with Java 1.8.0_11

    Hi there,

    I'm currently using trove for a few projects, so naturally I wanted to benchmark hftc in comparison. However, the build buildMain always stops at > Building 8% > :lib:impl:compileJava.

    My setup is Java 1.8.0_11 (oracle) on LUbuntu 13.10 (64bit), gradle 2.0, groovy 2.3.3, ant 1.9.3.

    PS. It would be great if you could release binaries.

    bug Performance & Benchmarks Build 
    opened by llogiq 9
  • Expand Implemented Interfaces

    Expand Implemented Interfaces

    Is there a possibility to expand the number of collections interfaces implemented so these classes can be used there any other collection interface is used.

    question new functionality 
    opened by sirinath 9
  • Extract code generator into a separate project

    Extract code generator into a separate project

    Code generator appears to be perfectly reusable.

    Currenlty it is located in buildSrc/ module, it acts as Gradle jpsg (Java Primitive Specializations Generator) plugin.

    opened by leventov 7
  • 0.6.2 does not work under java-se-1.6: java.lang.NoClassDefFoundError java.util.concurrent.ThreadLocalRandom

    0.6.2 does not work under java-se-1.6: java.lang.NoClassDefFoundError java.util.concurrent.ThreadLocalRandom

    The generated code contains calls to java.util.concurent.ThreadLocalRandom, which is not present in java 6. See http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadLocalRandom.html

    (Env. Mac OSX 10.9.4, JRE Java SE6 version 1.6.0_33)

    public class Test {
        public static void main(final String[] args) {
    
            new LHashParallelKVLongDoubleMapFactoryImpl();
        }
    }
    
    Exception in thread "main" java.lang.NoClassDefFoundError: java/util/concurrent/ThreadLocalRandom
        at com.busintel.ponti.impl.MapBasedValueCollectionImplTest.main(MapBasedValueCollectionImplTest.java:62)
    Caused by: java.lang.ClassNotFoundException: java.util.concurrent.ThreadLocalRandom
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        ... 1 more
    
    bug Java 6 
    opened by h2000 6
  • possibility to generate only a subset of implementations

    possibility to generate only a subset of implementations

    Hello leventov,

    At first thanks for the very cool library.

    Is it possible to build the library with a subset of primitives e.g. key={int,long} value={int,long,double} to reduce the jar size?

    question Build 
    opened by h2000 5
  • Koloboke jar size

    Koloboke jar size

    The koloboke jars are quite big (api: 850KB, impl: 18.4MB), so users may want to create fat jars to keep only the classes they actually use, for instance with maven-shade-plugin.

    But koloboke uses ServiceLoader to locate the implementation factories, so the jar minification tools do not detect the dependencies and strip down all the implementation classes.

    Here is a sample workaround configuration for maven-shade-plugin (saves 9MB in this case):

          <plugin>
            <artifactId>maven-shade-plugin</artifactId>
            <executions>
              <execution>
                <phase>package</phase>
                <goals>
                  <goal>shade</goal>
                </goals>
                <configuration>
                  <filters>
                    <filter>
                      <artifact>com.koloboke:*</artifact>
                      <includes>
                        <include>**/*IntSet*</include>
                        <include>**/*IntIntMap*</include>
                        <include>**/*IntObjMap*</include>
                      </includes>
                    </filter>
                  </filters>
                </configuration>
              </execution>
            </executions>
          </plugin>
    

    Here are some solutions I can imagine:

    1. Publish one big jar containing API+impl, not using ServiceLoader to locate the default factory, but a straight import
    2. Split the impl jar in several smaller ones (eg: by type ?)

    (by the way, the minification is only one aspect of the issue, this causes the same issues with obfuscation/bytecode manipulation tools such as Proguard)

    opened by nicoulaj 4
  • HashObjIntMap performance problem

    HashObjIntMap performance problem

    Hello Roman, I'm currently trying to brew my own benchmarks using Mikhail Vorontsov's as inspiration (http://java-performance.info) using my own put() version modified for different inputs. Well, it turns out Koloboke is badly misbehaving, to the point of pathological, if you compare to Fastutil for instance. I've reproduced the case using a Random distrib [- 6000000; 6000000] with 6000000 as target preallocated size, 0.75 load factor, using the same allocation logic as Mikhail did:

    testedMap = HashObjIntMaps. getDefaultFactory() .withHashConfig(HashConfig.fromLoads(KolobokeObjectIntIssue.LOAD_FACTOR / 2, KolobokeObjectIntIssue.LOAD_FACTOR, KolobokeObjectIntIssue.LOAD_FACTOR)) .newMutableMap(KolobokeObjectIntIssue.TARGET_SIZE);

    Here is the code: https://gist.github.com/vsonnier/35b68b109528884e2bdd

    The int ==> int version does not seem to suffer the same problem, under the same input.

    Performance & Benchmarks 
    opened by vsonnier 3
  • The ensureCapacity implementation

    The ensureCapacity implementation

    Hi,

    We have a change in the Trove ensureCapacity method which takes care that the grow of the underlying tables (_set, _values) is following the normal grow where the capacity is always doubled. Your (and Trove) implementation will always increase the capacity just to fit the new additional (or Trove's extra desired) size which will cause a lot of rehashing (including array re-allocations) when ensureCapacity is called many times for small increments. If the ensureCapacity is going to rehash, it should follow the normal capacity grow (Trove - double it, Koloboke - grow by configured scale).

    Reimplemented ensureCapacity for Trove 2.0.x

      @Override
      public void ensureCapacity(int desiredExtraSize) {
        if(desiredExtraSize > (super._maxSize - size())) {
          final int desiredCapacity = ((int) ((desiredExtraSize + size()) / super._loadFactor)) + 1;
          final int doubleCapacity = this.capacity() << 1;
          this.rehash(PrimeFinder.nextPrime(Math.max(desiredCapacity, doubleCapacity)));      
          final int capacity = capacity();
          this.computeMaxSize(capacity);
          this.computeNextAutoCompactionAmount(capacity);
        } else {
          ;
        }
      }
    

    BTW: Do you have a page listing all changes made to the Trove interface? The ensureCapacity is to be called now with the expected total size which is quite a change to the desired size on Trove.

    Michal

    enhancement question wontfix API & Naming 
    opened by frajt 3
  • Eliminate fillArray for ObjObj Map by using FREE=null

    Eliminate fillArray for ObjObj Map by using FREE=null

    Hi,

    we are using Trove library with some extra modifications where one of them is about null reference being used for the FREE value. The initial array creation and rehashing do not require to fill the array with FREE values (one extra iteration over the complete array is now required). Additionally it can be cheaper for a garbage collector to see a null reference than many references to a singleton object representing FREE - null should be for free. Obviously there is some cost in handling key slots where the new NULL object represents null key - could be reduced by noNull case (like you do with noRemoved).

    protected static final Object REMOVED = new Object(), FREE = null, NULL = new Object();
    

    Michal

    enhancement Performance & Benchmarks 
    opened by frajt 3
  • MutableQHashSeparateKVLongObjMap rehash BUG

    MutableQHashSeparateKVLongObjMap rehash BUG

    public static void rehashTest() { // RuntimeType:MutableQHashSeparateKVLongObjMap HashLongObjMap newMutableMap = HashLongObjMaps.getDefaultFactory() .withHashConfig(HashConfig.fromLoads(0.80, 0.89, 0.97).withGrowthFactor(1.15)) .newMutableMap(); newMutableMap.put(3, 3); newMutableMap.put(6, 6); newMutableMap.put(2, 2); newMutableMap.put(8, 8); newMutableMap.put(4, 4); newMutableMap.put(5, 5); newMutableMap.put(1, 1); System.out.println(newMutableMap.size()); // 7 System.out.println(newMutableMap); // {5=5, 6=6, 2=2, 1=1, 3=3, 8=8, 4=4} newMutableMap.remove(3); newMutableMap.remove(4); newMutableMap.remove(5); System.out.println(newMutableMap.shrink());### // TODO will trigger rehash and reset "removedSlots" flag, cause copy arrays error! System.out.println(newMutableMap.size()); // 4 System.out.println(newMutableMap); // {8=8, 2=2, 269536850670147407=null, 269536850670147407=null, 6=6, 269536850670147407=null, 1=1} }

    opened by tuhuiming 0
  • Could not initialize class com.koloboke.collect.map.hash.HashIntObjMaps$DefaultFactoryHolder

    Could not initialize class com.koloboke.collect.map.hash.HashIntObjMaps$DefaultFactoryHolder

    Using AdoptOpenJDK 11.0.5 First of all, is ver 1.0.0 compatible with JDK 11?

    We get a NoClassDefFoundError with below stacktrace:

    Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.koloboke.collect.map.hash.HashIntObjMaps$DefaultFactoryHolder at com.koloboke.collect.map.hash.HashIntObjMaps.getDefaultFactory(HashIntObjMaps.java:55) ~[koloboke-api-jdk8-1.0.0.jar:?] at com.koloboke.collect.map.hash.HashIntObjMaps.newImmutableMap(HashIntObjMaps.java:1499) ~[koloboke-api-jdk8-1.0.0.jar:?]

    Code that breaks: TreeMap<Integer, DeserializeAction> map = new TreeMap<>(); ..do stuff this.map = HashIntObjMaps.newImmutableMap(map); //this line throws exception

    The above code is initiated in a class in its initialization public class Myclass { DoStuff stuff = new DoStuff(); // triggers the above call to HashIntObjMaps.newImmutableMap(map);

    }

    We have following jar files in the classpath: koloboke-api-jdk8-1.0.0.jar koloboke-impl-common-jdk8-1.0.0.jar koloboke-impl-jdk8-1.0.0.jar

    We have not been able to reproduce it consequently, but seem to happen from time to time.

    opened by persal 1
  • Incorrect behavior of MutableLHashSeparateKVLongObjMapGO cursor

    Incorrect behavior of MutableLHashSeparateKVLongObjMapGO cursor

    Hello @leventov ,

    Found an issue when using the cursor to modify the MutableLHashSeparateKVLongObjMapGO map. The following snippet can be used to replicate the issue:

    `HashLongObjMap map = HashLongObjMaps.newMutableMap(); map.put(108, "null"); map.put(117, "null"); map.put(116, "null"); map.put(146, "null"); map.put(131, "null"); map.put(111, "null"); map.put(113, "null"); map.put(173, "null"); map.put(107, "null"); map.put(110, "null"); map.put(118, "null"); map.put(109, "null"); map.put(114, "null"); map.put(185, "null"); map.put(112, "null"); map.put(177, "null"); map.put(135, "null"); map.put(115, "null"); map.put(186, "null"); LongObjCursor cursor = map.cursor();

        while (cursor.moveNext()) {
            long key = cursor.key();
            if (key == 177 || key == 186 || key == 185 || key == 173) {
                cursor.remove();
            } else {
                cursor.setValue(String.valueOf(key));
            }
        }
    
        System.out.println(map);`
    

    The output is that key 109 has value "null" instead of "109".

    The version used is koloboke-impl-jdk8-1.0.0.

    opened by marespane 0
  • Koloboke annotation issue running Chronicle XCL tests...

    Koloboke annotation issue running Chronicle XCL tests...

    Hi Roman, et.al. I get this on Java SDK 8 ... any work arounds?

    Error:(18, 17) java: Koloboke Compile annotation processor threw an exception while processing this type,
      please report this on https://github.com/leventov/Koloboke/issues:
      zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.problem.AbortCompilation: Pb(324) The type java.util.function.BiFunction cannot be resolved. It is indirectly referenced from required .class files
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.problem.ProblemHandler.handle(ProblemHandler.java:159)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.problem.ProblemHandler.handle(ProblemHandler.java:226)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.problem.ProblemReporter.handle(ProblemReporter.java:2391)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.problem.ProblemReporter.isClassPathCorrect(ProblemReporter.java:4727)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.UnresolvedReferenceBinding.resolve(UnresolvedReferenceBinding.java:110)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.BinaryTypeBinding.resolveType(BinaryTypeBinding.java:212)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.PackageBinding.getTypeOrPackage(PackageBinding.java:186)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.CompilationUnitScope.findImport(CompilationUnitScope.java:466)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.CompilationUnitScope.findSingleImport(CompilationUnitScope.java:520)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.CompilationUnitScope.resolveSingleImport(CompilationUnitScope.java:733)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.Scope.getTypeOrPackage(Scope.java:3208)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.Scope.getType(Scope.java:2940)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.ast.ParameterizedSingleTypeReference.internalResolveLeafType(ParameterizedSingleTypeReference.java:187)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.ast.ParameterizedSingleTypeReference.internalResolveType(ParameterizedSingleTypeReference.java:165)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.ast.ParameterizedSingleTypeReference.resolveType(ParameterizedSingleTypeReference.java:378)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.ast.TypeReference.resolveType(TypeReference.java:582)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.SourceTypeBinding.resolveTypesFor(SourceTypeBinding.java:1906)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.SourceTypeBinding.methods(SourceTypeBinding.java:1516)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.ReferenceBinding.unResolvedMethods(ReferenceBinding.java:1857)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.MethodVerifier.computeInheritedMethods(MethodVerifier.java:539)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.MethodVerifier.computeInheritedMethods(MethodVerifier.java:511)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.MethodVerifier.verify(MethodVerifier.java:1009)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.MethodVerifier15.verify(MethodVerifier15.java:1005)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.MethodVerifier.verify(MethodVerifier.java:1019)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.SourceTypeBinding.verifyMethods(SourceTypeBinding.java:2604)
      	at zzz_koloboke_compile.shaded.org.$eclipse$.jdt.internal.compiler.lookup.CompilationUnitScope.verifyMethods(CompilationUnitScope.java:824)
      	at zzz_koloboke_compile.shaded.$spoon$.support.compiler.jdt.TreeBuilderCompiler.buildUnits(TreeBuilderCompiler.java:63)
      	at zzz_koloboke_compile.shaded.$spoon$.support.compiler.jdt.JDTBatchCompiler.getUnits(JDTBatchCompiler.java:193)
      	at zzz_koloboke_compile.shaded.$spoon$.support.compiler.jdt.JDTBasedSpoonCompiler.buildSources(JDTBasedSpoonCompiler.java:444)
      	at zzz_koloboke_compile.shaded.$spoon$.support.compiler.jdt.JDTBasedSpoonCompiler.build(JDTBasedSpoonCompiler.java:126)
      	at zzz_koloboke_compile.shaded.$spoon$.support.compiler.jdt.JDTBasedSpoonCompiler.build(JDTBasedSpoonCompiler.java:109)
      	at zzz_koloboke_compile.shaded.$spoon$.Launcher.buildModel(Launcher.java:686)
      	at com.koloboke.compile.processor.KolobokeCollectionProcessor$ProcessingCxt$GenerationCxt.parallelProcess(KolobokeCollectionProcessor.kt:741)
      	at com.koloboke.compile.processor.KolobokeCollectionProcessor$ProcessingCxt.parallelProcess(KolobokeCollectionProcessor.kt:358)
      	at com.koloboke.compile.processor.KolobokeCollectionProcessor$process$2$$special$$inlined$map$lambda$lambda$1.invoke(KolobokeCollectionProcessor.kt:215)
      	at com.koloboke.compile.processor.KolobokeCollectionProcessor$process$2$$special$$inlined$map$lambda$lambda$1.invoke(KolobokeCollectionProcessor.kt:79)
      	at com.koloboke.compile.processor.KolobokeCollectionProcessor.safeTypeProcess(KolobokeCollectionProcessor.kt:232)
      	at com.koloboke.compile.processor.KolobokeCollectionProcessor.access$safeTypeProcess(KolobokeCollectionProcessor.kt:79)
      	at com.koloboke.compile.processor.KolobokeCollectionProcessor$process$2$$special$$inlined$map$lambda$1.call(KolobokeCollectionProcessor.kt:214)
      	at com.koloboke.compile.processor.KolobokeCollectionProcessor$process$2$$special$$inlined$map$lambda$1.call(KolobokeCollectionProcessor.kt:79)
      	at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1447)
      	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
      	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
      	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
      	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
      	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
    
    
    
    opened by Cotton-Ben 1
  • java.lang.ClassCastException in very specific case on compiled ObjIntMap<String>.keySet().retainAll(...)

    java.lang.ClassCastException in very specific case on compiled ObjIntMap.keySet().retainAll(...)

    Please see the attached file for a test reproducing the behavior: objIntMapRetainAll.zip

    This problem happens when using Koloboke compile to implement a ObjIntMap<String> map, and then trying to remove elements using keySet().retainAll(...).

    This issue seems to happen when the first element of the backing array needs to be removed.

    In the generated source code, this block adds a tombstone element:

    if (indexToShift > indexToRemove) {
        firstDelayedRemoved = i;
        keys[indexToRemove] = XXX.Support.ObjHash.REMOVED;
        break closeDeletion;
    } 
    

    But then, the check on key existence for each iteration of the loop is:

    if ((key = ((String) (keys[i]))) != null) {
        ...
    }
    

    So when the check is executed against the REMOVED object, it fails with this exception:

    java.lang.ClassCastException: java.lang.Object cannot be cast to java.lang.String
    	at koloboke.KolobokeObjIntMapRetainAllTest_StringKeyObjIntMap.retainAll(KolobokeObjIntMapRetainAllTest_StringKeyObjIntMap.java:2120)
    	at koloboke.KolobokeObjIntMapRetainAllTest_StringKeyObjIntMap$KeyView.retainAll(KolobokeObjIntMapRetainAllTest_StringKeyObjIntMap.java:353)
    	at koloboke.ObjIntMapRetainAllTest.performTest(ObjIntMapRetainAllTest.java:35)
    	at koloboke.ObjIntMapRetainAllTest.testWithStringKey(ObjIntMapRetainAllTest.java:16)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
    	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
    	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
    

    The workaround, as demonstrated by the attached project, it just to use a generic ObjIntMap instead.

    opened by jmassenet 2
    Releases(compile-0.5.1)
    Owner
    Roman Leventov
    Software engineer and designer, author
    Roman Leventov
    High Performance data structures and utility methods for Java

    Agrona Agrona provides a library of data structures and utility methods that are a common need when building high-performance applications in Java. Ma

    Real Logic 2.4k Dec 31, 2022
    Java Collections till the last breadcrumb of memory and performance

    Koloboke A family of projects around collections in Java (so far). The Koloboke Collections API A carefully designed extension of the Java Collections

    Roman Leventov 967 Nov 14, 2022
    Java Collections till the last breadcrumb of memory and performance

    Koloboke A family of projects around collections in Java (so far). The Koloboke Collections API A carefully designed extension of the Java Collections

    Roman Leventov 967 Nov 14, 2022
    GS Collections has been migrated to the Eclipse Foundation, re-branded as Eclipse Collections. https://www.eclipse.org/collections/

    GS Collections is now Eclipse Collections We are pleased to announce that GS Collections has been migrated to the Eclipse Foundation, re-branded as Ec

    null 1.8k Dec 30, 2022
    GS Collections has been migrated to the Eclipse Foundation, re-branded as Eclipse Collections. https://www.eclipse.org/collections/

    GS Collections is now Eclipse Collections We are pleased to announce that GS Collections has been migrated to the Eclipse Foundation, re-branded as Ec

    null 1.8k Dec 30, 2022
    Eclipse Collections is a collections framework for Java with optimized data structures and a rich, functional and fluent API.

    English | 中文 | Deutsch | Español | Ελληνικά | Français | 日本語 | Norsk (bokmål) | Português-Brasil | Русский | हिंदी Eclipse Collections is a comprehens

    Eclipse Foundation 2.1k Jan 5, 2023
    Eclipse Collections is a collections framework for Java with optimized data structures and a rich, functional and fluent API.

    English | 中文 | Deutsch | Español | Ελληνικά | Français | 日本語 | Norsk (bokmål) | Português-Brasil | Русский | हिंदी Eclipse Collections is a comprehens

    Eclipse Foundation 2.1k Dec 29, 2022
    光 HikariCP・A solid, high-performance, JDBC connection pool at last.

    HikariCP It's Faster.Hi·ka·ri [hi·ka·'lē] (Origin: Japanese): light; ray. Fast, simple, reliable. HikariCP is a "zero-overhead" production ready JDBC

    Brett Wooldridge 17.7k Jan 1, 2023
    fasttuple - Collections that are laid out adjacently in both on- and off-heap memory.

    FastTuple Introduction There are lots of good things about working on the JVM, like a world class JIT, operating system threads, and a world class gar

    BMC TrueSight Pulse (formerly Boundary) 137 Sep 30, 2022
    You are missed at island with another 23 solider, loot chests and fight another opponents to survive, last man stands will win.

    SurvivalGames The best SurvivalGames project at the market Wiki • Report issue • Request feature About This Project For years ago, a lot of players wa

    ID2R Studios 10 Sep 8, 2022
    High Performance Primitive Collections for Java

    HPPC: High Performance Primitive Collections Collections of primitive types (maps, sets, stacks, lists) with open internals and an API twist (no java.

    Carrot Search 890 Dec 28, 2022
    Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-only access.

    Hollow Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-on

    Netflix, Inc. 1.1k Dec 25, 2022
    A Primitive Collection library that reduces memory usage and improves performance

    Primitive-Collections This is a Simple Primitive Collections Library i started as a hobby Project. It is based on Java's Collection Library and FastUt

    Speiger 26 Dec 25, 2022
    Operating Systems - Concepts of computer operating systems including concurrency, memory management, file systems, multitasking, performance analysis, and security. Offered spring only.

    Nachos for Java README Welcome to Nachos for Java. We believe that working in Java rather than C++ will greatly simplify the development process by p

    Sabir Kirpal 1 Nov 28, 2021
    The Java collections framework provides a set of interfaces and classes to implement various data structures and algorithms.

    Homework #14 Table of Contents General Info Technologies Used Project Status Contact General Information Homework contains topics: Sorting an ArrayLis

    Mykhailo 1 Feb 12, 2022
    Persistent (immutable) collections for Java and Kotlin

    What are Dexx Collections? Dexx Collections are a port of Scala's immutable, persistent collection classes to pure Java. Persistent in the context of

    Andrew O'Malley 208 Sep 30, 2022
    fastutil extends the Java Collections Framework by providing type-specific maps, sets, lists and queues.

    fastutil is a collection of type-specific Java classes that extend the Java Collections Framework by providing several containers, such as maps, sets, lists and prority queues, implementing the interfaces of the java.util package; it also provides big (64-bit) arrays, sets, lists, and fast, practical I/O classes for binary and text files.

    Sebastiano Vigna 1.5k Jan 7, 2023
    Annotation processor to create immutable objects and builders. Feels like Guava's immutable collections but for regular value objects. JSON, Jackson, Gson, JAX-RS integrations included

    Read full documentation at http://immutables.org // Define abstract value type using interface, abstract class or annotation @Value.Immutable public i

    Immutables 3.2k Dec 31, 2022
    A Persistent Java Collections Library

    PCollections A Persistent Java Collections Library Overview PCollections serves as a persistent and immutable analogue of the Java Collections Framewo

    harold cooper 708 Dec 28, 2022
    Ultra-fast SQL-like queries on Java collections

    CQEngine - Collection Query Engine CQEngine – Collection Query Engine – is a high-performance Java collection which can be searched with SQL-like quer

    Niall Gallagher 1.6k Dec 30, 2022