A Primitive Collection library that reduces memory usage and improves performance

Overview

build

Primitive-Collections

This is a Simple Primitive Collections Library i started as a hobby Project.
It is based on Java's Collection Library and FastUtil.
But its focus is a different one.

Main Features:

  • ArrayLists / LinkedLists
  • HashSets/Maps (Linked & HashControl)
  • TreeSets/Maps (RB & AVL)
  • EnumMaps
  • Immutable Maps/Lists/Sets
  • Priority Queues
  • Streams & Functional Queries
  • SplitIterators
  • Iterators
  • Pairs
  • Unary/Functions
  • Suppliers
  • Bi/Consumers

Current Level of Stability

Since this is a relatively new Library, stability was not perfect and some areas are not perfect yet.
Thanks to ben-manes we now have Roughly 160k unit test covering Maps/Sets/Lists.
These tests cover Javas Collection API completely and ensuring a Stable implementation.
These freshly added tests allowed me to squash thousands of issues according to Googles Test Library (Guava-Tests).
These will be expanded on as time goes on.

Benchmarks

Benchmarks can be found here: Click Me

Specialized Functions

New Specialized functions that were added to increase performance or reduce allocations or Quality Of life. To highlight things that may be wanted.

  • Iterable:
    • map/flatMap/arrayFlatMap: A Light weight version of Stream.map().
    • findFirst: Allows to find the first element of a Predicated Iterable.
    • filter: Allows to filter unwanted elements for wrapped Iterable
    • matchAny/matchNone/matchAll: Allows to find elements in a collection.
    • count: counts all valid elements in a collection.
    • forEach: Allows to input a second element into a forEach move allowing for more flexibility for Method References
    • reduce/limit/peek/distinct: Light Versions of the Stream variant, to reduce Stream usage.
    • pour: a function that allows to collect all elements within the Collection
  • Collection:
    • addAll: addAll array version
    • containsAny: Allows to test if another collection contains an of the elements of the tested collection.
    • primitiveStream: Provides access to the closest Java Stream Type.
    • copy: shallowCopies the collection, used instead of clone because this is better to use. (subCollections not supported for obvious reasons)
    • toArray: the ToArray function from Java9 and newer that uses a Functional interface and can use a method reference
  • List:
    • add/get/removeElements (From FastUtil): Allows to add/get/remove an Array into/from a list. Just with less overhead
    • extractElements: Allows to remove a Range of elements from the List and get what was removed.
    • Unstable Sort(From FastUtil): Uses a faster but not stable sort (Quick-Sort as example) to sort the list.
    • addIfAbsent/Present: adds a element only if absent/present in the list
    • swapRemove: deletes a desired element and inserts the last element in its place instead of leftshifting elements.
  • SortedSet:
    • addAndMoveToFirst/Last (From FastUtil but moved to Interface): Allows to add a element to the first/last position of a sorted set.
    • moveToFirst/Last: Moves the desired element at the first/last position of the SortedSet.
    • pollFirst/Last: Allows to poll the first/last element of the set.
  • Map:
    • putAll: putAll but in Array form.
    • putAllIfAbsent: Puts only the elements that are absent.
    • addTo (Only Primitives Values) (From FastUtil but moved to Interface): allows to add to the value of a given key. If not present it will be added. (Accumulator)
    • subFrom (Only Primitive Values): allows to subtract from the value of a given key. If value reached getDefaultReturnValue() element will be removed.
    • addToAll: Same as addTo but bulkVersion.
    • removeOrDefault: removes a Element and if not present returns the default value instead of the present value.
    • mergeAll: BulkVersion of Merge function.
    • supplyIfAbsent: A Supplier based computeIfAbsent
  • Sorted Map:
    • addAndMoveToFirst/Last (From FastUtil but moved to Interface): Allows to add a element to the first/last position of a sorted Map.
    • moveToFirst/Last: Moves the desired element at the first/last position of the Map.
    • getAndMoveToFirst/Last: gets the element and moves it to the first/last position. Replicating a Optional LinkedHashMap feature.
    • pollFirst/LastKey: Allows to poll the first/last element.
    • first/LastValue: Allows to get the first/last value from the Map.

Notes about Versions

Any 0.x.0 version (Minor) can be reason for massive changes including API.
To ensure that problems can be dealt with even if it is breaking the current API.
Any breaking changes will be Documented (once 1.0 is released)

Also to save space every 0.0.x (Patch) that is 2 Minor Versions behind will be removed. So if 0.5.0 is released every 0.3.x patch will be deleted, except for the last patch for that minor version.

How to install

Using Gradle:

repositories {
    maven {
        url = "https://maven.speiger.com/repository/main"
    }
}
dependencies {
	compile 'de.speiger:Primitive-Collections:0.5.3'
}

Direct:

Version Jar Sources Java Doc
0.5.3 Download Download Download
0.5.2 Download Download Download
0.5.1 Download Download Download
0.5.0 Download Download Download
0.4.5 Download Download Download
0.4.4 Download Download Download
0.4.3 Download Download Download
0.4.2 Download Download Download
0.4.1 Download Download Download
0.4.0 Download Download Download
0.3.6 Download Download Download

Contributing

If you want to contribute.
This project is created using gradle and java and my Template Library only. Nothing extra.
If you setup gradle the library will be downloaded automatically.

Where is everything stored?

  • Variables and ClassNames are define here
  • Templates are stored here
  • Tests can be found here

Please if you want to contribute follow the Rule-Sheet. It keeps everything in line.

How to Build

The SourceCode can be generated via:
/gradlew.bat generateSource

to build the jar:
/gradlew.bat build
do not combine the commands because they can not be executed at the same time.

Current Down Sides (Random order)

  • Documentation is only present at the lowest level for most cases and needs a typo fixing.
Comments
  • [Bugs] Guava test suits.

    [Bugs] Guava test suits.

    After finding @ben-manes FastUtil test suit using Guavas Test Library. I ran my library through the same tests and found loads of bugs... (56 in the example that was provided to FastUtil)

    So the goal is now to address these bugs before the next update and implement even more Test-cases.

    Tests that need to be done and completed:

    • [x] Maps

      • [x] HashMap,
      • [x] LinkedHashMap
      • [x] CustomHashMap
      • [x] LinkedCustomHashMap
      • [X] ImmutableHashMap
      • [x] ArrayMap
      • [x] EnumMap
      • [x] LinkedEnumMap
      • [x] RBTreeMap
      • [x] AVLTreeMap
      • [x] SortedMap
      • [x] NavigableMap
    • [x] Lists:

      • [x] ArrayList
      • [x] LinkedList
      • [x] ImmutableList
    • [x] Sets

      • [x] HashSet
      • [x] LinkedHashSet
      • [x] CustomHashSet
      • [x] LinkedCustomHashSet
      • [x] ImmutableHashSet
      • [x] ArraySet
      • [x] RBTreeSet
      • [x] AVLTreeSet
      • [x] SortedSet
      • [x] NavigableSet

    Thanks @ben-manes for bringing that to the attention. I know I kinda leach of fastutils work here but If you are willing I would love to add even more testsuits to my library to reduce bugs.

    opened by Speiger 17
  • Expanding Test Suite

    Expanding Test Suite

    Right now I am using Guavas Test-suite to ensure the quality of Primitive Collections. But that right now only covers Javas Collection functions and only generics. While Generics delegate to primitives, this isn't perfect either. Since we have 7 implementations and a lot of custom functions, that means we don't have the coverage that I would like.

    The end goal would be to have a Library like guavas Test-suite but for primitive collections. Where a lot of features are supported and adding to that would be a lot easier. This will most likely be its own library with maybe a dependency on Primitive Collections.

    How would we get there is now the question.

    Fairly easily actually. Since Guava provides a lot of the base that we actually want, we can start with it. Then we just append our features on top of that. Because we want to keep the Java Class Type to primitive type checks anyways that would ensure a Dual Check System.

    Now since that still is a lot of work this isn't going to be completed anytime soon. But it is here and it allows me to document progress properly.

    So lets throw in a todo-list of what needs to be done.

    • [x] Design a Framework that runs on top of Guavas Test Suite or makes use of it.
    • [x] Write the designed Framework
      • [x] Collections
      • [x] Collections Template
      • [x] Lists
      • [x] Lists Template
      • [x] Sets
      • [x] Sets Template
      • [x] Queues
      • [x] Queues Template
      • [x] Maps
      • [x] Maps Template
      • [x] Extras
      • [x] Extras Template

    If anyone wants to help with this you are welcome to do so. Even with providing inputs or Sources how to improve this even more.

    enhancement help wanted 
    opened by Speiger 14
  • Better Name suggestion

    Better Name suggestion

    When I created this Library this was only a "Temporary" name that kinda became final...

    I am not perfectly happy with that name but I keeping it until there is a better one found... Since people will most likely ask if I could rename it:

    Throw in suggestions, If I like it I will most likely change to it.

    Thanks for reading.

    enhancement help wanted 
    opened by Speiger 3
  • Modularization of the Library (In Progress)

    Modularization of the Library (In Progress)

    The Library is really big, 25MB iirc, but this just limits a lot of things if you just need 1 part. So i could just either make module releases and have at it, or provide the developers ways to just trim everything they don't need.

    So the goal is implement a Module System where features can be enabled/disabled as needed. To what extend this should be possible isn't clear yet, but having the option to just disable implementations or might be really useful. Especially if you only need 1 map implementation but don't want to have the other 1000 that exist this can trim down everything a lot.

    In the result this will increase template complexity a bit, but not by too much... Allow for what is needed instead of everything.

    Edit lets add some progress information. I have to work from top down, and I won't be able to disable "Sets" when you want "Maps". But you can disable Set implementations at the very least if you want "Maps" Progress:

    • [x] Base
    • [X] Async
    • [x] Pairs
    • [x] Collections
    • [x] Lists
    • [x] Queues
    • [x] Sets
    • [x] Functions
    • [X] Maps
    documentation enhancement Fixed 
    opened by Speiger 2
  • PollLast might not work properly (needs confirmation)

    PollLast might not work properly (needs confirmation)

    As Reported to FastUtil by @Z-Kris FastUtils HashSet removeLast (or pollLast in this library) has a issue with iterators afterwards. I am pretty sure I caught that in my Unit test implementations but I would rather double check this myself, to make sure it is fixed.

    If it is a issue this will be addressed with the 0.7.1 or 0.8.0 patch (the next patch might not be 0.7.1, but 0.8.0).

    Also @Z-Kris thanks for the report.

    invalid 
    opened by Speiger 1
  • putIfAbsent doesn't replace

    putIfAbsent doesn't replace "defaultValues"

    Reference to the google issue. https://github.com/google/guava/issues/6217

    @ben-manes found a bug that putIfAbsent doesn't replace null (default) values. Thank you buddy, I will address that soon. I am almost done with my other project cleanup. (6/12 projects are done and the other half should be completed this week)

    Fixed 
    opened by Speiger 0
  • [Java17] Add Support for custom Random Generators

    [Java17] Add Support for custom Random Generators

    Since java17, there is now a RandomGenerator Interface, meaning that custom Random Generators can be created. Since the normal random Generator isn't thread safe and seems to be outdated there should be support for it. Maybe add a wrapper for older java versions.

    That isn't sure. But since i am able to detect these things it would be good to produce them. Though for releases i am not sure how to handle this. Since this kinda creates a compilation split...

    Have to look into that. But forwards compat should be also a thing as backwards compat... (To some degree)

    enhancement Fixed 
    opened by Speiger 0
  • Optimize Empty Version ToArray

    Optimize Empty Version ToArray

    as reported to @amaembo to FastUtil. If the inputted array is null and the collection is empty the outputted array should not create a new array but use a pre-allocated empty array.

    Might be worth locking into.

    enhancement Fixed 
    opened by Speiger 0
  • Functional interface overhaul

    Functional interface overhaul

    Since this is technically a recreation of FastUtil, this means I also copied some of its flaws. The functional interface part of Primitive Collections is kinda a mess.

    When I copied and redesigned it to be less clutter (I removed a lot of functions FastUtil had) I still left the messy design in it. To give a example.

    Primitive Collections adds Consumers, but not ObjectConsumers since java has them already, but IntConsumers exist and extend of javas IntConsumer. Also imports of Javas Consumer is a special case that needs to be accounted for. Functions duplicate for some cases because there exists 2 consumers.

    And Function.class has the same problem where it has already duplicates and the function also renames the function you are going to call or delegates it. Just for the Map Interface which extends it.

    For this the whole Functional interfaces kinda need a major overhaul.

    This only breaks code if you specifically use said functional interfaces directly. Not even in lambdas or method references. And maps are free from that too.

    While I don't have right now no real solution for that. This needs to be addressed. Preferably with the next minor release. (Since I treat them as Major releases until 1.0)

    enhancement Fixed 
    opened by Speiger 0
  • ConcurrentHashMaps

    ConcurrentHashMaps

    Since i am right now working on that it should maybe its own card.

    Some details for anyone interested. I am not using Javas implementation of ConcurrentMaps due to the nature of the implementation. I use Guavas approach instead.

    The approach is not as good as Javas because in Javas case you can have as many threads as you want access the Map and only if the same node is tried to be accessed is there a synchronization happening, while guavas approach is a pool system where the amount of concurrency you define creates that amount of pools and each pool can only be accessed by 1 thread at a time. This can/will lead up to a bit more synchronization but if you only work with a couple threads which is very likely you barely will feel that especially if your concurrency level is high enough.

    opened by Speiger 0
  • [Feature] Add Jitpack Support

    [Feature] Add Jitpack Support

    Maven central is great, but Jitpack seems to be suiting me a lot better for what I want from a Maven source. So I am planning to add jitpack support soon.

    opened by Speiger 0
Releases(0.8.0)
  • 0.8.0(Dec 20, 2022)

    0.8.0 Release

    • Added: ISizeProvider interface (Optimization Helper)
    • Added: ISizeProvider into most Iterable implementations (Distinct/Filter/FlatMap/ArrayFlatMap don't support it, for obvious reasons)
    • Added: ToArray function into Iterable which uses ISizeProvider to reduce overhead of duplicating arrays.
    • Added: Functions that have the same type, Int2IntFunction as example, have now a identity function.
    • Added: Functions of a BooleanValue have now alwaysTrue/False function.
    • Added: ForEachIndexed for all Iterable implementations
    • Added: RandomGenerator support (Java17), though requires self compilation
    • Added: Optimizations for HashUtils next power of function.
    • Added: toArray() now returns a cached empty array if the collection is empty.
    • Added: toArray function for AsyncBuilder
    • Added: Modularization to the library where feature can be disabled as needed. (Requires Self-Compilation)
    • Fixed: putIfAbsent now replaces defaultValues
    • Fixed: OpenHashSet/Map and their Custom Variants no longer rely on List implementations.
    • Fixed: ObjectCopyOnWriteList.of did create a ObjectArrayList instead of the CopyOnWrite variant.
    • Removed: BooleanSet and Maps that start with a Boolean classes since they can not be used anyways.
    • Breaking Change: Function classes now use the "apply/applyAs/test" format from Java itself, instead of the "get" format. This cleans up a lot of things. But will break existing function class implementations
    • Breaking Change: Classes that used PrimitiveCollection functions now default to java functions where applicable, this is to increase compat.
    • Breaking Change: Some function classes now get closer to javas terms. (Predicate/UnaryOperator etc)
    Source code(tar.gz)
    Source code(zip)
Owner
Speiger
Speiger
Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-only access.

Hollow Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-on

Netflix, Inc. 1.1k Dec 25, 2022
Java Collections till the last breadcrumb of memory and performance

Koloboke A family of projects around collections in Java (so far). The Koloboke Collections API A carefully designed extension of the Java Collections

Roman Leventov 967 Nov 14, 2022
Library for creating In-memory circular buffers that use direct ByteBuffers to minimize GC overhead

Overview This project aims at creating a simple efficient building block for "Big Data" libraries, applications and frameworks; thing that can be used

Tatu Saloranta 132 Jul 28, 2022
fasttuple - Collections that are laid out adjacently in both on- and off-heap memory.

FastTuple Introduction There are lots of good things about working on the JVM, like a world class JIT, operating system threads, and a world class gar

BMC TrueSight Pulse (formerly Boundary) 137 Sep 30, 2022
Immutable in-memory R-tree and R*-tree implementations in Java with reactive api

rtree In-memory immutable 2D R-tree implementation in java using RxJava Observables for reactive processing of search results. Status: released to Mav

Dave Moten 999 Dec 20, 2022
External-Memory Sorting in Java

Externalsortinginjava External-Memory Sorting in Java: useful to sort very large files using multiple cores and an external-memory algorithm. The vers

Daniel Lemire 235 Dec 29, 2022
Lightning Memory Database (LMDB) for Java: a low latency, transactional, sorted, embedded, key-value store

LMDB for Java LMDB offers: Transactions (full ACID semantics) Ordered keys (enabling very fast cursor-based iteration) Memory-mapped files (enabling o

null 680 Dec 23, 2022
jproblemgenerator creates scenarios in which Java programs leak memory or crash the JVM

jproblemgenerator creates scenarios in which Java programs leak memory or crash the JVM. It is intended to train the use of debugging tools

null 1 Jan 6, 2022
A high performance caching library for Java

Caffeine is a high performance, near optimal caching library. For more details, see our user's guide and browse the API docs for the latest release. C

Ben Manes 13k Jan 5, 2023
Table-Computing (Simplified as TC) is a distributed light weighted, high performance and low latency stream processing and data analysis framework. Milliseconds latency and 10+ times faster than Flink for complicated use cases.

Table-Computing Welcome to the Table-Computing GitHub. Table-Computing (Simplified as TC) is a distributed light weighted, high performance and low la

Alibaba 34 Oct 14, 2022
High Performance data structures and utility methods for Java

Agrona Agrona provides a library of data structures and utility methods that are a common need when building high-performance applications in Java. Ma

Real Logic 2.5k Jan 5, 2023
Replicate your Key Value Store across your network, with consistency, persistance and performance.

Chronicle Map Version Overview Chronicle Map is a super-fast, in-memory, non-blocking, key-value store, designed for low-latency, and/or multi-process

Chronicle Software : Open Source 2.5k Dec 29, 2022
High performance Java implementation of a Cuckoo filter - Apache Licensed

Cuckoo Filter For Java This library offers a similar interface to Guava's Bloom filters. In most cases it can be used interchangeably and has addition

Mark Gunlogson 161 Dec 30, 2022
Simple Binary Encoding (SBE) - High Performance Message Codec

Simple Binary Encoding (SBE) SBE is an OSI layer 6 presentation for encoding and decoding binary application messages for low-latency financial applic

Real Logic 2.8k Dec 28, 2022
A Java library for quickly and efficiently parsing and writing UUIDs

fast-uuid fast-uuid is a Java library for quickly and efficiently parsing and writing UUIDs. It yields the most dramatic performance gains when compar

Jon Chambers 142 Jan 1, 2023
A fork of Cliff Click's High Scale Library. Improved with bug fixes and a real build system.

High Scale Lib This is Boundary's fork of Cliff Click's high scale lib. We will be maintaining this fork with bug fixes, improvements and versioned bu

BMC TrueSight Pulse (formerly Boundary) 402 Jan 2, 2023
LWJGL is a Java library that enables cross-platform access to popular native APIs useful in the development of graphics (OpenGL, Vulkan), audio (OpenAL), parallel computing (OpenCL, CUDA) and XR (OpenVR, LibOVR) applications.

LWJGL - Lightweight Java Game Library 3 LWJGL (https://www.lwjgl.org) is a Java library that enables cross-platform access to popular native APIs usef

Lightweight Java Game Library 4k Dec 29, 2022
A modern I/O library for Android, Kotlin, and Java.

Okio See the project website for documentation and APIs. Okio is a library that complements java.io and java.nio to make it much easier to access, sto

Square 8.2k Dec 31, 2022