Microstream - High-Performance Java-Native-Persistence

Related tags

Database java storage-engine serializer cache persistence in-memory-storage in-memory-database object-graph
Overview

GitHub GitHub issues Maven Central Twitter Follow

High-Performance Java-Native-Persistence

Store and load any Java Object Graph or Subgraphs partially, Relieved of Heavy-weight JPA. Microsecond Response Time. Ultra-High Throughput. Minimum of Latencies. Create Ultra-Fast In-Memory Database Applications & Microservices.

License

MicroStream is available under Eclipse Public License - v 2.0.

Documentation

Build

To build MicroStream you need Java 8 and Maven.

Just run

mvn install

Contribute

If you want to contribute to MicroStream, please read our guidelines.

Links

Comments
  • store()/storeAll() causes

    store()/storeAll() causes "Storage is shut down." exception when garbage collection is involved

    Environment Details

    • MicroStream Version: 05.00.02-MS-GA

    Describe the bug

    Since I synchronize larger amounts of data at night, I activated the MicroStream GarbageCollector, carried out the GarbageCollection and deactivated the MicroStream GarbageCollector again. After an indefinite period of time, the already known problem with store () / storeAll () and the exception "Storage is shut down" occurs. This also occurs after restarting the application and without a shutdown of the storage that I initiated or intended. No exceptions were shown to me while starting the storage. Reading the db is still partially possible.

    I noticed that if this exception occurred when saving to the storage and then the application is started without the following garbage collection, then the storage can be used again without problems, i.e. store () / storeAll () is also possible again.

    StorageEntityCache.Default.setGarbageCollectionEnabled (true);
STORAGE.issueFullGarbageCollection ();
StorageEntityCache.Default.setGarbageCollectionEnabled (false);

    However, if the GarbageCollection is activated again the next time it is started, the store () / storeAll () problem occurs again.

    Are there possibilities / methods to better document the start, the behavior or the state of the storage or the GC, or to receive specific outputs? Could you give me some code examples in this regard? I might be able to provide further / helpful information to solve this issue. It would also be possible for me to create a ThreadDump in the db after the unsuccessful write attempt, shortly after the db started.

    I've attached the exceptions and the code for starting and shutting down.

    Output Console/Log (Update of the user via store (...)):

one.microstream.persistence.exceptions.PersistenceExceptionTransfer
   at one.microstream.storage.embedded.types.EmbeddedStorageBinaryTarget$Default.write(EmbeddedStorageBinaryTarget.java:90)
   at one.microstream.storage.embedded.types.EmbeddedStorageBinaryTarget$Default.write(EmbeddedStorageBinaryTarget.java:1)
   at one.microstream.persistence.binary.types.BinaryStorer$Default.commit(BinaryStorer.java:501)
   at one.microstream.persistence.types.PersistenceManager$Default.store(PersistenceManager.java:298)
   at one.microstream.storage.types.StorageConnection.store(StorageConnection.java:367)
   at de.ui.beans.RegisterOrLogin.login(RegisterOrLogin.java:162)
   at de.ui.beans.RegisterOrLogin.txtLoginPassword_onKeyDown(RegisterOrLogin.java:298)
   at com.vaadin.flow.component.ComponentEventBus.fireEventForListener(ComponentEventBus.java:205)
   at com.vaadin.flow.component.ComponentEventBus.handleDomEvent(ComponentEventBus.java:373)
   at com.vaadin.flow.component.ComponentEventBus.lambda$addDomTrigger$dd1b7957$1(ComponentEventBus.java:264)
   at com.vaadin.flow.internal.nodefeature.ElementListenerMap.lambda$fireEvent$2(ElementListenerMap.java:441)
   at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
   at com.vaadin.flow.internal.nodefeature.ElementListenerMap.fireEvent(ElementListenerMap.java:441)
   at com.vaadin.flow.server.communication.rpc.EventRpcHandler.handleNode(EventRpcHandler.java:59)
   at com.vaadin.flow.server.communication.rpc.AbstractRpcInvocationHandler.handle(AbstractRpcInvocationHandler.java:64)
   at com.vaadin.flow.server.communication.ServerRpcHandler.handleInvocationData(ServerRpcHandler.java:409)
   at com.vaadin.flow.server.communication.ServerRpcHandler.lambda$handleInvocations$1(ServerRpcHandler.java:390)
   at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
   at com.vaadin.flow.server.communication.ServerRpcHandler.handleInvocations(ServerRpcHandler.java:390)
   at com.vaadin.flow.server.communication.ServerRpcHandler.handleRpc(ServerRpcHandler.java:317)
   at com.vaadin.flow.server.communication.UidlRequestHandler.synchronizedHandleRequest(UidlRequestHandler.java:89)
   at com.vaadin.flow.server.SynchronizedRequestHandler.handleRequest(SynchronizedRequestHandler.java:40)
   at com.vaadin.flow.server.VaadinService.handleRequest(VaadinService.java:1547)
   at com.vaadin.flow.server.VaadinServlet.service(VaadinServlet.java:247)
   at com.rapidclipse.framework.server.RapServlet.service(RapServlet.java:159)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:228)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
   at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
   at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
   at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:687)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
   at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382)
   at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
   at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
   at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1723)
   at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: one.microstream.storage.exceptions.StorageExceptionNotRunning: Storage is shut down.
   at one.microstream.storage.types.StorageTaskBroker$Default.enqueueTask(StorageTaskBroker.java:212)
   at one.microstream.storage.types.StorageTaskBroker$Default.enqueueTask(StorageTaskBroker.java:197)
   at one.microstream.storage.types.StorageTaskBroker$Default.enqueueTaskAndNotifyAll(StorageTaskBroker.java:187)
   at one.microstream.storage.types.StorageTaskBroker$Default.enqueueStoreTask(StorageTaskBroker.java:396)
   at one.microstream.storage.types.StorageRequestAcceptor$Default.storeData(StorageRequestAcceptor.java:177)
   at one.microstream.storage.embedded.types.EmbeddedStorageBinaryTarget$Default.write(EmbeddedStorageBinaryTarget.java:86)
   ... 47 more

    Output Console/Log (Update of the user via storeAll (...)):

one.microstream.persistence.exceptions.PersistenceExceptionTransfer
   at one.microstream.storage.embedded.types.EmbeddedStorageBinaryTarget$Default.write(EmbeddedStorageBinaryTarget.java:90)
   at one.microstream.storage.embedded.types.EmbeddedStorageBinaryTarget$Default.write(EmbeddedStorageBinaryTarget.java:1)
   at one.microstream.persistence.binary.types.BinaryStorer$Default.commit(BinaryStorer.java:501)
   at one.microstream.persistence.types.PersistenceManager$Default.storeAll(PersistenceManager.java:316)
   at one.microstream.storage.types.StorageConnection.storeAll(StorageConnection.java:376)
   at de.dao.UserDAO.update(UserDAO.java:131)
   at de.ui.beans.CartDialog.lambda$setContent$0(CartDialog.java:113)
   at de.ui.bits.CartProductBox.btnRemove_onClick(CartProductBox.java:267)
   at com.vaadin.flow.component.ComponentEventBus.fireEventForListener(ComponentEventBus.java:205)
   at com.vaadin.flow.component.ComponentEventBus.handleDomEvent(ComponentEventBus.java:373)
   at com.vaadin.flow.component.ComponentEventBus.lambda$addDomTrigger$dd1b7957$1(ComponentEventBus.java:264)
   at com.vaadin.flow.internal.nodefeature.ElementListenerMap.lambda$fireEvent$2(ElementListenerMap.java:441)
   at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
   at com.vaadin.flow.internal.nodefeature.ElementListenerMap.fireEvent(ElementListenerMap.java:441)
   at com.vaadin.flow.server.communication.rpc.EventRpcHandler.handleNode(EventRpcHandler.java:59)
   at com.vaadin.flow.server.communication.rpc.AbstractRpcInvocationHandler.handle(AbstractRpcInvocationHandler.java:64)
   at com.vaadin.flow.server.communication.ServerRpcHandler.handleInvocationData(ServerRpcHandler.java:409)
   at com.vaadin.flow.server.communication.ServerRpcHandler.lambda$handleInvocations$1(ServerRpcHandler.java:390)
   at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
   at com.vaadin.flow.server.communication.ServerRpcHandler.handleInvocations(ServerRpcHandler.java:390)
   at com.vaadin.flow.server.communication.ServerRpcHandler.handleRpc(ServerRpcHandler.java:317)
   at com.vaadin.flow.server.communication.UidlRequestHandler.synchronizedHandleRequest(UidlRequestHandler.java:89)
   at com.vaadin.flow.server.SynchronizedRequestHandler.handleRequest(SynchronizedRequestHandler.java:40)
   at com.vaadin.flow.server.VaadinService.handleRequest(VaadinService.java:1547)
   at com.vaadin.flow.server.VaadinServlet.service(VaadinServlet.java:247)
   at com.rapidclipse.framework.server.RapServlet.service(RapServlet.java:159)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:228)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
   at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)
   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
   at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
   at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:687)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
   at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382)
   at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
   at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
   at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1723)
   at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: one.microstream.storage.exceptions.StorageExceptionNotRunning: Storage is shut down.
   at one.microstream.storage.types.StorageTaskBroker$Default.enqueueTask(StorageTaskBroker.java:212)
   at one.microstream.storage.types.StorageTaskBroker$Default.enqueueTask(StorageTaskBroker.java:197)
   at one.microstream.storage.types.StorageTaskBroker$Default.enqueueTaskAndNotifyAll(StorageTaskBroker.java:187)
   at one.microstream.storage.types.StorageTaskBroker$Default.enqueueStoreTask(StorageTaskBroker.java:396)
   at one.microstream.storage.types.StorageRequestAcceptor$Default.storeData(StorageRequestAcceptor.java:177)
   at one.microstream.storage.embedded.types.EmbeddedStorageBinaryTarget$Default.write(EmbeddedStorageBinaryTarget.java:86)
   ... 48 more

    Database.java

public class Database {
    private static Logger logger = LoggerFactory.getLogger(Database.class);

	public static final Reference<Root> ROOT = X.Reference(new Root());
	public static final EmbeddedStorageManager STORAGE;
	private static final Path PATH;

	static {
		PATH = Paths.get(System.getProperty("user.home"), "MicroStream", "Shop");

		try {
			if (!PATH.toFile().exists()) {
				Files.createDirectory(Database.PATH);
				logger.info("MS-DB: No existing path to DB-files found. Path was created.");
			} else {
				logger.info("MS-DB: Already existing path to DB-files found.");		
			}
		} catch (final IOException e) {
			e.printStackTrace();
		}

        STORAGE = EmbeddedStorage.start(Database.ROOT, Database.PATH);
        StorageEntityCache.Default.setGarbageCollectionEnabled(true);
        storage.issueFullGarbageCollection();

        // As a precaution, the GC is deactivated again so that there is no risk of the shop being shut down during garbage collection, which could result in inconsistencies in the DB.
		StorageEntityCache.Default.setGarbageCollectionEnabled(false);
    }
}

    MicroStreamContextDestroyer.java

@WebListener
public class MicroStreamContextDestroyer implements ServletContextListener {
	private static Logger logger = LoggerFactory.getLogger(MicroStreamContextDestroyer.class);

	@Override
	public void contextInitialized(final ServletContextEvent sce) {
	}

	@Override
	public void contextDestroyed(final ServletContextEvent sce) {
		logger.info("MicroStreamContextDestroyer: Destruction of context started.");
		
		logger.info("MS-DB STORAGE Shutdown: Check MS-DB STORAGE for shutdown.");
		if (Database.STORAGE != null) {
			if (Database.STORAGE.shutdown()) {
				logger.info("MS-DB STORAGE Shutdown: All active threads that manage the MicroStream memory have been shut down successfully.");
			} else {
				logger.error("MS-DB STORAGE Shutdown: An internal InterruptedException occurred during the shutdown of all active threads that manage the MicroStream memory.");
			}
		} else {
			logger.info("MS-DB Shutdown: No started MS-DB found to shut down.");
		}
		
		// Stop the LazyReference Manager to prevent memory leaks.
		LazyReferenceManager lazyReferenceManager = LazyReferenceManager.get();
		if (lazyReferenceManager != null) {
			lazyReferenceManager.stop();
			logger.info("MS-DB LazyReferenceManager Shutdown: LazyReferenceManager triggered to stop.");
		} else {
			logger.error("MS-DB LazyReferenceManager Shutdown: No LazyReferenceManager found to stop.");
		}
		logger.info("MicroStreamContextDestroyer: Destruction of context finished.");
	}
}

    Additional context

    • This issue has existed since version 04.01.00-MS-GA.
    • No lazy references are used.
    bug 
    opened by MutenRoshi84 18
  • Paths with leading separator not handled correctly

    Paths with leading separator not handled correctly

    Edit TM: this issue turned out to actually be about leading separators in paths. See below

    Configuration - setBaseDirectory() has no effect.

    Project: Microstream-test branch: master test: microstream-test/intern/src/main/java/test/java/microstream/configuration/baseDirectoryTest.java

    test:

    class BaseDirectoryTest {

    @TempDir
    Path location;

    Path configFilePath;


    @Test
    void baseDirectoryTest() throws IOException {
        configFilePath = location.resolve("baseDirectory.ini");
        FileUtils.writeStringToFile(configFilePath.toFile(),"baseDirectory = "+ location.toString(), "UTF-8");

        Customer customer = CustomerGenerator.generateNewCustomer();

        Configuration configuration = Configuration.LoadIni(configFilePath);

        EmbeddedStorageManager storageManager = configuration.createEmbeddedStorageFoundation().createEmbeddedStorageManager(customer).start();

        List<File> files = (List<File>) FileUtils.listFiles(location.toFile(), null, true);

        assertTrue(files.size() > 2,files.toString());

        storageManager.shutdown();
    }
}

    image

    opened by zdenek-jonas 12
  • IndexOutOfBounds when storing with channel count >= 2 (more likely with higher channel count)

    IndexOutOfBounds when storing with channel count >= 2 (more likely with higher channel count)

    Tested on current master (80fc56f5d88dc743f1aae3cc86ecdec5dcfbebec)

    Whenever I start the storage with backup enabled and a channel count of 2, the first start has no problems. But when I start the application again, it will give me an IndexOutOfBounds when trying to copy a channel file to the backup location. This doesn't happen when the channel count is 1.

    If the channel count is greater or equal to 4 then I always get this exception on the first store.

    This is my code:

    final List<String> root = new ArrayList<>();
final NioFileSystem fs = NioFileSystem.New();

System.out.println("Creating and starting...");
final EmbeddedStorageManager x = EmbeddedStorage.Foundation()
	.setConfiguration(
		StorageConfiguration.Builder()
			.setChannelCountProvider(StorageChannelCountProvider.New(4))
			.setBackupSetup(
				StorageBackupSetup.New(
					StorageBackupFileProvider.Builder(fs)
						.setDirectory(fs.ensureDirectoryPath("backup"))
						.createFileProvider()
				)
			)
			.createConfiguration()
	)
	.setRoot(root)
	.createEmbeddedStorageManager()
	.start();

root.add("HEHE");

System.out.println("Storing...");
x.storeRoot();

Thread.sleep(1000);
System.out.println("Shutting down...");
x.shutdown();

    (The sleep is there so the exception output doesn't overlap with the exception message that comes from the shutdown)

    Exception when starting with channel count of 4:

    Exception in thread "MicroStream-StorageChannel-3" Exception in thread "MicroStream-StorageChannel-0" Exception in thread "MicroStream-StorageChannel-1" Exception in thread "MicroStream-StorageChannel-2" one.microstream.storage.exceptions.StorageExceptionDisruptingExceptions: Disruptions: {one.microstream.storage.exceptions.StorageExceptionBackupCopying:storage/channel_3/channel_3_1.dat@220+0 -> Pommes/channel_3/channel_3_1.dat,}
	at one.microstream.storage.types.StorageOperationController$Default.checkProcessingEnabled(StorageOperationController.java:119)
	at one.microstream.storage.types.StorageChannel$Default.work(StorageChannel.java:308)
	at one.microstream.storage.types.StorageChannel$Default.run(StorageChannel.java:385)
	at java.base/java.lang.Thread.run(Thread.java:834)
one.microstream.storage.exceptions.StorageExceptionDisruptingExceptions: Disruptions: {one.microstream.storage.exceptions.StorageExceptionBackupCopying:storage/channel_3/channel_3_1.dat@220+0 -> Pommes/channel_3/channel_3_1.dat,}
	at one.microstream.storage.types.StorageOperationController$Default.checkProcessingEnabled(StorageOperationController.java:119)
	at one.microstream.storage.types.StorageChannel$Default.work(StorageChannel.java:308)
	at one.microstream.storage.types.StorageChannel$Default.run(StorageChannel.java:385)
	at java.base/java.lang.Thread.run(Thread.java:834)
one.microstream.storage.exceptions.StorageExceptionDisruptingExceptions: Disruptions: {one.microstream.storage.exceptions.StorageExceptionBackupCopying:storage/channel_3/channel_3_1.dat@220+0 -> Pommes/channel_3/channel_3_1.dat,}
	at one.microstream.storage.types.StorageOperationController$Default.checkProcessingEnabled(StorageOperationController.java:119)
	at one.microstream.storage.types.StorageChannel$Default.work(StorageChannel.java:308)
	at one.microstream.storage.types.StorageChannel$Default.run(StorageChannel.java:385)
	at java.base/java.lang.Thread.run(Thread.java:834)
one.microstream.storage.exceptions.StorageExceptionDisruptingExceptions: Disruptions: {one.microstream.storage.exceptions.StorageExceptionBackupCopying:storage/channel_3/channel_3_1.dat@220+0 -> Pommes/channel_3/channel_3_1.dat,}
	at one.microstream.storage.types.StorageOperationController$Default.checkProcessingEnabled(StorageOperationController.java:119)
	at one.microstream.storage.types.StorageChannel$Default.work(StorageChannel.java:308)
	at one.microstream.storage.types.StorageChannel$Default.run(StorageChannel.java:385)
	at java.base/java.lang.Thread.run(Thread.java:834)
Exception in thread "MicroStream-StorageBackupHandler" one.microstream.storage.exceptions.StorageExceptionBackupCopying: storage/channel_3/channel_3_1.dat@220+0 -> Pommes/channel_3/channel_3_1.dat
	at one.microstream.storage.types.StorageBackupHandler$Default.copyFilePart(StorageBackupHandler.java:471)
	at one.microstream.storage.types.StorageBackupHandler$Default.copyFilePart(StorageBackupHandler.java:485)
	at one.microstream.storage.types.StorageBackupItemQueue$Default$Item.processBy(StorageBackupItemQueue.java:178)
	at one.microstream.storage.types.StorageBackupItemQueue$Default.processNextItem(StorageBackupItemQueue.java:109)
	at one.microstream.storage.types.StorageBackupHandler$Default.run(StorageBackupHandler.java:212)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: one.microstream.storage.exceptions.StorageExceptionBackupCopying: storage/channel_3/channel_3_1.dat@220+0 -> Pommes/channel_3/channel_3_1.dat
	at one.microstream.storage.types.StorageBackupHandler$Default.copyFilePart(StorageBackupHandler.java:462)
	... 5 more
Caused by: one.microstream.storage.exceptions.StorageExceptionIoReading
	at one.microstream.storage.types.StorageFileEntityDataIterator$Default.fillBuffer(StorageFileEntityDataIterator.java:174)
	at one.microstream.storage.types.StorageFileEntityDataIterator$Internal.iterateEntityData(StorageFileEntityDataIterator.java:45)
	at one.microstream.storage.types.StorageDataFileValidator$Default.validateFile(StorageDataFileValidator.java:112)
	at one.microstream.storage.types.StorageBackupHandler$Default.copyFilePart(StorageBackupHandler.java:457)
	... 5 more
Caused by: one.microstream.storage.exceptions.StorageExceptionIoReading
	at one.microstream.storage.types.StorageFile$Abstract.readBytes(StorageFile.java:149)
	at one.microstream.storage.types.StorageFileEntityDataIterator$Default.fillBuffer(StorageFileEntityDataIterator.java:170)
	... 8 more
Caused by: one.microstream.exceptions.IndexBoundsException: Index out of bounds: 220 not in [0;220[ (StartIndex >= bound)
	at one.microstream.io.XIO.internalRead(XIO.java:1176)
	at one.microstream.io.XIO.read(XIO.java:1162)
	at one.microstream.afs.nio.NioIoHandler$Default.specificReadBytes(NioIoHandler.java:372)
	at one.microstream.afs.nio.NioIoHandler$Default.specificReadBytes(NioIoHandler.java:1)
	at one.microstream.afs.AIoHandler$Abstract.readBytes(AIoHandler.java:752)
	at one.microstream.afs.AReadableFile.readBytes(AReadableFile.java:84)
	at one.microstream.storage.types.StorageFile$Abstract.readBytes(StorageFile.java:145)
	... 9 more
    opened by m-stoer 11
  • Android - Legacy Type Mapping works different from JDK

    Android - Legacy Type Mapping works different from JDK

    I have two classes:

    public class ByteLegacy {

    private byte byteTo_float;
    private byte byteTo_double;
    private byte byteTo_short;
    private byte byteTo_long;
    private byte byteTo_byte;
    private byte byteTo_int;
    private byte byteToString;
    private byte byteTo_boolean;
    private byte byteTo_char;
    private byte byteToByte;
    private Byte copyByteTo_byte;


    public static ByteLegacy fillSample() {
        ByteLegacy legacy = new ByteLegacy();

        legacy.byteTo_float = '1';
        legacy.byteTo_double = '2';
        legacy.byteTo_short = '3';
        legacy.byteTo_long = '4';
        legacy.byteTo_byte = 'a';
        legacy.byteTo_int = '6';
        legacy.byteToString = '5';
        legacy.byteTo_boolean = '0';
        legacy.byteTo_char = 'c';
        legacy.byteToByte = 'x';
        legacy.copyByteTo_byte = 'i';
        return legacy;
    }
}

public class ByteLegacy2 {

    private float byteTo_float;
    private double byteTo_double;
    private short byteTo_short;
    private long byteTo_long;
    private byte byteTo_byte;
    private int byteTo_int;
    private char byteToString;
    private boolean byteTo_boolean;
    private char byteTo_char;
    private Byte bytToByte;
    private byte copyByteTo_byte;
}

    I save the first one, change the class and start with the second class A received:

    ByteLegacy{byteTo_float=49, byteTo_double=50, byteTo_short=51, byteTo_long=52, byteTo_byte=97, byteTo_int=54, byteToString=53, byteTo_boolean=48, byteTo_char=99, byteToByte=120, ByteTo_byte=105}

ByteLegacy2{byteTo_float=0.0, byteTo_double=0.0, byteTo_short=49, byteTo_long=0, byteTo_byte=97, byteTo_int=50, byteToString=5, byteTo_boolean=true, byteTo_char=c, bytToByte=120, copyByteTo_byte=105}

    This is not an conversion what i expect. On the classic JVM works as intended:

    ByteLegacy{byteTo_float=49, byteTo_double=50, byteTo_short=51, byteTo_long=52, byteTo_byte=97, byteTo_int=54, byteToString=53, byteTo_boolean=48, byteTo_char=99, byteToByte=120, ByteTo_byte=105}

ByteLegacy2{byteTo_float=49.0, byteTo_double=50.0, byteTo_short=51, byteTo_long=52, byteTo_byte=97, byteTo_int=54, byteToString=5, byteTo_boolean=true, byteTo_char=c, bytToByte=120, copyByteTo_byte=105}
    opened by zdenek-jonas 10
  • Configuration validation exceptions must tell validation rules/values

    Configuration validation exceptions must tell validation rules/values

    Our Documentation in page: https://manual.docs.microstream.one/data-store/configuration/properties says: Minimum file size in bytes of a storage file to avoid merging with other files during housekeeping. Must be greater than 1, maximum is 2GB.

    It should be there, which format should be used.

    When i write: 500 I received: one.microstream.storage.exceptions.StorageExceptionInvalidConfiguration: Invalid file size: 500 Why is this number invalid?

    When i write 10Mb I received: java.lang.IllegalArgumentException: Nonsensical size limits: min file size = 10000000, max file size = 8388608 I would write: Min size limit cannot be higher as actual max size limit. This number has many '0', its hard to read it.

    Ok, I write: 20000000 I received: one.microstream.storage.exceptions.StorageExceptionInvalidConfiguration: Invalid file size: 20000000 Why is this number invalid?

    I write: 5kb dann passt. (previous edit was from ZJ via my computer during a meeting)

    opened by zdenek-jonas 9
  • Add OSGi bundle headers to released jar file

    Add OSGi bundle headers to released jar file

    In order to use MicroStream in OSGi environments it is necessary to ship the OSGi bundle headers within our jar files. This can be done with a maven plugin in the package phase of the build.

    https://blog.osoco.de/2016/05/building-osgi-bundles-with-apache-maven/

    opened by fh-ms 9
  • Convenience start methods must shutdown on problems

    Convenience start methods must shutdown on problems

    Bane of convenience methods: If complex logic is hidden in a short methode, there is no way to add a try-catch and care for cleanup (e.g. shutdown a started storage instance). So the convenience methods have to handle such cases.

    Especially ensuring that all file handles are closed. Maybe even with an internal "forceAbort" or something like that, in case task processing does not work anymore.

    opened by tm-ms 8
  • Communication: Trivial class with enum cannot be transferred

    Communication: Trivial class with enum cannot be transferred

    I have a problem to send this class:

        static class CustomEnumTrivialData  {
        CustomEnumTrivialEnumData value;

        public void fillSampleData() {
            value = CustomEnumTrivialEnumData.FIRST_VALUE;
        }

        CustomEnumTrivialEnumData getValue() {
            return value;
        }


        private enum CustomEnumTrivialEnumData {
            FIRST_VALUE
        }
    }

    I received:

    /usr/lib/jvm/java-8-openjdk-amd64/bin/java -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:/snap/intellij-idea-community/226/lib/idea_rt.jar=35497:/snap/intellij-idea-community/226/bin -Dfile.encoding=UTF-8 -classpath /snap/intellij-idea-community/226/lib/idea_rt.jar:/snap/intellij-idea-community/226/plugins/junit/lib/junit5-rt.jar:/snap/intellij-idea-community/226/plugins/junit/lib/junit-rt.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/cldrdata.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/dnsns.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/icedtea-sound.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/jaccess.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/java-atk-wrapper.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/nashorn.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunec.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/zipfs.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/management-agent.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar:/home/jonas/devel/microstream-test/communication/target/test-classes:/home/jonas/devel/microstream-test/communication/target/classes:/home/jonas/.m2/repository/one/microstream/communication.binary/03.01.00-MS-CO-SNAPSHOT/communication.binary-03.01.00-MS-CO-SNAPSHOT.jar:/home/jonas/.m2/repository/one/microstream/communication/03.01.00-MS-CO-SNAPSHOT/communication-03.01.00-MS-CO-20200528.100330-2.jar:/home/jonas/.m2/repository/one/microstream/persistence/03.01.00-MS-CO-SNAPSHOT/persistence-03.01.00-MS-CO-SNAPSHOT.jar:/home/jonas/.m2/repository/one/microstream/base/03.01.00-MS-CO-SNAPSHOT/base-03.01.00-MS-CO-SNAPSHOT.jar:/home/jonas/.m2/repository/one/microstream/persistence.binary/03.01.00-MS-CO-SNAPSHOT/persistence.binary-03.01.00-MS-CO-20200528.100320-2.jar:/home/jonas/.m2/repository/org/assertj/assertj-core/3.16.1/assertj-core-3.16.1.jar:/home/jonas/.m2/repository/org/junit/jupiter/junit-jupiter/5.5.2/junit-jupiter-5.5.2.jar:/home/jonas/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.5.2/junit-jupiter-api-5.5.2.jar:/home/jonas/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/home/jonas/.m2/repository/org/junit/platform/junit-platform-commons/1.5.2/junit-platform-commons-1.5.2.jar:/home/jonas/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.5.2/junit-jupiter-params-5.5.2.jar:/home/jonas/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.5.2/junit-jupiter-engine-5.5.2.jar:/home/jonas/.m2/repository/org/junit/platform/junit-platform-surefire-provider/1.3.2/junit-platform-surefire-provider-1.3.2.jar:/home/jonas/.m2/repository/org/apiguardian/apiguardian-api/1.0.0/apiguardian-api-1.0.0.jar:/home/jonas/.m2/repository/org/apache/maven/surefire/surefire-api/2.22.0/surefire-api-2.22.0.jar:/home/jonas/.m2/repository/org/apache/maven/surefire/surefire-logger-api/2.22.0/surefire-logger-api-2.22.0.jar:/home/jonas/.m2/repository/org/apache/maven/surefire/common-java5/2.22.0/common-java5-2.22.0.jar:/home/jonas/.m2/repository/org/junit/platform/junit-platform-launcher/1.5.2/junit-platform-launcher-1.5.2.jar:/home/jonas/.m2/repository/org/junit/platform/junit-platform-engine/1.5.2/junit-platform-engine-1.5.2.jar:/home/jonas/.m2/repository/commons-io/commons-io/2.6/commons-io-2.6.jar:/home/jonas/.m2/repository/com/github/javafaker/javafaker/0.18/javafaker-0.18.jar:/home/jonas/.m2/repository/org/apache/commons/commons-lang3/3.5/commons-lang3-3.5.jar:/home/jonas/.m2/repository/org/yaml/snakeyaml/1.23/snakeyaml-1.23-android.jar:/home/jonas/.m2/repository/com/github/mifmif/generex/1.0.2/generex-1.0.2.jar:/home/jonas/.m2/repository/dk/brics/automaton/automaton/1.11-8/automaton-1.11-8.jar:/home/jonas/.m2/repository/org/jacoco/org.jacoco.agent/0.8.5/org.jacoco.agent-0.8.5-runtime.jar com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit5 one.microstream.communication.BinaryHandlerCustomEnumTrivialTest

Exception in thread "Thread-1" one.microstream.com.ComException
	at one.microstream.com.ComChannelExceptionHandler.defaultHandleException(ComChannelExceptionHandler.java:18)
	at one.microstream.com.ComChannelExceptionHandler$Default.handleException(ComChannelExceptionHandler.java:46)
	at one.microstream.com.ComConnectionAcceptor$Default.acceptConnection(ComConnectionAcceptor.java:111)
	at one.microstream.com.ComHost$Default.synchAcceptConnection(ComHost.java:155)
	at one.microstream.com.ComHost$Default.acceptConnections(ComHost.java:147)
	at one.microstream.com.ComHost$Default.run(ComHost.java:113)
	at one.microstream.communication.AbstractComTest.lambda$startServer$1(AbstractComTest.java:73)
	at java.lang.Thread.run(Thread.java:748)
Caused by: one.microstream.persistence.exceptions.PersistenceExceptionTypeHandlerConsistencyUnhandledTypeId: No type handler found for type id "1000060".
	at one.microstream.persistence.binary.types.BinaryLoader$Default.lookupTypeHandler(BinaryLoader.java:190)
	at one.microstream.persistence.binary.types.BinaryLoader$Default.createBuildItem(BinaryLoader.java:198)
	at one.microstream.persistence.binary.types.BinaryLoader$Default.internalReadBinaryEntitiesByteReversing(BinaryLoader.java:168)
	at one.microstream.persistence.binary.types.BinaryLoader$Default.readBinaryEntities(BinaryLoader.java:138)
	at one.microstream.persistence.binary.types.ChunksWrapper.iterateEntityData(ChunksWrapper.java:70)
	at one.microstream.persistence.binary.types.BinaryLoader$Default.addChunks(BinaryLoader.java:801)
	at one.microstream.persistence.binary.types.BinaryLoader$Default.readLoadOnce(BinaryLoader.java:707)
	at one.microstream.persistence.binary.types.BinaryLoader$Default.get(BinaryLoader.java:824)
	at one.microstream.persistence.types.PersistenceManager$Default.get(PersistenceManager.java:379)
	at one.microstream.com.binarydynamic.ComChannelDynamic.requestUnhandled(ComChannelDynamic.java:44)
	at one.microstream.com.binarydynamic.ComHandlerSendMessageNewType.sendMessage(ComHandlerSendMessageNewType.java:34)
	at one.microstream.com.binarydynamic.ComHandlerSendMessageNewType.sendMessage(ComHandlerSendMessageNewType.java:48)
	at one.microstream.com.binarydynamic.ComChannelDynamic.send(ComChannelDynamic.java:59)
	at one.microstream.com.binarydynamic.ComTypeDescriptionRegistrationObserver.observeTypeDefinitionRegistration(ComTypeDescriptionRegistrationObserver.java:34)
	at one.microstream.persistence.types.PersistenceTypeDictionary$Default.synchRegisterType(PersistenceTypeDictionary.java:303)
	at one.microstream.persistence.types.PersistenceTypeDictionary$Default.registerTypeDefinition(PersistenceTypeDictionary.java:356)
	at one.microstream.persistence.types.PersistenceTypeDictionary$Default.registerRuntimeTypeDefinition(PersistenceTypeDictionary.java:393)
	at one.microstream.persistence.types.PersistenceTypeDictionaryManager$Abstract.registerRuntimeTypeDefinition(PersistenceTypeDictionaryManager.java:134)
	at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.synchUnvalidatedRegisterTypeHandler(PersistenceTypeHandlerManager.java:639)
	at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.registerTypeHandler(PersistenceTypeHandlerManager.java:608)
	at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.internalEnsureTypeHandler(PersistenceTypeHandlerManager.java:588)
	at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.ensureTypeHandler(PersistenceTypeHandlerManager.java:357)
	at one.microstream.persistence.types.PersistenceTypeHandlerManager$Default.ensureTypeHandler(PersistenceTypeHandlerManager.java:333)
	at one.microstream.persistence.binary.types.BinaryStorer$Default.registerGuaranteed(BinaryStorer.java:557)
	at one.microstream.persistence.binary.types.BinaryStorer$Default.registerLazyOptional(BinaryStorer.java:572)
	at one.microstream.persistence.types.PersistenceObjectManager$Default.ensureObjectId(PersistenceObjectManager.java:182)
	at one.microstream.persistence.binary.types.BinaryStorer$Default.register(BinaryStorer.java:591)
	at one.microstream.persistence.binary.types.BinaryStorer$Default.apply(BinaryStorer.java:298)
	at one.microstream.persistence.binary.types.BinaryValueFunctions$17.storeValueFromMemory(BinaryValueFunctions.java:296)
	at one.microstream.persistence.binary.types.Binary.storeFixedSize(Binary.java:1149)
	at one.microstream.persistence.binary.internal.AbstractBinaryHandlerReflective.store(AbstractBinaryHandlerReflective.java:497)
	at one.microstream.persistence.binary.internal.AbstractBinaryHandlerReflective.store(AbstractBinaryHandlerReflective.java:1)
	at one.microstream.persistence.binary.types.BinaryStorer$Default.storeItem(BinaryStorer.java:414)
	at one.microstream.persistence.binary.types.BinaryStorer$Default.storeGraph(BinaryStorer.java:403)
	at one.microstream.persistence.binary.types.BinaryStorer$Default.store(BinaryStorer.java:421)
	at one.microstream.persistence.types.PersistenceManager$Default.store(PersistenceManager.java:274)
	at one.microstream.com.binarydynamic.ComChannelDynamic.send(ComChannelDynamic.java:63)
	at one.microstream.communication.AbstractComTest.lambda$null$0(AbstractComTest.java:62)
	at one.microstream.com.ComConnectionAcceptor$Default.acceptConnection(ComConnectionAcceptor.java:107)
	... 5 more

Process finished with exit code 130 (interrupted by signal 2: SIGINT)
    opened by zdenek-jonas 7
  • Process is preventing itself from copying files

    Process is preventing itself from copying files

    JDK moronity strikes again. The following code snippet:

    Path source = XIO.Path("source.txt");
Path target = XIO.Path("target.bak");
StorageLockedFile slf = StorageLockedFile.openLockedFile(source);
System.out.println("LockedFile: " + slf);
Files.copy(source, target);

    Causes the following exception with a completely wrong and useless message: source.txt -> target.bak: Der Prozess kann nicht auf die Datei zugreifen, da ein anderer Prozess einen Teil der Datei gesperrt hat. (I won't translate that here and showing system messages in a localized form is another moronity in itself, but whatever)

    It seems that the recent switch from File to Path and the associated change from opening a FileChannel via an InputStream to the actually more modern and supposedly better FileChannel#open method caused this behavior.

    Maybe Files#copy doesn't work if there is an open FileChannel oder an existing FileLock or whatever. Or it need a "Create target file if it does not exist, yet"-option. Whatever the reason is, it is definitely NOT that another process has locked any of the two files involved, because there is no such process.

    If someone needs me, I'll be knee-deep in researching JDK idiocy once again...

    opened by tm-ms 7
  • Missing PersistenceTypeDictionary.ptd in backup directory

    Missing PersistenceTypeDictionary.ptd in backup directory

    After load XML configuration with backupDirectory, the file PersistenceTypeDictionary.ptd has been not created in backup dir.

    I have following xml file:

    <?xml version="1.0" encoding="UTF-8"?>
<properties>
	<property name="backupDirectory" value="C://Users//ZDENEK~1//AppData//Local//Temp//junit8758453482108621550//backup"/>
</properties>

    And this code snippet:

    Configuration configuration = Configuration.LoadXml(configFilePath.toUri().toURL());
 Customer customer = CustomerGenerator.generateNewCustomer();
EmbeddedStorageManager storageManager = configuration.createEmbeddedStorageFoundation().createEmbeddedStorageManager(customer).start();

    With this call the file PersistenceTypeDictionary.ptd has been not created in backup dir.

    When i called:

    EmbeddedStorageManager storageManager = configuration.setBaseDirectory(location.toString()).createEmbeddedStorageFoundation().createEmbeddedStorageManager(customer).start();

    Then the file has been created.

    So the behavior is: When i create store manager without define the baseDirectory, the file PersistenceTypeDictionary.ptd has been not created in backup dir.

    The similar problem has been already mentioned in Stack Overflow. Unfortunately at that time without further information.

    Update: After 18.02.2020 Is this file in backup directory not created at all.

    Main Method to test it:

        public static void main(String[] args) throws IOException, InterruptedException {
        Path location = new File("c:\\temp").toPath();

        Path configFilePath = location.resolve("backupDirectory.xml");
        Path backupLocation = location.resolve("backup");

        StringBuilder builder = new StringBuilder();
        builder.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
        builder.append("<properties>\n");
        builder.append("\t<property name=\"backupDirectory\" value=\""+ backupLocation.toString().replace("\\","//") + "\"/>\n");
        builder.append("</properties>");

        FileUtils.writeStringToFile(configFilePath.toFile(),builder.toString(), "UTF-8");

        Customer customer = CustomerGenerator.generateNewCustomer();

        Configuration configuration = Configuration.LoadXml(configFilePath.toUri().toURL());

        EmbeddedStorageManager storageManager = configuration.setBaseDirectory(location.toString()).createEmbeddedStorageFoundation().createEmbeddedStorageManager(customer).start();
        //EmbeddedStorageManager storageManager = configuration.createEmbeddedStorageFoundation().createEmbeddedStorageManager(customer).start();
        //https://github.com/microstream-one/microstream-private/issues/227 test after fix

        Thread.sleep(5000);

        List<File> files = (List<File>) FileUtils.listFiles(backupLocation.toFile(), null, true);

        assertTrue(files.size() > 2, files.toString());

        storageManager.shutdown();
    }
    opened by zdenek-jonas 7
  • Reduce StorageEntity memory consuming

    Reduce StorageEntity memory consuming

    Backgroup

    Let see a situation: Database size: is 271M(build with garbage collection) ➜ storage_db tree -h
    [ 128] . ├── [ 17K] PersistenceTypeDictionary.ptd └── [ 128] channel_0 ├── [271M] channel_0_1.dat └── [ 170] transactions_0.sft 1 directory, 3 files

    Heap size: just load database without others framework(no spring ... Used heap: 472M(after manual jvm gc) one.microstream.storage.types.StorageEntity$Default 328,551,808 B (64.6%) 5,133,622 (60.8%) this.is.myown.class 2,300,208 B (0.4%) 47,921 (0.6%)

    Problem

    1. StorageEntity instances consume up to 65% of total used heap memory.
    2. StorageEntity instances consume much more memory than my own class instances(0.6%), payload is too much.
    3. StorageEntity instances can't gc by jvm until EmbeddedStorageManager is closed.
    4. As ms database size growing, StorageEntity instances memory consuming grow faster.
    5. All StorageEntity instances loaded into heap(a whole db), normal obj references(String str;) and Lazy reference(Lazy<String> str;), which make Lazy reference less useful. (1 StorageEntity for 1 lazy ref, StorageEntity loaded into heap anywhere and StorageEntity use more memory than your own object)

    Expected solution

    1. Sub object graph under Lazy reference(Lazy subGraph) should not be loaded into heap at database startup, only be loaded on demand.
    2. No StorageEntity instance will be created for object within lazy sub object graph, until it's loaded.
    3. StorageEntity instance can be gc(by JVM) after the whole lazy sub object graph is gc by JVM.
    4. Reduce StorageEntity heap size consuming if it's possible.
      5. opened by 54446776 6
    5. How to recover from StorageExceptionConsistency?

      How to recover from StorageExceptionConsistency?

      Environment Details

      • MicroStream Version: latest (8)
      • JDK version: 17
      • OS: Linux
      • Used frameworks: None

      Describe the bug

      I had to kill a test process generating a lot of data and it seems that it corrupted something. I am wondering if this can be properly recovered somehow.

      one.microstream.storage.exceptions.StorageException: Problem in channel #0 at app//one.microstream.storage.types.StorageChannelTask$Abstract.checkForProblems(StorageChannelTask.java:114) at app//one.microstream.storage.types.StorageChannelTask$Abstract.waitOnCompletion(StorageChannelTask.java:176) at app//one.microstream.storage.types.StorageSystem$Default.startThreads(StorageSystem.java:336) at app//one.microstream.storage.types.StorageSystem$Default.internalStartUp(StorageSystem.java:519) at app//one.microstream.storage.types.StorageSystem$Default.start(StorageSystem.java:604) at app//one.microstream.storage.types.StorageSystem$Default.start(StorageSystem.java:78) at app//one.microstream.storage.embedded.types.EmbeddedStorageManager$Default.start(EmbeddedStorageManager.java:247) at app//one.microstream.storage.embedded.types.EmbeddedStorageManager$Default.start(EmbeddedStorageManager.java:99) at app//one.microstream.storage.embedded.types.EmbeddedStorage.createAndStartStorageManager(EmbeddedStorage.java:664) at app//one.microstream.storage.embedded.types.EmbeddedStorage.start(EmbeddedStorage.java:521) at app//one.microstream.storage.embedded.types.EmbeddedStorage.start(EmbeddedStorage.java:357) at app//com.arhs.mas.perf.GenerateUsersPerfTestDataTest.generateUsers(GenerateUsersPerfTestDataTest.java:81) at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at [email protected]/java.lang.reflect.Method.invoke(Method.java:568) at app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727) at app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at app//org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45) at app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1511) at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1511) at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86) at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86) at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53) at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99) at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79) at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75) at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61) at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at [email protected]/java.lang.reflect.Method.invoke(Method.java:568) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33) at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94) at jdk.proxy2/jdk.proxy2.$Proxy5.stop(Unknown Source) at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193) at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129) at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100) at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60) at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56) at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133) at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71) at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69) at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74) Caused by: one.microstream.storage.exceptions.StorageExceptionConsistency: 0 Length 8388563 of file 42 is inconsinstent with the transactions entry's length of 8388546 at app//one.microstream.storage.types.StorageFileManager$Default.validateStorageDataFilesLength(StorageFileManager.java:811) at app//one.microstream.storage.types.StorageFileManager$Default.initializeStorage(StorageFileManager.java:878) at app//one.microstream.storage.types.StorageChannel$Default.initializeStorage(StorageChannel.java:768) at app//one.microstream.storage.types.StorageChannelTaskInitialize$Default.succeed(StorageChannelTaskInitialize.java:204) at app//one.microstream.storage.types.StorageChannelTaskInitialize$Default.succeed(StorageChannelTaskInitialize.java:36) at app//one.microstream.storage.types.StorageChannelSynchronizingTask$AbstractCompletingTask.synchronizedComplete(StorageChannelSynchronizingTask.java:84) at app//one.microstream.storage.types.StorageChannelSynchronizingTask$AbstractCompletingTask.complete(StorageChannelSynchronizingTask.java:132) at app//one.microstream.storage.types.StorageChannelTask$Abstract.processBy(StorageChannelTask.java:268) at app//one.microstream.storage.types.StorageChannel$Default.work(StorageChannel.java:428) at app//one.microstream.storage.types.StorageChannel$Default.run(StorageChannel.java:520) at [email protected]/java.lang.Thread.run(Thread.java:833)

      opened by ghilainm 1
    6. Cannot store any entry in map

      Cannot store any entry in map

      Environment Details

      • MicroStream Version:
      • JDK version: JDK 17
      • OS: Linux
      • Used frameworks: None

      Describe the bug

      Cannot store any entry in map. When loading it after restart it is always empty. Following the same pattern with a list does not show the problem.

      Here is code snippet showcasing issue.

          @Test
    void test() {

        EmbeddedStorageManager storageManager = EmbeddedStorage.start(Paths.get("/home/my-data"));
        PerfData root = getRoot(storageManager);
        if (root.getUsers() == null) {
            root.setUsers(new LinkedList<>());
        }
        if (root.getUserData() == null) {
            root.setUserData(new HashMap<>());
        }
        System.out.println("User ? "+root.getUsers().size());
        PerfData.UserData xxx = new PerfData.UserData("xxx");
        root.getUserData().put("abc", xxx);
        storageManager.store(root.getUserData());
        root.getUsers().add(xxx);
        storageManager.store(root.getUsers());
        storageManager.storeRoot();
        storageManager.shutdown();
    }

    private PerfData getRoot(EmbeddedStorageManager storageManager) {
        if (storageManager.root() == null) {
            PerfData perfData = new PerfData();
            storageManager.setRoot(perfData);
            storageManager.storeRoot();
            storageManager.store(perfData.getUserData());
            log.info("Generating new root for data.");
            return perfData;
        } else {
            PerfData root = (PerfData) storageManager.root();
            log.info("Reusing data. Already {} users generated", root.getUserData().keySet().size());
            return root;
        }
    }

@Getter
@Setter
class PerfData {

    Map<String, UserData> userData = new HashMap<>();
    List<UserData> users = new LinkedList<>();
    String name = UUID.randomUUID().toString();

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    record UserId(String userId) {
    }
    record UserData(String name) {
    }
}
      opened by ghilainm 3
    7. Support multiple StorageManagers within Spring Boot Integration

      Support multiple StorageManagers within Spring Boot Integration

      It should be possible to support multiple StorageManagers within the integration using Qualifiers.

      Due to limitations in looking up annotations on injection point from within Bean producer method, some developer actions are required.

      @Configuration
public class DefineStorageManagers {

    private final StorageManagerProvider provider;

    public DefineStorageManagers(StorageManagerProvider provider) {
        this.provider = provider;
    }

    @Bean
    @Qualifier("green")
    public EmbeddedStorageManager getGreenManager() {
        return provider.get(DatabaseColor.GREEN.getName());
    }
    @Bean
    @Qualifier("red")
    public EmbeddedStorageManager getRedManager() {
        return provider.get(DatabaseColor.RED.getName());
    }
}

      The qualifier (label) is used within the configuration key

      one.microstream.red.storage-directory=red-db
one.microstream.red.channel-count=2

one.microstream.green.storage-directory=green-db
one.microstream.green.channel-count=1

      The automatic Root bean creation (and injection), EmbeddedStorageFoundationCustomizer, and StorageManagerInitializer are supported.

      enhancement 
      opened by rdebusscher 0
    8. Use provided PersistenceTypeDictionary-Name for the FullBackup

      Use provided PersistenceTypeDictionary-Name for the FullBackup

      I'm initialising my MicroStream-Datastore with the following code:

      	public static final EmbeddedStorageFoundation<?> createStorageFoundation(
		final File                        directory      
	)
	{
		final EmbeddedStorageFoundation<?> foundation = EmbeddedStorage.Foundation();
		foundation.onConnectionFoundation(
			f ->
			{
				f
				.setTypeDictionaryIoHandler( dictionaryStorage(, directory))
			}
		);
		return foundation;
	}
		
	static final PersistenceTypeDictionaryFileHandler dictionaryStorage(final File directory)
	{
		final File dictFile = new File(directory, "MyTypeDict.ptd");
		final NioFileSystem fileSystem = NioFileSystem.New();

		return dispatch(PersistenceTypeDictionaryFileHandler.New(fileSystem.ensureFile(dictFile.toPath())));
	}

      Perfect! It works. Then i create a Backup like this:

      public void createBackup(EmbeddedStorageManager       storageManager)
{
  storageManager.issueFullBackup(backupFileSystem.ensureDirectoryPath("."));
}

      Now the backup works as well, with one exception: The created PersistenceTypeDictionary is named 'PersistenceTypeDictionary.ptd' instead of the configured 'MyTypeDict.ptd'. That means if i want to use the backup, i have to rename the file.

      The MicroStream support already offered me a solution:

      final ADirectory targetDirectory = fileSystem.ensureDirectoryPath("c:/temp/MPS15/bkup");
storage.issueFullBackup(
	StorageLiveFileProvider.New(targetDirectory),
	PersistenceTypeDictionaryExporter.New(
		PersistenceTypeDictionaryFileHandler.New(targetDirectory.ensureFile("MyTypeDict.ptd"))
	)
);

      My feature request is simple enough: Make it work without the offered solution.

      opened by JohannesRabauer 1
      9. Releases(08.00.00-MS-EA1)

      • 08.00.00-MS-EA1(Dec 8, 2022)

        Features

        • Upgrade to Java 11 [#359]
        • Quarkus integration [#422]
        • Java 17 type handler improvement [#462]
        • Read-only mode [#427][#436]
        • Serializer with type information [#376]
        • Helidon MP example update [#444]
        • Channel count migration tool [#371]

        Bugfixes

        • Maven 4 build compliance [#471]
        • Helidon example build fix [#463]
        • Spring integration fixes [#415]
        • File deletion fix [#408]
        • Cache read through fix [#369]
        • Define service providers in module info [#368]
        Source code(tar.gz)
        Source code(zip)

      • 07.01.00-MS-GA(Sep 14, 2022)

        Features

        • New public AFS connectors
          • AWS S3 and DynamoDB
          • Azure Storage
          • Google Cloud Firestore
          • Hazelcast
          • Kafka
          • Oracle Coherence, DB and Cloud Object Store
          • Redis
          • Hana DB
        • Finalized storage garbage collector, activated by default [#402]
        • Improve CDI integration [#377]
        • CDI Integration: Allow customization of the EmbeddedStorageFoundation before StorageManager is created [#388][#398]
        • Spring Boot integration: Add interceptor logic to store marked objects at end of method [#384]
        • Clarification around cache expiry when using StorageManager [#387]
        • Store reference inside Lazy when Lazy is marked (CDI integration) [#386]
        • Enable microstream cache [#351]
        • Validate Channel Folders to Channel Count [#192]

        Bugfixes

        • Remove @Store from CDI integration [#397][#394]
        • Fix CDI integration cache configuration keys [#395]
        • Active the test in the CDI extension [#354]
        Source code(tar.gz)
        Source code(zip)

      • 07.01.00-MS-beta2(Aug 30, 2022)

        Features

        • Improve CDI integration [#377]
        • CDI Integration: Allow customization of the EmbeddedStorageFoundation before StorageManager is created [#388][#398]
        • Spring Boot integration: Add interceptor logic to store marked objects at end of method [#384]
        • Clarification around cache expiry when using StorageManager [#387]
        • Store reference inside Lazy when Lazy is marked (CDI integration) [#386]
        • Enable microstream cache [#351]

        Bugfixes

        • Remove @Store from CDI integration [#397][#394]
        • Fix CDI integration cache configuration keys [#395]
        • Active the test in the CDI extension [#354]
        Source code(tar.gz)
        Source code(zip)

      • 07.01.00-MS-beta1(Aug 9, 2022)

      • 07.00.00-MS-GA(Apr 26, 2022)

        Features

        • CDI Extension
        • Logging [#293]
        • Serializer API [#286]
        • Android Type Handlers [#325]

        Bugfixes

        • Reducing memory usage [#323]
        • Exclusive attempt conflict [#282][#284]
        • Fixed storage stalls in case of errors during the housekeeping phase [#285][#303]
        • Stop LazyrefenceManager automatically [#343]
        • Eliminate null in StorageConfiguration.toString() [#205][#297]
        • Fixed exclusive file access conflicts when restarting storage [#282]
        Source code(tar.gz)
        Source code(zip)

      • 07.00.00-MS-beta2(Mar 29, 2022)

      • 07.00.00-MS-beta1(Mar 21, 2022)

      • 06.01.00-MS-GA(Nov 16, 2021)

        Bugfixes

        • Fixed various bugs in backup [#194][#272][#273][#278][#280]
        • Fixed storage restart error with same configuration instance [#279]
        • Fixed Javadoc warnings [#281]
        Source code(tar.gz)
        Source code(zip)

      • 06.00.00-MS-GA(Oct 21, 2021)

        Features

        • Spring Boot Integration
        • Specialized type handlers for Java 17 collection
        • Deep-copy utility
        • Examples included in reactor build

        Bugfixes

        • Fixed swallowed exceptions in storage channels [#265]
        • Fixed failing import with multiple channels [#254]
        • Fixed zero byte blocks in storage [#255]
        • Fixed faulty loading behaviour in persisted cache [#226]
        • Fixed possible deadlock in persistence manager [#240]
        • Fixed bug in type handler for PriorityQueue [#234]
        Source code(tar.gz)
        Source code(zip)

      • 05.00.02-MS-GA(Jul 23, 2021)

      • 05.00.01-MS-GA(Jul 16, 2021)

      • 05.00.00-MS-GA(Jul 15, 2021)

        This release introduces some breaking changes. To fully support the Java Module System (Jigsaw) and to meet Maven’s naming standards, we had to rename some packages and our artifact’s IDs. But don’t worry. Just change the dependency configuration and organize some imports, and you are good to go. Existing storages are not affected and are fully compatible with this version. For detailed information, see the migration guide below.

        Features

        • General purpose configuration layer on which the storage and cache configuration are now based on, with following key features:
          • Tree like structure to support sub-configurations
          • Input from various sources (ini, xml, yaml, …​)
          • Typed getters for properties
          • Extendable: custom property parsers and configuration loaders
          • Adaptable: easy adaption to other framework’s configurations, like e.g. Spring Boot
        • Storage configuration now supports different file systems for live storage and backup
        • GraalVM support: MicroStream’s optimized memory adapter is now used automatically when running on GraalVM
        • Restructured MicroStream’s module and package structure to fully support the Java Module System and adopt Maven naming standards,

        Bugfixes

        • Fixed sporadic index out of bounds exception when storage and backup had different storage targets with a channel count greater than one

        Migration Guide

        https://docs.microstream.one/manual/intro/changelog.html#_migration_guide

        Source code(tar.gz)
        Source code(zip)
      Owner
      MicroStream
      Next Generation Serialization for Java and Native Java Data Store for developing ultra-fast Java in-memory database applications.
      MicroStream
      GitHub https://microstream.one/
      This is an Android application that deals with storage or "data persistence". The app is suitable for any shelter house to store the data of pets such as name, breed, gender and weight of the pet. The app uses a SQLite Database to store the data. The data is stored locally on the users phone. This app uses many other concepts such as building a ContentProvider and using a CursorAdapter and CursorLoader to automatically load the data.

      Pets App This app displays a list of pets and their related com.example.android.pets.data that the user inputs. Used in a Udacity course in the Androi

      null 3 Sep 2, 2021
      Apache Cayenne is an open source persistence framework licensed under the Apache License

      Apache Cayenne is an open source persistence framework licensed under the Apache License, providing object-relational mapping (ORM) and remoting services.

      The Apache Software Foundation 284 Dec 31, 2022
      MixStack lets you connects Flutter smoothly with Native pages, supports things like Multiple Tab Embeded Flutter View, Dynamic tab changing, and more. You can enjoy a smooth transition from legacy native code to Flutter with it.

      中文 README MixStack MixStack lets you connects Flutter smoothly with Native pages, supports things like Multiple Tab Embeded Flutter View, Dynamic tab

      Yuewen Engineering 80 Dec 19, 2022
      Apache Druid: a high performance real-time analytics database.

      Website | Documentation | Developer Mailing List | User Mailing List | Slack | Twitter | Download Apache Druid Druid is a high performance real-time a

      The Apache Software Foundation 12.3k Jan 1, 2023
      光 HikariCP・A solid, high-performance, JDBC connection pool at last.

      HikariCP It's Faster.Hi·ka·ri [hi·ka·'lē] (Origin: Japanese): light; ray. Fast, simple, reliable. HikariCP is a "zero-overhead" production ready JDBC

      Brett Wooldridge 17.7k Jan 1, 2023
      eXist Native XML Database and Application Platform

      eXist-db Native XML Database eXist-db is a high-performance open source native XML database—a NoSQL document database and application platform built e

      eXist-db.org 363 Dec 30, 2022
      Provides many useful CRUD, Pagination, Sorting operations with Thread-safe Singleton support through the native JDBC API.

      BangMapleJDBCRepository Inspired by the JpaRepository of Spring framework which also provides many capabilities for the CRUD, Pagination and Sorting o

      Ngô Nguyên Bằng 5 Apr 7, 2022
      Replicate your Key Value Store across your network, with consistency, persistance and performance.

      Chronicle Map Version Overview Chronicle Map is a super-fast, in-memory, non-blocking, key-value store, designed for low-latency, and/or multi-process

      Chronicle Software : Open Source 2.5k Dec 29, 2022
      Redisson - Redis Java client with features of In-Memory Data Grid. Over 50 Redis based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Tomcat, Scheduler, JCache API, Hibernate, MyBatis, RPC, local cache ...

      Redisson - Redis Java client with features of In-Memory Data Grid Quick start | Documentation | Javadocs | Changelog | Code examples | FAQs | Report a

      Redisson 20.4k Jan 5, 2023
      H2 is an embeddable RDBMS written in Java.

      Welcome to H2, the Java SQL database. The main features of H2 are: Very fast, open source, JDBC API Embedded and server modes; disk-based or in-memory

      H2 Database Engine 3.6k Jan 5, 2023
      A blazingly small and sane redis java client

      Jedis Jedis is a blazingly small and sane Redis java client. Jedis was conceived to be EASY to use. Jedis is fully compatible with redis 2.8.x, 3.x.x

      Redis 10.8k Dec 31, 2022
      Elasticsearch Java Rest Client.

      JEST Jest is a Java HTTP Rest client for ElasticSearch. ElasticSearch is an Open Source (Apache 2), Distributed, RESTful, Search Engine built on top o

      Searchly 2.1k Jan 1, 2023
      Java binding for etcd

      jetcd: Java binding for etcd TravisCI: CircleCI: A simple Java client library for the awesome etcd Uses the Apache HttpAsyncClient to implement watche

      Justin Santa Barbara 134 Jan 26, 2022
      LINQ-style queries for Java 8

      JINQ: Easy Database Queries for Java 8 Jinq provides developers an easy and natural way to write database queries in Java. You can treat database data

      Ming Iu 641 Dec 28, 2022
      MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap-memory. It is a fast and easy to use embedded Java database engine.

      MapDB: database engine MapDB combines embedded database engine and Java collections. It is free under Apache 2 license. MapDB is flexible and can be u

      Jan Kotek 4.6k Dec 30, 2022
      MariaDB Embedded in Java JAR

      What? MariaDB4j is a Java (!) "launcher" for MariaDB (the "backward compatible, drop-in replacement of the MySQL(R) Database Server", see FAQ and Wiki

      Michael Vorburger ⛑️ 720 Jan 4, 2023
      Unified Queries for Java

      Querydsl Querydsl is a framework which enables the construction of type-safe SQL-like queries for multiple backends including JPA, MongoDB and SQL in

      Querydsl 4.1k Dec 31, 2022
      Speedment is a Stream ORM Java Toolkit and Runtime

      Java Stream ORM Speedment is an open source Java Stream ORM toolkit and runtime. The toolkit analyzes the metadata of an existing SQL database and aut

      Speedment 2k Dec 21, 2022
      A blazingly small and sane redis java client

      Jedis Jedis is a blazingly small and sane Redis java client. Jedis was conceived to be EASY to use. Jedis is fully compatible with redis 2.8.x, 3.x.x

      Redis 10.9k Jan 8, 2023

      2022.JavaRepos