In the introduction, "The model class can be an abstract class or an interface" is stated.

Why not support POJOs that are not final and that do not have any final methods?

Incidentally, a later bullet: "This makes life easier, but there is no requirement that the model class extend or implement anything" should be indented.

BerkeleyKVDatabase kvdb = new BerkeleyKVDatabase();
kvdb.setDirectory(new File("."));
kvdb.setDatabaseName("fred");
kvdb.start();
jdb = new JSimpleDBFactory()
    .setDatabase(new Database(kvdb))
    .setSchemaVersion(-1)
    .setModelClasses(Person.class).newJSimpleDB();
final JTransaction txn = jdb.createTransaction(true, ValidationMode.AUTOMATIC);
JTransaction.setCurrent(txn);

The above gives:

org.jsimpledb.kv.RetryTransactionException: transaction must be retried

	at org.jsimpledb.kv.bdb.BerkeleyKVTransaction.wrapException(BerkeleyKVTransaction.java:265)
	at org.jsimpledb.kv.bdb.BerkeleyKVTransaction$CursorIterator.findNext(BerkeleyKVTransaction.java:395)
	at org.jsimpledb.kv.bdb.BerkeleyKVTransaction$CursorIterator.hasNext(BerkeleyKVTransaction.java:312)
	at org.jsimpledb.core.Database.verifySchemas(Database.java:471)
	at org.jsimpledb.core.Database.createTransaction(Database.java:304)
	at org.jsimpledb.core.Database.createTransaction(Database.java:254)
	at org.jsimpledb.JSimpleDB.createTransaction(JSimpleDB.java:581)
	at org.jsimpledb.JSimpleDB.createTransaction(JSimpleDB.java:551)
	at de.protubero.ajs.PersonStoreImpl.selectById(PersonStoreImpl.java:80)
	at de.protubero.ajs.PersonStoreTest.shouldBeAbleToDeleteKnownPerson(PersonStoreTest.java:33)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: com.sleepycat.je.LockTimeoutException: (JE 6.4.9) Lock expired. Locker 1328238652 11_main_Txn: waited for lock on database=fred LockAddr:785570251 LSN=0x0/0x1265 type=RANGE_READ grant=WAIT_NEW timeoutMillis=500 startTime=1502933461314 endTime=1502933461816
Owners: [<LockInfo locker="1926096844 10_main_Txn" type="RANGE_WRITE"/>]
Waiters: []

	at com.sleepycat.je.txn.LockManager.newLockTimeoutException(LockManager.java:736)
	at com.sleepycat.je.txn.LockManager.makeTimeoutException(LockManager.java:590)
	at com.sleepycat.je.txn.LockManager.lockInternal(LockManager.java:417)
	at com.sleepycat.je.txn.LockManager.lock(LockManager.java:295)
	at com.sleepycat.je.txn.Txn.lockInternal(Txn.java:567)
	at com.sleepycat.je.txn.Locker.lock(Locker.java:504)
	at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:3551)
	at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:3346)
	at com.sleepycat.je.dbi.CursorImpl.lockLNAndCheckDeleted(CursorImpl.java:2149)
	at com.sleepycat.je.dbi.CursorImpl.lockAndGetCurrent(CursorImpl.java:2074)
	at com.sleepycat.je.dbi.CursorImpl.getNext(CursorImpl.java:2552)
	at com.sleepycat.je.Cursor.retrieveNextCheckForInsertion(Cursor.java:3212)
	at com.sleepycat.je.Cursor.retrieveNextNoDups(Cursor.java:3094)
	at com.sleepycat.je.Cursor.retrieveNext(Cursor.java:2836)
	at com.sleepycat.je.Cursor.getNext(Cursor.java:1117)
	at org.jsimpledb.kv.bdb.BerkeleyKVTransaction$CursorIterator.findNext(BerkeleyKVTransaction.java:379)
	... 30 more

If I comment out the start() operation:

java.lang.IllegalStateException: not started

	at com.google.common.base.Preconditions.checkState(Preconditions.java:444)
	at org.jsimpledb.kv.bdb.BerkeleyKVDatabase.createTransaction(BerkeleyKVDatabase.java:274)
	at org.jsimpledb.kv.bdb.BerkeleyKVDatabase.createTransaction(BerkeleyKVDatabase.java:267)
	at org.jsimpledb.kv.bdb.BerkeleyKVDatabase.createTransaction(BerkeleyKVDatabase.java:41)
	at org.jsimpledb.core.Database.createTransaction(Database.java:251)
	at org.jsimpledb.JSimpleDB.createTransaction(JSimpleDB.java:581)
	at org.jsimpledb.JSimpleDB.createTransaction(JSimpleDB.java:551)

So I can't see an example of how to programmatically start BDB via JSimpleDB, and don't know what to do next to get it going. What am I doing wrong?

Oh, and it would be cool to have fluent interfaces like withDirectory() and withDatabaseName() for in-line fun:

jdb = new JSimpleDBFactory()
    .setDatabase(new Database(new BerkeleyKVDatabase()
        .withDirectory(new File(".")).withDatabaseName("fred")))
    .setSchemaVersion(-1)
    .setModelClasses(Person.class).newJSimpleDB();

Maven pom's are an ide agnostic way of sharing project.

The maven pom is unusable jsimpledb/src/mvn/pom.xml

The pom.xml can not be imported into an ide.

For modern web frameworks, JSON can be made automatically for return values from method invocation that are bound to HTTP-GET operations. For example List<Person> can be turned into a list a people in JSON.

Strategies are needed for excluding the fields from the JSimpleDB generated subclass of Person though. If the serialization tech is Jackson, there are a number of strategies but each is quite a bit of obscure code. Especially if the web-framework in question instantiates its own ObjectMapper.

Jackson allows @JsonIgnore to prefix any field that should not be serialized. Java is quite generous for annotations that are missing from the classpath at runtime so adding that to every @JsimpleDB generated field would work.

Alternatively (and best of all) making each @JsimpleDB generated field transient would work too. This one would work for Gson too.

Thoughts?

Does SQL Server consider 0x1234 equal to 0x123400 ?

This is of course incompatible with JSimpleDB's assumptions about key/value stores and would mean MSSQLKVDatabase is broken, even though it appears to pass the unit tests OK (in my tests).

Some worrisome data points:

http://stackoverflow.com/questions/26255195/how-does-varbinary-data-sorted/26257328?noredirect=1#comment74644636_26257328

https://dba.stackexchange.com/questions/48660/comparing-binary-0x-and-0x00-turns-out-to-be-equal-on-sql-server

Kotlin unifies int and java.lang.Integer at the language level. Likewise for other primitives. This confuses JSimpleDB and yields errors like this one:

Exception in thread "main" java.lang.IllegalArgumentException: invalid value type int for index query on field `age' in class net.plan99.jsimdb.Person: should be a super-type of java.lang.Integer

given this sort of code:

fun queryByAge() = dbtx.queryIndex(Person::class.java, "age", Int::class.java).asMap()

It works if I change it to:

fun queryByAge() = dbtx.queryIndex(Person::class.java, "age", Integer::class.java).asMap()

but this triggers Kotlin warnings and really, it'd be nicer if int and Integer were treated as equivalent.

Here's my POJO with annotations added for JSimpleDB:

@JSimpleClass
public class Person {

	private int id;
	private String name;
	private String email;
	private int age;

	@JField(indexed = true)
	public int getId() {
		return id;
	}
	public void setId(int id) {
		this.id = id;
	}

	@JField
	public String getName() {
		return name;
	}
	public void setName(String name) {
		this.name = name;
	}

	@JField
	public String getEmail() {
		return email;
	}
	public void setEmail(String email) {
		this.email = email;
	}

	@JField
	public int getAge() {
		return age;
	}
	public void setAge(int age) {
		this.age = age;
	}

	public void updateWith(Person upToDatePerson) {
		this.age = upToDatePerson.age;
		this.email = upToDatePerson.email;
		this.name = upToDatePerson.name;
	}

//	@Override
//	public String toString() {
//		return "Person{" +
//				"id=" + id +
//				", name='" + name + '\'' +
//				", email='" + email + '\'' +
//				", age=" + age +
//				'}';
//	}

	@Override
	public String toString() {
		return "Person{" +
				"id=" + getId() +
				", name='" + getName() + '\'' +
				", email='" + getEmail() + '\'' +
				", age=" + getAge() +
				'}';
	}
}

There's two alternate toString() implementations there. One that Intellij made for me that uses fields to compose the RV. In use those were 0 and null despite prior setXyz() calls on the instance. When I changed it to use the getters it worked as expected. I deduce that the POJOs own fields are effectively disabled. While I understand that JSimpleDB wants to subclass POJOs, I feel it could be more faithful to the definition of POJOs if each generated method were to also invoke the super with the same args.

Or give up on POJOs and suggest abstractions only.

The trouble with the latter is that web frameworks like Spring want to create a temp 'model' object from request params (and cookie vars and form fields) to hand to a controller method, and according to MVC* that model object could be discarded and not at all saved - subject to the control flow within the controller method (and whatever collaborating methods it calls). Given 'not save' is a viable lifecycle for a Person instance (in this case) that'd be coming through a web-UI, then we cannot insist that all Person instances are made via JTransaction.create(modelClass). If we did it would be data-binding*.

Say we did insist that. I'd end up with Person.class for saving to JSimpleDB, and PersonUI.class with all the same fields and methods for the UI framework to casually create at will, and a checklist activity for the dev team to ensure that the two Person classes tracked each other's fields etc over the months and years. Or a code gen situation.

* https://paulhammant.com/2015/04/29/mvc-misunderstood-for-37-years - find "Late to MVC Party" in page

First I've got to say, this looks like a really awesome project!

I'm hoping to use it in a new project, but for a database I'm required to use MS SQL 2012, and I encountered an infinite loop.

To get started, I pulled the required jars for jsimpledb-kv-3.4.0, and I fetched the latest JTDS/MS drivers. I then followed the instructions at the getting started page. I first ran everything with the XMLKVDatabase and had no issues.

I then extended the SQLKVDatabase to override createPutStatement() with a transaction. Writing works, but during the read I ran into the loop at getFriendsToMe().

I overrode createGetRangeStatement() and changed the range from column>=? to column>?, and then received the expected output without any issue.

Do you believe this problem is specific to MS SQL? And is there a test procedure I should use to ensure interoperability with the database?

Whether you can open a directory as a regular file is highly platform specific. The following code fails on Windows.

Offending code:

if(!this.directory.isDirectory()) {
    throw new ArrayKVException("file `" + this.directory + "\' is not a directory");
}
this.directoryChannel = FileChannel.open(this.directory.toPath(), new OpenOption[0]);

Stacktrace: Caused by: java.nio.file.AccessDeniedException: C:\x\y\z\kvstore at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115) at java.nio.channels.FileChannel.open(FileChannel.java:287) at java.nio.channels.FileChannel.open(FileChannel.java:335) at org.jsimpledb.kv.array.AtomicArrayKVStore.start(AtomicArrayKVStore.java:353) ... 72 more

I've spent a lot of time over the holiday break reading and understanding some of the Permazen code. As part of that I fixed a bunch of typos, broken JavaDocs etc that IntelliJ flagged. I also accepted a few of its suggestions for code simplification, albeit not most of them to avoid too much pointless churn.

Where code has been deleted it's because it's dead. I left most dead methods as they may be useful in future, but some stuff would clearly never be used.

Bumps jetty-server from 9.4.44.v20210927 to 10.0.10.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps spring-web from 5.3.20 to 6.0.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Examples

Unit tests don't give enough info for the end-user to get up and running with. Consider BDB. There should perhaps be a README in here - https://github.com/archiecobbs/jsimpledb/tree/master/jsimpledb-kv-bdb. From that links to example projects.

Implementations

Note too that https://github.com/archiecobbs/jsimpledb (readme) does not mention Berkeley. First linked wiki page Information does, but that links to JavaDocs. One third of the dev world is happy with ref docs, one third with tutorials and one third cohesive examples (like me). Note that StackOverflow is Mecca for example-code people (I have your JSimpleDB answers an upvote).

The unit test for the bdb implementation doesn't yield any larger "how to" information. Nor does the CLI tool.

By contrast, the Jooby project provides a number of cohesive example applications, complete with build unit test and integration tests. Like this one - https://github.com/jooby-project/greeting. Easy to use as a starting point for your own app.

Information rift:

• the Introduction wiki page (listed first) shows snippets of key and value interop.
• the getting started wiki page (listed second) shows wrapping JDB setup and try/finally stuff that must wrap all snippets. Cruicial, IMO, is the usage of .newJSimpleDB() but there's only two refs to that in that page, and not at the top. Note also, you've two orphaned </code> markup vestiges in that doc to excise.
• There should be a wiki page that lists all the implementations of JSimpleDB, including the BDB one. It should be listed in the readme, and linked to again in Introduction and Getting Started. That should include the differentiating setup per implementation.
• .start() and .stop are not really discussed anywhere. Why? What if I don't stop the DB before shutdown? Why can't it start itself on first use?

Don't be put off - I think your stuff is great - and my nooooob experience report should be useful to you.

This isn't an issue per say, but after setting up the GUI editor for a schema I wrote, I realized I was no longer able to edit objects with a Set, since the JSet/JMap/JList field builders in JObjectEditorWindow.java are still a TODO and result in an "Invalid value(s)" message.

Though technically not needed, I'd really love to use the editor. Any recommendation to get past this in the meantime? Alternatively, I'm thinking I could hold myself over by writing a very simple implementation of the field builders (a text field of json to represent the set/map/list of either internal IDs or simple data)