Planned changes:

• [x] API update only
• [x] Throw UnsupportedOperationException for most operations
• [x] Disable activemq-camel from build (rebased from #701)
• [x] geronimo-jms for karaf
• [x] Add deliveryDelay
• [x] Add unit test
• [x] Formatting fixes

• [x] Create JMSContext
• [x] Create JMSProducer
• [x] Create JMSConsumer
• [x] Add JMSException -> JMSRuntimeException wrapping
• [x] Add methods for working with queues
• [x] Update unit test to send-recv using JMS 2.0 API objects (JMSProducer, JMSConsumer)
• [x] JMSContext autostart handling
• [x] QueueBrowser unit tests (autostart, false, and delayed)
• [x] JMSContext multi-context and instance counting support
• [x] JMSProducer review fixes and improvements
• [x] All message types and destination types unit tests
• [x] All message property types and min/max values unit tests
• [x] JMSProducer priority and deliveryMode bounds checking and unit tests
• [x] JMSConsumer Topic durable subscription support and unit tests
• [x] Add unit tests for producer overrides and disableTimestamp handling
• [x] Update for thread safety and state locking
• [x] Add unit test and consider for disableMessageID being ignored
• [x] Add ack mode unit tests
• [x] Add message listener unit tests
• [x] Add mixed producer tests (disableTimestamp vs not)
• [x] Add producer invalid property name and value(s) unit tests
• [x] Document JMS2 page on website that disableMessageID not supported by default provider. Everything is wired through to the ActiveMQSession in the event that someone wants to swap out some protocol handler to do it.
• [x] Refactor message consumer pattern
• [x] Refactor test message listener to use countdown latch and concurrency safe collections for verification
• [x] Refactor support test class send patterns to use a builder class (MessageData) for legibility
• [x] Reduce total unit test time

We recently encountered a similar performance degradation as is described here: https://issues.apache.org/jira/browse/AMQ-7340

A specific example we encountered was with around 20k messages scheduled very quickly where each message adds 100ms delay times with the previous message. Example: message 1 > delay time 100ms message 2 > delay time 200ms message 3 > delay time 300ms ... message 10,000 >> delay time 1000000ms (1000s) ... message 20,000 >> delay time 2000000ms (2000s)

If delivered on schedule all these messages should be moved to the queue within ~33 minutes but we observed that it took nearly 3 hours for all the messages to be moved to the queue.

After diving deep on the issue it seems the main loop that processes scheduled messages process them by traversing the B+ Tree index from the root to find the leftmost leaf node which contains the scheduled jobs with the earliest executionTime. This traversal does a disk read at each branch and unmarshals the raw data before moving to the next branch and is not cached for future reads as far as I can tell. The scheduled jobs in that leaf are processed then we repeat the traversal to find the execution time of the next batch of jobs to calculate how long the loop should sleep for. So for every loop, we find the left most node twice, at the start and end of the loop. It seems like this loop can take a long time and we end up falling behind and not being able to catch up since we're processing one node at a time.

This change still does the traversal from the root node to the leftmost node at the start of the loop but once it finds that leftmost node it will iterate over the leaf nodes sequentially until it finds a job with an execution time greater than the current time. This means that it will always "catch up" on every iteration of the loop. The index is locked for the duration of the iteration so there's probably some risk that scheduled messages can't make it in but in practice it doesn't seem like an issue. I ran 100 connections against a broker scheduling messages as fast as they could with 100ms delays as in the example above and I was able to schedule 120k messages within a couple of seconds without any issues.

Added a getMessageSize method to MessageStore to support retrieving the total message size of all stored messages for a destination. This is useful for metrics.

This resolves https://issues.apache.org/jira/browse/AMQ-5748

Hi Community,

Milap here an engineer turned a designer. I have designed few logo which you can here. I have referred Apache foundation style guide. Would like to know your feedback.

Thanks, Milap :)

Sus test:

• [ ] org.apache.activemq.broker.region.QueuePurgeTest (hangs long time)
• [ ] org.apache.activemq.broker.scheduler.JmsSchedulerTest (hangs long time)

Failing tests:

Passing tests:

• [x] org.apache.activemq.ProducerFlowControlTest
• [x] org.apache.activemq.ReconnectWithSameClientIDTest
• [x] org.apache.activemq.TcpTransportCloseConnectionTest
• [x] org.apache.activemq.broker.region.DestinationGCStressTest
• [x] org.apache.activemq.broker.scheduler.JobSchedulerManagementTest
• [x] org.apache.activemq.broker.scheduler.JobSchedulerRedliveryPluginDLQStoreCleanupTest
• [x] org.apache.activemq.broker.scheduler.JobSchedulerStoreCheckpointTest
• [x] org.apache.activemq.broker.scheduler.LostScheduledMessagesTest
• [x] org.apache.activemq.bugs.AMQ1936Test
• [x] org.apache.activemq.bugs.AMQ2902Test
• [x] org.apache.activemq.bugs.AMQ3567Test
• [x] org.apache.activemq.bugs.AMQ3622Test
• [x] org.apache.activemq.bugs.AMQ3625Test
• [x] org.apache.activemq.bugs.AMQ3779Test
• [x] org.apache.activemq.bugs.AMQ4221Test
• [x] org.apache.activemq.bugs.AMQ5426Test
• [x] org.apache.activemq.bugs.AMQ5844Test
• [x] org.apache.activemq.bugs.AMQ6131Test
• [x] org.apache.activemq.bugs.AMQ6432Test
• [x] org.apache.activemq.bugs.AMQ6463Test
• [x] org.apache.activemq.bugs.AMQ7067Test
• [x] org.apache.activemq.bugs.AMQ6133PersistJMSRedeliveryTest
• [x] org.apache.activemq.bugs.MKahaDBTxRecoveryTest
• [x] org.apache.activemq.bugs.TransactedStoreUsageSuspendResumeTest
• [x] org.apache.activemq.bugs.TrapMessageInJDBCStoreTest
• [x] org.apache.activemq.jmx.JmxAuditLogTest
• [x] org.apache.activemq.security.SecurityJMXTest
• [x] org.apache.activemq.store.jdbc.JDBCConcurrentDLQTest
• [x] org.apache.activemq.store.jdbc.JDBCIOExceptionHandlerTest
• [x] org.apache.activemq.store.kahadb.KahaDBTest
• [x] org.apache.activemq.transport.failover.AMQ1925Test
• [x] org.apache.activemq.transport.failover.FailoverUpdateURIsTest
• [x] org.apache.activemq.transport.nio.NIOSSLBasicTest
• [x] org.apache.activemq.transport.vm.VmTransportNetworkBrokerTest
• [x] org.apache.activemq.usage.StoreUsageLimitsTest
• [x] org.apache.activemq.usecases.AMQ6446Test
• [x] org.apache.activemq.usecases.ClientRebalanceTest
• [x] org.apache.activemq.usecases.DurableSubCacheTest
• [x] org.apache.activemq.usecases.EmptyTransactionTest
• [x] org.apache.activemq.usecases.OfflineDurableSubscriberTimeoutTest
• [x] org.apache.activemq.usecases.RequestReplyTempDestRemovalAdvisoryRaceTest
• [x] org.apache.activemq.usecases.TopicProducerFlowControlTest
• [x] org.apache.activemq.usecases.UsageBlockedDispatchTest
• [x] org.apache.activemq.store.SharedFileLockerTest
• [x] org.apache.activemq.store.kahadb.KahaDBPersistenceAdapterTest
• [x] org.apache.activemq.transport.tcp.TcpTransportCloseSocketNoWarnTest
• [x] org.apache.activemq.broker.jmx.Log4JConfigTest
• [x] org.apache.activemq.transport.tcp.TcpTransportInactiveDuringHandshakeTest

File changed that is not a test

• [x] org.apache.activemq.bugs.embedded.EmbeddedActiveMQ
• [x] org.apache.activemq.bugs.embedded.ThreadExplorer

This narrows the lock that was added to serviceRead() to secureRead() which prevents processing commands while locked which should solve the deadlock issues

…tory#tmFromJndi parameter to Boolean for JCA validation

Without this change deploying activemq-rar-5.13.3.rar on WildFly leads to "Invalid archive" warning as described in WFLY-6597.

this won't fix every issue, but improve the situation see https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/apache/activemq/README.md

This change will allow to pass sortColumnId and sortOrder url parameters to console's queues view, thus ordering the data on-load. This is very useful when visually monitoring dequeued messages change over time.

This PR first adds a test that proves the behavior is broken: c3729b0

When you run this test against master it will fail:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.activemq.transport.stomp.Stomp12Test
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 62.016 sec <<< FAILURE! - in org.apache.activemq.transport.stomp.Stomp12Test
testClientIndividualAckPrefetchTransaction(org.apache.activemq.transport.stomp.Stomp12Test)  Time elapsed: 61.822 sec  <<< FAILURE!
java.lang.AssertionError: null
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.junit.Assert.assertTrue(Assert.java:52)
	at org.apache.activemq.transport.stomp.Stomp12Test.testClientIndividualAckPrefetchTransaction(Stomp12Test.java:413)


Results :

Failed tests: 
  Stomp12Test.testClientIndividualAckPrefetchTransaction:413 null

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

The test output shows that it only took 4 messages to trigger this condition:

2019-05-20 21:54:48,710 [me-limited test] - INFO  Stomp12Test                    - Broker sent: MESSAGE
expires:0
destination:/queue/org.apache.activemq.transport.stomp.Stomp12Test.testClientIndividualAckPrefetchTransaction
JMSXUserID:system
ack:ID:eborgstrom-NW3081-55336-1558414488397-6:5
subscription:1
priority:4
message-id:ID:eborgstrom-NW3081-55336-1558414488397-5:1:-1:1:5
timestamp:1558414488648

4
2019-05-20 21:54:48,711 [0.1:55337@55324] - WARN  ProtocolConverter              - Exception occurred processing: ACK -> org.apache.activemq.transport.stomp.ProtocolException: Unexpected ACK received for message-id [null]
2019-05-20 21:54:48,712 [0.1:55337@55324] - DEBUG ProtocolConverter              - Exception detail
org.apache.activemq.transport.stomp.ProtocolException: Unexpected ACK received for message-id [null]
	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompAck(ProtocolConverter.java:476)
	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:251)
	at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:85)
	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:233)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
	at java.lang.Thread.run(Thread.java:748)
2019-05-20 21:54:48,716 [me-limited test] - INFO  Stomp12Test                    - Broker sent: ERROR
message:Unexpected ACK received for message-id [null]
content-type:text/plain

org.apache.activemq.transport.stomp.ProtocolE...d.java:748)

Which is the exception I reported in https://issues.apache.org/jira/browse/AMQ-7211

The fix for this (be19dcaa3) is simply to NOT clear pending ACKs on transaction COMMIT & ABORT.

Looking at the ticket referenced in the commit that added the pedingAcks.clear() lines (52d95ee01 / https://issues.apache.org/jira/browse/AMQ-5423) I believe that changing the this.pedingAcks.get to this.pedingAcks.remove is the real solution to the memory leak that prompted the change and that clearing the acks at the end of a transaction is not needed and incorrect behavior.

CC @tabish121 -- since you commited 52d95ee01 and worked on AMQ-5423

AMQ-9188 Augment scheduler to forward scheduled and dispatched messages to a destination, if such is configured. e.g. <broker ... schedulerSupport="true" schedulerActivityDestination="topic://ActiveMQ.Scheduler.Activity" > If schedulerActivityDestination is not set, current behavior is unaffected. Otherwise... When message is successfully scheduled scheduler should set property AMQ_SCHEDULER_ACTIVITY to SCHEDULED and forward scheduled message to the schedulerActivityDestination When message is successfully dispatched scheduler should set property AMQ_SCHEDULER_ACTIVITY to DISPATCHED and forward scheduled message to the schedulerActivityDestination

Hi, Added a test case to demonstrate the bug described in AMQ-9187 and a fix.

The fix is very simple - swapping the order of the AdvisoryBroker and SchedulerBroker interceptors. I don't think there will be any weird side effects of doing this because the SchedulerBroker only intercepts the 'send' method whereas the AdvisoryBroker doesn't intercept that 'send' method. Let me know what you think.

I would like to backport this to the 5.17 and 5.16 branches once we get this accepted into main. Thanks!

regards,

• Martin

JIRA issue: https://issues.apache.org/jira/projects/AMQ/issues/AMQ-9166?filter=allopenissues&orderby=created+DESC%2C+priority+DESC%2C+updated+DESC

• [x] Relocate the subset of activeio classes used by jdbc journal to jdbc-store module
• [x] Remove activeio-core (no other module uses it)
• [x] Rename System properties activeio -> activemq.store.journal
• [x] Update javadoc @see: references
• [x] Deprecate JDBC Journal

Note: The activeio classes are used by the JDBC Journal from 4.x era, not the more modern JDBCPersistenceAdapter.

Build warnings that need review

Reading from file activemq/activemq-client/src/main/grammar/SelectorParser.jj . . .
Warning: Line 22, Column 3: Command line setting of "STATIC" modifies option value in file.
Warning: Line 23, Column 3: Command line setting of "UNICODE_INPUT" modifies option value in file.
Warning: Line 26, Column 3: Command line setting of "ERROR_REPORTING" modifies option value in file.
File "TokenMgrError.java" does not exist.  Will create one.
File "ParseException.java" does not exist.  Will create one.
File "Token.java" does not exist.  Will create one.
File "SimpleCharStream.java" does not exist.  Will create one.
Parser generated with 0 errors and 3 warnings.