Mirror of Apache ActiveMQ Artemis

Overview

ActiveMQ Artemis

This file describes some minimum 'stuff one needs to know' to get started coding in this project.

Source

For details about the modifying the code, building the project, running tests, IDE integration, etc. see our Hacking Guide.

Build Status

Build Status: Build Status

Building the ASYNC IO library

ActiveMQ Artemis provides two journal persistence types, NIO (which uses the Java NIO libraries), and ASYNCIO which interacts with the linux kernel libaio library. The ASYNCIO journal type should be used where possible as it is far superior in terms of performance.

ActiveMQ Artemis does not ship with the Artemis Native ASYNCIO library in the source distribution. These need to be built prior to running "mvn install", to ensure that the ASYNCIO journal type is available in the resulting build. Don't worry if you don't want to use ASYNCIO or your system does not support libaio, ActiveMQ Artemis will check at runtime to see if the required libraries and system dependencies are available, if not it will default to using NIO.

To build the ActiveMQ Artemis ASYNCIO native libraries, please follow the instructions in the artemis-native/README.

Documentation

Our documentation is always in sync with our releases at the Apache ActiveMQ Artemis website.

Or you can also look at the current master version on github.

Examples

To run an example firstly make sure you have run

$ mvn -Prelease install

If the project version has already been released then this is unnecessary.

Each individual example can be run using this command from its corresponding directory:

$ mvn verify

If you wish to run groups of examples then use this command from a parent directory (e.g. examples/features/standard):

$ mvn -Pexamples verify

Recreating the examples

If you are trying to copy the examples somewhere else and modifying them. Consider asking Maven to explicitly list all the dependencies:

# if trying to modify the 'topic' example:
cd examples/jms/topic && mvn dependency:list

Open Web Application Security Project (OWASP) Report

If you wish to generate the report for CCV dependencies, you may run it with the -Powasp profile

$ mvn -Powasp verify

The output will be under ./target/dependency-check-report.html for each sub-module.

Comments
  • ARTEMIS-1156: FIX: Long Autoboxing occurring on Hot Path

    ARTEMIS-1156: FIX: Long Autoboxing occurring on Hot Path

    Building on ARTEMIS-905 JCtools ConcurrentMap replacement first proposed but currently parked by @franz1981, replace the collections with primitive key concurrent collections to avoid auto boxing.

    The goal of this is to reduce/remove autoboxing on the hot path. We are just adding jctools to the broker (should not be in client dependencies) Like wise targeting specific use case with specific implementation rather than a blanket replace all.

    We have to create our own primitive long concurrent hash set, based on jctools as jctools currently don't have one.

    opened by michaelandrepearce 64
  • ARTEMIS-3106  - Support for SASL-SCRAM

    ARTEMIS-3106 - Support for SASL-SCRAM

    This adds Support for ARTEMIS-3106 and adds the following parts:

    • An implementation for SASL-SCRAM-SHA1/256/512 based on SCRAM SASL authentication for Java
    • A properties based SASL-SCRAM LoginModule to manage user and roles
    • A LoginModule using the acquired credentials from the SASL-SCRAM Module to perform the login in Artemis
    • An example Server/Client that demonstrates the use
    opened by laeubi 53
  • ARTEMIS-2216 Use a specific executor for pageSyncTimer

    ARTEMIS-2216 Use a specific executor for pageSyncTimer

    Improve paging throughput by using a specific executor for pageSyncTimer

    Improving throughput on paging mode is one of our concerns since our cluster uses paging a lot. We found that pageSyncTimer in PagingStoreImpl shared the same executor with pageCursorProvider from thread pool. In heavy load scenario like hundreds of consumers receiving messages simultaneously, it became difficult for pageSyncTimer to get the executor due to race condition. Therefore page sync was delayed and producers suffered low throughput.

    To achieve higher performance we assign a specific executor to pageSyncTimer to avoid racing. And we run a small-scale test on a single modified broker.

    Broker: 4C/8G/500G SSD Producer: 200 threads, non-transactional send Consumer 200 threads, transactional receive Message text size: 100-200 bytes randomly AddressFullPolicy: PAGE

    Test result:

      | Only Send TPS | Only Receive TPS | Send&Receive TPS -- | -- | -- | -- Original ver | 38k | 33k | 3k/30k Modified ver | 38k | 34k | 30k/12.5k

    The chart above shows that on modified broker send TPS improves from “poor” to “extremely fast”, while receive TPS drops from “extremely fast” to “not-bad” under heavy load. Considering consumer systems usually have a long processing chain after receiving messages, we don’t need too fast receive TPS. Instead, we want to guarantee send TPS to cope with traffic peak and lower producer’s delay time. Moreover, send and receive TPS in total raises from 33k to about 43k. From all above this trade-off seems beneficial and acceptable.

    opened by qihongxu 53
  • ARTEMIS-2823 Use datasource with JDBC store db connections

    ARTEMIS-2823 Use datasource with JDBC store db connections

    Replaces direct jdbc connections with dbcp2 datasource. Adda configuration options to use alternative datasources and to alter the parameters. While adding slight overhead, this vastly improves the management and pooling capabilities with db connections.

    opened by uomik 47
  • ARTEMIS-196 Implement Consumer Priority

    ARTEMIS-196 Implement Consumer Priority

    @franz1981 an alternative so we don't have to have a copy of CopyOnWriteArrayList, it does mean on add or remove consumer we have to invoke toArray which causes a copy, but this is not on hot path, so i think we should be good, and avoids us having to clone a jvm class.

    opened by michaelandrepearce 46
  • ARTEMIS-2614 Create queues based on FQQN for AMQP protocol

    ARTEMIS-2614 Create queues based on FQQN for AMQP protocol

    This is my attempt to address the problem of the auto-creation of queues based on FQQN. The simple scenario seems to work, but some decisions need to be made before I can move forward with this.

    1. There is a problem with the RoutingType mismatch. Previously it didn't matter what RoutingType receiver tries to attach to, as the implementation assumed pre-existence of the queue. Now the routing type matters, as we need to create a properly configured queue if it doesn't exist. That's why testQueueConsumerReceiveTopicUsingFQQN test fails currently. To make it pass I would need to subscribe to a topic instead of a queue. I'm not sure but is this a breaking change or not?
    2. I'm not sure what is the expected behavior when durability configuration mismatches. The problem is described in a test testAttachToPreConfiguredNonDurableQueueUsingDurableFQQN. Should we reject the attempt to attach when the pre-configured queue is non-durable and attach request has the source with TerminusDurability of UNSETTLED_STATE or CONFIGURATION? And what about the opposite, pre-configured queue is durable and we received source with NONE value for TerminusDurability?

    I would greatly appreciate your help. :)

    opened by Havret 41
  • ARTEMIS-1626 Disable thread leak check for failing tests

    ARTEMIS-1626 Disable thread leak check for failing tests

    The ThreadLeakCheckRule is used to check thread leaks after each test is finished. However when a test fails, it is not necessary to check leaking threads because the test failure should be fixed anyway. And leaking threads in a failed test may well be a result of the failure (once the test is fixed the thread leak may be gone).

    If a failed test also leaks threads, it takes a long time before the thread leak check finishes (60 seconds checking time), thus it takes a long time to finish, especially when tests are run in batches with failures.

    So to improve this, it should be reasonable to just enable the thread leaking check for each test passes, and disable the check when a test fails.

    opened by gaohoward 37
  • ARTEMIS-2336 Use zero copy to replicate journal/page/large message file (AGAIN)

    ARTEMIS-2336 Use zero copy to replicate journal/page/large message file (AGAIN)

    I've opened this PR for discussion: I would like to re-introduce ARTEMIS-2336, but I've allowed wildfly or any user that doesn't want/can to use zero copy to be able to use the existing artemis code.

    I've opened https://github.com/netty/netty/pull/9592 too to "enhance" ChunkedNioFile in order to solve a bug on our implementation: in the meantime I've "shadowed" my solution on Netty directly into AbsoluteChunkedNioFile to not rely on any specific Netty version.

    This PR could make use for InVM connection the same optimization sent on https://github.com/apache/activemq-artemis/pull/2844 while reading file (RandomAccessFile) on ReplicationSyncFileMessage.

    DO-NOT-MERGE-YET 
    opened by franz1981 36
  • ARTEMIS-2399 Improve performance when there are a lot of subscribers

    ARTEMIS-2399 Improve performance when there are a lot of subscribers

    opened by wy96f 35
  • ARTEMIS-994 Support Netty Native Epoll on Linux

    ARTEMIS-994 Support Netty Native Epoll on Linux

    The following changes are made to support Epoll.

    • Refactored SharedNioEventLoopGroup into renamed SharedEventLoopGroup to be generic (as so we can re-use for both Nio and Epoll)
    • Add support and toggles for Epoll in NettyAcceptor and NettyConnector (with fall back to NIO if cannot load Epoll)
    • Removal from code of PartialPooledByteBufAllocator, caused bad address when doing native, and no longer needed - see jira discussion

    New Connector Properties:

    useEpoll - toggles to use epoll or not, default true (but we failback to nio gracefully) epollRemotingThreads = same behaviour as nioRemotingThreads but for Epoll. useEpollGlobalWorkerPool = same behaviour as useNioGlobalWorkerPool but for Epoll.

    New Acceptor Properties:

    useEpoll - toggles to use epoll or not, default true (but we failback to nio gracefully) useEpollGlobalWorkerPool = same behaviour as useNioGlobalWorkerPool but for Epoll.

    opened by michaelandrepearce 32
  • ARTEMIS-1883 Improve FQQN validation

    ARTEMIS-1883 Improve FQQN validation

    This pull request improves the validation of Fully Qualified Queue Name (FQQN) strings as per issue ARTEMIS-1883 by validating the presence of content in both sides of the "::" separator.

    opened by rtista 30
  • ARTEMIS-4122 support timed refresh for LegacyLDAPSecuritySettingPlugin

    ARTEMIS-4122 support timed refresh for LegacyLDAPSecuritySettingPlugin

    Some LDAP servers (e.g. OpenLDAP) do not support the "persistent search" feature and therefore the existing "listener" feature does not actually fetch updates. This commit implements a "pull" feature controlled by a configurable interval equivalent to what is implemented in the cached LDAP authorization module from ActiveMQ "Classic."

    opened by jbertram 0
  • ARTEMIS-3707 fixing tests

    ARTEMIS-3707 fixing tests

    A handful of tests started to fail after the original fix was committed. This commit fixes those failures mainly by using a mock TransactionSynchronizationRegistry.

    I changed o.a.a.a.r.ActiveMQRAManagedConnection#checkTransactionActive slightly because getTransactionStatus will never return null unlike getTransaction would. The semantics should still be the same, though.

    opened by jbertram 0
  • Update ActiveMQSession.java

    Update ActiveMQSession.java

         When calling Session.recover(), unacknowledged messages must be returned to the queue, that is, ClientSession.rollback (false). If you call ClientSession.rollback (true), the ActiveMQServerMessagePlugin.messageAcknowledged () is called. Which is illogical and misleading. However, this introduces the problem that deliveryCount does not increase on rereads. ut this needs to be fixed elsewhere.
    
    opened by Yesenkov 16
Owner
The Apache Software Foundation
The Apache Software Foundation
Mirror of Apache Kafka

Apache Kafka See our web site for details on the project. You need to have Java installed. We build and test Apache Kafka with Java 8, 11 and 15. We s

The Apache Software Foundation 23.9k Jan 5, 2023
Mirror of Apache RocketMQ

Apache RocketMQ Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level c

The Apache Software Foundation 18.5k Dec 28, 2022
FLiP: StreamNative: Cloud-Native: Streaming Analytics Using Apache Flink SQL on Apache Pulsar

StreamingAnalyticsUsingFlinkSQL FLiP: StreamNative: Cloud-Native: Streaming Analytics Using Apache Flink SQL on Apache Pulsar Running on NVIDIA XAVIER

Timothy Spann 5 Dec 19, 2021
Apache Camel is an open source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data.

Apache Camel Apache Camel is a powerful, open-source integration framework based on prevalent Enterprise Integration Patterns with powerful bean integ

The Apache Software Foundation 4.7k Dec 31, 2022
Apache Pulsar - distributed pub-sub messaging system

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API. Learn more about Pulsar at https:

The Apache Software Foundation 12.1k Jan 4, 2023
An XMPP server licensed under the Open Source Apache License.

Openfire About Openfire is a real time collaboration (RTC) server licensed under the Open Source Apache License. It uses the only widely adopted open

Ignite Realtime 2.6k Jan 3, 2023
Kryptonite is a turn-key ready transformation (SMT) for Apache Kafka® Connect to do field-level 🔒 encryption/decryption 🔓 of records. It's an UNOFFICIAL community project.

Kryptonite - An SMT for Kafka Connect Kryptonite is a turn-key ready transformation (SMT) for Apache Kafka® to do field-level encryption/decryption of

Hans-Peter Grahsl 53 Jan 3, 2023
RocketMQ-on-Pulsar - A protocol handler that brings native RocketMQ protocol to Apache Pulsar

RocketMQ on Pulsar(RoP) RoP stands for RocketMQ on Pulsar. Rop broker supports RocketMQ-4.6.1 protocol, and is backed by Pulsar. RoP is implemented as

StreamNative 88 Jan 4, 2023
Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data.

Dagger Dagger or Data Aggregator is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processi

Open DataOps Foundation 238 Dec 22, 2022
Template for an Apache Flink project.

Minimal Apache Flink Project Template It contains some basic jobs for testing if everything runs smoothly. How to Use This Repository Import this repo

Timo Walther 2 Sep 20, 2022
Mirror of Apache ActiveMQ

Welcome to Apache ActiveMQ Apache ActiveMQ is a high performance Apache 2.0 licensed Message Broker and JMS 1.1 implementation. Getting Started To hel

The Apache Software Foundation 2.1k Jan 2, 2023
Mirror of Apache Deltaspike

Apache DeltaSpike Documentation Mailing Lists Contribution Guide JIRA Apache License v2.0 Apache DeltaSpike is a suite of portable CDI Extensions inte

The Apache Software Foundation 141 Jan 1, 2023
Mirror of Apache Mahout

Welcome to Apache Mahout! The goal of the Apache Mahout™ project is to build an environment for quickly creating scalable, performant machine learning

The Apache Software Foundation 2k Jan 4, 2023
Mirror of Apache Kafka

Apache Kafka See our web site for details on the project. You need to have Java installed. We build and test Apache Kafka with Java 8, 11 and 15. We s

The Apache Software Foundation 23.9k Jan 5, 2023
Mirror of Apache RocketMQ

Apache RocketMQ Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level c

The Apache Software Foundation 18.5k Dec 28, 2022
Mirror of Apache Storm

Master Branch: Storm is a distributed realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processi

The Apache Software Foundation 6.4k Dec 26, 2022
Mirror of Apache SIS

============================================= Welcome to Apache SIS <http://sis.apache.org> ============================================= SIS is a Ja

The Apache Software Foundation 81 Dec 26, 2022
Mirror of Apache Cassandra

Apache Cassandra Apache Cassandra is a highly-scalable partitioned row store. Rows are organized into tables with a required primary key. Partitioning

The Apache Software Foundation 7.7k Jan 1, 2023
Mirror of Apache SystemML

Apache SystemDS Overview: SystemDS is a versatile system for the end-to-end data science lifecycle from data integration, cleaning, and feature engine

The Apache Software Foundation 940 Dec 25, 2022
Mirror of Apache SystemML

Apache SystemDS Overview: SystemDS is a versatile system for the end-to-end data science lifecycle from data integration, cleaning, and feature engine

The Apache Software Foundation 940 Dec 25, 2022