Toolkit for testing multi-threaded and asynchronous applications

Overview

ConcurrentUnit

Build Status Maven Central License JavaDoc

A simple, zero-dependency toolkit for testing multi-threaded code. Supports Java 1.6+.

Introduction

ConcurrentUnit was created to help developers test multi-threaded or asynchronous code. It allows you to perform assertions and wait for operations in any thread, with failures being properly reported back to the main test thread. If an assertion fails, your test fails, regardless of which thread the assertion came from.

Usage

  1. Create a Waiter
  2. Use Waiter.await to block the main test thread.
  3. Use the Waiter.assert calls from any thread to perform assertions.
  4. Once expected assertions are completed, use Waiter.resume call to unblock the awaiting thread.

When your test runs, assertion failures will result in the main thread being interrupted and the failure thrown. If an await call times out before all expected resume calls occur, the test will fail with a TimeoutException.

Examples

Consider a test for a message bus that delivers messages asynchronously:

@Test
public void shouldDeliverMessage() throws Throwable {
  final Waiter waiter = new Waiter();

  messageBus.registerHandler(message -> {
    // Called on separate thread
    waiter.assertEquals(message, "foo");
    waiter.resume();
  };
  
  messageBus.send("foo");
  
  // Wait for resume() to be called
  waiter.await(1000);
}

We can also handle wait for multiple resume calls:

@Test
public void shouldDeliverMessages() throws Throwable {
  final Waiter waiter = new Waiter();

  messageBus.registerHandler(message -> {
    waiter.assertEquals(message, "foo");
    waiter.resume();
  };
  
  messageBus.send("foo");
  messageBus.send("foo");
  messageBus.send("foo");
  
  // Wait for resume() to be called 3 times
  waiter.await(1000, 3);
}

If an assertion fails in any thread, the test will fail as expected:

@Test(expected = AssertionError.class)
public void shouldFail() throws Throwable {
  final Waiter waiter = new Waiter();

  new Thread(() -> {
    waiter.assertTrue(false);
  }).start();
  
  waiter.await();
}

TimeoutException is thrown if resume is not called before the await time is exceeded:

@Test(expected = TimeoutException.class)
public void shouldTimeout() throws Throwable {
  new Waiter().await(1);
}

Alternatively

As a more concise alternative to using the Waiter class, you can extend the ConcurrentTestCase:

class SomeTest extends ConcurrentTestCase {
  @Test
  public void shouldSucceed() throws Throwable {
    new Thread(() -> {
      doSomeWork();
      threadAssertTrue(true);
      resume();
    }).start();

    await(1000);
  }
}

Assertions

ConcurrentUnit's Waiter supports the standard assertions along with Hamcrest Matcher assertions:

waiter.assertEquals(expected, result);
waiter.assertThat(result, is(equalTo(expected)));

Since Hamcrest is an optional dependency, users need to explicitly add it to their classpath (via Maven/Gradle/etc).

Other Examples

More example usages can be found in the WaiterTest or in the following projects:

Additional Notes

On await / resume Timing

Since it is not always possible to ensure that resume is called after await in multi-threaded tests, ConcurrentUnit allows them to be called in either order. If resume is called before await, the resume calls are recorded and await will return immediately if the expected number of resumes have already occurred. This ability comes with a caveat though: it is not possible to detect when additional unexpected resume calls are made since ConcurrentUnit allows an await call to follow.

Additional Resources

License

Copyright 2011-2016 Jonathan Halterman - Released under the Apache 2.0 license.

Comments
  • All unit tests after a failing one are not executed

    All unit tests after a failing one are not executed

    After a test failure, all the remaining tests are always reported as successfully executed no matter the real test result. Here a simple reproducer:

        @Test
        public void testOne() throws Throwable {
            performTest();
        }
    
        @Test
        public void testTwo() throws Throwable {
            performTest();
        }
    
        void performTest() throws Throwable {
              final Waiter waiter = new Waiter();
              new Thread(new Runnable() {
                @Override
                public void run() {
                  // ------- HERE A FAILING ASSERTION -------
                  waiter.assertTrue(false);
                  waiter.resume();
                }
              }).start();
              waiter.await(100);
        }
    

    Both the tests are supposed to fail, but instead only the first one fails. If additional tests are added to the same class, all of them will be marked as "passed" even if they fail. This can be easily reproduced adding

        @Test
        public void testThree() throws Throwable {
            performTest();
        }
    

    to the previous example.

    Due to this issue it is not really possible to know which tests are failing and which are really passing.

    opened by ufoscout 9
  • Licensing

    Licensing

    Source files are missing license headers. Would you please contact upstream to fix it? https://fedoraproject.org/wiki/Packaging:LicensingGuidelines?rd=Packaging/LicensingGuidelines#License_Clarification concurrentunit-concurrentunit-0.4.2/src/main/java/net/jodah/concurrentunit/ConcurrentTestCase.java concurrentunit-concurrentunit-0.4.2/src/main/java/net/jodah/concurrentunit/Waiter.java concurrentunit-concurrentunit-0.4.2/src/main/java/net/jodah/concurrentunit/internal/ReentrantCircuit.java

    From my point of view the Apache Software License 2.0 is fine, see https://fedoraproject.org/wiki/Licensing:Main?rd=Licensing#SoftwareLicenses But it looks like that upstream has forgotten to add a header to the License in tarball. See APPENDIX: How to apply the Apache License to your work http://www.apache.org/licenses/LICENSE-2.0

    Downstream package review: https://bugzilla.redhat.com/show_bug.cgi?id=1305365

    opened by rapgro 3
  • Support Hamcrest's matcher

    Support Hamcrest's matcher

    Hamcrest's matcher allows to write readable and typeable tests. https://code.google.com/p/hamcrest/wiki/Tutorial

    The following code snippet would be enough to implement this feature but I haven't tried it yet.

    public class Waiter {
        // ...
        public <T> void assertThat(T actual, org.hamcrest.Matcher<T> matcher) {
            try {
                org.hamcrest.MatcherAssert.assertThat("", actual, matcher);
            } catch (AssertionError e) {
                fail(e);
            }
        }
        // ...
    }
    

    When declaring a dependency for hamcrest, if you set its optional flag to true, you can still make concurrentunit zero-dependency. That means user should declare hamcrest dependency to use this feature.

    <dependency>
        <groupId>org.hamcrest</groupId>
        <artifactId>hamcrest-all</artifactId>
        <version>1.3</version>
        <optional>true</optional>
    </dependency>
    

    This feature can reduce some assertion code by half.

    // Using assertTrue
    threadAssertTrue(http.headerNames().containsAll(Arrays.asList("a", "b")) || http.headerNames().containsAll(Arrays.asList("A", "B")));
    
    // Using assertThat
    threadAssertThat(http.headerNames(), either(hasItems("a", "b")).or(hasItems("A", "B")));
    

    What do you think? if you like it, I can work on a pull request.

    opened by flowersinthesand 3
  • ConcurrentTestCase doesn't work with JUnit's timeout support

    ConcurrentTestCase doesn't work with JUnit's timeout support

    Here's a demo to reproduce the issue. https://github.com/flowersinthesand/concurrentunit-junit-timeout-issue

    Clone that repository and type mvn test. Then you will get:

    [INFO] Scanning for projects...
    [INFO]                                                                         
    [INFO] ------------------------------------------------------------------------
    [INFO] Building issue-report 1.0.0-SNAPSHOT
    [INFO] ------------------------------------------------------------------------
    [INFO] 
    [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ issue-report ---
    [INFO] Deleting c:\Users\Donghwan\Documents\GitHub\concurrentunit-junit-issue\target
    [INFO] 
    [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ issue-report ---
    [INFO] Using 'UTF-8' encoding to copy filtered resources.
    [INFO] skip non existing resourceDirectory c:\Users\Donghwan\Documents\GitHub\concurrentunit-junit-issue\src\main\resources
    [INFO] 
    [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ issue-report ---
    [INFO] No sources to compile
    [INFO] 
    [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ issue-report ---
    [INFO] Using 'UTF-8' encoding to copy filtered resources.
    [INFO] skip non existing resourceDirectory c:\Users\Donghwan\Documents\GitHub\concurrentunit-junit-issue\src\test\resources
    [INFO] 
    [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ issue-report ---
    [INFO] Changes detected - recompiling the module!
    [INFO] Compiling 1 source file to c:\Users\Donghwan\Documents\GitHub\concurrentunit-junit-issue\target\test-classes
    [INFO] 
    [INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ issue-report ---
    [INFO] Surefire report directory: c:\Users\Donghwan\Documents\GitHub\concurrentunit-junit-issue\target\surefire-reports
    
    -------------------------------------------------------
     T E S T S
    -------------------------------------------------------
    Running DemoTest
    on test1's await main
    on test1's resume Thread-0
    on test2's await Time-limited test
    on test2's resume Thread-1
    on test3's await Time-limited test
    Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.281 sec <<< FAILURE!
    test3(DemoTest)  Time elapsed: 0 sec  <<< ERROR!
    java.lang.IllegalStateException: Must be called from within the main test thread
        at net.jodah.concurrentunit.Waiter.await(Waiter.java:149)
        at net.jodah.concurrentunit.Waiter.await(Waiter.java:90)
        at net.jodah.concurrentunit.ConcurrentTestCase.await(ConcurrentTestCase.java:73)
        at DemoTest.test3(DemoTest.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.lang.Thread.run(Thread.java:745)
    
    
    Results :
    
    Tests in error: 
      test3(DemoTest): Must be called from within the main test thread
    
    Tests run: 3, Failures: 0, Errors: 1, Skipped: 0
    
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD FAILURE
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 2.936 s
    [INFO] Finished at: 2015-08-01T20:48:37+09:00
    [INFO] Final Memory: 12M/163M
    [INFO] ------------------------------------------------------------------------
    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project issue-report: There are test failures.
    [ERROR] 
    [ERROR] Please refer to c:\Users\Donghwan\Documents\GitHub\concurrentunit-junit-issue\target\surefire-reports for the individual test results.
    [ERROR] -> [Help 1]
    [ERROR] 
    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
    [ERROR] Re-run Maven using the -X switch to enable full debug logging.
    [ERROR] 
    [ERROR] For more information about the errors and possible solutions, please read the following articles:
    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
    

    And here's the cause: https://github.com/flowersinthesand/concurrentunit-junit-timeout-issue/blob/master/src%2Ftest%2Fjava%2FDemoTest.java#L62-L63

    As a workaround, I created a waiter per a test: https://github.com/cettia/asity/blob/5bc9d2bc19e1916626426bb601f42e29ec3d87da/test/src/main/java/io/cettia/asity/test/support/ConcurrentTestBase.java#L13-L23

    opened by flowersinthesand 3
  • Await for a specific time unit always uses milliseconds

    Await for a specific time unit always uses milliseconds

    Await for a specific time unit does not delegate to the waiter's appropriate method. Is that the expected behaviour?

    https://github.com/jhalterman/concurrentunit/blob/c88779fec230189cd14724fc0217e20736e0d07d/src/main/java/net/jodah/concurrentunit/ConcurrentTestCase.java#L117

    opened by xaviarias 2
  • Await not waiting in junit test case.

    Await not waiting in junit test case.

    Is there a trick to getting await() to actually hold up junit from completing the test?

    It's plowing right through and not waiting for resume.

    Junit 4.12 ConcurrentUnit 0.4.2

        @Test
        public void testRequestSuccess() throws Exception {
            final Waiter waiter = new Waiter();
    
            LoginTestClass api_model = new LoginTestClass();
    
            HttpUrl url = HttpUrl.parse(
                    String.format(
                            "%s/%s/login",
                            api_model.x(),
                            api_model.y()));
    
            api_model.request(
                    url,
                    new APIModel.APICallback() {
                        @Override
                        public void onFailure(Call call, APIError apiError, IOException error) {
                            Assert.assertNull(apiError);
                            Assert.assertNull(error);
                            waiter.resume();
                        }
    
                        @Override
                        public void onResponse(Call call, APIModel.APIResult result, Response response) throws IOException {
                            Assert.assertNotNull(result);
                            Assert.assertEquals(
                                    "some-id",
                                    result.apiResult().get("id"));
                            waiter.resume();
                        }
                    }
            );
    
            waiter.await(2000);
        }
    
    Awaiting Feedback 
    opened by williamsjj 2
  • Include example for multiple threads

    Include example for multiple threads

    Something like the following but with concurrentunit:

        final CountDownLatch l = new CountDownLatch(count);
        for (int x=0; x < count; ++x) {
            new Thread(new Runnable() {
                @Override
                public void run() {
                    assertEquals(1, 2);
                    l.countDown();
                }
            }).start();
        }
        l.await();
    
    opened by letmaik 2
  • enable GitHub Actions CI

    enable GitHub Actions CI

    replace TravisCI with GitHub Actions https://arstechnica.com/information-technology/2021/09/travis-ci-flaw-exposed-secrets-for-thousands-of-open-source-projects/

    opened by sullis 1
  • Include expected vs actual resume counts in TIMEOUT_MESSAGE

    Include expected vs actual resume counts in TIMEOUT_MESSAGE

    When developing tests it's helpful to set the expected number of resumes to a value larger than you expect to confirm that no unexpected resume()'s have been introduced.

    I realize that you could search (+/- 1) the count but it's easier just to print the value.

    opened by dtrott 1
  • Added OSGi support

    Added OSGi support

    I've OSGi-ed the library by adding the maven-bundle-plugin. This makes it easier to use it in an OSGi-project. If necessary, I'll try to add an Pax Exam integration-test to prove it works in an OSGi-container.

    Please have a look...

    opened by ponziani 1
  • Fixes #10 support Hamcrest's matcher

    Fixes #10 support Hamcrest's matcher

    It seems there are no tests for assertion itself like one verifying assertEquals in WaiterTest so I didn't add a test for assertThat.

    Please let me know that if it has some issues.

    opened by flowersinthesand 1
  • GitHub Actions setup-java v3

    GitHub Actions setup-java v3

    https://github.com/actions/setup-java/releases/tag/v3.0.0

    https://github.blog/changelog/2021-08-30-github-actions-setup-java-now-supports-dependency-caching/

    opened by sullis 0
  • Capture assertions failures without waiting

    Capture assertions failures without waiting

    In some use cases, it would be nice to be able to capturing assertion failures to report later without necessarily waiting on anything. Ex:

    runInThread() -> {
      waiter.assertTrue(false);
    });
    
    waiter.reportFailures();
    
    Feature 
    opened by jhalterman 0
  • Use OpenTest4j assertions

    Use OpenTest4j assertions

    There's now a standard for exceptions in testing frameworks that IDE's can universally understand.

    Converting to these assertions would make the exceptions reported by this tool more useful.

    https://github.com/ota4j-team/opentest4j

    opened by JLLeitschuh 0
  • Facilitate starting threads' work at the same time

    Facilitate starting threads' work at the same time

    As you yourself apparently noted at https://www.planetgeek.ch/2009/08/25/how-to-find-a-concurrency-bug-with-java/ :

    This pattern looks similar to what was generalized into the ConcurrentUnit library for testing multi-threaded code

    But there's one thing missing, this part from the article:

    // wait until all threads are ready
    assertTrue("Timeout initializing threads! Perform long lasting initializations before passing runnables to assertConcurrent", allExecutorThreadsReady.await(runnables.size() * 10, TimeUnit.MILLISECONDS));
    // start all test runners
    afterInitBlocker.countDown();
    

    Waiting for threads' actual readiness would maximize test's chances at revealing concurrency problems.

    Question 
    opened by VsevolodGolovanov 1
  • Consider failure handling wrapper

    Consider failure handling wrapper

    Consider an API to wrap closures that can handle any failures and re-throw them in the main test thread. Ex:

    messageBus.registerHandler(waiter.handleFailures(() -> msg -> {
      throw SomethingBadException();
    });
    
    opened by jhalterman 0
Owner
Jonathan Halterman
Jonathan Halterman
Java testing framework for testing pojo methods

Java testing framework for testing pojo methods. It tests equals, hashCode, toString, getters, setters, constructors and whatever you report in issues ;)

Piotr Joński 48 Aug 23, 2022
Awaitility is a small Java DSL for synchronizing asynchronous operations

Testing asynchronous systems is hard. Not only does it require handling threads, timeouts and concurrency issues, but the intent of the test code can

Awaitility 3.3k Dec 31, 2022
Awaitility is a small Java DSL for synchronizing asynchronous operations

Testing asynchronous systems is hard. Not only does it require handling threads, timeouts and concurrency issues, but the intent of the test code can

Awaitility 3.3k Jan 2, 2023
JVM version of Pact. Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.

pact-jvm JVM implementation of the consumer driven contract library pact. From the Ruby Pact website: Define a pact between service consumers and prov

Pact Foundation 962 Dec 31, 2022
A modern testing and behavioural specification framework for Java 8

Introduction If you're a Java developer and you've seen the fluent, modern specification frameworks available in other programming languages such as s

Richard Warburton 250 Sep 12, 2022
The Enterprise-ready testing and specification framework.

Spock Framework Spock is a BDD-style developer testing and specification framework for Java and Groovy applications. To learn more about Spock, visit

Spock Framework 3.3k Jan 5, 2023
Layout and functional testing framework for websites

Galen Framework master: Galen is an open-source tool for testing layout and responsive design of web applications. It is also a powerfull functional t

Galen Framework 1.4k Dec 10, 2022
Advanced Java library for integration testing, mocking, faking, and code coverage

Codebase for JMockit 1.x releases - Documentation - Release notes How to build the project: use JDK 1.8 or newer use Maven 3.6.0 or newer; the followi

The JMockit Testing Toolkit 439 Dec 9, 2022
Java DSL for easy testing of REST services

Testing and validation of REST services in Java is harder than in dynamic languages such as Ruby and Groovy. REST Assured brings the simplicity of usi

REST Assured 6.2k Dec 31, 2022
Cucumber DSL for testing RESTful Web Services

cukes-rest takes simplicity of Cucumber and provides bindings for HTTP specification. As a sugar on top, cukes-rest adds steps for storing and using r

C.T.Co 100 Oct 18, 2022
Randomized Testing (Core JUnit Runner, ANT, Maven)

RANDOMIZED TESTING ================== JUnit test runner and plugins for running JUnit tests with pseudo-randomness. See the following for more infor

null 167 Dec 26, 2022
Captures log entries for unit testing purposes

LogCaptor Install with maven <dependency> <groupId>io.github.hakky54</groupId> <artifactId>logcaptor</artifactId> <version>2.4.0</version>

null 215 Jan 1, 2023
A programmer-oriented testing framework for Java.

JUnit 4 JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks. For more infor

JUnit 8.4k Jan 4, 2023
TestNG testing framework

Documentation available at TestNG's main web site. Release Notes 7.4.0 7.3.0 7.1.0 7.0.0 Need help? Before opening a new issue, did you ask your quest

Cedric Beust 1.8k Jan 5, 2023
Java DSL for easy testing of REST services

Testing and validation of REST services in Java is harder than in dynamic languages such as Ruby and Groovy. REST Assured brings the simplicity of usi

REST Assured 6.2k Dec 25, 2022
🎉Back end module of Sonic UI automation testing platform. Sonic-UI自动化测试平台后端模块。

?? Sonic UI automation testing platform. English | 简体中文 Background What is sonic ? Nowadays, automation testing, remote control and other technologies

Eason 1.7k Jan 1, 2023
JVM version of Pact Enables consumer driven contract testing

JVM version of Pact. Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.

Pact Foundation 961 Dec 30, 2022
State of the art mutation testing system for the JVM

Pitest (aka PIT) is a state of the art mutation testing system for Java and the JVM. Read all about it at http://pitest.org Releases 1.7.3 #952 Mutate

Henry Coles 1.5k Dec 26, 2022