AllPairs4J - an open source Java library for generation of minimal set of test combinations

Related tags

Testing java testing test-automation testing-tools allpairs pairwise
Overview

AllPairs4J

GitHub Workflow Status Maven Central

AllPairs4J is an open source Java library for generation of minimal set of test combinations.

AllPairs4J is a Java port of allpairspy project (with some improvements and bugfixes) developed by MetaCommunications Engineering and Tsuyoshi Hombashi.

Pairwise Testing

In computer science, all-pairs testing or pairwise testing is a combinatorial method of software testing that, for each pair of input parameters to a system, tests all possible discrete combinations of those parameters. For example, if you want to create a test suite for browser testing, the domain can be described by the following parameters:

Browser:    Chrome, Firefox, Safari, Edge
OS:         Windows, Linux, macOS
RAM:        1024, 2048, 4096, 8192, 16384
Drive:      HDD, SSD

There are hundreds of possible combinations of these values. Instead of spending unreasonable amount of time testing them all, pairwise suggests testing all possible pairs of values. For example, {Chrome, Windows} is one pair, {4096, SSD} is another; together they represent a test case that covers many pairs: {Chrome, Windows, 4096, SSD}. In the end, you have good coverage while the number of test cases remains manageable.

With AllPairs4J, you are also able to add limitations on the domain to restrict generation of certain pairs. For example, specify that Safari can only be paired with macOS, and Edge can only be paired with Windows.

For more info on pairwise testing see https://www.pairwise.org/.

Features

  • Specify constraints to add limitations on the test combinations generation.
  • Generate pairwise, triplewise, n-wise test combinations.
  • Integrate with any Java project. There are pairwise tools that can sometimes be faster or generate smaller set covering all the test combinations (pict is probably your best choice). However, they are missing features mentioned above and/or do not integrate with Java.

Installation

Requirements

Java 8 or higher.

Gradle

dependencies {
    implementation 'io.github.pavelicii:allpairs4j:1.0.0'
}

Maven

<dependency>
    <groupId>io.github.paveliciigroupId>
    <artifactId>allpairs4jartifactId>
    <version>1.0.0version>
dependency>

Usage

Generate Pairwise Combinations

You can configure AllPairs generation using AllPairsBuilder. After it is built, the test cases are generated.
As a minimal setup, you need to describe Parameters to generate test combinations from.

Sample code:

AllPairs allPairs = new AllPairs.AllPairsBuilder()
        .withParameters(Arrays.asList(
                new Parameter("Browser", "Chrome", "Safari", "Edge"),
                new Parameter("OS", "Windows", "Linux", "macOS"),
                new Parameter("RAM", 2048, 4096, 8192, 16384),
                new Parameter("Drive", "HDD", "SSD")))
        .build();

System.out.println(allPairs);

Output:

Show 
  1: {Browser=Chrome, OS=Windows, RAM=2048, Drive=HDD}
  2: {Browser=Safari, OS=Linux, RAM=4096, Drive=HDD}
  3: {Browser=Edge, OS=macOS, RAM=8192, Drive=HDD}
  4: {Browser=Edge, OS=Linux, RAM=16384, Drive=SSD}
  5: {Browser=Safari, OS=Windows, RAM=16384, Drive=SSD}
  6: {Browser=Chrome, OS=macOS, RAM=4096, Drive=SSD}
  7: {Browser=Chrome, OS=Linux, RAM=8192, Drive=SSD}
  8: {Browser=Safari, OS=macOS, RAM=2048, Drive=SSD}
  9: {Browser=Edge, OS=Windows, RAM=4096, Drive=HDD}
 10: {Browser=Edge, OS=Windows, RAM=2048, Drive=HDD}
 11: {Browser=Safari, OS=macOS, RAM=16384, Drive=HDD}
 12: {Browser=Chrome, OS=Linux, RAM=16384, Drive=SSD}
 13: {Browser=Safari, OS=Linux, RAM=8192, Drive=SSD}
 14: {Browser=Chrome, OS=Windows, RAM=8192, Drive=HDD}
 15: {Browser=Edge, OS=Linux, RAM=2048, Drive=HDD}

Generate Filtered Pairwise Combinations

To limit generated pairs, you need to describe Constraints. Each Constraint is a Predicate that is tested against each generated Case. If it evaluates to true, the Case is deleted from the result.

For example, let's create limitations so that:

  • Browser=Safari can only be paired with OS=macOS
  • Browser=Edge can only be paired with OS=Windows
  • RAM can't be less than 4000

Sample code:

c.get("Browser").equals("Safari") && !c.get("OS").equals("macOS")) .withConstraint(c -> c.get("Browser").equals("Edge") && !c.get("OS").equals("Windows")) .withConstraint(c -> (int) c.get("RAM") < 4000) .build(); int index = 1; for (Case c : allPairs) { System.out.printf("%3d: %s%n", index++, c); }"> 
AllPairs allPairs = new AllPairs.AllPairsBuilder()
        .withParameter(new Parameter("Browser", "Chrome", "Safari", "Edge"))
        .withParameter(new Parameter("OS", "Windows", "Linux", "macOS"))
        .withParameter(new Parameter("RAM", 2048, 4096, 8192, 16384))
        .withParameter(new Parameter("Drive", "HDD", "SSD"))
        .withConstraint(c -> c.get("Browser").equals("Safari") && !c.get("OS").equals("macOS"))
        .withConstraint(c -> c.get("Browser").equals("Edge") && !c.get("OS").equals("Windows"))
        .withConstraint(c -> (int) c.get("RAM") < 4000)
        .build();

int index = 1;
for (Case c : allPairs) {
    System.out.printf("%3d: %s%n", index++, c);
}

Output:

Show 
  1: {Browser=Chrome, OS=Windows, RAM=4096, Drive=HDD}
  2: {Browser=Safari, OS=macOS, RAM=8192, Drive=HDD}
  3: {Browser=Edge, OS=Windows, RAM=16384, Drive=SSD}
  4: {Browser=Edge, OS=Windows, RAM=8192, Drive=SSD}
  5: {Browser=Safari, OS=macOS, RAM=4096, Drive=SSD}
  6: {Browser=Chrome, OS=Linux, RAM=16384, Drive=HDD}
  7: {Browser=Edge, OS=Windows, RAM=4096, Drive=HDD}
  8: {Browser=Chrome, OS=macOS, RAM=16384, Drive=SSD}
  9: {Browser=Safari, OS=macOS, RAM=16384, Drive=HDD}
 10: {Browser=Chrome, OS=Linux, RAM=8192, Drive=SSD}
 11: {Browser=Chrome, OS=Linux, RAM=4096, Drive=SSD}

Generate Triplewise Combinations

You can specify test combination size to go beyond pairs.

Sample code:

AllPairs allPairs = new AllPairs.AllPairsBuilder()
        .withTestCombinationSize(3)
        .withParameters(Arrays.asList(
                new Parameter("Browser", "Chrome", "Safari", "Edge"),
                new Parameter("OS", "Windows", "Linux", "macOS"),
                new Parameter("RAM", 2048, 4096, 8192, 16384),
                new Parameter("Drive", "HDD", "SSD")))
        .build();

System.out.println(allPairs.toString());

Output:

Show 
  1: {Browser=Chrome, OS=Windows, RAM=2048, Drive=HDD}
  2: {Browser=Safari, OS=Linux, RAM=4096, Drive=HDD}
  3: {Browser=Edge, OS=macOS, RAM=8192, Drive=HDD}
  4: {Browser=Edge, OS=macOS, RAM=16384, Drive=SSD}
  5: {Browser=Safari, OS=Linux, RAM=16384, Drive=SSD}
  6: {Browser=Chrome, OS=Windows, RAM=8192, Drive=SSD}
  7: {Browser=Chrome, OS=Windows, RAM=4096, Drive=HDD}
  8: {Browser=Safari, OS=Linux, RAM=2048, Drive=HDD}
  9: {Browser=Edge, OS=macOS, RAM=2048, Drive=HDD}
 10: {Browser=Edge, OS=macOS, RAM=4096, Drive=HDD}
 11: {Browser=Safari, OS=Linux, RAM=8192, Drive=SSD}
 12: {Browser=Chrome, OS=Windows, RAM=16384, Drive=SSD}
 13: {Browser=Chrome, OS=Windows, RAM=16384, Drive=HDD}
 14: {Browser=Safari, OS=Linux, RAM=4096, Drive=SSD}
 15: {Browser=Edge, OS=macOS, RAM=2048, Drive=SSD}
 16: {Browser=Edge, OS=macOS, RAM=8192, Drive=SSD}
 17: {Browser=Safari, OS=Linux, RAM=8192, Drive=HDD}
 18: {Browser=Chrome, OS=Windows, RAM=2048, Drive=SSD}
 19: {Browser=Chrome, OS=Windows, RAM=4096, Drive=SSD}
 20: {Browser=Safari, OS=Linux, RAM=16384, Drive=HDD}
 21: {Browser=Edge, OS=macOS, RAM=16384, Drive=HDD}
 22: {Browser=Edge, OS=macOS, RAM=4096, Drive=SSD}
 23: {Browser=Safari, OS=Linux, RAM=2048, Drive=SSD}
 24: {Browser=Chrome, OS=Windows, RAM=8192, Drive=HDD}
 25: {Browser=Chrome, OS=Linux, RAM=8192, Drive=HDD}
 26: {Browser=Safari, OS=macOS, RAM=2048, Drive=SSD}
 27: {Browser=Edge, OS=Windows, RAM=4096, Drive=SSD}
 28: {Browser=Edge, OS=Windows, RAM=16384, Drive=HDD}
 29: {Browser=Safari, OS=macOS, RAM=16384, Drive=HDD}
 30: {Browser=Chrome, OS=Linux, RAM=4096, Drive=SSD}
 31: {Browser=Chrome, OS=Linux, RAM=2048, Drive=SSD}
 32: {Browser=Safari, OS=macOS, RAM=8192, Drive=HDD}
 33: {Browser=Edge, OS=Windows, RAM=8192, Drive=HDD}
 34: {Browser=Safari, OS=macOS, RAM=4096, Drive=SSD}
 35: {Browser=Edge, OS=Windows, RAM=2048, Drive=SSD}
 36: {Browser=Chrome, OS=Linux, RAM=16384, Drive=HDD}
 37: {Browser=Chrome, OS=macOS, RAM=16384, Drive=HDD}
 38: {Browser=Edge, OS=Linux, RAM=2048, Drive=SSD}
 39: {Browser=Safari, OS=Windows, RAM=4096, Drive=SSD}
 40: {Browser=Safari, OS=Windows, RAM=8192, Drive=HDD}
 41: {Browser=Edge, OS=Linux, RAM=8192, Drive=HDD}
 42: {Browser=Chrome, OS=macOS, RAM=4096, Drive=SSD}
 43: {Browser=Chrome, OS=macOS, RAM=2048, Drive=SSD}
 44: {Browser=Edge, OS=Linux, RAM=16384, Drive=HDD}
 45: {Browser=Safari, OS=Windows, RAM=16384, Drive=HDD}
 46: {Browser=Safari, OS=Windows, RAM=2048, Drive=SSD}
 47: {Browser=Edge, OS=Linux, RAM=4096, Drive=SSD}
 48: {Browser=Chrome, OS=macOS, RAM=8192, Drive=HDD}

Configuration summary

Sample code:

AllPairs allPairs = new AllPairs.AllPairsBuilder()
        .withParameters( List<Parameter> )        // required
        .withParameter( Parameter )               // alternative way to specify Parameters one by one
        .withConstraints( List<Predicate<Case>> ) // not required, default is no Constraints
        .withConstraint( Predicate<Case> )        // alternative way to specify Constraints one by one
        .withTestCombinationSize( int )           // not required, default is 2
        .printEachCaseDuringGeneration()          // not required, useful for debug
        .build();

List<Case> generatedCases = allPairs.getGeneratedCases();

Data types:

  • Parameter: Named List storing all input values.
  • Case: Map storing one generated test case, where key is mapped to the Parameter name, value is mapped to one of the Parameter values.
  • Predicate : constraint to test against Case. When evaluates to true, the Case is considered invalid and is filtered out form the resulting set.

    • Contributing

    Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

    Please make sure to update tests as appropriate.

    SemVer is used for versioning. For the versions available, see the releases on this repository.

You might also like...

The ultimate KitPvP Core with a ton of features! Fully configurable & Open source.

KitPvP-Core The ultimate KitPvP Core with a ton of features! Fully configurable & Open source. Placeholders My core plugin offers several features wit

Nov 14, 2022

Never debug a test again: Detailed failure reports and hassle free assertions for Java tests - Power Asserts for Java

Never debug a test again: Detailed failure reports and hassle free assertions for Java tests - Power Asserts for Java

Scott Test Reporter for Maven and Gradle Get extremely detailed failure messages for your tests without assertion libraries, additional configuration

Nov 17, 2022

A sample repo to help you handle basic auth for automation test in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

A sample repo to help you handle basic auth for automation test in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

How to handle basic auth for automation test in Java-selenium on LambdaTest Prerequisites Install and set environment variable for java. Windows - htt

Jul 13, 2022

A sample repo to help you run automation test in incognito mode in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

A sample repo to help you run automation test in incognito mode in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

How to run automation test in incognito mode in Java-selenium on LambdaTest Prerequisites Install and set environment variable for java. Windows - htt

Jul 13, 2022

A sample repo to help you handle cookies for automation test in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

A sample repo to help you handle cookies for automation test in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

How to handle cookies for automation test in Java-selenium on LambdaTest Prerequisites Install and set environment variable for java. Windows - https:

Jul 13, 2022

A sample repo to help you capture JavaScript exception for automation test in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

A sample repo to help you capture JavaScript exception for automation test in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

How to capture JavaScript exception for automation test in Java-selenium on LambdaTest Prerequisites Install and set environment variable for java. Wi

Jul 13, 2022

A sample repo to help you find an element by text for automation test in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

A sample repo to help you find an element by text for automation test in Java-selenium on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

How to find an element by text for automation test in Java-selenium on LambdaTest Prerequisites Install and set environment variable for java. Windows

Jul 13, 2022

A sample repo to help you emulate network conditions in Java-selenium automation test on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

A sample repo to help you emulate network conditions in Java-selenium automation test on LambdaTest. Run your Java Selenium tests on LambdaTest platform.

How to emulate network conditions in Java-selenium automation test on LambdaTest Prerequisites Install and set environment variable for java. Windows

Jul 13, 2022

PowerMock is a Java framework that allows you to unit test code normally regarded as untestable.

PowerMock is a Java framework that allows you to unit test code normally regarded as untestable.

Writing unit tests can be hard and sometimes good design has to be sacrificed for the sole purpose of testability. Often testability corresponds to go

Dec 28, 2022
Owner
Pavel Nazimok
QA Automation Engineer / SDET
Pavel Nazimok
GitHub
A sample repo to help you set geolocation for automation test in Java-TestNG on LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

How to set geolocation for automation test in Java-TestNG on LambdaTest Environment Setup Global Dependencies Install Maven Or Install Maven with Home

null 12 Jul 13, 2022
A sample repo to help you set device mode using CDP in Java-TestNG automation test on LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

How to set device mode using CDP in Java-TestNG automation test on LambdaTest Environment Setup Global Dependencies Install Maven Or Install Maven wit

null 11 Jul 13, 2022
JUnit 5 Parameterized Test Yaml Test Data Source

Yamaledt — JUnit 5 Parameterized Tests Using Yaml and Jamal Introduction and usage Note This is the latest development documentation. This is a SNAPSH

Peter Verhas 4 Mar 23, 2022
Apache JMeter - An Open Source Java application designed to measure performance and load test applications

An Open Source Java application designed to measure performance and load test applications. By The Apache Software Foundation What Is It? Apache JMete

The Apache Software Foundation 6.7k Jan 1, 2023
A powerful open source test automation platform for Web Apps, Mobile Apps, and APIs

A powerful open source test automation platform for Web Apps, Mobile Apps, and APIs. Build stable and reliable end-to-end tests @ DevOps speed.

Testsigma Technologies Inc 466 Dec 31, 2022
A Java architecture test library, to specify and assert architecture rules in plain Java

ArchUnit is a free, simple and extensible library for checking the architecture of your Java code. That is, ArchUnit can check dependencies between pa

TNG Technology Consulting GmbH 2.5k Jan 2, 2023
A library for setting up Java objects as test data.

Beanmother Beanmother helps to create various objects, simple and complex, super easily with fixtures for testing. It encourages developers to write m

Jaehyun Shin 113 Nov 7, 2022
ScalaTest is a free, open-source testing toolkit for Scala and Java programmers

ScalaTest is a free, open-source testing toolkit for Scala and Java programmers.

ScalaTest 1.1k Dec 26, 2022
Serenity BDD is a test automation library designed to make writing automated acceptance tests easier, and more fun.

That feeling you get when you know you can trust your tests Serenity BDD is a library designed to make writing automated acceptance tests easier, and

Serenity BDD 654 Dec 28, 2022
Core for open source libraries, included some important Classes for those libs.

OpenSource Core You could also read the CHINESE version of README This is a very useful Java class library. In this update, we have merged the origina

Theodore Hills 10 Nov 16, 2022