Trust-java - Test Results Verification library for Java

Overview

Build Status Coverage Status Maven Central Download MIT License Free

TRUST - Test Results Verification library for Java

The TRUST's primary goal is to provide the simple way of different test results verification.

General description

Usually during or at the end of some test scenario you want to verify that the application under the tests works as expected. Sometimes it is enough to use simple assertions chain but sometimes you have to implement the complex verification logic in order to check the test scenario(s) results.
And this is the exact that stage where the TRUST might be useful since it brings some benefits:

  • Unified test results verification approach for different testing types (e.g. UI, API)
  • No need in sources recompilation after the verification rules updates (unless you store these rules inside the compilation units)
  • XPath and JSONPath selectors support (for extracting the API response nodes value and using them during the further verification stages)
  • Simple yet powerful verification rules specification (anyone with the basic knowledge of the XPath, JSONPath, JS comparison and logical operators can write these rules)

As it was mentioned above, the TRUST is based on the user defined verification rules which are the subject for the further processing by the built-in JavaScript engine (the Nashorn by default).

The verification rules are represented by the String and they consist of the following:

  • Verification type (can be any String value (please check the list of predefined types) or API_RESPONSE)
  • ':  ' symbols (note: colon and two spaces (by default), without quotes)
  • JS expression(s) which should be evaluated into the Boolean
    • JS expressions should be separated by using the ';  ' symbols (note: semicolon and two spaces (by default), without quotes)
    • JS expressions are able to utilize the Lodash JS library (by default) extra functionality
    • JS expressions should contain placeholders/templates (depending on chosen Verification Type) which will be substituted with the test result values during the expressions evaluation
      • For the UI tests the placeholders/templates are represented by the entities on the page of the test results, value of which you want to use in the JS expression
      • For the API tests the placeholders/templates are represented by the XPath / JSONPath, JSONPath expressions
  • Verification Types should be separated by using the |&| symbols
  • Verification rules are processed softly

Please note that the substituted value(s) of the placeholders/templates will be normalized before use.

You can override the defaults by using corresponding system properties. Please refer to the TrustConfig file (the TRUST settings section).

Examples

//
// The UI verification rules examples
//

// Verifies that the "Status" field value of the test results page will contain "DONE" text;
UI_COMMON:  _.includes("${Status:}", "DONE")

// Verifies that the error message of the test results page will contain "System Error" or "Was not able to perform requested action" text;
UI_ERROR:  _.includes("${ERROR_MSG}", "System Error") || _.includes("${ERROR_MSG}", "Was not able to perform requested action")

// Verifies that the alert text will not contain "TestMePlease" text;
UI_ALERT:  !_.includes("${ALERT_MSG}", "TestMePlease")

// Verifies that the "Amount" field value of the test results page will be greater than 0 and less than 100
MY_RULE:  _.gt(${Amount:}, 0) && _.lt(${Amount:}, 100)

// Together
UI_COMMON:  _.includes("${Status:}", "DONE")|&|MY_RULE:  _.gt(${Amount:}, 0) && _.lt(${Amount:}, 100)

// 
// The API verification rules examples
//

// Verifies that the API response node of the executed request will contain "Valid response" text;
// ${$.description} or ${/Response/description} placeholders will be replaced with the real value of the <Response> -> <description> response node (JSON and XML content-type respectively);
API_RESPONSE:  _.includes("${$.description}", "Valid response")
API_RESPONSE:  _.includes("${/Response/description}", "Valid response")

UI test results verification

Suppose you have the next HTML table on the page as the results of some UI test activities:

Field Name Value
Field1: data1
Field2: data2
Description: Description
Status: DONE

Suppose also you have the next verification rule for this particular test:
UI_COMMON: _.includes("${Status:}", "DONE")

In this case your job will be to parse the HTML table into the key-value pairs (the Map) and invoke the VerificationUtils.performTestResultsVerification(...) method with this Map as the testResults parameter.

During the verification procedure the ${Status:} template will be substituted with the DONE value.
After this activities the resulting expression to verify will be _.includes("DONE", "DONE") <=> Lodash syntax + JS expression(s) evaluation <=> and the current test will be marked as passed since the "DONE" string includes/contains the "DONE" string.

API test results verification

Suppose you have the API test which will produce the next response payload:

<Response>
    <node1>1</node1>
    <node2>2</node2>
    <node3>3</node3>
    <description>Valid response</description>
    <node5>5</node5>
</Response>

Suppose also you have the next verification rule for this particular test:
API_RESPONSE: _.includes("${/Response/description}", "Valid response")

So the /Response/description XPath expression will be evaluated into the Valid response value for you and will be substituted instead of the ${/Response/description} template
After this activities the resulting expression to verify will be _.includes("Valid response", "Valid response") <=> Lodash syntax + JS expression(s) evaluation <=> and the current test will be marked as passed since the "Valid response" string includes/contains the "Valid response" string.

Basic usage

Let's say that for some particular API test scenario you're using the DataProvider implemented as Iterator which consumes the test data from the CSV file.

In this case you can add the new column into the CSV file which will hold the test results verification rules:

... VERIFICATION_RULES
... API_RESPONSE: _.includes("${/Response/description}", "Valid response")

And then you can invoke the corresponding method from your test:

package my.test;

import static com.shimkiv.trust.ValidationUtils.*;
import static com.shimkiv.trust.VerificationUtils.*;

public class MyApiTests {
        @Test
        public void apiResponseTest(int testDataIndex,
                                    Map<String, String> testData) {
            String httpResponsePayload;
            String httpResponseContentType;
            
            try {
                // do some test activities 
                // in order to receive 
                // the API response payload and content-type
            } catch (Exception e) {
                // process exceptions if required
            } finally {
                // for the XML payloads only !
                validateApiResponseAgainstXsd(
                        myApiXsd, 
                        httpResponsePayload);
                
                performApiResponseVerification(
                        httpResponsePayload,
                        httpResponseContentType,
                        generateVerificationEntities(
                                testData.
                                    get(VERIFICATION_RULES_NODE_NAME)));
            }
        }
}

The Schema itself can be initialized (if required) something like this:

package my.config;

// imports

public class MyApiConfig {
        private static Schema myApiXsd = null;
        
        static {
            LOG.info("API Schemas initialization ...");
            
            myApiSchemaInit();
        }
        
        // getters/setters/etc.
        
        private static void myApiSchemaInit() {
            try {
                myApiXsd =
                        SchemaFactory.
                                newInstance(W3C_XML_SCHEMA_NS_URI).
                                newSchema(
                                        new StreamSource(
                                                findFileInClassPath(
                                                        COMMON_CONFIG_DIR,
                                                        MY_API_XSD_FILE_NAME)));
            } catch (Exception e) {
                LOG.debug(COMMON_ERROR_MESSAGE, e);
            }
        }
}

More examples

The similar approach can be used in order to write your own verification methods like:

package my.utils;

import static com.shimkiv.trust.VerificationUtils.*;

public class MyVerificationUtils {
        private static final Logger LOG;
        
        /**
         * Performs UI alert message text verification
         * 
         * For verification rule like: 
         * UI_ALERT:  !_.includes("${ALERT_MSG}", "TestMePlease")
         * 
         * @param verificationEntities {@link VerificationEntities}
         */
        public static void performUiAlertTextVerification(VerificationEntities 
                                                                verificationEntities) {
            LOG.info("About to verify the UI alert text ...");
    
            assertThat(alertIsPresent()).
                    isTrue();
    
            Map<String, String> testResults =
                    new HashMap<>();
    
            testResults.put(
                    "ALERT_MSG",
                    switchTo().
                            alert().
                            getText());
    
            verifyTestResults(
                    testResults,                                                              
                    verificationEntities.
                            getVerificationEntity(
                                    UI_ALERT.
                                        name()).
                            getVerificationRules());
    
            if(ALERTS_AUTO_CONFIRM) {
                confirmAlertQuietly();
            }
        }
    
        /**
         * Performs UI error message text verification
         * 
         * For verification rule like: 
         * UI_ERROR:  _.includes("${ERROR_MSG}", "System Error") || _.includes("${ERROR_MSG}", "Was not able to perform requested action")
         *
         * @param verificationEntities {@link VerificationEntities}
         * @param errorContainerLabel The displayed label of the error container
         */
        public static void performUiErrorMessageVerification(VerificationEntities 
                                                                    verificationEntities,
                                                             String errorContainerLabel) {
            LOG.info("About to verify the UI error ...");
    
            SelenideElement errorContainer =
                    getParentContainer(
                            errorContainerLabel);
            Map<String, String> testResults =
                    new HashMap<>();
    
            testResults.put(
                    "ERROR_MSG",
                    errorContainer.
                            getText());
    
            errorContainer.
                    shouldBe(visible);
            verifyTestResults(
                    testResults,
                    verificationEntities.
                            getVerificationEntity(
                                    UI_ERROR.
                                        name()).
                            getVerificationRules());
        }
}

Usage with Maven

You can import the dependency of the TRUST into your pom.xml from the Maven Central repository:

<dependency>
    <groupId>com.shimkiv</groupId>
    <artifactId>trust</artifactId>
    <version>LATEST</version>
</dependency>
  • And then you can import the following methods into your code:
    • import static com.shimkiv.trust.VerificationUtils.*;
    • Optional (for the API responses validation against the XSD schema):
      • import static com.shimkiv.trust.ValidationUtils.*;

Changelog

CHANGELOG

Build from sources

git clone https://github.com/shimkiv/trust-java.git
cd trust-java
mvn clean package
# or
mvn clean install

Authors

TRUST was originally designed and developed by Serhii Shymkiv in 2017

License

TRUST is open-source project and distributed under the MIT license

You might also like...

GreenMail is an open source, intuitive and easy-to-use test suite of email servers for testing purposes.

GreenMail GreenMail is an open source, intuitive and easy-to-use test suite of email servers for testing purposes. Supports SMTP, POP3, IMAP with SSL

Dec 28, 2022

Conformance test suite for OpenShift

Origin Kubernetes This repo was previously the core Kubernetes tracking repo for OKD, and where OpenShift's hyperkube and openshift-test binaries were

Jan 4, 2023

This project will help to test the Log4j CVE-2021-44228 vulnerability.

Log4j-JNDIServer This project will help to test the Log4j CVE-2021-44228/CVE-2021-45046 vulnerabilities. Installation and Building Load the project on

Jun 30, 2022

Connection Pool Test Setup

Connection Pool Test Setup Setup to test the behavior of http connection pools with the following setup server-apache apache server with proxy-pass to

Jan 21, 2022

Android PackageInstaller API silent update test app

Android PackageInstaller API silent update test app

Dec 13, 2022

Set of project to test how fast is compilation on your computer

Бенчмарк сборки Android проектов Репозиторий содержит несколько проектов, для которые необходимо запустить тесты и зарепортить их результаты Методика

Jul 29, 2022

Section B of Assignment 1. Setup project and collaborate on GitHub by writing test fixtures.

Section B of Assignment 1. Setup project and collaborate on GitHub by writing test fixtures.

Task Each member (including the team leader) should create a branch (use student number as the branch name) and include a small program in the branch

Apr 6, 2022

Spring Boot Rest API unit test with Junit 5, Mockito, Maven

Spring Boot Rest API unit testing with Junit 5, Mockito, Maven Apply Spring Boot @WebMvcTest for Rest Controller Unit Test with JUnit 5 and Mockito. F

Dec 22, 2022

httpx - CLI to test HTTP/gRPC/RSocket/Kafka... services by HTTP DSL

httpx: CLI for run http file httpx is a CLI to execute requests from JetBrains Http File. Request types supported by httpx HTTP REST PUB/SUB - Apache

Dec 15, 2022
Comments
  • [Snyk] Security upgrade com.jayway.jsonpath:json-path from 2.4.0 to 2.6.0

    [Snyk] Security upgrade com.jayway.jsonpath:json-path from 2.4.0 to 2.6.0

    Snyk has created this PR to fix one or more vulnerable packages in the `maven` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • pom.xml

    Vulnerabilities that will be fixed

    With an upgrade:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 479/1000
    Why? Has a fix available, CVSS 5.3 | Denial of Service (DoS)
    SNYK-JAVA-NETMINIDEV-1078499 | com.jayway.jsonpath:json-path:
    2.4.0 -> 2.6.0
    | No | No Known Exploit medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Denial of Service (DoS)
    SNYK-JAVA-NETMINIDEV-1298655 | com.jayway.jsonpath:json-path:
    2.4.0 -> 2.6.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
Auto-Unit-Test-Case-Generator automatically generates high-level code-coverage JUnit test suites for Java, widely used within the ANT Group.

中文README传送门 What is Auto-Unit-Test-Case-Generator Auto-Unit-Test-Case-Generator generates JUnit test suites for Java class just as its name. During te

TRaaS 108 Dec 22, 2022
Tzatziki - Decathlon library to ease and promote Test Driven Development of Java microservices!

Tzatziki Steps Library This project is a collection of ready-to-use Cucumber steps making it easy to TDD Java microservices by focusing on an outside-

Decathlon 32 Dec 15, 2022
A unit testing library for varying test data.

Burst A unit testing library for varying test data. DEPRECATED: Burst remains stable and functional, but you should check out TestParameterInjector fr

Square 464 Oct 9, 2022
Test assignment for the course from MTS.Teta Java Middle Developer. 2022

Сервис по планированию задач Список фичей: Пользователь может добавлять, удалять, закрывать и заново открывать задачи Названия задач должны быть уника

Semyon Kirekov 7 Nov 1, 2022
Test-Driven Security

Test-Driven Security Run tests ./gradlew test References Spring Security test support https://docs.spring.io/spring-security/site/docs/current/referen

Eleftheria Stein-Kousathana 60 Oct 11, 2022
Integration Test Framework for Paper!

MiniTestFramework Integration Test Framework for Paper! Usage Install plugin Create Test Structure /test create <filename.testname> [width] Build cont

MiniDigger | Martin 12 Nov 25, 2022
An examples of creating test records in the database with Spring Boot + Spring Data + JPA usage.

Spring Boot + JPA — Clear Tests An examples of creating test records in the database with Spring Boot + Spring Data + JPA usage. Check out the article

Semyon Kirekov 8 Nov 24, 2022
CVE-2021-44228 - Apache log4j RCE quick test

Build ./build.sh Start log4j RCE Server ./start-log4j-rce-server.sh Test Run java -cp log4j-rce-1.0-SNAPSHOT-all.jar log4j Check if you get logs in ha

Jeffrey Li 3 Feb 1, 2022
Test case to check if the Log4Shell/CVE-2021-44228 hotfix will raise any unexpected exceptions

Log4Shell Hotfix Side Effect Test Case I wanted to know if any ClassNotFoundException or similar unexpected exception is raised when one applies the C

Malte S. Stretz 3 Nov 9, 2022
MessageEngine by afkvido. Alpha test is the most updated but may contain many bugs

MessageEngine Alpha Alpha Testing This is the most frequently updated, fresh, and potentially most glitchy version of MessageEngine. This version will

gemsvidø 3 Feb 7, 2022