a concolic testing engine for Java

Overview

CATG

CATG is a concolic unit testing engine for Java programs. The implementation uses ASM for instrumentation. ASM instrumentation instruments (see janala.instrument.) class files at runtime and dumps (see janala.logger.) to a file a log of all instructions executed by the program and all values loaded from local stacks and heaps. A concolic execution engine (see janala.interpreters.*) then takes the log and performs both symbolic and concrete interpretation of the logged instructions.

You must have java, gradle, cvc4 (http://cvc4.cs.nyu.edu/downloads/), and python 2.7 in your PATH. Two extra jar files are needed to run the tool. Create a lib directory in the root directory and download the following jar files

place them in the lib directory. Then invoke

gradle build

and

gradle integrationTest

To run the the full integration tests, use

./setup.sh  # This builds CATG and copy it to lib/
python testall.py

To run tests and see coverage report, use

gradle build
gradle jacocoTestReport

This runs the tests using online concolic execution. To run tests with offline concolic execution, use

python --offline testall.py

If you want to generate tests on a Java class file having a main method, you need to use the concolic.py script. For example, the following command generates test inputs for the class tests.Testme (the java source of this class can be found in src/integration/java/tests/Testme.java.

python concolic.py --coverage 100 tests.Testme

Usage

python janala/concolic.py -h usage: concolic.py [-h] [--offline] [-v] [-c] [-D D] maxIterations className [arguments [arguments ...]]

positional arguments:

maxIterations   Maximum number of times the program under test can be
              executed.
className       Java class to be tested.
arguments       Arguments passed to the program under test.

optional arguments:

-h, --help      show this help message and exit
--offline       Perform concolic testing offline. An intermediate trace file
              is generated during the execution of the program. offilne
              mode results in 2X slowdown that non-offline mode
-v, --verbose   Print commands that are executed.
-c, --coverage  Compute detailed coverage by rerunning tests.
-D D            JVM options
You might also like...

Java DSL for easy testing of REST services

Java DSL for easy testing of REST services

Testing and validation of REST services in Java is harder than in dynamic languages such as Ruby and Groovy. REST Assured brings the simplicity of usi

Dec 25, 2022

Sniffy - interactive profiler, testing and chaos engineering tool for Java

Sniffy - interactive profiler, testing and chaos engineering tool for Java

Sniffy Sniffy is a Java profiler which shows the results directly in your browser. It also brings profiling to your unit (or rather component) tests a

Dec 23, 2022

QuickPerf is a testing library for Java to quickly evaluate and improve some performance-related properties

QuickPerf is a testing library for Java to quickly evaluate and improve some performance-related properties

QuickPerf is a testing library for Java to quickly evaluate and improve some performance-related properties quickperf.io 📙 Documentation Annotations

Dec 15, 2022

A collection of JUnit rules for testing code which uses java.lang.System.

System Rules System Rules is a collection of JUnit rules for testing code which uses java.lang.System. System Lambda is an alternative to System Rules

Dec 22, 2022

Advanced Java library for integration testing, mocking, faking, and code coverage

Codebase for JMockit 1.x releases - Documentation - Release notes How to build the project: use JDK 1.8 or newer use Maven 3.6.0 or newer; the followi

Dec 9, 2022

ScalaTest is a free, open-source testing toolkit for Scala and Java programmers

ScalaTest is a free, open-source testing toolkit for Scala and Java programmers.

Dec 26, 2022

Practice and testing with Java 11, Prometheus, and Spring-boot with MicroService Architecture. Designed to run on Kubernetes in minikube.

Practice and testing with Java 11, Prometheus, and Spring-boot with MicroService Architecture. Designed to run on Kubernetes in minikube.

This application was written by Andrew Aslakson Built to run on minikube using kubernetes General race tracking system? Secure with Firebase Authentic

Feb 5, 2022

Spring REST API for financial management, developed with Java 11, JWT for authentication, JUnit for unit testing and Oracle Database

Spring REST API for financial management, developed with Java 11, JWT for authentication, JUnit for unit testing and Oracle Database

control_financial Spring REST API for financial management, developed with Java 11, JWT for authentication, JUnit for unit testing and Oracle Database

May 27, 2022

Benchmark testing number reading/writing in Java.

double-reader-writer Benchmark testing number reading/writing in Java. Relates to FasterXML/jackson-core#577 So far, FastDoubleParser looks useful if

Apr 12, 2022

Isolated MinIO container management for Java code testing

TestContainers for MinIO MinIO support for the test containers project. Installation Unfortunately, TestContainers for MinIO is not available in any p

Sep 30, 2022

F5 BIG-IP iControl REST vulnerability RCE exploit with Java including a testing LAB

F5 BIG-IP iControl REST vulnerability RCE exploit with Java including a testing LAB

CVE-2022-1388 F5 BIG-IP iControl REST vulnerability RCE exploit with Java and ELF. Included Scan a single target Scan many targets Exploit with a shel

Sep 24, 2022

Toolkit for testing multi-threaded and asynchronous applications

ConcurrentUnit A simple, zero-dependency toolkit for testing multi-threaded code. Supports Java 1.6+. Introduction ConcurrentUnit was created to help

Dec 30, 2022

Cucumber DSL for testing RESTful Web Services

Cucumber DSL for testing RESTful Web Services

cukes-rest takes simplicity of Cucumber and provides bindings for HTTP specification. As a sugar on top, cukes-rest adds steps for storing and using r

Oct 18, 2022

Randomized Testing (Core JUnit Runner, ANT, Maven)

RANDOMIZED TESTING ================== JUnit test runner and plugins for running JUnit tests with pseudo-randomness. See the following for more infor

Dec 26, 2022

JVM version of Pact. Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.

pact-jvm JVM implementation of the consumer driven contract library pact. From the Ruby Pact website: Define a pact between service consumers and prov

Dec 31, 2022

Captures log entries for unit testing purposes

LogCaptor Install with maven dependency groupIdio.github.hakky54/groupId artifactIdlogcaptor/artifactId version2.4.0/version

Jan 1, 2023

Utility for developers and QAs what helps minimize time wasting on writing the same data for testing over and over again. Made by Stfalcon

 Utility for developers and QAs what helps minimize time wasting on writing the same data for testing over and over again. Made by Stfalcon

Stfalcon Fixturer A Utility for developers and QAs which helps minimize time wasting on writing the same data for testing over and over again. You can

Nov 29, 2021

Testing tools for javac and annotation processors

Compile Testing A library for testing javac compilation with or without annotation processors. See the javadoc for usage examples. License Copyright 2

Dec 14, 2022

Simple and clean testing for JavaFX.

TestFX Simple and clean testing for JavaFX. TestFX requires a minimum Java version of 8 (1.8). Documentation See the Javadocs for latest master. See t

Dec 28, 2022
Comments
  • Java 1.7 support?

    Java 1.7 support?

    Does the application work with .class files compiled with 1.7?

    Whenever I try to examine a simple HelloWorld have file compiled in 1.7, I get a message

    Exception in thread "main" java.lang.VerifyError: Expecting a stackmap frame at branch target 24 Exception Details: Location: HelloWorld.()V @18: invokespecial Reason: Expected stackmap frame at this location. Bytecode: 0000000: 0304 03b8 0022 2a04 0412 2312 2412 25b8 0000010: 0029 b700 01a7 0007 b800 2cbf b800 2fb8 0000020: 0032 0504 b800 36b1
    Exception Handler Table: bci [18, 24] => handler: 24

    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2570)
    at java.lang.Class.getMethod0(Class.java:2813)
    at java.lang.Class.getMethod(Class.java:1663)
    at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
    at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
    

    However, when I test out the same HelloWorld code compiled in 1.6, everything seems to work fine.

    opened by dan7800 3
  • java.lang.ArrayIndexOutOfBoundsException

    java.lang.ArrayIndexOutOfBoundsException

    Hello,

    I am running CATG on javapoet and I am facing to an exception:

    java.lang.ArrayIndexOutOfBoundsException: 6
    	at janala.interpreters.ObjectValue.setField(ObjectValue.java:104)
    	at janala.interpreters.ConcolicInterpreter.visitPUTFIELD(ConcolicInterpreter.java:1384)
    	at janala.logger.inst.PUTFIELD.visit(PUTFIELD.java:16)
    	at janala.logger.DirectConcolicExecution.log(DirectConcolicExecution.java:58)
    	at janala.logger.AbstractLogger.SPECIAL(AbstractLogger.java:706)
    	at janala.logger.DJVM.SPECIAL(DJVM.java:708)
    	at com.squareup.javapoet.CodeWriter.<init>(CodeWriter.java:41)
    	at com.squareup.javapoet.CodeWriter.<init>(CodeWriter.java:67)
    	at com.squareup.javapoet.CodeWriter.<init>(CodeWriter.java:63)
    	at com.squareup.javapoet.TypeName.toString(TypeName.java:256)
    	at com.squareup.javapoet.AnnotatedTypeNameTest.main(AnnotatedTypeNameTest.java:146)
    

    happening when I try to run CATG on:

    public static void main(String[] args) {
            String lit1 = catg.CATG.readString("java.util.List<@");
            String lit2 = catg.CATG.readString(" java.lang.String>");
            String expected = (lit1 + (AnnotatedTypeNameTest.NN)) + lit2;
            TypeName type = TypeName.get(String.class).annotated(annotatedTypeNameTest.NEVER_NULL);
            ClassName list = ClassName.get(List.class);
            String actual = ParameterizedTypeName.get(list, type).toString();
    }
    

    I assumed that there is a problem with the fields of the class com.squareup.javapoet.CodeWriter.

    Find here, an excerpt of this class:

    final class CodeWriter {
      /** Sentinel value that indicates that no user-provided package has been set. */
      private static final String NO_PACKAGE = new String();
    
      private final String indent;
      private final Appendable out;
      private int indentLevel;
    
      private boolean javadoc = false;
      private boolean comment = false;
      private String packageName = NO_PACKAGE;
      private final List<TypeSpec> typeSpecStack = new ArrayList<>();
      private final Set<String> staticImportClassNames;
      private final Set<String> staticImports;
      private final Map<String, ClassName> importedTypes;
      private final Map<String, ClassName> importableTypes = new LinkedHashMap<>();
      private final Set<String> referencedNames = new LinkedHashSet<>();
      private boolean trailingNewline;
    
      /**
       * When emitting a statement, this is the line of the statement currently being written. The first
       * line of a statement is indented normally and subsequent wrapped lines are double-indented. This
       * is -1 when the currently-written line isn't part of a statement.
       */
      int statementLine = -1;
    
      CodeWriter(Appendable out) {
        this(out, "  ", Collections.<String>emptySet());
      }
    
      CodeWriter(Appendable out, String indent, Set<String> staticImports) {
        this(out, indent, Collections.<String, ClassName>emptyMap(), staticImports);
      }
    
      CodeWriter(Appendable out, String indent, Map<String, ClassName> importedTypes,
          Set<String> staticImports) {
        this.out = checkNotNull(out, "out == null");
        this.indent = checkNotNull(indent, "indent == null");
        this.importedTypes = checkNotNull(importedTypes, "importedTypes == null");
        this.staticImports = checkNotNull(staticImports, "staticImports == null");
        this.staticImportClassNames = new LinkedHashSet<>();
        for (String signature : staticImports) {
          staticImportClassNames.add(signature.substring(0, signature.lastIndexOf('.')));
        }
      }
    

    Do you have any idea or tips to fix this issues?

    Best regards,

    Benjamin.

    opened by danglotb 2
  • How to test the class without main()?

    How to test the class without main()?

    Hello,

    I meet problem when testing the class without main() method. The concolic.py seems to always look for main() and report errors when there is no main(). Anyone can point out how can I test the one without main()? I cannot find it in README.

    Thanks!

    opened by zhenyu-zhou 1
Owner
Koushik Sen
Koushik Sen
Java Concolic Unit Testing Engine

jCUTE The Java Concolic Unit Testing Engine (jCUTE) automatically generates unit tests for Java programs. Concolic execution combines randomized concr

Open Systems Laboratory 81 Nov 7, 2022
Softmotions 32 Jul 4, 2022
Drools is a rule engine, DMN engine and complex event processing (CEP) engine for Java.

An open source rule engine, DMN engine and complex event processing (CEP) engine for Javaâ„¢ and the JVM Platform. Drools is a business rule management

KIE (Drools, OptaPlanner and jBPM) 4.9k Dec 31, 2022
Java testing framework for testing pojo methods

Java testing framework for testing pojo methods. It tests equals, hashCode, toString, getters, setters, constructors and whatever you report in issues ;)

Piotr Joński 48 Aug 23, 2022
Te4j (Template Engine For Java) - Fastest and easy template engine

Te4j About the project Te4j (Template Engine For Java) - Fastest and easy template engine Pros Extremely fast (127k renders per second on 4790K) Easy

Lero4ka16 19 Nov 11, 2022
cglib - Byte Code Generation Library is high level API to generate and transform Java byte code. It is used by AOP, testing, data access frameworks to generate dynamic proxy objects and intercept field access.

cglib Byte Code Generation Library is high level API to generate and transform JAVA byte code. It is used by AOP, testing, data access frameworks to g

Code Generation Library 4.5k Jan 8, 2023
Java DSL for easy testing of REST services

Testing and validation of REST services in Java is harder than in dynamic languages such as Ruby and Groovy. REST Assured brings the simplicity of usi

REST Assured 6.2k Dec 31, 2022
A modern testing and behavioural specification framework for Java 8

Introduction If you're a Java developer and you've seen the fluent, modern specification frameworks available in other programming languages such as s

Richard Warburton 250 Sep 12, 2022
A programmer-oriented testing framework for Java.

JUnit 4 JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks. For more infor

JUnit 8.4k Jan 4, 2023