mBERT is a mutation testing tool that uses a pre-trained language model (CodeBERT) to generate mutants.

Overview

mBERT

mBERT is a mutation testing tool that uses CodeBERT, a pre-trained language model, to generate mutants.

Requirements

  • Maven
  • CodeBERT dependencies:
    • pip install torch
    • pip install transformers

Installation

  1. Run mkdir -p pre-trained/codebert-base-mlm to create the folder where CodeBERT pre-trained model will be saved.
  2. Run python3 download-codebert.py to download CodeBERT pre-trained model.
  3. Try CodeBERT by running: python3 run-codebert.py "int = b;"

{'score': 0.23396340012550354, 'token': 740, 'token_str': 'c', 'sequence': 'int c= b;'}

{'score': 0.05450829118490219, 'token': 939, 'token_str': 'i', 'sequence': 'int i= b;'}

{'score': 0.05004948750138283, 'token': 741, 'token_str': 'b', 'sequence': 'int b= b;'}

{'score': 0.04164685681462288, 'token': 10, 'token_str': 'a', 'sequence': 'int a= b;'}

{'score': 0.023635799065232277, 'token': 181, 'token_str': 'p', 'sequence': 'int p= b;'}

  1. Compile mBERT: mvn compile.
  2. Done! Try mBERT by running: ./mBERT.sh

Executing mBERT

mBERT provides some flags that you can configure:

  • -in=source_file_name
  • -out=mutants_directory
  • -N=max_num_of_mutants
  • -m=method_name
  • -m=method_name:method_definition_line
  • -l=line_to_mutate

Compile Mutants Generated

You can use script compile-mutants.sh to compile the mutants generated by mBERT.

  • Usage: ./compile-mutants.sh mutants_dir subject_name
  • Information: After compiling the mutants, you will find the details in files mutants_dir/subject_name.csv and mutants_dir/subject_name.log
  • Example: ./compile-mutants.sh examples/generated-mutants/gcd/ gcd

Examples

Forlder examples provide the examples discussed in the paper.

Inside examples/generated-mutants you can find the mutants generated by mBERT.

You might also like...

Java Statistical Analysis Tool, a Java library for Machine Learning

Java Statistical Analysis Tool JSAT is a library for quickly getting started with Machine Learning problems. It is developed in my free time, and made

Dec 20, 2022

DFA来过滤敏感词工具。--- The sensitive word tool for java with DFA.

sensitive-word-plus sensitive-word-plus 基于 DFA 算法实现的高性能敏感词工具。 站在巨人肩膀上,本项目是根据sensitive-word 做的升级 创作目的 基于sensitive-word-plus 实现返回敏感词类型 实现一款好用敏感词工具。 基于 D

Sep 22, 2022

A tool to assemble Magic: The Gathering proxies from a set of template images

Proximity A tool to assemble Magic: The Gathering proxies from a set of template images. To get started, check out the wiki. Building Proximity can be

Oct 17, 2022

This is a tool to visualize search algorithms

This is a tool to visualize search algorithms

Path-Finding-Visualizer Purpose This is a tool to visualize search algorithms Algorithms featured Breadth First Search Deapth First Search Gready Best

Oct 20, 2022

MathParser - a simple but powerful open-source math tool that parses and evaluates algebraic expressions written in pure java

MathParser - a simple but powerful open-source math tool that parses and evaluates algebraic expressions written in pure java

MathParser is a simple but powerful open-source math tool that parses and evaluates algebraic expressions written in pure java. This projec

Dec 24, 2022

Code4Me provides automatic intelligent code completion based on large pre-trained language models

Code4Me Code4Me provides automatic intelligent code completion based on large pre-trained language models. Code4Me predicts statement (line) completio

Dec 5, 2022

State of the art mutation testing system for the JVM

Pitest (aka PIT) is a state of the art mutation testing system for Java and the JVM. Read all about it at http://pitest.org Releases 1.7.3 #952 Mutate

Dec 26, 2022

cglib - Byte Code Generation Library is high level API to generate and transform Java byte code. It is used by AOP, testing, data access frameworks to generate dynamic proxy objects and intercept field access.

cglib Byte Code Generation Library is high level API to generate and transform JAVA byte code. It is used by AOP, testing, data access frameworks to g

Jan 8, 2023

Development Driven Testing (DDT) lets you generate unit tests from a running application. Reproduce a bug, generate a properly mocked test

DDTJ: It kills bugs DDT is the flip side of TDD (Test-driven development). It stands for "Development Driven Tests". Notice that it doesn’t contradict

Dec 30, 2021

lazy-language-loader improves loading times when changing your language by only reloading the language instead of all the game resources!

lazy-language-loader lazy-language-loader improves loading times when changing your language by only reloading the language instead of all the game re

Sep 7, 2022

For English vocabulary analysis and sentence analysis in natural language, model trainin

Sword Come 🌈 For English vocabulary analysis and sentence analysis in natural language, model training, intelligent response and emotion analysis rea

Apr 9, 2022

Lightweight installer written in java, made for minecraft mods, The installer uses JPanel and uses a URL to install to the specific area (Discord URL's work the best i've seen)

InstallerForJava Lightweight installer written in java, made for minecraft mods, The installer uses JPanel and uses a URL to install to the specific a

Dec 9, 2022

This is a simple realization of custom messages pre/post processing in spring-boot HTTP/Stream requests & responses

spring-boot-custom-message-converting-instances This is a simple realization of custom messages converting in spring-boot HTTP requests and responses.

Jul 22, 2022

Pre-release of JavaGachi

JavaGachi-PreRelease Pre-release of JavaGachi Despite having Gradle resources, delegate building to IDEA if you are interested in deploying the applic

Jan 6, 2022

A pre-authenticated RCE exploit for Inductive Automation Ignition

A pre-authenticated RCE exploit for Inductive Automation Ignition

Randy What This is a pre-authenticated RCE exploit for Inductive Automation Ignition that impacts versions = 8.1.16. We failed to exploit the bugs at

Sep 23, 2022

Google's ML-Kit-Vision demo (android) for pre encoded video.

Google's ML-Kit-Vision demo (android) for pre encoded video.

Android ML Kit Vision demo with Video Google's ML-Kit-Vision demo (android) for pre encoded video. Demos for camera preview and still image are also i

Dec 29, 2022

A collection of JUnit rules for testing code which uses java.lang.System.

System Rules System Rules is a collection of JUnit rules for testing code which uses java.lang.System. System Lambda is an alternative to System Rules

Dec 22, 2022

How to configure Replica Set with Embedded Mongo using Spring Boot and Flapdoodle for unit testing code that uses mongodb transactions

Spring Boot Embedded Mongo with Replica Set This project defines a basic rest service that allows users to update records of a Person (name and email)

Nov 1, 2022
Comments
  • COMPILATION ERROR

    COMPILATION ERROR

    Hi, I am trying to run mBERT on my machine but here comes an error,

    [ERROR] COMPILATION ERROR : 
    [INFO] -------------------------------------------------------------
    [ERROR] ***/MuBERT/src/main/java/mutation/CodeBERTMutation.java:[81,64] cannot find symbol
      symbol:   method of(java.lang.String)
      location: interface java.nio.file.Path
    [INFO] 1 error
    [INFO] -------------------------------------------------------------
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD FAILURE
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time:  1.776 s
    [INFO] Finished at: 2022-10-25T09:42:43+08:00
    [INFO] ------------------------------------------------------------------------
    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project mBERT: Compilation failure
    [ERROR] ***/MuBERT/src/main/java/mutation/CodeBERTMutation.java:[81,64] cannot find symbol
    [ERROR]   symbol:   method of(java.lang.String)
    [ERROR]   location: interface java.nio.file.Path
    

    my mvn and java version,

    Maven home: /usr/local/apache-maven-3.8.6
    Java version: 1.8.0_341, vendor: Oracle Corporation, runtime: /usr/local/java/jdk8/jre
    Default locale: en, platform encoding: UTF-8
    OS name: "linux", version: "4.15.0-76-generic", arch: "amd64", family: "unix"
    

    Thanks, Albert

    opened by Albert-Gong 2
  • A Small Bug 'java.lang.ArrayIndexOutOfBoundsException'

    A Small Bug 'java.lang.ArrayIndexOutOfBoundsException'

    Hi, I find a small bug(or typo) in src/main/java/Main.java at line 39

      if (method.contains(":")) {
          String [] ml = method.split(":");
          method = ml[0];
          line = Integer.parseInt(ml[2]);
      }
    

    where m[2] should be ml[1].

    Albert

    opened by Albert-Gong 1
  • SLF4J: Failed to load class

    SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder"

    Hi, Thank you for this interesting tool. I faced another issue while running mBERT on the same SampleMethods.java in the example folder. But It generated zero mutant files. It generates map text file and mapping csv file but with zerto entry.

    The only msg or error that I receive is:

    SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
    SLF4J: Defaulting to no-operation (NOP) logger implementation
    SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
    792: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
    

    I tried to add log4j dependencies into pom.xml, but It doesn't solve the issue.

    Here is the whole output:

    Input: ./SimpleMethods.java
    Output: ./method_sample/Out/
    Settings{CODEBERT_TIMEOUT=60, MAX_NUM_OF_MUTANTS=3}
    SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
    SLF4J: Defaulting to no-operation (NOP) logger implementation
    SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
    Here is the standard error of the command (if any):
    
    2022-11-16 18:37:47.121643: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
    Here is the standard error of the command (if any):
    
    2022-11-16 18:37:54.992443: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
    Here is the standard error of the command (if any):
    
    2022-11-16 18:38:03.246794: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
    Here is the standard error of the command (if any):
    
    2022-11-16 18:38:11.397682: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
    Here is the standard error of the command (if any):
    
    2022-11-16 18:38:19.358529: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
    Here is the standard error of the command (if any):
    
    2022-11-16 18:38:27.984133: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
    Here is the standard error of the command (if any):
    
    2022-11-16 18:38:37.259234: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
    Saving mutants...
    Settings{CODEBERT_TIMEOUT=60, MAX_NUM_OF_MUTANTS=3}
    
    
    
    opened by arghavanmo 1
  • Doesn't work on a single java method

    Doesn't work on a single java method

    Hey, I am trying to run mBERT on different java methods but it seems that it cannot accept a single method as an input source file. For example instead of a java class with different methods, If I put only the below method in the source file, It failed to execute:

    public int min(int a, int b) {
    	    if (a <= b)
                return a;
            else 
                return b;
    
        }
    

    Is there any way to run it on a single method? I am using this command: Error: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0

    ./mBERT.sh -in=./method_sample/In/test.java -out=./method_sample/Out/ -N=4 -m=parse

    opened by arghavanmo 1
Owner
null
Model import deployment framework for retraining models (pytorch, tensorflow,keras) deploying in JVM Micro service environments, mobile devices, iot, and Apache Spark

The Eclipse Deeplearning4J (DL4J) ecosystem is a set of projects intended to support all the needs of a JVM based deep learning application. This mean

Eclipse Foundation 12.7k Dec 30, 2022
calculator when you be using a model that employs RPN (Reverse Polish Notation)

calculator when you be using a model that employs RPN (Reverse Polish Notation) in its calculations and be a custom build all at the same time? The kids may have colour TFTs and graphing functions, but your keyboard has no equals sign, and that means something.

Eslam ElBeak 8 Oct 28, 2021
Archinsight project tends to implement architecture-as-code definition of a standard c4 architecture model

Archinsight project tends to implement architecture-as-code definition of a standard c4 architecture model. This project offers a new Insight language designed in such way that an Architect can focus on architecture definition, not visualization. Compared to UML, the Insight language is more specific and is unable to describe an arbitrary entity, but shorter and probably easier to use.

null 25 Nov 24, 2022
A Recruitment bot that uses K-Nearest Neighbors Algorithm to determine whether you should hire the applicant or not

knn_recruitment A Recruitment bot that uses K-Nearest Neighbors Algorithm to determine whether you should hire the applicant or not It uses the data i

null 1 Feb 11, 2022
Abstract machine for formal semantics of SIMP (Simple Imperative Language)

SIMP-abstract-machine In 2020/21 I was a Teaching Assistant for the second year module 5CCS2PLD Programming Language Paradigms at King's College Londo

Sten Arthur Laane 25 Oct 10, 2022
👨‍🏫ITMO University first 4 labs. They are about object oriented programming and Java language

Java-Programming-1st-semester 1st lab - math operations, formatted output. 2nd lab - object oriented programming. 3rd lab - SOLID and STUPID principle

Andrey Vasiliev 4 Dec 1, 2022
👄 The most accurate natural language detection library for Java and the JVM, suitable for long and short text alike

Quick Info this library tries to solve language detection of very short words and phrases, even shorter than tweets makes use of both statistical and

Peter M. Stahl 532 Dec 28, 2022
MALLET is a Java-based package for statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text.

MALLET is a Java-based package for statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text.

null 900 Jan 2, 2023
Workflow management system fully supporting low-code language Petriflow.

Netgrif Application Engine MySQL Create NETGRIF user: CREATE USER 'netgrif_nae'@'localhost' IDENTIFIED BY 'netgrif_nae'; GRANT ALL PRIVILEGES ON * . *

NETGRIF 17 Dec 23, 2022
Data Structures and Algorithms (DSA) - Java Language Using Integrated Development Environments NetBeans

Data Structures and Algorithms (DSA) Course Code : CSC211 Credit Hours : 4 Language : JAVA Integrated development environments : NETBEANS Topic Covere

Ossama Mehmood 샘 2 Oct 1, 2022