BinAbsInspector: Vulnerability Scanner for Binaries

Overview

What is BinAbsInspector?

BinAbsInspector (Binary Abstract Inspector) is a static analyzer for automated reverse engineering and scanning vulnerabilities in binaries, which is a long-term research project incubated at Keenlab. It is based on abstract interpretation with the support from Ghidra. It works on Ghidra's Pcode instead of assembly. Currently it supports binaries on x86,x64, armv7 and aarch64.

Installation

  • Install Ghidra according to Ghidra's documentation
  • Install Z3 (tested version: 4.8.15)
  • Note that generally there are two parts for Z3 library: one is Java package, the other one is native library. The Java package is already included in "/lib" directory, but we suggest that you replace it with your own Java package for version compatibility.
    • For Windows, download a pre-built package from here, extract the zip file and add a PATH environment variable pointing to z3-${version}-win/bin
    • For Linux, install with package manager is NOT recommended, there are two options:
      1. You can download suitable pre-build package from here, extract the zip file and copy z3-${version}-win/bin/*.so to /usr/local/lib/
      2. or you can build and install z3 according to Building Z3 using make and GCC/Clang
    • For MacOS, it is similar to Linux.
  • Download the extension zip file from release page
  • Install the extension according to Ghidra Extension Notes

Building

Build the extension by yourself, if you want to develop a new feature, please refer to development guide.

  • Install Ghidra and Z3
  • Install Gradle 7.x (tested version: 7.4)
  • Pull the repository
  • Run gradle buildExtension under repository root
  • The extension will be generated at dist/${GhidraVersion}_${date}_BinAbsInspector.zip

Usage

You can run BinAbsInspector in headless mode, GUI mode, or with docker.

  • With Ghidra headless mode.
$GHIDRA_INSTALL_DIR/support/analyzeHeadless <projectPath> <projectName> -import <file> -postScript BinAbsInspector "@@<scriptParams>"

<projectPath> -- Ghidra project path.
<projectName> -- Ghidra project name.
<scriptParams> -- The argument for our analyzer, provides following options:

Parameter Description
[-K <kElement>] KSet size limit K
[-callStringK <callStringMaxLen>] Call string maximum length K
[-Z3Timeout <timeout>] Z3 timeout
[-timeout <timeout>] Analysis timeout
[-entry <address>] Entry address
[-externalMap <file>] External function model config
[-json] Output in json format
[-disableZ3] Disable Z3
[-all] Enable all checkers (by default)
[-debug] Enable debugging log output
[-check "<cweNo1>[;<cweNo2>...]"] Enable specific checkers
  • With Ghidra GUI

    1. Run Ghidra and import the target binary into a project
    2. Analyze the binary with default settings
    3. When the analysis is done, open Window -> Script Manager and find BinAbsInspector.java
    4. Double-click on BinAbsInspector.java entry, set the parameters in configuration window and click OK
    5. When the analysis is done, you can see the CWE reports in console window, double-click the addresses from the report can jump to corresponding address
  • With Docker

git clone [email protected]:KeenSecurityLab/BinAbsInspector.git
cd BinAbsInspector
docker build . -t bai
docker run -v $(pwd):/data/workspace bai "@@<script parameters>" -import <file>

Implemented Checkers

So far BinAbsInspector supports following checkers:

  • CWE78 (OS Command Injection)
  • CWE119 (Buffer Overflow (generic case))
  • CWE125 (Buffer Overflow (Out-of-bounds Read))
  • CWE134 (Use of Externally-Controlled Format string)
  • CWE190 (Integer overflow or wraparound)
  • CWE367 (Time-of-check Time-of-use (TOCTOU))
  • CWE415 (Double free)
  • CWE416 (Use After Free)
  • CWE426 (Untrusted Search Path)
  • CWE467 (Use of sizeof() on a pointer type)
  • CWE476 (NULL Pointer Dereference)
  • CWE676 (Use of Potentially Dangerous Function)
  • CWE787 (Buffer Overflow (Out-of-bounds Write))

Project Structure

The structure of this project is as follows, please refer to technical details for more details.

├── main
│   ├── java
│   │   └── com
│   │       └── bai
│   │           ├── checkers                       checker implementatiom
│   │           ├── env
│   │           │   ├── funcs                      function modeling
│   │           │   │   ├── externalfuncs          external function modeling
│   │           │   │   └── stdfuncs               cpp std modeling
│   │           │   └── region                     memory modeling
│   │           ├── solver                         analyze core and grpah module
│   │           └── util                           utilities
│   └── resources
└── test

You can also build the javadoc with gradle javadoc, the API documentation will be generated in ./build/docs/javadoc.

Acknowledgement

We employ Ghidra as our foundation and frequently leverage JImmutable Collections for better performance.
Here we would like to thank them for their great help!

Comments
  • Java error

    Java error

    I am using Ubuntu 22.04.1 Desktop OS and I followed the installation of BinAbsInspector strictly. I have ghidra 10.1.2 I got the *.so files like this: https://github.com/Z3Prover/z3/releases/download/z3-4.8.15/z3-4.8.15-x64-glibc-2.31.zip $ unzip z3-4.8.15-x64-glibc-2.31.zip $ cd ~/z3-4.8.15-x64-glibc-2.31/bin $ sudo cp *.so /usr/local/lib/

    I imported the ghidra_10.1.2_PUBLIC_20220420_BinAbsInspector.zip file successfully, but I get this error in ghidra:

    kép

    Tis is the java environment: $ java --version openjdk 11.0.16 2022-07-19 OpenJDK Runtime Environment (build 11.0.16+8-post-Ubuntu-0ubuntu122.04) OpenJDK 64-Bit Server VM (build 11.0.16+8-post-Ubuntu-0ubuntu122.04, mixed mode, sharing)

    I have no idea what to do, please help!

    opened by torabi12 11
  • MacOS installation error

    MacOS installation error

    I tried using BinAbsInspector on MacOS, but got the following error: BinAbsInspector.java> Cannot detect z3 solver library, please check your z3 solver installation or disable z3 solver in configuration

    But I have successfully installed Z3 using the command: brew install z3, and I can find the z3 library in /usr/local/lib. I don't know where I went wrong. Could you give me some advice?

    opened by rssample 8
  • Paper / Internals doc?

    Paper / Internals doc?

    Hi, could you document some of the ideas behind BinAbsInspector, how it uses Z3? Is there a research paper to understand this? Super interesting! Thanks.

    documentation 
    opened by dgutson 6
  • TaintMap.getTaints may return the same taint for different callsites in the same function.

    TaintMap.getTaints may return the same taint for different callsites in the same function.

    Take com.bai.env.funcs.externalfuncs.GetenvFunction as an example. (External function model for char *getenv(const char *name))

    the source is:

        public void invoke(PcodeOp pcode, AbsEnv inOutEnv, AbsEnv tmpEnv, Context context, Function callFunc) {
            ALoc retALoc = getReturnALoc(callFunc, false);
            if (retALoc == null) {
                return;
            }
            long taints = TaintMap.getTaints(context, callFunc);
            inOutEnv.set(retALoc, KSet.getTop(taints), true);
        }
    

    context doesn't include the current callsite of external function. So, for different callsites in the current function, TaintMap.getTaints will return the same taint.

    If I analyze this shared object:

    // clang  -shared -o test.so -fPIC ./test.c 
    char *getenv(const char *name);
    
    int main() {
        getenv("test");
        getenv("aaa");
        return 0;
    }
    

    and set breakpoint at GetenvFunction.invoke, I can see the same context (main[0, 0, 0]) and the same taints (taints = 2) twice.

    I think it should return different taints?

    (by the way, I am working on a research project that heavily uses BinAbsInspector)

    opened by am009 4
  • can not choose cwe types in ghidra UI mode

    can not choose cwe types in ghidra UI mode

    image image image

    Althouth i choose the specific cwe types,it does not work as i expected。I don't known how to fix this problem。Or should i use it in headless mode? Besides,the results of cwe 787 check are really Outrageous。

    opened by hackoflpf 4
  • 启动工具失败

    启动工具失败

    严格按照安装指导书进行安装,启动工具失败,报错如下: java.lang.StackOverflowError at com.bai.solver.CFG.visit(CFG.java:80) at com.bai.solver.CFG.visit(CFG.java:84) at com.bai.solver.CFG.visit(CFG.java:84)


    Build Date: 2022-Jan-25 1526 EST Ghidra Version: 10.1.2 Java Home: C:\Program Files\Eclipse Adoptium\jdk-11.0.14.101-hotspot JVM Version: Eclipse Adoptium 11.0.14.1 OS: Windows 10 10.0 amd64

    opened by Bling1234 4
  • headless mode

    headless mode

    My command is : analyzeHeadless ./bai bai_2022_09_09 -import ./test2 -postScript BinAbsInspector "@@ -all" And i get the error followed: root@3893d60a101e:/bai# analyzeHeadless ./bai bai_2022_09_09 -import ./test2 -postScript BinAbsInspector "@@ -all" openjdk version "17.0.4.1" 2022-08-12 OpenJDK Runtime Environment Temurin-17.0.4.1+1 (build 17.0.4.1+1) OpenJDK 64-Bit Server VM Temurin-17.0.4.1+1 (build 17.0.4.1+1, mixed mode) INFO Using log config file: jar:file:/bai/ghidra_10.1.2_PUBLIC/Ghidra/Framework/Generic/lib/Generic.jar!/generic.log4j.xml (LoggingInitialization) INFO Using log file: /root/.ghidra/.ghidra_10.1.2_PUBLIC/application.log (LoggingInitialization) INFO Loading user preferences: /root/.ghidra/.ghidra_10.1.2_PUBLIC/preferences (Preferences) INFO Loading previous preferences: /root/.ghidra/.ghidra_10.1.5_PUBLIC/preferences (Preferences) INFO Class search complete (1497 ms) (ClassSearcher) INFO Initializing SSL Context (SSLContextInitializer) INFO Initializing Random Number Generator... (SecureRandomFactory) INFO Random Number Generator initialization complete: NativePRNGNonBlocking (SecureRandomFactory) INFO Trust manager disabled, cacerts have not been set (ApplicationTrustManagerFactory) INFO HEADLESS Script Paths: /bai/ghidra_10.1.2_PUBLIC/Ghidra/Features/MicrosoftCodeAnalyzer/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Features/Base/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Processors/DATA/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Features/FileFormats/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Debug/Debugger/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Processors/PIC/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Debug/Debugger-agent-dbgmodel-traceloader/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Features/Python/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Features/BytePatterns/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Extensions/BinAbsInspector/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Features/Decompiler/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Processors/8051/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Features/FunctionID/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Features/VersionTracking/ghidra_scripts /bai/ghidra_10.1.2_PUBLIC/Ghidra/Features/GnuDemangler/ghidra_scripts (HeadlessAnalyzer) ERROR REPORT SCRIPT ERROR: BinAbsInspector : Missing plugin needed to run scripts of this type. Please ensure you have installed the necessary plugin. (HeadlessAnalyzer) ERROR Abort due to Headless analyzer error: Invalid script: BinAbsInspector (HeadlessAnalyzer) java.lang.IllegalArgumentException: Invalid script: BinAbsInspector at ghidra.app.util.headless.HeadlessAnalyzer.checkScript(HeadlessAnalyzer.java:788) at ghidra.app.util.headless.HeadlessAnalyzer.checkScriptsList(HeadlessAnalyzer.java:801) at ghidra.app.util.headless.HeadlessAnalyzer.compileScripts(HeadlessAnalyzer.java:835) at ghidra.app.util.headless.HeadlessAnalyzer.processLocal(HeadlessAnalyzer.java:408) at ghidra.app.util.headless.AnalyzeHeadless.launch(AnalyzeHeadless.java:121) at ghidra.GhidraLauncher.launch(GhidraLauncher.java:59) at ghidra.Ghidra.main(Ghidra.java:47)

    I have tried many methods but failed in the end, could you please help me? thanks a lot

    opened by now4yreal 3
  • 工具似乎在识别64位arm寄存器上有点问题?

    工具似乎在识别64位arm寄存器上有点问题?

                     LAB_001695f8                                    XREF[2]:     001695b0(j), 00169684(j)  
            001695f8 73 52 4b b9     ldr        w19,[x19, #0xb50]=>PTR_002cfb50                  = 00000000
            001695fc e0 03 14 2a     mov        w0,w20
            00169600 a2 8f 40 b9     ldr        w2,[x29, #local_4]
            00169604 61 02 40 b9     ldr        w1,[x19]                                         CWE476: Null pointer dereference
    
    

    这里报空指针异常,应该是指[X19]寄存器里面的内容吧,实际上它是在w19中被赋值了的。

    opened by diablo0822 3
  • Expected CODE symbol

    Expected CODE symbol

    this line has a error in function getVarArgsSignature

    DataTypeSymbol symbol = HighFunctionDBUtil.readOverride(symbols[0]);

    log: [INFO - BinAbsInspector] Running solver on "entry()" function ERROR REPORT SCRIPT ERROR: ( /bin/ls ) /root/.ghidra/.ghidra_10.1.2_PUBLIC/Extensions/BinAbsInspector/ghidra_scripts/BinAbsInspector.java : Expected CODE symbol (HeadlessAnalyzer) java.lang.IllegalArgumentException: Expected CODE symbol at ghidra.program.model.pcode.DataTypeSymbol.readSymbol(DataTypeSymbol.java:128) at ghidra.program.model.pcode.HighFunctionDBUtil.readOverride(HighFunctionDBUtil.java:704) at com.bai.env.funcs.externalfuncs.VarArgsFunctionBase.getVarArgsSignature(VarArgsFunctionBase.java:157) at com.bai.checkers.MemoryCorruption.checkExternalCallParameters(MemoryCorruption.java:284) at com.bai.solver.PcodeVisitor.visit_CALL(PcodeVisitor.java:684) at com.bai.solver.PcodeVisitor.visit(PcodeVisitor.java:1334) at com.bai.solver.PcodeVisitor.visit(PcodeVisitor.java:1466) at com.bai.env.Context.loop(Context.java:304) at com.bai.env.Context.mainLoop(Context.java:463) at com.bai.solver.InterSolver.run(InterSolver.java:35) at BinAbsInspector.analyze(BinAbsInspector.java:95) at BinAbsInspector.run(BinAbsInspector.java:152) at ghidra.app.script.GhidraScript.executeNormal(GhidraScript.java:379) at ghidra.app.script.GhidraScript.doExecute(GhidraScript.java:234) at ghidra.app.script.GhidraScript.execute(GhidraScript.java:212) at ghidra.app.util.headless.HeadlessAnalyzer.runScript(HeadlessAnalyzer.java:576) at ghidra.app.util.headless.HeadlessAnalyzer.runScriptsList(HeadlessAnalyzer.java:909) at ghidra.app.util.headless.HeadlessAnalyzer.analyzeProgram(HeadlessAnalyzer.java:1057) at ghidra.app.util.headless.HeadlessAnalyzer.processFileWithImport(HeadlessAnalyzer.java:1550) at ghidra.app.util.headless.HeadlessAnalyzer.processWithImport(HeadlessAnalyzer.java:1688) at ghidra.app.util.headless.HeadlessAnalyzer.processWithImport(HeadlessAnalyzer.java:1753) at ghidra.app.util.headless.HeadlessAnalyzer.processLocal(HeadlessAnalyzer.java:445) at ghidra.app.util.headless.AnalyzeHeadless.launch(AnalyzeHeadless.java:121) at ghidra.GhidraLauncher.launch(GhidraLauncher.java:59) at ghidra.Ghidra.main(Ghidra.java:47)

    startup command: analyzeHeadless ~ tmp "-deleteProject" "-overwrite" "-postScript" "BinAbsInspector.java" "@@-all" -import /bin/ls

    opened by snowhax 3
  • getEntryFunction doesn't handle PE format

    getEntryFunction doesn't handle PE format

    Current getEntryFunction implementation can only handle an ELF header, thus the extension can't locate an entry point for PE/PE+ files automatically.

    ghidra

    The logic of analyze function seems to be odd, as there's a generic way of locating an entry point through the analyzeFromMain function, which isn't called at all if the executable header wasn't successfully parsed.

    Another thing, that analyzeFromMain is only trying to locate a global main function, which isn't always present in, for example, executables created in MASM. Probably it should locate and utilize an entry function address, if main is not present.

    opened by kaimi- 3
  • How could I install Z3 solver?

    How could I install Z3 solver?

    I have downloaded z3-4.8.17-x86-win.zip and unziped it.And set the PATH environment to z3-4.8.17-x86-win\bin .But BinAbsInspector.java still tell me :Cannot detect z3 solver library, please check your z3 solver installation or disable z3 solver in configuration. How could I install z3 solver?

    opened by lightbordwin 2
  • Exception in thread

    Exception in thread "main" ghidra.util.exception.InvalidInputException: /data/workspace/BinAbsInspector/test is not a valid directory or file

    您好,我使用docker方式进行搭建这个程序,在分析一个elf文件的时候出现这种错误,请问大佬如何解决

    image

    docker run -v $(pwd):/data/workspace bai "@@