Description
This repo can be used as demo repo for finding memory leaks.
Example spring-boot project to show how to find and fix
- JVM HEAP memory leak
- Native memory leak from the
.so
shared library file written in C++
More explanation about the how to find memory leaks is explained in the blog post: BLOG_POST_URL
Application details
The Application contains 2 rest end points
- To demonstrate how JVM HEAP memory leak looks like
Method: POST
URL: http://localhost:8080/users/heap
Example Body:
{
"first_name" : "Iranna",
"last_name" : "Patil",
"email" : "[email protected]"
}
- To demonstrate Native shared library memory leak
Method: POST
URL: http://localhost:8080/users/native
Example Body:
{
"first_name" : "Iranna",
"last_name" : "Patil",
"email" : "[email protected]"
}
Prerequisites
- Ubuntu(.so) and mac-os(.dylib)
- Java-15: OpenJDK-15
- Maven - 3.6.3
- g++ compiler for C++ code
hey
load generator: https://github.com/rakyll/heyjconsol
from jdk- Yourkit profiles for JVM application profiling: https://www.yourkit.com/
jemalloc
for process memory profiling: https://www.yourkit.com/
Running the Application.
Below are the steps to run the application.
- As the application makes JNI call, it needs shared library in the class path, so we need to build
.so
for the code written in C++. All the C++ code present in the folder "cpp", I have already generated header file for the JAVA native method code and also implemented corresponding logic in C++. Below are the steps to generate.so
library file from C++ code usingg++
compiler.- Compile the CPP file
g++ -c -fPIC -I{path}/jdk-15.0.2/include/ -I{path}/jdk-15.0.2/include/linux -I{path_to_cpp_files} {path_to_cpp_files}/datastore.cpp -o {output_directory_path}/datastore.o
- Convert the CPP compiled code to shared library`
g++ -shared -fPIC -o {output_directory_path}/libdatastore.so {output_directory_path}/datastore.o -lc
.so
file in foldercpp/lib/libdatastore
, and this can be used for the demo if you have any issues in generating this file from C++ code. - Once we generated the
.so
we can build our applicationjar
file.
mvn clean install
-
After generating JAR, now we can run the application and start sending the request. As we want to profile the whole process we want to use
jemallo
instead ofglibc/libc
to allocate memory. Please follow the steps to createjemalloc
library and how to use https://github.com/jemalloc/jemalloc/wiki/ Once we havejemalloc
built now we can start our application using thejemalloc
and our own native code shared liblibdatastore.so
as shown below. Open new terminal as do below steps.- Make jemalloc as memory allocator instead of glibc/lic malloc
export LD_PRELOAD=/usr/local/lib/libjemalloc.so
Make sure you built
jemalloc
lib locally and placed it in above directory by following instructions fromjemalloc
documentation.- Configure jemalloc so that it will write heap details of the process at every fixed interval to a file.
export MALLOC_CONF=prof:true,lg_prof_interval:31,lg_prof_sample:17,prof_prefix:/tmp/heap/jeprof
- Set library path to folder were our
.so
native code library present so that JVM can call native methods, or else we will getLinking exceptions
export LD_LIBRARY_PATH={output_directory_path}
- Now run the java application from the jar.
java -Xmx500m -Xms250m -jar target/MemoryLeakDemo-1.0-SNAPSHOT.jar
We can check whether application is alive or not using alive endpoint http://localhost:8080/alive
Generating load
We can generate load using light weight async load generator following below steps.
- Create a
user.json
file having json content which will be used as body for requests.
user.json
{
"first_name" : "Iranna",
"last_name" : "Patil",
"email" : "[email protected]"
}
- Now using this file we can generate load on any of the
/users/heas
or/users/native
endpoint as shown below
hey -c 5 -z 20m -m POST -T "application/json" -D user.json http://localhost:8080/users/native
Checking memory leak
Heap
We can check the heap leak by connecting profiler to the application(Yourkit or jconsol)
Native
As we told the jemalloc
to generate the heap details files in a directory, while starting the application. Now we can use jeprof
to plot the graph or text file to show the memory details and also show the leak.
#!/bin/bash
jeprof --svg /tmp/heap/jeprof.$1.* >/tmp/heap/$1-report.svg 2>/dev/null
jeprof --text /tmp/heap/jeprof.$1.* >/tmp/heap/$1-report.txt 2>/dev/null
The parameter will be the porcess_id
Same script file can be found in this repo under folder profile
Once this svg
and txt
files generated, we can open them and see for memory leak details.