A streaming JsonPath processor in Java

Overview

JsonSurfer - Let's surf on Json!

Join the chat at https://gitter.im/jsurfer/JsonSurfer Coverage Status Maven Central

Why JsonSurfer

  • Streaming

    No need to deserialize entire json into memory.

  • JsonPath

    Selectively extract json data by the power of JsonPath.

  • Stoppable

    JsonSurfer is built on stoppable SAX-like interface that allows the processor to stop itself if necessary.

  • Non-Blocking

    JsonSurfer is event-driven and offers non-blocking parser interface.

  • Binary format

    Support multiple binary data formats including Avro, CBOR, Protobuf, Smile and Ion.

Getting started

What is JsonPath?

  • Supported JsonPath operator in JsonSurfer:
Operator Description
$ root
@ current node
* wildcard
.. recursive descent
.<name> child
['<name>' (, '<name>')] child/children
[<number> (, <number>)] index/indices
[start:end] array slice
[?(<expression>)] filter expression
  • JsonSurfer is available in cetral maven repository.

JsonSurfer has drivers for most of popular json libraries including: Gson, Jackson, FastJson and JsonSimple. Choose one and add to your POM.

<dependency>
    <groupId>com.github.jsurfer</groupId>
    <artifactId>jsurfer-gson</artifactId>
    <version>1.6.0</version>
</dependency>

<dependency>
    <groupId>com.github.jsurfer</groupId>
    <artifactId>jsurfer-jackson</artifactId>
    <version>1.6.0</version>
</dependency>

<dependency>
    <groupId>com.github.jsurfer</groupId>
    <artifactId>jsurfer-fastjson</artifactId>
    <version>1.6.0</version>
</dependency>

<dependency>
    <groupId>com.github.jsurfer</groupId>
    <artifactId>jsurfer-jsonsimple</artifactId>
    <version>1.6.0</version>
</dependency>

Usage:

Create your JsonSurfer:

  • JsonSurfer has flexible constructor. You can create yourself or pick a prebuilt one according the json library you used:
  1. Gson
        // use gson parser and use gson provider use to deserialize json into gson model i.e.com.google.gson.JsonElement
        JsonSurfer surfer = new JsonSurfer(GsonParser.INSTANCE, GsonProvider.INSTANCE);

or

        JsonSurfer surfer = JsonSurferGson.INSTANCE;
  1. Jackson
        JsonSurfer surfer = new JsonSurfer(JacksonParser.INSTANCE, JacksonProvider.INSTANCE);

or

        JsonSurfer surfer = JsonSurferJackson.INSTANCE;
  1. JsonSimple
        // use json-simple parser and json-simple provider to deserialize json into json-simple model i.e.org.json.simple.JSONObject or org.json.simple.JSONArray
        JsonSurfer surfer = new JsonSurfer(JsonSimpleParser.INSTANCE, JsonSimpleProvider.INSTANCE);

or

        JsonSurfer surfer = JsonSurferJsonSimple.INSTANCE;
  1. Fastjson
        JsonSurfer surfer = new JsonSurfer(FastJsonParser.INSTANCE, FastJsonProvider.INSTANCE);

or

        JsonSurfer surfer = JsonSurferFastJson.INSTANCE;

Collect value by JsonPath

        Collector collector = surfer.collector(sample);
        ValueBox<String> box1 = collector.collectOne("$.store.book[1].category", String.class);
        ValueBox<Object> box2 = collector.collectOne("$.store.book[2].isbn");
        ValueBox<Collection<Object>> box3 = collector.collectAll("$.store.book[*]");
        collector.exec(); // make sure exec() invoked before getting value from boxes
        box1.get();
        box2.get();
        box3.get();

【DEPRECATED】 Collect the first matched value and stop immediately

        JsonSurfer jsonSurfer = JsonSurferGson.INSTANCE;
        Object singleResult = jsonSurfer.collectOne(sample, "$.store.book[0]");

【DEPRECATED】 Colllect every matched value into a collection

        JsonSurfer jsonSurfer = JsonSurferGson.INSTANCE;
        Collection<Object> multipleResults = jsonSurfer.collectAll(sample, "$.store.book[*]");

"Surfing" in Json and collecting matched value in the listeners

        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$.store.book[*]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Reuse listener binding.

SurfingConfiguration is thread-safe as long as your listeners are stateless.

        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        SurfingConfiguration config = surfer.configBuilder()
                .bind("$.store.book[*]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .build();        
        surfer.surf(sample1, config);
        surfer.surf(sample2, config);

Compiled JsonPath

JsonPath object is immutable and can be reused safely.

Tips: Most of JsonSurfer API have two version: accepting raw JsonPath string or JsonPath object. The latter is always faster than the former without compiling JsonPath.

        JsonPath compiledPath = JsonPathCompiler.compile("$..book[1,3]['author','title']");
        String value = surfer.collectOne(read("sample.json"), String.class, compiledPath);

JsonPath Filters

  • Filter operators
Operator Description
== equal
< less than
> greater than

You can use logical operators '&&' and '||' to create more complex filter expression. For example:

$.store.book[?(@.price < 10 || @.category && @.isbn && @.price>10)].volumes[?(@.chapter == 1)]

Resolver API:

  • Limitation: Wildcard and Recursive Descent are NOT supported.
  • As of 1.2.6, JsonSurfer provides another way of processing json. You can directly resolve value with JsonPath from a well-built DOM like HashMap or even POJO:
        Book book = new Book();
        book.setAuthor("Leo");
        book.setCategory("Fiction");
        book.setPrice(100.0d);
        book.setTitle("JsonSurfer is great!");
        System.out.print(compile("$.author").resolve(book, new PoJoResolver()));

which prints "Leo".

        List<String> list = Arrays.asList("foo", "bar");
        HashMap<String, Object> map = new HashMap<String, Object>();
        map.put("list", list);
        System.out.println(compile("$.list[1]").resolve(map, JavaCollectionProvider.INSTANCE));

which prints "bar".

  • If you want to process POJO with full JsonPath feature, you can convert the POJO into binary format and then surfer on it.

Binaray format (Jackson only)

By importing Jackson binary format backend, JsonSurfer is capable to surfer with multiple binary object representation formats such as Avro, CBOR, Protobuf(A known bug to be fixed in Jackson 2.9.6), Smile and Ion.

For example, if you want to surfer with CBOR data, firstly, CBOR format backend need to be imported as dependency.

    <dependency>
        <groupId>com.fasterxml.jackson.dataformat</groupId>
        <artifactId>jackson-dataformat-cbor</artifactId>
        <version>${jackson.version}</version>
    </dependency>

Then create a JsonSurfer with CBOR-backed JacksonParser and surfer as usual

    surfer = new JsonSurfer(new JacksonParser(new CBORFactory()), provider);

Find more examples here: https://github.com/jsurfer/JsonSurfer/blob/master/jsurfer-all/src/test/java/org/jsfr/json/JacksonParserTest.java

Share data among processors

Since JsonSurfer emit data in the way of callback, it would be difficult if one of your processing depends one another. Therefore a simple transient map is added for sharing data among your processors. Following unit test shows how to use it:

        surfer.configBuilder().bind("$.store.book[1]", new JsonPathListener() {
            @Override
            public void onValue(Object value, ParsingContext context) {
                context.save("foo", "bar");
            }
        }).bind("$.store.book[2]", new JsonPathListener() {
            @Override
            public void onValue(Object value, ParsingContext context) {
                assertEquals("bar", context.load("foo", String.class));
            }
        }).bind("$.store.book[0]", new JsonPathListener() {
            @Override
            public void onValue(Object value, ParsingContext context) {
                assertNull(context.load("foo", String.class));
            }
        }).buildAndSurf(read("sample.json"));

Control parsing

  • How to pause and resume parsing.
    SurfingConfiguration config = surfer.configBuilder()
            .bind("$.store.book[0]", new JsonPathListener() {
                @Override
                public void onValue(Object value, ParsingContext context) {
                    LOGGER.info("The first pause");
                    context.pause();
                }
            })
            .bind("$.store.book[1]", new JsonPathListener() {
                @Override
                public void onValue(Object value, ParsingContext context) {
                    LOGGER.info("The second pause");
                    context.pause();
                }
            }).build();
    ResumableParser parser = surfer.getResumableParser(read("sample.json"), config);
    assertFalse(parser.resume());
    LOGGER.info("Start parsing");
    parser.parse();
    LOGGER.info("Resume from the first pause");
    assertTrue(parser.resume());
    LOGGER.info("Resume from the second pause");
    assertTrue(parser.resume());
    LOGGER.info("Parsing stopped");
    assertFalse(parser.resume());

Java 8 Streams API support

As of 1.4, JsonSurfer can create an iterator from Json and JsonPath. Matched value can be pulled from the iterator one by one without loading entire json into memory.

    Iterator iterator = surfer.iterator(read("sample.json"), JsonPathCompiler.compile("$.store.book[*]"));

Java8 user can also convert the iterator into a Stream

    Stream<Object> targetStream = StreamSupport.stream(
          Spliterators.spliteratorUnknownSize(iterator, Spliterator.ORDERED),
          false);

Functions implmented by Streams API

    public static Stream<Object> toStream(String json, String path) {
        Iterator<Object> iterator = JsonSurferJackson.INSTANCE.iterator(json, JsonPathCompiler.compile(path));
        return StreamSupport.stream(Spliterators.spliteratorUnknownSize(iterator, Spliterator.ORDERED), false);
    }

    public static void main(String[] s) throws Exception {
        String json = "[1,3,5,7,11]";
        // Count
        System.out.println(toStream(json, "$[*]").count());
        // Max
        toStream(json, "$[*]").mapToInt(o -> ((LongNode) o).asInt()).max().ifPresent(System.out::println);
        // Min
        toStream(json, "$[*]").mapToInt(o -> ((LongNode) o).asInt()).min().ifPresent(System.out::println);
        // Average
        toStream(json, "$[*]").mapToDouble(o -> ((LongNode) o).asDouble()).average().ifPresent(System.out::println);
    }

Non-Blocking parsing

As of 1.4, JsonSurfer support non-blocking parsing for JacksonParser. You can achieve 100% non-blocking JSON processing with JsonSurfer in a NIO application. Let's take a Vertx request handler as an example:

    Vertx vertx = Vertx.vertx();
    HttpServer server = vertx.createHttpServer(new HttpServerOptions());
    JsonSurfer surfer = JsonSurferJackson.INSTANCE;
    SurfingConfiguration config = surfer.configBuilder()
            .bind("$[*]", (JsonPathListener) (value, context) -> {
                // Handle json
                System.out.println(value);
            }).build();
    server.requestHandler(request -> {
        NonBlockingParser parser = surfer.createNonBlockingParser(config);
        request.handler(buffer -> {
            byte[] bytes = buffer.getBytes();
            System.out.println("Received " + bytes.length + " bytes");
            parser.feed(bytes, 0, bytes.length);
        });
        request.endHandler(aVoid -> {
            parser.endOfInput();
            System.out.println("End of request");
            request.response().end();
        });
    }).listen(8080);

Examples

Sample Json:

{
    "store": {
        "book": [
            {
                "category": "reference",
                "author": "Nigel Rees",
                "title": "Sayings of the Century",
                "price": 8.95
            },
            {
                "category": "fiction",
                "author": "Evelyn Waugh",
                "title": "Sword of Honour",
                "price": 12.99
            },
            {
                "category": "fiction",
                "author": "Herman Melville",
                "title": "Moby Dick",
                "isbn": "0-553-21311-3",
                "price": 8.99
            },
            {
                "category": "fiction",
                "author": "J. R. R. Tolkien",
                "title": "The Lord of the Rings",
                "isbn": "0-395-19395-8",
                "price": 22.99
            }
        ],
        "bicycle": {
            "color": "red",
            "price": 19.95
        }
    },
    "expensive": 10
}
JsonPath Result
$.store.book[*].author Find the authors of all books
$..author All authors
$.store.* All things in store
$.store..price The price of everything in the store
$..book[2] The third book
$..book[0,1] The first two books
$.store.book[?(@.price==8.95)] Filter all books whose price equals to 8.95
$.store.book[?(@.category=='fiction')]             Filter all books which belong to fiction category                  
$.store.book[?(@.author=~/tolkien/i)] All books matching regex

Find the authors of all books:

$.store.book[*].author
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$.store.book[*].author", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

"Nigel Rees"
"Evelyn Waugh"
"Herman Melville"
"J. R. R. Tolkien"

All authors

$..author
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$..author", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

"Nigel Rees"
"Evelyn Waugh"
"Herman Melville"
"J. R. R. Tolkien"

All things in store

$.store.*
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$.store.*", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

[{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95},{"category":"fiction","author":"Evelyn Waugh","title":"Sword of Honour","price":12.99},{"category":"fiction","author":"Herman Melville","title":"Moby Dick","isbn":"0-553-21311-3","price":8.99},{"category":"fiction","author":"J. R. R. Tolkien","title":"The Lord of the Rings","isbn":"0-395-19395-8","price":22.99}]
{"color":"red","price":19.95}

The price of everything in the store

$.store..price
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$.store..price", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

8.95
12.99
8.99
22.99
19.95

The third book

$..book[2]
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$..book[2]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

{"category":"fiction","author":"Herman Melville","title":"Moby Dick","isbn":"0-553-21311-3","price":8.99}

The first two books

$..book[0,1]
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$..book[0,1]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95}
{"category":"fiction","author":"Evelyn Waugh","title":"Sword of Honour","price":12.99}

Filter all books whose price equals to 8.95

$.store.book[?(@.price==8.95)]
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$.store.book[?(@.price==8.95)]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95}

Filter all books which belong to fiction category

$.store.book[?(@.category=='fiction')]
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$.store.book[?(@.category=='fiction')]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

{"category":"fiction","author":"Evelyn Waugh","title":"Sword of Honour","price":12.99}
{"category":"fiction","author":"Herman Melville","title":"Moby Dick","isbn":"0-553-21311-3","price":8.99}
{"category":"fiction","author":"J. R. R. Tolkien","title":"The Lord of the Rings","isbn":"0-395-19395-8","price":22.99}

All books matching regex

$.store.book[?(@.author=~/tolkien/i)]
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$.store.book[?(@.author=~/tolkien/i)]')]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

{"author":"J. R. R. Tolkien","price":22.99,"isbn":"0-395-19395-8","category":"fiction","title":"The Lord of the Rings"}

Stoppable parsing

The parsing is stopped when the first book found and printed.

$..book[0,1]
        JsonSurfer surfer = JsonSurferGson.INSTANCE;
        surfer.configBuilder()
                .bind("$..book[0,1]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {                        
                        System.out.println(value);
                        context.stop();
                    }
                })
                .buildAndSurf(sample);

Output

{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95}

Benchmark

  • JsonSurfer is fast !!! The benchmark is powered by JMH
Benchmark                                                       Mode  Cnt       Score       Error  Units
BenchmarkCollectSingleValue.benchmarkFastjson                  thrpt   10  139772.275      8854.369  ops/s
BenchmarkCollectSingleValue.benchmarkFastjsonWithJsonSurfer    thrpt   10  699176.961      23396.619  ops/s
BenchmarkCollectSingleValue.benchmarkGson                      thrpt   10  139394.358      6019.764  ops/s
BenchmarkCollectSingleValue.benchmarkGsonWithJsonSurfer        thrpt   10  632155.657      15484.499  ops/s
BenchmarkCollectSingleValue.benchmarkJackson                   thrpt   10  160545.079      7006.525  ops/s
BenchmarkCollectSingleValue.benchmarkJacksonWithJsonSurfer     thrpt   10  451870.586      13132.576  ops/s
BenchmarkCollectSingleValue.benchmarkJsonSimpleWithJsonSurfer  thrpt   10  155094.948      4457.502  ops/s
Comments
  • Support closing of underlying parser (important performance aspect for Jackson backend) automatically or explicitly

    Support closing of underlying parser (important performance aspect for Jackson backend) automatically or explicitly

    Looking at benchmark I noticed that Jackson-backed surfer seemed to be little bit slower than GSON or FastJSON one. While this could happen in some cases (different libraries have different trade-offs) I was bit surprised compared to my own tests of parsing speed, and decided to try bit of benchmarking using Java async profiler (https://github.com/jvm-profiling-tools/async-profiler).

    What I found out strongly suggests that the underlying Jackson JsonParser does not get close()d after being used, nor is all the content read (when reaching end of content, parser is auto-closed, basically, working as equivalent to close() call). This unfortunately has significant performance overhead -- Jackson uses aggressive recycling of buffers and symbol table. As a result, I think performance of Jackson-backed version may be as much as 50% lower as otherwise.

    I hope to have time to see if there was a way to augment internal APIs to call close() on at least some of the cases. I realize that due to life-cycle it is possible that this may not be trivial to do, especially on all cases. But I hope to at least show the performance difference if this could be done.

    It is also possible that other parser backend could have similar optimizations but I am not as familiar with them.

    opened by cowtowncoder 14
  • Add Support For Knowing Parsing Completed

    Add Support For Knowing Parsing Completed

    Right now on trying to convert this Api to Reactive Streams we don't have any way to know that parsing completed, hence can con signal onComplete event in Reactive Streams. So please add support for same. If I am missing something then correct me.

    opened by ankurpathak 8
  • Wildward does not work on Pojo

    Wildward does not work on Pojo

    When using jsonpath on objects having arrays or list, wildcard exception does not work. The following exception is thrown : java.lang.UnsupportedOperationException: Not supported at org.jsfr.json.path.PathOperator.resolve(PathOperator.java:48) at org.jsfr.json.path.JsonPath.resolve(JsonPath.java:177)

    package test;
    
    import java.util.Arrays;
    import java.util.List;
    
    import org.jsfr.json.compiler.JsonPathCompiler;
    import org.jsfr.json.path.JsonPath;
    import org.jsfr.json.resolver.PoJoResolver;
    import org.junit.jupiter.api.Test;
    
    public class JsonSurferTest {
    
        class B {
            private boolean c = true;
    
        }
    
        class A {
            private List<B> b = Arrays.asList(new B[] { new B(), new B()});
            private B[] b2 = new B[] { new B(), new B()};
    
        }
        
        @Test
        public void testList() {
            A a = new A();
            JsonPath compiledPath = JsonPathCompiler.compile("$.b[*].c");
            compiledPath.resolve(a, new PoJoResolver());
            
        }
        
        @Test
        public void testArray() {
            A a = new A();
            JsonPath compiledPath = JsonPathCompiler.compile("$.b2[*].c");
            compiledPath.resolve(a, new PoJoResolver());
            
        }
    }
    
    
    opened by acommuni 8
  • Add support for Java 8 Stream

    Add support for Java 8 Stream

    Thanks for your great work. Is it possible to add in support for Java 8 Stream? e.g. return a stream of objects so that the stream can emit new elements as soon as they arrive from the Json input stream.

    opened by charlesxucheng 8
  • Streaming support

    Streaming support

    JsonSurf is currently event-driven, let's consider use case of processing big json with JsonPath expression:

    1. it will not load entire json in memory
    2. it will load all results in memory (in a collection) in event listener, in order to return it to caller.

    The proposal is to provide an iterator, so that client code can interact with parser in pull manner, getting events one by one.

    opened by oleg-smith 7
  • Improve filtering on objects (argument matching nested object)

    Improve filtering on objects (argument matching nested object)

    This is related to comparison test case https://cburgmer.github.io/json-path-comparison/results/filter_expression_after_recursive_descent.html#Java_com.github.jsurfer

    Consider the following JSON:

    [
      {"letter" : "A", "number": 1, "boolean": true},
      {"letter" : "B", "number": 2, "boolean": false, "next":  {"letter" : "X", "number": 1.1, "boolean": true } },
      {"letter" : "C", "number": 3, "boolean": true, "next":  {"letter" : "X", "number": 1.2, "boolean": false } },
      {"letter" : "D", "number": 4, "boolean": true, "next":  {"letter" : "X", "number": 1.3, "boolean": true } }
    ]
    

    and the following Java snippet:

    String expression = // The JsonPath expression
    InputStream is = getClass().getClassLoader().getResourceAsStream("MultiJsonPath.json");
    JsonSurfer surfer = new JsonSurfer(JacksonParser.INSTANCE, JacksonProvider.INSTANCE);
    Collector collector = surfer.collector(is);
    ValueBox<Collection<Object>> matches = collector.collectAll(expression);
    collector.exec();
    
    Iterator<Object> iter = matches.get().iterator();
    while(iter.hasNext()){
    	Object o = iter.next();
    	System.err.println(o.getClass());
    	if(o instanceof Map) {
    		Iterator<Entry> iten = ((Map) o).entrySet().iterator();
    		while (iten.hasNext()) {
    			System.err.println(" > " + iten.next());
    	    }
        }
    }
    

    I get the following behaviour with the path expressions

    Expression: $..[?(@.letter == 'X')] (does not work as expected)

    class java.util.LinkedHashMap
     > letter=B
     > number=2
     > boolean=false
     > next={letter=X, number=1.1, boolean=true}
    class java.util.LinkedHashMap
     > letter=C
     > number=3
     > boolean=true
     > next={letter=X, number=1.2, boolean=false}
    class java.util.LinkedHashMap
     > letter=D
     > number=4
     > boolean=true
     > next={letter=X, number=1.3, boolean=true}
    

    I would have expected instead, matches of the inner JsonObjects only:

    class java.util.LinkedHashMap
     > letter=X
     > number=1.1
     > boolean=true
    class java.util.LinkedHashMap
     > letter=X
     > number=1.2
     > boolean=false
    class java.util.LinkedHashMap
     > letter=X
     > number=1.3
     > boolean=true
    

    Changing the expression to matching one object in the first 'layer' works as expected.

    Expression $..[?(@.letter == 'A')] (works as expected)

    class java.util.LinkedHashMap
     > letter=A
     > number=1
     > boolean=true
    

    Expression $..[?(@.number == 1.2)] (does not work as expected)

    class java.util.LinkedHashMap
     > letter=C
     > number=3
     > boolean=true
     > next={letter=X, number=1.2, boolean=false}
    

    I think the problem is indeed the fact that the match is including the containing object, but it should not (at least, from my understanding of JsonPath). The behaviour I would have expected is the one of Jayway, I tried with https://jsonpath.herokuapp.com . However, I want to stick to JsonSurfer for the streaming capabilities.

    For reference, the case is implemented as a test here and relates to this feature issue

    opened by enridaga 6
  • Property binding not working when it's coupled with a filtered binding

    Property binding not working when it's coupled with a filtered binding

    Hi,

    I found a strange behaviour, not sure if it's a bug or it's just me using the API in the wrong manner. (At least) With JsonSurfer and FastJsonParser, if I try this code

    public void testStrangeBehavihour() throws Exception {
            surfer.configBuilder()
                    .bind("$.store.book[?(@.category == 'reference')]", new JsonPathListener() {
                        @Override
                        public void onValue(Object value, ParsingContext context) {
                            System.out.println(value);
                        }
                    })
                    .bind("$.store.bicycle.color", new JsonPathListener() {
                        @Override
                        public void onValue(Object value, ParsingContext context) {
                            System.out.println(value); //This never happen
                        }
                    })
                    .bind("$.store.bicycle", new JsonPathListener() {
                        @Override
                        public void onValue(Object value, ParsingContext context) {
                            System.out.println(value);
                        }
                    }).buildAndSurf(read("sample.json"));
        }
    

    the second listener will never be called.

    If I remove the first binding everything works well:

        public void testStrangBehavihour() throws Exception {
            surfer.configBuilder()
                    .bind("$.store.bicycle.color", new JsonPathListener() {
                        @Override
                        public void onValue(Object value, ParsingContext context) {
                            System.out.println(value);
                        }
                    })
                    .bind("$.store.bicycle", new JsonPathListener() {
                        @Override
                        public void onValue(Object value, ParsingContext context) {
                            System.out.println(value);
                        }
                    }).buildAndSurf(read("sample.json"));
        }
    

    I tried to find an explanation in the code but found nothing.

    bug 
    opened by maciesse 6
  • Regression in 1.5.0

    Regression in 1.5.0

    Hello,

    In 1.4.3, I have a code working well like that : surfer.configBuilder() .bind("$.header", (JsonPathListener) (final Object value, final ParsingContext context) -> { }).bind("$.data[?(@.device-type=='vmanage')]", (JsonPathListener) (final Object value, final ParsingContext context) -> { }) .bind("$.data[?(@.device-type=='vsmart')]", (JsonPathListener) (final Object value, final ParsingContext context) -> { }) .buildAndSurf(entity.getContent()); Now, in 1.5.0, only the first bind found data. No data for the second and third bind (we have data with version 1.4.3) If you removed the third bind, the second bind has data. From my point of view, you have an issue when you have several binding with a filter on same attribute.

    bug 
    opened by thierrylegrain 6
  • Support script expressions

    Support script expressions

    The current JsonPath spec allows script expressions aka functions, are there any plans to support it?

    $.books.size()

    $.somestring.split(".")[0]

    opened by aljim 6
  • Not working if node key is a number

    Not working if node key is a number

    It looks like the path could not be parsed if the node key is a number, such as "$.tree.taxonomy.100177". Is there a dedicated expression that used for this case to work?

    opened by brolinuk 6
  • Empty results for root array

    Empty results for root array

    JsonSurfer returns empty results for the following test, whereas Jayway implementation returns 2 maps correctly:

    JsonSurferJackson.INSTANCE.collectAll(
    	"[\n" +
    	"    {\n" +
    	"      \"type\"  : \"iPhone\",\n" +
    	"      \"number\": \"0123-4567-8888\"\n" +
    	"    },\n" +
    	"    {\n" +
    	"      \"type\"  : \"home\",\n" +
    	"      \"number\": \"0123-4567-8910\"\n" +
    	"    }\n" +
    	"  ]", JsonPathCompiler.compile("$.*"))
    	
    
    opened by oleg-smith 5
  • Support for feeding via bytebuffers

    Support for feeding via bytebuffers

    Jackson has had NonBlockingByteBufferJsonParser from 2.14.

    Could NonBlockingParserbe upgraded to accommodate to ByteBufferFeeder to avoid having to do memory copying when source is ByteBuffer

    opened by TuomasKiviaho 2
  • Gson before 2.8.9 are vulnerable

    Gson before 2.8.9 are vulnerable

    Gson before 2.8.9 are vulnerable to Deserialization of Untrusted Data via the writeReplace() method in internal classes, which may lead to DoS attacks, according to https://security.snyk.io/vuln/SNYK-JAVA-COMGOOGLECODEGSON-1730327

    opened by dohongdayi 1
  • Create JPATH Compare using not equals (!=)

    Create JPATH Compare using not equals (!=)

    I am not sure if it was implemented but if I want to compare like say:-

    JsonPathCompiler.compile("$.data[?(@.age!=18)]")
    

    it throws exception

    |  Exception org.antlr.v4.runtime.misc.ParseCancellationException
    |        at BailErrorStrategy.recoverInline (BailErrorStrategy.java:66)
    |        at Parser.match (Parser.java:206)
    |        at JsonPathParser.filter (JsonPathParser.java:777)
    |        at JsonPathParser.relativePath (JsonPathParser.java:293)
    |        at JsonPathParser.path (JsonPathParser.java:145)
    |        at JsonPathCompiler.compile (JsonPathCompiler.java:283)
    |        at (#18:1)
    |  Caused by: org.antlr.v4.runtime.InputMismatchException
    |        at BailErrorStrategy.recoverInline (BailErrorStrategy.java:61)
    |        ...
    
    opened by sansmoraxz 0
  • JsonPath/SQL standard

    JsonPath/SQL standard

    The JsonPath language isn't standardized. When SQL accommodated it, they actually formally specified the language, however in a different flavor. Example differences:

    • array ranges are indexed as $[1 to 2] instead of $[1:2]
    • properties are quoted as $."property with space" instead of $['property with space']
    • separate strict and lax modes: in strict mode, non-existent property or out-of-bounds array index throw, in lax mode they return null.

    I'm not deeply familiar with the SQL's nuances, there might be a couple more differences. We're implementing JsonPath support in an SQL engine. My question is have you ever considered supporting the SQL flavor? Would you accept PRs adding support for them, provided that the old syntax is unaffected and the SQL way will be an alternative syntax? So far this seems possible. Or to provide a configuration setting to pick the flavor?

    opened by viliam-durina 1
  • Add benchmarks using streaming parser

    Add benchmarks using streaming parser

    I would prefer comparing the performance to the streaming apis of the competing parsers (i.e. jacksons and so), instead of first mapping to objects. At least for the simpler benchmarks, this would be more fair.

    opened by skjolber 2
  • Filter does not apply to object fields when wildcard used

    Filter does not apply to object fields when wildcard used

    Steps to reproduce

    Example JSON:

    {
        "a": {
            "size": 15
        },
        "b": {
            "size": 20
        },
        "c": {
            "size": 10
        }
    }
    

    Queries:

    $[?(@.size > 13)]
    $.*[?(@.size > 13)]
    

    Code:

    String json = "...";  // see JSON file above
    Collector collector = JsonSurferJackson.INSTANCE.collector(json);
    ValueBox<Collection<Object>> first = collector.collectAll("$[?(@.size > 10)]");
    ValueBox<Collection<Object>> second = collector.collectAll("$.*[?(@.size > 10)]");
    collector.exec();
    
    System.out.println(first.get());
    System.out.println(second.get());
    

    Expected

    The queries $.*[?(@.size > 13)] and $[?(@.size > 13)] return an array of two items:

    [ { "size": 15 }, { "size": 20 } ]
    

    Actual

    Both queries return an empty array:

    []
    

    Note

    The queries $.* and $[*] return an array of all three items:

    [
        "a": {
            "size": 15
        },
        "b": {
            "size": 20
        },
        "c": {
            "size": 10
        }
    ]
    

    So I would expect that a filter after a wildcard is working fine.

    Environment

    JsonSurfer 1.6.0 + Jackson Parser + Jackson Provider.

    opened by ghost 2
Releases(v1.6.4)
Owner
null
Java JsonPath implementation

Jayway JsonPath A Java DSL for reading JSON documents. Jayway JsonPath is a Java port of Stefan Goessner JsonPath implementation. News 10 Dec 2020 - R

null 7.6k Jan 5, 2023
A Java annotation processor used for automatically generating better builder codes.

BetterBuilder BetterBuilder is a Java annotation processor used for automatically generating better builder codes(builder design pattern), which can m

LEO D PEN 9 Apr 6, 2021
Core part of Jackson that defines Streaming API as well as basic shared abstractions

Overview This project contains core low-level incremental ("streaming") parser and generator abstractions used by Jackson Data Processor. It also incl

FasterXML, LLC 2.1k Jan 1, 2023
Convert Java to JSON. Convert JSON to Java. Pretty print JSON. Java JSON serializer.

json-io Perfect Java serialization to and from JSON format (available on Maven Central). To include in your project: <dependency> <groupId>com.cedar

John DeRegnaucourt 303 Dec 30, 2022
A Java serialization/deserialization library to convert Java Objects into JSON and back

Gson Gson is a Java library that can be used to convert Java Objects into their JSON representation. It can also be used to convert a JSON string to a

Google 21.7k Jan 8, 2023
A universal types-preserving Java serialization library that can convert arbitrary Java Objects into JSON and back

A universal types-preserving Java serialization library that can convert arbitrary Java Objects into JSON and back, with a transparent support of any kind of self-references and with a full Java 9 compatibility.

Andrey Mogilev 9 Dec 30, 2021
A simple java JSON deserializer that can convert a JSON into a java object in an easy way

JSavON A simple java JSON deserializer that can convert a JSON into a java object in an easy way. This library also provide a strong object convertion

null 0 Mar 18, 2022
Java with functions is a small java tools and utils library.

Java with functions is a small java tools and utils library.

null 4 Oct 14, 2022
Set of support modules for Java 8 datatypes (Optionals, date/time) and features (parameter names)

Overview This is a multi-module umbrella project for Jackson modules needed to support Java 8 features, especially with Jackson 2.x that only requires

FasterXML, LLC 372 Dec 23, 2022
A modern JSON library for Kotlin and Java.

Moshi Moshi is a modern JSON library for Android and Java. It makes it easy to parse JSON into Java objects: String json = ...; Moshi moshi = new Mos

Square 8.7k Dec 31, 2022
A fast JSON parser/generator for Java.

fastjson Fastjson is a Java library that can be used to convert Java Objects into their JSON representation. It can also be used to convert a JSON str

Alibaba 25.1k Dec 31, 2022
JSON to JSON transformation library written in Java.

Jolt JSON to JSON transformation library written in Java where the "specification" for the transform is itself a JSON document. Useful For Transformin

Bazaarvoice 1.3k Dec 30, 2022
Sawmill is a JSON transformation Java library

Update: June 25, 2020 The 2.0 release of Sawmill introduces a breaking change to the GeoIpProcessor to comply with the updated license of the MaxMind

Logz.io 100 Jan 1, 2023
Genson a fast & modular Java <> Json library

Genson Genson is a complete json <-> java conversion library, providing full databinding, streaming and much more. Gensons main strengths? Easy to use

null 212 Jan 3, 2023
A reference implementation of a JSON package in Java.

JSON in Java [package org.json] Click here if you just want the latest release jar file. Overview JSON is a light-weight language-independent data int

Sean Leary 4.2k Jan 6, 2023
Fast JSON parser for java projects

ig-json-parser Fast JSON parser for java projects. Getting started The easiest way to get started is to look at maven-example. For more comprehensive

Instagram 1.3k Dec 26, 2022
Generate Java types from JSON or JSON Schema and annotates those types for data-binding with Jackson, Gson, etc

jsonschema2pojo jsonschema2pojo generates Java types from JSON Schema (or example JSON) and can annotate those types for data-binding with Jackson 2.x

Joe Littlejohn 5.9k Jan 5, 2023
A JSON Schema validation implementation in pure Java, which aims for correctness and performance, in that order

Read me first The current version of this project is licensed under both LGPLv3 (or later) and ASL 2.0. The old version (2.0.x) was licensed under LGP

Java Json Tools 1.5k Jan 4, 2023
Lean JSON Library for Java, with a compact, elegant API.

mJson is an extremely lightweight Java JSON library with a very concise API. The source code is a single Java file. The license is Apache 2.0. Because

Borislav Iordanov 77 Dec 25, 2022