Rest.li is a REST+JSON framework for building robust, scalable service architectures using dynamic discovery and simple asynchronous APIs.

In CompatibilityChecker the removing an optional field is not considered an error. As a result it is not considered an error to remove an optional field and then add a new optional field with the same field name as the field just removed, but with a different type. This is clearly inconsistent with the rule that it is a compatibility error to change the type of field.

Consider this case:

• An optional field, let's call it 'title' of type string exists in a record schema.
• Writers exist that produce data with "title", and readers exist that expect it to be of type string when present.
• The optional field is removed from the schema. This is not considered an error according to CompatibilityChecker.
• A new optional field is added to the schema, also called "title", but of type Title (a record type). This also is not considered an error according to CompatibilityChecker.

It is now possible to have writers producing either a string or a record for the 'title' field. It is also possible to have readers expecting either a string or a record for the field. They are clearly not compatible with one another.

I believe the correct fix it to make removing optional fields a compatibility error.

When I run gradle startExampleBasicClient command I get this build error:

Execution failed for task ':restli-example-api:generateMainGeneratedRestRestClient'.
> 1,1: "com.linkedin.restli.example.AlbumEntry" cannot be resolved.

More complete build log:

:restli-example-api:generateMainGeneratedRestRestClient
Add interface file: D:\Projects\Research\rest.li\restli-example-api\src\main\idl\com.linkedin.restli.example.photos.albumEntry.restspec.json
Add interface file: D:\Projects\Research\rest.li\restli-example-api\src\main\idl\com.linkedin.restli.example.photos.albums.restspec.json
Add interface file: D:\Projects\Research\rest.li\restli-example-api\src\main\idl\com.linkedin.restli.example.photos.photos.restspec.json
Generating rest client source files for: com.linkedin.restli.example.photos.albumEntry.restspec.json
Destination directory: D:\Projects\Research\rest.li\restli-example-api\src\mainGeneratedRest\java
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/D:/Projects/Java/gradle-1.7/lib/logback-classic-1.0.9.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/Oleg/.gradle/caches/artifacts-26/filestore/org.slf4j/slf4j-log4j12/1.6.2/jar/7dd950c7eeb3004a91720ba8782aa0ded9037110/slf4j-log4j12-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
:restli-example-api:generateMainGeneratedRestRestClient FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':restli-example-api:generateMainGeneratedRestRestClient'.
> 1,1: "com.linkedin.restli.example.AlbumEntry" cannot be resolved.

This gives developers the option run the validator in a mode that disallows or filters out unrecognized fields when needed. For most use cases ignoring unrecognized fields for backward compatibility reasons. But there are a couple system boundaries where these approaches have been found useful.

Note that I've included a specialized version of com.linkedin.data.it.Remover.ToRemove called ToTrim. Alternatively, could modified and then exposed some of the currently private functionality from Remover, but that would have resulted in a public API where developers could directly accumulate Remover.ToRemove objects, which is risky because if the order of those objects is not correct, the wrong elements can be removed from arrays. Also, ToTrim needs to track the paths of the elements it visits for validation message purposes.

Questions for reviewers:

• Does the naming look reasonable for the new Enum values? "TRIM" could alternatively have been named "FILTER".
• Did I handle fixup correctly here?

These dependencies are brought in when using latest release of rest.li

\--- org.antlr:antlr4:4.7.2
     +--- org.antlr:antlr4-runtime:4.7.2
     +--- org.antlr:antlr-runtime:3.5.2
     +--- org.antlr:ST4:4.1
     |    \--- org.antlr:antlr-runtime:3.5.2
     +--- org.abego.treelayout:org.abego.treelayout.core:1.0.3
     +--- org.glassfish:javax.json:1.0.4
     \--- com.ibm.icu:icu4j:61.1

OWASP scanner on a sample project shows these vulnerabilities

Upgrading to latest 4.10.1 we get these

\--- org.antlr:antlr4:4.10.1
     +--- org.antlr:antlr4-runtime:4.10.1
     +--- org.antlr:antlr-runtime:3.5.3
     +--- org.antlr:ST4:4.3.3
     |    \--- org.antlr:antlr-runtime:3.5.2 -> 3.5.3
     +--- org.abego.treelayout:org.abego.treelayout.core:1.0.3
     +--- org.glassfish:javax.json:1.0.4
     \--- com.ibm.icu:icu4j:69.1

After update 0 CVEs

Ref

• https://plugins.gradle.org/plugin/org.owasp.dependencycheck
• https://jeremylong.github.io/DependencyCheck/dependency-check-gradle/index.html

When rest.li process schema files to generate data template java code, the order of the input source file will cause the change in the annotation of the generated java class. For example: When a PDL or PDSC file A contains a data schema definition in B, and B is a typeref, when the input is in order [A, B], the generated B's class annotation comment will have a different class name compare to the two file in order [B, A] In depth, this is caused by parsing result's _schemaAndLocations entry was populated in different order when parsing is processed.

Hi Rest.li team, We trying to build 2 servers. One of them (Server A) need to send a request to the other (Server B).

This is our structure: root project: "apis"

• Contains the serves api (pdsc + the generated java class). for each server there is a separate module.

|- build.gradle
|- settings.gradle
|
|- serverA-api/build.gradle
|- serverA-api/src/main/pegasus/net/servername/api/*.pdsc
|- serverA-api/src/mainGeneratedDataTemplate/java/.../*.java
|
|- serverB-api/build.gradle
|- serverB-api/src/main/pegasus/net/servername/api/*.pdsc
|- serverB-api/src/mainGeneratedDataTemplate/java/.../*.java

root project: "server-A"

• Contains the "serverA-api" as a library and Implemented the api use cases.

root project: "server-B"

• Contains the "serverA-api" as a library - because we want to make a requests to server A.
• Contains the "serverB-api" as a library and implemented the api use cases.

While implementing the servers api use cases everything works well, but with implementing the rest client we got troubles... We are a bit confused with how to make a restClient requests on our current structure from server A to server B.

do we have to create a request builder class? - if yes, how? The docs for this part are a bit confusing, what is the "product_spec.json"? https://github.com/linkedin/rest.li/wiki/Rest.li-User-Guide#depending-on-data-templates

BTW,

is the Web UI dead? it doesn't work - also the "restli-apihub-0.0.2.zip" download link broken. https://github.com/linkedin/rest.li-api-hub

Thank you

Passing an entity with missing required files causes a RequiredFieldNotPresentException, which in turn is wrapped by a RestLiServiceException and the service returns a 500 internal server error response.

Since this is not a server but a client error, the status code should be 400 Bad Request.

This plugin can prevent user from using of old API before we release the new major version that completely get rid of legacy Avro versions (1.4 & 1.6)

After all migration is finished, we might or might not need to remove it. (It should be OK to remove it)

This is a backwards incompatible change that aims to remove the V1 request builders and associated APIs. These should all have replacements:

1. MyResourceBuilders => MyResourceRequestBuilders
2. MyResourceMethodBuilder => MyResourceMethodRequestBuilder
3. BatchCreateRequest => BatchCreateIdRequest
4. CreateRequest => CreateIdRequest
5. CreateResponse => IdResponse
6. BatchGetRequest / BatchGetKVRequest => BatchGetEntityRequest
7. Response#getId => IdResponse#getEntity

There's no urgency to merge and push this out, but keeping this up to date with the code base will require some work, so I am hoping to merge this some time this quarter and include it as part of the next major version.

This primarily cleans up DataSchemaParser to better support alternate generator target languages as well as allow for alternate schema parsers.

This is primarily for Courier (coursera.github.io/courier) but is general enough to be useful to anyone extending pegasus.

Courier supports multiple target languages (Swift, Scala, Java for Android) and also has it's own custom file format for authoring pegasus schemas, and would benefit significantly from having these extension points.

Separate from this pull-request I'd like to discuss merging the custom schema language used by Courier into Pegasus, but regardless what we decide on that topic, this seems generally useful.

Testing:

./gradlew test
...
SUCCESS

This code exists internally at LI. I've done the minimal amount of work to copy it here to open source and get it building.

Note I did need to disable Werror for this one project. We can clean it up later.

question: hello. I would like to know where there has plugins fixed for resolve the .pdl file ?

ide: intellij idea / microsoft visual studio code.

thanks.

example url: https://github.com/linkedin/rest.li/tree/master/examples/quickstart

exec cmd: gradle publishRestliIdl

error log:

Task :api:generateDataTemplate FAILED There are 1 data schema input files. Using input root folder: D:\code\tool\linkedin\rest.li\examples\quickstart\api\src\main\pegasus

FAILURE: Build failed with an exception.

• What went wrong: Execution failed for task ':api:generateDataTemplate'.

'other' has different root

• Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.6/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 15s 1 actionable task: 1 executed

env: os: windows10 gradle version: 6.6

This PR drops the support of all Gradle versions below 5.6.4. It is a foundation to improve the plugin's support to Gradle 7 and above. It is usually considered a good practice for a Gradle plugin to support only two major versions at a time.

Testing done

• local build ok

Hi Team, We use the maven repo https://mvnrepository.com/artifact/com.linkedin.pegasus/data/27.7.18.

Keys starting with numeric string are throwing the following exception while generating PDL schema.

Exception in thread "main" java.lang.IllegalArgumentException: 1,93: "00100030" is an invalid field name.

at com.linkedin.data.template.DataTemplateUtil.parseSchema(DataTemplateUtil.java:313)
at com.linkedin.data.template.DataTemplateUtil.parseSchema(DataTemplateUtil.java:291)

Details: We are trying to convert 'dicom' json data/schema to 'PDL'. 'dicom' data has lot of keys(string) that start with a numeral say "0020000D"

eg: { "0020000D": { "vr": "UI", "Value": [ "1.2.392.200036.9116.2.2.2.1762893313.1029997326.945873" ] } } Note: The complete dicom data file can be found https://dicom.nema.org/dicom/2013/output/chtml/part18/sect_F.4.html

But the "linkedin pegasus data" library does not allow keys which start with a number, eg "0020000D", while creating PDL schema.

Please let us know if you need more details