Backport of Java 8's lambda expressions to Java 7, 6 and 5

Related tags

retrolambda
Overview

Retrolambda: Use Lambdas on Java 7

Retrolambda presentation video

Just as there was Retroweaver et al. for running Java 5 code with generics on Java 1.4, Retrolambda lets you run Java 8 code with lambda expressions, method references and try-with-resources statements on Java 7, 6 or 5. It does this by transforming your Java 8 compiled bytecode so that it can run on an older Java runtime. After the transformation they are just a bunch of normal .class files, without any additional runtime dependencies. Read more details.

There is also limited support for backporting default methods and static methods on interfaces. This feature is disabled by default.

Retrolambda supports backporting to Java 7, Java 6 and Java 5 runtimes. And for adventurous developers there are other backporting tools that may let you go from Java 5 down to Java 1.4.

Nowadays Android Studio has built-in support for Java 8 features, so that is probably the first thing to try out. Otherwise, Retrolambda works also for Android: Serge Zaitsev has written an article about it and there is a Gradle plugin which makes it easy.

Retrolambda does not backport the new Java 8 APIs, but there are other projects that have backported some of them:

Additionally Animal Sniffer and IntelliJ IDEA can warn about the use of Java 8 APIs.

User Guide

Retrolambda can be run as a Maven plugin, Gradle plugin or command line application. Also have a look at some tips for using Retrolambda effectively.

Maven Plugin

To run Retrolambda using Maven, add the following to your pom.xml:

<plugin>
    <groupId>net.orfjackal.retrolambda</groupId>
    <artifactId>retrolambda-maven-plugin</artifactId>
    <version>2.5.7</version>
    <executions>
        <execution>
            <goals>
                <goal>process-main</goal>
                <goal>process-test</goal>
            </goals>
        </execution>
    </executions>
</plugin>

See the plugin documentation for all possible parameters. There is also a usage example in end-to-end-tests/pom.xml

Gradle Plugin

Gradle Retrolamba Plugin is developed by Evan Tatarka. See its site for usage instructions.

Command Line Application

Download the latest retrolambda.jar from Maven Central.

Use JDK 8 to compile your source code.

Run Retrolambda, using Java 8, on the class files produced by JDK 8. Run java -jar retrolambda.jar without any additional options to see the instructions (for your convenience they are also shown below).

Your class files should now run on Java 7 or older.

Usage: java -Dretrolambda.inputDir=? -Dretrolambda.classpath=? [-javaagent:retrolambda.jar] -jar retrolambda.jar

Retrolambda takes Java 8 classes and backports lambda expressions and
some other language features to work on Java 7, 6 or 5.
Web site: https://github.com/luontola/retrolambda

Copyright (c) 2013-2017  Esko Luontola and other Retrolambda contributors
This software is released under the Apache License 2.0.
The license text is at http://www.apache.org/licenses/LICENSE-2.0

Configurable system properties:

  retrolambda.bytecodeVersion
      Major version number for the generated bytecode. For a list, see
      offset 7 at http://en.wikipedia.org/wiki/Java_class_file#General_layout
      Default value is 51 (i.e. Java 7)

  retrolambda.defaultMethods
      Whether to backport default methods and static methods on interfaces.
      LIMITATIONS: All backported interfaces and all classes which implement
      them or call their static methods must be backported together,
      with one execution of Retrolambda.
      Disabled by default. Enable by setting to "true"

  retrolambda.inputDir (required)
      Input directory from where the original class files are read.

  retrolambda.outputDir
      Output directory into where the generated class files are written.
      Defaults to same as retrolambda.inputDir

  retrolambda.classpath (required)
      Classpath containing the original class files and their dependencies.
      Uses ; or : as the path separator, see java.io.File#pathSeparatorChar

  retrolambda.classpathFile (alternative)
      File listing the classpath entries.
      Alternative to retrolambda.classpath for avoiding the command line
      length limit. The file must list one file per line with UTF-8 encoding.

  retrolambda.includedFiles
      List of files to process, instead of processing all files.
      This is useful for a build tool to support incremental compilation.
      Uses ; or : as the path separator, see java.io.File#pathSeparatorChar

  retrolambda.includedFilesFile (alternative)
      File listing the files to process, instead of processing all files.
      Alternative to retrolambda.includedFiles for avoiding the command line
      length limit. The file must list one file per line with UTF-8 encoding.

  retrolambda.javacHacks
      Attempts to fix javac bugs (type-annotation emission for local variables).
      Disabled by default. Enable by setting to "true"

  retrolambda.quiet
      Reduces the amount of logging.
      Disabled by default. Enable by setting to "true"

If the Java agent is used, then Retrolambda will use it to capture the
lambda classes generated by Java. Otherwise Retrolambda will hook into
Java's internal lambda dumping API, which is more susceptible to suddenly
stopping to work between Java releases.

Tips

Be sure to run comprehensive tests on your target JVM version (e.g. Java 7), in case the code accidentally uses Java 8 APIs or language features that Retrolambda doesn't backport.

During development, inside an IDE, it's the easiest to use Java 8, without Retrolamba, to compile and run tests. But in your continuous integration and release builds you should run all tests using the target Java version. For example, you can configure Maven Surefire Plugin to run tests using a different JVM.

I recommend setting up environment variables JAVA8_HOME, JAVA7_HOME etc. and referring to those variables in the build configuration, instead of relying on what happens to be the default Java version in JAVA_HOME.

You will need Java 8 for compiling and also for generating Javadocs. JDK 7's Javadoc tool will fail for some valid Java 8 code.

Backported Language Features

Lambda expressions are backported by converting them to anonymous inner classes. This includes the optimization of using a singleton instance for stateless lambda expressions to avoid repeated object allocation.

Method references are basically just syntax sugar for lambda expressions and they are backported in the same way.

Try-with-resources statements are backported by removing calls to Throwable.addSuppressed if the target bytecode version is below Java 7. If you would like the suppressed exceptions to be logged instead of swallowed, please create a feature request and we'll make it configurable.

Objects.requireNonNull calls are replaced with calls to Object.getClass if the target bytecode version is below Java 7. The synthetic null checks generated by JDK 9 use Objects.requireNonNull, whereas earlier JDK versions used Object.getClass.

Optionally also:

Default methods are backported by copying the default methods to a companion class (interface name + "$") as static methods, replacing the default methods in the interface with abstract methods, and by adding the necessary method implementations to all classes which implement that interface.

Static methods on interfaces are backported by moving the static methods to a companion class (interface name + "$"), and by changing all methods calls to call the new method location.[1]

[1] The static methods are moved to a companion class even with default method support disabled, because some of them may be lambda implementation methods, but the method calls to static methods are not updated. This may cause weird error messages if static methods on interfaces are accidentally used without enabling default method support.

Known Limitations

Does not backport Java 8 APIs.

Backporting default methods and static methods on interfaces requires all backported interfaces and all classes which implement them or call their static methods to be backported together, with one execution of Retrolambda. In other words, you must always do a clean build. Also, backporting default methods won't work across module or dependency boundaries.

May break if a future JDK 8 build stops generating a new class for each invokedynamic call. Retrolambda works so that it captures the bytecode that java.lang.invoke.LambdaMetafactory generates dynamically, so optimizations to that mechanism may break Retrolambda.

Java 9 and higher are not supported; just build your project with Java 8. The new JDKs mostly just add new APIs, which you anyways wouldn't be able to use on on older JREs. To backport new language features, create a new tool for it yourself or pay someone to do it, if you think it's worth the effort. ;)

Version History

Retrolambda 2.5.7 (2020-01-23)

  • Improved error messages for Java 12 and newer, which have been confirmed to not work without the Java agent (Issue #154)
  • Upgraded the ASM library to improve compatibility with Java 13 (Issue #154)

Retrolambda 2.5.6 (2018-11-30)

  • Fix a NullPointerException crash in the Maven plugin on Java 10 & 11
  • Fix Java agent to work on Java 9+ (Pull request #148)

Retrolambda 2.5.5 (2018-08-14)

  • Fix an ArrayIndexOutOfBoundsException crash in ASM due to incorrect bytecode produced by javac under some circumstances. See JDK-8073658 and ASM-317845. Enable the javacHacks parameter for a workaround to this issue. (Pull request #143)

Retrolambda 2.5.4 (2018-05-30)

  • Fix regression in Maven plugin; use classpathFile (Issue #141)

Retrolambda 2.5.3 (2017-12-28)

  • Copy the SourceFile attribute of the enclosing class into the lambda class (Issue #131)

Retrolambda 2.5.2 (2017-12-28)

  • Fixed running Retrolambda under Java 9 (Issue #137)
  • Consider module-info.class as a resource and do not try backporting it (Issue #122)

Retrolambda 2.5.1 (2017-02-23)

  • Fixed the enclosing method attribute of anonymous classes declared inside lambda expressions (Issue #121)

Retrolambda 2.5.0 (2017-01-22)

  • Fixed lambda expressions in subclasses accidentally overriding lambda expressions in their parent. If you are using version 2.2.0 or greater, it is strongly recommended to upgrade to this version. (Issue #109)

Retrolambda 2.4.0 (2017-01-11)

  • Added an option to reduce the amount of logging (Issue #103)
  • Removes java/lang/invoke/LambdaForm$Hidden annotations from the generated lambda classes to avoid issues with ProGuard (Pull request #118)
  • Fixed backporting classes in the default package (Issue #105)
  • Fixed backporting java.lang.Object itself (Pull request #113)

Retrolambda 2.3.0 (2016-04-30)

  • Optimize generated code to reduce method count (Issue #81)
  • Fix method reference to protected method in base class in other package failing with IllegalAccessError (Issue #89)

Retrolambda 2.2.0 (2016-04-29)

  • Backports calls to Objects.requireNonNull, improving JDK 9 support (Issue #75)
  • Optimize generated code to reduce method count (Issue #81)

Retrolambda 2.1.0 (2015-12-19)

  • Added the -Dretrolambda.classpathFile parameter to avoid the command line length limit (Issue #70)
  • Added the -Dretrolambda.includedFilesFile parameter to avoid the command line length limit (Pull request #74)
  • Made it easier to invoke Retrolambda as a library. Made Config an interface and fixed an assumption of using the default file system (Pull request #71)
  • Don't create a companion class when an interface has just a static initialization block because of constant fields (Issue #66)
  • Improved error messages: report the name of the class or lambda method which crashed Retrolambda (Issue #69)

Retrolambda 2.0.6 (2015-09-06)

  • Fixed method references to constructors causing VerifyError on Android (Issue #67)

Retrolambda 2.0.5 (2015-07-19)

  • Support for lambdas with marker interfaces (Issue #62)

Retrolambda 2.0.4 (2015-07-08)

  • Fixed a compile error when calling default methods from another module (Issue #56)
  • Fixed method references to constructors of the current class (Issue #60)
  • Removes bytecode references to java.lang.invoke.MethodHandles.Lookup on Java 6 and older (Issue #61)
  • Copies non-class files from input to output directory (Issue #54)

Retrolambda 2.0.3 (2015-06-07)

  • Fixed Retrolambda generating stack map frames for Java 5 bytecode, causing some bytecode tools to fail (Issue #55)

Retrolambda 2.0.2 (2015-04-14)

  • Fixed a hack which caused lambdas in interfaces to be backported twice, possibly producing broken method calls in the bytecode (Issue #48)
  • Fixed the handling of non-static lambda implementation methods in interfaces, i.e. lambdas which capture this (Issue #48)
  • Removes generic method signatures from the default method implementation methods which are placed in the interface's companion class, to avoid them getting out of sync with their erased method descriptors (Issue #48)

Retrolambda 2.0.1 (2015-04-06)

  • Fixed not backporting lambda expressions in default methods and static methods on interfaces (Issue #48)

Retrolambda 2.0.0 (2015-03-28)

  • Backports default methods and static methods on interfaces (Issue #31)

Retrolambda 1.8.1 (2015-01-06)

  • Backports lambda expressions in an interface's constant initializer (Issue #42)

Retrolambda 1.8.0 (2014-11-16)

  • Backports try-with-resources statements to Java 6 and older by removing calls to Throwable.addSuppressed (Issue #38)

Retrolambda 1.7.0 (2014-10-21)

  • Support for serializable lambdas (Issue #35)

Retrolambda 1.6.2 (2014-10-03)

  • Fixed a crash when trying to backport Android classes (Issue #34)

Retrolambda 1.6.1 (2014-08-25)

  • Fixed a crash when trying backport classes which are nominally the same as those included in the JRE, but which have different bytecode (Issue #29)

Retrolambda 1.6.0 (2014-08-20)

  • Does not anymore require the use of a Java agent (Issue #27)
  • Maven plugin: by default run Retrolambda in the same process as Maven, making it a bit faster. If Maven is not running under Java 8, then will fall back to forking the process and using the Java agent mechanism

Retrolambda 1.5.0 (2014-07-19)

Retrolambda 1.4.0 (2014-07-04)

  • Added an optional -Dretrolambda.includedFiles parameter to support the incremental compilers of build tools (Issue #23)
  • Decides which lambda classes to save based on the current class being processed, instead of the class loader that loaded the lambda class (Issue #21)

Retrolambda 1.3.0 (2014-06-04)

  • Maven plugin: made the input and output directories configurable (Issue #20)
  • Maven plugin: by default use the current JRE for running Retrolambda. For the old behavior, add <java8home>${env.JAVA8_HOME}</java8home> to the plugin configuration

Retrolambda 1.2.3 (2014-05-19)

  • Android: Fixed NoSuchMethodError when calling a private method to which there is a method reference (Issue #18)
  • Fixed the possibility of accidentally overriding private methods to which there is method reference (Issue #19)

Retrolambda 1.2.2 (2014-05-15)

  • Fixed method references to private methods; will now make them package-private the same way as lambda implementation methods (Issue #17)

Retrolambda 1.2.1 (2014-05-04)

  • Fixed the Retrolambda Maven plugin not using the project's classpath (Issue #16)
  • Maven plugin: save retrolambda.jar under target/retrolambda/
  • Suppress false warning about class initializer methods on interfaces

Retrolambda 1.2.0 (2014-05-02)

  • Maven plugin for running Retrolambda (thanks, Dave Moten)

Retrolambda 1.1.4 (2014-03-29)

  • Removes from interfaces bridge methods which were generated by JDK 8 e.g. when an interface overrides a method and refines its return type (Issue #13)

Retrolambda 1.1.3 (2014-03-25)

  • Fixed incompatibility with the Eclipse JDT compiler, version Kepler SR2 with the Java 8 support patch 1.0.0.v20140317-1959 (Issue #12)

Retrolambda 1.1.2 (2014-01-08)

  • Updated to work with JDK 8 Early Access Build b121 (2013-12-19) (Issue #3)

Retrolambda 1.1.1 (2013-11-27)

  • Show help if the -javaagent parameter is missing (Issue #2)

Retrolambda 1.1.0 (2013-07-25)

  • Create only one instance of lambdas which do not capture arguments; i.e. the same optimization as what JDK 8 does
  • Start the sequence number of lambda classes from one (e.g. com.example.Foo$$Lambda$1) for each enclosing class

Retrolambda 1.0.0 (2013-07-23)

  • Backports lambda expressions and method references to Java 7 and older
  • Tested to work with JDK 8 Early Access Build b99 (2013-07-19)
Comments
  • NoClassDefFoundError sometimes.

    NoClassDefFoundError sometimes.

    I have strange issue for a quite some time, sometimes when I build my app, following exception occur.

        java.lang.NoClassDefFoundError: com.wheely.app.utils.PhoneCodeUtils$$Lambda$1
                at com.wheely.app.utils.PhoneCodeUtils.preloadPhoneNumberUtil(PhoneCodeUtils.java:133)
                at com.wheely.app.WheelyApplication.onCreate(WheelyApplication.java:92)
                at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1007)
                at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4328)
                at android.app.ActivityThread.access$1500(ActivityThread.java:135)
                at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1256)
                at android.os.Handler.dispatchMessage(Handler.java:102)
                at android.os.Looper.loop(Looper.java:136)
                at android.app.ActivityThread.main(ActivityThread.java:5001)
                at java.lang.reflect.Method.invokeNative(Native Method)
                at java.lang.reflect.Method.invoke(Method.java:515)
                at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:785)
                at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:601)
                at dalvik.system.NativeStart.main(Native Method)
    

    This is first occurrence of lambda on code path, so I think that either retrolambda didn't run or it's result classes not included in final dex. Cleaning the project usually makes the deal. But it seems this happens more frequently eventually. I tend to think it is somehow related to incremental building, but not sure.

    gradle-plugin 
    opened by pepyakin 57
  • Problems with backported default and static methods

    Problems with backported default and static methods

    I'm trying to backport some default and static methods with new version 2.0. PRE: The project I'm backporting is successfully backported with old version of retrolambda if I remove all default and static method. The project is also obfuscated with proguard 5.1 successfully.

    When trying the new version I have two problems:

    1. Proguard enabled: I got a long series of warnings when obfuscating: [java] Warning: org.plibrary.test.TestLinq$$Lambda$55: can't find referenced method 'boolean lambda$negate$38(org.plibrary.compat.Predicate,java.lang.Object)' in program class org.plibrary.compat.Predicate$

    Note that "negate" is the name of the default method but no class has been generated with that name

    1. Proguard disabled. I can compile but when I run tests I got: initializationError(org.plibrary.test.TestLinq) java.lang.ClassFormatError: Method lambda$and$27 in class org/plibrary/compat/Consumer has illegal modifiers: 0x1402

    "and" is another default method and Consumer is an interface.

    As an additional info these are the options I'm using as printed by retrolambda [java] Retrolambda 2.0.0 [java] Bytecode version: 50 (Java 6) [java] Default methods: true [java] Input directory: build\classes [java] Output directory: build\classes

    I tried also with bytecode 51, nothing changes.

    Any idea of what I can do? Thanks.

    opened by pietrodev 20
  • Crash with

    Crash with "VerifyError: Expecting a stackmap frame" when loading Android classes

    I'm using the gradle plugin which can be found here and I'm getting the following crash during compilation:

    15:54:46.846 [INFO] [system.out] Error! Failed to transform some classes
    15:54:46.851 [INFO] [system.out] java.lang.RuntimeException: java.lang.IllegalAccessException: no such method: my.package.android.view.MarkerMapFragment.lambda$init$11(View)void/invokeSpecial
    15:54:46.857 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaReifier.reifyLambdaClass(LambdaReifier.java:42)
    15:54:46.862 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaUsageBackporter$InvokeDynamicInsnConvertingMethodVis
    itor.backportLambda(LambdaUsageBackporter.java:165) 
    15:54:46.866 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaUsageBackporter$InvokeDynamicInsnConvertingMethodVis
    itor.visitInvokeDynamicInsn(LambdaUsageBackporter.java:154)
    15:54:46.870 [INFO] [system.out]        at net.orfjackal.retrolambda.asm.ClassReader.readCode(ClassReader.java:1439)
    15:54:46.875 [INFO] [system.out]        at net.orfjackal.retrolambda.asm.ClassReader.readMethod(ClassReader.java:1017)
    15:54:46.880 [INFO] [system.out]        at net.orfjackal.retrolambda.asm.ClassReader.accept(ClassReader.java:693)
    15:54:46.886 [INFO] [system.out]        at net.orfjackal.retrolambda.asm.ClassReader.accept(ClassReader.java:506)
    15:54:46.891 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaUsageBackporter.transform(LambdaUsageBackporter.java
    :22)                                                
    15:54:46.896 [INFO] [system.out]        at net.orfjackal.retrolambda.Main$1.transform(Main.java:46)
    15:54:46.899 [INFO] [system.out]        at net.orfjackal.retrolambda.BytecodeTransformingFileVisitor.visitFile(BytecodeTransfor
    mingFileVisitor.java:25)                            
    15:54:46.903 [INFO] [system.out]        at net.orfjackal.retrolambda.BytecodeTransformingFileVisitor.visitFile(BytecodeTransfor
    mingFileVisitor.java:11)                            
    15:54:46.908 [INFO] [system.out]        at java.nio.file.Files.walkFileTree(Files.java:2667)
    15:54:46.914 [INFO] [system.out]        at java.nio.file.Files.walkFileTree(Files.java:2739)
    15:54:46.919 [INFO] [system.out]        at net.orfjackal.retrolambda.Main.visitFiles(Main.java:60)
    15:54:46.925 [INFO] [system.out]        at net.orfjackal.retrolambda.Main.main(Main.java:43)
    15:54:46.929 [INFO] [system.out] Caused by: java.lang.IllegalAccessException: no such method: my.package.android.view.Ma
    rkerMapFragment.lambda$init$11(View)void/invokeSpecial
    15:54:46.934 [INFO] [system.out]        at java.lang.invoke.MemberName.makeAccessException(MemberName.java:872)
    15:54:46.938 [INFO] [system.out]        at java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:993)
    15:54:46.943 [INFO] [system.out]        at java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.ja9):137
    15:54:46.949 [INFO] [system.out]        at java.lang.invoke.MethodHandles$Lookup.findSpecial(MethodHandles.java:997)
    15:54:46.955 [INFO] [system.out]        at net.orfjackal.retrolambda.Types.toMethodHandle(Types.java:42)
    15:54:46.961 [INFO] [system.out]        at net.orfjackal.retrolambda.Types.asmToJdkType(Types.java:19)
    15:54:46.966 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaReifier.callBootstrapMethod(LambdaReifier.java:106)
    15:54:46.970 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaReifier.reifyLambdaClass(LambdaReifier.java:37)
    15:54:46.974 [INFO] [system.out]        ... 14 more 
    15:54:46.978 [INFO] [system.out] Caused by: java.lang.VerifyError: Expecting a stackmap frame at branch target 11
    Exception Details:                                  
      Location:
        com/google/android/gms/maps/SupportMapFragment.getMap()Lcom/google/android/gms/maps/GoogleMap; @6: ifnonnull
      Reason:
        Expected stackmap frame at this location.
      Bytecode:
        0000000: 2ab6 0023 4c2b c700 0501 b02b b900 3601
        0000010: 004d a700 0d4e bb00 0c59 2db7 0033 bf2c
        0000020: c700 0501 b02a b400 0ec6 0018 2ab4 000e
        0000030: b600 20b9 0035 0100 2cb9 0035 0100 a500
        0000040: 0f2a bb00 0559 2cb7 001f b500 0e2a b400
        0000050: 0eb0
      Exception Handler Table:
        bci [11, 18] => handler: 21
    
    15:54:46.984 [INFO] [system.out]        at java.lang.invoke.MethodHandleNatives.resolve(Native Method)
    15:54:46.990 [INFO] [system.out]        at java.lang.invoke.MemberName$Factory.resolve(MemberName.java:965)
    15:54:46.997 [INFO] [system.out]        at java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:990)
    15:54:47.003 [INFO] [system.out]        ... 20 more 
    

    It seems that any class that extends Google's SupportMapFragment from the Google Play Services and uses any lambdas will produce this crash.

    Here's a minimal app that produces the crash: https://gist.github.com/cypressious/5ccfe84a31fd836fe125

    wontfix gradle-plugin 
    opened by cypressious 18
  • Method references to private methods

    Method references to private methods

    Hi. The problem is:

    {
      someObserable.finallyDo(this::myProblemMethod);
    }
    
    private void myProblemMethod(){}
    

    The code is compiled and dexed to run on Davlik VM. But in runtime on an android device I get warnings and the method isn't called, of course.

     DexOpt: illegal method access (ca;.myProblemMethod ()V from (Fragment$$Lambda$1;)
     I/dalvikvm﹕ Could not find method myProblemMethod, referenced from method Fragment$$Lambda$1.call
    VFY: unable to resolve virtual method 43525:
    

    The code works as a lambda if myProblemMethod is package private, public, everything except private. The method can be called explicitly. But it doesn't work as a method reference. Warnings appear only once.

    So, is it a problem, that retrolambda somehow processes references, so that dex excludes them from the build?

    opened by naixx 14
  • Anonymous class inside lambda expression causes Proguard to fail with 2.5.0

    Anonymous class inside lambda expression causes Proguard to fail with 2.5.0

    I have a problem with proguard after updating to retrolambda 2.5.0 (No problem with 2.3.0 and 2.4.0) The errors are the following, is something changed in 2.5.0 that can cause this error? My code hasn't changed and, if I revert to retrolambda 2.4.0 all works as expected.

    Warning: org.plibrary.linq.LinqQueryableMap$1: can't find enclosing method 'java.util.Iterator lambda$values$186(java.lang.Iterable)' in program class org.plibrary.linq.LinqQueryableMap Warning: org.plibrary.util.MultiHeaderList$1: can't find enclosing method 'java.util.Iterator lambda$invertedIterable$325()' in program class org.plibrary.util.MultiHeaderList

    P.S.: I used Proguard 5.2.1, so I updated it to the latest version (5.3.2) to be sure but it also fails

    bug 
    opened by pietrodev 13
  • Android N

    Android N

    I am thinking about adding lambas to my project and I know that android N will support this feature with jack in a while So my question is : is there any reason to implement retrolamba or wait till android N will re released ?

    question 
    opened by kotya341 10
  • Java 9 question & issue

    Java 9 question & issue

    Are there any plans to support java 9, meaning that the generated bytecode should be compatible with java 9, right?

    I'm asking this because I encountered an interesting issue:

    I have a project lambdamatchers which has some lambdas based on Stream class, which I agree they are not supposed to work on Java < 8.

    I run retrolambda on the classes and after I run unit tests using Java 8, it works fine with the code created by retrolambda (using as a target 1.5, 1.6, 1.7 and even 1.8), but when running the unit tests using Java 9 i get the IncompatibleClassChangeError:

    java.lang.IncompatibleClassChangeError: Method java.util.stream.Stream.of([Ljava/lang/Object;)Ljava/util/stream/Stream; must be InterfaceMethodref constant
        at ro.derbederos.hamcrest.StreamTest.streamOf(StreamTest.java:32)
    ...
    

    This can be easily reproduced with:

    public class StreamTest {
        @Test
        public void streamIsEmpty() {
            Stream.empty();
        }
    }
    

    Stream::empty is a static interface method, maybe the way they are invoked is the issue.

    Also, running unit tests with Java 9 without retrolambda-ing the project does not reproduce the issue. I'm not 100% sure, but this might to be an issue with the jvm.

    Note: This is not a real issue, I observed it during some tests, but I can work around it, as only the test code had issues, but I wanted to bring it into attention. Hopefully this does not cause problems for anybody. The test code in my case can be read as "the client code" and the client should not use retrolambda if the target is Java >= 8.

    question wontfix 
    opened by csoroiu 9
  • Compilation time has increased ten fold

    Compilation time has increased ten fold

    Ever since I updated to v3.2.2, compilation has started to take a really long time (10-50x the normal). Things are fine in every version upto v3.0.1. In the latest version, most modules are getting stuck in the compileReleaseJava step. I'd be happy to provide you more data, please let me know how.

    gradle-plugin 
    opened by vishnukvmd 9
  • Eliminate the need for VM agent

    Eliminate the need for VM agent

    I eliminated the need for VM agent by hooking into internal lambda classes dumping mechanism. I don't know if that is direction you'd like to move, so I am not starting a pull request, rather than recording my work in an issue.

    
    Eliminating need for agent (and thus starting separate VM) by hooking into internal lamda classes dump mechanism. Works on 1.8.0_11-b12
    
    diff --git a/retrolambda-maven-plugin/src/main/java/net/orfjackal/retrolambda/maven/ProcessClassesMojo.java b/retrolambda-maven-plugin/src/main/java/net/orfjackal/retrolambda/maven/ProcessClassesMojo.java
    index a945a78..f766043 100644
    --- a/retrolambda-maven-plugin/src/main/java/net/orfjackal/retrolambda/maven/ProcessClassesMojo.java
    +++ b/retrolambda-maven-plugin/src/main/java/net/orfjackal/retrolambda/maven/ProcessClassesMojo.java
    @@ -6,14 +6,18 @@ package net.orfjackal.retrolambda.maven;
    
     import com.google.common.base.Joiner;
     import com.google.common.collect.ImmutableMap;
    +import java.io.*;
    +import java.lang.reflect.Method;
    +import java.net.URL;
    +import java.net.URLClassLoader;
    +import java.util.*;
    +import org.apache.maven.artifact.Artifact;
     import org.apache.maven.execution.MavenSession;
     import org.apache.maven.plugin.*;
     import org.apache.maven.plugins.annotations.*;
     import org.apache.maven.project.MavenProject;
     import org.apache.maven.toolchain.*;
    
    -import java.io.*;
    -import java.util.*;
    
     import static org.twdata.maven.mojoexecutor.MojoExecutor.*;
    
    @@ -132,28 +136,29 @@ abstract class ProcessClassesMojo extends AbstractMojo {
    
         private void processClasses() throws MojoExecutionException {
             String retrolambdaJar = getRetrolambdaJarPath();
    -        executeMojo(
    -                plugin(groupId("org.apache.maven.plugins"),
    -                        artifactId("maven-antrun-plugin"),
    -                        version("1.7")),
    -                goal("run"),
    -                configuration(element(
    -                        "target",
    -                        element("property",
    -                                attributes(attribute("name", "the_classpath"),
    -                                        attribute("refid", getClasspathId()))),
    -                        element("exec",
    -                                attributes(
    -                                        attribute("executable", getJavaCommand()),
    -                                        attribute("failonerror", "true")),
    -                                element("arg", attribute("value", "-Dretrolambda.bytecodeVersion=" + targetBytecodeVersions.get(target))),
    -                                element("arg", attribute("value", "-Dretrolambda.inputDir=" + getInputDir().getAbsolutePath())),
    -                                element("arg", attribute("value", "-Dretrolambda.outputDir=" + getOutputDir().getAbsolutePath())),
    -                                element("arg", attribute("value", "-Dretrolambda.classpath=${the_classpath}")),
    -                                element("arg", attribute("value", "-javaagent:" + retrolambdaJar)),
    -                                element("arg", attribute("value", "-jar")),
    -                                element("arg", attribute("value", retrolambdaJar))))),
    -                executionEnvironment(project, session, pluginManager));
    +        File jar = new File(retrolambdaJar);
    +        
    +        try {
    +            StringBuilder sb = new StringBuilder();
    +            sb.append(getInputDir());
    +            for (Artifact a : project.getArtifacts()) {
    +                if (a.getFile() != null) {
    +                    sb.append(File.pathSeparator);
    +                    sb.append(a.getFile());
    +                }
    +            }
    +        
    +            URLClassLoader url = new URLClassLoader(new URL[] { jar.toURI().toURL() });
    +            Class<?> mainClass = Class.forName("net.orfjackal.retrolambda.Main", true, url);
    +            System.setProperty("retrolambda.bytecodeVersion", "" + targetBytecodeVersions.get(target));
    +            System.setProperty("retrolambda.inputDir", getInputDir().getAbsolutePath());
    +            System.setProperty("retrolambda.outputDir", getOutputDir().getAbsolutePath());
    +            System.setProperty("retrolambda.classpath", sb.toString());
    +            Method main = mainClass.getMethod("main", String[].class);
    +            main.invoke(null, (Object) new String[0]);
    +        } catch (Exception ex) {
    +            throw new MojoExecutionException("Cannot initialize classloader for " + retrolambdaJar, ex);
    +        }
         }
    
         private String getRetrolambdaJarPath() {
    diff --git a/retrolambda/src/main/java/net/orfjackal/retrolambda/Config.java b/retrolambda/src/main/java/net/orfjackal/retrolambda/Config.java
    index 7d07447..502716d 100644
    --- a/retrolambda/src/main/java/net/orfjackal/retrolambda/Config.java
    +++ b/retrolambda/src/main/java/net/orfjackal/retrolambda/Config.java
    @@ -43,7 +43,7 @@ public class Config {
         }
    
         public boolean isFullyConfigured() {
    -        return hasAllRequiredProperties() && PreMain.isAgentLoaded();
    +        return hasAllRequiredProperties();
         }
    
         private boolean hasAllRequiredProperties() {
    diff --git a/retrolambda/src/main/java/net/orfjackal/retrolambda/LambdaSavingClassFileTransformer.java b/retrolambda/src/main/java/net/orfjackal/retrolambda/LambdaSavingClassFileTransformer.java
    index fef4f1d..eda6e2d 100644
    --- a/retrolambda/src/main/java/net/orfjackal/retrolambda/LambdaSavingClassFileTransformer.java
    +++ b/retrolambda/src/main/java/net/orfjackal/retrolambda/LambdaSavingClassFileTransformer.java
    @@ -4,13 +4,28 @@
    
     package net.orfjackal.retrolambda;
    
    -import org.objectweb.asm.ClassReader;
    -
    -import java.lang.instrument.*;
    +import java.io.ByteArrayOutputStream;
    +import java.io.File;
    +import java.io.IOException;
    +import java.lang.reflect.Constructor;
    +import java.lang.reflect.Field;
    +import java.lang.reflect.Modifier;
    +import java.net.URI;
    +import java.nio.ByteBuffer;
    +import java.nio.channels.Channels;
    +import java.nio.channels.SeekableByteChannel;
    +import java.nio.channels.WritableByteChannel;
     import java.nio.file.*;
    -import java.security.ProtectionDomain;
    +import java.nio.file.attribute.BasicFileAttributes;
    +import java.nio.file.attribute.FileAttribute;
    +import java.nio.file.attribute.FileAttributeView;
    +import java.nio.file.attribute.UserPrincipalLookupService;
    +import java.nio.file.spi.FileSystemProvider;
    +import java.util.Iterator;
    +import java.util.Map;
    +import java.util.Set;
    
    -public class LambdaSavingClassFileTransformer implements ClassFileTransformer {
    +public class LambdaSavingClassFileTransformer {
    
         private final Path outputDir;
         private final int targetVersion;
    @@ -20,17 +35,35 @@ public class LambdaSavingClassFileTransformer implements ClassFileTransformer {
             this.targetVersion = targetVersion;
         }
    
    -    @Override
    -    public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
    -        if (className == null) {
    -            // Since JDK 8 build b121 or so, lambda classes have a null class name,
    -            // but we can read it from the bytecode where the name still exists.
    -            className = new ClassReader(classfileBuffer).getClassName();
    +    private Field field;
    +    void registerDumper() {
    +        try {
    +            Class<?> dumper = Class.forName("java.lang.invoke.ProxyClassesDumper");
    +            Constructor<?> cnstr = dumper.getDeclaredConstructor(Path.class);
    +            cnstr.setAccessible(true);
    +            Class<?> mf = Class.forName("java.lang.invoke.InnerClassLambdaMetafactory");
    +            field = mf.getDeclaredField("dumper");
    +            Field m = field.getClass().getDeclaredField("modifiers");
    +            m.setAccessible(true);
    +            int mod = m.getInt(field);
    +            m.setInt(field, mod & ~Modifier.FINAL);
    +            field.setAccessible(true);
    +            
    +            Path p = new VirtualPath("");
    +            field.set(null, cnstr.newInstance(p));
    +        } catch (Exception ex) {
    +            throw new IllegalStateException("Cannot initialize dumper", ex);
             }
    -        if (LambdaReifier.isLambdaClassToReify(className)) {
    -            reifyLambdaClass(className, classfileBuffer);
    +    }
    +    
    +    void unregisterDumper() {
    +        if (field != null) {
    +            try {
    +                field.set(null, null);
    +            } catch (Exception ex) {
    +                throw new IllegalArgumentException(ex);
    +            }
             }
    -        return null;
         }
    
         private void reifyLambdaClass(String className, byte[] classfileBuffer) {
    @@ -47,4 +80,361 @@ public class LambdaSavingClassFileTransformer implements ClassFileTransformer {
                 t.printStackTrace(System.out);
             }
         }
    +    
    +    private final class VirtualProvider extends FileSystemProvider {
    +
    +        @Override
    +        public String getScheme() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public FileSystem newFileSystem(URI uri, Map<String, ?> env) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public FileSystem getFileSystem(URI uri) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path getPath(URI uri) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public SeekableByteChannel newByteChannel(Path path, Set<? extends OpenOption> options, FileAttribute<?>... attrs) throws IOException {
    +            return new ClassChannel(path);
    +        }
    +
    +        @Override
    +        public DirectoryStream<Path> newDirectoryStream(Path dir, DirectoryStream.Filter<? super Path> filter) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public void createDirectory(Path dir, FileAttribute<?>... attrs) throws IOException {
    +        }
    +
    +        @Override
    +        public void delete(Path path) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public void copy(Path source, Path target, CopyOption... options) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public void move(Path source, Path target, CopyOption... options) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public boolean isSameFile(Path path, Path path2) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public boolean isHidden(Path path) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public FileStore getFileStore(Path path) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public void checkAccess(Path path, AccessMode... modes) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public <V extends FileAttributeView> V getFileAttributeView(Path path, Class<V> type, LinkOption... options) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public <A extends BasicFileAttributes> A readAttributes(Path path, Class<A> type, LinkOption... options) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Map<String, Object> readAttributes(Path path, String attributes, LinkOption... options) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public void setAttribute(Path path, String attribute, Object value, LinkOption... options) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +        
    +    }
    +    
    +    private final class VirtualFS extends FileSystem {
    +
    +        @Override
    +        public FileSystemProvider provider() {
    +            return new VirtualProvider();
    +        }
    +
    +        @Override
    +        public void close() throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public boolean isOpen() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public boolean isReadOnly() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public String getSeparator() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Iterable<Path> getRootDirectories() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Iterable<FileStore> getFileStores() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Set<String> supportedFileAttributeViews() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path getPath(String first, String... more) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public PathMatcher getPathMatcher(String syntaxAndPattern) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public UserPrincipalLookupService getUserPrincipalLookupService() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public WatchService newWatchService() throws IOException {
    +            throw new IllegalStateException();
    +        }
    +        
    +    }
    +    
    +    private final class VirtualPath implements Path {
    +        private final String path;
    +
    +        public VirtualPath(String path) {
    +            this.path = path;
    +        }
    +
    +        @Override
    +        public FileSystem getFileSystem() {
    +            return new VirtualFS();
    +        }
    +
    +        @Override
    +        public boolean isAbsolute() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path getRoot() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path getFileName() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path getParent() {
    +            return this;
    +        }
    +
    +        @Override
    +        public int getNameCount() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path getName(int index) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path subpath(int beginIndex, int endIndex) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public boolean startsWith(Path other) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public boolean startsWith(String other) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public boolean endsWith(Path other) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public boolean endsWith(String other) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path normalize() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path resolve(Path other) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path resolve(String other) {
    +            assert path.isEmpty();
    +            return new VirtualPath(other);
    +        }
    +
    +        @Override
    +        public Path resolveSibling(Path other) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path resolveSibling(String other) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path relativize(Path other) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public URI toUri() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path toAbsolutePath() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Path toRealPath(LinkOption... options) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public File toFile() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public WatchKey register(WatchService watcher, WatchEvent.Kind<?>[] events, WatchEvent.Modifier... modifiers) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public WatchKey register(WatchService watcher, WatchEvent.Kind<?>... events) throws IOException {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public Iterator<Path> iterator() {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public int compareTo(Path other) {
    +            throw new IllegalStateException();
    +        }
    +
    +        @Override
    +        public String toString() {
    +            return path;
    +        }
    +    }
    +    
    +    private final class ClassChannel implements SeekableByteChannel {
    +        private final Path path;
    +        private final ByteArrayOutputStream os;
    +        private final WritableByteChannel ch;
    +
    +        public ClassChannel(Path path) {
    +            this.path = path;
    +            this.os = new ByteArrayOutputStream();
    +            this.ch = Channels.newChannel(os);
    +        }
    +        
    +        @Override
    +        public int read(ByteBuffer dst) throws IOException {
    +            throw new IOException();
    +        }
    +
    +        @Override
    +        public int write(ByteBuffer src) throws IOException {
    +            return ch.write(src);
    +        }
    +
    +        @Override
    +        public long position() throws IOException {
    +            throw new IOException();
    +        }
    +
    +        @Override
    +        public SeekableByteChannel position(long newPosition) throws IOException {
    +            throw new IOException();
    +        }
    +
    +        @Override
    +        public long size() throws IOException {
    +            throw new IOException();
    +        }
    +
    +        @Override
    +        public SeekableByteChannel truncate(long size) throws IOException {
    +            throw new IOException();
    +        }
    +
    +        @Override
    +        public boolean isOpen() {
    +            return true;
    +        }
    +
    +        @Override
    +        public void close() throws IOException {
    +            String className = path.toString();
    +            className = className.substring(0, className.length() - 6);
    +            if (LambdaReifier.isLambdaClassToReify(className)) {
    +                reifyLambdaClass(className, os.toByteArray());
    +            }
    +        }
    +    } // end of ClassCastException
     }
    diff --git a/retrolambda/src/main/java/net/orfjackal/retrolambda/Main.java b/retrolambda/src/main/java/net/orfjackal/retrolambda/Main.java
    index feba36e..98d98c0 100644
    --- a/retrolambda/src/main/java/net/orfjackal/retrolambda/Main.java
    +++ b/retrolambda/src/main/java/net/orfjackal/retrolambda/Main.java
    @@ -43,7 +43,11 @@ public class Main {
                 visitFiles(inputDir, includedFiles, new BytecodeTransformingFileVisitor(inputDir, outputDir) {
                     @Override
                     protected byte[] transform(byte[] bytecode) {
    -                    return LambdaUsageBackporter.transform(bytecode, bytecodeVersion);
    +                    final LambdaSavingClassFileTransformer trans = new LambdaSavingClassFileTransformer(outputDir, bytecodeVersion);
    +                    trans.registerDumper();
    +                    byte[] ret = LambdaUsageBackporter.transform(bytecode, bytecodeVersion);
    +                    trans.unregisterDumper();
    +                    return ret;
                     }
                 });
             } catch (Throwable t) {
    diff --git a/retrolambda/src/main/java/net/orfjackal/retrolambda/PreMain.java b/retrolambda/src/main/java/net/orfjackal/retrolambda/PreMain.java
    deleted file mode 100644
    index ddb1a6d..0000000
    --- a/retrolambda/src/main/java/net/orfjackal/retrolambda/PreMain.java
    +++ /dev/null
    @@ -1,25 +0,0 @@
    -// Copyright © 2013 Esko Luontola <www.orfjackal.net>
    -// This software is released under the Apache License 2.0.
    -// The license text is at http://www.apache.org/licenses/LICENSE-2.0
    -
    -package net.orfjackal.retrolambda;
    -
    -import java.lang.instrument.Instrumentation;
    -import java.nio.file.Path;
    -
    -public class PreMain {
    -
    -    private static boolean agentLoaded = false;
    -
    -    public static void premain(String agentArgs, Instrumentation inst) {
    -        Config config = new Config(System.getProperties());
    -        int bytecodeVersion = config.getBytecodeVersion();
    -        Path outputDir = config.getOutputDir();
    -        inst.addTransformer(new LambdaSavingClassFileTransformer(outputDir, bytecodeVersion));
    -        agentLoaded = true;
    -    }
    -
    -    public static boolean isAgentLoaded() {
    -        return agentLoaded;
    -    }
    -}
    
    opened by jtulach 9
  • Using obfuscated dependency results in

    Using obfuscated dependency results in "java finished with non-zero exit value 1"

    Short version:

    I created a minimal self-contained Gradle project that reproduces the issue: RetrolambdaTest.zip

    Please run ./gradlew clean :app:assembleRelease in the project directory. It will fail with:

    ...
    :app:transformClassesWithRetrolambdaForRelease FAILED
    
    FAILURE: Build failed with an exception.
    
    * What went wrong:
    Execution failed for task ':app:transformClassesWithRetrolambdaForRelease'.
    > Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1
    

    Slightly longer version:

    I'm developing a closed source Android library that I'm obfuscating using proguard before uploading it to Maven. The library is using Java 6 syntax, so no retrolambda.

    When I try to use this library in my app (using retrolambda, gradle-retrolambda and Java 8 syntax), I get the aforementioned error. The app itself is not proguarded, only the library.

    If I include a non-obfuscated version of my library in the app, it works fine. If I include -noverify in my JVM args as mentioned in #25, it also works fine, but I'm not exactly comfortable with this solution.

    I tried to narrow down the exact cause of the error. I've found that those few lines of code in the test project are necessary to reproduce it. In particular, please take a look at the MyLibService and TestService classes. It turned out to not matter whether the obfuscated library comes from Maven or just another gradle module, so in the test project, it's in the lib module.

    duplicate wontfix 
    opened by kustra 8
  • Optimize generated code to reduce method count.

    Optimize generated code to reduce method count.

    Android's file format, dex, is highly sensitive to the number of methods in the contained class files. There's a method table with a 16-bit index which thus restricts the number of methods to 65536. "65536 methods?" you might say, "that should be enough for anyone!" Unfortunately these add up quick for all kinds of reasons.

    As I'm sure you are aware, Retrolambda has a big userbase in the Android community due to the restriction of the toolchain only understanding Java 6/7 classfiles. Minimizing the number of methods generated per lambda and method reference would go a long way to help keep applications under this limit.

    There's currently four cases whose generated code can be altered to reduce the number of required methods required for each lambda and method reference. By my estimation this will remove between 2000 and 2500 method references from our app! For our current usage count, that's approximately 5%!

    Instance factory method

    All $$Lambda$ classes have a lambdaFactory$ method which either returns a new instance for capturing lambdas or a cached instance for non-capturing lambdas. The implementation either delegates to the private constructor or a private static field. Removing the private modifier and making these package scoped would allow the constructor or field to be reference directly from the original call site avoiding the need for this method altogether.

    Non-capturing example:

    class Test {
      public static void main(String... args) {
        run(() -> System.out.println("Hey!"));
      }
      private static void run(Runnable run) {
        run.run();
      }
    }
    

    Generates:

    final class Test$$Lambda$1 implements java.lang.Runnable {
      private static final Test$$Lambda$1 instance;
    
      // <omitted>
    
      public static java.lang.Runnable lambdaFactory$();
        Code:
           0: getstatic     #22                 // Field instance:LTest$$Lambda$1;
           3: areturn
    }
    
    class Test {
      public static void main(java.lang.String...);
        Code:
           0: invokestatic  #22                 // Method Test$$Lambda$1.lambdaFactory$:()Ljava/lang/Runnable;
           3: invokestatic  #26                 // Method run:(Ljava/lang/Runnable;)V
           6: return
    
      // <omitted>
    }
    

    Capturing example:

    class Test {
      public static void main(String... args) {
        run(() -> System.out.println("Hey!" + args));
      }
      private static void run(Runnable run) {
        run.run();
      }
    }
    
    final class Test$$Lambda$1 implements java.lang.Runnable {
      private Test$$Lambda$1(java.lang.String[]);
        Code:
           0: aload_0
           1: invokespecial #13                 // Method java/lang/Object."<init>":()V
           4: aload_0
           5: aload_1
           6: putfield      #15                 // Field arg$1:[Ljava/lang/String;
           9: return
    
      // <omitted>
    
      public static java.lang.Runnable lambdaFactory$(java.lang.String[]);
        Code:
           0: new           #2                  // class Test$$Lambda$1
           3: dup
           4: aload_0
           5: invokespecial #19                 // Method "<init>":([Ljava/lang/String;)V
           8: areturn
    }
    
    class Test {
      public static void main(java.lang.String...);
        Code:
           0: aload_0
           1: invokestatic  #22                 // Method Test$$Lambda$1.lambdaFactory$:([Ljava/lang/String;)Ljava/lang/Runnable;
           4: invokestatic  #26                 // Method run:(Ljava/lang/Runnable;)V
           7: return
    
     // <omitted>
    }
    

    Completed Parts 🎉

    ~~Non-private method references~~ (#84)

    Method references on non-private methods (both static and instance) generate a needless package-scoped accessor method. This only needs generated for private methods. All other versions can effectively "inline" the method call directly to the generated $$Lambda$ class.

    Example:

    class Test {
      public static void main(String... args) {
        run(Test::sayHi);
      }
      private static void run(Runnable run) {
        run.run();
      }
      static void sayHi() {
        System.out.println("Hey!");
      }
    }
    

    Generates:

    final class Test$$Lambda$1 implements java.lang.Runnable {
      // <omitted>
    
      public void run();
        Code:
           0: aload_0
           1: getfield      #15                 // Field arg$1:LTest;
           4: invokestatic  #25                 // Method Test.access$lambda$0:(LTest;)V
           7: return
    }
    
    class Test {
      // <omitted>
    
      void sayHi();
        Code:
           0: getstatic     #41                 // Field java/lang/System.out:Ljava/io/PrintStream;
           3: ldc           #43                 // String Hey!
           5: invokevirtual #49                 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
           8: return
    
      static void access$lambda$0(Test);
        Code:
           0: aload_0
           1: invokevirtual #52                 // Method sayHi:()V
           4: return
    }
    

    ~~Lambda body methods~~ (#86)

    The body of a lambda is copied into a private static method on the defining class with any captured references hoisted into parameters. Being a private method, though, an additional accessor method has to be created so that the $$Lambda$ class can call back into it. Promoting the lambda body method itself to package scoped eliminates the need for this extra indirection.

    Example:

    class Test {
      public static void main(String... args) {
        run(() -> System.out.println("Hey!" + args));
      }
      private static void run(Runnable run) {
        run.run();
      }
    }
    

    Generates:

    class Test {
      // <omitted>
    
      private static void lambda$main$0(java.lang.String[]);
        Code:
           0: getstatic     #37                 // Field java/lang/System.out:Ljava/io/PrintStream;
           3: new           #39                 // class java/lang/StringBuilder
           6: dup
           7: invokespecial #40                 // Method java/lang/StringBuilder."<init>":()V
          10: ldc           #42                 // String Hey!
          12: invokevirtual #46                 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
          15: aload_0
          16: invokevirtual #49                 // Method java/lang/StringBuilder.append:(Ljava/lang/Object;)Ljava/lang/StringBuilder;
          19: invokevirtual #53                 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
          22: invokevirtual #59                 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
          25: return
    
      static void access$lambda$0(java.lang.String[]);
        Code:
           0: aload_0
           1: invokestatic  #62                 // Method lambda$main$0:([Ljava/lang/String;)V
           4: return
    }
    

    ~~Unused factory method~~ (#82)

    A factory method named get$Lambda shows up on the $$Lamda$ classes for capturing lambdas. This method duplicates the implementation of the lambdaFactory$ on the same class but is completely unused.

    class Test {
      public static void main(String... args) {
        run(() -> System.out.println("Hey!" + args));
      }
      private static void run(Runnable run) {
        run.run();
      }
    }
    
    final class Test$$Lambda$1 implements java.lang.Runnable {
      // <omitted>
    
      private static java.lang.Runnable get$Lambda(java.lang.String[]);
        Code:
           0: new           #2                  // class Test$$Lambda$1
           3: dup
           4: aload_0
           5: invokespecial #19                 // Method "<init>":([Ljava/lang/String;)V
           8: areturn
    
      // <omitted>
    }
    
    enhancement 
    opened by JakeWharton 8
  • Bug on multi-module projects when java.lang.invoke.InnerClassLambdaMetafactory is JIT'd

    Bug on multi-module projects when java.lang.invoke.InnerClassLambdaMetafactory is JIT'd

    Preconditions:

    • Multi-module Maven build with retrolambda in multiple modules.
    • retrolambda-maven-plugin with fork=false
    • JDK 8, tested with various versions, my most recent test used Oracle JDK 1.8.0_281-b09.

    If java.lang.invoke.InnerClassLambdaMetafactory is JIT'd, the static final reference to dumper will be inlined in a PrivilegedAction anonymous class (at least that's my guess given the observed behaviour) and remain unchanged even when later retrolambda executions change the dumper variable with reflection. The relevant JDK code is the following:

    https://github.com/AdoptOpenJDK/openjdk-jdk8u/blob/9a751dc19fae78ce58fb0eb176522070c992fb6f/jdk/src/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java#L314-L320

    In practice, that means that once that class is JIT'd, for example, after processing 1000 lambdas, further executions of retrolambda will write .class files to the output directory of the previous module.

    Here's a repository to reproduce the problem: https://github.com/smola/retrolambda-jit-bug

    In short: run ./repro.sh and you should see .class files from module2 written to module1/target/classes.

    Workarounds:

    • Use fork=true
    • Use interpreter execution only (very slow): MAVEN_OPTS="-Xint" ./repro.sh
    opened by smola 1
  • Don't process default methods at all

    Don't process default methods at all

    Hi. I am trying to upgrade Bck2Brwsr VM usage of Retrolamda from 2.1.0 to 2.5.7. I am facing problems. Bck2Brwsr doesn't need help with default methods, only with lambdas. Looks like that version 2.5.7 isn't really ready for that.

    This change, together with 3a5bb2a change seem to do the trick. Can you please review them and accept or suggest better way to do what is necessary?

    One surprising issue is that I have to call analyzer.analyse myself as it is not called when defaultMethodsEnabled is false.

    Then I needed to somehow disable creation of the companion class. The protected method which allows me to return existing class name seems to do the trick.

    My changes just outline what is necessary. I am sure you can come up with better fixes.

    Btw. it maybe hard to test with defaultMethodsEnabled being off - if you are interested, I could prepare some testcase using Bck2Brwsr AOT compilation & execution. It might make upgrading to new retrolambda version less painful in the future.

    opened by jtulach 1
  • breaks on JDK 15

    breaks on JDK 15

    Unfortunately, I'm failing to use JDK 15 even though JDK 14 is fine now.

         [exec] Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
         [exec] 00:00 ERROR: Failed to transform java/lang/invoke/InnerClassLambdaMetafactory, cannot enable the Java agent. Please report an issue to Retrolamb
    da with full logs. Probably you're running on an unsupported Java version.
         [exec] java.lang.RuntimeException: Could not find the toByteArray call
         [exec]     at net.orfjackal.retrolambda.lambdas.InnerClassLambdaMetafactoryTransformer.transformMetafactory(InnerClassLambdaMetafactoryTransformer.java
    :73)
         [exec]     at net.orfjackal.retrolambda.lambdas.InnerClassLambdaMetafactoryTransformer.transform(InnerClassLambdaMetafactoryTransformer.java:22)
         [exec]     at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:246)
         [exec]     at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
         [exec]     at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
         [exec]     at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
         [exec]     at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:167)
         [exec]     at net.orfjackal.retrolambda.PreMain.premain(PreMain.java:22)
         [exec]     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         [exec]     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
         [exec]     at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         [exec]     at java.base/java.lang.reflect.Method.invoke(Method.java:564)
         [exec]     at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
         [exec]     at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
         [exec] Retrolambda 2.5.7
         [exec] 00:00 ERROR: Failed to run Retrolambda
         [exec] java.lang.IllegalStateException: Cannot initialize dumper; unexpected JDK implementation. Please run Retrolambda using the Java agent (enable forking in the Maven plugin).
         [exec]     at net.orfjackal.retrolambda.lambdas.LambdaClassDumper.install(LambdaClassDumper.java:38)
         [exec]     at net.orfjackal.retrolambda.Retrolambda.run(Retrolambda.java:67)
         [exec]     at net.orfjackal.retrolambda.Main.main(Main.java:28)
         [exec]     Suppressed: java.lang.RuntimeException: java.lang.IllegalAccessException: class net.orfjackal.retrolambda.lambdas.LambdaClassDumper cannot access a member of class java.lang.invoke.InnerClassLambdaMetafactory (in module java.base) with modifiers "private static final"
         [exec]             at net.orfjackal.retrolambda.lambdas.LambdaClassDumper.uninstall(LambdaClassDumper.java:48)
         [exec]             at net.orfjackal.retrolambda.lambdas.LambdaClassDumper.close(LambdaClassDumper.java:55)
         [exec]             at net.orfjackal.retrolambda.Retrolambda.run(Retrolambda.java:102)
         [exec]             ... 1 more
         [exec]     Caused by: java.lang.IllegalAccessException: class net.orfjackal.retrolambda.lambdas.LambdaClassDumper cannot access a member of class java.lang.invoke.InnerClassLambdaMetafactory (in module java.base) with modifiers "private static final"
         [exec]             at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:385)
         [exec]             at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:693)
         [exec]             at java.base/java.lang.reflect.Field.checkAccess(Field.java:1096)
         [exec]             at java.base/java.lang.reflect.Field.set(Field.java:791)
         [exec]             at net.orfjackal.retrolambda.lambdas.LambdaClassDumper.uninstall(LambdaClassDumper.java:46)
         [exec]             ... 3 more
         [exec] Caused by: java.lang.RuntimeException: Failed to make a field non-final (private static final java.lang.invoke.ProxyClassesDumper java.lang.invoke.InnerClassLambdaMetafactory.dumper). This known to fail on Java 12 and newer. Prefer using Java 8 or try using the Java agent (fork=true in the Maven plugin).
         [exec]     at net.orfjackal.retrolambda.lambdas.LambdaClassDumper.makeNonFinal(LambdaClassDumper.java:65)
         [exec]     at net.orfjackal.retrolambda.lambdas.LambdaClassDumper.install(LambdaClassDumper.java:32)
         [exec]     ... 2 more
         [exec] Caused by: java.lang.NoSuchFieldException: modifiers
         [exec]     at java.base/java.lang.Class.getDeclaredField(Class.java:2569)
         [exec]     at net.orfjackal.retrolambda.lambdas.LambdaClassDumper.makeNonFinal(LambdaClassDumper.java:60)
         [exec]     ... 3 more
    

    These links might be helpful (or random waste of your time!)..

    https://github.com/dusanboskovic/openj9/commit/b62eb85b5fd4d011a2c8becec33c01e6eeac47f6 https://github.com/oracle/graal/pull/2455

    opened by codefromthecrypt 2
  • Avoid OSGi resolution issues caused by jdk.internal.vm.annotation (with JDK 14)

    Avoid OSGi resolution issues caused by jdk.internal.vm.annotation (with JDK 14)

    This is an odd one @dkulp identified and @reta found out. When we switched to JDK 14, generated code could claim a dependency on jdk.internal.vm.annotation due to the Hidden annotation. Can we avoid that? as it isn't worth the trouble to hunt and open these deps. See https://github.com/openzipkin/brave/pull/1247

    opened by codefromthecrypt 0
  • Is retrolambda threadSafe?

    Is retrolambda threadSafe?

    When I compile my project I get the following warning: mvn clean install -T6

    [WARNING] ***************************************************************** [WARNING] * Your build is requesting parallel execution, but project * [WARNING] * contains the following plugin(s) that have goals not marked * [WARNING] * as @threadSafe to support parallel building. * [WARNING] * While this /may/ work fine, please look for plugin updates * [WARNING] * and/or request plugins be made thread-safe. * [WARNING] * If reporting an issue, report it against the plugin in * [WARNING] * question, not against maven-core * [WARNING] ***************************************************************** [WARNING] The following plugins are not marked @threadSafe in ...: [WARNING] net.orfjackal.retrolambda:retrolambda-maven-plugin:2.5.7 [WARNING] Enable debug to see more precisely which goals are not marked @threadSafe. [WARNING] *****************************************************************

    opened by andresluuk 0
  • retrolambda convertion from jdk 11 to bytecode java 1.6

    retrolambda convertion from jdk 11 to bytecode java 1.6

    I am using net.orfjackal.retrolambda.retrolambda jar (version 2.5.6) for converting a jar which is from jdk 11 to compatible with 1.6 jvm needed by one the module by using ant build.

    Ant target

    <target name="retroLambdaArchive" description="make class files compatible with 1.6 jvm needed by module">
    
    	<property name="retroFileName" value="test.jar"/>
    	<property name="retroFileName" value="MANIFEST.MF"/>
    	<property name="retroClasspath" value="" />
    
    	<delete dir="/home/tmp/retroMe/" />
    	<delete dir="/home/tmp/retroMe_in/" />
    	<delete dir="/home/tmp/retroMe_out/" />
    
    	<!-- get the retrolambda jar -->
    	<artifact:dependencies filesetId="retrolambda.fileset" useScope="runtime">
    		<dependency groupId="net.orfjackal.retrolambda"
    					artifactId="retrolambda"
    					version="2.5.6"
    					type="jar">
    		</dependency>
    
    	</artifact:dependencies>
    	<copy todir="/home/tmp/retroMe">
    		<fileset refid="retrolambda.fileset" />
    		<mapper type="flatten" />
    	</copy>
    
    	<unzip src="${retroFileName}" dest="/home/tmp/retroMe_in/" />
    	<java classname="net.orfjackal.retrolambda.Main" fork="true" failonerror="true" classpath="/home/tmp/retroMe/retrolambda-2.5.6.jar:${retroClasspath}">
    		<sysproperty key="retrolambda.inputDir" value="/home/tmp/retroMe_in/"/>
    		<sysproperty key="retrolambda.classpath" value="/home/tmp/retroMe_in/"/>
    		<sysproperty key="retrolambda.outputDir" value="/home/tmp/retroMe_out/"/>
    		<sysproperty key="retrolambda.bytecodeVersion" value="50"/>
    	</java>
    	<jar destfile="50test.jar" manifest="/home/test/META-INF/MANIFEST.MF">
    		<fileset dir="/home/tmp/retroMe_out/" />
    	</jar>
    	<delete dir="/home/tmp/retroMe/" />
    	<delete dir="/home/tmp/retroMe_in/" />
    	<delete dir="/home/tmp/retroMe_out/" />
    </target>
    

    Whenever I use Lambda expression in any of the class, build get failed with below error:

     [java] 00:02 ERROR: Failed to run Retrolambda
     [java] java.lang.RuntimeException: Failed to backport class: com/test/TestClass
     [java] 	at net.orfjackal.retrolambda.Transformers.transform(Transformers.java:129)
     [java] 	at net.orfjackal.retrolambda.Transformers.transform(Transformers.java:107)
     [java] 	at net.orfjackal.retrolambda.Transformers.backportClass(Transformers.java:47)
     [java] 	at net.orfjackal.retrolambda.Retrolambda.run(Retrolambda.java:94)
     [java] 	at net.orfjackal.retrolambda.Main.main(Main.java:28)
     [java] Caused by: java.lang.RuntimeException: Failed to backport lambda or method reference: com/test/TestClass.lambda$appendPluginList$0(Ljava/lang/StringBuilder;Ljava/lang/String;)V (6)
    
    java9+ 
    opened by rajaveld 1
Owner
Esko Luontola
Esko Luontola
Java 8 annotation processor and framework for deriving algebraic data types constructors, pattern-matching, folds, optics and typeclasses.

Derive4J: Java 8 annotation processor for deriving algebraic data types constructors, pattern matching and more! tl;dr Show me how to write, say, the

null 543 Nov 23, 2022
vʌvr (formerly called Javaslang) is a non-commercial, non-profit object-functional library that runs with Java 8+. It aims to reduce the lines of code and increase code quality.

Vavr is an object-functional language extension to Java 8, which aims to reduce the lines of code and increase code quality. It provides persistent co

vavr 5.1k Jan 3, 2023
An advanced, but easy to use, platform for writing functional applications in Java 8.

Getting Cyclops X (10) The latest version is cyclops:10.4.0 Stackoverflow tag cyclops-react Documentation (work in progress for Cyclops X) Integration

AOL 1.3k Dec 29, 2022
Stream utilities for Java 8

protonpack A small collection of Stream utilities for Java 8. Protonpack provides the following: takeWhile and takeUntil skipWhile and skipUntil zip a

Dominic Fox 464 Nov 8, 2022
Enhancing Java Stream API

StreamEx 0.7.3 Enhancing Java Stream API. This library defines four classes: StreamEx, IntStreamEx, LongStreamEx, DoubleStreamEx which are fully compa

Tagir Valeev 2k Jan 3, 2023
Functional patterns for Java

λ Functional patterns for Java Table of Contents Background Installation Examples Semigroups Monoids Functors Bifunctors Profunctors Applicatives Mona

null 825 Dec 29, 2022
java port of Underscore.js

underscore-java Requirements Java 1.8 and later or Java 11. Installation Include the following in your pom.xml for Maven: <dependencies> <dependency

Valentyn Kolesnikov 411 Dec 6, 2022
A library that simplifies error handling for Functional Programming in Java

Faux Pas: Error handling in Functional Programming Faux pas noun, /fəʊ pɑː/: blunder; misstep, false step Faux Pas is a library that simplifies error

Zalando SE 114 Dec 5, 2022
RustScript is a functional scripting language with as much relation to Rust as Javascript has to Java.

RustScript RustScript is a scripting language as much relation to Rust as JavaScript has to Java I made this for a school project; it's meant to be im

Mikail Khan 25 Dec 24, 2022
Backport of functionality based on JSR-310 to Java SE 6 and 7. This is NOT an implementation of JSR-310.

ThreeTen backport project JSR-310 provides a new date and time library for Java SE 8. This project is the backport to Java SE 6 and 7. See the main ho

ThreeTen 541 Jan 8, 2023
A complete and performing library to highlight text snippets (EditText, SpannableString and TextView) using Spannable with Regular Expressions (Regex) for Android.

Highlight A complete and performing library to highlight text snippets (EditText/Editable and TextView) using Spannable with Regular Expressions (Rege

Irineu A. Silva 16 Dec 22, 2022
MathParser - a simple but powerful open-source math tool that parses and evaluates algebraic expressions written in pure java

MathParser is a simple but powerful open-source math tool that parses and evaluates algebraic expressions written in pure java. This projec

AmirHosseinAghajari 40 Dec 24, 2022
100% Java, Lambda Enabled, Lightweight Rules Engine with a Simple and Intuitive DSL

RuleBook » A Simple & Intuitive Rules Abstraction for Java 100% Java · Lambda Enabled · Simple, Intuitive DSL · Lightweight Why RuleBook? RuleBook rul

Delivered Technologies Labs 666 Dec 21, 2022
Java regular expressions made easy.

JavaVerbalExpressions VerbalExpressions is a Java library that helps to construct difficult regular expressions. Getting Started Maven Dependency: <de

null 2.6k Dec 30, 2022
EvalEx is a handy expression evaluator for Java, that allows to evaluate expressions.

EvalEx - Java Expression Evaluator EvalEx is a handy expression evaluator for Java, that allows to parse and evaluate expression strings. Key Features

null 18 Sep 18, 2022
Human friendly alternative to Regular Expressions

Cucumber Expressions Cucumber Expressions is an alternative to Regular Expressions with a more intuitive syntax. Try Cucumber Expressions in your brow

Cucumber 86 Jan 8, 2023
Easy-to-use placeholder library to focus on replacement of regular expressions inside a text.

WPlaceholder Flexible library to replace placeholders inside messages/texts with Regex expressions. For what it's good for Placeholder replacement is

Luiz Otávio 7 Oct 11, 2022
An example project showing how to enable tiered compilation on a Java AWS Lambda function.

AWS Lambda Tiered Compilation Sample Getting started Download or clone the repository. To install prerequisite software: Install AWS CDK Install Apach

AWS Samples 18 Dec 13, 2022
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Oryx 2 is a realization of the lambda architecture built on Apache Spark and Apache Kafka, but with specialization for real-time large scale machine l

Oryx Project 1.8k Dec 28, 2022