gRPC and protocol buffers for Android, Kotlin, and Java.

Related tags

Data Structures wire
Overview

Wire

“A man got to have a code!” - Omar Little

See the project website for documentation and APIs.

As our teams and programs grow, the variety and volume of data also grows. Success will turn your simple data models into complex ones! Whether your application is storing data to disk or transmitting it over a network, the structure and interpretation of that data should be clear. Consumers work best with data they understand!

Schemas describe and document data models. If you have data, you should have a schema.

Protocol Buffers

Google's Protocol Buffers are built around a great schema language:

  • It's cross platform and language independent. Whatever programming language you use, you'll be able to use proto schemas with your application.

  • Proto schemas are backwards-compatible and future-proof. You can evolve your schema as your application loses old features and gains new ones.

  • It's focused. Proto schemas describe your data models. That's it.

Here's a sample message definition:

syntax = "proto3";

package squareup.dinosaurs;

option java_package = "com.squareup.dinosaurs";

import "squareup/geology/period.proto";

message Dinosaur {
  // Common name of this dinosaur, like "Stegosaurus".
  string name = 1;

  // URLs with images of this dinosaur.
  repeated string picture_urls = 2;

  squareup.geology.Period period = 5;
}

And here's an enum definition:

syntax = "proto3";

package squareup.geology;

option java_package = "com.squareup.geology";

enum Period {
  // 145.5 million years ago — 66.0 million years ago.
  CRETACEOUS = 0;

  // 201.3 million years ago — 145.0 million years ago.
  JURASSIC = 1;

  // 252.17 million years ago — 201.3 million years ago.
  TRIASSIC = 2;
}

This schema language is Protocol Buffers' best feature. You might even use it purely for documentation purposes, such as to describe a JSON API.

Protocol Buffers also defines a compact binary encoding of messages that conform to the schema. This encoding is fast to encode, fast to decode, small to transmit, and small to store. The binary encoding uses numeric tags from the schema, like the 5 for period above.

For example, let's encode this dinosaur:

{
  name: "Stegosaurus",
  period: JURASSIC
}

The encoded value is just 15 bytes:

Hex  Description
 0a  tag: name(1), field encoding: LENGTH_DELIMITED(2). 1 << 3 | 2
 0b  "Stegosaurus".length()
 53  'S'
 74  't'
 65  'e'
 67  'g'
 6f  'o'
 73  's'
 61  'a'
 75  'u'
 72  'r'
 75  'u'
 73  's'
 28  tag: period(5), field encoding: VARINT(0). 5 << 3 | 0
 02  JURASSIC(2)

Why Wire?

The Protocol Buffers schema language and binary encoding are both defined by Google. Wire is an independent implementation from Square that's specifically designed for Android and Java.

For each message type defined in the schema, Wire generates an immutable model class and its builder. The generated code looks like code you'd write by hand: it's documented, formatted, and simple. Wire's APIs should feel at home to programmers who like Effective Java.

That said, there are some interesting design decisions in Wire:

  • Wire messages declare public final fields instead of the usual getter methods. This cuts down on both code generated and code executed. Less code is particularly beneficial for Android programs.

  • Wire avoids case mapping. A field declared as picture_urls in a schema yields a Java field picture_urls and not the conventional pictureUrls camel case. Though the name feels awkward at first, it's fantastic whenever you use grep or more sophisticated search tools. No more mapping when navigating between schema, Java source code, and data. It also provides a gentle reminder to calling code that proto messages are a bit special.

  • Primitive types are always boxed. If a field is absent, its value is null. This is used for naturally optional fields, such as a dinosaur whose period is unknown. A field may also be null due to schema evolution: if tomorrow we add a carnivore boolean to our message definition, today's data won’t have a value for that field.

Here's the compact generated code for the Dinosaur message defined above:

// Code generated by Wire protocol buffer compiler, do not edit.
// Source: squareup.dinosaurs.Dinosaur in squareup/dinosaurs/dinosaur.proto
package com.squareup.dinosaurs;

import com.squareup.geology.Period;
import com.squareup.wire.Message;
import com.squareup.wire.ProtoAdapter;
import com.squareup.wire.Syntax;
import com.squareup.wire.WireField;
import com.squareup.wire.internal.Internal;
import java.lang.Object;
import java.lang.Override;
import java.lang.String;
import java.util.List;
import okio.ByteString;

public final class Dinosaur extends Message<Dinosaur, Dinosaur.Builder> {
  public static final ProtoAdapter<Dinosaur> ADAPTER = ProtoAdapter.newMessageAdapter(Dinosaur.class, "type.googleapis.com/squareup.dinosaurs.Dinosaur", Syntax.PROTO_3);

  private static final long serialVersionUID = 0L;

  /**
   * Common name of this dinosaur, like "Stegosaurus".
   */
  @WireField(
      tag = 1,
      adapter = "com.squareup.wire.ProtoAdapter#STRING",
      label = WireField.Label.OMIT_IDENTITY
  )
  public final String name;

  /**
   * URLs with images of this dinosaur.
   */
  @WireField(
      tag = 2,
      adapter = "com.squareup.wire.ProtoAdapter#STRING",
      label = WireField.Label.REPEATED,
      jsonName = "pictureUrls"
  )
  public final List<String> picture_urls;

  @WireField(
      tag = 5,
      adapter = "com.squareup.geology.Period#ADAPTER",
      label = WireField.Label.OMIT_IDENTITY
  )
  public final Period period;

  public Dinosaur(String name, List<String> picture_urls, Period period) {
    this(name, picture_urls, period, ByteString.EMPTY);
  }

  public Dinosaur(String name, List<String> picture_urls, Period period, ByteString unknownFields) {
    super(ADAPTER, unknownFields);
    if (name == null) {
      throw new IllegalArgumentException("name == null");
    }
    this.name = name;
    this.picture_urls = Internal.immutableCopyOf("picture_urls", picture_urls);
    if (period == null) {
      throw new IllegalArgumentException("period == null");
    }
    this.period = period;
  }

  @Override
  public Builder newBuilder() {
    Builder builder = new Builder();
    builder.name = name;
    builder.picture_urls = Internal.copyOf(picture_urls);
    builder.period = period;
    builder.addUnknownFields(unknownFields());
    return builder;
  }

  @Override
  public boolean equals(Object other) {
    if (other == this) return true;
    if (!(other instanceof Dinosaur)) return false;
    Dinosaur o = (Dinosaur) other;
    return unknownFields().equals(o.unknownFields())
        && Internal.equals(name, o.name)
        && picture_urls.equals(o.picture_urls)
        && Internal.equals(period, o.period);
  }

  @Override
  public int hashCode() {
    int result = super.hashCode;
    if (result == 0) {
      result = unknownFields().hashCode();
      result = result * 37 + (name != null ? name.hashCode() : 0);
      result = result * 37 + picture_urls.hashCode();
      result = result * 37 + (period != null ? period.hashCode() : 0);
      super.hashCode = result;
    }
    return result;
  }

  public static final class Builder extends Message.Builder<Dinosaur, Builder> {
    public String name;

    public List<String> picture_urls;

    public Period period;

    public Builder() {
      name = "";
      picture_urls = Internal.newMutableList();
      period = Period.CRETACEOUS;
    }

    /**
     * Common name of this dinosaur, like "Stegosaurus".
     */
    public Builder name(String name) {
      this.name = name;
      return this;
    }

    /**
     * URLs with images of this dinosaur.
     */
    public Builder picture_urls(List<String> picture_urls) {
      Internal.checkElementsNotNull(picture_urls);
      this.picture_urls = picture_urls;
      return this;
    }

    public Builder period(Period period) {
      this.period = period;
      return this;
    }

    @Override
    public Dinosaur build() {
      return new Dinosaur(name, picture_urls, period, super.buildUnknownFields());
    }
  }
}

The Java code to create and access proto models is compact and readable:

Dinosaur stegosaurus = new Dinosaur.Builder()
    .name("Stegosaurus")
    .period(Period.JURASSIC)
    .build();

System.out.println("My favorite dinosaur existed in the " + stegosaurus.period + " period.");

Each type has a corresponding ProtoAdapter that can encode a message to bytes and decode bytes back into a message.

Dinosaur stegosaurus = ...
byte[] stegosaurusBytes = Dinosaur.ADAPTER.encode(stegosaurus);

byte[] tyrannosaurusBytes = ...
Dinosaur tyrannosaurus = Dinosaur.ADAPTER.decode(tyrannosaurusBytes);

When accessing a field, use Wire.get() to replace null values with the corresponding default:

Period period = Wire.get(stegosaurus.period, Dinosaur.DEFAULT_PERIOD);

This is equivalent to the following:

Period period = stegosaurus.period != null ? stegosaurus.period : Dinosaur.DEFAULT_PERIOD;

Wire Kotlin

Since version 3.0.0, Wire can generate Kotlin code. See Wire Compiler & Gradle Plugin to learn how to configure your build.

Kotlin is a pragmatic and expressive programming language that makes it easy to model data. Here's how we used Kotlin to model Protocol Buffers messages:

  • Messages feel like data classes, but in fact they're not. Compiler still generates equals(), hashCode(), toString() and copy() for you. Wire does not generate componentN() functions though, we believe that destructuring declarations are not a good fit for Protocol Buffers: a change in the schema that removes or adds a field might lead to a situation when your destructuring declaration still compiles but now describes a completely different subset of fields, rendering your code incorrect.

  • copy() is a substitute for the Builder, which is not used anymore. If your program relies on the Builder to be present, you may generate code in Java interoperability mode - Wire Compiler & Gradle Plugin explains how that works.

  • Fields are generated as properties. While this is idiomatic in Kotlin, Java code will now have to access fields using getters. If your program relies on accessing fields directly, use Java interoperability mode - the compiler will generate @JvmField annotations for each field.

  • The nullability of each field's type depends on its label: required, repeated and map fields get non-nullable types, whereas optional fields are of nullable types.

  • With the exception of required fields, each field has a default value:

    • null for optional fields,
    • emptyList() for repeated fields,
    • emptyMap() for map fields.

Here's the same Dinosaur message in Kotlin:

// Code generated by Wire protocol buffer compiler, do not edit.
// Source: squareup.dinosaurs.Dinosaur in squareup/dinosaurs/dinosaur.proto
package com.squareup.dinosaurs

import com.squareup.geology.Period
import com.squareup.wire.FieldEncoding
import com.squareup.wire.Message
import com.squareup.wire.ProtoAdapter
import com.squareup.wire.ProtoReader
import com.squareup.wire.ProtoWriter
import com.squareup.wire.Syntax.PROTO_3
import com.squareup.wire.WireField
import com.squareup.wire.internal.immutableCopyOf
import com.squareup.wire.internal.sanitize
import kotlin.Any
import kotlin.AssertionError
import kotlin.Boolean
import kotlin.Deprecated
import kotlin.DeprecationLevel
import kotlin.Int
import kotlin.Long
import kotlin.Nothing
import kotlin.String
import kotlin.collections.List
import kotlin.hashCode
import kotlin.jvm.JvmField
import okio.ByteString

class Dinosaur(
  /**
   * Common name of this dinosaur, like "Stegosaurus".
   */
  @field:WireField(
    tag = 1,
    adapter = "com.squareup.wire.ProtoAdapter#STRING",
    label = WireField.Label.OMIT_IDENTITY
  )
  val name: String = "",
  picture_urls: List<String> = emptyList(),
  @field:WireField(
    tag = 5,
    adapter = "com.squareup.geology.Period#ADAPTER",
    label = WireField.Label.OMIT_IDENTITY
  )
  val period: Period = Period.CRETACEOUS,
  unknownFields: ByteString = ByteString.EMPTY
) : Message<Dinosaur, Nothing>(ADAPTER, unknownFields) {
  /**
   * URLs with images of this dinosaur.
   */
  @field:WireField(
    tag = 2,
    adapter = "com.squareup.wire.ProtoAdapter#STRING",
    label = WireField.Label.REPEATED,
    jsonName = "pictureUrls"
  )
  val picture_urls: List<String> = immutableCopyOf("picture_urls", picture_urls)

  @Deprecated(
    message = "Shouldn't be used in Kotlin",
    level = DeprecationLevel.HIDDEN
  )
  override fun newBuilder(): Nothing = throw AssertionError()

  override fun equals(other: Any?): Boolean {
    if (other === this) return true
    if (other !is Dinosaur) return false
    if (unknownFields != other.unknownFields) return false
    if (name != other.name) return false
    if (picture_urls != other.picture_urls) return false
    if (period != other.period) return false
    return true
  }

  override fun hashCode(): Int {
    var result = super.hashCode
    if (result == 0) {
      result = unknownFields.hashCode()
      result = result * 37 + name.hashCode()
      result = result * 37 + picture_urls.hashCode()
      result = result * 37 + period.hashCode()
      super.hashCode = result
    }
    return result
  }

  override fun toString(): String {
    val result = mutableListOf<String>()
    result += """name=${sanitize(name)}"""
    if (picture_urls.isNotEmpty()) result += """picture_urls=${sanitize(picture_urls)}"""
    result += """period=$period"""
    return result.joinToString(prefix = "Dinosaur{", separator = ", ", postfix = "}")
  }

  fun copy(
    name: String = this.name,
    picture_urls: List<String> = this.picture_urls,
    period: Period = this.period,
    unknownFields: ByteString = this.unknownFields
  ): Dinosaur = Dinosaur(name, picture_urls, period, unknownFields)

  companion object {
    @JvmField
    val ADAPTER: ProtoAdapter<Dinosaur> = object : ProtoAdapter<Dinosaur>(
      FieldEncoding.LENGTH_DELIMITED,
      Dinosaur::class,
      "type.googleapis.com/squareup.dinosaurs.Dinosaur",
      PROTO_3,
      null
    ) {
      override fun encodedSize(value: Dinosaur): Int {
        var size = value.unknownFields.size
        if (value.name != "") size += ProtoAdapter.STRING.encodedSizeWithTag(1, value.name)
        size += ProtoAdapter.STRING.asRepeated().encodedSizeWithTag(2, value.picture_urls)
        if (value.period != Period.CRETACEOUS) size += Period.ADAPTER.encodedSizeWithTag(5,
            value.period)
        return size
      }

      override fun encode(writer: ProtoWriter, value: Dinosaur) {
        if (value.name != "") ProtoAdapter.STRING.encodeWithTag(writer, 1, value.name)
        ProtoAdapter.STRING.asRepeated().encodeWithTag(writer, 2, value.picture_urls)
        if (value.period != Period.CRETACEOUS) Period.ADAPTER.encodeWithTag(writer, 5, value.period)
        writer.writeBytes(value.unknownFields)
      }

      override fun decode(reader: ProtoReader): Dinosaur {
        var name: String = ""
        val picture_urls = mutableListOf<String>()
        var period: Period = Period.CRETACEOUS
        val unknownFields = reader.forEachTag { tag ->
          when (tag) {
            1 -> name = ProtoAdapter.STRING.decode(reader)
            2 -> picture_urls.add(ProtoAdapter.STRING.decode(reader))
            5 -> try {
              period = Period.ADAPTER.decode(reader)
            } catch (e: ProtoAdapter.EnumConstantNotFoundException) {
              reader.addUnknownField(tag, FieldEncoding.VARINT, e.value.toLong())
            }
            else -> reader.readUnknownField(tag)
          }
        }
        return Dinosaur(
          name = name,
          picture_urls = picture_urls,
          period = period,
          unknownFields = unknownFields
        )
      }

      override fun redact(value: Dinosaur): Dinosaur = value.copy(
        unknownFields = ByteString.EMPTY
      )
    }

    private const val serialVersionUID: Long = 0L
  }
}

Creating and accessing proto models is easy:

val stegosaurus = Dinosaur(
    name = "Stegosaurus",
    period = Period.JURASSIC
)

println("My favorite dinosaur existed in the ${stegosaurus.period} period.")

Here's how you can modify the object to add extra fields:

val stegosaurus = stegosaurus.copy(
    picture_urls = listOf("https://www.flickr.com/photos/tags/Stegosaurus/")
)

println("Here are some photos of ${stegosaurus.name}: ${stegosaurus.picture_urls}")

Wire gRPC

Since version 3.0.0, Wire supports gRPC.

Generating Code With Wire

Wire can read .proto files from the local file system and from within .jar files.

The compiler can optionally prune your schema to a subset of root types and their transitive dependencies. This is useful when sharing a schema between projects: a Java service and Android app may each use a subset of a larger shared schema.

For more info on how to get started, see Wire Compiler & Gradle Plugin.

If you don't use Gradle, the compiler also has a command line interface. Just substitute wire-compiler-VERSION-jar-with-dependencies.jar with the path to your jar. Download the latest precompiled jar.

% java -jar wire-compiler-VERSION-jar-with-dependencies.jar \
    --proto_path=src/main/proto \
    --java_out=out \
    squareup/dinosaurs/dinosaur.proto \
    squareup/geology/period.proto
Writing com.squareup.dinosaurs.Dinosaur to out
Writing com.squareup.geology.Period to out

Supplying the --android flag to the compiler causes Wire messages to implement Parcelable.

If you use Proguard, then you need to add keep rules. The simplest option is to tell Proguard not to touch the Wire runtime library and your generated protocol buffers (of course these simple rules will miss opportunities to shrink and optimize the code):

-keep class com.squareup.wire.** { *; }
-keep class com.yourcompany.yourgeneratedcode.** { *; }

Get Wire

The wire-runtime package contains runtime support libraries that must be included in applications that use Wire-generated code.

With Maven:

<dependency>
  <groupId>com.squareup.wire</groupId>
  <artifactId>wire-runtime</artifactId>
  <version>3.6.1</version>
</dependency>

With Gradle:

api "com.squareup.wire:wire-runtime:3.6.1"

Snapshots of the development version are available in Sonatype's snapshots repository.

Unsupported

Wire does not support:

  • Groups - they are skipped when parsing binary input data

Wire supports custom options on messages and fields. Other custom options are ignored. Pass --excludes=google.protobuf.* to the compiler to omit options from the generated code.

Further Documentation

See Google's excellent documentation on the structure and syntax of proto schemas.

Comments
  • proto3 Working Document

    proto3 Working Document

    As kicked off by issue 210, here’s what we need for proto3:

    • [x] Forbid optional / required labels
    • [x] First enum value with tag 0 (#575)
      • [x] Default Java enums to that value
      • [x] Forbid null for Java enums
    • [x] Any
      • [x] support in parser (TODO oldergod: update)
      • [x] support in schema
      • [x] code gen and runtime API (requires google/protobuf/any.proto)
      • [x] serialization & deserialization
    • [x] map
      • [x] support in parser
      • [x] support in schema (#578)
      • [x] code gen and runtime API (#596)
      • [x] serialization & deserialization (#596)
    • [x] Timestamp and Duration (#1382)
      • [x] code gen and runtime API (timestamp.proto, duration.proto)
      • [x] serialization & deserialization
    • [x] Packed is default for repeated scalars (#1383)
    • [x] reserved
      • [x] support in parser (#548)
      • [x] support enforcement in schema (#283)
    • [x] stream keyword for RPC response types (#282, #580)
    • [x] No extensions unless it's a custom option. (#1384)
    • [x] ~No unknown fields~ They got reverted in 3.5.
    • [x] Forbid user-defined default values (#1385)
    • [x] Scalars become non-nullable, but message fields remain nullable. (#1386)
      • Test case: empty string, default enum, 0 int in a repeated field should all be encoded.
      • Test case: map entries that have default values.
    • [x] Support in Java.
      • [x] Crash if we pass a null to a non-nullable type (like int32)
      • [x] Default Java strings to ""
      • [x] Forbid null for Java strings
    • [x] Support in Gson.
    • [x] Tests for Gson and proto3.
    • [x] Tests for Java and proto3.
    • [x] AnyMessage documentation. (@oldergod)
    • [x] Wrappers. (@swankjesse and @oldergod)
      • [x] Runtime.
      • [x] JSON.
    • [x] JSON support.
      • [x] Moshi release for custom keys (@swankjesse to cut Moshi release)
      • [x] MapAdapters. (@swankjesse)
        • [x] Unignore WireJsonTest#wrappers().
        • [x] Map values in Gson (@swankjesse).
    • [x] Create a doc to list differences we took in designing our generated code differently from Google. (@oldergod)
    • [x] Proto2 and Proto3 interoperability tests. (@oldergod)
    • [x] Remove proto3 unsupported option. (@oldergod)
    • [x] Forbid usage for proto2 enums in proto3 messages. (@oldergod)

    Goal: All done September 1?

    Proto3 
    opened by swankjesse 40
  • Add Additional statements for exporting individual Enum Values See #74

    Add Additional statements for exporting individual Enum Values See #74

    After exposing last enum method in [https://github.com/square/javawriter/pull/42] I have added additional statements which allow individual Enum values to be exposed.

    Now Enums look like this:

    public enum FooBarBazEnum {
        @ProtoEnum(1)
        FOO,
        @ProtoEnum(2)
        BAR,
        @ProtoEnum(3)
        BAZ;
        public static final Integer FOO_VALUE = 1;
        public static final Integer BAR_VALUE = 2;
        public static final Integer BAZ_VALUE = 3;
      }
    

    This change would require a SNAPSHOT version of JavaWriter and I am not quite sure which tests it's going to break and need fixing.

    Comments? @danrice-square

    opened by creativepsyco 22
  • Consider switch to Gradle

    Consider switch to Gradle

    Now that we have variants of generated code, the inability to compile the variants (and necessity to suffix their filenames) as a sanity check is unfortunate. With Gradle, various test source sets could be created in which the generation could be done and independent dependencies (such as Android) could be provided to test compilation as well as the normal file comparison checks.

    enhancement 
    opened by JakeWharton 21
  • Unable to decode proto3 message from Retrofit's response error body

    Unable to decode proto3 message from Retrofit's response error body

    Problem

    Hey, I am trying to decode and create a new protobuf model, from the bytes located in Retrofit's Response.errorBody().

    Reproducing

    The way I am doing this, is get an HttpException with status code 402 from our backend, containing a specific proto message in the error body. After accessing the errorBody() from the response, I have the following:

    val errorBodyBytes = response?.errorBody()?.bytes()
    

    The above in hex has the following value: 2a092801300138e8074801. Specifically I get a failure to decode the value in the bool e = 5; of the following proto message. It resolves the value of 40 when reading the next byte and fails to match that to a boolean in ProtoAdapter.COMMON_BOOL.

    The proto3 message

    message Subscription {
      bool a = 1;
      bool b = 2;
      bool c = 3;
      bool d = 4;
      bool e = 5;
      bool f = 6;
      int32 g = 7;
      int32 h = 8;
      bool i = 9;
    }
    

    Dependencies versions

    Wire-runtime: 3.0.0-alpha03 Wire Gradle plugin: 3.0.0-alpha03 Retrofit: 2.6.0 Okio: 2.3.0-SNAPSHOT OkHttp: 4.0.0

    opened by pavlospt 19
  • Document the proper way to use protoPath for protos defined in a module dependency

    Document the proper way to use protoPath for protos defined in a module dependency

    I have a project with two modules, app and lib. Both modules use Wire and define protos; moreover, app depends on lib and the proto in app wants to use definitions from a proto in lib.

    Looking at the documentation, protoPath does not appear to support something like project(':lib'), as this is neither "a local directory, a local .jar file, [n]or an external artifact specified with Maven coordinates."

    Currently I work around this by exploiting the fact that I know the directory structure of the project to located the other module's protos: app/build.gradle.

    What is the proper way to use protoPath to identify protos in another module of the same project?

    opened by jdkoren 13
  • Allow filtering by path for srcJar dependencies

    Allow filtering by path for srcJar dependencies

    We use proto schema published as jar, but we need just a subset of that. But include option of sourcePath closure only works for directory dependencies. It would be nice to allow it for jar dependencies too

    opened by Mishkun 13
  • WIP: Switch code generation to instance visitor and static reader.

    WIP: Switch code generation to instance visitor and static reader.

    This changes Wire to be reflection-free in every path except for the initial message adapter creation where a static field lookup has to be done.

    Other wins:

    • Debuggable operations with stack traces that match object hierarchy.
    • ProtoReader and ProtoWriter syntax for 100% dynamic reading and writing of messages.
    • Massive upgrade in ProGuard compatibility with the ADAPTER field being the only thing needing to be kept.
    • Massive reduction in temporary objects during every operation.

    Note: This doesn't completely work yet. 7 tests are failing, but they should be passing in another day. Just wanted to get this out for early review. There's some tiny design problems in the API that I need to rectify, but it works in 99.9% of cases as-is!

    The Gson module is disabled. It made it 90% of the way until I put the nail in the coffin of the @ProtoField annotation. I don't think it's viable as a reflection adapter and we should instead leverage the schema to code-gen Gson or Moshi adapters instead.

    opened by JakeWharton 13
  • Initial commit of a gradle plugin for Wire.

    Initial commit of a gradle plugin for Wire.

    This plugin adds support for generating Java code from .protos using the Wire compiler to Android library/application projects.

    For basic usage, add .proto files to src/main/proto and apply the 'com.squareup.wire' plugin. More customization is available via the 'wire' extension on any sourceSet.

    Fixes #176

    opened by jpd236 13
  • Unable to use wire-gradle-plugin:3.5.0 using Gradle Kotlin DSL?

    Unable to use wire-gradle-plugin:3.5.0 using Gradle Kotlin DSL?

    Hi All - apologies for the intrusion, I have what seems a very similar problem to the one which is described by @Globegitter in #1029 (responded to by @jrodbx) and I'm looking for a clear statement of whether I am doing the wrong thing, please?

    • I am building a small Kotlin app
    • I want to use Wire for Kotlin code generation from protobuf specs
    • I am using Gradle with the Kotlin DSL

    output of a build

    This is what happens:

    $ gradlew shadowJar
    FAILURE: Build failed with an exception.
    * Where:
    Build file '/Users/alecm/GitHome/code.repo/foo/build.gradle.kts' line: 13
    * What went wrong:
    Plugin [id: 'com.squareup.wire', version: '3.5.0', apply: false] was not found in any of the following sources:
    - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
    - Plugin Repositories (could not resolve plugin artifact 'com.squareup.wire:com.squareup.wire.gradle.plugin:3.5.0')
      Searched in the following repositories:
        Gradle Central Plugin Repository
        MavenRepo
    

    settings.gradle.kts

    After several experiments, my settings currently look like the attached, though I am not sure how much of it is really necessary.

    rootProject.name = "foo"
    pluginManagement {
        repositories {
            gradlePluginPortal()
            mavenCentral()
        }
    }
    dependencies { ...
    

    build.gradle.kts

    I have tried:

    plugins {
        // both with-and-without "application"
        kotlin("jvm") version "1.4.10"
        id("com.squareup.wire") version "3.5.0" // both with-and-without "apply false"
        id("com.github.johnrengelman.shadow") version "6.1.0"
    }
    buildscript { // <- both with-and-without this
        repositories {
            mavenCentral()
        }
    }
    

    issue

    Put simply, it looks like the plugin does not resolve; the plugin documentation for Groovy-Kotlin appear to be suspicious (because id 'com.squareup.wire' does not specify a version?) - and I can't find any other documentation on how to coerce the plugins block to look for the plugin elsewhere, or under a different, perhaps hyphenated, name?

    opened by alecmuffett 12
  • Add java.io.Serializable to Message, add trivial tests

    Add java.io.Serializable to Message, add trivial tests

    Went with the trivially-easy approach, as there really isn't much justification for figuring out a wrapper situation.

    I figured the best approach for tests was to piggyback the already-built SimpleMessage test. Let me know if that doesn't work for you guys.

    opened by asarazan 12
  • Support fully qualified type names parsing non-scalar fields in ProtoParser

    Support fully qualified type names parsing non-scalar fields in ProtoParser

    I am using ProtoParser to parse below Protobuf file to a ProtoFileElement using following code,

    syntax = "proto3";
    
    package com.foo.bar;
    
    message Address {
      string street = 1;
    
      int32 zip = 2;
    
      string city = 3;
    }
    
    message Customer {
      string name = 1;
      Address address = 3;
    }
    
    
    ProtoFileElement protoFileElement = ProtoParser.Companion.parse(Location.get(""), schema);
    MessageElement msgElement = (MessageElement) protoFileElement.getTypes().get(1);
    FieldElement addressField = msgElement.getFields().get(1);
    

    Contents of AddressField

    FieldElement(location=:15:3, label=REQUIRED, type=Address, name=address, defaultValue=null, jsonName=null, tag=3, documentation=, options=[])
    

    The type of addressField is set to Address. However, the Protobuf Java compiler recognizes it's type as .com.foo.bar.Address Reference

    Can the type names for non-scalar types be updated to have fully qualified names?

    opened by blacktooth 11
  • Type is Map<String, *>? but Any was expected

    Type is Map? but Any was expected

    Hello, I got a problem when using the wire protobuf and grpc client. I use latest version of wire. And got compile error Type mismatch: inferred type is Map<String, *>? but Any was expected The grpc service is as follows: rpc CreateOrder(RequestClass) returns (.google.protobuf.Struct); Code generated:

      public override fun CreateOrder(): GrpcCall<RequestClass,  Map<String, *>?> =
          client.newCall(GrpcMethod(
          path = "/retail_core.v1.Retail/CreateOrder",
          requestAdapter = PaypalCreateOrderRequest.ADAPTER,
          responseAdapter = ProtoAdapter.STRUCT_MAP
      ))
    

    ProtoAdapter.STRUCT_MAP return only Map<String, *>? witch is not subtype of Any class GrpcMethod<S : Any, R : Any>

    Can you add possibility to change adapter to custom or add @JvmSuppressWildCards or do STRUCT_MAP with Any return?

    opened by tdv-rf 0
  • wire-gradle-plugin not supprt kotlin > 1.6.20

    wire-gradle-plugin not supprt kotlin > 1.6.20

    Just default configuration.

    image

    If i use kotlin 1.6.20 all compiled ok. But any upper verison for example 1.7.20 not compiled with exception. image

    Couldn't find debug in {androidTest=null, androidTestDebug=null, androidTestRelease=null, debug=null, main=null, release=null, test=null, testDebug=null, testRelease=null}

    opened by tdv-rf 0
  • Decode grpc-status-details-bin and add it to GrpcException

    Decode grpc-status-details-bin and add it to GrpcException

    Only attempt to base64-decode it when there's no transport error. An invalid base64 value will be treated as IOException.

    It is up to the user to unmarshal it into a string if desired.

    This is a de-facto standard used by grpc: https://groups.google.com/g/grpc-io/c/p_gCk1bn2JE

    Note that this is not included into responseMetadata because gRPC metadata must not include reserved names grpc-* per spec: https://grpc.io/docs/what-is-grpc/core-concepts/#metadata In other words, by definition this is not "metadata".

    Fixes #2343.

    opened by Hexcles 3
  • class java.time.Instant cannot be cast to class java.util.Map

    class java.time.Instant cannot be cast to class java.util.Map

    Hello!

    Seems there is no way to serialize google.protobuf.Timestamp dynamicaly.

    Here is test case

    
    package com.mypackage;
    
    import com.squareup.wire.internal.RuntimeMessageAdapter;
    import com.squareup.wire.schema.Schema;
    import com.squareup.wire.SchemaBuilder;
    import lombok.SneakyThrows;
    import okio.Path;
    import org.junit.jupiter.api.Test;
    
    import java.time.Instant;
    import java.util.List;
    import java.util.Map;
    
    import static org.assertj.core.api.Assertions.assertThat;
    
    public class SerializationTest {
    
      @Test
      @SneakyThrows
      public void timestampTest() {
        Schema schema =
            new SchemaBuilder()
                .add(
                    Path.get("message.proto"),
                    """
                                syntax = "proto3";
    
                                import "google/protobuf/timestamp.proto";
    
                                message Message {
                                  google.protobuf.Timestamp timestamp_field = 1;
                                  string string_field = 2;
                                }
                                """)
                .build();
    
        RuntimeMessageAdapter<Object, Object> adapter =
            (RuntimeMessageAdapter<Object, Object>) schema.protoAdapter("Message", true);
        {
          Map<String, String> value = Map.of("string_field", "value");
          byte[] result = adapter.encode(value);
          assertThat(result).isNotEmpty();
          Object map = adapter.decode(result);
          assertThat(map).isEqualTo(value);
        }
        {
          Map<String, Object> value = Map.of("timestamp_field", Instant.now());
          // Map<String, Object> value = Map.of("timestamp_field", Map.of("1", List.of(123131234L), "2",
          // List.of(23432423L)));
          byte[] result = adapter.encode(value); // <--- exception here class java.time.Instant cannot be cast to class java.util.Map
          assertThat(result).isNotEmpty();
          Object map = adapter.decode(result);
          assertThat(map).isEqualTo(value);
        }
      }
    }
    

    This code produces exception:

    java.lang.ClassCastException: class java.time.Instant cannot be cast to class java.util.Map (java.time.Instant and java.util.Map are in module java.base of loader 'bootstrap')
    
    	at com.squareup.wire.schema.SchemaProtoAdapterFactory$SchemaMessageBinding.unknownFields(SchemaProtoAdapterFactory.kt:160)
    	at com.squareup.wire.internal.RuntimeMessageAdapter.encode(RuntimeMessageAdapter.kt:92)
    	at com.squareup.wire.ProtoAdapter.encodeWithTag(ProtoAdapter.kt:355)
    	at com.squareup.wire.internal.RuntimeMessageAdapter.encode(RuntimeMessageAdapter.kt:96)
    	at com.squareup.wire.ProtoAdapter.encode(ProtoAdapter.kt:363)
    	at com.squareup.wire.ProtoAdapter.encode(ProtoAdapter.kt:367)
    	at com.mypackage.SerializationTest.timestampTest(SerializationTest.java:50)
    	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
    	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
    	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
    	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
    	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
    	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
    	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
    	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
    	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
    	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
    	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
    	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
    	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
    	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
    	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
    	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
    	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
    	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
    	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
    	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
    

    Timestamp deserialization produces something like

    Map<String, Object> value = Map.of("timestamp_field", Map.of("1", List.of(1666954064), "2", List.of(499908000)));
    

    But even this map can't be serialized back. Result is empty.

    So, is there way to serialize java.time.Instant properly?

    Thanks!

    opened by gigi 0
  • Coroutines based server

    Coroutines based server

    The original gPRC docs show that they are generating some ImplBase(), for example, GreeterGrpcKt.GreeterCoroutineImplBase(), which suspend functions.

    Is it planned to do the same with Wire?

    Using rpcCallStyle = "suspending" doesn't work as it generates code that doesn't compile, and there is no suspend function in the generated code.

    opened by JavierSegoviaCordoba 0
Releases(parent-3.0.0-rc02)
This repository contains codes for various data structures and algorithms in C, C++, Java, Python, C#, Go, JavaScript and Kotlin.

Overview The goal of this project is to have codes for various data structures and algorithms - in C, C++, Java, Python, C#, Go, JavaScript and Kotlin

Manan 25 Mar 2, 2022
A lightning fast, transactional, file-based FIFO for Android and Java.

Tape by Square, Inc. Tape is a collection of queue-related classes for Android and Java. QueueFile is a lightning-fast, transactional, file-based FIFO

Square 2.4k Dec 30, 2022
A repository that contains Data Structure and Algorithms coded on Java

A repository that contains Data Structure and Algorithms coded on Java . It will also contain solutions of questions from Leetcode.

Akshat Gupta 6 Oct 15, 2022
🎓☕ Repository of lessons and exercises from loiane.training's course on data structure with Java

☕ Curso estrutura de dados com Java by @loiane.training Repositório com as aulas e exercícios do curso de estrutura de dados com Java da loiane.traini

Leticia Campos 2 Feb 1, 2022
Worker-queue implementation on top of Java and database

Database Queue Library provides worker-queue implementation on top of Java and database. Fintech company YooMoney uses db-queue in cases where reliabi

null 17 Dec 12, 2022
Data structures and algorithms exercises in java

Data structure and algorithms in Java About The Project [] In this repository you can find examples of data structure exercises solved in java and som

Luis Perez Contreras 1 Nov 25, 2021
Data structures & algorithms implemented in Java and solutions to leetcode problems.

Hello, World! ?? Hey everyone, I'm Sharad ☃ , and I'm a Software Engineer ?? at eGain! This repository ?? is all about data structures & algorithms an

Sharad Dutta 16 Dec 16, 2022
A Persistent Java Collections Library

PCollections A Persistent Java Collections Library Overview PCollections serves as a persistent and immutable analogue of the Java Collections Framewo

harold cooper 708 Dec 28, 2022
A better compressed bitset in Java

RoaringBitmap Bitsets, also called bitmaps, are commonly used as fast data structures. Unfortunately, they can use too much memory. To compensate, we

Roaring bitmaps: A better compressed bitset 2.9k Dec 29, 2022
A fast, simple persistent queue written in Java

Ladder Introduction Ladder is a lightning fast persistent queue written in Java. Usage Installation // TODO publish to Maven Central Create persistent

null 6 Sep 9, 2022
Example of implementing data structures in Java

Data Structures Example of implementing data structures in Java Implemented data structure : Queue Stack Linked List Binary Search Tree (BST) Trie Hea

Ismaïl BENHALLAM 2 Sep 27, 2021
Material de apoio da turma de introdução ao JAVA da Let's Code

Introdução ao Java Esse repositório foi criado com o intuito de servir como material de apoio ao módulo de Introdução a Java da Let's Code. Sumário Li

Bruno Pereira Pinho 5 Oct 25, 2022
Facebook Clone created using java based on Graph data Structure

Facebook Clone Facebook Clone created using java based on Graph data Structure Representation of Social Media using Graph Data Structure in Java It is

yogita pandurang chaudhari 1 Jan 16, 2022
JAVA DASAR PROGRAMING

JAVA DASAR PROGRAMING Halo , Selamat datang di repository kami ! BELAJAR JAVA DASAR HANYA DALAM WAKTU 14 HARI Ayo berpartisipasi menjadi bagian dari J

Radja Aulia Al Ramdani 2 Nov 5, 2022
C++ / JAVA Optimised Solutions for LeetCode Problems

LeetCode Solutions Algorithms Math Bit Manipulation Array String Matrix Hash Table Binary Search Sorting Two Pointers Dynamic Programming My Most Vote

Akshaya Amar 191 Jan 5, 2023
This repository contains all the Data Structures and Algorithms concepts and their implementation in several ways

An Open-Source repository that contains all the Data Structures and Algorithms concepts and their implementation in several ways, programming questions and Interview questions. The main aim of this repository is to help students who are learning Data Structures and Algorithms or preparing for an interview.

Pranay Gupta 691 Dec 31, 2022
A big, fast and persistent queue based on memory mapped file.

Big Queue A big, fast and persistent queue based on memory mapped file. Notice, bigqueue is just a standalone library, for a high-throughput, persiste

bulldog 520 Dec 30, 2022
Union, intersection, and set cardinality in loglog space

HyperMinHash-java A Java implementation of the HyperMinHash algorithm, presented by Yu and Weber. HyperMinHash allows approximating set unions, inters

LiveRamp 48 Sep 22, 2022
Popular Algorithms and Data Structures implemented in popular languages

Algos Community (college) maintained list of Algorithms and Data Structures implementations. Implemented Algorithms Algorithm C CPP Java Python Golang

IIIT Vadodara Open Source 1k Dec 28, 2022