Note: the latest design is here

Goals

In the past two years, Ribbon has been used as an inter-processing communication client with load balancing capabilities. Some best practices have been observed when using Ribbon together with other open source libraries. We feel it is best to offer such best practices in a form of a set of APIs in a new Ribbon module.

Here is that we want to achieve:

• Use annotation/template to drive the process of a request creation as opposed to hand-craft the request
• Use Hystrix for fault tolerance, fallback and call collapsing and make Ribbon client properties visible to Hystrix to close the gap of the two layers
• Use cache, (e.g. EVCache) for low latency data access before reaching out to the actual resource
• Built on top of reactive and non-blocking networking layer RxNetty
• Protocol agnostic

The ultimate goal is that the clients created with this set of API should be more consistent, agile, reliable, efficient, and resilient.

Design proposal

The center pieces of the new API are AsyncRequest and RibbonRequest

public interface AsyncRequest<T> {
    public T execute();

    public Future<T> queue();

    public Observable<T> observe();

    public Observable<T> toObservable();
}

public interface RibbonRequest<T> extends AsyncRequest<T> {
    public RibbonRequest<HystrixResponse<T>> withHystrixInfo();
}

RequestTemplate is used to represent all the information, including cache providers, Hystrix fallback and any protocol specific information (e.g. URI for HTTP) required to create RibbonRequest. Information supplied to create the RequestTemplate may include variables which should be substituted with real values at the time of request creation.

/**
 * @param <I> request input entity type
 * @param <O> response entity type
 * @param <R> response meta data, e.g. HttpClientResponse
 */
public interface RequestTemplate<I, O, R> {

    RequestBuilder<O> requestBuilder();

    RequestTemplate<I, O, R> withFallbackProvider(FallbackProvider<O> fallbackProvider);

    RequestTemplate<I, O, R> withFallbackDeterminator(FallbackDeterminator<R> fallbackDeterminator);

    RequestTemplate<I, O, R> addCacheProvider(CacheProvider<O> cacheProvider, String cacheKeyTemplate);

    RequestTemplate<I, O, R> withHystrixCommandPropertiesDefaults(HystrixCommandProperties.Setter setter);

    RequestTemplate<I, O, R> withHystrixThreadPoolPropertiesDefaults(HystrixThreadPoolProperties.Setter setter);

    RequestTemplate<I, O, R> withHystrixCollapserPropertiesDefaults(HystrixCollapserProperties.Setter setter);

    public abstract class RequestBuilder<O> {
        public abstract RequestBuilder<O> withValue(String key, Object value);

        public abstract RibbonRequest<O> build();
    }
}

FallbackDeterminator is used to determine whether a response with protocol specific meta data should cause throw an error during Hystrix command execution and cause a fallback:

public interface FallbackDeterminator<T> {
    public boolean shouldTriggerFallback(T responseMetaData);
}

For example:

Ribbon.from(httpClient)
        .newRequestTemplate()
        .withFallbackDeterminator(new FallbackDeterminator<HttpClientResponse<ByteBuf>>() {
            @Override
            public boolean shouldTriggerFallback(
                    HttpClientResponse<ByteBuf> response) {
                return response.getStatus().code() >= 500;
            }
        })   

Once FallbackDeterminator determines that a protocol specific response should trigger Hystrix fallback, HystrixFallbackProvider is used to get the actual fallback:

public interface FallbackProvider<T> extends Func1<HystrixCommand<T>, Observable<T>> {
}

CacheProvider is the interface to provide access to cache of the resource

public interface CacheProvider<T> {
    Observable<T> get(String key);
}

Ribbon is the factory that creates

• clients whose responsibility is to create protocol specific template
• service object from the interface definition of a service contract defined by annotations

public final class Ribbon {

    private Ribbon() {
    }

    public static <I, O> RibbonHttpClient<I, O> from(HttpClient<I, O> transportClient) {
        // ...
    }

    public static <I, O, T> T create(Class<T> contract, HttpClient<I, O> transportClient) {
        // ...
    }
}

/**
 * @param <I> Request input entity type
 * @param <O> Response entity type
 * @param <T> Type of RequestTemplate
 */
public interface RibbonClient<I, O, T extends RequestTemplate<I, O, ?>> {
    public T newRequestTemplate();
}

The ribbon-transport module in Ribbon will provide clients used by Ribbon for different protocol with load balancing capabilities. You can also use any client created directly RxNetty if load balancing is not required.

The RibbonTransport in ribbon-transport module is the factory to create load balancing clients:

public final class RibbonTransport {

    private RibbonTransport() {
    }

    public static RxClient<ByteBuf, ByteBuf> newTcpClient(ILoadBalancer loadBalancer, IClientConfig config) {
        // ...
    }

    public static <I, O> RxClient<I, O> newTcpClient(ILoadBalancer loadBalancer, PipelineConfigurator<O, I> pipelineConfigurator, 
            IClientConfig config) {
        // ...
    }

    public static <I, O> RxClient<I, O> newTcpClient(PipelineConfigurator<O, I> pipelineConfigurator, 
            IClientConfig config) {
        // ...
    }

    // ...

    public static HttpClient<ByteBuf, ByteBuf> newHttpClient(IClientConfig config) {
        // ...
    }

    public static <I, O> HttpClient<I, O> newHttpClient(PipelineConfigurator<HttpClientResponse<O>, HttpClientRequest<I>> pipelineConfigurator, 
            ILoadBalancer loadBalancer, IClientConfig config) {
       // ...
    }

    public static <I, O> HttpClient<I, O> newHttpClient(PipelineConfigurator<HttpClientResponse<O>, HttpClientRequest<I>> pipelineConfigurator, 
            IClientConfig config) {
        // ...
    }
}

Compared with Netty, Apache's HttpAsyncClient seems to be easier to integrate into Ribbon's higher level async interface design. Another significant advantage is that it has built-in connection pool support.

So the plan is to merge this first and continue working on Netty's integration with Ribbon.

Here are the new features provided:

• Execute simple async request/response with response buffering
• Support streaming with customized stream decoder
• Use a wrapper to provide RxJava's Observable APIs, while providing the simple callback based APIs at the same time
• Async execution with load balancers
• Customizable serialization
• Async execution with cancelable back up requests

The new HttpRequest and HttpResponse will be used by both RestClient and AsyncHttpClient while RestClient's specific request/response classes are hidden or deprecated.

Some APIs in AbstractLoadBalancerAwareClient is factored out to LoadBalancerContext to be used by AsyncLoadBalancingClient.

Examples are provided in ribbon-examples sub project.

To enable ribbon in Guice just add RibbonModule() when creating the injector (either via Guice or Governator). Alternatively one can override the bindings for HttpResourceGroupFactory and HttpRequestTemplateFactory to override HttpResourceGroup and HttpRequestTemplate.

Replaced the AtomicLong implementation of our cyclic counting code with an AtomicInteger that rotates back to 0 if it hits its max value so we do the "right thing".

Also did a bit of clean-up and fixed one of the unit tests that was unstable.

add ability for ribbon/eureka to use ipAddr instead of hostName configed via Archaius

this relates to the discussion on the mailing list here:

https://groups.google.com/forum/#!topic/eureka_netflix/7M28bK-SCZU

I want to write a class implemented IRule to choose a service instance based on the loadBalancerKey, the 'IRule' implementation just like this:

public class MyLoadBalancerRule extends AbstractLoadBalancerRule {
    @Override
    public void initWithNiwsConfig(IClientConfig clientConfig) {
    }

    @Override
    public Server choose(Object key) {
        List<Server> servers = getLoadBalancer().getReachableServers();
        // choose and return a server from servers based on the key
    }
}

but the key is always null.

LoadBalancerCommand passes its attribute loadBalancerKey as parameter to the choose method. And LoadBalancerCommand object is constructed in AbstractLoadBalancerAwareClient#executeWithLoadBalancer without setting the attribute loadBalancerKey.

So I report this pr to add the loadBalancerKey when building a LoadBalancerCommand object.

We have encountered difficulty in getting SSL communication to work with the NettyHttpClient. I am instantiating this via the RibbonTransport.newHttpClient(lb,config) static constructor. I am populating the keystore and keystore password in the client config, as well as the withSecure() attribute. The server is jetty.

With SSL debug turned on, we are essentially getting no handshake activity to occur. There is no exception or timeout, just multiple attempts and eventual failure. I have tested the key and trust stores with success using other clients, browsers and openssl with success, so the certs seem good.

During research into how to get this to work, I came across this issue in RxNetty, which has a similar feel to it: https://github.com/ReactiveX/RxNetty/issues/213

I then switched to the RxNetty client, which provides a hook to the SSLEngineFactory. By creating my own ssl engine factory and overriding the createSSLEngine method to set use client mode to true, I was able to get the following to work:

   try {
        sslContextFactory.start();
    }
    catch (Exception e) { ;  }

    SSLEngineFactory myFactory = new DefaultFactories.SSLContextBasedFactory(sslContextFactory.getSslContext()) {
        @Override
        public SSLEngine createSSLEngine(ByteBufAllocator allocator) {
            SSLEngine myEngine = super.createSSLEngine(allocator);
            myEngine.setUseClientMode(true);
            return myEngine;
        };
    }
    HttpClient<ByteBuf, ByteBuf> rxClient = RxNetty.<ByteBuf, ByteBuf>newHttpClientBuilder("localhost", 8443)
            .withSslEngineFactory(myFactory)
            .build();

However, now we have lost the load balancing internal to the Ribbon client, so I couldn't use it for anything other than proving out the theory. Following the theory further, I was able to get the NettyHttpClient working by following a similar SSLEngineFactory creation path and adding a new PipelineConfigurator onto the newHttpClient constructor in RibbonTransport:

        PipelineConfigurator<HttpClientResponse<ByteBuf>, HttpClientRequest<ByteBuf>> configurators =
                new PipelineConfiguratorComposite<HttpClientResponse<ByteBuf>, HttpClientRequest<ByteBuf>>(PipelineConfigurators.httpClientConfigurator(),sslPipelineConfigurator);

        wrappedClient = RibbonTransport.newHttpClient(configurators, loadBalancer, clientConfig);

This also works, but I feel it isn't ideal. It seems like somewhere along the line, the underlying SSL implementations have possibly changed to require the setUseClientMode() call and we're not doing it now. Is that possible?

My belief is that changing the LoadBalancingHttpClient to do a similar overriding on the SslEngineFactory it uses in the clientBuilder in protected HttpClient<I, O> cacheLoadRxClient(Server server) might solve the issue, assuming its something that is an issue and not something I have messed up. I am planning on forking and testing this out and will respond to this issue with those results, but I thought it might be useful for others to weigh in or as help for others experiencing the same issue.

The constructor calls the LoadBalancerBuilder API which always return a load balancer with a ConfigurationBasedServerList, regardless whether NIWSServerListClassName is set in the config.

AbstractLoadBalancerAwareClient.computeFinalUriWithLoadBalancer() rebuilds the URI but corrupts query parameters. Line 672 is:

newURI = new URI(scheme, theUrl.getUserInfo(), host, port, urlPath, theUrl.getQuery(), theUrl.getFragment());

It should be:

newURI = new URI(scheme, theUrl.getUserInfo(), host, port, urlPath, theUrl.getRawQuery(), theUrl.getFragment());

Here's a test that shows the problem:

import com.netflix.client.ClientException;
import com.netflix.niws.client.http.HttpClientRequest;
import com.netflix.niws.client.http.RestClient;
import org.junit.Assert;
import org.junit.Test;

import java.net.URI;
import java.net.URLEncoder;
import java.util.concurrent.atomic.AtomicReference;

public class RibbonTest {
    @Test
    public void showBadEncodingInRibbon() throws Exception {
        final String lotsOfToughChars = "éƎ&=*%!@#$%^&*()";

        final AtomicReference<URI> computedUri = new AtomicReference<>();
        RestClient restClient = new RestClient() {
            @Override
            protected HttpClientRequest computeFinalUriWithLoadBalancer(HttpClientRequest original) throws ClientException {
                HttpClientRequest httpClientRequest = super.computeFinalUriWithLoadBalancer(original);
                computedUri.set(httpClientRequest.getUri());
                return httpClientRequest;
            }
        };

        URI uri = new URI("http://foo.com:8080/bar?test=" + URLEncoder.encode(lotsOfToughChars, "UTF-8"));
        HttpClientRequest request = HttpClientRequest.newBuilder().setUri(uri).build();
        try {
            restClient.executeWithLoadBalancer(request);
        } catch (ClientException e) {
            // ignore
        }

        Assert.assertEquals(uri, computedUri.get());
    }
}

Fix for issue #218: Taking the cue from the existing ServerListChangeListener I have added a ServerStatusChangeListener that may be registered with BaseLoadBalancer to receive notifications on server status changes (server down/up).

We have had this code in production for some time now and would like to see it included in Ribbon.

First batch of update for ribbon 1.x branch.

• Added ribbon-rxnetty module for asynchronous HTTP communication based on Netty and RxJava
• IClientConfig and IClientConfigKey supports typed property
• Updates to serialization factory and serializer interfaces (replaced Guava TypeToken with TypeDef in the interfaces)
• Updates to HttpRequest and HttpResponse interfaces with addition of HttpHeaders