...
▽

Package grpc_retry

import "github.com/grpc-ecosystem/go-grpc-middleware/retry"
Overview
Index
Examples

Overview ▾

`grpc_retry` provides client-side request retry logic for gRPC.

Client-Side Request Retry Interceptor

It allows for automatic retry, inside the generated gRPC code of requests based on the gRPC status of the reply. It supports unary (1:1), and server stream (1:n) requests.

By default the interceptors *are disabled*, preventing accidental use of retries. You can easily override the number of retries (setting them to more than 0) with a `grpc.ClientOption`, e.g.: 

myclient.Ping(ctx, goodPing, grpc_retry.WithMax(5))

Other default options are: retry on `ResourceExhausted` and `Unavailable` gRPC codes, use a 50ms linear backoff with 10% jitter.

For chained interceptors, the retry interceptor will call every interceptor that follows it whenever a retry happens.

Please see examples for more advanced use.

▾ Example (Initialization)

Simple example of using the default interceptor configuration.

Code:

grpc.Dial("myservice.example.com",
    grpc.WithStreamInterceptor(grpc_retry.StreamClientInterceptor()),
    grpc.WithUnaryInterceptor(grpc_retry.UnaryClientInterceptor()),
)

▾ Example (InitializationWithExponentialBackoff)

Example with an exponential backoff starting with 100ms. Each next interval is the previous interval multiplied by 2.

Code:

opts := []grpc_retry.CallOption{
    grpc_retry.WithBackoff(grpc_retry.BackoffExponential(100 * time.Millisecond)),
}
grpc.Dial("myservice.example.com",
    grpc.WithStreamInterceptor(grpc_retry.StreamClientInterceptor(opts...)),
    grpc.WithUnaryInterceptor(grpc_retry.UnaryClientInterceptor(opts...)),
)

▾ Example (InitializationWithOptions)

Complex example with a 100ms linear backoff interval, and retry only on NotFound and Unavailable.

Code:

opts := []grpc_retry.CallOption{
    grpc_retry.WithBackoff(grpc_retry.BackoffLinear(100 * time.Millisecond)),
    grpc_retry.WithCodes(codes.NotFound, codes.Aborted),
}
grpc.Dial("myservice.example.com",
    grpc.WithStreamInterceptor(grpc_retry.StreamClientInterceptor(opts...)),
    grpc.WithUnaryInterceptor(grpc_retry.UnaryClientInterceptor(opts...)),
)

▾ Example (SimpleCall)

Simple example of an idempotent `ServerStream` call, that will be retried automatically 3 times.

Code:

client := pb_testproto.NewTestServiceClient(cc)
stream, _ := client.PingList(newCtx(1*time.Second), &pb_testproto.PingRequest{}, grpc_retry.WithMax(3))

for {
    pong, err := stream.Recv() // retries happen here
    if err == io.EOF {
        break
    } else if err != nil {
        return
    }
    fmt.Printf("got pong: %v", pong)
}

Constants

const (
    AttemptMetadataKey = "x-retry-attempty"
)

Variables

var (
    // DefaultRetriableCodes is a set of well known types gRPC codes that should be retri-able.
    //
    // `ResourceExhausted` means that the user quota, e.g. per-RPC limits, have been reached.
    // `Unavailable` means that system is currently unavailable and the client should retry again.
    DefaultRetriableCodes = []codes.Code{codes.ResourceExhausted, codes.Unavailable}
)

func StreamClientInterceptor 

func StreamClientInterceptor(optFuncs ...CallOption) grpc.StreamClientInterceptor

StreamClientInterceptor returns a new retrying stream client interceptor for server side streaming calls.

The default configuration of the interceptor is to not retry *at all*. This behaviour can be changed through options (e.g. WithMax) on creation of the interceptor or on call (through grpc.CallOptions).

Retry logic is available *only for ServerStreams*, i.e. 1:n streams, as the internal logic needs to buffer the messages sent by the client. If retry is enabled on any other streams (ClientStreams, BidiStreams), the retry interceptor will fail the call.

func UnaryClientInterceptor 

func UnaryClientInterceptor(optFuncs ...CallOption) grpc.UnaryClientInterceptor

UnaryClientInterceptor returns a new retrying unary client interceptor.

The default configuration of the interceptor is to not retry *at all*. This behaviour can be changed through options (e.g. WithMax) on creation of the interceptor or on call (through grpc.CallOptions).

type BackoffFunc

BackoffFunc denotes a family of functions that control the backoff duration between call retries.

They are called with an identifier of the attempt, and should return a time the system client should hold off for. If the time returned is longer than the `context.Context.Deadline` of the request the deadline of the request takes precedence and the wait will be interrupted before proceeding with the next iteration. 

type BackoffFunc func(attempt uint) time.Duration

func BackoffExponential 

func BackoffExponential(scalar time.Duration) BackoffFunc

BackoffExponential produces increasing intervals for each attempt.

The scalar is multiplied times 2 raised to the current attempt. So the first retry with a scalar of 100ms is 100ms, while the 5th attempt would be 1.6s.

func BackoffExponentialWithJitter 

func BackoffExponentialWithJitter(scalar time.Duration, jitterFraction float64) BackoffFunc

BackoffExponentialWithJitter creates an exponential backoff like BackoffExponential does, but adds jitter.

func BackoffLinear 

func BackoffLinear(waitBetween time.Duration) BackoffFunc

BackoffLinear is very simple: it waits for a fixed period of time between calls.

func BackoffLinearWithJitter 

func BackoffLinearWithJitter(waitBetween time.Duration, jitterFraction float64) BackoffFunc

BackoffLinearWithJitter waits a set period of time, allowing for jitter (fractional adjustment).

For example waitBetween=1s and jitter=0.10 can generate waits between 900ms and 1100ms.

type BackoffFuncContext

BackoffFuncContext denotes a family of functions that control the backoff duration between call retries.

They are called with an identifier of the attempt, and should return a time the system client should hold off for. If the time returned is longer than the `context.Context.Deadline` of the request the deadline of the request takes precedence and the wait will be interrupted before proceeding with the next iteration. The context can be used to extract request scoped metadata and context values. 

type BackoffFuncContext func(ctx context.Context, attempt uint) time.Duration

type CallOption

CallOption is a grpc.CallOption that is local to grpc_retry. 

type CallOption struct {
    grpc.EmptyCallOption // make sure we implement private after() and before() fields so we don't panic.
    // contains filtered or unexported fields
}

func Disable 

func Disable() CallOption

Disable disables the retry behaviour on this call, or this interceptor.

Its semantically the same to `WithMax`

func WithBackoff 

func WithBackoff(bf BackoffFunc) CallOption

WithBackoff sets the `BackoffFunc` used to control time between retries.

func WithBackoffContext 

func WithBackoffContext(bf BackoffFuncContext) CallOption

WithBackoffContext sets the `BackoffFuncContext` used to control time between retries.

func WithCodes 

func WithCodes(retryCodes ...codes.Code) CallOption

WithCodes sets which codes should be retried.

Please *use with care*, as you may be retrying non-idempotent calls.

You cannot automatically retry on Cancelled and Deadline, please use `WithPerRetryTimeout` for these.

func WithMax 

func WithMax(maxRetries uint) CallOption

WithMax sets the maximum number of retries on this call, or this interceptor.

func WithPerRetryTimeout 

func WithPerRetryTimeout(timeout time.Duration) CallOption

WithPerRetryTimeout sets the RPC timeout per call (including initial call) on this call, or this interceptor.

The context.Deadline of the call takes precedence and sets the maximum time the whole invocation will take, but WithPerRetryTimeout can be used to limit the RPC time per each call.

For example, with context.Deadline = now + 10s, and WithPerRetryTimeout(3 * time.Seconds), each of the retry calls (including the initial one) will have a deadline of now + 3s.

A value of 0 disables the timeout overrides completely and returns to each retry call using the parent `context.Deadline`.

Note that when this is enabled, any DeadlineExceeded errors that are propagated up will be retried.

▾ Example

This is an example of an `Unary` call that will also retry on deadlines. Because the passed in context has a `5s` timeout, the whole `Ping` invocation should finish within that time. However, by default all retried calls will use the parent context for their deadlines. This means, that unless you shorten the deadline of each call of the retry, you won't be able to retry the first call at all. `WithPerRetryTimeout` allows you to shorten the deadline of each retry call, allowing you to fit multiple retries in the single parent deadline.

Code:

client := pb_testproto.NewTestServiceClient(cc)
pong, _ := client.Ping(
    newCtx(5*time.Second),
    &pb_testproto.PingRequest{},
    grpc_retry.WithMax(3),
    grpc_retry.WithPerRetryTimeout(1*time.Second))

fmt.Printf("got pong: %v", pong)