1# GEP-1748: Gateway API Interaction with Multi-Cluster Services
2
3* Issue: [#1748](https://github.com/kubernetes-sigs/gateway-api/issues/1748)
4* Status: Experimental
5
6> **Note**: This GEP is exempt from the [Probationary Period][expprob] rules of
7> our GEP overview as it existed before those rules did, and so it has been
8> explicitly grandfathered in.
9
10[expprob]:https://gateway-api.sigs.k8s.io/geps/overview/#probationary-period
11
12## TLDR
13
14The Kubernetes Multi-Cluster Services API enables Services to span multiple
15clusters. Gateway API enables advanced traffic routing, serving as the next
16generation Ingress, Load Balancing, and Mesh API. This document describes how
17these APIs can be used together.
18
19## Goals
20
21* Define the interaction between Gateway API and Multi-Cluster Services
22* Define any situations where Gateway API may span multiple clusters without the
23 Multi-Cluster Services API
24
25## Non-Goals
26
27* Make significant changes to either API
28
29## Introduction
30
31Gateway API and the Multi-Cluster Services API represent two of the newest
32Kubernetes networking APIs. As they’ve been developed in parallel, there’s been
33some cross-project discussion about how they can interact, but that has never
34formally been written down. This GEP aims to change that.
35
36## Overview
37
38Multi-Cluster Services can be used within Gateway API wherever Services can be
39used. The difference is that Services refer only to cluster-local endpoints while
40Multi-Cluster Services can refer to endpoints throughout an entire ClusterSet.
41
42### ServiceImport as a Backend
43
44A Route can forward traffic to the endpoints attached to an imported Service.
45This behaves identically to how forwarding to Services work in Kubernetes, with
46the exception that the endpoints attached to a ServiceImport may span multiple
47clusters. For example, the following HTTPRoute would forward traffic to
48endpoints attached to the "store" ServiceImport:
49
50```yaml
51{% include 'standard/multicluster/httproute-simple.yaml' %}
52```
53
54#### Routing to Specific Clusters
55
56In some cases, it may be helpful to route certain paths to a specific cluster
57(or region). Similar to single-cluster Services, this can be accomplished by
58creating multiple Multi-Cluster Services, one for each subset of endpoints you
59would like to route to. For example, the following configuration will send
60requests with paths prefixed with “/west” to the store-west ServiceImport, and
61“/east” to the store-east ServiceImport. Requests that don’t match either of
62these paths will be routed to the “store” ServiceImport which represents a
63superset of the “store-west” and “store-east” ServiceImports.
64
65```yaml
66{% include 'standard/multicluster/httproute-location.yaml' %}
67```
68
69#### Advanced Routing With ServiceImports
70
71All Routing capabilities in Gateway API should apply equally whether the backend
72is a Service or ServiceImport. For example, when routing to a system with
73multiple read replicas, it could be beneficial to route requests based on HTTP
74Method. In the following example, requests with POST, PUT, and DELETE methods
75are routed to `api-primary` while the rest are routed to `api-replicas`:
76
77```yaml
78{% include 'standard/multicluster/httproute-method.yaml' %}
79```
80
81#### Routing to Both Services and ServiceImports
82
83There are some situations where it will be useful to split traffic between a
84Service and ServiceImport. In the following example, 90% of traffic would go to
85endpoints attached to the cluster-local "store" Service, and the remaining 10%
86would go to endpoints attached to the Multi-Cluster "store-global" Service:
87
88```yaml
89{% include 'standard/multicluster/httproute-hybrid.yaml' %}
90```
91
92#### Cross-Namespace References with ReferenceGrant
93
94It is possible to use ReferenceGrant to enable cross-namespace references to
95ServiceImports. For example, the following HTTPRoute would forward traffic to
96endpoints attached to the “bar” Multi-Cluster Service in the “bar” namespace:
97
98```yaml
99{% include 'standard/multicluster/httproute-referencegrant.yaml' %}
100```
101
102### Mesh: ServiceImport as Parent
103
104In some cases, you may want to override traffic destined for a Multi-Cluster
105Service with a mesh. As part of the broader GAMMA initiative, ServiceImport can
106be used in the same way that Service is used as a ParentRef. When a Service is
107specified as a parent, meshes will intercept traffic destined for the ClusterIP
108of the Service and apply any policies or routing decisions defined by the Route.
109Similarly, when a ServiceImport is specified as a parent, meshes will intercept
110traffic destined for the ClusterSetIP and apply any policies or routing
111decisions defined by the Route. In the following example, the mesh would
112intercept traffic destined for the store ClusterSetIP matching the `/cart` path
113and redirect it to the `cart` Multi-Cluster Service.
114
115```yaml
116{% include 'standard/multicluster/httproute-gamma.yaml' %}
117```
118
119### Services vs ServiceImports
120
121It is important that all implementations provide a consistent experience. That
122means that references to Services SHOULD always be interpreted as references to
123endpoints within the same cluster for that Service. References to ServiceImports
124MUST be interpreted as routing to Multi-Cluster endpoints across the ClusterSet
125for the given ServiceImport. In practice, that means that users should use
126“Service” when they want to reference cluster-local endpoints, and
127“ServiceImport” when they want to route to Multi-Cluster endpoints across the
128ClusterSet for the given ServiceImport. This behavior should be analogous to
129using `.cluster.local` versus `.clusterset.local` DNS for a given Service.
130
131## API Changes
132
133* ServiceImport is recognized as backend with “Extended” conformance
134* ServiceImport is included in GAMMA GEP(s) with “Extended” conformance
135* Clarification that Services refer to endpoints within the same cluster
136
137## Alternatives
138
139### Develop Custom Multi-Cluster Concepts Independently from Multi-Cluster Services
140
141We could theoretically develop an entirely new way to handle multi-cluster routing. We’re choosing not to do that because the existing APIs are sound and can work well together.
142
143## References
144
145* [Original Doc](https://docs.google.com/document/d/1akwzBKtMKkkUV8tX-O7tPcI4BPMOLZ-gmS7Iz-7AOQE/edit#)
View as plain text