1# GEP-746: Replace Cert Refs on HTTPRoute with Cross Namespace Refs from Gateway
2
3* Issue: [#746](https://github.com/kubernetes-sigs/gateway-api/issues/746)
4* Status: Standard
5
6## TLDR
7
8This GEP proposes that we should remove TLS Certificate references from
9HTTPRoute and replace them with Cross Namespace Certificate references from
10Gateways. Although that is not a complete replacement on its own, this GEP shows
11how a controller could provide the rest of the functionality with this approach.
12
13## Goals
14
15* Remove a confusing and underspecified part of the API - cert refs on
16 HTTPRoute.
17* Add the ability to reference certificates in other namespaces from Gateways
18 to replace much of the functionality that was enabled by cert refs on
19 HTTPRoute.
20* Describe how a controller could automate self service cert attachment to
21 Gateway listeners.
22
23## Non-Goals
24
25* Actually provide a core implementation of a controller that can enable self
26 service cert attachment. This may be worth considering at a later point, but
27 is out of scope for this GEP.
28
29## Introduction
30
31TLS Certificate references on HTTPRoute have always been a confusing part of the
32Gateway API. In the v1alpha2 release, we should consider removing this feature
33while we still can. This GEP proposes an alternative that is simpler to work
34with and understand, while also leaving sufficient room to enable all the same
35capabilities that certificate references on HTTPRoute enabled.
36
37### Attaching TLS Certificates with Routes is Confusing
38One of the most confusing parts of the Gateway API is how certificates can be
39attached to Routes. There are a variety of different factors that lead to
40confusion here:
41
42* It can be natural to assume that a certificate attached to a Route only
43 applies to that Route. In reality, it applies to the entire listener(s)
44 associated with that Route.
45* This means that a Route can affect any other Routes attached to the same
46 Gateway Listener. By attaching a Route to a Gateway Listener, you’re
47 implicitly trusting all other Routes attached to that Gateway Listener.
48* When multiple Routes specify a certificate for the same Listener, it’s
49 possible that they will conflict and create more confusion.
50
51### Why We Did It
52To understand how we ended up with the ability to attach TLS certificates with
53Routes, it’s helpful to look at the use cases for this capability:
54
551. Some users want Route owners to be able to attach arbitrary domains and certs
56 to a Gateway listener.
57 [#103](https://github.com/kubernetes-sigs/gateway-api/issues/103)
581. Some users want Route owners to control certs for their applications.
59
60### Alternative Solutions
61
62#### 1. Automation with tools like Cert-Manager
63When automation is acceptable, the first use case is entirely possible with
64tools like cert-manager that can watch Routes, generate certs for them, and
65attach them to a Gateway.
66
67#### 2. Cross Namespace Cert Direct References from Gateways
68With the already established ReferenceGrant concept, we have established a safe
69way to reference resources across namespaces. Although this would require some
70coordination between Gateway and App owners, it would enable App owners to
71retain full control of the certs used by their app without the extra confusion
72that certs in HTTPRoute have led to.
73
74### Enabling Self-Service Certificate Attachment for App Owners
75Although this dramatically simplifies the API, it does not completely replace
76the functionality that certs attached to HTTPRoutes enabled. Most notably, it
77would be difficult to attach arbitrary self-provided certificates to a Gateway
78listener without requiring manual changes from a Gateway admin.
79
80There are a couple potential solutions here:
81
82#### 1. Implement a selector for cert references instead of direct references
83Although the simplicity of this approach is nice, it ends up with many of the
84same problems as certificates attached to Routes have and feels inconsistent
85with how Routes attach to Gateways.
86
87#### 2. Implement a controller that attaches certificates to Gateway listeners
88Similar to cert-manager, it could be possible to implement a controller that
89watches for Secrets with a certain label, and attaches those to the specified
90Gateway. Although it's out of scope for this GEP to completely define what a
91controller like this could look like, it would likely need to include at least
92one of the following safeguards:
93
941. A way to configure which namespaces could attach certificates for each
95 domain.
962. A way to configure which namespaces could attach certificates to each
97 Gateway (or Listener).
983. A way to use ReferenceGrant to indicate where references from Secrets to
99 Gateways were trusted from and to.
100
101## API
102
103The API changes proposed here are quite small, mostly removing fields.
104
105### Changes
1061. The `LocalObjectReference` used for the `CertificateRef` field in
107 `GatewayTLSConfig` would be replaced with an `ObjectReference`.
1081. `ReferenceGrant` would be updated to note that references from Gateways to
109 Secrets were part of the Core support level.
110
111### Removals
112
113From HTTPRouteSpec:
114```go
115 // TLS defines the TLS certificate to use for Hostnames defined in this
116 // Route. This configuration only takes effect if the AllowRouteOverride
117 // field is set to true in the associated Gateway resource.
118 //
119 // Collisions can happen if multiple HTTPRoutes define a TLS certificate
120 // for the same hostname. In such a case, conflict resolution guiding
121 // principles apply, specifically, if hostnames are same and two different
122 // certificates are specified then the certificate in the
123 // oldest resource wins.
124 //
125 // Please note that HTTP Route-selection takes place after the
126 // TLS Handshake (ClientHello). Due to this, TLS certificate defined
127 // here will take precedence even if the request has the potential to
128 // match multiple routes (in case multiple HTTPRoutes share the same
129 // hostname).
130 //
131 // Support: Core
132 //
133 // +optional
134 TLS *RouteTLSConfig `json:"tls,omitempty"`
135```
136
137And the associated struct:
138```go
139// RouteTLSConfig describes a TLS configuration defined at the Route level.
140type RouteTLSConfig struct {
141 // CertificateRef is a reference to a Kubernetes object that contains a TLS
142 // certificate and private key. This certificate is used to establish a TLS
143 // handshake for requests that match the hostname of the associated HTTPRoute.
144 // The referenced object MUST reside in the same namespace as HTTPRoute.
145 //
146 // CertificateRef can reference a standard Kubernetes resource, i.e. Secret,
147 // or an implementation-specific custom resource.
148 //
149 // Support: Core (Kubernetes Secrets)
150 //
151 // Support: Implementation-specific (Other resource types)
152 //
153 CertificateRef LocalObjectReference `json:"certificateRef"`
154}
155```
156
157From GatewayTlsConfig:
158```go
159 // RouteOverride dictates if TLS settings can be configured
160 // via Routes or not.
161 //
162 // CertificateRef must be defined even if `routeOverride.certificate` is
163 // set to 'Allow' as it will be used as the default certificate for the
164 // listener.
165 //
166 // Support: Core
167 //
168 // +optional
169 // +kubebuilder:default={certificate:Deny}
170 RouteOverride *TLSOverridePolicy `json:"routeOverride,omitempty"`
171```
172
173And the associated types:
174```go
175type TLSRouteOverrideType string
176
177const (
178 // Allows the parameter to be configured from all routes.
179 TLSROuteOVerrideAllow TLSRouteOverrideType = "Allow"
180
181 // Prohibits the parameter from being configured from any route.
182 TLSRouteOverrideDeny TLSRouteOverrideType = "Deny"
183)
184
185// TLSOverridePolicy defines a schema for overriding TLS settings at the Route
186// level.
187type TLSOverridePolicy struct {
188 // Certificate dictates if TLS certificates can be configured
189 // via Routes. If set to 'Allow', a TLS certificate for a hostname
190 // defined in a Route takes precedence over the certificate defined in
191 // Gateway.
192 //
193 // Support: Core
194 //
195 // +optional
196 // +kubebuilder:default=Deny
197 Certificate *TLSRouteOverrideType `json:"certificate,omitempty"`
198}
199```
200
201## Prior Art
202
203OpenShift already supports configuring TLS certificates on Routes. Although
204largely similar to the Gateway API approach, there are some notable differences:
205
206* Each Route can specify a maximum of 1 hostname
207* When a Route is attached to a hostname, newer Routes can't use the same
208 hostname unless all of the following are true:
209 * The Routes are in the same namespace or the Router is configured to allow
210 sharing hostnames across namespaces
211 * The Routes have unique, non-overlapping paths specified
212 * The Routes are not TCP or TLS routes
213
214A typical configuration would involve a Router with `*.example.com` that has a
215wildcard cert. Routes could be attached within those constraints without the
216need for a cert. Routes can also use a different hostname if they also provide a
217cert.
218
219## Alternatives
220
221### 1. Improved Documentation + Extended Support Level
222My first attempt to improve this was to create a
223[PR](https://github.com/kubernetes-sigs/gateway-api/pull/739) that would clarify
224the documentation around how this works and lower the support level to extended.
225
226Trying to improve the documentation around this feature made it clear how easy
227it would be to get confused by how it worked. It would be only natural to assume
228that a cert attached to a Route would only apply to that Route. The conflict
229resolution semantics associated with this were both complicated and difficult to
230surface to a user through status or other means.
231
232Lowering the support level from core to extended also didn't make sense.
233Although some implementers were uncomfortable with supporting this feature due
234to the potential for vulnerabilities, that was not a sufficient reason to lower
235the support level. An extended support level should only be used for features
236that cannot be universally supported. That was not the case here. Instead there
237were just very real questions around the safety of the feature.
238
239The combination of those 2 factors led me to believe that this feature was not
240well thought out and should be removed. Since this was essentially just a
241shortcut to attaching certificates to a Gateway listener from different sources,
242it seemed like there had to be a way that was both safer and easier to
243understand. That led to this proposal.
244
245### 2. Implement Hostname Restrictions
246Similar to the OpenShift approach described above, we could enforce the
247following:
248
2491. Only a single hostname may be specified for HTTPRoutes with a certificate
250 reference.
2511. The oldest HTTPRoute to attach a certificate to a hostname would effectively
252 own that hostname. No other HTTPRoutes could be attached with the same
253 hostname unless they were explicitly allowed by that HTTPRoute.
254
255The second condition would be difficult to validate. As we've seen elsewhere in
256the API, it's difficult to determine which resource was first to claim a
257hostname or path. Instead we have to rely on the oldest resource, which can
258result in some weird and potentially breaking changes if an older resource
259chooses to claim a hostname.
260
261## References
262
263Docs:
264
265* [Gateway API: Replacing TLS Certificates in Routes](https://docs.google.com/document/d/1Cv95XFCL6S_9pIyS0drnsDLsfinWc2tHOFl_x3-_SWI/edit)View as plain text