1# GEP-724: Refresh Route-Gateway Binding
2
3* Issue: [#724](https://github.com/kubernetes-sigs/gateway-api/issues/724)
4* Status: Standard
5
6## TLDR
7
8This GEP proposes changes to Route-Gateway binding that will result in Routes
9attaching to Gateways with direct references. When supporting Routes in multiple
10namespaces, Gateways will need to specify the namespaces they trust Routes in.
11These changes will slightly simplify the Route-Gateway relationship and make way
12for the future addition of Route inclusion (Routes including other Routes).
13
14## Goals
15
16Refactor cross-namespace Route-Gateway binding to:
17
18* Be more consistent with [cross-namespace references from
19 Routes](/geps/gep-709)
20* Provide a clear path to enable Route inclusion (Routes including Routes).
21* Simplify user experience based on initial feedback.
22* Enable other kinds of Route parents in addition to Gateway, this could include:
23 * Routes (as part of one potential approach to Route inclusion)
24 * Custom Gateway resources
25 * Mesh resources
26
27## Out of scope
28
29* Defining how Route inclusion will work.
30
31## Existing Approach
32
33The existing API already supports cross-namespace references. Gateways configure
34the following:
35
36* A Route label selector
37* Namespaces: Same, Selector, or All
38
39Routes then have three options as far as which Gateways can bind to them:
40
41* Same namespace (default)
42* From a list of Gateways
43* All
44
45Although this enables a great deal of flexibility, it can lead to confusion. For
46example, 2 separate label selectors from Gateway can be challenging for users to
47compute. It requires users to do to label selector lookups and then compute the
48intersection of that result. Additionally, the default behavior of selecting all
49Routes in the same namespace makes it easy to accidentally expose applications
50(see [#515](https://github.com/kubernetes-sigs/gateway-api/issues/515)).
51
52## Proposed Changes
53
54Although the API changes proposed here are relatively small, this represents a
55larger conceptual change. Instead of Gateways selecting Routes, Routes would
56directly reference the Gateways they wanted to attach to. This pattern was
57already possible with the existing API, but not clearly documented.
58
59One of the key concepts in the [cross-namespace references from Routes
60GEP](/geps/gep-709) was that of a handshake for
61references that cross namespace boundaries. A key part of that handshake was
62that one direction included a direct reference, while the other direction
63provided a way to denote trust for a set of Namespaces and kind of resources.
64
65It seems to make sense to carry that same underlying principle through to the
66Route - Gateway relationship. Given that each Gateway is likely to support many
67Routes, it would not be practical to support direct references from Gateways to
68Routes. Instead, it is simpler if Routes directly reference the Gateways they
69want to attach to. Gateways can then specify the namespaces they trust Routes
70to attach from.
71
72
73
74In the following example, the lb Gateway indicates that it trusts Routes from
75the foo Namespace, and the HTTPRoute in that namespace attaches directly to the
76Gateway.
77
78```yaml
79kind: Gateway
80metadata:
81 name: lb
82 namespace: infra
83spec:
84 listeners:
85 - name: foo
86 hostname: foo.com
87 port: 80
88 routes:
89 namespaces:
90 from: Selector
91 selector:
92 kubernetes.io/metadata.name: foo
93---
94kind: HTTPRoute
95metadata:
96 name: foo
97 namespace: foo
98spec:
99 attachTo:
100 - kind: Gateway
101 namespace: infra
102 name: lb
103 sectionName: foo
104 rules:
105 - name: abc
106 matches:
107 - path: /bar
108```
109
110## Rationale
111
112#### 1. Remove Complexity While it is Still Possible
113A goal of v1alpha2 is to make any breaking changes we think we'll need to for
114the lifetime of the API. After this release, we plan to provide a fully
115convertible, and hopefully also fully backwards compatible, API. If we really
116do need more advanced functionality in the future, it will be fairly
117straightforward to add in a future release. On the other hand, it will be near
118impossible to remove this complexity post-v1alpha2 if we were to leave it as is.
119
120#### 2. Route Selection Added Confusion and Did Not Enhance Security
121Although it was easy to look at the selector from Gateway -> Route as providing
122Gateway admins some form of control over the Routes attached to their Gateway,
123it was nothing but security theater. Route owners still had ultimate control by
124deciding how their Routes were labeled. This also made it difficult to document
125how Gateway and Route owners should interact. One possible explanation was that
126Gateway owners should provide Route owners with a set of labels that they should
127add to their Route to bind to a given Gateway. At that point, we were still
128ultimately relying on Route owners to attach a Route to a Gateway, just making
129it a rather clunky process.
130
131It should be noted that this proposal does still retain the ability for Gateways
132to restrict the namespaces they trust Routes to attach from. This was the only
133real control Gateway admins had before this proposed change.
134
135#### 3. The Existing Defaults Were Too Permissive
136One of the common complaints about the existing API was that the defaults made
137it far too easy to accidentally expose a Route. By default, a Gateway would be
138bound to all Routes in the same namespace. Although that made the getting
139started guide simple, it would inevitably lead to some unfortunate mistakes in
140the future. As we've already learned with Kubernetes, it's very difficult to
141recover from insecure defaults. Instead, it's much safer to start with more
142explicit configuration that demonstrates clear intent to connect resources.
143
144#### 4. We Need to Support non-Gateway Route Parents
145With the expansion of this API, it's clear that a Route may have non-Gateway
146parents. This may be other Routes, mesh implementations, or custom Gateways.
147Although none of these concepts are well specified at this point, making this
148change now will give us more flexibility in the future.
149
150#### 5. Users Want Control over the Gateways Their Routes Are Attached To
151Initial feedback we've received has shown that users want to have very clear
152control over the Gateways their Routes are attached to. Even in the case of
153Gateway replacement, many Route owners would prefer to be involved in the
154process.
155
156As we get more feedback and usage of the API, we may identify more users that
157are interested in the more advanced capabilities that some form of selection may
158enable, but at this point it's clear that a large portion of users value an
159explicit way to attach a Route to a Gateway.
160
161#### 6. We Need to Maintain a Handshake Between Gateways and Routes
162Of course we do still need a handshake that will enable cross-namespace
163references between Routes and Gateways. This proposal leaves in the core
164capabilities of the v1alpha1 API for this. Gateways can specify the namespaces
165they trust Routes to bind from, and Routes directly reference the Gateways they
166want to attach to. This is largely similar to the ReferenceGrant model proposed
167for Route->Service references, but is embedded within the Route and Gateway
168resources. The alternatives below explore what this could look like with
169ReferenceGrant.
170
171## API Changes
172
173The proposed changes here can be summarized as:
174
175* Remove Route selector from the RouteBindingSelector in Gateway listeners.
176* Replace Route kind and group with optional list of accepted kinds and groups
177 in RouteBindingSelector.
178* Rename RouteBindingSelector to ListenerRoutes.
179* Replace the 3 options from Route -> Gateway (All, FromList, SameNamespace)
180 with a reference list that supports arbitrary kinds.
181* Add a name to Gateway listeners.
182* Restructure listener status to include name, routeRefs, and supportedKinds
183 fields.
184
185### Gateway Spec
186
187In Gateway spec, the only change involves removing the Route selector field.
188Everything else remains the same.
189
190#### Removed
191The following fields would be removed from RouteBindingSelector:
192
193```go
194 // Selector specifies a set of route labels used for selecting
195 // routes to associate with the Gateway. If this Selector is defined,
196 // only routes matching the Selector are associated with the Gateway.
197 // An empty Selector matches all routes.
198 //
199 // Support: Core
200 //
201 // +optional
202 Selector *metav1.LabelSelector `json:"selector,omitempty"`
203 // Group is the group of the route resource to select. Omitting the value or specifying
204 // the empty string indicates the gateway.networking.k8s.io API group.
205 // For example, use the following to select an HTTPRoute:
206 //
207 // routes:
208 // kind: HTTPRoute
209 //
210 // Otherwise, if an alternative API group is desired, specify the desired
211 // group:
212 //
213 // routes:
214 // group: acme.io
215 // kind: FooRoute
216 //
217 // Support: Core
218 //
219 // +optional
220 // +kubebuilder:default=gateway.networking.k8s.io
221 // +kubebuilder:validation:MinLength=1
222 // +kubebuilder:validation:MaxLength=253
223 Group *string `json:"group,omitempty"`
224 // Kind is the kind of the route resource to select.
225 //
226 // Kind MUST correspond to kinds of routes that are compatible with the
227 // application protocol specified in the Listener's Protocol field.
228 //
229 // If an implementation does not support or recognize this
230 // resource type, it SHOULD set the "ResolvedRefs" condition to false for
231 // this listener with the "InvalidRoutesRef" reason.
232 //
233 // Support: Core
234 Kind string `json:"kind"`
235```
236
237#### Added
238Note: The ListMapKey annotation for listeners would also have to change to name
239for this.
240
241```go
242type Listener struct {
243 // Name is the name of the Listener. If more than one Listener is present
244 // each Listener MUST specify a name. The names of Listeners MUST be unique
245 // within a Gateway.
246 //
247 // Support: Core
248 //
249 // +kubebuilder:validation:MinLength=1
250 // +kubebuilder:validation:MaxLength=253
251 // +optional
252 Name *string `json:"name,omitempty"`
253 // ...
254}
255```
256
257The RouteBindingSelector struct would be renamed to ListenerRoutes, and a Kinds
258field would be added. Note that the Selector, Group, and Kind field would be
259removed from this struct as described above.
260
261```go
262type ListenerRoutes struct {
263 // ...
264 // Kinds specifies the groups and kinds of Routes that are allowed to bind to
265 // this Gateway listener. When unspecified or empty, the only limitation on
266 // the kinds of Routes supported is the Listener protocol. Kind MUST
267 // correspond to kinds of Routes that are compatible with the application
268 // protocol specified in the Listener's Protocol field. If an implementation
269 // does not support or recognize this resource type, it SHOULD set the
270 // "ResolvedRefs" condition to false for this listener with the
271 // "InvalidRoutesRef" reason.
272 //
273 // Support: Core
274 //
275 // +optional
276 // +kubebuilder:validation:MaxItems=10
277 Kinds []RouteGroupKind `json:"kinds,omitempty"`
278}
279
280type RouteGroupKind struct {
281 // Group is the group of the Route.
282 //
283 // Support: Core
284 //
285 // +optional
286 // +kubebuilder:default=gateway.networking.k8s.io
287 // +kubebuilder:validation:MaxLength=253
288 Group *string `json:"group,omitempty"`
289 // Kind is the kind of the Route.
290 //
291 // Support: Core
292 //
293 // +kubebuilder:validation:MinLength=1
294 // +kubebuilder:validation:MaxLength=253
295 Kind string `json:"kind"`
296}
297```
298
299### Gateway Status
300The most significant addition to the Gateway resource is in status. It may be
301helpful to share a sample of what the YAML would look like:
302
303```yaml
304status:
305 listeners:
306 - name: foo
307 supportedKinds:
308 - group: gateway.networking.k8s.io
309 kind: HTTPRoute
310 attachedRoutes: 1
311 conditions:
312 - ...
313```
314
315The key changes here all involve Listener status:
316
317* Replace the `port`, `protocol`, and `hostname` field with `name` to take
318 advantage of the new Listener name concept.
319* Add a new `supportedKinds` field. This will be most useful when the
320 corresponding field in the spec is left blank or when a user specifies kinds
321 that a controller does not support.
322
323Note: The ListMapKey annotation for listener status would also have to change to
324name for this.
325
326```go
327// ListenerStatus is the status associated with a Listener.
328type ListenerStatus struct {
329 // Name is the name of the Listener.
330 //
331 // +kubebuilder:validation:MinLength=1
332 // +kubebuilder:validation:MaxLength=253
333 // +optional
334 Name *string `json:"name,omitempty"`
335
336 // SupportedKinds is the list indicating the Kinds supported by this
337 // listener. When this is not specified on the Listener, this MUST represent
338 // the kinds an implementation supports for the specified protocol. When
339 // there are kinds specified on the Listener, this MUST represent the
340 // intersection of those kinds and the kinds supported by the implementation
341 // for the specified protocol.
342 //
343 // +kubebuilder:validation:MaxItems=10
344 // +optional
345 SupportedKinds []RouteGroupKind `json:"supportedKinds,omitempty"`
346
347 // AttachedRoutes represents the total number of Routes that have been
348 // successfully attached to this Listener.
349 AttachedRoutes int32 `json:"attachedRoutes"`
350
351 // Conditions...
352}
353```
354
355### Routes
356
357On Routes, we remove the `RouteGateways` struct and replace it with a list of
358parent references to attach to.
359
360#### Removed
361From Route Specs:
362```go
363 // Gateways defines which Gateways can use this Route.
364 //
365 // +optional
366 // +kubebuilder:default={allow: "SameNamespace"}
367 Gateways *RouteGateways `json:"gateways,omitempty"`
368```
369
370And the structs that references:
371```go
372// RouteGateways defines which Gateways will be able to use a route. If this
373// field results in preventing the selection of a Route by a Gateway, an
374// "Admitted" condition with a status of false must be set for the Gateway on
375// that Route.
376type RouteGateways struct {
377 // Allow indicates which Gateways will be allowed to use this route.
378 // Possible values are:
379 // * All: Gateways in any namespace can use this route.
380 // * FromList: Only Gateways specified in GatewayRefs may use this route.
381 // * SameNamespace: Only Gateways in the same namespace may use this route.
382 //
383 // +optional
384 // +kubebuilder:validation:Enum=All;FromList;SameNamespace
385 // +kubebuilder:default=SameNamespace
386 Allow *GatewayAllowType `json:"allow,omitempty"`
387
388 // GatewayRefs must be specified when Allow is set to "FromList". In that
389 // case, only Gateways referenced in this list will be allowed to use this
390 // route. This field is ignored for other values of "Allow".
391 //
392 // +optional
393 GatewayRefs []GatewayReference `json:"gatewayRefs,omitempty"`
394}
395```
396
397#### Added
398To Route Specs:
399```go
400 // ParentRefs references the resources that can attach to this Route. The only
401 // kind of parent resource with "Core" support is Gateway. This API may be
402 // extended in the future to support additional kinds of parent resources such
403 // as one of the route kinds. It is invalid to reference an identical parent
404 // more than once. It is valid to reference multiple distinct sections within
405 // the same parent resource, such as 2 Listeners within a Gateway.
406 //
407 // It is possible to separately reference multiple distinct objects that may
408 // be collapsed by an implementation. For example, some implementations may
409 // choose to merge compatible Gateway Listeners together. If that is the case,
410 // the list of routes attached to those resources should also be merged.
411 //
412 // +optional
413 // +kubebuilder:validation:MaxItems=16
414 ParentRefs []ParentRef `json:"parentRefs,omitempty"`
415```
416
417And the struct that references:
418```go
419// ParentRef identifies an API object that should be considered a parent of this
420// resource. The only kind of parent resource with "Core" support is Gateway.
421// This API may be extended in the future to support additional kinds of parent
422// resources, such as HTTPRoute.
423type ParentRef struct {
424 // Group is the group of the referent.
425 //
426 // Support: Core
427 //
428 // +kubebuilder:validation:MinLength=1
429 // +kubebuilder:validation:MaxLength=253
430 // +kubebuilder:default=gateway.networking.k8s.io
431 Group string `json:"group"`
432
433 // Kind is kind of the referent.
434 //
435 // Support: Core (Gateway)
436 // Support: Extended (Other Resources)
437 //
438 // +kubebuilder:validation:MinLength=1
439 // +kubebuilder:validation:MaxLength=253
440 // +kubebuilder:default=Gateway
441 // +optional
442 Kind *string `json:"kind,omitempty"`
443
444 // Namespace is the namespace of the referent. When unspecified (empty
445 // string), this will either be:
446 //
447 // * local namespace of the target is a namespace scoped resource
448 // * no namespace (not applicable) if the target is cluster-scoped.
449 //
450 // Support: Extended
451 //
452 // +kubebuilder:validation:MinLength=1
453 // +kubebuilder:validation:MaxLength=253
454 // +optional
455 Namespace *string `json:"namespace,omitempty"`
456
457 // Scope represents if this refers to a cluster or namespace scoped resource.
458 // This may be set to "Cluster" or "Namespace".
459 //
460 // Support: Core (Namespace)
461 // Support: Extended (Cluster)
462 //
463 // +kubebuilder:validation:Enum=Cluster;Namespace
464 // +kubebuilder:default=Namespace
465 // +optional
466 Scope *string `json:"scope,omitempty"`
467
468 // Name is the name of the referent.
469 //
470 // Support: Core
471 //
472 // +kubebuilder:validation:MinLength=1
473 // +kubebuilder:validation:MaxLength=253
474 Name string `json:"name"`
475
476 // SectionName is the name of a section within the target resource. In the
477 // following resources, SectionName is interpreted as the following:
478 //
479 // * Gateway: Listener Name
480 //
481 // Implementations MAY choose to support attaching Routes to other resources.
482 // If that is the case, they MUST clearly document how SectionName is
483 // interpreted.
484 //
485 // When unspecified (empty string), this will reference the entire resource.
486 // For the purpose of status, an attachment is considered successful if at
487 // least one section in the parent resource accepts it. For example, Gateway
488 // listeners can restrict which Routes can bind to them by Route kind,
489 // namespace, or hostname. If 1 of 2 Gateway listeners accept attachment from
490 // the referencing Route, the Route MUST be considered successfully
491 // attached. If no Gateway listeners accept attachment from this Route, the
492 // Route MUST be considered detached from the Gateway.
493 //
494 // Support: Core
495 //
496 // +kubebuilder:validation:MinLength=1
497 // +kubebuilder:validation:MaxLength=253
498 // +optional
499 SectionName *string `json:"sectionName,omitempty"`
500}
501```
502
503#### Changed
504
505To accommodate Routes with arbitrary types of parents, `RouteGatewayStatus` would
506be renamed to `RouteParentStatus`. Similarly, the `GatewayRef` inside that
507struct would be replaced with the `ParentRef` struct included above.
508
509### Advantages
510
511* Simplifies the API by providing a single way to attach Routes to Gateway.
512* Assigns clear responsibilities to Gateway and Route owners.
513* Consistent with pattern of direct references from Routes to all associated
514 resources.
515* Enables attaching Routes to arbitrary parents, such as custom Gateways, other
516 Routes (to be defined), or Meshes.
517* Prevents accidental exposure of Routes.
518* Easy to understand which Gateways/parents a Route is attached to.
519* Further simplifies path to Route inclusion.
520* Follows pattern of direct reference in one direction with a broader trust
521 reference in the other direction.
522* Aligns with initial user feedback.
523
524### Disadvantages
525
526* Attaching a Route to a named listener with SectionName may be a bit confusing.
527* Does not utilize existing ReferenceGrant mechanism.
528* May be more difficult to understand which Routes are attached to a Gateway.
529* Adding/replacing a Gateway requires changes to Routes.
530
531### Potential Expansion
532In the future, it may be useful to add a selector from Route -> Parent. Although
533this would enable greater flexibility, it also significantly increases
534complexity.
535
536## Alternatives
537
538### 1. ReferenceGrant with Gateways selecting Routes
539
540
541
542A compelling alternative to this proposal would involve retaining the Route
543selector in Gateway and replacing the trust concept in Routes with
544ReferenceGrant. To represent the same example as above, we'd use a Route
545selector on Gateway, a corresponding label on the HTTPRoute, and a
546ReferenceGrant that allowed it:
547
548```yaml
549kind: Gateway
550metadata:
551 name: xlb
552 namespace: infra
553spec:
554 listeners:
555 - name: foo
556 hostname: foo.com
557 port: 80
558 routes:
559 kind: HTTPRoute
560 selector:
561 gateway: xlb
562 namespaces:
563 from: Selector
564 selector:
565 kubernetes.io/metadata.name: foo
566---
567kind: HTTPRoute
568metadata:
569 name: foo
570 namespace: foo
571 labels:
572 gateway: xlb
573spec:
574 rules:
575 - name: abc
576 matches:
577 - path: /bar
578---
579kind: ReferenceGrant
580metadata:
581 name: infra-gateways
582 namespace: foo
583spec:
584 from:
585 - group: gateway.networking.k8s.io
586 kind: Gateway
587 namespace: infra
588 to:
589 - group: gateway.networking.k8s.io
590 kind: HTTPRoute
591```
592
593#### Advantages
594
595* Consistent use of ReferenceGrant throughout the API.
596* Provides a single way of binding Gateways to Routes.
597
598#### Disadvantages
599
600* Even the simplest cross-namespace reference from Gateway -> Route would
601 require a ReferenceGrant in each target namespace.
602* Existing demos and examples would become significantly more verbose.
603* Does not support attaching Routes to arbitrary parents.
604* Does not prevent accidental exposure of Routes.
605* Route owners have limited control in terms of which Gateways their Route is
606 attached to.
607
608### 2. ReferenceGrant with Routes referencing Gateways
609
610
611
612The other way we could use ReferenceGrant would be with Routes referencing
613Gateways. Unfortunately the nested structure of Gateways makes this nearly
614impossible to do effectively. A core concept for Gateways is that each listener
615should be able to attach to an entirely different set of Routes. For example,
616a Gateway may want to delegate foo.com to the foo namespace and bar.com to the
617bar namespace. Unfortunately that would be very difficult to recreate with
618ReferenceGrant.
619
620ReferenceGrant is fundamentally about trusting references from resource of kind
621Foo in to resources of kind Bar. Names and section names are intentionally
622excluded. If we added both of those concepts to ReferenceGrant, this would be
623possible, but quite complex and verbose. This is what the example from above
624would look like with this approach:
625
626```yaml
627kind: Gateway
628metadata:
629 name: lb
630 namespace: infra
631spec:
632 listeners:
633 - name: foo
634 hostname: foo.com
635 port: 80
636---
637kind: ReferenceGrant
638metadata:
639 name: foo-lb
640 namespace: infra
641spec:
642 from:
643 - group: gateway.networking.k8s.io
644 kind: HTTPRoute
645 namespace: foo
646 to:
647 - group: gateway.networking.k8s.io
648 kind: Gateway
649 name: lb
650 sectionName: foo
651---
652kind: HTTPRoute
653metadata:
654 name: foo
655 namespace: foo
656spec:
657 parentRefs:
658 - kind: Gateway
659 namespace: infra
660 name: lb
661 sectionName: foo
662 rules:
663 - name: abc
664 matches:
665 - path: /bar
666```
667
668#### Advantages
669
670* Consistent use of ReferenceGrant throughout the API.
671* Provides a single way of binding Gateways to Routes.
672* Supports attaching Routes to arbitrary parents.
673* Prevents accidental exposure of Routes.
674
675#### Disadvantages
676
677* In most cases, each listener in a Gateway would require a unique
678 ReferenceGrant resource.
679* Even the simplest cross-namespace reference from Route -> Gateway would
680 require a ReferenceGrant in each target namespace. This could either rule
681 out or significantly complicate self-service use-cases.
682* Existing demos and examples would become significantly more verbose.
683* ReferenceGrant would become more complex for all other use cases.
684
685## References
686
687**GEPs**
688
689* [GEP 709: ReferenceGrant + Cross Namespace References from Routes](/geps/gep-709)
690
691**Docs:**
692
693* [Gateway API References Doc](https://docs.google.com/document/d/18MoabVA-fr5XL9cYdf6cxclqRwFpOvHUXV_UYzSiooY/edit)
694* [Simplifying Gateway Route Binding](https://docs.google.com/document/d/1YVyB2dizACWrn8Rj31hQFBwCYqgyFKsxKeeGlv-iQCs/edit)
695
696**Issues:**
697
698* [Simplify Gateway Route Binding](https://github.com/kubernetes-sigs/gateway-api/issues/594)
699* [Route selector improvements on Gateway](https://github.com/kubernetes-sigs/gateway-api/issues/515)View as plain text