1# GEP-1686: Mesh conformance testing plan
2
3- Issue: [#1686](https://github.com/kubernetes-sigs/gateway-api/issues/1686)
4- Status: Experimental
5
6> **Note**: This GEP is exempt from the [Probationary Period][expprob] rules
7> of our GEP overview as it existed before those rules did, and so it has been
8> explicitly grandfathered in.
9
10[expprob]:https://gateway-api.sigs.k8s.io/geps/overview/#probationary-period
11
12## TLDR
13
14This testing plan specifies a new set of tests to define a "Mesh" [conformance profile](https://github.com/kubernetes-sigs/gateway-api/issues/1709).
15
16## Goals
17
18* Define a strategy for segmenting GAMMA tests from the existing conformance test suite
19* Define a set of test scenarios to capture conformance with the GAMMA spec
20* Rely on existing tests for non-GAMMA-specific Gateway API conformance
21
22## Focus
23
24Currently the GAMMA spec consists of two Gateway API GEPs [defining terminology and goals of Gateway API for service meshes](https://gateway-api.sigs.k8s.io/geps/gep-1324/)
25and specifically [how route resources work in a service mesh context](https://gateway-api.sigs.k8s.io/geps/gep-1426/).
26The goal of the initial conformance testing is to check the essential behavior as defined by GEP-1426, as it differs from the wider Gateway API spec. This GEP focuses on using a `Service` object as an `xRoute` `parentRef` to control how the GAMMA implementation directs traffic to the endpoints specified by the `Services` in `backendRefs` and how the traffic is filtered and modified.
27
28## Conformance Profile
29
30GAMMA intends to introduce a "Mesh" [conformance profile](https://gateway-api.sigs.k8s.io/geps/gep-1709/) to isolate tests specific to East/West functionality from both existing tests focused on North/South functionality and common Gateway API functionality shared by N/S and E/W implementations. A conformance profile is a set of tests that implementations can run to check their conformance to some subset of the Gateway API spec.
31
32This appropach will enable service meshes to certify that an implementation follows the GAMMA spec without requiring a North/South implementation, and importantly avoid any expectation that North/South Gateway API implementations expand their scope to understand GAMMA and E/W traffic flows.
33
34Leveraging existing tests for common functionality between N/S and E/W implementations will both ensure consistency across Gateway API implementations and help limit the maintence burden for the conformance testing suite.
35
36### Support Levels
37
38Using a conformance profile will enable granular conformance definitions for GAMMA, splitting functionality along the existing Gateway API [support levels](https://gateway-api.sigs.k8s.io/concepts/conformance/?h=conformance+levels#2-support-levels), with required functionality as Core, standardized but optional functionality as Extended, and Implementation-specific for things beyond the current or intended scope of GAMMA configuration. It is expected that some capabilities will begin as Implementation-specific and eventually migrate to Extended or Core conformance as GAMMA matures.
39
40## Tests
41
42Testing GAMMA implementations requires both a new suite of test cases as well as refactoring the existing test framework setup.
43
44### Runner and Setup
45
46The existing Gateway API conformance tests use a relatively simple implementation to send requests from outside a Kubernetes cluster to a gateway sitting at the edge, [capture the request and response](https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/utils/roundtripper/roundtripper.go), and [assert a match against an expected response](https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/utils/http/http.go).
47
48GAMMA conformance tests should still be based around a request/expected response suite, but requests will need to originate from _inside the cluster_, from either the same or different namespace as the target service. Adopting or developing tooling to enable this is being explored in [gateway-api#1340](https://github.com/kubernetes-sigs/gateway-api/issues/1340).
49
50### Scenarios
51
52All requests are sent from a client inside the same cluster/mesh and the same `Namespace`
53as the `Service` under test.
54Test scenarios are largely focused on the `backendRefs` and the
55`Namespace` of an `xRoute` resource.
56
57#### `Service` as `parentRef`
58
59- Given a simple `HTTPRoute` with a single `backendRef`
60 - With an explicit `port` in `parentRef`
61 - Assert that only requests to this `Service` and `port` are directed to the
62 backend
63 - Without a `port` in `parentRef`
64 - Assert that all requests to this `Service` are directed to the backend
65
66#### Omitted `backendRefs`
67
68- Given a simple `HTTPRoute` without `backendRefs`
69 - Assert that requests are directed to the endpoints defined by the `Service`
70 `parentRef` in its backend role
71
72#### Only `Services` as frontends are affected
73
74- Given a simple `HTTPRoute` with a single `backendRef`
75 - Send requests directly the endpoints of the `parentRef` `Service`'s backend
76 - Assert that traffic is not affected by the `HTTPRoute` resource
77
78#### `Namespace`-dependent behavior, producer vs consumer
79
80A producer `HTTPRoute` is in the same namespace as the `parentRef` `Service` (the
81producer).
82
83- Given a producer `HTTPRoute`
84 - Assert that traffic from a client in the producer `Namespace` is routed by the
85 `HTTPRoute`
86 - Assert that traffic from a client in a different `Namespace` is routed by the
87 `HTTPRoute`
88
89A consumer `HTTPRoute` is in the same `Namespace` as the the request sender (the
90consumer), a different `Namespace` as the `parentRef` `Service`.
91
92- Given a consumer `HTTPRoute`
93 - Assert that traffic from the consumer client is routed by the `HTTPRoute`
94 - Assert that traffic from a client in a different `Namespace` is _not_ routed by the
95 `HTTPRoute`
96
97Consumer routes have priority over producer routes.
98
99- Given both a consumer `HTTPRoute` and a producer `HTTPRoute`
100 - Assert that traffic from the consumer client is routed by the consumer `HTTPRoute`
101 - Assert that traffic from a client in a different `Namespace` is routed by
102 the producer `HTTPRoute`
103
104#### `xRoute`-specific
105
106- Given multiple `xRoutes` of different types
107 - Assert that routes take affect according to the specificity as defined in the spec
108- Given an `HTTPRoute` without `matches`, all requests are received at the `Service` endpoints as if no `HTTPRoute` existed
109- Given an `HTTPRoute` with `matches`, unmatched requests are dropped with a 404
110
111#### Filters
112
113Filters have the same effects on requests as any implementation. Gateway API conformance test framework can be
114refactored to extract checks on filter behavior for use on both GAMMA and Gateway API tests.
View as plain text