1# API serialization compatibility tests
2
3This directory tree contains serialized API objects in json and yaml formats.
4
5This creates JSON and YAML files for three APIs exposed by kubevirt in group-version "kubevirt.io/v1", versioned by the release. The main branch is will be stored in `HEAD` directory, previous versions are in `release-0.yy` release directory. While more APIs can be added in the future, the current list includes:
6 ```
7 VirtualMachineInstance
8 VirtualMachine
9 KubeVirt
10 ```
11
12## DEVELOPERS GUIDE
13
14### Populating data for each release
15
16When KubeVirt cuts a new release of the project, the current version files will be copied to the release version
17
18For example, to capture compatibility data for `v1.2.0`:
19
20```sh
21export VERSION=release-1.2
22git checkout ${VERSION}
23cp -fr staging/src/kubevirt.io/api/apitesting/testdata/{HEAD,${VERSION}}
24git checkout -b add-${VERSION}-api-testdata master
25git add .
26git commit -m "Add ${VERSION} API testdata"
27```
28
29### Current version
30
31The `HEAD` subdirectory contains serialized API objects generated from the current commit:
32
33```
34HEAD/
35 <group>.<version>.<kind>.[json|yaml]
36```
37
38To run serialization tests just for the current version:
39
40```sh
41go test kubevirt.io/api/apitesting -run //HEAD
42```
43
44All the formats of a given group/version/kind are expected to decode successfully to identical objects,
45and to round-trip back to serialized form with identical bytes.
46Adding new fields or deprecating new fields or API types *is* expected to modify these fixtures. To regenerate them, run:
47
48```sh
49UPDATE_COMPATIBILITY_FIXTURE_DATA=true go test kubevirt.io/api/apitesting -run //HEAD
50```
51
52### Previous versions
53
54The vX.Y.0 subdirectories contain serialized API objects from previous releases:
55
56```
57release-X.Y
58 <group>.<version>.<kind>.[json|yaml]
59```
60
61To run serialization tests for a previous version, like `v1.1.0`:
62
63```sh
64go test kubevirt.io/api/apitesting -run //release-1.1
65```
66
67To run serialization tests for a particular group/version/kind, like `apps/v1` `Deployment`:
68```sh
69go test kubevirt.io/api/apitesting -run /apps.v1.Deployment/
70```
71
72Example output:
73
74```
75--- FAIL: TestCompatibility/kubevirt.io.v1.VirtualMachineInstance (0.01s)
76 --- FAIL: TestCompatibility/kubevirt.io.v1.VirtualMachineInstance/release-0.50 (0.01s)
77 compatibility.go:416: json differs
78 compatibility.go:417: (
79 """
80 ... // 215 identical lines
81 "readonly": true
82 },
83 - "floppy": {
84 - "readonly": true,
85 - "tray": "trayValue"
86 - },
87 "cdrom": {
88 "bus": "busValue",
89 ... // 678 identical lines
90 "tscFrequency": -12
91 },
92 - "virtualMachineRevisionName": "virtualMachineRevisionNameValue"
93 + "virtualMachineRevisionName": "virtualMachineRevisionNameValue",
94 + "runtimeUser": 0
95 }
96 }
97 """
98 )
99
100 compatibility.go:422: yaml differs
101 compatibility.go:423: (
102 """
103 ... // 237 identical lines
104 pciAddress: pciAddressValue
105 readonly: true
106 - floppy:
107 - readonly: true
108 - tray: trayValue
109 io: ioValue
110 lun:
111 ... // 341 identical lines
112 qosClass: qosClassValue
113 reason: reasonValue
114 + runtimeUser: 0
115 topologyHints:
116 tscFrequency: -12
117 ... // 22 identical lines
118 """
119 )
120
121```
122
123The above output shows that for VirtualMachineInstance:
1241. api-field: `spec.domain.devices.disks.floppy` was dropped. [ref-1](https://github.com/kubevirt/kubevirt/issues/2016)[ref-2](https://github.com/kubevirt/kubevirt/pull/2164)
1252. api-field: `status.runtimeUser` field was added[ref-3](https://github.com/kubevirt/kubevirt/pull/6709)
126
127Here is the basic list of what is not allowed when modifying the API and are covered using these tests to ensure backward compatibility.
1281. Removing existing fields.
1292. Adding a new required field.
1303. Changing the type of existing fields.
1314. Renaming the fields.
1325. Changing field behaviour(e.g., converting a required field to optional or vice versa)
133
134## REVIEWERS GUIDE
135
136With any change in go structs of the APIs, the corresponding JSON and YAML files will be updated.
137
138When decoding, round-tripping identical bytes, or decoding identical objects from JSON/YAML, failures trigger detailed error outputs.
139These errors specify the differences found during the comparison.
140
141Some cases involve adding a new non-pointer fields to existing API types. These fields serialize zero values, which may lead to additional fields being output during the round-tripping process with data from previous releases.
142
143Example output:
144
145```
146--- FAIL: TestCompatibility/kubevirt.io.v1.VirtualMachine/release-1.0 (0.09s)
147 compatibility.go:411: json differs
148 compatibility.go:412: (
149 """
150 ... // 1113 identical lines
151 "status": {}
152 }
153 - ]
154 + ],
155 + "dummyField": null
156 },
157 "status": {
158 ... // 111 identical lines
159 """
160 )
161
162 compatibility.go:417: yaml differs
163 compatibility.go:418: (
164 """
165 ... // 181 identical lines
166 volumeName: volumeNameValue
167 status: {}
168 + dummyField: null
169 instancetype:
170 inferFromVolume: inferFromVolumeValue
171 ... // 654 identical lines
172 """
173 )
174
175```
176
177Before dummyField was added, the serialized representation did not include `dummyField` field. With dummyField added, the serialized representation now includes dummyField set to null.
178
179This discrepancy causes round-trip tests to fail, as the output includes unexpected fields.
180
181To address this, an expected data file (<group>.<version>.<kind>_after_roundtrip.[json|yaml]) is placed next to the serialized data file from a previous release.
182
183This file contains the expected data after the round-tripping process, accounting for any new fields or changes.
184
185These after_roundtrip files can be generated by running the failing tests with the environment variable UPDATE_COMPATIBILITY_FIXTURE_DATA=true. This updates the compatibility fixture data to reflect the current expected output.
186
187Using this API reviewers can say if changes in current version will break older clients upon upgrade.
188
189
190### Open Issues
191
192 Consider a scenario where upgrade blocked right after creating the crd and the admin had to abort the upgrade.
193
194 Check if this is a valid supported scenario? and how can it be tested?
View as plain text