ConformanceReport is a report of conformance testing results including the specific conformance profiles that were tested and the results of the tests with summaries and statistics.
type ConformanceReport struct {
metav1.TypeMeta `json:",inline"`
Implementation `json:"implementation"`
// Date indicates the date that this report was generated.
Date string `json:"date"`
// GatewayAPIVersion indicates which release version of Gateway API this
// test report was made for.
GatewayAPIVersion string `json:"gatewayAPIVersion"`
// ProfileReports is a list of the individual reports for each conformance
// profile that was enabled for a test run.
ProfileReports []ProfileReport `json:"profiles"`
}
ExtendedStatus shows the testing results for the extended support level.
type ExtendedStatus struct {
Status `json:",inline"`
// SupportedFeatures indicates which extended features were flagged as
// supported by the implementation and tests will be attempted for.
SupportedFeatures []string `json:"supportedFeatures,omitempty"`
// UnsupportedFeatures indicates which extended features the implementation
// does not have support for and therefore will not attempt to test.
UnsupportedFeatures []string `json:"unsupportedFeatures,omitempty"`
}
Implementation provides metadata information on the downstream implementation of Gateway API which ran conformance tests.
type Implementation struct {
// Organization refers to the company, group or individual which maintains
// the named implementation. Organizations can provide reports for any
// number of distinct Gateway API implementations they maintain, but need
// to identify themselves using this organization field for grouping.
Organization string `json:"organization"`
// Project indicates the name of the project or repository for a Gateway API
// implementation.
Project string `json:"project"`
// URL indicates a human-usable URL where more information about the
// implementation can be found. For open source projects this should
// generally link to the code repository.
URL string `json:"url"`
// Version indicates the version of the implementation that was used for
// testing. This should generally be a semver version when applicable.
Version string `json:"version"`
// Contact is contact information for the maintainers so that Gateway API
// maintainers can get ahold of them as needed. Ideally this should be
// Github usernames (in the form of `@<username>`) or team names (in the
// form of `@<team>/<name>`), but when that's not possible it can be email
// addresses.
// Rather than Github usernames or email addresses you can provide a URL to the relevant
// support pages for the project. Ideally this would be something like the issue creation page
// on a repository, but for projects without a publicly exposed repository a general support
// page URL can be provided.
Contact []string `json:"contact"`
}
ProfileReport is the generated report for the test results of a specific named conformance profile.
type ProfileReport struct {
// Name indicates the name of the conformance profile (e.g. "HTTP",
// "TLS", "Mesh", e.t.c.).
Name string `json:"name"`
// Core indicates the core support level which includes the set of tests
// which are the minimum the implementation must pass to be considered at
// all conformant.
Core Status `json:"core"`
// Extended indicates the extended support level which includes additional
// optional features which the implementation may choose to implement
// support for, but are not required.
Extended *ExtendedStatus `json:"extended,omitempty"`
}
Result is a simple high-level summary describing the conclusion of a test run.
type Result string
var (
// Success indicates that the test run concluded in all required tests
// passing.
Success Result = "success"
// Partial indicates that the test run concluded in some of the required
// tests passing without any failures, but some were skipped.
Partial Result = "partial"
// Failure indicates that the test run concluded in one ore more tests
// failing to complete successfully.
Failure Result = "failure"
)
Statistics includes numerical summaries of the number of conformance tests that passed, failed or were intentionally skipped.
type Statistics struct {
// Passed indicates how many tests completed successfully.
Passed uint32
// Skipped indicates how many tests were intentionally not run, whether due
// to lack of feature support or whether they were explicitly disabled in
// the test suite.
Skipped uint32
// Failed indicates how many tests were unsuccessful.
Failed uint32
}
Status includes details on the results of a test.
type Status struct {
Result `json:"result"`
// Summary is a human-readable message intended for end-users to understand
// the overall status at a glance.
Summary string `json:"summary"`
// Statistics includes numerical statistics on the result of the test run.
Statistics `json:"statistics"`
// SkippedTests indicates which tests were explicitly disabled in the test
// suite. Skipping tests for Core level support implicitly identifies the
// results as being partial and the implementation will not be considered
// conformant at any level.
SkippedTests []string `json:"skippedTests,omitempty"`
// FailedTests indicates which tests were failing during the execution of
// test suite.
FailedTests []string `json:"failedTests,omitempty"`
}