1# kstatus
2
3kstatus provides tools for checking the status of Kubernetes resources. The primary use case is knowing when
4(or if) a given set of resources has been fully reconciled after an apply operation.
5
6## Concepts
7
8This effort has several goals, some with a shorter timeline than others. Initially, we want to provide
9a library that makes it easier to decide when changes to a set of resources have been reconciled in a cluster.
10To support types that do not yet publish status information, we will initially fallback on type specific rules.
11The library already contains rules for many of the most common built-in types such a Deployment and StatefulSet.
12
13For custom resource definitions (CRDs), there currently isn't much guidance on which properties should be exposed in the status
14object and which conditions should be used. As part of this effort we want to define a set of standard conditions
15that the library will understand and that we encourage developers to adopt in their CRDs. These standard conditions will
16be focused on providing the necessary information for understanding status of the reconcile after `apply` and it is not
17expected that these will necessarily be the only conditions exposed in a custom resource. Developers will be free to add as many conditions
18as they wish, but if the CRDs adopt the standard conditions defined here, this library will handle them correctly.
19
20The `status` objects for built-in types don't all conform to a common behavior. Not all built-in types expose conditions,
21and even among the types that does, the types of conditions vary widely. Long-term, we hope to add support for the
22standard conditions to the built-in types as well. This would remove the need for type-specific rules for determining
23status.
24
25### Statuses
26
27The status of a resource is a single value that represents the reconcile state for
28the resource at a single point in time.
29
30The library currently defines the following statuses for resource:
31* __InProgress__: The actual state of the resource has not yet reached the desired state as specified in the
32resource manifest, i.e. the resource reconcile has not yet completed. Newly created resources will usually
33start with this status, although some resources like ConfigMaps are Current immediately.
34* __Failed__: The process of reconciling the actual state with the desired state has encountered and error
35or it has made insufficient progress.
36* __Current__: The actual state of the resource matches the desired state. The reconcile process is considered
37complete until there are changes to either the desired or the actual state.
38* __Terminating__: The resource is in the process of being deleted.
39* __NotFound__: The resource does not exist in the cluster.
40* __Unknown__: This is for situations when the library are unable to determine the status of a resource.
41
42### Conditions
43
44The conditions defined in the library are designed to adhere to the "abnormal-true" polarity pattern
45(https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties)
46, i.e. that conditions are present and with a value of true whenever something unusual happens. So the absence of
47any conditions means everything is normal. Normal in this situation simply means that the latest observed
48generation of the resource manifest by the controller have been fully reconciled with the actual state.
49
50* __Reconciling__: The controller is currently working on reconciling the latest changes.
51* __Stalled__: The controller has encountered an error during the reconcile process or it has made
52insufficient progress (timeout).
53
54The use of the "abnormal-true" pattern has some challenges. If the controller is not running, or for some
55reason not able to update the resource, it will look like it is in a good state when that is not true. The
56solution to this issue is to adopt the pattern used by several of the built-in types where there is an
57`observedGeneration` property on the status object which is set by the controller during the reconcile loop.
58If the `generation` and the `observedGeneration` of a resource does not match, it means there are changes
59that the controller has not yet seen, and therefore not acted upon.
60
61#### The `Ready` Condition
62For resources that have not adopted the recommended conditions described above
63and we don't have a type-specific rule for the resource, the library
64will look for the `Ready` condition. If there is a `Ready` condition and it
65is `True`, the library will consider the resource to be fully reconciled. If
66there is a `Ready` condition and it is `False`, the library will consider the
67resource to be in the process of reconciling.
68
69There are a few corner cases here:
70 * If a resource doesn't set the Ready condition until it is True,
71the library have no way of telling whether the resource is using the
72Ready condition, so it will fall back to the strategy for unknown
73resources, which is to assume they are always reconciled.
74 * If the library sees the resource before the controller has had
75a chance to update the conditions, it also will not realize the
76resource use the Ready condition.
77 * There is no way to determine if a resource with the Ready condition
78set to False is making progress or is doomed.
79
80## Features
81
82The library is currently separated into two packages, one that provides the basic functionality, and another that
83builds upon the basics to provide a higher level API.
84
85**sigs.k8s.io/kustomize/kstatus/status**: Provides two basic functions. First, it provides the `Compute` function
86that takes a single resource and computes the status for this resource based on the fields in the status object for
87the resource. Second, it provides the `Augment` function that computes the appropriate standard conditions based on
88the status object and then amends them to the conditions in the resource. Both of these functions currently operate
89on Unstructured types, but this should eventually be changed to rely on the kyaml library. Both of these functions
90compute the status and conditions solely based on the data in the resource passed in. It does not communicate with
91a cluster to get the latest state of the resources.
92
93**sigs.k8s.io/kustomize/kstatus/polling**: This package builds upon the status package and provides functionality for
94polling the cluster for the latest state for all specified resources and compute status. The polling will terminate
95either when status for all resources reach the desired value, or when it is cancelled by the caller.
96
97## Challenges
98
99### Status is not obvious for all resource types
100
101For some types of resources, it is pretty clear what the different statuses mean. For others, it
102is far less obvious. For example, what does it mean that a PodDisruptionBudget is Current? Based on
103the assumptions above it probably should be whenever the controller has observed the resource
104and updated the status object of the PDB with information on allowed disruptions. But currently, a PDB is
105considered Current when the number of healthy replicas meets the threshold given in the PDB. Also, should
106the presence of a PDB influence when a Deployment is considered Current? This would mean that a Deployment
107should be considered Current whenever the number of replicas reach the threshold set by the corresponding
108PDB. This is not currently supported as described below.
109
110### Status for a resource depends on the status of other resources
111The status package computes the status of a resource solely based on the
112state of that particular resource. But not all resources expose sufficient
113information in their status object to fully determine the reconcile status
114of the resource. In particular this is challenging for some built-in resources
115that create other resources, such as Services that create Endpoints.
116
117The polling package address this by having a framework that allows the
118ResourceReader for a specific type to also look up the state of other
119resources and use their state in the computation.
120
121### Status is decided based on single resource
122Currently the status of a resource is decided solely based on information from
123the state of that resource. This is an issue for resources that create other resources
124and that doesn't provide sufficient information within their own status object. An example
125is the Service resource that doesn't provide much status information but do generate Endpoint
126resources that could be used to determine status. Similar, the status of a Deployment could be
127based on its generated ReplicaSets and Pods.
128
129Not having the generated resources also limits the amount of details that can be provided
130when something isn't working as expected.
131
132## Future
133
134### Depend on kyaml instead of k8s libraries
135The sigs.k8s.io/kustomize/kstatus/status package currently depends on k8s libraries. This can be
136challenging if someone wants to vendor the library within their own project. We want to replace
137the dependencies on k8s libraries with kyaml for the status package. The `polling` package needs to
138talk to a k8s cluster, so this package will continue to rely on the k8s libraries.
View as plain text