...

Package controllerruntime

import "sigs.k8s.io/controller-runtime"
Overview
Index
Examples
Subdirectories

Overview ▾

Package controllerruntime provides tools to construct Kubernetes-style controllers that manipulate both Kubernetes CRDs and aggregated/built-in Kubernetes APIs.

It defines easy helpers for the common use cases when building CRDs, built on top of customizable layers of abstraction. Common cases should be easy, and uncommon cases should be possible. In general, controller-runtime tries to guide users towards Kubernetes controller best-practices.

Getting Started

The main entrypoint for controller-runtime is this root package, which contains all of the common types needed to get started building controllers:

import (
	ctrl "sigs.k8s.io/controller-runtime"
)

The examples in this package walk through a basic controller setup. The kubebuilder book (https://book.kubebuilder.io) has some more in-depth walkthroughs.

controller-runtime favors structs with sane defaults over constructors, so it's fairly common to see structs being used directly in controller-runtime.

Organization

A brief-ish walkthrough of the layout of this library can be found below. Each package contains more information about how to use it.

Frequently asked questions about using controller-runtime and designing controllers can be found at https://github.com/kubernetes-sigs/controller-runtime/blob/main/FAQ.md.

Managers

Every controller and webhook is ultimately run by a Manager (pkg/manager). A manager is responsible for running controllers and webhooks, and setting up common dependencies, like shared caches and clients, as well as managing leader election (pkg/leaderelection). Managers are generally configured to gracefully shut down controllers on pod termination by wiring up a signal handler (pkg/manager/signals).

Controllers

Controllers (pkg/controller) use events (pkg/event) to eventually trigger reconcile requests. They may be constructed manually, but are often constructed with a Builder (pkg/builder), which eases the wiring of event sources (pkg/source), like Kubernetes API object changes, to event handlers (pkg/handler), like "enqueue a reconcile request for the object owner". Predicates (pkg/predicate) can be used to filter which events actually trigger reconciles. There are pre-written utilities for the common cases, and interfaces and helpers for advanced cases.

Reconcilers

Controller logic is implemented in terms of Reconcilers (pkg/reconcile). A Reconciler implements a function which takes a reconcile Request containing the name and namespace of the object to reconcile, reconciles the object, and returns a Response or an error indicating whether to requeue for a second round of processing.

Clients and Caches

Reconcilers use Clients (pkg/client) to access API objects. The default client provided by the manager reads from a local shared cache (pkg/cache) and writes directly to the API server, but clients can be constructed that only talk to the API server, without a cache. The Cache will auto-populate with watched objects, as well as when other structured objects are requested. The default split client does not promise to invalidate the cache during writes (nor does it promise sequential create/get coherence), and code should not assume a get immediately following a create/update will return the updated resource. Caches may also have indexes, which can be created via a FieldIndexer (pkg/client) obtained from the manager. Indexes can used to quickly and easily look up all objects with certain fields set. Reconcilers may retrieve event recorders (pkg/recorder) to emit events using the manager.

Schemes

Clients, Caches, and many other things in Kubernetes use Schemes (pkg/scheme) to associate Go types to Kubernetes API Kinds (Group-Version-Kinds, to be specific).

Webhooks

Similarly, webhooks (pkg/webhook/admission) may be implemented directly, but are often constructed using a builder (pkg/webhook/admission/builder). They are run via a server (pkg/webhook) which is managed by a Manager.

Logging and Metrics

Logging (pkg/log) in controller-runtime is done via structured logs, using a log set of interfaces called logr (https://pkg.go.dev/github.com/go-logr/logr). While controller-runtime provides easy setup for using Zap (https://go.uber.org/zap, pkg/log/zap), you can provide any implementation of logr as the base logger for controller-runtime.

Metrics (pkg/metrics) provided by controller-runtime are registered into a controller-runtime-specific Prometheus metrics registry. The manager can serve these by an HTTP endpoint, and additional metrics may be registered to this Registry as normal.

Testing

You can easily build integration and unit tests for your controllers and webhooks using the test Environment (pkg/envtest). This will automatically stand up a copy of etcd and kube-apiserver, and provide the correct options to connect to the API server. It's designed to work well with the Ginkgo testing framework, but should work with any testing setup.

Example

This example creates a simple application Controller that is configured for ReplicaSets and Pods. * Create a new application for ReplicaSets that manages Pods owned by the ReplicaSet and calls into ReplicaSetReconciler. * Start the application.

Code:

log := ctrl.Log.WithName("builder-examples")

manager, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{})
if err != nil {
    log.Error(err, "could not create manager")
    os.Exit(1)
}

err = ctrl.
    NewControllerManagedBy(manager). // Create the Controller
    For(&appsv1.ReplicaSet{}).       // ReplicaSet is the Application API
    Owns(&corev1.Pod{}).             // ReplicaSet owns Pods created by it
    Complete(&ReplicaSetReconciler{Client: manager.GetClient()})
if err != nil {
    log.Error(err, "could not create controller")
    os.Exit(1)
}

if err := manager.Start(ctrl.SetupSignalHandler()); err != nil {
    log.Error(err, "could not start manager")
    os.Exit(1)
}

Example (CustomHandler)

This example creates a simple application Controller that is configured for ExampleCRDWithConfigMapRef CRD. Any change in the configMap referenced in this Custom Resource will cause the re-reconcile of the parent ExampleCRDWithConfigMapRef due to the implementation of the .Watches method of "sigs.k8s.io/controller-runtime/pkg/builder".Builder.

Code:

log := ctrl.Log.WithName("builder-examples")

manager, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{})
if err != nil {
    log.Error(err, "could not create manager")
    os.Exit(1)
}

err = ctrl.
    NewControllerManagedBy(manager).
    For(&ExampleCRDWithConfigMapRef{}).
    Watches(&corev1.ConfigMap{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, cm client.Object) []ctrl.Request {
        // map a change from referenced configMap to ExampleCRDWithConfigMapRef, which causes its re-reconcile
        crList := &ExampleCRDWithConfigMapRefList{}
        if err := manager.GetClient().List(ctx, crList); err != nil {
            manager.GetLogger().Error(err, "while listing ExampleCRDWithConfigMapRefs")
            return nil
        }

        reqs := make([]ctrl.Request, 0, len(crList.Items))
        for _, item := range crList.Items {
            if item.ConfigMapRef.Name == cm.GetName() {
                reqs = append(reqs, ctrl.Request{
                    NamespacedName: types.NamespacedName{
                        Namespace: item.GetNamespace(),
                        Name:      item.GetName(),
                    },
                })
            }
        }

        return reqs
    })).
    Complete(reconcile.Func(func(ctx context.Context, r reconcile.Request) (reconcile.Result, error) {
        // Your business logic to implement the API by creating, updating, deleting objects goes here.
        return reconcile.Result{}, nil
    }))
if err != nil {
    log.Error(err, "could not create controller")
    os.Exit(1)
}

if err := manager.Start(ctrl.SetupSignalHandler()); err != nil {
    log.Error(err, "could not start manager")
    os.Exit(1)
}

Example (UpdateLeaderElectionDurations)

This example creates a simple application Controller that is configured for ReplicaSets and Pods. This application controller will be running leader election with the provided configuration in the manager options. If leader election configuration is not provided, controller runs leader election with default values. Default values taken from: https://github.com/kubernetes/component-base/blob/master/config/v1alpha1/defaults.go * defaultLeaseDuration = 15 * time.Second * defaultRenewDeadline = 10 * time.Second * defaultRetryPeriod = 2 * time.Second * Create a new application for ReplicaSets that manages Pods owned by the ReplicaSet and calls into ReplicaSetReconciler. * Start the application.

Code:

log := ctrl.Log.WithName("builder-examples")
leaseDuration := 100 * time.Second
renewDeadline := 80 * time.Second
retryPeriod := 20 * time.Second
manager, err := ctrl.NewManager(
    ctrl.GetConfigOrDie(),
    ctrl.Options{
        LeaseDuration: &leaseDuration,
        RenewDeadline: &renewDeadline,
        RetryPeriod:   &retryPeriod,
    })
if err != nil {
    log.Error(err, "could not create manager")
    os.Exit(1)
}

err = ctrl.
    NewControllerManagedBy(manager). // Create the Controller
    For(&appsv1.ReplicaSet{}).       // ReplicaSet is the Application API
    Owns(&corev1.Pod{}).             // ReplicaSet owns Pods created by it
    Complete(&ReplicaSetReconciler{Client: manager.GetClient()})
if err != nil {
    log.Error(err, "could not create controller")
    os.Exit(1)
}

if err := manager.Start(ctrl.SetupSignalHandler()); err != nil {
    log.Error(err, "could not start manager")
    os.Exit(1)
}

Variables

var (
    // RegisterFlags registers flag variables to the given FlagSet if not already registered.
    // It uses the default command line FlagSet, if none is provided. Currently, it only registers the kubeconfig flag.
    RegisterFlags = config.RegisterFlags

    // GetConfigOrDie creates a *rest.Config for talking to a Kubernetes apiserver.
    // If --kubeconfig is set, will use the kubeconfig file at that location.  Otherwise will assume running
    // in cluster and use the cluster provided kubeconfig.
    //
    // Will log an error and exit if there is an error creating the rest.Config.
    GetConfigOrDie = config.GetConfigOrDie

    // GetConfig creates a *rest.Config for talking to a Kubernetes apiserver.
    // If --kubeconfig is set, will use the kubeconfig file at that location.  Otherwise will assume running
    // in cluster and use the cluster provided kubeconfig.
    //
    // Config precedence
    //
    // * --kubeconfig flag pointing at a file
    //
    // * KUBECONFIG environment variable pointing at a file
    //
    // * In-cluster config if running in cluster
    //
    // * $HOME/.kube/config if exists.
    GetConfig = config.GetConfig

    // NewControllerManagedBy returns a new controller builder that will be started by the provided Manager.
    NewControllerManagedBy = builder.ControllerManagedBy

    // NewWebhookManagedBy returns a new webhook builder that will be started by the provided Manager.
    NewWebhookManagedBy = builder.WebhookManagedBy

    // NewManager returns a new Manager for creating Controllers.
    // Note that if ContentType in the given config is not set, "application/vnd.kubernetes.protobuf"
    // will be used for all built-in resources of Kubernetes, and "application/json" is for other types
    // including all CRD resources.
    NewManager = manager.New

    // CreateOrUpdate creates or updates the given object obj in the Kubernetes
    // cluster. The object's desired state should be reconciled with the existing
    // state using the passed in ReconcileFn. obj must be a struct pointer so that
    // obj can be updated with the content returned by the Server.
    //
    // It returns the executed operation and an error.
    CreateOrUpdate = controllerutil.CreateOrUpdate

    // SetControllerReference sets owner as a Controller OwnerReference on owned.
    // This is used for garbage collection of the owned object and for
    // reconciling the owner object on changes to owned (with a Watch + EnqueueRequestForOwner).
    // Since only one OwnerReference can be a controller, it returns an error if
    // there is another OwnerReference with Controller flag set.
    SetControllerReference = controllerutil.SetControllerReference

    // SetupSignalHandler registered for SIGTERM and SIGINT. A stop channel is returned
    // which is closed on one of these signals. If a second signal is caught, the program
    // is terminated with exit code 1.
    SetupSignalHandler = signals.SetupSignalHandler

    // Log is the base logger used by controller-runtime.  It delegates
    // to another logr.Logger.  You *must* call SetLogger to
    // get any actual logging.
    Log = log.Log

    // LoggerFrom returns a logger with predefined values from a context.Context.
    // The logger, when used with controllers, can be expected to contain basic information about the object
    // that's being reconciled like:
    // - `reconciler group` and `reconciler kind` coming from the For(...) object passed in when building a controller.
    // - `name` and `namespace` from the reconciliation request.
    //
    // This is meant to be used with the context supplied in a struct that satisfies the Reconciler interface.
    LoggerFrom = log.FromContext

    // LoggerInto takes a context and sets the logger as one of its keys.
    //
    // This is meant to be used in reconcilers to enrich the logger within a context with additional values.
    LoggerInto = log.IntoContext

    // SetLogger sets a concrete logging implementation for all deferred Loggers.
    SetLogger = log.SetLogger
)

type Builder

Builder builds an Application ControllerManagedBy (e.g. Operator) and returns a manager.Manager to start it.

type Builder = builder.Builder

type GroupResource

GroupResource specifies a Group and a Resource, but does not force a version. This is useful for identifying concepts during lookup stages without having partially valid types.

type GroupResource = schema.GroupResource

type GroupVersion

GroupVersion contains the "group" and the "version", which uniquely identifies the API.

type GroupVersion = schema.GroupVersion

type Manager

Manager initializes shared dependencies such as Caches and Clients, and provides them to Runnables. A Manager is required to create Controllers.

type Manager = manager.Manager

type ObjectMeta

ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.

type ObjectMeta = metav1.ObjectMeta

type Options

Options are the arguments for creating a new Manager.

type Options = manager.Options

type Request

Request contains the information necessary to reconcile a Kubernetes object. This includes the information to uniquely identify the object - its Name and Namespace. It does NOT contain information about any specific Event or the object contents itself.

type Request = reconcile.Request

type Result

Result contains the result of a Reconciler invocation.

type Result = reconcile.Result

type SchemeBuilder

SchemeBuilder builds a new Scheme for mapping go types to Kubernetes GroupVersionKinds.

type SchemeBuilder = scheme.Builder

type TypeMeta

TypeMeta describes an individual object in an API response or request with strings representing the type of the object and its API schema version. Structures that are versioned or persisted should inline TypeMeta.

+k8s:deepcopy-gen=false

type TypeMeta = metav1.TypeMeta

Subdirectories

Name Synopsis
..
examples
builtins
crd
pkg +kubebuilder:object:generate=true +groupName=chaosapps.metamagical.io
tokenreview
pkg Package pkg provides libraries for building Controllers.
builder Package builder wraps other controller-runtime libraries and exposes simple patterns for building common Controllers.
cache Package cache provides object caches that act as caching client.Reader instances and help drive Kubernetes-object-based event handlers.
informertest
certwatcher Package certwatcher is a helper for reloading Certificates from disk to be used with tls servers.
metrics
client Package client contains functionality for interacting with Kubernetes API servers.
apiutil Package apiutil contains utilities for working with raw Kubernetes API machinery, such as creating RESTMappers and raw REST clients, and extracting the GVK of an object.
config Package config contains libraries for initializing REST configs for talking to the Kubernetes API
fake Package fake provides a fake client for testing.
interceptor
cluster
config
controller Package controller provides types and functions for building Controllers.
controllertest Package controllertest contains fake informers for testing controllers When in doubt, it's almost always better to test against a real API server using envtest.Environment.
controllerutil Package controllerutil contains utility functions for working with and implementing Controllers.
conversion Package conversion provides interface definitions that an API Type needs to implement for it to be supported by the generic conversion webhook handler defined under pkg/webhook/conversion.
envtest Package envtest provides libraries for integration testing by starting a local control plane
komega
event Package event contains the definitions for the Event types produced by source.Sources and transformed into reconcile.Requests by handler.EventHandler.
finalizer
handler Package handler defines EventHandlers that enqueue reconcile.Requests in response to Create, Update, Deletion Events observed from Watching Kubernetes APIs.
healthz Package healthz contains helpers from supporting liveness and readiness endpoints.
leaderelection Package leaderelection contains a constructor for a leader election resource lock.
fake Package fake mocks a resource lock for testing purposes.
log Package log contains utilities for fetching a new logger when one is not already available.
zap Package zap contains helpers for setting up a new logr.Logger instance using the Zap logging framework.
manager Package manager is required to create Controllers and provides shared dependencies such as clients, caches, schemes, etc.
signals Package signals contains libraries for handling signals to gracefully shutdown the manager in combination with Kubernetes pod graceful termination policy.
metrics Package metrics contains controller related metrics utilities
filters
server Package server provides the metrics server implementation.
predicate Package predicate defines Predicates used by Controllers to filter Events before they are provided to EventHandlers.
ratelimiter Package ratelimiter defines rate limiters used by Controllers to limit how frequently requests may be queued.
reconcile Package reconcile defines the Reconciler interface to implement Kubernetes APIs.
recorder Package recorder defines interfaces for working with Kubernetes event recorders.
scheme Package scheme contains utilities for gradually building Schemes, which contain information associating Go types with Kubernetes groups, versions, and kinds.
source Package source provides event streams to hook up to Controllers with Controller.Watch.
webhook Package webhook provides methods to build and bootstrap a webhook server.
admission Package admission provides implementation for admission webhook and methods to implement admission webhook handlers.
authentication Package authentication provides implementation for authentication webhook and methods to implement authentication webhook handlers.
conversion Package conversion provides implementation for CRD conversion webhook that implements handler for version conversion requests for types that are convertible.