1# Building Kubernetes
2
3Building Kubernetes is easy if you take advantage of the containerized build environment. This document will help guide you through understanding this build process.
4
5## Requirements
6
71. Docker, using one of the following configurations:
8 * **macOS** Install Docker for Mac. See installation instructions [here](https://docs.docker.com/desktop/install/mac-install/).
9 **Note**: You will want to set the Docker VM to have at least 8GB of initial memory or building will likely fail. (See: [#11852]( http://issue.k8s.io/11852)).
10 * **Linux with local Docker** Install Docker according to the [instructions](https://docs.docker.com/installation/#installation) for your OS.
11 * **Windows with Docker Desktop WSL2 backend** Install Docker according to the [instructions](https://docs.docker.com/docker-for-windows/wsl-tech-preview/). Be sure to store your sources in the local Linux file system, not the Windows remote mount at `/mnt/c`.
12
13 **Note**: You will need to check if Docker CLI plugin buildx is properly installed (`docker-buildx` file should be present in `~/.docker/cli-plugins`). You can install buildx according to the [instructions](https://github.com/docker/buildx/blob/master/README.md#installing).
14
152. **Optional** [Google Cloud SDK](https://developers.google.com/cloud/sdk/)
16
17You must install and configure Google Cloud SDK if you want to upload your release to Google Cloud Storage and may safely omit this otherwise.
18
19## Overview
20
21While it is possible to build Kubernetes using a local golang installation, we have a build process that runs in a Docker container. This simplifies initial set up and provides for a very consistent build and test environment.
22
23## Key scripts
24
25The following scripts are found in the [`build/`](.) directory. Note that all scripts must be run from the Kubernetes root directory.
26
27* [`build/run.sh`](run.sh): Run a command in a build docker container. Common invocations:
28 * `build/run.sh make`: Build just linux binaries in the container. Pass options and packages as necessary.
29 * `build/run.sh make cross`: Build all binaries for all platforms. To build only a specific platform, add `KUBE_BUILD_PLATFORMS=<os>/<arch>`
30 * `build/run.sh make kubectl KUBE_BUILD_PLATFORMS=darwin/amd64`: Build the specific binary for the specific platform (`kubectl` and `darwin/amd64` respectively in this example)
31 * `build/run.sh make test`: Run all unit tests
32 * `build/run.sh make test-integration`: Run integration test
33 * `build/run.sh make test-cmd`: Run CLI tests
34* [`build/copy-output.sh`](copy-output.sh): This will copy the contents of `_output/dockerized/bin` from the Docker container to the local `_output/dockerized/bin`. It will also copy out specific file patterns that are generated as part of the build process. This is run automatically as part of `build/run.sh`.
35* [`build/make-clean.sh`](make-clean.sh): Clean out the contents of `_output`, remove any locally built container images and remove the data container.
36* [`build/shell.sh`](shell.sh): Drop into a `bash` shell in a build container with a snapshot of the current repo code.
37
38## Basic Flow
39
40The scripts directly under [`build/`](.) are used to build and test. They will ensure that the `kube-build` Docker image is built (based on [`build/build-image/Dockerfile`](build-image/Dockerfile) and after base image's `KUBE_BUILD_IMAGE_CROSS_TAG` from Dockerfile is replaced with one of those actual tags of the base image, like `v1.13.9-2`) and then execute the appropriate command in that container. These scripts will both ensure that the right data is cached from run to run for incremental builds and will copy the results back out of the container. You can specify a different registry/name and version for `kube-cross` by setting `KUBE_CROSS_IMAGE` and `KUBE_CROSS_VERSION`, see [`common.sh`](common.sh) for more details.
41
42The `kube-build` container image is built by first creating a "context" directory in `_output/images/build-image`. It is done there instead of at the root of the Kubernetes repo to minimize the amount of data we need to package up when building the image.
43
44There are 3 different containers instances that are run from this image. The first is a "data" container to store all data that needs to persist across to support incremental builds. Next there is an "rsync" container that is used to transfer data in and out to the data container. Lastly there is a "build" container that is used for actually doing build actions. The data container persists across runs while the rsync and build containers are deleted after each use.
45
46`rsync` is used transparently behind the scenes to efficiently move data in and out of the container. This will use an ephemeral port picked by Docker. You can modify this by setting the `KUBE_RSYNC_PORT` env variable.
47
48All Docker names are suffixed with a hash derived from the file path (to allow concurrent usage on things like CI machines) and a version number. When the version number changes all state is cleared and clean build is started. This allows the build infrastructure to be changed and signal to CI systems that old artifacts need to be deleted.
49
50## Build artifacts
51The build system output all its products to a top level directory in the source repository named `_output`.
52These include the binary compiled packages (e.g. kubectl, kube-scheduler etc.) and archived Docker images.
53If you intend to run a component with a docker image you will need to import it from this directory with
54the appropriate command (e.g. `docker import _output/release-images/amd64/kube-scheduler.tar k8s.io/kube-scheduler:$(git describe)`).
55
56## Releasing
57
58The [`build/release.sh`](release.sh) script will build a release. It will build binaries, run tests, (optionally) build runtime Docker images.
59
60The main output is a tar file: `kubernetes.tar.gz`. This includes:
61* Cross compiled client utilities.
62* Script (`kubectl`) for picking and running the right client binary based on platform.
63* Examples
64* Cluster deployment scripts for various clouds
65* Tar file containing all server binaries
66
67In addition, there are some other tar files that are created:
68* `kubernetes-client-*.tar.gz` Client binaries for a specific platform.
69* `kubernetes-server-*.tar.gz` Server binaries for a specific platform.
70
71When building final release tars, they are first staged into `_output/release-stage` before being tar'd up and put into `_output/release-tars`.
72
73## Reproducibility
74`make release` and its variant `make quick-release` provide a
75hermetic build environment which should provide some level of reproducibility
76for builds. `make` itself is **not** hermetic.
77
78The Kubernetes build environment supports the [`SOURCE_DATE_EPOCH` environment
79variable](https://reproducible-builds.org/specs/source-date-epoch/) specified by
80the Reproducible Builds project, which can be set to a UNIX epoch timestamp.
81This will be used for the build timestamps embedded in compiled Go binaries,
82and maybe someday also Docker images.
83
84One reasonable setting for this variable is to use the commit timestamp from the
85tip of the tree being built; this is what the Kubernetes CI system uses. For
86example, you could use the following one-liner:
87
88```bash
89SOURCE_DATE_EPOCH=$(git show -s --format=format:%ct HEAD)
90```
View as plain text