1# <a name="linuxContainerConfiguration" />Linux Container Configuration
2
3This document describes the schema for the [Linux-specific section](config.md#platform-specific-configuration) of the [container configuration](config.md).
4The Linux container specification uses various kernel features like namespaces, cgroups, capabilities, LSM, and filesystem jails to fulfill the spec.
5
6## <a name="configLinuxDefaultFilesystems" />Default Filesystems
7
8The Linux ABI includes both syscalls and several special file paths.
9Applications expecting a Linux environment will very likely expect these file paths to be set up correctly.
10
11The following filesystems SHOULD be made available in each container's filesystem:
12
13| Path | Type |
14| -------- | ------ |
15| /proc | [proc][] |
16| /sys | [sysfs][] |
17| /dev/pts | [devpts][] |
18| /dev/shm | [tmpfs][] |
19
20## <a name="configLinuxNamespaces" />Namespaces
21
22A namespace wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource.
23Changes to the global resource are visible to other processes that are members of the namespace, but are invisible to other processes.
24For more information, see the [namespaces(7)][namespaces.7_2] man page.
25
26Namespaces are specified as an array of entries inside the `namespaces` root field.
27The following parameters can be specified to set up namespaces:
28
29* **`type`** *(string, REQUIRED)* - namespace type. The following namespace types SHOULD be supported:
30 * **`pid`** processes inside the container will only be able to see other processes inside the same container or inside the same pid namespace.
31 * **`network`** the container will have its own network stack.
32 * **`mount`** the container will have an isolated mount table.
33 * **`ipc`** processes inside the container will only be able to communicate to other processes inside the same container via system level IPC.
34 * **`uts`** the container will be able to have its own hostname and domain name.
35 * **`user`** the container will be able to remap user and group IDs from the host to local users and groups within the container.
36 * **`cgroup`** the container will have an isolated view of the cgroup hierarchy.
37 * **`time`** the container will be able to have its own clocks.
38* **`path`** *(string, OPTIONAL)* - namespace file.
39 This value MUST be an absolute path in the [runtime mount namespace](glossary.md#runtime-namespace).
40 The runtime MUST place the container process in the namespace associated with that `path`.
41 The runtime MUST [generate an error](runtime.md#errors) if `path` is not associated with a namespace of type `type`.
42
43 If `path` is not specified, the runtime MUST create a new [container namespace](glossary.md#container-namespace) of type `type`.
44
45If a namespace type is not specified in the `namespaces` array, the container MUST inherit the [runtime namespace](glossary.md#runtime-namespace) of that type.
46If a `namespaces` field contains duplicated namespaces with same `type`, the runtime MUST [generate an error](runtime.md#errors).
47
48### Example
49
50```json
51"namespaces": [
52 {
53 "type": "pid",
54 "path": "/proc/1234/ns/pid"
55 },
56 {
57 "type": "network",
58 "path": "/var/run/netns/neta"
59 },
60 {
61 "type": "mount"
62 },
63 {
64 "type": "ipc"
65 },
66 {
67 "type": "uts"
68 },
69 {
70 "type": "user"
71 },
72 {
73 "type": "cgroup"
74 },
75 {
76 "type": "time"
77 }
78]
79```
80
81## <a name="configLinuxUserNamespaceMappings" />User namespace mappings
82
83**`uidMappings`** (array of objects, OPTIONAL) describes the user namespace uid mappings from the host to the container.
84**`gidMappings`** (array of objects, OPTIONAL) describes the user namespace gid mappings from the host to the container.
85
86Each entry has the following structure:
87
88* **`containerID`** *(uint32, REQUIRED)* - is the starting uid/gid in the container.
89* **`hostID`** *(uint32, REQUIRED)* - is the starting uid/gid on the host to be mapped to *containerID*.
90* **`size`** *(uint32, REQUIRED)* - is the number of ids to be mapped.
91
92The runtime SHOULD NOT modify the ownership of referenced filesystems to realize the mapping.
93Note that the number of mapping entries MAY be limited by the [kernel][user-namespaces].
94
95### Example
96
97```json
98"uidMappings": [
99 {
100 "containerID": 0,
101 "hostID": 1000,
102 "size": 32000
103 }
104],
105"gidMappings": [
106 {
107 "containerID": 0,
108 "hostID": 1000,
109 "size": 32000
110 }
111]
112```
113
114## <a name="configLinuxTimeOffset" />Offset for Time Namespace
115
116**`timeOffsets`** (object, OPTIONAL) sets the offset for Time Namespace. For more information
117see the [time_namespaces][time_namespaces.7].
118
119The name of the clock is the entry key.
120Entry values are objects with the following properties:
121
122* **`secs`** *(int64, OPTIONAL)* - is the offset of clock (in seconds) in the container.
123* **`nanosecs`** *(uint32, OPTIONAL)* - is the offset of clock (in nanoseconds) in the container.
124
125## <a name="configLinuxDevices" />Devices
126
127**`devices`** (array of objects, OPTIONAL) lists devices that MUST be available in the container.
128The runtime MAY supply them however it likes (with [`mknod`][mknod.2], by bind mounting from the runtime mount namespace, using symlinks, etc.).
129
130Each entry has the following structure:
131
132* **`type`** *(string, REQUIRED)* - type of device: `c`, `b`, `u` or `p`.
133 More info in [mknod(1)][mknod.1].
134* **`path`** *(string, REQUIRED)* - full path to device inside container.
135 If a [file][] already exists at `path` that does not match the requested device, the runtime MUST generate an error.
136 The path MAY be anywhere in the container filesystem, notably outside of `/dev`.
137* **`major, minor`** *(int64, REQUIRED unless `type` is `p`)* - [major, minor numbers][devices] for the device.
138* **`fileMode`** *(uint32, OPTIONAL)* - file mode for the device.
139 You can also control access to devices [with cgroups](#configLinuxDeviceAllowedlist).
140* **`uid`** *(uint32, OPTIONAL)* - id of device owner in the [container namespace](glossary.md#container-namespace).
141* **`gid`** *(uint32, OPTIONAL)* - id of device group in the [container namespace](glossary.md#container-namespace).
142
143The same `type`, `major` and `minor` SHOULD NOT be used for multiple devices.
144
145Containers MAY NOT access any device node that is not either explicitly
146referenced in the **`devices`** array or listed as being part of the
147[default devices](#configLinuxDefaultDevices).
148Rationale: runtimes based on virtual machines need to be able to adjust the node
149devices, and accessing device nodes that were not adjusted could have undefined
150behaviour.
151
152
153### Example
154
155```json
156"devices": [
157 {
158 "path": "/dev/fuse",
159 "type": "c",
160 "major": 10,
161 "minor": 229,
162 "fileMode": 438,
163 "uid": 0,
164 "gid": 0
165 },
166 {
167 "path": "/dev/sda",
168 "type": "b",
169 "major": 8,
170 "minor": 0,
171 "fileMode": 432,
172 "uid": 0,
173 "gid": 0
174 }
175]
176```
177
178### <a name="configLinuxDefaultDevices" />Default Devices
179
180In addition to any devices configured with this setting, the runtime MUST also supply:
181
182* [`/dev/null`][null.4]
183* [`/dev/zero`][zero.4]
184* [`/dev/full`][full.4]
185* [`/dev/random`][random.4]
186* [`/dev/urandom`][random.4]
187* [`/dev/tty`][tty.4]
188* `/dev/console` is set up if [`terminal`](config.md#process) is enabled in the config by bind mounting the pseudoterminal pty to `/dev/console`.
189* [`/dev/ptmx`][pts.4].
190 A [bind-mount or symlink of the container's `/dev/pts/ptmx`][devpts].
191
192## <a name="configLinuxControlGroups" />Control groups
193
194Also known as cgroups, they are used to restrict resource usage for a container and handle device access.
195cgroups provide controls (through controllers) to restrict cpu, memory, IO, pids, network and RDMA resources for the container.
196For more information, see the [kernel cgroups documentation][cgroup-v1].
197
198A runtime MAY, during a particular [container operation](runtime.md#operation),
199such as [create](runtime.md#create), [start](runtime.md#start), or
200[exec](runtime.md#exec), check if the container cgroup is fit for purpose,
201and MUST [generate an error](runtime.md#errors) if such a check fails.
202For example, a frozen cgroup or (for [create](runtime.md#create) operation)
203a non-empty cgroup. The reason for this is that accepting such configurations
204could cause container operation outcomes that users may not anticipate or
205understand, such as operation on one container inadvertently affecting other
206containers.
207
208### <a name="configLinuxCgroupsPath" />Cgroups Path
209
210**`cgroupsPath`** (string, OPTIONAL) path to the cgroups.
211It can be used to either control the cgroups hierarchy for containers or to run a new process in an existing container.
212
213The value of `cgroupsPath` MUST be either an absolute path or a relative path.
214
215* In the case of an absolute path (starting with `/`), the runtime MUST take the path to be relative to the cgroups mount point.
216* In the case of a relative path (not starting with `/`), the runtime MAY interpret the path relative to a runtime-determined location in the cgroups hierarchy.
217
218If the value is specified, the runtime MUST consistently attach to the same place in the cgroups hierarchy given the same value of `cgroupsPath`.
219If the value is not specified, the runtime MAY define the default cgroups path.
220Runtimes MAY consider certain `cgroupsPath` values to be invalid, and MUST generate an error if this is the case.
221
222Implementations of the Spec can choose to name cgroups in any manner.
223The Spec does not include naming schema for cgroups.
224The Spec does not support per-controller paths for the reasons discussed in the [cgroupv2 documentation][cgroup-v2].
225The cgroups will be created if they don't exist.
226
227You can configure a container's cgroups via the `resources` field of the Linux configuration.
228Do not specify `resources` unless limits have to be updated.
229For example, to run a new process in an existing container without updating limits, `resources` need not be specified.
230
231Runtimes MAY attach the container process to additional cgroup controllers beyond those necessary to fulfill the `resources` settings.
232
233### Cgroup ownership
234
235Runtimes MAY, according to the following rules, change (or cause to
236be changed) the owner of the container's cgroup to the host uid that
237maps to the value of `process.user.uid` in the [container
238namespace](glossary.md#container-namespace); that is, the user that
239will execute the container process.
240
241Runtimes SHOULD NOT change the ownership of container cgroups when
242cgroups v1 is in use. Cgroup delegation is not secure in cgroups
243v1.
244
245A runtime SHOULD NOT change the ownership of a container cgroup
246unless it will also create a new cgroup namespace for the container.
247Typically this occurs when the `linux.namespaces` array contains an
248object with `type` equal to `"cgroup"` and `path` unset.
249
250Runtimes SHOULD change the cgroup ownership if and only if the
251cgroup filesystem is to be mounted read/write; that is, when the
252configuration's `mounts` array contains an object where:
253
254- The `source` field is equal to `"cgroup"`
255- The `destination` field is equal to `"/sys/fs/cgroup"`
256- The `options` field does not contain the value `"ro"`
257
258If the configuration does not specify such a mount, the runtime
259SHOULD NOT change the cgroup ownership.
260
261A runtime that changes the cgroup ownership SHOULD only change the
262ownership of the container's cgroup directory and files within that
263directory that are listed in `/sys/kernel/cgroup/delegate`. See
264`cgroups(7)` for details about this file. Note that not all files
265listed in `/sys/kernel/cgroup/delegate` necessarily exist in every
266cgroup. Runtimes MUST NOT fail in this scenario, and SHOULD change
267the ownership of the listed files that do exist in the cgroup.
268
269If the `/sys/kernel/cgroup/delegate` file does not exist, the
270runtime MUST fall back to using the following list of files:
271
272```
273cgroup.procs
274cgroup.subtree_control
275cgroup.threads
276```
277
278The runtime SHOULD NOT change the ownership of any other files.
279Changing other files may allow the container to elevate its own
280resource limits or perform other unwanted behaviour.
281
282### Example
283
284```json
285"cgroupsPath": "/myRuntime/myContainer",
286"resources": {
287 "memory": {
288 "limit": 100000,
289 "reservation": 200000
290 },
291 "devices": [
292 {
293 "allow": false,
294 "access": "rwm"
295 }
296 ]
297}
298```
299
300### <a name="configLinuxDeviceAllowedlist" />Allowed Device list
301
302**`devices`** (array of objects, OPTIONAL) configures the [allowed device list][cgroup-v1-devices].
303The runtime MUST apply entries in the listed order.
304
305Each entry has the following structure:
306
307* **`allow`** *(boolean, REQUIRED)* - whether the entry is allowed or denied.
308* **`type`** *(string, OPTIONAL)* - type of device: `a` (all), `c` (char), or `b` (block).
309 Unset values mean "all", mapping to `a`.
310* **`major, minor`** *(int64, OPTIONAL)* - [major, minor numbers][devices] for the device.
311 Unset values mean "all", mapping to [`*` in the filesystem API][cgroup-v1-devices].
312* **`access`** *(string, OPTIONAL)* - cgroup permissions for device.
313 A composition of `r` (read), `w` (write), and `m` (mknod).
314
315#### Example
316
317```json
318"devices": [
319 {
320 "allow": false,
321 "access": "rwm"
322 },
323 {
324 "allow": true,
325 "type": "c",
326 "major": 10,
327 "minor": 229,
328 "access": "rw"
329 },
330 {
331 "allow": true,
332 "type": "b",
333 "major": 8,
334 "minor": 0,
335 "access": "r"
336 }
337]
338```
339
340### <a name="configLinuxMemory" />Memory
341
342**`memory`** (object, OPTIONAL) represents the cgroup subsystem `memory` and it's used to set limits on the container's memory usage.
343For more information, see the kernel cgroups documentation about [memory][cgroup-v1-memory].
344
345Values for memory specify the limit in bytes, or `-1` for unlimited memory.
346
347* **`limit`** *(int64, OPTIONAL)* - sets limit of memory usage
348* **`reservation`** *(int64, OPTIONAL)* - sets soft limit of memory usage
349* **`swap`** *(int64, OPTIONAL)* - sets limit of memory+Swap usage
350* **`kernel`** *(int64, OPTIONAL, NOT RECOMMENDED)* - sets hard limit for kernel memory
351* **`kernelTCP`** *(int64, OPTIONAL, NOT RECOMMENDED)* - sets hard limit for kernel TCP buffer memory
352
353The following properties do not specify memory limits, but are covered by the `memory` controller:
354
355* **`swappiness`** *(uint64, OPTIONAL)* - sets swappiness parameter of vmscan (See sysctl's vm.swappiness)
356 The values are from 0 to 100. Higher means more swappy.
357* **`disableOOMKiller`** *(bool, OPTIONAL)* - enables or disables the OOM killer.
358 If enabled (`false`), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.
359 The OOM killer is enabled by default in every cgroup using the `memory` subsystem.
360 To disable it, specify a value of `true`.
361* **`useHierarchy`** *(bool, OPTIONAL)* - enables or disables hierarchical memory accounting.
362 If enabled (`true`), child cgroups will share the memory limits of this cgroup.
363* **`checkBeforeUpdate`** *(bool, OPTIONAL)* - enables container memory usage check before setting a new limit.
364 If enabled (`true`), runtime MAY check if a new memory limit is lower than the current usage, and MUST
365 reject the new limit. Practically, when cgroup v1 is used, the kernel rejects the limit lower than the
366 current usage, and when cgroup v2 is used, an OOM killer is invoked. This setting can be used on
367 cgroup v2 to mimic the cgroup v1 behavior.
368
369#### Example
370
371```json
372"memory": {
373 "limit": 536870912,
374 "reservation": 536870912,
375 "swap": 536870912,
376 "kernel": -1,
377 "kernelTCP": -1,
378 "swappiness": 0,
379 "disableOOMKiller": false
380}
381```
382
383### <a name="configLinuxCPU" />CPU
384
385**`cpu`** (object, OPTIONAL) represents the cgroup subsystems `cpu` and `cpusets`.
386For more information, see the kernel cgroups documentation about [cpusets][cgroup-v1-cpusets].
387
388The following parameters can be specified to set up the controller:
389
390* **`shares`** *(uint64, OPTIONAL)* - specifies a relative share of CPU time available to the tasks in a cgroup
391* **`quota`** *(int64, OPTIONAL)* - specifies the total amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
392 If specified with any (valid) positive value, it MUST be no smaller than `burst` (runtimes MAY generate an error).
393* **`burst`** *(uint64, OPTIONAL)* - specifies the maximum amount of accumulated time in microseconds for which all tasks in a cgroup can run additionally for burst during one period (as defined by **`period`** below)
394 If specified, this value MUST be no larger than any positive `quota` (runtimes MAY generate an error).
395* **`period`** *(uint64, OPTIONAL)* - specifies a period of time in microseconds for how regularly a cgroup's access to CPU resources should be reallocated (CFS scheduler only)
396* **`realtimeRuntime`** *(int64, OPTIONAL)* - specifies a period of time in microseconds for the longest continuous period in which the tasks in a cgroup have access to CPU resources
397* **`realtimePeriod`** *(uint64, OPTIONAL)* - same as **`period`** but applies to realtime scheduler only
398* **`cpus`** *(string, OPTIONAL)* - list of CPUs the container will run in
399* **`mems`** *(string, OPTIONAL)* - list of Memory Nodes the container will run in
400* **`idle`** *(int64, OPTIONAL)* - cgroups are configured with minimum weight, 0: default behavior, 1: SCHED_IDLE.
401
402#### Example
403
404```json
405"cpu": {
406 "shares": 1024,
407 "quota": 1000000,
408 "burst": 1000000,
409 "period": 500000,
410 "realtimeRuntime": 950000,
411 "realtimePeriod": 1000000,
412 "cpus": "2-3",
413 "mems": "0-7",
414 "idle": 0
415}
416```
417
418### <a name="configLinuxBlockIO" />Block IO
419
420**`blockIO`** (object, OPTIONAL) represents the cgroup subsystem `blkio` which implements the block IO controller.
421For more information, see the kernel cgroups documentation about [blkio][cgroup-v1-blkio] of cgroup v1 or [io][cgroup-v2-io] of cgroup v2, .
422
423Note that I/O throttling settings in cgroup v1 apply only to Direct I/O due to kernel implementation constraints, while this limitation does not exist in cgroup v2.
424
425The following parameters can be specified to set up the controller:
426
427* **`weight`** *(uint16, OPTIONAL)* - specifies per-cgroup weight. This is default weight of the group on all devices until and unless overridden by per-device rules.
428* **`leafWeight`** *(uint16, OPTIONAL)* - equivalents of `weight` for the purpose of deciding how much weight tasks in the given cgroup has while competing with the cgroup's child cgroups.
429* **`weightDevice`** *(array of objects, OPTIONAL)* - an array of per-device bandwidth weights.
430 Each entry has the following structure:
431 * **`major, minor`** *(int64, REQUIRED)* - major, minor numbers for device.
432 For more information, see the [mknod(1)][mknod.1] man page.
433 * **`weight`** *(uint16, OPTIONAL)* - bandwidth weight for the device.
434 * **`leafWeight`** *(uint16, OPTIONAL)* - bandwidth weight for the device while competing with the cgroup's child cgroups, CFQ scheduler only
435
436 You MUST specify at least one of `weight` or `leafWeight` in a given entry, and MAY specify both.
437
438* **`throttleReadBpsDevice`**, **`throttleWriteBpsDevice`** *(array of objects, OPTIONAL)* - an array of per-device bandwidth rate limits.
439 Each entry has the following structure:
440 * **`major, minor`** *(int64, REQUIRED)* - major, minor numbers for device.
441 For more information, see the [mknod(1)][mknod.1] man page.
442 * **`rate`** *(uint64, REQUIRED)* - bandwidth rate limit in bytes per second for the device
443
444* **`throttleReadIOPSDevice`**, **`throttleWriteIOPSDevice`** *(array of objects, OPTIONAL)* - an array of per-device IO rate limits.
445 Each entry has the following structure:
446 * **`major, minor`** *(int64, REQUIRED)* - major, minor numbers for device.
447 For more information, see the [mknod(1)][mknod.1] man page.
448 * **`rate`** *(uint64, REQUIRED)* - IO rate limit for the device
449
450#### Example
451
452```json
453"blockIO": {
454 "weight": 10,
455 "leafWeight": 10,
456 "weightDevice": [
457 {
458 "major": 8,
459 "minor": 0,
460 "weight": 500,
461 "leafWeight": 300
462 },
463 {
464 "major": 8,
465 "minor": 16,
466 "weight": 500
467 }
468 ],
469 "throttleReadBpsDevice": [
470 {
471 "major": 8,
472 "minor": 0,
473 "rate": 600
474 }
475 ],
476 "throttleWriteIOPSDevice": [
477 {
478 "major": 8,
479 "minor": 16,
480 "rate": 300
481 }
482 ]
483}
484```
485
486### <a name="configLinuxHugePageLimits" />Huge page limits
487
488**`hugepageLimits`** (array of objects, OPTIONAL) represents the `hugetlb` controller which allows to limit the HugeTLB reservations (if supported) or usage (page fault).
489By default if supported by the kernel, `hugepageLimits` defines the hugepage sizes and limits for HugeTLB controller
490reservation accounting, which allows to limit the HugeTLB reservations per control group and enforces the controller
491limit at reservation time and at the fault of HugeTLB memory for which no reservation exists.
492Otherwise if not supported by the kernel, this should fallback to the page fault accounting, which allows users to limit
493the HugeTLB usage (page fault) per control group and enforces the limit during page fault.
494
495Note that reservation limits are superior to page fault limits, since reservation limits are enforced at reservation
496time (on mmap or shget), and never causes the application to get SIGBUS signal if the memory was reserved before hand.
497This allows for easier fallback to alternatives such as non-HugeTLB memory for example. In the case of page fault
498accounting, it's very hard to avoid processes getting SIGBUS since the sysadmin needs precisely know the HugeTLB usage
499of all the tasks in the system and make sure there is enough pages to satisfy all requests. Avoiding tasks getting
500SIGBUS on overcommited systems is practically impossible with page fault accounting.
501
502For more information, see the kernel cgroups documentation about [HugeTLB][cgroup-v1-hugetlb].
503
504Each entry has the following structure:
505
506* **`pageSize`** *(string, REQUIRED)* - hugepage size.
507 The value has the format `<size><unit-prefix>B` (64KB, 2MB, 1GB), and must match the `<hugepagesize>` of the
508 corresponding control file found in `/sys/fs/cgroup/hugetlb/hugetlb.<hugepagesize>.rsvd.limit_in_bytes` (if
509 hugetlb_cgroup reservation is supported) or `/sys/fs/cgroup/hugetlb/hugetlb.<hugepagesize>.limit_in_bytes` (if not
510 supported).
511 Values of `<unit-prefix>` are intended to be parsed using base 1024 ("1KB" = 1024, "1MB" = 1048576, etc).
512* **`limit`** *(uint64, REQUIRED)* - limit in bytes of *hugepagesize* HugeTLB reservations (if supported) or usage.
513
514#### Example
515
516```json
517"hugepageLimits": [
518 {
519 "pageSize": "2MB",
520 "limit": 209715200
521 },
522 {
523 "pageSize": "64KB",
524 "limit": 1000000
525 }
526]
527```
528
529### <a name="configLinuxNetwork" />Network
530
531**`network`** (object, OPTIONAL) represents the cgroup subsystems `net_cls` and `net_prio`.
532For more information, see the kernel cgroups documentations about [net\_cls cgroup][cgroup-v1-net-cls] and [net\_prio cgroup][cgroup-v1-net-prio].
533
534The following parameters can be specified to set up the controller:
535
536* **`classID`** *(uint32, OPTIONAL)* - is the network class identifier the cgroup's network packets will be tagged with
537* **`priorities`** *(array of objects, OPTIONAL)* - specifies a list of objects of the priorities assigned to traffic originating from processes in the group and egressing the system on various interfaces.
538 The following parameters can be specified per-priority:
539 * **`name`** *(string, REQUIRED)* - interface name in [runtime network namespace](glossary.md#runtime-namespace)
540 * **`priority`** *(uint32, REQUIRED)* - priority applied to the interface
541
542#### Example
543
544```json
545"network": {
546 "classID": 1048577,
547 "priorities": [
548 {
549 "name": "eth0",
550 "priority": 500
551 },
552 {
553 "name": "eth1",
554 "priority": 1000
555 }
556 ]
557}
558```
559
560### <a name="configLinuxPIDS" />PIDs
561
562**`pids`** (object, OPTIONAL) represents the cgroup subsystem `pids`.
563For more information, see the kernel cgroups documentation about [pids][cgroup-v1-pids].
564
565The following parameters can be specified to set up the controller:
566
567* **`limit`** *(int64, REQUIRED)* - specifies the maximum number of tasks in the cgroup
568
569#### Example
570
571```json
572"pids": {
573 "limit": 32771
574}
575```
576
577### <a name="configLinuxRDMA" />RDMA
578
579**`rdma`** (object, OPTIONAL) represents the cgroup subsystem `rdma`.
580For more information, see the kernel cgroups documentation about [rdma][cgroup-v1-rdma].
581
582The name of the device to limit is the entry key.
583Entry values are objects with the following properties:
584
585* **`hcaHandles`** *(uint32, OPTIONAL)* - specifies the maximum number of hca_handles in the cgroup
586* **`hcaObjects`** *(uint32, OPTIONAL)* - specifies the maximum number of hca_objects in the cgroup
587
588You MUST specify at least one of the `hcaHandles` or `hcaObjects` in a given entry, and MAY specify both.
589
590#### Example
591
592```json
593"rdma": {
594 "mlx5_1": {
595 "hcaHandles": 3,
596 "hcaObjects": 10000
597 },
598 "mlx4_0": {
599 "hcaObjects": 1000
600 },
601 "rxe3": {
602 "hcaObjects": 10000
603 }
604}
605```
606
607## <a name="configLinuxUnified" />Unified
608
609**`unified`** (object, OPTIONAL) allows cgroup v2 parameters to be to be set and modified for the container.
610
611Each key in the map refers to a file in the cgroup unified hierarchy.
612
613The OCI runtime MUST ensure that the needed cgroup controllers are enabled for the cgroup.
614
615Configuration unknown to the runtime MUST still be written to the relevant file.
616
617The runtime MUST generate an error when the configuration refers to a cgroup controller that is not present or that cannot be enabled.
618
619### Example
620
621```json
622"unified": {
623 "io.max": "259:0 rbps=2097152 wiops=120\n253:0 rbps=2097152 wiops=120",
624 "hugetlb.1GB.max": "1073741824"
625}
626```
627
628If a controller is enabled on the cgroup v2 hierarchy but the configuration is provided for the cgroup v1 equivalent controller, the runtime MAY attempt a conversion.
629
630If the conversion is not possible the runtime MUST generate an error.
631
632## <a name="configLinuxIntelRdt" />IntelRdt
633
634**`intelRdt`** (object, OPTIONAL) represents the [Intel Resource Director Technology][intel-rdt-cat-kernel-interface].
635If `intelRdt` is set, the runtime MUST write the container process ID to the `tasks` file in a proper sub-directory in a mounted `resctrl` pseudo-filesystem. That sub-directory name is specified by `closID` parameter.
636If no mounted `resctrl` pseudo-filesystem is available in the [runtime mount namespace](glossary.md#runtime-namespace), the runtime MUST [generate an error](runtime.md#errors).
637
638If `intelRdt` is not set, the runtime MUST NOT manipulate any `resctrl` pseudo-filesystems.
639
640The following parameters can be specified for the container:
641
642* **`closID`** *(string, OPTIONAL)* - specifies the identity for RDT Class of Service (CLOS).
643
644* **`l3CacheSchema`** *(string, OPTIONAL)* - specifies the schema for L3 cache id and capacity bitmask (CBM).
645 The value SHOULD start with `L3:` and SHOULD NOT contain newlines.
646* **`memBwSchema`** *(string, OPTIONAL)* - specifies the schema of memory bandwidth per L3 cache id.
647 The value MUST start with `MB:` and MUST NOT contain newlines.
648
649The following rules on parameters MUST be applied:
650
651* If both `l3CacheSchema` and `memBwSchema` are set, runtimes MUST write the combined value to the `schemata` file in that sub-directory discussed in `closID`.
652
653* If `l3CacheSchema` contains a line beginning with `MB:`, the value written to `schemata` file MUST be the non-`MB:` line(s) from `l3CacheSchema` and the line from `memBWSchema`.
654
655* If either `l3CacheSchema` or `memBwSchema` is set, runtimes MUST write the value to the `schemata` file in the that sub-directory discussed in `closID`.
656
657* If neither `l3CacheSchema` nor `memBwSchema` is set, runtimes MUST NOT write to `schemata` files in any `resctrl` pseudo-filesystems.
658
659* If `closID` is not set, runtimes MUST use the container ID from [`start`](runtime.md#start) and create the `<container-id>` directory.
660
661* If `closID` is set, `l3CacheSchema` and/or `memBwSchema` is set
662 * if `closID` directory in a mounted `resctrl` pseudo-filesystem doesn't exist, the runtimes MUST create it.
663 * if `closID` directory in a mounted `resctrl` pseudo-filesystem exists, runtimes MUST compare `l3CacheSchema` and/or `memBwSchema` value with `schemata` file, and [generate an error](runtime.md#errors) if doesn't match.
664
665* If `closID` is set, and neither of `l3CacheSchema` and `memBwSchema` are set, runtime MUST check if corresponding pre-configured directory `closID` is present in mounted `resctrl`. If such pre-configured directory `closID` exists, runtime MUST assign container to this `closID` and [generate an error](runtime.md#errors) if directory does not exist.
666
667* **`enableCMT`** *(boolean, OPTIONAL)* - specifies if Intel RDT CMT should be enabled:
668 * CMT (Cache Monitoring Technology) supports monitoring of the last-level cache (LLC) occupancy
669 for the container.
670
671* **`enableMBM`** *(boolean, OPTIONAL)* - specifies if Intel RDT MBM should be enabled:
672 * MBM (Memory Bandwidth Monitoring) supports monitoring of total and local memory bandwidth
673 for the container.
674
675### Example
676
677Consider a two-socket machine with two L3 caches where the default CBM is 0x7ff and the max CBM length is 11 bits,
678and minimum memory bandwidth of 10% with a memory bandwidth granularity of 10%.
679
680Tasks inside the container only have access to the "upper" 7/11 of L3 cache on socket 0 and the "lower" 5/11 L3 cache on socket 1,
681and may use a maximum memory bandwidth of 20% on socket 0 and 70% on socket 1.
682
683```json
684"linux": {
685 "intelRdt": {
686 "closID": "guaranteed_group",
687 "l3CacheSchema": "L3:0=7f0;1=1f",
688 "memBwSchema": "MB:0=20;1=70"
689 }
690}
691```
692
693## <a name="configLinuxSysctl" />Sysctl
694
695**`sysctl`** (object, OPTIONAL) allows kernel parameters to be modified at runtime for the container.
696For more information, see the [sysctl(8)][sysctl.8] man page.
697
698### Example
699
700```json
701"sysctl": {
702 "net.ipv4.ip_forward": "1",
703 "net.core.somaxconn": "256"
704}
705```
706
707## <a name="configLinuxSeccomp" />Seccomp
708
709Seccomp provides application sandboxing mechanism in the Linux kernel.
710Seccomp configuration allows one to configure actions to take for matched syscalls and furthermore also allows matching on values passed as arguments to syscalls.
711For more information about Seccomp, see [Seccomp][seccomp] kernel documentation.
712The actions, architectures, and operators are strings that match the definitions in seccomp.h from [libseccomp][] and are translated to corresponding values.
713
714**`seccomp`** (object, OPTIONAL)
715
716The following parameters can be specified to set up seccomp:
717
718* **`defaultAction`** *(string, REQUIRED)* - the default action for seccomp. Allowed values are the same as `syscalls[].action`.
719* **`defaultErrnoRet`** *(uint, OPTIONAL)* - the errno return code to use.
720 Some actions like `SCMP_ACT_ERRNO` and `SCMP_ACT_TRACE` allow to specify the errno code to return.
721 When the action doesn't support an errno, the runtime MUST print and error and fail.
722 If not specified then its default value is `EPERM`.
723* **`architectures`** *(array of strings, OPTIONAL)* - the architecture used for system calls.
724 A valid list of constants as of libseccomp v2.5.0 is shown below.
725
726 * `SCMP_ARCH_X86`
727 * `SCMP_ARCH_X86_64`
728 * `SCMP_ARCH_X32`
729 * `SCMP_ARCH_ARM`
730 * `SCMP_ARCH_AARCH64`
731 * `SCMP_ARCH_MIPS`
732 * `SCMP_ARCH_MIPS64`
733 * `SCMP_ARCH_MIPS64N32`
734 * `SCMP_ARCH_MIPSEL`
735 * `SCMP_ARCH_MIPSEL64`
736 * `SCMP_ARCH_MIPSEL64N32`
737 * `SCMP_ARCH_PPC`
738 * `SCMP_ARCH_PPC64`
739 * `SCMP_ARCH_PPC64LE`
740 * `SCMP_ARCH_S390`
741 * `SCMP_ARCH_S390X`
742 * `SCMP_ARCH_PARISC`
743 * `SCMP_ARCH_PARISC64`
744 * `SCMP_ARCH_RISCV64`
745
746* **`flags`** *(array of strings, OPTIONAL)* - list of flags to use with seccomp(2).
747
748 A valid list of constants is shown below.
749
750 * `SECCOMP_FILTER_FLAG_TSYNC`
751 * `SECCOMP_FILTER_FLAG_LOG`
752 * `SECCOMP_FILTER_FLAG_SPEC_ALLOW`
753 * `SECCOMP_FILTER_FLAG_WAIT_KILLABLE_RECV`
754
755* **`listenerPath`** *(string, OPTIONAL)* - specifies the path of UNIX domain socket over which the runtime will send the [container process state](#containerprocessstate) data structure when the `SCMP_ACT_NOTIFY` action is used.
756 This socket MUST use `AF_UNIX` domain and `SOCK_STREAM` type.
757 The runtime MUST send exactly one [container process state](#containerprocessstate) per connection.
758 The connection MUST NOT be reused and it MUST be closed after sending a seccomp state.
759 If sending to this socket fails, the runtime MUST [generate an error](runtime.md#errors).
760 If the `SCMP_ACT_NOTIFY` action is not used this value is ignored.
761
762 The runtime sends the following file descriptors using `SCM_RIGHTS` and set their names in the `fds` array of the [container process state](#containerprocessstate):
763
764 * **`seccompFd`** (string, REQUIRED) is the seccomp file descriptor returned by the seccomp syscall.
765
766* **`listenerMetadata`** *(string, OPTIONAL)* - specifies an opaque data to pass to the seccomp agent.
767 This string will be sent as the `metadata` field in the [container process state](#containerprocessstate).
768 This field MUST NOT be set if `listenerPath` is not set.
769
770* **`syscalls`** *(array of objects, OPTIONAL)* - match a syscall in seccomp.
771 While this property is OPTIONAL, some values of `defaultAction` are not useful without `syscalls` entries.
772 For example, if `defaultAction` is `SCMP_ACT_KILL` and `syscalls` is empty or unset, the kernel will kill the container process on its first syscall.
773 Each entry has the following structure:
774
775 * **`names`** *(array of strings, REQUIRED)* - the names of the syscalls.
776 `names` MUST contain at least one entry.
777 * **`action`** *(string, REQUIRED)* - the action for seccomp rules.
778 A valid list of constants as of libseccomp v2.5.0 is shown below.
779
780 * `SCMP_ACT_KILL`
781 * `SCMP_ACT_KILL_PROCESS`
782 * `SCMP_ACT_KILL_THREAD`
783 * `SCMP_ACT_TRAP`
784 * `SCMP_ACT_ERRNO`
785 * `SCMP_ACT_TRACE`
786 * `SCMP_ACT_ALLOW`
787 * `SCMP_ACT_LOG`
788 * `SCMP_ACT_NOTIFY`
789
790 * **`errnoRet`** *(uint, OPTIONAL)* - the errno return code to use.
791 Some actions like `SCMP_ACT_ERRNO` and `SCMP_ACT_TRACE` allow to specify the errno code to return.
792 When the action doesn't support an errno, the runtime MUST print and error and fail.
793 If not specified its default value is `EPERM`.
794
795 * **`args`** *(array of objects, OPTIONAL)* - the specific syscall in seccomp.
796 Each entry has the following structure:
797
798 * **`index`** *(uint, REQUIRED)* - the index for syscall arguments in seccomp.
799 * **`value`** *(uint64, REQUIRED)* - the value for syscall arguments in seccomp.
800 * **`valueTwo`** *(uint64, OPTIONAL)* - the value for syscall arguments in seccomp.
801 * **`op`** *(string, REQUIRED)* - the operator for syscall arguments in seccomp.
802 A valid list of constants as of libseccomp v2.3.2 is shown below.
803
804 * `SCMP_CMP_NE`
805 * `SCMP_CMP_LT`
806 * `SCMP_CMP_LE`
807 * `SCMP_CMP_EQ`
808 * `SCMP_CMP_GE`
809 * `SCMP_CMP_GT`
810 * `SCMP_CMP_MASKED_EQ`
811
812### Example
813
814```json
815"seccomp": {
816 "defaultAction": "SCMP_ACT_ALLOW",
817 "architectures": [
818 "SCMP_ARCH_X86",
819 "SCMP_ARCH_X32"
820 ],
821 "syscalls": [
822 {
823 "names": [
824 "getcwd",
825 "chmod"
826 ],
827 "action": "SCMP_ACT_ERRNO"
828 }
829 ]
830}
831```
832
833### <a name="containerprocessstate" />The Container Process State
834
835The container process state is a data structure passed via a UNIX socket.
836The container runtime MUST send the container process state over the UNIX socket as regular payload serialized in JSON and file descriptors MUST be sent using `SCM_RIGHTS`.
837The container runtime MAY use several `sendmsg(2)` calls to send the aforementioned data.
838If more than one `sendmsg(2)` is used, the file descriptors MUST be sent only in the first call.
839
840The container process state includes the following properties:
841
842* **`ociVersion`** (string, REQUIRED) is version of the Open Container Initiative Runtime Specification with which the container process state complies.
843* **`fds`** (array, OPTIONAL) is a string array containing the names of the file descriptors passed.
844 The index of the name in this array corresponds to index of the file descriptors in the `SCM_RIGHTS` array.
845* **`pid`** (int, REQUIRED) is the container process ID, as seen by the runtime.
846* **`metadata`** (string, OPTIONAL) opaque metadata.
847* **`state`** ([state](runtime.md#state), REQUIRED) is the state of the container.
848
849Example sending a single `seccompFd` file descriptor in the `SCM_RIGHTS` array:
850
851```json
852{
853 "ociVersion": "1.0.2",
854 "fds": [
855 "seccompFd"
856 ],
857 "pid": 4422,
858 "metadata": "MKNOD=/dev/null,/dev/net/tun;BPF_MAP_TYPES=hash,array",
859 "state": {
860 "ociVersion": "1.0.2",
861 "id": "oci-container1",
862 "status": "creating",
863 "pid": 4422,
864 "bundle": "/containers/redis",
865 "annotations": {
866 "myKey": "myValue"
867 }
868 }
869}
870```
871
872## <a name="configLinuxRootfsMountPropagation" />Rootfs Mount Propagation
873
874**`rootfsPropagation`** (string, OPTIONAL) sets the rootfs's mount propagation.
875Its value is either `shared`, `slave`, `private` or `unbindable`.
876It's worth noting that a peer group is defined as a group of VFS mounts that propagate events to each other.
877A nested container is defined as a container launched inside an existing container.
878
879* **`shared`**: the rootfs mount belongs to a new peer group.
880 This means that further mounts (e.g. nested containers) will also belong to that peer group and will propagate events to the rootfs.
881 Note this does not mean that it's shared with the host.
882* **`slave`**: the rootfs mount receives propagation events from the host (e.g. if something is mounted on the host it will also appear in the container) but not the other way around.
883* **`private`**: the rootfs mount doesn't receive mount propagation events from the host and further mounts in nested containers will be isolated from the host and from the rootfs (even if the nested container `rootfsPropagation` option is shared).
884* **`unbindable`**: the rootfs mount is a private mount that cannot be bind-mounted.
885
886The [Shared Subtrees][sharedsubtree] article in the kernel documentation has more information about mount propagation.
887
888### Example
889
890```json
891"rootfsPropagation": "slave",
892```
893
894## <a name="configLinuxMaskedPaths" />Masked Paths
895
896**`maskedPaths`** (array of strings, OPTIONAL) will mask over the provided paths inside the container so that they cannot be read.
897The values MUST be absolute paths in the [container namespace](glossary.md#container_namespace).
898
899### Example
900
901```json
902"maskedPaths": [
903 "/proc/kcore"
904]
905```
906
907## <a name="configLinuxReadonlyPaths" />Readonly Paths
908
909**`readonlyPaths`** (array of strings, OPTIONAL) will set the provided paths as readonly inside the container.
910The values MUST be absolute paths in the [container namespace](glossary.md#container-namespace).
911
912### Example
913
914```json
915"readonlyPaths": [
916 "/proc/sys"
917]
918```
919
920## <a name="configLinuxMountLabel" />Mount Label
921
922**`mountLabel`** (string, OPTIONAL) will set the Selinux context for the mounts in the container.
923
924### Example
925
926```json
927"mountLabel": "system_u:object_r:svirt_sandbox_file_t:s0:c715,c811"
928```
929
930## <a name="configLinuxPersonality" />Personality
931
932**`personality`** (object, OPTIONAL) sets the Linux execution personality. For more information
933see the [personality][personality.2] syscall documentation. As most of the options are
934obsolete and rarely used, and some reduce security, the currently supported set is a small
935subset of the available options.
936
937* **`domain`** *(string, REQUIRED)* - the execution domain.
938 The valid list of constants is shown below. `LINUX32` will set the `uname` system call to show
939 a 32 bit CPU type, such as `i686`.
940
941 * `LINUX`
942 * `LINUX32`
943
944* **`flags`** *(array of strings, OPTIONAL)* - the additional flags to apply.
945 Currently no flag values are supported.
946
947
948[cgroup-v1]: https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt
949[cgroup-v1-blkio]: https://www.kernel.org/doc/Documentation/cgroup-v1/blkio-controller.txt
950[cgroup-v1-cpusets]: https://www.kernel.org/doc/Documentation/cgroup-v1/cpusets.txt
951[cgroup-v1-devices]: https://www.kernel.org/doc/Documentation/cgroup-v1/devices.txt
952[cgroup-v1-hugetlb]: https://www.kernel.org/doc/Documentation/cgroup-v1/hugetlb.txt
953[cgroup-v1-memory]: https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt
954[cgroup-v1-net-cls]: https://www.kernel.org/doc/Documentation/cgroup-v1/net_cls.txt
955[cgroup-v1-net-prio]: https://www.kernel.org/doc/Documentation/cgroup-v1/net_prio.txt
956[cgroup-v1-pids]: https://www.kernel.org/doc/Documentation/cgroup-v1/pids.txt
957[cgroup-v1-rdma]: https://www.kernel.org/doc/Documentation/cgroup-v1/rdma.txt
958[cgroup-v2]: https://www.kernel.org/doc/Documentation/cgroup-v2.txt
959[cgroup-v2-io]: https://docs.kernel.org/admin-guide/cgroup-v2.html#io
960[devices]: https://www.kernel.org/doc/Documentation/admin-guide/devices.txt
961[devpts]: https://www.kernel.org/doc/Documentation/filesystems/devpts.txt
962[file]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_164
963[libseccomp]: https://github.com/seccomp/libseccomp
964[proc]: https://www.kernel.org/doc/Documentation/filesystems/proc.txt
965[seccomp]: https://www.kernel.org/doc/Documentation/prctl/seccomp_filter.txt
966[sharedsubtree]: https://www.kernel.org/doc/Documentation/filesystems/sharedsubtree.txt
967[sysfs]: https://www.kernel.org/doc/Documentation/filesystems/sysfs.txt
968[tmpfs]: https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt
969
970[full.4]: http://man7.org/linux/man-pages/man4/full.4.html
971[mknod.1]: http://man7.org/linux/man-pages/man1/mknod.1.html
972[mknod.2]: http://man7.org/linux/man-pages/man2/mknod.2.html
973[namespaces.7_2]: http://man7.org/linux/man-pages/man7/namespaces.7.html
974[null.4]: http://man7.org/linux/man-pages/man4/null.4.html
975[personality.2]: http://man7.org/linux/man-pages/man2/personality.2.html
976[pts.4]: http://man7.org/linux/man-pages/man4/pts.4.html
977[random.4]: http://man7.org/linux/man-pages/man4/random.4.html
978[sysctl.8]: http://man7.org/linux/man-pages/man8/sysctl.8.html
979[tty.4]: http://man7.org/linux/man-pages/man4/tty.4.html
980[zero.4]: http://man7.org/linux/man-pages/man4/zero.4.html
981[user-namespaces]: http://man7.org/linux/man-pages/man7/user_namespaces.7.html
982[intel-rdt-cat-kernel-interface]: https://www.kernel.org/doc/Documentation/x86/intel_rdt_ui.txt
983[time_namespaces.7]: https://man7.org/linux/man-pages/man7/time_namespaces.7.html
View as plain text