1# `tarball`
2
3[](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/tarball)
4
5This package produces tarballs that can consumed via `docker load`. Note
6that this is a _different_ format from the [`legacy`](/pkg/legacy/tarball)
7tarballs that are produced by `docker save`, but this package is still able to
8read the legacy tarballs produced by `docker save`.
9
10## Usage
11
12```go
13package main
14
15import (
16 "os"
17
18 "github.com/google/go-containerregistry/pkg/name"
19 "github.com/google/go-containerregistry/pkg/v1/tarball"
20)
21
22func main() {
23 // Read a tarball from os.Args[1] that contains ubuntu.
24 tag, err := name.NewTag("ubuntu")
25 if err != nil {
26 panic(err)
27 }
28 img, err := tarball.ImageFromPath(os.Args[1], &tag)
29 if err != nil {
30 panic(err)
31 }
32
33 // Write that tarball to os.Args[2] with a different tag.
34 newTag, err := name.NewTag("ubuntu:newest")
35 if err != nil {
36 panic(err)
37 }
38 f, err := os.Create(os.Args[2])
39 if err != nil {
40 panic(err)
41 }
42 defer f.Close()
43
44 if err := tarball.Write(newTag, img, f); err != nil {
45 panic(err)
46 }
47}
48```
49
50## Structure
51
52<p align="center">
53 <img src="/images/tarball.dot.svg" />
54</p>
55
56Let's look at what happens when we write out a tarball:
57
58
59### `ubuntu:latest`
60
61```
62$ crane pull ubuntu ubuntu.tar && mkdir ubuntu && tar xf ubuntu.tar -C ubuntu && rm ubuntu.tar
63$ tree ubuntu/
64ubuntu/
65├── 423ae2b273f4c17ceee9e8482fa8d071d90c7d052ae208e1fe4963fceb3d6954.tar.gz
66├── b6b53be908de2c0c78070fff0a9f04835211b3156c4e73785747af365e71a0d7.tar.gz
67├── de83a2304fa1f7c4a13708a0d15b9704f5945c2be5cbb2b3ed9b2ccb718d0b3d.tar.gz
68├── f9a83bce3af0648efaa60b9bb28225b09136d2d35d0bed25ac764297076dec1b.tar.gz
69├── manifest.json
70└── sha256:72300a873c2ca11c70d0c8642177ce76ff69ae04d61a5813ef58d40ff66e3e7c
71
720 directories, 6 files
73```
74
75There are a couple interesting files here.
76
77`manifest.json` is the entrypoint: a list of [`tarball.Descriptor`s](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/tarball#Descriptor)
78that describe the images contained in this tarball.
79
80For each image, this has the `RepoTags` (how it was pulled), a `Config` file
81that points to the image's config file, a list of `Layers`, and (optionally)
82`LayerSources`.
83
84```
85$ jq < ubuntu/manifest.json
86[
87 {
88 "Config": "sha256:72300a873c2ca11c70d0c8642177ce76ff69ae04d61a5813ef58d40ff66e3e7c",
89 "RepoTags": [
90 "ubuntu"
91 ],
92 "Layers": [
93 "423ae2b273f4c17ceee9e8482fa8d071d90c7d052ae208e1fe4963fceb3d6954.tar.gz",
94 "de83a2304fa1f7c4a13708a0d15b9704f5945c2be5cbb2b3ed9b2ccb718d0b3d.tar.gz",
95 "f9a83bce3af0648efaa60b9bb28225b09136d2d35d0bed25ac764297076dec1b.tar.gz",
96 "b6b53be908de2c0c78070fff0a9f04835211b3156c4e73785747af365e71a0d7.tar.gz"
97 ]
98 }
99]
100```
101
102The config file and layers are exactly what you would expect, and match the
103registry representations of the same artifacts. You'll notice that the
104`manifest.json` contains similar information as the registry manifest, but isn't
105quite the same:
106
107```
108$ crane manifest ubuntu@sha256:0925d086715714114c1988f7c947db94064fd385e171a63c07730f1fa014e6f9
109{
110 "schemaVersion": 2,
111 "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
112 "config": {
113 "mediaType": "application/vnd.docker.container.image.v1+json",
114 "size": 3408,
115 "digest": "sha256:72300a873c2ca11c70d0c8642177ce76ff69ae04d61a5813ef58d40ff66e3e7c"
116 },
117 "layers": [
118 {
119 "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
120 "size": 26692096,
121 "digest": "sha256:423ae2b273f4c17ceee9e8482fa8d071d90c7d052ae208e1fe4963fceb3d6954"
122 },
123 {
124 "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
125 "size": 35365,
126 "digest": "sha256:de83a2304fa1f7c4a13708a0d15b9704f5945c2be5cbb2b3ed9b2ccb718d0b3d"
127 },
128 {
129 "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
130 "size": 852,
131 "digest": "sha256:f9a83bce3af0648efaa60b9bb28225b09136d2d35d0bed25ac764297076dec1b"
132 },
133 {
134 "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
135 "size": 163,
136 "digest": "sha256:b6b53be908de2c0c78070fff0a9f04835211b3156c4e73785747af365e71a0d7"
137 }
138 ]
139}
140```
141
142This makes it difficult to maintain image digests when roundtripping images
143through the tarball format, so it's not a great format if you care about
144provenance.
145
146The ubuntu example didn't have any `LayerSources` -- let's look at another image
147that does.
148
149### `hello-world:nanoserver`
150
151```
152$ crane pull hello-world:nanoserver@sha256:63c287625c2b0b72900e562de73c0e381472a83b1b39217aef3856cd398eca0b nanoserver.tar
153$ mkdir nanoserver && tar xf nanoserver.tar -C nanoserver && rm nanoserver.tar
154$ tree nanoserver/
155nanoserver/
156├── 10d1439be4eb8819987ec2e9c140d44d74d6b42a823d57fe1953bd99948e1bc0.tar.gz
157├── a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053.tar.gz
158├── be21f08f670160cbae227e3053205b91d6bfa3de750b90c7e00bd2c511ccb63a.tar.gz
159├── manifest.json
160└── sha256:bc5d255ea81f83c8c38a982a6d29a6f2198427d258aea5f166e49856896b2da6
161
1620 directories, 5 files
163
164$ jq < nanoserver/manifest.json
165[
166 {
167 "Config": "sha256:bc5d255ea81f83c8c38a982a6d29a6f2198427d258aea5f166e49856896b2da6",
168 "RepoTags": [
169 "index.docker.io/library/hello-world:i-was-a-digest"
170 ],
171 "Layers": [
172 "a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053.tar.gz",
173 "be21f08f670160cbae227e3053205b91d6bfa3de750b90c7e00bd2c511ccb63a.tar.gz",
174 "10d1439be4eb8819987ec2e9c140d44d74d6b42a823d57fe1953bd99948e1bc0.tar.gz"
175 ],
176 "LayerSources": {
177 "sha256:26fd2d9d4c64a4f965bbc77939a454a31b607470f430b5d69fc21ded301fa55e": {
178 "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
179 "size": 101145811,
180 "digest": "sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053",
181 "urls": [
182 "https://mcr.microsoft.com/v2/windows/nanoserver/blobs/sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053"
183 ]
184 }
185 }
186 }
187]
188```
189
190A couple things to note about this `manifest.json` versus the other:
191* The `RepoTags` field is a bit weird here. `hello-world` is a multi-platform
192 image, so We had to pull this image by digest, since we're (I'm) on
193 amd64/linux and wanted to grab a windows image. Since the tarball format
194 expects a tag under `RepoTags`, and we didn't pull by tag, we replace the
195 digest with a sentinel `i-was-a-digest` "tag" to appease docker.
196* The `LayerSources` has enough information to reconstruct the foreign layers
197 pointer when pushing/pulling from the registry. For legal reasons, microsoft
198 doesn't want anyone but them to serve windows base images, so the mediaType
199 here indicates a "foreign" or "non-distributable" layer with an URL for where
200 you can download it from microsoft (see the [OCI
201 image-spec](https://github.com/opencontainers/image-spec/blob/master/layer.md#non-distributable-layers)).
202
203We can look at what's in the registry to explain both of these things:
204```
205$ crane manifest hello-world:nanoserver | jq .
206{
207 "manifests": [
208 {
209 "digest": "sha256:63c287625c2b0b72900e562de73c0e381472a83b1b39217aef3856cd398eca0b",
210 "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
211 "platform": {
212 "architecture": "amd64",
213 "os": "windows",
214 "os.version": "10.0.17763.1040"
215 },
216 "size": 1124
217 }
218 ],
219 "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
220 "schemaVersion": 2
221}
222
223
224# Note the media type and "urls" field.
225$ crane manifest hello-world:nanoserver@sha256:63c287625c2b0b72900e562de73c0e381472a83b1b39217aef3856cd398eca0b | jq .
226{
227 "schemaVersion": 2,
228 "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
229 "config": {
230 "mediaType": "application/vnd.docker.container.image.v1+json",
231 "size": 1721,
232 "digest": "sha256:bc5d255ea81f83c8c38a982a6d29a6f2198427d258aea5f166e49856896b2da6"
233 },
234 "layers": [
235 {
236 "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
237 "size": 101145811,
238 "digest": "sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053",
239 "urls": [
240 "https://mcr.microsoft.com/v2/windows/nanoserver/blobs/sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053"
241 ]
242 },
243 {
244 "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
245 "size": 1669,
246 "digest": "sha256:be21f08f670160cbae227e3053205b91d6bfa3de750b90c7e00bd2c511ccb63a"
247 },
248 {
249 "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
250 "size": 949,
251 "digest": "sha256:10d1439be4eb8819987ec2e9c140d44d74d6b42a823d57fe1953bd99948e1bc0"
252 }
253 ]
254}
255```
256
257The `LayerSources` map is keyed by the diffid. Note that `sha256:26fd2d9d4c64a4f965bbc77939a454a31b607470f430b5d69fc21ded301fa55e` matches the first layer in the config file:
258```
259$ jq '.[0].LayerSources' < nanoserver/manifest.json
260{
261 "sha256:26fd2d9d4c64a4f965bbc77939a454a31b607470f430b5d69fc21ded301fa55e": {
262 "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
263 "size": 101145811,
264 "digest": "sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053",
265 "urls": [
266 "https://mcr.microsoft.com/v2/windows/nanoserver/blobs/sha256:a35da61c356213336e646756218539950461ff2bf096badf307a23add6e70053"
267 ]
268 }
269}
270
271$ jq < nanoserver/sha256\:bc5d255ea81f83c8c38a982a6d29a6f2198427d258aea5f166e49856896b2da6 | jq .rootfs
272{
273 "type": "layers",
274 "diff_ids": [
275 "sha256:26fd2d9d4c64a4f965bbc77939a454a31b607470f430b5d69fc21ded301fa55e",
276 "sha256:601cf7d78c62e4b4d32a7bbf96a17606a9cea5bd9d22ffa6f34aa431d056b0e8",
277 "sha256:a1e1a3bf6529adcce4d91dce2cad86c2604a66b507ccbc4d2239f3da0ec5aab9"
278 ]
279}
280```
View as plain text