1# go-tuf
2
3[](https://github.com/theupdateframework/go-tuf/actions?query=workflow%3Abuild) [](https://coveralls.io/github/theupdateframework/go-tuf) [](https://pkg.go.dev/github.com/theupdateframework/go-tuf) [](https://goreportcard.com/report/github.com/theupdateframework/go-tuf)
4
5This is a Go implementation of [The Update Framework (TUF)](http://theupdateframework.com/),
6a framework for securing software update systems.
7
8## Directory layout
9
10A TUF repository has the following directory layout:
11
12```bash
13.
14├── keys
15├── repository
16│ └── targets
17└── staged
18 └── targets
19```
20
21The directories contain the following files:
22
23- `keys/` - signing keys (optionally encrypted) with filename pattern `ROLE.json`
24- `repository/` - signed metadata files
25- `repository/targets/` - hashed target files
26- `staged/` - either signed, unsigned or partially signed metadata files
27- `staged/targets/` - unhashed target files
28
29## CLI
30
31`go-tuf` provides a CLI for managing a local TUF repository.
32
33### Install
34
35`go-tuf` is tested on Go versions 1.18.
36
37```bash
38go install github.com/theupdateframework/go-tuf/cmd/tuf@latest
39```
40
41### Commands
42
43#### `tuf init [--consistent-snapshot=false]`
44
45Initializes a new repository.
46
47This is only required if the repository should not generate consistent
48snapshots (i.e. by passing `--consistent-snapshot=false`). If consistent
49snapshots should be generated, the repository will be implicitly
50initialized to do so when generating keys.
51
52#### `tuf add-key [--scheme=<scheme>] [--expires=<days>] [--public-key=<path>] <role>`
53
54Adds a new signing key for the given role.
55
56The root metadata file will be staged
57with the addition of the key's ID to the role's list of key IDs.
58
59The public value can be specified as a path or passed in via stdin.
60
61#### `tuf gen-key [--expires=<days>] <role>`
62
63Prompts the user for an encryption passphrase (unless the
64`--insecure-plaintext` flag is set), then generates a new signing key and
65writes it to the relevant key file in the `keys` directory. It also stages
66the addition of the new key to the `root` metadata file. Alternatively, passphrases
67can be set via environment variables in the form of `TUF_{{ROLE}}_PASSPHRASE`
68
69#### `tuf revoke-key [--expires=<days>] <role> <id>`
70
71Revoke a signing key
72
73The key will be removed from the root metadata file, but the key will remain in the
74"keys" directory if present.
75
76#### `tuf add [<path>...]`
77
78Hashes files in the `staged/targets` directory at the given path(s), then
79updates and stages the `targets` metadata file. Specifying no paths hashes all
80files in the `staged/targets` directory.
81
82#### `tuf remove [<path>...]`
83
84Stages the removal of files with the given path(s) from the `targets` metadata file
85(they get removed from the filesystem when the change is committed). Specifying
86no paths removes all files from the `targets` metadata file.
87
88#### `tuf snapshot [--expires=<days>]`
89
90Expects a staged, fully signed `targets` metadata file and stages an appropriate
91`snapshot` metadata file. Optionally one can set number of days after which
92the `snapshot` metadata will expire.
93
94#### `tuf timestamp [--expires=<days>]`
95
96Stages an appropriate `timestamp` metadata file. If a `snapshot` metadata file is staged,
97it must be fully signed. Optionally one can set number of days after which
98the timestamp metadata will expire.
99
100#### `tuf sign <metadata>`
101
102Signs the given role's staged metadata file with all keys present in the `keys`
103directory for that role.
104
105#### `tuf commit`
106
107Verifies that all staged changes contain the correct information and are signed
108to the correct threshold, then moves the staged files into the `repository`
109directory. It also removes any target files which are not in the `targets`
110metadata file.
111
112#### `tuf regenerate [--consistent-snapshot=false]`
113
114Note: Not supported yet
115
116Recreates the `targets` metadata file based on the files in `repository/targets`.
117
118#### `tuf clean`
119
120Removes all staged metadata files and targets.
121
122#### `tuf root-keys`
123
124Outputs a JSON serialized array of root keys to STDOUT. The resulting JSON
125should be distributed to clients for performing initial updates.
126
127#### `tuf set-threshold <role> <threshold>`
128
129Sets `role`'s threshold (required number of keys for signing) to
130`threshold`.
131
132#### `tuf get-threshold <role>`
133
134Outputs `role`'s threshold (required number of keys for signing).
135
136#### `tuf change-passphrase <role>`
137
138Changes the passphrase for given role keys file. The CLI supports reading
139both the existing and the new passphrase via the following environment
140variables - `TUF_{{ROLE}}_PASSPHRASE` and respectively `TUF_NEW_{{ROLE}}_PASSPHRASE`
141
142#### `tuf payload <metadata>`
143
144Outputs the metadata file for a role in a ready-to-sign (canonicalized) format.
145
146See also `tuf sign-payload` and `tuf add-signatures`.
147
148#### `tuf sign-payload --role=<role> <path>`
149
150Sign a file (outside of the TUF repo) using keys (in the TUF keys database,
151typically produced by `tuf gen-key`) for the given `role` (from the TUF repo).
152
153Typically, `path` will be a file containing the output of `tuf payload`.
154
155See also `tuf add-signatures`.
156
157#### `tuf add-signatures [--signatures <sig_file>] [--format=<format>] [--key-id=<key-id>] <metadata>`
158
159Adds signatures (the output of `tuf sign-payload`) to the given role metadata file.
160
161If the signature does not verify, it will not be added. Signature can be a json file
162or json passed in via `stdin`.
163
164#### `tuf status --valid-at <date> <role>`
165
166Check if the role's metadata will be expired on the given date.
167
168#### Usage of environment variables
169
170The `tuf` CLI supports receiving passphrases via environment variables in
171the form of `TUF_{{ROLE}}_PASSPHRASE` for existing ones and
172`TUF_NEW_{{ROLE}}_PASSPHRASE` for setting new ones.
173
174For a list of supported commands, run `tuf help` from the command line.
175
176### Examples
177
178The following are example workflows for managing a TUF repository with the CLI.
179
180The `tree` commands do not need to be run, but their output serve as an
181illustration of what files should exist after performing certain commands.
182
183Although only two machines are referenced (i.e. the "root" and "repo" boxes),
184the workflows can be trivially extended to many signing machines by copying
185staged changes and signing on each machine in turn before finally committing.
186
187Some key IDs are truncated for illustrative purposes.
188
189#### Create signed root metadata file
190
191Generate a root key on the root box:
192
193```bash
194$ tuf gen-key root
195Enter root keys passphrase:
196Repeat root keys passphrase:
197Generated root key with ID 184b133f
198
199$ tree .
200.
201├── keys
202│ └── root.json
203├── repository
204└── staged
205 ├── root.json
206 └── targets
207```
208
209Copy `staged/root.json` from the root box to the repo box and generate targets,
210snapshot and timestamp keys:
211
212```bash
213$ tree .
214.
215├── keys
216├── repository
217└── staged
218 ├── root.json
219 └── targets
220
221$ tuf gen-key targets
222Enter targets keys passphrase:
223Repeat targets keys passphrase:
224Generated targets key with ID 8cf4810c
225
226$ tuf gen-key snapshot
227Enter snapshot keys passphrase:
228Repeat snapshot keys passphrase:
229Generated snapshot key with ID 3e070e53
230
231$ tuf gen-key timestamp
232Enter timestamp keys passphrase:
233Repeat timestamp keys passphrase:
234Generated timestamp key with ID a3768063
235
236$ tree .
237.
238├── keys
239│ ├── snapshot.json
240│ ├── targets.json
241│ └── timestamp.json
242├── repository
243└── staged
244 ├── root.json
245 └── targets
246```
247
248Copy `staged/root.json` from the repo box back to the root box and sign it:
249
250```bash
251$ tree .
252.
253├── keys
254│ ├── root.json
255├── repository
256└── staged
257 ├── root.json
258 └── targets
259
260$ tuf sign root.json
261Enter root keys passphrase:
262```
263
264The staged `root.json` can now be copied back to the repo box ready to be
265committed alongside other metadata files.
266
267#### Alternate signing flow
268
269Instead of manually copying `root.json` into the TUF repository on the root box,
270you can use the `tuf payload`, `tuf sign-payload`, `tuf add-signatures` flow.
271
272On the repo box, get the `root.json` payload in a canonical format:
273
274``` bash
275$ tuf payload root.json > root.json.payload
276```
277
278Copy `root.json.payload` to the root box and sign it:
279
280
281``` bash
282$ tuf sign-payload --role=root root.json.payload > root.json.sigs
283Enter root keys passphrase:
284```
285
286Copy `root.json.sigs` back to the repo box and import the signatures:
287
288``` bash
289$ tuf add-signatures --signatures root.json.sigs root.json
290```
291
292This achieves the same state as the above flow for the repo box:
293
294```bash
295$ tree .
296.
297├── keys
298│ ├── snapshot.json
299│ ├── targets.json
300│ └── timestamp.json
301├── repository
302└── staged
303 ├── root.json
304 └── targets
305```
306
307#### Add a target file
308
309Assuming a staged, signed `root` metadata file and the file to add exists at
310`staged/targets/foo/bar/baz.txt`:
311
312```bash
313$ tree .
314.
315├── keys
316│ ├── snapshot.json
317│ ├── targets.json
318│ └── timestamp.json
319├── repository
320└── staged
321 ├── root.json
322 └── targets
323 └── foo
324 └── bar
325 └── baz.txt
326
327$ tuf add foo/bar/baz.txt
328Enter targets keys passphrase:
329
330$ tree .
331.
332├── keys
333│ ├── snapshot.json
334│ ├── targets.json
335│ └── timestamp.json
336├── repository
337└── staged
338 ├── root.json
339 ├── targets
340 │ └── foo
341 │ └── bar
342 │ └── baz.txt
343 └── targets.json
344
345$ tuf snapshot
346Enter snapshot keys passphrase:
347
348$ tuf timestamp
349Enter timestamp keys passphrase:
350
351$ tree .
352.
353├── keys
354│ ├── snapshot.json
355│ ├── targets.json
356│ └── timestamp.json
357├── repository
358└── staged
359 ├── root.json
360 ├── snapshot.json
361 ├── targets
362 │ └── foo
363 │ └── bar
364 │ └── baz.txt
365 ├── targets.json
366 └── timestamp.json
367
368$ tuf commit
369
370$ tree .
371.
372├── keys
373│ ├── snapshot.json
374│ ├── targets.json
375│ └── timestamp.json
376├── repository
377│ ├── root.json
378│ ├── snapshot.json
379│ ├── targets
380│ │ └── foo
381│ │ └── bar
382│ │ └── baz.txt
383│ ├── targets.json
384│ └── timestamp.json
385└── staged
386```
387
388#### Remove a target file
389
390Assuming the file to remove is at `repository/targets/foo/bar/baz.txt`:
391
392```bash
393$ tree .
394.
395├── keys
396│ ├── snapshot.json
397│ ├── targets.json
398│ └── timestamp.json
399├── repository
400│ ├── root.json
401│ ├── snapshot.json
402│ ├── targets
403│ │ └── foo
404│ │ └── bar
405│ │ └── baz.txt
406│ ├── targets.json
407│ └── timestamp.json
408└── staged
409
410$ tuf remove foo/bar/baz.txt
411Enter targets keys passphrase:
412
413$ tree .
414.
415├── keys
416│ ├── snapshot.json
417│ ├── targets.json
418│ └── timestamp.json
419├── repository
420│ ├── root.json
421│ ├── snapshot.json
422│ ├── targets
423│ │ └── foo
424│ │ └── bar
425│ │ └── baz.txt
426│ ├── targets.json
427│ └── timestamp.json
428└── staged
429 └── targets.json
430
431$ tuf snapshot
432Enter snapshot keys passphrase:
433
434$ tuf timestamp
435Enter timestamp keys passphrase:
436
437$ tree .
438.
439├── keys
440│ ├── snapshot.json
441│ ├── targets.json
442│ └── timestamp.json
443├── repository
444│ ├── root.json
445│ ├── snapshot.json
446│ ├── targets
447│ │ └── foo
448│ │ └── bar
449│ │ └── baz.txt
450│ ├── targets.json
451│ └── timestamp.json
452└── staged
453 ├── snapshot.json
454 ├── targets.json
455 └── timestamp.json
456
457$ tuf commit
458
459$ tree .
460.
461├── keys
462│ ├── snapshot.json
463│ ├── targets.json
464│ └── timestamp.json
465├── repository
466│ ├── root.json
467│ ├── snapshot.json
468│ ├── targets.json
469│ └── timestamp.json
470└── staged
471```
472
473#### Regenerate metadata files based on targets tree (Note: Not supported yet)
474
475```bash
476$ tree .
477.
478├── keys
479│ ├── snapshot.json
480│ ├── targets.json
481│ └── timestamp.json
482├── repository
483│ ├── root.json
484│ ├── snapshot.json
485│ ├── targets
486│ │ └── foo
487│ │ └── bar
488│ │ └── baz.txt
489│ ├── targets.json
490│ └── timestamp.json
491└── staged
492
493$ tuf regenerate
494Enter targets keys passphrase:
495
496$ tree .
497.
498├── keys
499│ ├── snapshot.json
500│ ├── targets.json
501│ └── timestamp.json
502├── repository
503│ ├── root.json
504│ ├── snapshot.json
505│ ├── targets
506│ │ └── foo
507│ │ └── bar
508│ │ └── baz.txt
509│ ├── targets.json
510│ └── timestamp.json
511└── staged
512 └── targets.json
513
514$ tuf snapshot
515Enter snapshot keys passphrase:
516
517$ tuf timestamp
518Enter timestamp keys passphrase:
519
520$ tree .
521.
522├── keys
523│ ├── snapshot.json
524│ ├── targets.json
525│ └── timestamp.json
526├── repository
527│ ├── root.json
528│ ├── snapshot.json
529│ ├── targets
530│ │ └── foo
531│ │ └── bar
532│ │ └── baz.txt
533│ ├── targets.json
534│ └── timestamp.json
535└── staged
536 ├── snapshot.json
537 ├── targets.json
538 └── timestamp.json
539
540$ tuf commit
541
542$ tree .
543.
544├── keys
545│ ├── snapshot.json
546│ ├── targets.json
547│ └── timestamp.json
548├── repository
549│ ├── root.json
550│ ├── snapshot.json
551│ ├── targets
552│ │ └── foo
553│ │ └── bar
554│ │ └── baz.txt
555│ ├── targets.json
556│ └── timestamp.json
557└── staged
558```
559
560#### Update timestamp.json
561
562```bash
563$ tree .
564.
565├── keys
566│ └── timestamp.json
567├── repository
568│ ├── root.json
569│ ├── snapshot.json
570│ ├── targets
571│ │ └── foo
572│ │ └── bar
573│ │ └── baz.txt
574│ ├── targets.json
575│ └── timestamp.json
576└── staged
577
578$ tuf timestamp
579Enter timestamp keys passphrase:
580
581$ tree .
582.
583├── keys
584│ └── timestamp.json
585├── repository
586│ ├── root.json
587│ ├── snapshot.json
588│ ├── targets
589│ │ └── foo
590│ │ └── bar
591│ │ └── baz.txt
592│ ├── targets.json
593│ └── timestamp.json
594└── staged
595 └── timestamp.json
596
597$ tuf commit
598
599$ tree .
600.
601├── keys
602│ └── timestamp.json
603├── repository
604│ ├── root.json
605│ ├── snapshot.json
606│ ├── targets
607│ │ └── foo
608│ │ └── bar
609│ │ └── baz.txt
610│ ├── targets.json
611│ └── timestamp.json
612└── staged
613```
614
615#### Adding a new root key
616
617Copy `staged/root.json` to the root box and generate a new root key on the root box:
618
619```bash
620$ tuf gen-key root
621$ tuf sign root.json
622```
623
624Copy `staged/root.json` from the root box and commit:
625
626```bash
627$ tuf commit
628```
629
630#### Rotating root key(s)
631
632Copy `staged/root.json` to the root box to do the rotation, where `abcd` is the keyid of the key that is being replaced:
633
634```bash
635$ tuf gen-key root
636$ tuf revoke-key root abcd
637$ tuf sign root.json
638```
639
640Note that `revoke-key` removes the old key from `root.json`, but the key remains in the `keys/` directory on the root box as it is needed to sign the next `root.json`. After this signing is done, the old key may be removed from `keys/`. Any number of keys may be added or revoked during this step, but ensure that at least a threshold of valid keys remain.
641
642Copy `staged/root.json` from the root box to commit:
643
644```bash
645$ tuf commit
646```
647
648## Client
649
650For the client package, see https://godoc.org/github.com/theupdateframework/go-tuf/client.
651
652For the client CLI, see https://github.com/theupdateframework/go-tuf/tree/master/cmd/tuf-client.
653
654## Contributing and Development
655
656For local development, `go-tuf` requires Go version 1.18.
657
658The [Python interoperability tests](client/python_interop/) require Python 3
659(available as `python` on the `$PATH`) and the [`python-tuf`
660package](https://github.com/theupdateframework/python-tuf) installed (`pip
661install tuf`). To update the data for these tests requires Docker and make (see
662test data [README.md](client/python_interop/testdata/README.md) for details).
663
664Please see [CONTRIBUTING.md](docs/CONTRIBUTING.md) for contribution guidelines before making your first contribution!
665
666[](https://gitpod.io/#https://github.com/...)
667
668Users wishing to use remote IDEs can also make use of [Gitpod](https://www.gitpod.io/) to make changes to this project.
669
670## Comparison to other implementations
671
672There are TUF implementations in a variety of programming languages. Some other Go implementations of TUF include:
673
674* [Notary](https://github.com/notaryproject/notary): A version of TUF designed specifically for publishing and managing trusted collections of content. It was used by Docker Content Trust, and has since been superseded by the [Notation](https://github.com/notaryproject/notation) project. In contrast, go-tuf is a direct implementation of TUF and has been updated to conform to 1.0.0 of the TUF specification.
675
View as plain text