1# Buildozer
2
3Buildozer is a command line tool to rewrite multiple
4[Bazel](https://github.com/bazelbuild/bazel) BUILD files using
5standard commands.
6
7## Installation
8
91. Build a binary and put it into your $GOPATH/bin:
10
11```bash
12go install github.com/bazelbuild/buildtools/buildozer@latest
13```
14
15## Usage
16
17```shell
18buildozer [OPTIONS] ['command arg...'...|-f FILE] [label]...
19```
20
21Here, `label...` is a (space-separated, possibly empty) list of Bazel labels,
22for example `//path/to/pkg1:rule1 relative/path/to/pkg2:rule2`. In addition to
23the Bazel label syntax for specifying a package, Buildozer also allows the
24package part to refer to a BUILD-like file, for example `//WORKSPACE:all` or
25`toolchains/BUILD.tpl:host_toolchain`.
26
27Buildozer commands are passed as single positional arguments, and thus have to
28be quoted (or otherwise escaped). Multiple commands and multiple labels can be
29passed. Buildozer will execute all commands on all targets. (So if you do not
30specify at least one command and one target, nothing will happen.) Commands are
31executed in order, files are processed in parallel.
32
33When `-f FILE` is used instead of literal commands, buildozer reads commands
34from `FILE`. `FILE` can be `-`, in which case commands are read from the
35standard input.
36
37The format of the command file is as follows: Empty lines and lines beginning
38with `#` are ignored (including leading whitespace). Non-ignored lines consist
39of `|`-separated sets of commands and labels:
40
41```shell
42command arg arg...|command arg arg...|...|label|label|...
43```
44
45(In fact, commands and labels can appear interleaved in arbitrary order.) `|`
46characters in commands can be escaped like `\|`, but double null bytes
47(`\x00\x00`) are not valid in command files. See below for special handling of
48labels to allow reading from the standard input. When a line in a command file
49uses the single label '*', then the command(s) will be applied to all elements
50of the list `label...` from the command line.
51
52### Targets
53
54Targets look like Bazel labels, but there can be some differences in presence of
55macros.
56
57 * Use the label notation to refer to a rule: `//buildtools/buildozer:edit`
58 * Use the `__pkg__` suffix to refer to the package declaration:
59 `//buildtools/buildozer:__pkg__`
60 * Use an asterisk to refer to all rules in a file: `//pkg:*`
61 * Use `...` to refer to all descendant BUILD files in a directory: `//pkg/...:*`
62 * Use percent to refer to all rules of a certain kind: `//pkg:%java_library`
63 * Use percent-and-number to refer to a rule that begins at a certain line:
64 `//pkg:%123`.
65 * Use the special package name `-` to read the BUILD file from the standard
66 input instead of from a local file in the package directory: `-:all_tests`.
67 (It is presumably not useful to both use a `-` package name and use the `-f
68 -` flag to read commands from the standard input.)
69
70### Options
71
72OPTIONS include the following options:
73
74 * `-stdout` : write changed BUILD file to stdout
75 * `-buildifier` : format output using a specific buildifier binary. If empty, use built-in formatter.
76 * `-k` : apply all commands, even if there are failures
77 * `-quiet` : suppress informational messages
78 * `-shorten_labels` : convert added labels to short form, e.g. //foo:bar => :bar
79 * `-types`: Filter the targets, keeping only those of the given types, e.g.
80 `buildozer -types go_library,go_binary 'print rule' '//buildtools/buildozer:*'`
81 * `-eol-comments=false`: When adding new comments, put them on a separate line.
82
83See `buildozer -help` for the full list.
84
85### Edit commands
86
87Buildozer supports the following commands(`'command args'`):
88
89 * `add <attr> <value(s)>`: Adds value(s) to a list attribute of a rule. If a
90 value is already present in the list, it is not added.
91 * `new_load <path> <[to=]from(s)>`: Add a load statement for the given path,
92 importing the symbols. Before using this, make sure to run
93 `buildozer 'fix movePackageToTop'`. Afterwards, consider running
94 `buildozer 'fix unusedLoads'`.
95 * `replace_load <path> <[to=]from(s)>`: Similar to `new_load`, but removes
96 existing load statements for the requested symbols before adding new loads.
97 * `substitute_load <old_regexp> <new_template>` Replaces modules of loads which
98 match `old_regexp` according to `new_template`. The regular expression must
99 follow [RE2 syntax](https://github.com/google/re2/wiki/Syntax).
100 `new_template` may be a simple replacement string, but it may also expand
101 numbered or named groups using `$0` or `$x`.
102 * `comment <attr>? <value>? <comment>`: Add a comment to a rule, an attribute,
103 or a specific value in a list. Spaces in the comment should be escaped with
104 backslashes.
105 * `print_comment <attr>? <value>?`
106 * `delete`: Delete a rule.
107 * `fix <fix(es)>?`: Apply a fix.
108 * `move <old_attr> <new_attr> <value(s)>`: Moves `value(s)` from the list `old_attr`
109 to the list `new_attr`. The wildcard `*` matches all values.
110 * `new <rule_kind> <rule_name> [(before|after) <relative_rule_name>]`: Add a
111 new rule at the end of the BUILD file (before/after `<relative_rule>`). The
112 identifier `__pkg__` can be used to position rules relative to package().
113 * `print <attr(s)>`
114 * `remove <attr>`: Removes attribute `attr`. The wildcard `*` matches all
115 attributes except `name`.
116 * `remove <attr> <value(s)>`: Removes `value(s)` from the list `attr`. The
117 wildcard `*` matches all attributes. Lists containing none of the `value(s)` are
118 not modified.
119 * `remove_comment <attr>? <value>?`: Removes the comment attached to the rule,
120 an attribute, or a specific value in a list.
121 * `remove_if_equal <attr> <value>`: Removes the attribute `attr` if its value
122 is equal to `value`.
123 * `rename <old_attr> <new_attr>`: Rename the `old_attr` to `new_attr` which must
124 not yet exist.
125 * `replace <attr> <old_value> <new_value>`: Replaces `old_value` with `new_value`
126 in the list `attr`. Wildcard `*` matches all attributes. Lists not containing
127 `old_value` are not modified.
128 * `substitute <attr> <old_regexp> <new_template>`: Replaces strings which
129 match `old_regexp` in the list `attr` according to `new_template`. Wildcard
130 `*` matches all attributes. The regular expression must follow
131 [RE2 syntax](https://github.com/google/re2/wiki/Syntax). `new_template` may
132 be a simple replacement string, but it may also expand numbered or named
133 groups using `$0` or `$x`. Lists without strings that match `old_regexp`
134 are not modified.
135 * `set <attr> <value(s)>`: Sets the value of an attribute. If the attribute
136 was already present, its old value is replaced.
137 * `set_if_absent <attr> <value(s)>`: Sets the value of an attribute. If the
138 attribute was already present, no action is taken.
139 * `set kind <value>`: Set the target type to value.
140 * `set_select <attr> <key_1> <value_1> <key_n> <value_n>`
141 * `copy <attr> <from_rule>`: Copies the value of `attr` between rules. If it
142 exists in the `to_rule`, it will be overwritten.
143 * `copy_no_overwrite <attr> <from_rule>`: Copies the value of `attr` between
144 rules. If it exists in the `to_rule`, no action is taken.
145 * `dict_add <attr> <(key:value)(s)>`: Sets the value of a key for the dict
146 attribute `attr`. If the key was already present, it will _not_ be overwritten
147 * `dict_set <attr> <(key:value)(s)>`: Sets the value of a key for the dict
148 attribute `attr`. If the key was already present, its old value is replaced.
149 * `dict_remove <attr> <key(s)>`: Deletes the key for the dict attribute `attr`.
150 * `dict_list_add <attr> <key> <value(s)>`: Adds value(s) to the list in the
151 dict attribute `attr`.
152 * `format`: Force formatting of all files, even if they were not changed by
153 other commands.
154
155Here, `<attr>` represents an attribute (being `add`ed/`rename`d/`delete`d etc.),
156e.g.: `srcs`, `<value(s)>` represents values of the attribute and so on.
157A '?' indicates that the preceding argument is optional.
158
159The fix command without a fix specified applied to all eligible fixes.
160Use `//path/to/pkg:__pkg__` as label for file level changes like `new_load` and
161`new`.
162A transformation can be applied to all rules of a particular kind by using
163`%rule_kind` at the end of the label(see examples below).
164
165The following commands only apply to `MODULE.bazel` files (e.g. the target
166`//MODULE.bazel:all`):
167
168 * `use_repo_add <use_extension variable name> <repo(s)>`:
169 Ensures that the given repositories are imported via `use_repo` for the
170 extension for which the given top-level variable contains the return value
171 of a `use_extension` call.
172 * `use_repo_add <use_extension variable name> <repo(s)>`:
173 Ensures that the given repositories are *not* imported via `use_repo` for
174 the extension for which the given top-level variable contains the return
175 value of a `use_extension` call.
176 * `use_repo_add [dev] <extension .bzl file> <extension name> <repo(s)>`:
177 Ensures that the given repositories generated by the given extension are
178 imported via `use_repo`. If the `dev` argument is given, extension usages
179 with `dev_dependency = True` will be considered instead. Extension usages
180 with `isolated = True` are ignored.
181 * `use_repo_remove [dev] <extension .bzl file> <extension name> <repo(s)>`:
182 Ensures that the given repositories generated by the given extension are
183 *not* imported via `use_repo`. If the `dev` argument is given, extension
184 usages with `dev_dependency = True` will be considered instead. Extension
185 usages with `isolated = True` are ignored.
186
187#### Examples
188
189```bash
190# Edit //pkg:rule and //pkg:rule2, and add a dependency on //base
191buildozer 'add deps //base' //pkg:rule //pkg:rule2
192
193# A load for a skylark file in //pkg
194buildozer 'new_load //tools/build_rules:build_test.bzl build_test' //pkg:__pkg__
195
196# Replaces existing loads for build_test in //pkg
197buildozer 'replace_load @rules_build//build:defs.bzl build_test' //pkg:__pkg__
198
199# Replaces modules of loads using regular expressions.
200#
201# In this example
202# load("@rules_foo//foo:defs.bzl", "foo_library", "foo_test")
203# will be replaced with
204# load("//third_party/build_defs/rules_foo/foo:defs.bzl", "foo_library", "foo_test")
205buildozer 'substitute_load ^@([^/]*)//([^:].*)$ //third_party/build_defs/${1}/${2}' //pkg:__pkg__
206
207# Change the default_visibility to public for the package //pkg
208buildozer 'set default_visibility //visibility:public' //pkg:__pkg__
209
210# Change all gwt_module targets to java_library in the package //pkg
211buildozer 'set kind java_library' //pkg:%gwt_module
212
213# Replace the dependency on pkg_v1 with a dependency on pkg_v2
214buildozer 'replace deps //pkg_v1 //pkg_v2' //pkg:rule
215
216# Replace all dependencies using regular expressions.
217buildozer 'substitute deps //old/(.*) //new/${1}' //pkg:rule
218
219# Delete the dependency on foo in every cc_library in the package
220buildozer 'remove deps foo' //pkg:%cc_library
221
222# Delete the testonly attribute in every rule in the package
223buildozer 'remove testonly' '//pkg:*'
224
225# Add a comment to the timeout attribute of //pkg:rule_test
226buildozer 'comment timeout Delete\ this\ after\ 2015-12-31.' //pkg:rule_test
227
228# Add a new rule at the end of the file
229buildozer 'new java_library foo' //pkg:__pkg__
230
231# Add a cc_binary rule named new_bin before the rule named tests
232buildozer 'new cc_binary new_bin before tests' //:__pkg__
233
234# Copy an attribute from `protolib` to `py_protolib`.
235buildozer 'copy testonly protolib' //pkg:py_protolib
236
237# Set two attributes in the same rule
238buildozer 'set compile 1' 'set srcmap 1' //pkg:rule
239
240# Make a default explicit in all soy_js rules in a package
241buildozer 'set_if_absent allowv1syntax 1' //pkg:%soy_js
242
243# Add an attribute new_attr with value "def_val" to all cc_binary rules
244# Note that special characters will automatically be escaped in the string
245buildozer 'add new_attr def_val' //:%cc_binary
246```
247
248### Print commands
249
250These commands are not modifying files, Buildifier returns 0 after a successful
251execution.
252
253 * `print <attribute(s)>`: For each target, prints the value of the attributes
254 (see below).
255 * `print_comment <attr>? <value>?`: Prints a comment associated with a rule,
256 an attribute or a specific value in a list.
257
258The print command prints the value of the attributes. If a target doesn't have
259the attribute, a warning is printed on stderr.
260
261There are some special attributes in the `print` command:
262
263 * `kind`: displays the name of the function
264 * `label`: the fully qualified label
265 * `rule`: the entire rule definition
266 * `startline`: the line number on which the rule begins in the BUILD file
267 * `endline`: the line number on which the rule ends in the BUILD file
268 * `path`: the absolute path to the BUILD file that contains the rules
269
270#### Examples
271
272```shell
273# Print the kind of a target
274buildozer 'print kind' base # output: cc_library
275
276# Print the name of all cc_library in //base
277buildozer 'print name' base:%cc_library
278
279# Get the default visibility of the //base package
280buildozer 'print default_visibility' base:%package
281
282# Print labels of cc_library targets in //base that have a deps attribute
283buildozer 'print label deps' base:%cc_library 2>/dev/null | cut -d' ' -f1
284
285# Print the list of labels in //base that explicitly set the testonly attribute:
286buildozer 'print label testonly' 'base:*' 2>/dev/null
287
288# Print the entire definition (including comments) of the //base:heapcheck rule:
289buildozer 'print rule' //base:heapcheck
290```
291
292## Converting labels
293
294Buildozer works at the syntax-level. It doesn't evaluate the BUILD files. If you
295need to query the information Bazel has, please use `bazel query`. If you have a
296list of Bazel labels, chances are that some of them are generated by BUILD
297extensions. Labels in Buildozer are slightly different from labels in Bazel.
298Bazel cares about the generated code, while Buildozer looks at the BUILD file
299before macro expansion.
300
301To see the expanded BUILD files, try:
302
303```shell
304bazel query --output=build //path/to/BUILD
305```
306
307## Do multiple changes at once
308
309Use `buildozer -f <file>` to load a list of commands and labels from a file (see
310[Usage](#usage) above).
311
312```shell
313$ cat /tmp/cmds
314# a comment
315new cc_library foo|//buildtools/buildozer/BUILD
316add deps //base //strings|add srcs foo.cc|//buildtools/buildozer:foo
317add deps :foo|//buildtools/buildozer
318
319$ buildozer -f /tmp/cmds
320fixed //buildtools/buildozer/BUILD
321```
322
323The list of commands will typically be generated and can be large. This is
324efficient: Commands are grouped so that each file is modified once. Files are
325processed in parallel.
326
327Alternatively, BUILD files can be read from the standard input and written to
328the standard output, by using the `-` package name:
329
330```shell
331$ cat /tmp/cmds
332add deps //base //strings|-:foo|-:bar
333
334$ cat some/path/BUILD | buildozer -f /tmp/cmds
335```
336
337This writes the result of updating the `:foo` and `:bar` targets in the input
338BUILD file to the standard output.
339
340Buildozer commands can be made executable by means of a shebang line, too:
341
342```shell
343#!/usr/bin/env -S buildozer -f
344#
345# Adds //base and //string dependencies to :foo and :bar.
346
347add deps //base //strings|-:foo|-:bar
348```
349
350## Error code
351
352The return code is:
353
354 * `0` on success, if changes were made or only readonly commands were executed
355 * `1` when there is a usage error
356 * `2` when at least one command has failed
357 * `3` on success, when no changes were made
358
359## Source Structure
360
361 * `buildozer/main.go` : Entry point for the buildozer binary
362 * `edit/buildozer.go` : Implementation of functions for the buildozer commands
363 * `edit/edit.go`: Library functions to perform various operations on ASTs. These
364 * functions are called by the impl functions in buildozer.go
365 * `edit/fix.go`: Functions for various fixes for the `buildozer 'fix <fix(es)>'`
366 command, like cleaning unused loads, changing labels to canonical notation, etc.
367 * `edit/types.go`: Type information for attributes
368
View as plain text