1Go Protocol buffers
2===================
3
4.. _proto_library: https://docs.bazel.build/versions/master/be/protocol-buffer.html#proto_library
5.. _default Go plugin: https://github.com/golang/protobuf
6.. _common plugins: #predefined-plugins
7.. _Go providers: /go/providers.rst
8.. _GoLibrary: /go/providers.rst#golibrary
9.. _GoSource: /go/providers.rst#gosource
10.. _GoArchive: /go/providers.rst#goarchive
11.. _Gazelle: https://github.com/bazelbuild/bazel-gazelle
12.. _Make variable substitution: https://docs.bazel.build/versions/master/be/make-variables.html#make-var-substitution
13.. _Bourne shell tokenization: https://docs.bazel.build/versions/master/be/common-definitions.html#sh-tokenization
14.. _gogoprotobuf: https://github.com/gogo/protobuf
15.. _compiler.bzl: compiler.bzl
16
17.. role:: param(kbd)
18.. role:: type(emphasis)
19.. role:: value(code)
20.. |mandatory| replace:: **mandatory value**
21
22rules_go provides rules that generate Go packages from .proto files. These
23packages can be imported like regular Go libraries.
24
25.. contents:: :depth: 2
26
27-----
28
29Overview
30--------
31
32Protocol buffers are built with the three rules below. ``go_proto_library`` and
33``go_proto_compiler`` may be loaded from ``@io_bazel_rules_go//proto:def.bzl``.
34
35* `proto_library`_: This is a Bazel built-in rule. It lists a set of .proto
36 files in its ``srcs`` attribute and lists other ``proto_library`` dependencies
37 in its ``deps`` attribute. ``proto_library`` rules may be referenced by
38 language-specific code generation rules like ``java_proto_library`` and
39 ``go_proto_library``.
40* `go_proto_library`_: Generates Go code from .proto files using one or more
41 proto plugins, then builds that code into a Go library. ``go_proto_library``
42 references ``proto_library`` sources via the ``proto`` attribute. They may
43 reference other ``go_proto_library`` and ``go_library`` dependencies via the
44 ``deps`` attributes. ``go_proto_library`` rules can be depended on or
45 embedded directly by ``go_library`` and ``go_binary``.
46* `go_proto_compiler`_: Defines a protoc plugin. By default,
47 ``go_proto_library`` generates Go code with the `default Go plugin`_, but
48 other plugins can be used by setting the ``compilers`` attribute. A few
49 `common plugins`_ are provided in ``@io_bazel_rules_go//proto``.
50
51The ``go_proto_compiler`` rule produces a `GoProtoCompiler`_ provider. If you
52need a greater degree of customization (for example, if you don't want to use
53protoc), you can implement a compatible rule that returns one of these.
54
55The ``go_proto_library`` rule produces the normal set of `Go providers`_. This
56makes it compatible with other Go rules for use in ``deps`` and ``embed``
57attributes.
58
59Avoiding conflicts
60------------------
61
62When linking programs that depend on protos, care must be taken to ensure that
63the same proto isn't registered by more than one package. This may happen if
64you depend on a ``go_proto_library`` and a vendored ``go_library`` generated
65from the same .proto files. You may see compile-time, link-time, or run-time
66errors as a result of this.
67
68There are two main ways to avoid conflicts.
69
70Option 1: Use go_proto_library exclusively
71~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
72
73You can avoid proto conflicts by using ``go_proto_library`` to generate code
74at build time and avoiding ``go_library`` rules based on pre-generated .pb.go
75files.
76
77Gazelle generates rules in this mode by default. When .proto files are present,
78it will generate ``go_proto_library`` rules and ``go_library`` rules that embed
79them (which are safe to use). Gazelle will automatically exclude .pb.go files
80that correspond to .proto files. If you have .proto files belonging to multiple
81packages in the same directory, add the following directives to your
82root build file:
83
84.. code:: bzl
85
86 # gazelle:proto package
87 # gazelle:proto_group go_package
88
89rules_go provides ``go_proto_library`` rules for commonly used proto libraries.
90The Well Known Types can be found in the ``@io_bazel_rules_go//proto/wkt``
91package. There are implicit dependencies of ``go_proto_library`` rules
92that use the default compiler, so they don't need to be written
93explicitly in ``deps``. You can also find rules for Google APIs and gRPC in
94``@go_googleapis//``. You can list these rules with the commands:
95
96.. code:: bash
97
98 $ bazel query 'kind(go_proto_library, @io_bazel_rules_go//proto/wkt:all)'
99 $ bazel query 'kind(go_proto_library, @go_googleapis//...)'
100
101Some commonly used Go libraries, such as ``github.com/golang/protobuf/ptypes``,
102depend on the Well Known Types. In order to avoid conflicts when using these
103libraries, separate versions of these libraries are provided with
104``go_proto_library`` dependencies. Gazelle resolves imports of these libraries
105automatically. For example, it will resolve ``ptypes`` as
106``@com_github_golang_protobuf//ptypes:go_default_library_gen``.
107
108Option 2: Use pre-generated .pb.go files
109~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
110
111You can also avoid conflicts by generating .pb.go files ahead of time and using
112those exclusively instead of using ``go_proto_library``. This may be a better
113option for established Go projects that also need to build with ``go build``.
114
115Gazelle can generate rules for projects built in this mode. Add the following
116comment to your root build file:
117
118.. code:: bzl
119
120 # gazelle:proto disable_global
121
122This prevents Gazelle from generating ``go_proto_library`` rules. .pb.go files
123won't be excluded, and all special cases for imports (such as ``ptypes``) are
124disabled.
125
126If you have ``go_repository`` rules in your ``WORKSPACE`` file that may
127have protos, you'll also need to add
128``build_file_proto_mode = "disable_global"`` to those as well.
129
130.. code:: bzl
131
132 go_repository(
133 name = "com_example_some_project",
134 importpath = "example.com/some/project",
135 tag = "v0.1.2",
136 build_file_proto_mode = "disable_global",
137 )
138
139A note on vendored .proto files
140~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
141
142By default, Bazel assumes imports in .proto files are relative to a repository
143root directory. This means, for example, if you import ``"foo/bar/baz.proto"``,
144that file must be in the directory ``foo/bar``, not
145``vendor/example.com/repo/foo/bar``.
146
147To deal with this, use the `strip_import_prefix` option in the proto_library_
148for the vendored file.
149
150API
151---
152
153go_proto_library
154~~~~~~~~~~~~~~~~
155
156``go_proto_library`` generates a set of .go files from a set of .proto files
157(specified in a ``proto_library`` rule), then builds a Go library from those
158files. ``go_proto_library`` can be imported like any ``go_library`` rule.
159
160Providers
161^^^^^^^^^
162
163* GoLibrary_
164* GoSource_
165* GoArchive_
166
167Attributes
168^^^^^^^^^^
169
170+---------------------+----------------------+-------------------------------------------------+
171| **Name** | **Type** | **Default value** |
172+---------------------+----------------------+-------------------------------------------------+
173| :param:`name` | :type:`string` | |mandatory| |
174+---------------------+----------------------+-------------------------------------------------+
175| A unique name for this rule. |
176| |
177| By convention, and in order to interoperate cleanly with Gazelle_, this |
178| should be a name like ``foo_go_proto``, where ``foo`` is the Go package name |
179| or the last component of the proto package name (hopefully the same). The |
180| ``proto_library`` referenced by ``proto`` should be named ``foo_proto``. |
181+---------------------+----------------------+-------------------------------------------------+
182| :param:`proto` | :type:`label` | |mandatory| |
183+---------------------+----------------------+-------------------------------------------------+
184| Points to the ``proto_library`` containing the .proto sources this rule |
185| should generate code from. Avoid using this argument, use ``protos`` instead. |
186+---------------------+----------------------+-------------------------------------------------+
187| :param:`protos` | :type:`label` | |mandatory| |
188+---------------------+----------------------+-------------------------------------------------+
189| List of ``proto_library`` targets containing the .proto sources this rule should generate |
190| code from. This argument should be used instead of ``proto`` argument. |
191+---------------------+----------------------+-------------------------------------------------+
192| :param:`deps` | :type:`label_list` | :value:`[]` |
193+---------------------+----------------------+-------------------------------------------------+
194| List of Go libraries this library depends on directly. Usually, this will be |
195| a list of ``go_proto_library`` rules that correspond to the ``deps`` of the |
196| ``proto_library`` rule referenced by ``proto``. |
197| |
198| Additional dependencies may be added by the proto compiler. For example, the |
199| default compiler implicitly adds dependencies on the ``go_proto_library`` |
200| rules for the Well Known Types. |
201+---------------------+----------------------+-------------------------------------------------+
202| :param:`importpath` | :type:`string` | |mandatory| |
203+---------------------+----------------------+-------------------------------------------------+
204| The source import path of this library. Other libraries can import this |
205| library using this path. This must be specified in ``go_proto_library`` or |
206| inherited from one of the targets in ``embed``. |
207| |
208| ``importpath`` must match the import path specified in ``.proto`` files using |
209| ``option go_package``. The option determines how ``.pb.go`` files generated |
210| for protos importing this proto will import this package. |
211+---------------------+----------------------+-------------------------------------------------+
212| :param:`importmap` | :type:`string` | :value:`""` |
213+---------------------+----------------------+-------------------------------------------------+
214| The Go package path of this library. This is mostly only visible to the |
215| compiler and linker, but it may also be seen in stack traces. This may be |
216| set to prevent a binary from linking multiple packages with the same import |
217| path, e.g., from different vendor directories. |
218+---------------------+----------------------+-------------------------------------------------+
219| :param:`embed` | :type:`label_list` | :value:`[]` |
220+---------------------+----------------------+-------------------------------------------------+
221| List of Go libraries that should be combined with this library. The ``srcs`` |
222| and ``deps`` from these libraries will be incorporated into this library when it |
223| is compiled. Embedded libraries must have the same ``importpath`` and |
224| Go package name. |
225+---------------------+----------------------+-------------------------------------------------+
226| :param:`gc_goopts` | :type:`string_list` | :value:`[]` |
227+---------------------+----------------------+-------------------------------------------------+
228| List of flags to add to the Go compilation command when using the gc |
229| compiler. Subject to `Make variable substitution`_ and `Bourne shell tokenization`_. |
230+---------------------+----------------------+-------------------------------------------------+
231| :param:`compiler` | :type:`label` | :value:`None` |
232+---------------------+----------------------+-------------------------------------------------+
233| Equivalent to ``compilers`` with a single label. |
234+---------------------+----------------------+-------------------------------------------------+
235| :param:`compilers` | :type:`label_list` | :value:`["@io_bazel_rules_go//proto:go_proto"]` |
236+---------------------+----------------------+-------------------------------------------------+
237| List of rules producing `GoProtoCompiler`_ providers (normally |
238| `go_proto_compiler`_ rules). This is usually understood to be a list of |
239| protoc plugins used to generate Go code. See `Predefined plugins`_ for |
240| some options. |
241+---------------------+----------------------+-------------------------------------------------+
242
243Example: Basic proto
244^^^^^^^^^^^^^^^^^^^^
245
246Suppose you have two .proto files in separate packages: foo/foo.proto and
247bar/bar.proto. foo/foo.proto looks like this:
248
249.. code:: proto
250
251 syntax = "proto3";
252
253 option go_package = "example.com/repo/foo";
254
255 import "google/protobuf/any.proto";
256 import "bar/bar.proto";
257
258 message Foo {
259 bar.Bar x = 1;
260 google.protobuf.Any y = 2;
261 };
262
263In foo/BUILD.bazel, we need to declare a ``proto_library`` rule that lists
264foo.proto in its ``srcs`` attribute. Since we import some other protos, we
265also need a label in ``deps`` for each imported package. We will need to
266create another ``proto_library`` in bar/BUILD.bazel, but we can use an
267existing library for any.proto, since it's one of the Well Known Types.
268
269.. code:: bzl
270
271 proto_library(
272 name = "foo_proto",
273 srcs = ["foo.proto"],
274 deps = [
275 "//bar:bar_proto",
276 "@com_google_protobuf//:any_proto",
277 ],
278 visibility = ["//visibility:public"],
279 )
280
281In order to these this proto in Go, we need to declare a ``go_proto_library``
282that references to ``proto_library`` to be built via the ``proto`` attribute.
283Like ``go_library``, an ``importpath`` attribute needs to be declared.
284Ideally, this should match the ``option go_package`` declaration in the .proto
285file, but this is not required. We also need to list Go packages that the
286generated Go code imports in the ``deps`` attributes. Generally, ``deps``
287in ``go_proto_library`` will correspond with ``deps`` in ``proto_library``,
288but the Well Known Types don't need to be listed (they are added automatically
289by the compiler in use).
290
291.. code:: bzl
292
293 load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")
294
295 go_proto_library(
296 name = "foo_go_proto",
297 importpath = "example.com/repo/foo",
298 proto = ":foo_proto",
299 visibility = ["//visibility:public"],
300 deps = ["//bar:bar_go_proto"],
301 )
302
303This library can be imported like a regular Go library by other rules.
304
305.. code:: bzl
306
307 load("@io_bazel_rules_go//go:def.bzl", "go_binary")
308
309 go_binary(
310 name = "main",
311 srcs = ["main.go"],
312 deps = ["//foo:foo_go_proto"],
313 )
314
315If you need to add additional source files to a package built from protos,
316you can do so with a separate ``go_library`` that embeds the
317``go_proto_library``.
318
319.. code:: bzl
320
321 load("@io_bazel_rules_go//go:def.bzl", "go_library")
322
323 go_library(
324 name = "foo",
325 srcs = ["extra.go"],
326 embed = [":foo_go_proto"],
327 importpath = "example.com/repo/foo",
328 visibility = ["//visibility:public"],
329 )
330
331For convenience, ``proto_library``, ``go_proto_library``, and ``go_binary``
332can all be generated by Gazelle_.
333
334Example: gRPC
335^^^^^^^^^^^^^
336
337To compile protos that contain service definitions, just use the ``go_grpc``
338plugin.
339
340.. code:: bzl
341
342 load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")
343
344 proto_library(
345 name = "foo_proto",
346 srcs = ["foo.proto"],
347 visibility = ["//visibility:public"],
348 )
349
350 go_proto_library(
351 name = "foo_go_proto",
352 compilers = ["@io_bazel_rules_go//proto:go_grpc"],
353 importpath = "example.com/repo/foo",
354 proto = ":foo_proto",
355 visibility = ["//visibility:public"],
356 deps = ["//bar:bar_go_proto"],
357 )
358
359go_proto_compiler
360~~~~~~~~~~~~~~~~~
361
362``go_proto_compiler`` describes a plugin for protoc, the proto compiler.
363Different plugins will generate different Go code from the same protos.
364Compilers may be chosen through the ``compilers`` attribute of
365``go_proto_library``.
366
367Several instances of this rule are listed in `Predefined plugins`_. You will
368only need to use this rule directly if you need a plugin which is not there.
369
370Providers
371^^^^^^^^^
372
373* GoProtoCompiler_
374* GoLibrary_
375* GoSource_
376
377Attributes
378^^^^^^^^^^
379
380+-----------------------------+----------------------+-----------------------------------------------------+
381| **Name** | **Type** | **Default value** |
382+-----------------------------+----------------------+-----------------------------------------------------+
383| :param:`name` | :type:`string` | |mandatory| |
384+-----------------------------+----------------------+-----------------------------------------------------+
385| A unique name for this rule. |
386+-----------------------------+----------------------+-----------------------------------------------------+
387| :param:`deps` | :type:`label_list` | :value:`[]` |
388+-----------------------------+----------------------+-----------------------------------------------------+
389| List of Go libraries that Go code *generated by* this compiler depends on |
390| implicitly. Rules in this list must produce the `GoLibrary`_ provider. This |
391| should contain libraries for the Well Known Types at least. |
392+-----------------------------+----------------------+-----------------------------------------------------+
393| :param:`options` | :type:`string_list` | :value:`[]` |
394+-----------------------------+----------------------+-----------------------------------------------------+
395| List of command line options to be passed to the compiler. Each option will |
396| be preceded by ``--option``. |
397+-----------------------------+----------------------+-----------------------------------------------------+
398| :param:`suffix` | :type:`string` | :value:`.pb.go` |
399+-----------------------------+----------------------+-----------------------------------------------------+
400| File name suffix of generated Go files. ``go_proto_compiler`` assumes that |
401| one Go file will be generated for each input .proto file. Output file names |
402| will have the .proto suffix removed and this suffix appended. For example, |
403| ``foo.proto`` will become ``foo.pb.go``. |
404+-----------------------------+----------------------+-----------------------------------------------------+
405| :param:`valid_archive` | :type:`bool` | :value:`True` |
406+-----------------------------+----------------------+-----------------------------------------------------+
407| Whether code generated by this compiler can be compiled into a standalone |
408| archive file without additional sources. |
409+-----------------------------+----------------------+-----------------------------------------------------+
410| :param:`import_path_option` | :type:`bool` | :value:`True` |
411+-----------------------------+----------------------+-----------------------------------------------------+
412| When true, the ``importpath`` attribute from ``go_proto_library`` rules |
413| using this compiler will be passed to the compiler on the command line as |
414| ``--option import_path={}``. |
415+-----------------------------+----------------------+-----------------------------------------------------+
416| :param:`plugin` | :type:`label` | :value:`@com_github_golang_protobuf//protoc-gen-go` |
417+-----------------------------+----------------------+-----------------------------------------------------+
418| The plugin to use with protoc via the ``--plugin`` option. This rule must |
419| produce an executable file. |
420+-----------------------------+----------------------+-----------------------------------------------------+
421
422Predefined plugins
423------------------
424
425Several ``go_proto_compiler`` rules are predefined in
426``@io_bazel_rules_go//proto``.
427
428* ``go_proto``: default plugin from github.com/golang/protobuf.
429* ``go_grpc``: default gRPC plugin.
430* gogoprotobuf_ plugins for the variants ``combo``, ``gofast``, ``gogo``,
431 ``gogofast``, ``gogofaster``, ``gogoslick``, ``gogotypes``, ``gostring``.
432 For each variant, there is a regular version (e.g., ``gogo_proto``) and a
433 gRPC version (e.g., ``gogo_grpc``).
434
435Providers
436---------
437
438Providers are objects produced by Bazel rules and consumed by other rules that
439depend on them. See `Go providers`_ for information about Go providers,
440specifically GoLibrary_, GoSource_, and GoArchive_.
441
442GoProtoCompiler
443~~~~~~~~~~~~~~~
444
445GoProtoCompiler is the provider returned by the ``go_proto_compiler`` rule and
446anything compatible with it. The ``go_proto_library`` rule expects any rule
447listed in its ``compilers`` attribute to provide ``GoProtoCompiler``. If the
448``go_proto_compiler`` rule doesn't do what you need (e.g., you don't want to
449use protoc), you can write a new rule that produces this.
450
451``GoProtoCompiler`` is loaded from ``@io_bazel_rules_go//proto:def.bzl``.
452
453``GoProtoCompiler`` has the fields described below. Additional fields may be
454added to pass information to the ``compile`` function. This interface is
455*not final* and may change in the future.
456
457+-----------------------------+-------------------------------------------------+
458| **Name** | **Type** |
459+-----------------------------+-------------------------------------------------+
460| :param:`deps` | :type:`Target list` |
461+-----------------------------+-------------------------------------------------+
462| A list of Go libraries to be added as dependencies to any |
463| ``go_proto_library`` compiled with this compiler. Each target must provide |
464| GoLibrary_, GoSource_, and GoArchive_. This list should include libraries |
465| for the Well Known Types and anything else considered "standard". |
466+-----------------------------+-------------------------------------------------+
467| :param:`compile` | :type:`Function` |
468+-----------------------------+-------------------------------------------------+
469| A function which declares output files and actions when called. See |
470| `compiler.bzl`_ for details. |
471+-----------------------------+-------------------------------------------------+
472| :param:`valid_archive` | :type:`bool` |
473+-----------------------------+-------------------------------------------------+
474| Whether the compiler produces a complete Go library. Compilers that just add |
475| methods to structs produced by other compilers will set this to false. |
476+-----------------------------+-------------------------------------------------+
477
478Dependencies
479------------
480
481In order to support protocol buffers, rules_go declares the external
482repositories listed below in ``go_rules_dependencies()``. These repositories
483will only be downloaded if proto rules are used.
484
485* ``@com_google_protobuf (github.com/google/protobuf)``: Well Known Types and
486 general proto support.
487* ``@com_github_golang_protobuf (github.com/golang/protobuf)``: standard
488 Go proto plugin.
489* ``@com_github_gogo_protobuf (github.com/gogo/protobuf)``: gogoprotobuf
490 plugins.
491* ``@org_golang_google_grpc (github.com/grpc/grpc-go``: gRPC support.
492* gRPC dependencies
493
494 * ``@org_golang_x_net (golang.org/x/net)``
495 * ``@org_golang_x_text (golang.org/x/text)``
496 * ``@org_golang_google_genproto (google.golang.org/genproto)``
View as plain text