1# Contributing
2
31. [File an issue](https://github.com/googleapis/google-cloud-go/issues/new/choose).
4 The issue will be used to discuss the bug or feature and should be created
5 before sending a PR.
6
71. [Install Go](https://golang.org/dl/).
8 1. Ensure that your `GOBIN` directory (by default `$(go env GOPATH)/bin`)
9 is in your `PATH`.
10 1. Check it's working by running `go version`.
11 * If it doesn't work, check the install location, usually
12 `/usr/local/go`, is on your `PATH`.
13
141. Sign one of the
15[contributor license agreements](#contributor-license-agreements) below.
16
171. Clone the repo:
18 `git clone https://github.com/googleapis/google-cloud-go`
19
201. Change into the checked out source:
21 `cd google-cloud-go`
22
231. Fork the repo.
24
251. Set your fork as a remote:
26 `git remote add fork git@github.com:GITHUB_USERNAME/google-cloud-go.git`
27
281. Make changes, commit to your fork.
29
30 Commit messages should follow the
31 [Conventional Commits Style](https://www.conventionalcommits.org). The scope
32 portion should always be filled with the name of the package affected by the
33 changes being made. For example:
34 ```
35 feat(functions): add gophers codelab
36 ```
37
381. Send a pull request with your changes.
39
40 To minimize friction, consider setting `Allow edits from maintainers` on the
41 PR, which will enable project committers and automation to update your PR.
42
431. A maintainer will review the pull request and make comments.
44
45 Prefer adding additional commits over amending and force-pushing since it can
46 be difficult to follow code reviews when the commit history changes.
47
48 Commits will be squashed when they're merged.
49
50## Policy on new dependencies
51
52While the Go ecosystem is rich with useful modules, in this project we try to
53minimize the number of direct dependencies we have on modules that are not
54Google-owned.
55
56Adding new third party dependencies can have the following effects:
57* broadens the vulnerability surface
58* increases so called "vanity" import routing infrastructure failure points
59* increases complexity of our own [`third_party`][] imports
60
61So if you are contributing, please either contribute the full implementation
62directly, or find a Google-owned project that provides the functionality. Of
63course, there may be exceptions to this rule, but those should be well defined
64and agreed upon by the maintainers ahead of time.
65
66## Testing
67
68We test code against two versions of Go, the minimum and maximum versions
69supported by our clients. To see which versions these are checkout our
70[README](README.md#supported-versions).
71
72### Integration Tests
73
74In addition to the unit tests, you may run the integration test suite. These
75directions describe setting up your environment to run integration tests for
76_all_ packages: note that many of these instructions may be redundant if you
77intend only to run integration tests on a single package.
78
79#### GCP Setup
80
81To run the integrations tests, creation and configuration of three projects in
82the Google Developers Console is required: one specifically for Firestore
83integration tests, one specifically for Bigtable integration tests, and another
84for all other integration tests. We'll refer to these projects as
85"Firestore project", "Bigtable project" and "general project".
86
87Note: You can skip setting up Bigtable project if you do not plan working on or running a few Bigtable
88tests that require a secondary project
89
90After creating each project, you must [create a service account](https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount)
91for each project. Ensure the project-level **Owner**
92[IAM role](https://console.cloud.google.com/iam-admin/iam/project) role is added to
93each service account. During the creation of the service account, you should
94download the JSON credential file for use later.
95
96Next, ensure the following APIs are enabled in the general project:
97
98- BigQuery API
99- BigQuery Data Transfer API
100- Cloud Dataproc API
101- Cloud Dataproc Control API Private
102- Cloud Datastore API
103- Cloud Firestore API
104- Cloud Key Management Service (KMS) API
105- Cloud Natural Language API
106- Cloud OS Login API
107- Cloud Pub/Sub API
108- Cloud Resource Manager API
109- Cloud Spanner API
110- Cloud Speech API
111- Cloud Translation API
112- Cloud Video Intelligence API
113- Cloud Vision API
114- Compute Engine API
115- Compute Engine Instance Group Manager API
116- Container Registry API
117- Firebase Rules API
118- Google Cloud APIs
119- Google Cloud Deployment Manager V2 API
120- Google Cloud SQL
121- Google Cloud Storage
122- Google Cloud Storage JSON API
123- Google Compute Engine Instance Group Updater API
124- Google Compute Engine Instance Groups API
125- Kubernetes Engine API
126- Cloud Error Reporting API
127- Pub/Sub Lite API
128
129Next, create a Datastore database in the general project, and a Firestore
130database in the Firestore project.
131
132Finally, in the general project, create an API key for the translate API:
133
134- Go to GCP Developer Console.
135- Navigate to APIs & Services > Credentials.
136- Click Create Credentials > API Key.
137- Save this key for use in `GCLOUD_TESTS_API_KEY` as described below.
138
139#### Local Setup
140
141Once the three projects are created and configured, set the following environment
142variables:
143
144- `GCLOUD_TESTS_GOLANG_PROJECT_ID`: Developers Console project's ID (e.g.
145bamboo-shift-455) for the general project.
146- `GCLOUD_TESTS_GOLANG_KEY`: The path to the JSON key file of the general
147project's service account.
148- `GCLOUD_TESTS_GOLANG_DATASTORE_DATABASES`: Comma separated list of developer's Datastore databases. If not provided, default database i.e. empty string is used.
149- `GCLOUD_TESTS_GOLANG_FIRESTORE_PROJECT_ID`: Developers Console project's ID
150(e.g. doorway-cliff-677) for the Firestore project.
151- `GCLOUD_TESTS_GOLANG_FIRESTORE_DATABASES` : Comma separated list of developer's Firestore databases. If not provided, default database is used.
152- `GCLOUD_TESTS_GOLANG_FIRESTORE_KEY`: The path to the JSON key file of the
153Firestore project's service account.
154- `GCLOUD_TESTS_API_KEY`: API key for using the Translate API created above.
155- `GCLOUD_TESTS_GOLANG_SECONDARY_BIGTABLE_PROJECT_ID`: Developers Console project's ID (e.g. doorway-cliff-677) for Bigtable optional secondary project. This can be same as Firestore project or any project other than the general project.
156- `GCLOUD_TESTS_BIGTABLE_CLUSTER`: Cluster ID of Bigtable cluster in general project
157- `GCLOUD_TESTS_BIGTABLE_PRI_PROJ_SEC_CLUSTER`: Optional. Cluster ID of Bigtable secondary cluster in general project
158
159As part of the setup that follows, the following variables will be configured:
160
161- `GCLOUD_TESTS_GOLANG_KEYRING`: The full name of the keyring for the tests,
162in the form
163"projects/P/locations/L/keyRings/R". The creation of this is described below.
164- `GCLOUD_TESTS_BIGTABLE_KEYRING`: The full name of the keyring for the bigtable tests,
165in the form
166"projects/P/locations/L/keyRings/R". The creation of this is described below. Expected to be single region.
167- `GCLOUD_TESTS_GOLANG_ZONE`: Compute Engine zone.
168
169Install the [gcloud command-line tool][gcloudcli] to your machine and use it to
170create some resources used in integration tests.
171
172From the project's root directory:
173
174``` sh
175# Sets the default project in your env.
176$ gcloud config set project $GCLOUD_TESTS_GOLANG_PROJECT_ID
177
178# Authenticates the gcloud tool with your account.
179$ gcloud auth login
180
181# Create the indexes for all the databases you want to use in the datastore integration tests.
182# Use empty string as databaseID or skip database flag for default database.
183$ gcloud alpha datastore indexes create --database=your-databaseID-1 --project=$GCLOUD_TESTS_GOLANG_PROJECT_ID testdata/index.yaml
184
185# Creates a Google Cloud storage bucket with the same name as your test project,
186# and with the Cloud Logging service account as owner, for the sink
187# integration tests in logging.
188$ gsutil mb gs://$GCLOUD_TESTS_GOLANG_PROJECT_ID
189$ gsutil acl ch -g cloud-logs@google.com:O gs://$GCLOUD_TESTS_GOLANG_PROJECT_ID
190
191# Creates a PubSub topic for integration tests of storage notifications.
192$ gcloud beta pubsub topics create go-storage-notification-test
193# Next, go to the Pub/Sub dashboard in GCP console. Authorize the user
194# "service-<numeric project id>@gs-project-accounts.iam.gserviceaccount.com"
195# as a publisher to that topic.
196
197# Creates a Spanner instance for the spanner integration tests.
198$ gcloud beta spanner instances create go-integration-test --config regional-us-central1 --nodes 10 --description 'Instance for go client test'
199# NOTE: Spanner instances are priced by the node-hour, so you may want to
200# delete the instance after testing with 'gcloud beta spanner instances delete'.
201
202$ export MY_KEYRING=some-keyring-name
203$ export MY_LOCATION=global
204$ export MY_SINGLE_LOCATION=us-central1
205# Creates a KMS keyring, in the same location as the default location for your
206# project's buckets.
207$ gcloud kms keyrings create $MY_KEYRING --location $MY_LOCATION
208# Creates two keys in the keyring, named key1 and key2.
209$ gcloud kms keys create key1 --keyring $MY_KEYRING --location $MY_LOCATION --purpose encryption
210$ gcloud kms keys create key2 --keyring $MY_KEYRING --location $MY_LOCATION --purpose encryption
211# Sets the GCLOUD_TESTS_GOLANG_KEYRING environment variable.
212$ export GCLOUD_TESTS_GOLANG_KEYRING=projects/$GCLOUD_TESTS_GOLANG_PROJECT_ID/locations/$MY_LOCATION/keyRings/$MY_KEYRING
213# Authorizes Google Cloud Storage to encrypt and decrypt using key1.
214$ gsutil kms authorize -p $GCLOUD_TESTS_GOLANG_PROJECT_ID -k $GCLOUD_TESTS_GOLANG_KEYRING/cryptoKeys/key1
215
216# Create KMS Key in one region for Bigtable
217$ gcloud kms keyrings create $MY_KEYRING --location $MY_SINGLE_LOCATION
218$ gcloud kms keys create key1 --keyring $MY_KEYRING --location $MY_SINGLE_LOCATION --purpose encryption
219# Sets the GCLOUD_TESTS_BIGTABLE_KEYRING environment variable.
220$ export GCLOUD_TESTS_BIGTABLE_KEYRING=projects/$GCLOUD_TESTS_GOLANG_PROJECT_ID/locations/$MY_SINGLE_LOCATION/keyRings/$MY_KEYRING
221# Create a service agent, https://cloud.google.com/bigtable/docs/use-cmek#gcloud:
222$ gcloud beta services identity create \
223 --service=bigtableadmin.googleapis.com \
224 --project $GCLOUD_TESTS_GOLANG_PROJECT_ID
225# Note the service agent email for the agent created.
226$ export SERVICE_AGENT_EMAIL=<service agent email, from last step>
227
228# Authorizes Google Cloud Bigtable to encrypt and decrypt using key1
229$ gcloud kms keys add-iam-policy-binding key1 \
230 --keyring $MY_KEYRING \
231 --location $MY_SINGLE_LOCATION \
232 --role roles/cloudkms.cryptoKeyEncrypterDecrypter \
233 --member "serviceAccount:$SERVICE_AGENT_EMAIL" \
234 --project $GCLOUD_TESTS_GOLANG_PROJECT_ID
235```
236
237It may be useful to add exports to your shell initialization for future use.
238For instance, in `.zshrc`:
239
240```sh
241#### START GO SDK Test Variables
242# Developers Console project's ID (e.g. bamboo-shift-455) for the general project.
243export GCLOUD_TESTS_GOLANG_PROJECT_ID=your-project
244
245# Developers Console project's ID (e.g. bamboo-shift-455) for the Bigtable project.
246export GCLOUD_TESTS_GOLANG_SECONDARY_BIGTABLE_PROJECT_ID=your-bigtable-optional-secondary-project
247
248# The path to the JSON key file of the general project's service account.
249export GCLOUD_TESTS_GOLANG_KEY=~/directory/your-project-abcd1234.json
250
251# Comma separated list of developer's Datastore databases. If not provided,
252# default database i.e. empty string is used.
253export GCLOUD_TESTS_GOLANG_DATASTORE_DATABASES=your-database-1,your-database-2
254
255# Developers Console project's ID (e.g. doorway-cliff-677) for the Firestore project.
256export GCLOUD_TESTS_GOLANG_FIRESTORE_PROJECT_ID=your-firestore-project
257
258# Comma separated list of developer's Firestore databases. If not provided, default database is used.
259export GCLOUD_TESTS_GOLANG_FIRESTORE_DATABASES=your-database-1,your-database-2
260
261# The path to the JSON key file of the Firestore project's service account.
262export GCLOUD_TESTS_GOLANG_FIRESTORE_KEY=~/directory/your-firestore-project-abcd1234.json
263
264# The full name of the keyring for the tests, in the form "projects/P/locations/L/keyRings/R".
265# The creation of this is described below.
266export MY_KEYRING=my-golang-sdk-test
267export MY_LOCATION=global
268export GCLOUD_TESTS_GOLANG_KEYRING=projects/$GCLOUD_TESTS_GOLANG_PROJECT_ID/locations/$MY_LOCATION/keyRings/$MY_KEYRING
269
270# API key for using the Translate API.
271export GCLOUD_TESTS_API_KEY=abcdefghijk123456789
272
273# Compute Engine zone. (https://cloud.google.com/compute/docs/regions-zones)
274export GCLOUD_TESTS_GOLANG_ZONE=your-chosen-region
275#### END GO SDK Test Variables
276```
277
278#### Running
279
280Once you've done the necessary setup, you can run the integration tests by
281running:
282
283``` sh
284$ go test -v ./...
285```
286
287Note that the above command will not run the tests in other modules. To run
288tests on other modules, first navigate to the appropriate
289subdirectory. For instance, to run only the tests for datastore:
290``` sh
291$ cd datastore
292$ go test -v ./...
293```
294
295#### Replay
296
297Some packages can record the RPCs during integration tests to a file for
298subsequent replay. To record, pass the `-record` flag to `go test`. The
299recording will be saved to the _package_`.replay` file. To replay integration
300tests from a saved recording, the replay file must be present, the `-short`
301flag must be passed to `go test`, and the `GCLOUD_TESTS_GOLANG_ENABLE_REPLAY`
302environment variable must have a non-empty value.
303
304## Contributor License Agreements
305
306Before we can accept your pull requests you'll need to sign a Contributor
307License Agreement (CLA):
308
309- **If you are an individual writing original source code** and **you own the
310intellectual property**, then you'll need to sign an [individual CLA][indvcla].
311- **If you work for a company that wants to allow you to contribute your
312work**, then you'll need to sign a [corporate CLA][corpcla].
313
314You can sign these electronically (just scroll to the bottom). After that,
315we'll be able to accept your pull requests.
316
317## Contributor Code of Conduct
318
319As contributors and maintainers of this project,
320and in the interest of fostering an open and welcoming community,
321we pledge to respect all people who contribute through reporting issues,
322posting feature requests, updating documentation,
323submitting pull requests or patches, and other activities.
324
325We are committed to making participation in this project
326a harassment-free experience for everyone,
327regardless of level of experience, gender, gender identity and expression,
328sexual orientation, disability, personal appearance,
329body size, race, ethnicity, age, religion, or nationality.
330
331Examples of unacceptable behavior by participants include:
332
333* The use of sexualized language or imagery
334* Personal attacks
335* Trolling or insulting/derogatory comments
336* Public or private harassment
337* Publishing other's private information,
338such as physical or electronic
339addresses, without explicit permission
340* Other unethical or unprofessional conduct.
341
342Project maintainers have the right and responsibility to remove, edit, or reject
343comments, commits, code, wiki edits, issues, and other contributions
344that are not aligned to this Code of Conduct.
345By adopting this Code of Conduct,
346project maintainers commit themselves to fairly and consistently
347applying these principles to every aspect of managing this project.
348Project maintainers who do not follow or enforce the Code of Conduct
349may be permanently removed from the project team.
350
351This code of conduct applies both within project spaces and in public spaces
352when an individual is representing the project or its community.
353
354Instances of abusive, harassing, or otherwise unacceptable behavior
355may be reported by opening an issue
356or contacting one or more of the project maintainers.
357
358This Code of Conduct is adapted from the [Contributor Covenant](https://contributor-covenant.org), version 1.2.0,
359available at [https://contributor-covenant.org/version/1/2/0/](https://contributor-covenant.org/version/1/2/0/)
360
361[gcloudcli]: https://developers.google.com/cloud/sdk/gcloud/
362[indvcla]: https://developers.google.com/open-source/cla/individual
363[corpcla]: https://developers.google.com/open-source/cla/corporate
364[`third_party`]: https://opensource.google/documentation/reference/thirdparty
View as plain text