# Running the Cloud SQL Proxy as a Service This example demonstrates how to run the Cloud SQL Auth Proxy with PgBouncer on Kubernetes as a service. It assumes you have already successfully completed all the steps in [Using the Cloud SQL Auth Proxy on Kubernetes][sidecar]. In this example, you will deploy [PgBouncer][] with the Cloud SQL Auth Proxy as a sidecar, in addition to configuring encryption between the application and PgBouncer. ## A Word of Warning Running PgBouncer with the Cloud SQL Auth Proxy may pose a significant operational burden and should be undertaken with caution given the attendant complexity. In general, we recommend [running the proxy as a sidecar][sidecar] to your application because it is simple, there is less overhead, it is secure out of the box, and there is less latency involved. However, the service pattern is useful when you are at very large scale, when you clearly need a database connection pooler, and when you are running into SQL Admin API quota problems. ## Initial Setup Before we deploy PgBouncer with the Cloud SQL Auth Proxy, there are three initial steps to take. ### Generate Certificates for PgBouncer First, you will need to generate certificates to encrypt the connection between the application and PgBouncer. We recommend using [CFSSL][] to handle certificate generation. Note: this example uses self-signed certificates. In some cases, using a certificate signed by a public certificate authority may be preferred. Alternatively, Kubernetes includes [an API for issuing certificates][k8s-tls]. See the documentation on [certificates][certificate-docs] for more details. The certificate signing request is encoded as JSON in [`ca_csr.json`](ca_csr.json) for the certificate authority and in [`server_csr.json`](server_csr.json) for the "server," here PgBouncer. First, we initialize our certificate authority. ``` shell # This step produces ca-key.pem (the CA private key) # and ca.pem (the CA certificate). cfssl genkey -initca ca_csr.json | cfssljson -bare ca ``` Next, we generate a public and private key for the server. These will be what we will use to encrypt traffic from the application to PgBouncer. ``` shell # This step produces server-key.pem (the server private key) # and server.pem (the server certicate). cfssl gencert -ca cert -ca-key key server_csr.json | cfssljson -bare server ``` ### Save the certificates as secrets Second, with all the necessary certificates generated, we will save them as secrets: ``` shell # First the CA cert kubectl create secret tls --key="ca-key.pem" --cert="ca.pem" # Next the server cert kubectl create secret tls --key="server-key.pem" \ --cert="server.pem" ``` ### Containerize PgBouncer Third, we will containerize PgBouncer. Some users may prefer to containerize PgBouncer themselves. For this example, we will make use of an open source container, [edoburu/pgbouncer][edoburu]. One nice benefit of `edoburu/pgbouncer` is that it will generate all the PgBouncer configuration based on environment variables passed to the container. ## Deploy PgBouncer as a Service With PgBouncer containerized, we will now create a deployment with PgBouncer and the proxy as a sidecar. First, we mount our CA certificate and server certificate and private key, renaming the certificate secrets to `cert.pem` and server private key to `key.pem`: > [`pgbouncer_deployment.yaml`](pgbouncer_deployment.yaml#L15-L29) ``` yaml volumes: - name: cacert secret: secretName: items: - key: tls.crt path: cert.pem - name: servercert secret: secretName: items: - key: tls.crt path: cert.pem - key: tls.key path: key.pem ``` Next, we specify volume mounts in our PgBouncer container where the secrets will be stored: > [`pgbouncer_deployment.yaml`](pgbouncer_deployment.yaml#L31-L41) ``` yaml - name: pgbouncer image: ports: - containerPort: 5432 volumeMounts: - name: cacert mountPath: "/etc/ca" readOnly: true - name: servercert mountPath: "/etc/server" readOnly: true ``` Then we configure PgBouncer through environment variables. Note: we use 5431 for `DB_PORT` to leave 5432 available. > [`pgbouncer_deployment.yaml`](pgbouncer_deployment.yaml#L42-L69) ``` yaml env: - name: DB_HOST value: "127.0.0.1" - name: DB_USER valueFrom: secretKeyRef: name: key: username - name: DB_PASSWORD valueFrom: secretKeyRef: name: key: password - name: DB_NAME valueFrom: secretKeyRef: name: key: database - name: DB_PORT value: "5431" - name: CLIENT_TLS_SSLMODE value: "require" - name: CLIENT_TLS_CA_FILE value: "/etc/ca/cert.pem" - name: CLIENT_TLS_KEY_FILE value: "/etc/server/key.pem" - name: CLIENT_TLS_CERT_FILE value: "/etc/server/cert.pem" ``` For the PgBouncer deployment, we add the proxy as a sidecar, starting it on port 5431: > [`pgbouncer_deployment.yaml`](pgbouncer_deployment.yaml#L70-L76) ``` yaml - name: cloud-sql-proxy image: gcr.io/cloudsql-docker/gce-proxy:1.22.0 # make sure the use the latest version command: - "/cloud_sql_proxy" - "-instances==tcp:5431" securityContext: runAsNonRoot: true ``` Next, we create a PgBouncer service, listening on port 5342: > [`pgbouncer_service.yaml`](pgbouncer_service.yaml#L1-L11) ``` yaml apiVersion: v1 kind: Service metadata: name: spec: selector: app: ports: - protocol: TCP port: 5432 targetPort: 5432 ``` With the PgBouncer service and deployment done, we are ready to point our application at it. ## Configure your application First, we configure a volume for the CA certificate, mapping the file name to `cert.pem`. > [`deployment.yaml`](deployment.yaml#L1-L11) ``` yaml volumes: - name: cacert secret: secretName: items: - key: tls.crt path: cert.pem ``` Next, we mount the volume within the application container: > [`deployment.yaml`](deployment.yaml#L28-L31) ``` yaml volumeMounts: - name: cacert mountPath: "/etc/ca" readOnly: true ``` Then, we configure environment variables for connecting to the database, this time including a `CA_CERT`: > [`deployment.yaml`](deployment.yaml#L32-L53) ``` yaml env: - name: DB_HOST value: ".default.svc.cluster.local" # using the "default" namespace - name: DB_USER valueFrom: secretKeyRef: name: key: username - name: DB_PASS valueFrom: secretKeyRef: name: key: password - name: DB_NAME valueFrom: secretKeyRef: name: key: database - name: DB_PORT value: "5432" - name: CA_CERT value: "/etc/ca/cert.pem" ``` Note: now the `DB_HOST` value uses an internal DNS record pointing at the PgBouncer service. Finally, when configuring a database connection string, the application must provide the additional properties: 1. `sslmode` must be set to at least `verify-ca` 1. `sslrootcert` must set to the environment variable `CA_CERT` [certificate-docs]: https://kubernetes.io/docs/tasks/administer-cluster/certificates/ [CFSSL]: https://github.com/cloudflare/cfssl [edoburu]: https://hub.docker.com/r/edoburu/pgbouncer [sidecar]: ../k8s-sidecar/README.md [k8s-tls]: https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/ [PgBouncer]: https://www.pgbouncer.org