In the past several posts we looked at Config Connector for Kubernetes. With Config Connector you can create GCP resources, like Spanner or PubSub, using declarative K8s model. In this post we will show how Config Connector works together with Anthos Config Management (ACM).
Why ACM?
In all the examples before, we actuated GCP resources by simply running kubectl apply
on a file or directory, containing declarative configs. This, by itself, however, does not provide full configuration-as-code workflow. First of all, you need to code review the configurations before they are applied. This is to make sure, that the same code, that was reviewed, was eventually submitted. Secondly, you want to apply same or similar configuration on many clusters reliably, without having to run the same script over and over. Finally, to prevent “shadow ops”, i.e. unapproved changes on your clusters.
In this post we will extend the configuration we built earlier, WordPress powered by GCP MySQL database with Workload Identity for secure keyless IAM. We will demonstrate how we can use Anthos Config Management and Config Connector to automatically sync and apply this configuration from Git repo to your K8s clusters and GCP projects.
To start, fork this repo: gcp-kcc-samples. Then, clone your fork:
git clone https://github.com/[my-git-user-name]/gcp-kcc-samples.git repo-name/
GCP project and cluster
Let’s provision a project that will be hosting our main Kubernetes cluster:
export PROJECT_ID=[PROJECT_ID] export BILLING_ACCOUNT=[BILLING_ACCOUNT] export SA_EMAIL="cnrm-system@${PROJECT_ID}.iam.gserviceaccount.com" gcloud projects create ${PROJECT_ID} --name=${PROJECT_ID} gcloud alpha billing projects link ${PROJECT_ID} --billing-account $BILLING_ACCOUNT gcloud config set project $PROJECT_ID # enable container API gcloud services enable container.googleapis.com --project ${PROJECT_ID} # enable SQL admin API gcloud services enable sqladmin.googleapis.com --project ${PROJECT_ID} # create service account that will be used for Config Connector gcloud iam service-accounts create cnrm-system --project ${PROJECT_ID} gcloud projects add-iam-policy-binding ${PROJECT_ID} --member "serviceAccount:${SA_EMAIL}" --role roles/owner gcloud iam service-accounts keys create --iam-account "${SA_EMAIL}" ./key.json
Let’s now create our Kubernetes cluster.
export CLUSTER_ID=[CLUSTER_NAME] export ZONE=[ZONE] gcloud beta container clusters create ${CLUSTER_ID} --identity-namespace=${PROJECT_ID}.svc.id.goog --zone $ZONE gcloud container clusters get-credentials $CLUSTER_ID --zone=$ZONE
Before we proceed with ACM configuration, let’s create two additional GCP projects: they will be holding GCP resources for DEV and PROD environments. Run the following configuration, after replacing [PROJECT_ID] and [BILLING_ACCOUNT] with your custom values:
export PROJECT_ID=[PROJECT_ID] export SA_EMAIL="cnrm-system@${PROJECT_ID}.iam.gserviceaccount.com" export BILLING_ACCOUNT=[BILLING_ACCOUNT] gcloud projects create ${PROJECT_ID}-dev --name=${PROJECT_ID}-dev gcloud alpha billing projects link ${PROJECT_ID}-dev --billing-account $BILLING_ACCOUNT gcloud projects add-iam-policy-binding ${PROJECT_ID}-dev --member "serviceAccount:${SA_EMAIL}" --role roles/owner gcloud services enable sqladmin.googleapis.com --project ${PROJECT_ID}-dev gcloud projects create ${PROJECT_ID}-prod --name=${PROJECT_ID}-prod gcloud alpha billing projects link ${PROJECT_ID}-prod --billing-account $BILLING_ACCOUNT gcloud projects add-iam-policy-binding ${PROJECT_ID}-prod --member "serviceAccount:${SA_EMAIL}" --role roles/owner gcloud services enable sqladmin.googleapis.com --project ${PROJECT_ID}-prod
As you see, this step currently requires using gcloud
command. Config Connector will soon have a capability to create GCP projects. At this point, this step, including project creation and IAM will then be replaced with declarative configuration directly through Anthos Config Management.
We are now ready to initialize our infrastructure. This diagram shows what we are about to build:

As you can see, we are going two environments running WordPress application powered by GCP MySQL database. The configuration between dev
and prod
environments will vary partially, for example, the size of DB, SQL credentials are examples of different configurations. When initialized, wp-dev
and wp-prod
will become Kubernetes namespaces. On GCP side, they will map to two different projects, that we created earlier.
Anthos Config Management
All the subsequent steps will be done assuming you are working in the local clone of your repo in src/wp-acm/
directory. Let’s install ACM operator. Here I’m explicitly referencing version 1.1.0.
# download gsutil cp gs://config-management-release/released/1.1.0/config-management-operator.yaml config-management-operator.yaml # apply CRD kubectl apply -f config-management-operator.yaml
Before we proceed to the next step, if you haven’t already, make sure that you check in all the changes in your local repo fork.
At this point, we are ready to initialize Anthos Config Management and start sync process. Let’s modify and apply the following configuration with kubectl apply
. Note that configConnector and policyController settings are enabled.
apiVersion: configmanagement.gke.io/v1 kind: ConfigManagement metadata: name: config-management spec: clusterName: cluster-1 git: syncRepo: https://github.com/[your-git-user-name]/gcp-kcc-samples.git syncBranch: master secretType: none policyDir: "src/wp-acm/acm-root" configConnector: enabled: true policyController: enabled: true
Please see ACM documentation for explanations of all fields.
Wait some time for resources to sync up. Once cnrm-system
namespace is created, propagate permissions from Config Connector service account to cnrm-system
namespace, where we have Config Connector pod.
kubectl create secret generic gcp-key --from-file ./key.json --namespace cnrm-system
If you run kubectl get namespaces
you will see new namespaces created and after a while if you run kubectl describe iamserviceaccount --all-namespaces
, you will see new SQL service accounts.
What We Built
Once this is applied, Anthos Config Management Syncer should start creating objects. Let us review in more detail all the configs and what they represent:
- acm-root
- cluster: folder containing the configs that applies for the whole cluster
- ns-must-have-cost-center.yaml
- namespaces
- online
- wp: abstract namespace, contains common objects across all namespaces
- wp-dev
- namespace.yaml
- <<more resource files, specific to wp-dev>>
- wp-prod
- namespace.yaml
- <<more resource files, specific to wp-prod>>
- <<common resources, same in wp-prod and wp-dev>>
- wp-dev
- wp: abstract namespace, contains common objects across all namespaces
- online
- system
- repo.yaml: mandatory file container repo config
- cluster: folder containing the configs that applies for the whole cluster
See ACM documentation for more details on the repro structure and what each folder represents.
Once everything is synced, this is what you should have running in each of the two namespaces.

To verify the WordPress sites that were created, you can obtain the external site IP address by running kubectl get svc --namespace=wp-dev
(or wp-prod). Let us review all the key components of our configuration.
GCP Resources Created using Config Connector
In each of the two projects, we created:
- SQL Instance (yaml), SQL User (yaml) and SQL Database (yaml)
- service account (yaml)
- IAM policy to give the service account cloudsql/client permission
- IAM policy to give our K8s account workloadIdentityUser permission on the service account (yaml)
This post gives more details on each of these resources and using Config Connector configuration.
Secure IAM with Workload Identity

Workload identity enables keyless and secure IAM by propagating permissions from your Google service account to your K8s service account. This requires:
- Google service account (yaml)
- K8s service account (yaml – note the annotation)
- Workload Identity Policy (yaml – note, that the member is K8s service account)
See this post for more details on declarative Workload Identity provisioning.
Constraints with Gatekeeper
Gatekeeper constraint that ensures that each namespace has required labels – this is ns-must-have-cost-center.yaml
under cluster
folder.
Importantly, you can apply Gatekeeper constraints on GCP objects created with Config Connector. More example on this in this post.
This is it! If you need to make any changes to your configuration, apply your changes with checking in the config to Git.