Integrating Azure Key Vault with AKS via External Secrets Operator

Managing secrets centrally is a best practice in software development. Secrets should be stored and rotated in a central location, even when they are used by multiple components of an application. However, when secrets need to be accessed in different places, such as CI/CD pipelines and Kubernetes, manually copying them can lead to errors and failures caused by human mistakes.

To address this in Kubernetes, we can use External Secrets Operators, which automatically sync secrets from external secret stores such as Azure Key Vaults into Kubernetes secrets. This removes the need for manual synchronization and reduces the risk of errors due to incorrect or outdated secrets.

Pre-requites:

• Azure Kubernetes Cluster
• Azure Key vault
• Managed identity

As the first step, open PowerShell or WSL and log in to the Azure subscription where your AKS cluster is running.

• Then, run the following CLI command to authenticate with the AKS cluster and enable cluster management from the local machine.

az aks get-credentials -n <CLUSTER_NAME> -g <RESOURCE_GROUP> --admin

• To enable secretless authentication with Azure AD, we need to enable the OIDC (OpenID Connect) issuer in AKS. Run the command below to verify the status of the OIDC issuer and Workload Identity for the AKS cluster.

az aks show -g <RESOURCE_GROUP> -n <CLUSTER_NAME> --query "[oidcIssuerProfile.enabled, securityProfile.workloadIdentity.enabled]"

• If the OIDC issuer is not enabled, run the command below to enable OIDC and Workload Identity.

az aks update --resource-group <RESOURCE_GROUP> --name <CLUSTER_NAME> --enable-oidc-issuer --enable-workload-identity

• We need the OIDC issuer URL and the managed identity client ID for the next steps of this blog. To get the OIDC issuer URL, run the command below.

OIDC_ISSUER=$(az aks show --resource-group <RESOURCE_GROUP> --name <CLUSTER_NAME> --query "oidcIssuerProfile.issuerUrl" -o tsv)

• Run the command below to get the client ID of the managed identity.

CLIENT_ID=$(az identity show --name <MANAGEDiDENTITY_NAME> --resource-group <RESOURCE_GROUP> --query 'clientId' -o tsv)

• Run below command to create a namespace.

kubectl create namespace <NAMESPACE_NAME>

• Federated credentials are needed to link your AKS workloads to Azure AD identities securely and secret Lessly. Without them, workloads would either not have access or would require storing secrets.

az identity federated-credential create --name <SERVICEACCOUNT_NAME> --identity-name <MANAGEDiDENTITY_NAME> --resource-group <RESOURCE_GROUP> --issuer $OIDC_ISSUER --subject system:serviceaccount:<NAMESPACE_NAME>:<SERVICEACCOUNT_NAME>

• Run the YAML below to create a service account. The AKS service account is needed to securely map a pod to an Azure AD identity, enabling secretless access to Azure resources. Without it, Workload Identity cannot be used properly.

ex: service-account.yaml

SERVICEACCOUNT_NAME (name of the service account)

NAMESPACE_NAME (name of the namespace)

CLIENT_ID (client id of the managed identity)

apiVersion: v1
kind: ServiceAccount
metadata:
  name: <SERVICEACCOUNT_NAME>
  namespace: <NAMESPACE_NAME>
  annotations:
    azure.workload.identity/client-id: <CLIENT_ID>

• Run below command to create service account.

kubectl apply -f <path to service-account.yaml>

• Run below to adds the External Secrets Helm chart repository to local Helm.

helm repo add external-secrets https://charts.external-secrets.io

• Run below command to Install the External Secrets Operator into the Kubernetes cluster.

helm install external-secrets-operator external-secrets/external-secrets

• Run the yaml below to create an AKS SecretStore. It is a Kubernetes Custom Resource Definition (CRD) that specifies which external secrets provider to use and how to authenticate to it

EX: secretstore.yaml

NAME_FOR_SECRETSTORE (give any name as secret store name)

NAMESPACE_NAME (Name of the namespace)
KEYVAULT_URL (Key vault URI)
TENANT_ID (Azure Tenant ID)
SERVICEACCOUNT_NAME (name of the service account)

apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
  name: <NAME_FOR_SECRETSTORE>
  namespace: <NAMESPACE_NAME>
  annotations:
    "helm.sh/hook": post-install, post-upgrade
    "helm.sh/hook-weight": "10"
spec:
  provider:
    azurekv:
      vaultUrl: <KEYVAULT_URL>
      authType: WorkloadIdentity
      tenantId: <TENANT_ID>
      serviceAccountRef:
        name: <SERVICEACCOUNT_NAME>

• Run below command to create SecretStore.

kubectl apply -f <path to secretstore.yaml>

• Run the below yaml to create an AKS ExternalSecret. It is a Kubernetes custom resource that defines which secrets from an external secret store should be synced into Kubernetes.

ex: externalsecret.yaml

NAME_OF_EXTERNALSECRET (Use suitable name as Name for ExternalSecret)

NAMESPACE_NAME (Name of the namespace)

NAME_FOR_SECRETSTORE (Name of the secret store)
NAME_OF_KUBERNETES_SECRET (Use any suitable name as Kubernetes secret)
NAME_OF_KUBERNETES_SECRET_KEY (Use any suitable name as Kubernetes secret key)
SECRET_NAME_IN_KEYVAULT (Name of the Azure Key Vault secret)

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: <NAME_OF_EXTERNALSECRET>
  namespace: <NAMESPACE_NAME>

spec:
  refreshInterval: 1m
  secretStoreRef:
    kind: SecretStore
    name: <NAME_FOR_SECRETSTORE>
  target:
    name: <NAME_OF_KUBERNETES_SECRET>
    creationPolicy: Owner
  data:
    - secretKey: <NAME_OF_KUBERNETES_SECRET_KEY>   
      remoteRef:
        key: <SECRET_NAME_IN_KEYVAULT>                  

• Run below command to create ExternalSecret.

kubectl apply -f 'path to <externalsecret.yaml>

After applying the ExternalSecret to the AKS cluster, a new Kubernetes Secret is created with the name <NAME_OF_KUBERNETES_SECRET> and the value from the <SECRET_NAME_IN_KEYVAULT> secret.