Introduce HashiCorp Vault secrets into Kubernetes via CSI secret store driver
In this article, I’ll walk through Secrets Store CSI Driver to integrate an external HashiCorp Vault with kubernetes via a Container Storage Interface (CSI) and inject sensitive information that is needed by the application to pods.
Prerequisites:
- vault server ( v1.9)
- csi-secret-store driver (v1.1.2)
- vault-provider (v1.9)
Let’s get into the practical part! we will use helm-chart for installing all prerequisites as it’s the recommended way by Hashicorp for Vault installation, it’s extremely easy and fast!
Run Vault Server:
The default installation , will install Vault in standalone mode, to make vault highly available you need to set ha.enabled to true in values.yaml or by passing -set “server.ha.enabled=true” to helm install command, and specify the desired number of replicas, default is 3. Please checkout the available configuration here for more.
Enable And Configure Kubernetes Authentication Method:
To allow Kubernetes cluster to authenticate with Vault , we need to enable auth/kubernetes access method on Vault (Authentication), then create a policy for specifying the access privileges (Authorization), lastly create Vault role to map Kubernetes service account to the access policy, we can consider the role as a bridge that links between the authentication and the authorization.
By completing above, you have your Vault server ready! Let’s move to Kubernetes cluster side.
Deploy CSI Secret Store Driver:
As you might already know, there are multiple ways to retrieve and inject secrets from external vault server to kubernetes cluster including:
- Vault-injector.
- Some open-source tools ( good referance: GitHub).
- CSI secret store.
I tried vault-injector at first to inject the secrets as environment variables, however, I found it a bit challenging when it came to exporting the secrets once the file retrieved from the vault. The Container Storage Interface (CSI) was the best option for my use case. Essentially, the CSI will attach vault secrets to your pod as ephemeral volumes. Once the volume is attached, the data in it is mounted into the container’s file system, and the application will use that data either directly or via Kubernetes secret resource, as I’ll describe in more detail below.
After confirming the csi driver is up and running and vault-provider as a daemonset on every node, You’ll get a new custom resource called SecretProviderClass which you’ll use to provide the vault configurations and secret details to the CSI driver.
I used identical names in objectName and secretKey parameters for readability purpose on the SecretProviderClass manifest.
objectName: defines the secret alias name to be written in mounted secret on your pod.secretPath: The path in Vault server.secretKey: secret key to read from vault .data
Inject secrets into your pod:
On Kubernetes side, create a ServiceAccount which will be used to authenticate with Vault.
The Service Account need to have a permission to TokenReview resource to be used by vault to verify the token of the service account of pod that want to connect to Vault and get secrets. To grant Kubernetes ServiceAccount a permissions on TokenReview, you need to create system:auth-delegator ClusterRoleBinding
If the service account token of the pod is successfully authenticated, then Vault will generate vault token and return it to the pod.
Lastly, edit your pod manifist to attach a volume and volume mount to your pod definition and add a secretRef for the secret created by csi driver.
The csi driver will communicate with vault provider using gRPC , secretProviderClass specified in the pod’s definition and the pod’s serviceaccount to retrieve the secrets from Vault and mount them into the pod’s CSI volume as well as build a Kubernetes Secret to mirror the mounted content at pod runtime!
Thanks for reading, I hope you enjoyed it :D
