Signing Docker Images with Notary V2 in Azure Pipelines
Let's learn how to sign Docker images using Notary V2 in Azure Pipelines.
Tools
Env
First, we'll set some helpful environment variables to make the rest of the tutorial easier to follow.
export RESOURCE_GROUP=notaryexport
export ACR_NAME=xeolexampleregistryexport
export AKV_NAME=xeolexamplekeyvault
And then setup some variables for our certificate and private key.
export CERT_SUBJECT="CN=wabbit-networks.io,O=Notary,L=Seattle,ST=WA,C=US"
export KEY_NAME=mysigningkey
export CERT_PATH=$KEY_NAME.pem
1.Setup the Infrastructure
We'll setup an Azure Container Registry, an Azure Key Vault, and then allow our Azure pipelines to have access to both of them.
1.1 Create the Container Registry
# (OPTIONAL): create a resource group
az group create
# create the container registry
acr create
1.2 Create the Key Vault
This will create an Azure Key Vault where we'll store our signing certificate and private key.
az keyvault create --name $AKV_NAME --resource-group $RESOURCE_GROUP --location eastus
2.Create the Signing Certificate
We need to create an x509 certificate and private key to sign and verify our images. Here we create a self-signed certificate, but if you have an existing certificate you can also upload it to Azure Key Vault.
This is a certificate policy that is used to create the certificate and private key with Azure Key Vault.
cat < ./cert_policy.json
{
"issuerParameters": {
"certificateTransparency": null,
"name": "Self"
},
"x509CertificateProperties": {
"ekus": [
"1.3.6.1.5.5.7.3.3"
],
"keyUsage": [
"digitalSignature"
],
"subject": "$CERT_SUBJECT",
"validityInMonths": 12
}
}
EOF
The EKU listed here (1.3.6.1.5.5.7.3.3) is an OID that specifies this certificate is for code signing. (ref). The subject is used later as trust identity when verifying the signature.
Now, we can use the policy to create the certificate and private key
az keyvault certificate create -n $KEY_NAME --vault-name $AKV_NAME -p @cert_policy.json
Get the links to the certificate and private key in order to use them later.
az keyvault certificate show -n $KEY_NAME --vault-name $AKV_NAME --query 'id' -o tsv
az keyvault certificate show -n $KEY_NAME --vault-name $AKV_NAME --query 'kid' -o tsv
Remember to note these values for later.
3.Setup pipelines access to the Key Vault and Container Registry
We'll now setup our Azure DevOps pipelines to have access to the Key Vault and Container Registry.
3.1. Setup Azure Key Vault Service Principal in ADO
- Visit Project Settings for your Azure DevOps project
- Click Service Connections and then New Service Connection and select Azure Resource Manager
- Select Service Principal (automatic) and then click Next.
- Select your Azure subscription and resource group, and then click Save.
- Set the name of the service connection to AzureServiceConnection and click Save.
- Click on the newly created service connection and then click Manage Service Principal and note
the Application (client) ID
3.2 Setup Azure Key Vault RBAC
We need to give our Azure DevOps service connection access to the Key Vault.
export KEY_VAULT_ID=$(az keyvault show --name $AKV_NAME --query id -o tsv)
az role assignment create --role "Key Vault Crypto User" --assignee $APPLICATION_ID --scope $KEY_VAULT_ID
az role assignment create --role "Key Vault Reader" --assignee $APPLICATION_ID --scope $KEY_VAULT_ID
These roles will allow the service principal to read and sign with the certificate and private key.
3.3 Setup Azure Container Registry Service Principal in ADO
- Visit Project Settings for your Azure DevOps project
- Click Service Connections and then New Service Connection and select Docker Registry.
- Select Azure Container Registry and then click Next.
- Select Authentication Type Service Principal. Choose your Azure subscription. And set Azure
Container Registry to the registry you created earlier. Click Save.
- Set the name of the service connection to ACRServiceConnection and click Save.
3.4 Setup Azure Container Registry RBAC
We need to give our Azure DevOps service connection access to the Container Registry.
export REGISTRY_ID=$(az acr show --name $ACR_NAME --query id -o tsv)
az role assignment create --role "AcrPush" --assignee $APPLICATION_ID --scope $REGISTRY_ID
az role assignment create --role "AcrPull" --assignee $APPLICATION_ID --scope $REGISTRY_ID
az role assignment create --role "AcrImageSigner" --assignee $APPLICATION_ID --scope $REGISTRY_ID
This will allow the service principal to push and pull images from the registry, as well as sign images with Notary V2.
4. Setup the Azure Pipeline
Now, we are finally ready to setup our Azure Pipeline. We'll use the Notation Azure Key Vault Plugin to sign our images.
branches:
include:
- main
pool:
vmImage: 'ubuntu-latest'
variables:
ACR_NAME: "xeolexampleregistry"
AKV_NAME: "xeolexamplekeyvault"
KEY_NAME: "xeol"
CERT_PATH: "./$(KEY_NAME).pem"
CERT_SUBJECT: "CN=xeol.io,O=Notary,L=San Fransisco,ST=CA,C=US"
CERT_ID: ""
KEY_ID: ""
IMAGE_REGISTRY: "$(ACR_NAME).azurecr.io"
IMAGE_REPOSITORY: "signed"
IMAGE_TAG: "$(Build.BuildId)"
NOTATION_VERSION: "1.0.0-rc.7"
NOTATION_AZURE_KV_VERSION: "1.0.0-rc.2"
jobs:
- job: Build
displayName: 'Build Docker Image'
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Docker@2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(IMAGE_REPOSITORY)
Dockerfile: '**/Dockerfile'
containerRegistry: ACRServiceConnection
tags: '$(IMAGE_TAG)'
- job: Sign
displayName: 'Sign Docker Image'
pool:
vmImage: 'ubuntu-latest'
steps:
- script: |
set -ex
checksum_file="notation_${NOTATION_VERSION}_checksums.txt"
tar_file="notation_${NOTATION_VERSION}_linux_amd64.tar.gz"
curl -Lo ${checksum_file} "https://github.com/notaryproject/notation/releases/download/v$NOTATION_VERSION/${checksum_file}"
curl -Lo ${tar_file} "https://github.com/notaryproject/notation/releases/download/v$NOTATION_VERSION/${tar_file}"
grep ${tar_file} ${checksum_file} | shasum --check
sudo tar xvzf ${tar_file} -C /usr/bin/ notation
displayName: 'Install Notation'
- script: |
set -ex
install_path=~/.config/notation/plugins/azure-kv
checksum_file="notation-azure-kv_${NOTATION_AZURE_KV_VERSION}_checksums.txt"
tar_file="notation-azure-kv_${NOTATION_AZURE_KV_VERSION}_linux_amd64.tar.gz"
mkdir -p ${install_path}
curl -Lo ${checksum_file} "https://github.com/Azure/notation-azure-kv/releases/download/v${NOTATION_AZURE_KV_VERSION}/${checksum_file}"
curl -Lo ${tar_file} "https://github.com/Azure/notation-azure-kv/releases/download/v${NOTATION_AZURE_KV_VERSION}/${tar_file}"
grep ${tar_file} ${checksum_file} | shasum --check
tar xvzf ${tar_file} -C ${install_path} ./notation-azure-kv
notation plugin ls
displayName: 'Install Notation Azure Key Vault Plugin'
- task: Docker@2
displayName: 'Docker Login'
inputs:
containerRegistry: 'AzureServiceConnection'
command: 'login'
- task: AzureCLI@2
inputs:
azureSubscription: 'AzureServiceConnection'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
set -ex
az keyvault certificate download --file $(CERT_PATH) --id $(CERT_ID) --encoding PEM
notation key add $(KEY_NAME) --plugin azure-kv --id $(KEY_ID)
notation sign --signature-format cose --key $KEY_NAME $(IMAGE_REGISTRY)/$(IMAGE_REPOSITORY):$(IMAGE_TAG)
notation ls $(IMAGE_REGISTRY)/$(IMAGE_REPOSITORY):$(IMAGE_TAG)
displayName: 'Sign Image with Notation'
If everything succeeds you should see a successful output from the notary sign command. 🥳
Successfully signed ***/signed@sha256:9c206fb5100b0d4972c9a5d3e6a6238c62307e1c28efbf56d8a06f31147c5f84