GitLab Secrets and OIDC

Stop storing long-lived credentials in CI variables. Use variable flags correctly for the ones you must keep, and federate via OIDC to Vault, AWS, and GCP for everything else.

Variable types and flags

File vs regular variable

deploy:
  environment: production
  script:
    # Regular var: single line, exposed in `env`
    - echo "$API_TOKEN" | wc -c

    # File var: GitLab writes the contents to a temp file and
    # puts the path in the env var. Multi-line safe, no shell escaping.
    - ssh -i "$SSH_KEY" deploy@host "systemctl restart app"
    - kubectl --kubeconfig="$KUBECONFIG_FILE" rollout restart deploy/app

Environment scoping

Every sensitive variable must be scoped. Settings → CI/CD → Variables → Environments column:

Then in the job:

deploy-prod:
  stage: deploy
  environment: { name: production }   # <-- this is what matches the scope
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
      when: manual
  script:
    - ansible-playbook -i inventories/prod site.yml

Masking: what it does and does not do

Masking replaces the value with [MASKED] in the job log if the runner sees it verbatim in output. It is the last line of defence, not a substitute for hygiene. Limits:

• Value must be at least 8 characters.
• Value must be a single line. Multi-line values cannot be masked — use a file variable.
• Value must match ^[\w+=/@:.~-]+$ (Base64-ish). Values with spaces, quotes, or ! will not be masked.
• Masking happens on the string as-stored. If your job prints base64 -d'd output, that is a different string — not masked.
• Masking is line-by-line. If a secret appears split across two write() calls from the process, only matching runs get caught.

Hardening against accidental echo

deploy:
  script:
    - set +x                               # never, ever turn this on with secrets
    - : "${VAULT_TOKEN:?missing}"          # fail loudly if the var is empty
    - curl -sSf -H "X-Vault-Token: $VAULT_TOKEN" "$VAULT_ADDR/v1/secret/..." \
        | jq -r .data.data.password        # do not `cat`, do not `echo`

OIDC federation: the idea

Every CI job can request a short-lived JSON Web Token signed by GitLab that contains verifiable claims about what the job is: which project, which branch, whether protected, which user triggered it. Vault, AWS, and GCP can verify that signature and exchange the JWT for their own credentials — no shared secret at rest.

deploy-prod:
  id_tokens:
    VAULT_ID_TOKEN:
      aud: https://vault.example.com
    AWS_ID_TOKEN:
      aud: sts.amazonaws.com

A real JWT payload looks like:

{
  "iss":              "https://gitlab.example.com",
  "aud":              "https://vault.example.com",
  "sub":              "project_path:infra/ansible:ref_type:branch:ref:main",
  "namespace_id":     "42",
  "namespace_path":   "infra",
  "project_id":       "1337",
  "project_path":     "infra/ansible",
  "user_id":          "91",
  "user_login":       "alice",
  "user_email":       "alice@example.com",
  "pipeline_id":      "987654",
  "job_id":           "12345678",
  "ref":              "main",
  "ref_type":         "branch",
  "ref_protected":    "true",
  "environment":      "production",
  "environment_protected": "true",
  "deployment_tier":  "production",
  "iat": 1735689600, "nbf": 1735689600, "exp": 1735693200
}

Every one of those claims is something you can bind against on the consuming side. The interesting ones are project_path, ref, ref_protected, environment, and environment_protected.

Vault via OIDC

Enable JWT auth and point it at GitLab

vault auth enable -path=jwt jwt
vault write auth/jwt/config \
  oidc_discovery_url="https://gitlab.example.com" \
  bound_issuer="https://gitlab.example.com" \
  default_role="ansible-dev"

A prod-bound role

vault write auth/jwt/role/ansible-prod \
  role_type="jwt" \
  user_claim="user_email" \
  token_policies="ansible-prod" \
  token_ttl="20m" \
  token_max_ttl="30m" \
  token_explicit_max_ttl="30m" \
  bound_audiences="https://vault.example.com" \
  bound_claims_type="glob" \
  bound_claims='{
    "project_path":   "infra/ansible",
    "ref":            "main",
    "ref_type":       "branch",
    "ref_protected":  "true",
    "environment":    "production",
    "environment_protected": "true"
  }'

Every claim in bound_claims must match on the incoming JWT or Vault returns invalid_claim. That is how you prevent a feature branch from assuming the prod role.

Consume it in the job

apply-prod:
  id_tokens:
    VAULT_ID_TOKEN:
      aud: https://vault.example.com
  variables:
    VAULT_ADDR: https://vault.example.com
  script:
    - export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=ansible-prod jwt=$VAULT_ID_TOKEN)"
    - export API_PASSWORD="$(vault kv get -field=password secret/prod/app/db)"
    - ansible-playbook -i inventories/prod site.yml
  after_script:
    - vault token revoke -self || true

AWS via OIDC

Create the IAM identity provider

aws iam create-open-id-connect-provider \
  --url https://gitlab.example.com \
  --client-id-list sts.amazonaws.com \
  --thumbprint-list "$(openssl s_client -servername gitlab.example.com -connect gitlab.example.com:443 < /dev/null 2>/dev/null \
    | openssl x509 -fingerprint -noout -sha1 | cut -d= -f2 | tr -d :)"

A role the job can assume

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "Federated": "arn:aws:iam::123456789012:oidc-provider/gitlab.example.com" },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "gitlab.example.com:aud": "sts.amazonaws.com"
      },
      "StringLike": {
        "gitlab.example.com:sub": "project_path:infra/terraform:ref_type:branch:ref:main"
      }
    }
  }]
}

The aud must match the aud you asked GitLab to sign. The sub is how you restrict to a project/branch. Use StringLike with * carefully — too broad is the whole point of getting wrong.

Consume in the job

terraform-apply:
  id_tokens:
    AWS_ID_TOKEN:
      aud: sts.amazonaws.com
  variables:
    AWS_WEB_IDENTITY_TOKEN_FILE: /tmp/aws.token
    AWS_ROLE_ARN: "arn:aws:iam::123456789012:role/gitlab-terraform-prod"
    AWS_DEFAULT_REGION: eu-west-1
  script:
    - echo -n "$AWS_ID_TOKEN" > /tmp/aws.token
    - aws sts get-caller-identity
    - terraform apply -auto-approve

No AWS_ACCESS_KEY_ID, no AWS_SECRET_ACCESS_KEY. The AWS CLI and SDKs read AWS_WEB_IDENTITY_TOKEN_FILE automatically.

GCP via OIDC

Workload Identity Pool

gcloud iam workload-identity-pools create "gitlab-pool" \
  --location="global" --display-name="GitLab CI"

gcloud iam workload-identity-pools providers create-oidc "gitlab" \
  --location="global" \
  --workload-identity-pool="gitlab-pool" \
  --issuer-uri="https://gitlab.example.com" \
  --attribute-mapping="google.subject=assertion.sub,attribute.project_path=assertion.project_path,attribute.ref_protected=assertion.ref_protected" \
  --attribute-condition="assertion.project_path=='infra/terraform' && assertion.ref_protected=='true'"

Bind the pool to a service account

gcloud iam service-accounts add-iam-policy-binding \
  ci-deploy@proj.iam.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects/12345/locations/global/workloadIdentityPools/gitlab-pool/attribute.project_path/infra/terraform"

Consume in the job

gcp-deploy:
  id_tokens:
    GCP_ID_TOKEN:
      aud: https://iam.googleapis.com/projects/12345/locations/global/workloadIdentityPools/gitlab-pool/providers/gitlab
  script:
    - echo "$GCP_ID_TOKEN" > /tmp/gcp.token
    - |
      cat > /tmp/gcp-sa.json <<JSON
      {
        "type": "external_account",
        "audience": "//iam.googleapis.com/projects/12345/locations/global/workloadIdentityPools/gitlab-pool/providers/gitlab",
        "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
        "token_url": "https://sts.googleapis.com/v1/token",
        "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/ci-deploy@proj.iam.gserviceaccount.com:generateAccessToken",
        "credential_source": { "file": "/tmp/gcp.token" }
      }
      JSON
    - export GOOGLE_APPLICATION_CREDENTIALS=/tmp/gcp-sa.json
    - gcloud auth login --cred-file=$GOOGLE_APPLICATION_CREDENTIALS

Rotation and revocation

• The JWT expires with the job. GitLab sets exp to the job's deadline; Vault/AWS/GCP token-TTL should be ≤ that. No post-job replay.
• Always revoke at after_script when the backend supports it (Vault does; AWS STS does not — TTL is the mechanism there).
• Rotate the Vault JWT config's bound audience or CA if a whole issuer needs to be retired; re-issue runner tokens if runners are suspect.
• Static CI variables rotate on a calendar. Any credential you must keep at rest gets an expiry date in your runbook.

What to do if a secret ends up in a log

1. Rotate the credential now. Do not wait to investigate.
2. Delete the job log: API DELETE /projects/:id/jobs/:job_id/artifacts, then UI "Erase job log" on the job.
3. If it was on a merge commit, rewriting history does not help — assume the secret is compromised.
4. Open a ticket; add a mask rule if the value has a recognisable shape; add a CI check that greps for common credential formats.

include, cross-project, and inherited secrets

When using include:project: to pull in a CI template from another project, variables defined in the template project do not come with it. Only the importing project's variables apply. That is almost always what you want — otherwise a library author could demand your prod secrets.

include:
  - project: 'infra/ci-templates'
    ref: v3.2.0                  # pin, never 'main'
    file: '/ansible/lint.yml'

Pin the ref to a tag. main is a moving target that a compromise of the template repo turns into a pipeline takeover of every consumer.

Troubleshooting

See also: HashiCorp Vault, GitLab CI/CD, CI for Ansible.