Secrets Patterns

Choose the smallest secret mechanism that fits the runtime: encrypted at rest in Git, injected at deploy time, or fetched just in time by the workload.

Secret hierarchy

Best: short-lived credentials or just-in-time fetch from a secret store such as Vault or a cloud-native manager.
Good: CI-injected file variables or deploy-time lookup when the app only needs the secret briefly.
Acceptable: Ansible Vault for secrets that must live in Git with the playbook.
Risky: plaintext environment variables, static shared tokens, or files on disk without ownership and rotation rules.
Bad: baking secrets into images, source code, terraform.tfvars, or shell history.

On this page

Choosing the right secret mechanism
Ansible Vault
GitLab masked and file variables
Vault and cloud secret stores
Environment variables vs files on disk
Rotation patterns
Anti-patterns and failure modes

Choosing the right secret mechanism

Not every secret needs the same delivery path. The right answer depends on who needs it, when they need it, and whether the value should still exist after deployment.

Mechanism	Good for	Main weakness
Ansible Vault	Secrets committed alongside infra code and decrypted only during playbook runs	Rotation is manual and the decrypted value still exists at runtime on the target
GitLab masked or file variables	CI jobs that need a short-lived deploy credential or multi-line secret	Still a static secret at rest in CI unless paired with OIDC or Vault
Vault or cloud secret store	Centralized secret fetch, dynamic credentials, runtime access control, auditability	Operational dependency and more moving parts
Environment variable	Simple apps that only read config from env and cannot read secret files	Easy to leak via `env`, crash dumps, debug output, or process inspection
File on disk	Services that support `*_file` or expect certs, keys, or kubeconfigs	Needs strict ownership, mode, cleanup, and backup exclusion

Operationally, the strongest general pattern is: store centrally, fetch late, expire early. When that is overkill, use the simpler mechanism deliberately and document why.

Ansible Vault

Ansible Vault is the right answer when the secret must live next to the playbook or inventory and be decrypted during automation. It is simple, reliable, and Git-friendly, but it does not magically solve runtime secret handling.

ansible-vault encrypt_string 'redacted' --name 'db_password'
ansible-vault encrypt group_vars/production/secrets.yml
ansible-playbook site.yml --ask-vault-pass

Use Ansible Vault for:

service passwords rendered into config files by Ansible
inventory-level secrets needed only during deployment
small teams that want one dependency fewer than running Vault everywhere

Do not pretend it gives you rotation, dynamic credentials, or runtime fetch. It gives you encrypted-at-rest in Git. That is valuable, but it is not the same as a secret control plane.

GitLab masked and file variables

CI variables are fine for deploy-time credentials, especially when the job only needs them briefly. Use file variables for multi-line material like SSH keys, kubeconfigs, or PEM bundles.

deploy:
  stage: deploy
  script:
    - ssh -i "$DEPLOY_KEY" deploy@app01.example.com 'systemctl restart myapp'
    - ansible-playbook site.yml --vault-password-file "$VAULT_PASS_FILE"

Rules for CI variables:

Use protected and masked where possible.
Scope by environment so feature branches cannot read prod.
Never print them with env, set, or echo.

If the pipeline platform supports OIDC federation, stop storing long-lived secrets there at all and move to the pattern described in GitLab Secrets & OIDC.

Vault and cloud secret stores

Central secret stores are the right answer when you want auditability, lease lifetimes, runtime retrieval, or one place to rotate credentials. HashiCorp Vault is the most flexible on-prem option; cloud-native stores are often the easiest choice inside one cloud.

# Vault
vault kv get kv/prod/app

# AWS Secrets Manager
aws secretsmanager get-secret-value --secret-id prod/app/db

# GCP Secret Manager
gcloud secrets versions access latest --secret=prod-app-db

# Azure Key Vault
az keyvault secret show --vault-name kv-prod --name app-db-password

Use a central store when:

multiple apps or pipelines need the same secret
you want centralized audit logs and access control
you need dynamic or revocable credentials
you do not want deploy repos to hold encrypted secrets long term

For Vault-specific auth, policies, and TTL patterns, see HashiCorp Vault.

Environment variables vs files on disk

Both are common. Neither is automatically safe.

Environment variables

They are convenient, but they leak easily into process listings, diagnostics, and careless logging.

[Service]
Environment="APP_DB_USER=myapp"
Environment="APP_DB_PASSWORD=redacted"

Use env vars only when the app expects them and the runtime is well-controlled. Never treat "not visible in the config file" as "not recoverable".

Files on disk

If the application supports a password_file or certificate path, a root-owned file with strict permissions is often safer than an env var.

install -o root -g myapp -m 0640 /dev/stdin /etc/myapp/db.password <<'EOF'
redacted
EOF

# /etc/myapp/config.ini
db_password_file = /etc/myapp/db.password

File rules:

own it explicitly
use the narrowest mode that still works
keep it out of broad backups unless required
delete or replace old versions cleanly during rotation

Rotation patterns

Rotation fails when the new value is generated before the rollout pattern is thought through. The safe sequence is usually create new, distribute new, validate new, revoke old.

Create a second credential or a new secret version.
Store it centrally without deleting the old one yet.
Roll consumers gradually and validate real logins or API calls.
Revoke the old value only after the fleet is confirmed clean.

Rotation example: database password
1. Create app_user_v2 in the database
2. Store app_user_v2 password in Vault / CI / Ansible Vault
3. Redeploy all consumers
4. Confirm new logins succeed
5. Disable app_user_v1

This same shape applies to API tokens, SSH keys, SMTP credentials, and shared service accounts. If the rotation path requires a maintenance window or backout plan, tie it to your Infra Change Lifecycle or a dedicated change runbook.

Anti-patterns and failure modes

Anti-pattern	Why it is bad	Better pattern
Secret committed in plain YAML, JSON, or shell script	Leaks to Git history forever and gets copied everywhere.	Use Ansible Vault or a secret store.
Secret baked into a VM image or container image	Rotation becomes a rebuild-and-redeploy incident across the whole fleet.	Inject at runtime or fetch on startup.
One shared admin token for CI, humans, and apps	No audit separation, huge blast radius, impossible revocation story.	Separate auth paths and short-lived tokens per caller type.
Prod and dev reuse the same secret	Dev compromise becomes prod compromise.	Separate secrets per environment and scope them tightly.
Printing env in CI for debugging	Secrets leak to job logs and artifacts.	Print only the specific safe values you need and rotate anything already exposed.
Plaintext secrets in `terraform.tfvars`	State and local files both become secret stores by accident.	Pass values from a secret manager or CI variable instead.
No owner or calendar for rotation	"Temporary" secrets become multi-year secrets.	Assign an owner, expiry expectation, and rotation runbook.