Skip to content

External Secrets

External Secrets Operator (ESO) syncs secrets from Doppler into native Kubernetes Secrets. Pods consume them via envFrom - no app-level Doppler SDK required.

Design

Doppler oep.base / oep-stg / mansety-prd / us-prd
    ↓  (ClusterSecretStore + service token)
ExternalSecret CR (per namespace, per secret)
    ↓  (ESO syncs on refreshInterval)
Kubernetes Secret (type=Opaque or kubernetes.io/tls)
    ↓  (envFrom / volumeMount)
Pod
  • One ClusterSecretStore per Doppler config: doppler-oep-stg, doppler-mansety-prd, doppler-us-prd. Each backed by a read-only Doppler service token in the external-secrets namespace (bootstrapped by TF k8s_secrets.tf).
  • No namespace-scoped SecretStores: ClusterSecretStores are reusable across namespaces. ExternalSecrets in any tenant ns reference the relevant store by name.
  • ADR-0001: full secret management rationale at adr/0001-secret-management.md.

ExternalSecret usage pattern

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: app-secrets
  namespace: oep-stg
spec:
  refreshInterval: 60s
  secretStoreRef:
    kind: ClusterSecretStore
    name: doppler-oep-stg
  target:
    name: app-secrets
    creationPolicy: Owner
  data:
    - secretKey: DB_PASSWORD
      remoteRef:
        key: DB_PASSWORD
  # or bulk-sync all keys:
  # dataFrom:
  #   - extract:
  #       key: /

refreshInterval: 60s is the default for most secrets. Use refreshInterval: 1h for rarely-changing config, refreshInterval: 30s for certs/tokens that rotate more often.

Doppler token rotation cascade

When a Doppler service token is rotated (see rotate-secrets.md):

  1. New token created in Doppler UI -> replaces K8s Secret via targeted TF apply.
  2. ESO detects the token Secret changed -> re-authenticates on next sync cycle.
  3. Force immediate re-sync if needed:
kubectl annotate externalsecret -n oep-stg --all \
  force-sync=$(date +%s) --overwrite
  1. kubectl get clustersecretstore shows Valid status after re-auth.

Rate limits

Doppler's API: 10,000 requests/minute per token on most plans. With refreshInterval: 60s and ~20 ExternalSecrets per namespace, that's ~60 req/min per tenant - well within limits. Avoid refreshInterval < 30s in production.

Debugging

ClusterSecretStore not Valid

kubectl describe clustersecretstore doppler-oep-stg
# Look for: Status.Conditions[type=Ready].Message
# Common: token wrong/expired, network to api.doppler.com blocked

ExternalSecret not syncing

kubectl describe externalsecret <name> -n <namespace>
# Status.Conditions[type=Ready].Message shows the error
# Common: key not found in Doppler, store not Valid, permission denied

# Check ESO controller logs
kubectl -n external-secrets logs -l app.kubernetes.io/name=external-secrets --tail=50

Secret value stale

Force immediate sync:

kubectl annotate externalsecret <name> -n <namespace> \
  force-sync=$(date +%s) --overwrite

Verify a secret value matches Doppler

kubectl -n oep-stg get secret <name> -o jsonpath='{.data.<key>}' | base64 -d
doppler secrets get <KEY> --plain --project oep --config oep-stg
# Values should match

laravel-app canonical pattern

The laravel-app Helm chart uses dataFrom.extract as the canonical bulk-pull pattern:

dataFrom:
  - extract:
      key: "/"

This pulls ALL keys from the Doppler config into one K8s Secret. All workloads (api, worker, websockets, scheduler, migrate) consume via envFrom.secretRef. Zero key maintenance in the chart - Doppler is the single source of truth.

ClusterSecretStore per tenant: doppler-oep-stg, doppler-mansety-prd, doppler-us-prd. RefreshInterval: 60s. K8s Secret named <release>-env.

See laravel-app.md "Secret flow" section for the full Doppler -> ESO -> pod diagram.