Skip to content

ADR-0010 — Doppler over HashiCorp Vault and AWS SSM Parameter Store

Status: Accepted Date: 2026-04-21 (workspace populated; decision crystallised by 2026-04-25) Author(s): Platform engineering (handover documented in memory reference_doppler_workspace.md)

Context

The Evospin codebase is three sibling repos (ebit-api, ebit-fe, ebit-admin-fe) each with multiple environments (dev, stg, prd, plus a perf-specific config dev_perf). Each repo has 12–120 environment variables — Postgres credentials, Redis passwords (cache + bot), JWT secrets, OAuth client secrets, payment-provider API keys, KYC vendor tokens, SendGrid template ids, OpenTelemetry endpoints, etc.

Pre-Doppler state:

  • Local development used .env files copied from .example.env / .local.env. These templates lived in git; secrets were filled in manually per developer machine.
  • There was no shared canonical store of secrets. New engineers re-discovered values from Slack, README files, and live deployments.
  • CI / Terraform had no programmatic access to secrets. The perf-test stack at terraform/perf/ could not bootstrap without manual env-file injection on each VM.

Pressures:

  1. Phase 2 perf-test bootstrap required containers to fetch secrets at runtime, not at build time. The terraform/perf/secrets/ directory needed scoped tokens for each project to pull only its own config.
  2. Multi-environment hygiene — the same JWT secret in dev and prd is a known security anti-pattern. Secrets need per-environment isolation enforceable from outside the codebase.
  3. Operability — two-person team, limited capacity to operate Vault. The store must be turn-key.

Decision

  1. Use Doppler as the single source of truth for environment variables across all three repos.
  2. Workspace ebit-devops (slug 4d59619bf8c6f858715b) holds three projects: ebit-api, ebit-fe, ebit-admin-fe.
  3. Per-project, per-config service tokens for runtime injection. Tokens are scoped to one project + one config; cannot read across boundaries.
  4. Containers run via doppler run --token <service-token> -- <cmd> to inject env vars at process start. No .env file lands on disk in production.
  5. Service tokens land in terraform/perf/secrets/ (gitignored). Rotated on workspace lockout.
  6. Local dev continues to use .env files (developer convenience), with Doppler as the canonical source. Devs doppler secrets download --format env > .env to refresh.

Considered alternatives

A. HashiCorp Vault (self-hosted)

The industry-standard secrets manager. Strong access policies, dynamic secrets, Kubernetes integration. Rejected on operational cost:

  • Self-hosted Vault requires a Raft cluster (3+ nodes) for HA, or a single-node accepting downtime risk.
  • Auto-unseal requires AWS KMS / GCP KMS integration — another moving part.
  • Token / role / policy authoring is non-trivial; mistakes lock teams out.
  • The team is two engineers; Vault ops would consume disproportionate capacity.
  • Vault Cloud / HCP Vault would remove ops cost but at much higher $/month than Doppler at our scale.

B. AWS SSM Parameter Store

Native AWS service, free at low scale, KMS-encrypted, IAM-integrated. Rejected on UX + scoping:

  • No per-config token model. Access is via IAM roles, which are per-IAM-principal not per-deployment. A leaked deployment role unlocks all parameters under its prefix.
  • Weaker CLI ergonomics. Reading a config requires aware-of-prefix path queries; bulk diffs and doppler run style env injection are unsupported natively (third-party wrappers exist).
  • No native multi-env model. You hand-roll the prefix scheme (/ebit-api/dev/, /ebit-api/prd/, etc.); Doppler ships this out of the box.
  • AWS-only. The dev workstation, the FE Vercel deploy (when wired), and any non-AWS environment would require their own auth path.

C. AWS Secrets Manager

Sister product to SSM Parameter Store; first-class secret rotation. Rejected on cost: $0.40 per secret per month × ~140 secrets ≈ $56/mo before any GET costs, against Doppler's free tier for our workspace size. Same UX limitations as SSM.

D. Sealed .env files in git via SOPS / Mozilla age

Encrypt secrets at rest with KMS or age keys; decrypt at deploy time. Rejected on workflow cost:

  • Every secret rotation requires a git commit.
  • Bulk diffs across environments require unsealing first.
  • Web UI for non-engineers (ops, security review) is awkward — they need the decryption key.
  • No real ACL — anyone with the master key reads everything.

E. Doppler

Picked. Strengths:

  • Per-project + per-config service tokens (dp.st.dev_perf.*) — scoped tokens cannot read across configs or projects.
  • doppler run CLI injects env vars cleanly at process start; works identically in local dev, Docker, and Terraform-provisioned VMs.
  • Free tier covers a single workspace with up to 5 users + small secret count — fits our team and footprint.
  • Web UI is usable by non-engineers for sign-off (e.g. confirming that a JWT secret in prd has been rotated).
  • Branch configs (e.g. dev_perf branched from dev) let us model perf-test-specific overrides without duplicating the whole config.

Trade-offs we accepted:

  • Vendor dependency. Doppler going down or going through an acquisition would force a migration. Mitigation: weekly export of all secrets to encrypted local backup (doppler secrets download --format env) so we can rebuild against any alternative provider.
  • Token sprawl. One service token per project per environment, plus the user's personal admin token, accumulates fast. Mitigation: tokens live in secrets/ (gitignored) and ~/.config/doppler/token (chmod 600).

Consequences

Operational

  • Workspace lockout is critical. Loss of admin access to ebit-devops is a P0. Mitigations: the personal admin token is stored at ~/.config/doppler/token (recovered from .bashrc export), and a copy of recent doppler secrets --format json is held in encrypted local backup.
  • CI / Terraform must read service tokens from a non-git location. terraform/perf/secrets/ is gitignored; CI fetches tokens from Doppler-the-runtime via a bootstrap secret injected by the CI environment (chicken-and-egg solved by the CI provider's own secret store).
  • Local dev still uses .env for developer convenience. Treat the .env file as a cache of Doppler — refresh weekly.

Security

  • Per-token scoping means a leaked perf-test token cannot read prd. The blast radius of a leaked token is one config.
  • Audit log — Doppler logs every read / write per token. Use it post-incident.
  • Rotation — single-secret rotation is doppler secrets set KEY=newvalue --project X --config prd; container restart picks up the new value at next doppler run.

Compliance / customer-facing

  • Customers running Evospin on their own infra may prefer a different secrets provider (their own Vault, their AWS account's SSM). The doppler run injection model is replaceable — any tool that exposes env vars works the same. The architectural commitment is to env-var-injected secrets at process start, not to Doppler specifically.
  • For a customer-self-hosted deployment, document the secret schema (env-var names + which env each is required in) and let them fill it via their preferred provider. See docs/recipes/customize-branding.md and docs/env-reference.md.

Known TBDs

  • MCP-server wiring — Doppler ships an MCP server for AI workflows; not yet wired (memoryreference_doppler_workspace.md``). When wired, agents can read/write secrets without leaking tokens to chat.
  • Runtime-token lifecycle — dp.st.dev_perf.* tokens have no automatic expiry. Schedule a quarterly rotation reminder.

References

  • Memory: reference_doppler_workspace.md — workspace + project + token inventory.
  • terraform/perf/secrets/ — service token storage (gitignored).
  • terraform/perf/.gitignore — confirms secrets/ exclusion.
  • .example.env / .local.env per repo — schema templates (no secret values).
  • docs/env-reference.md — every env var across the three repos with description.
  • docs/recipes/customize-branding.md §6 — references SendGrid template ids living in Doppler.
  • Doppler docs — Concepts.
  • Sibling ADR: this is the secrets-management peer to the broader infra story; cross-link docs/perf-run-checklist.md for the perf-test bootstrap that consumes the service tokens.