Secret Management: When Simple Beats Sophisticated

Jun 1, 2025

Your startup just hit Series A and suddenly everyone's talking about "enterprise-grade secret management." The security consultant you hired is pushing HashiCorp Vault. Your cloud architect wants AWS Secrets Manager. Your DevOps engineer discovered Doppler and thinks it'll solve everything. Meanwhile, your .env files have been working fine for two years.

Here's the uncomfortable truth: not every application needs a sophisticated secret management solution. After implementing everything from basic environment variables to full Vault deployments, I've learned that the "right" choice depends more on your team, scale, and operational maturity than on what's technically superior.

The Secret Management Spectrum

Secret management exists on a spectrum from "dead simple" to "enterprise complex." Each point on this spectrum has legitimate use cases, and choosing the wrong level of complexity for your organization can be worse than having no solution at all.

Level 1: Environment Variables and .env Files

Level 2: CI/CD Pipeline Injection

Level 3: Cloud Provider Secret Stores

Level 4: Dedicated Secret Management Platforms

Level 5: Self-Hosted Vault with Dynamic Secrets

Let's walk through each level and when it makes sense.

Level 1: Environment Variables and .env Files

"Just use environment variables" might sound naive, but for many applications, it's perfectly adequate.

How It Works

# .env file (never committed to git)
DATABASE_URL=postgresql://user:pass@localhost:5432/db
API_KEY=sk_live_abcd1234
STRIPE_SECRET=sk_test_xyz789

# docker-compose.yml
version: '3'
services:
  app:
    image: myapp:latest
    env_file: .env

Pros

  • Zero infrastructure overhead - no additional services to run
  • Universal compatibility - every language and framework supports env vars
  • Simple debugging - easy to see what values are being used
  • No network dependencies - secrets are available when the app starts
  • Perfect for development - developers can quickly get environments running

Cons

  • Secrets in plain text on disk and in process memory
  • No audit trail - who accessed what secret when?
  • Manual rotation - updating secrets requires app restarts
  • Process visibility - secrets visible in process lists
  • No centralized management across multiple services

When to Use .env Files

This approach works well for:

  • Early-stage startups with small teams
  • Development environments where security is less critical
  • Single-service applications with limited secret requirements
  • Teams without dedicated DevOps resources

Real talk: If you're a 3-person startup building an MVP, don't let anyone shame you into over-engineering secret management. Your .env file is fine.

Level 2: CI/CD Pipeline Injection

This is where many teams graduate from .env files—secrets are stored in your CI/CD platform and injected during deployment.

How It Works

# GitHub Actions example
name: Deploy
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Deploy to production
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
          API_KEY: ${{ secrets.API_KEY }}
        run: |
          echo "DATABASE_URL=${DATABASE_URL}" >> .env
          docker build -t myapp .
          docker run --env-file .env myapp

Pros

  • Centralized storage in your CI/CD platform
  • Access controls based on repository permissions
  • Encrypted at rest in most CI/CD platforms
  • Audit trails for secret changes
  • Team collaboration without sharing secrets manually

Cons

  • Build-time only - can't rotate secrets without redeployment
  • CI/CD platform dependency - vendor lock-in to GitHub/GitLab/etc.
  • Limited secret rotation capabilities
  • Deployment coupling - secret changes require new deployments

When to Use CI/CD Injection

This approach works for:

  • Teams already invested in GitHub Actions, GitLab CI, or similar
  • Applications with infrequent secret changes
  • Microservices where each service has its own repository
  • Organizations with strong GitOps practices

Level 3: Cloud Provider Secret Stores

Once you're running in AWS, GCP, or Azure, their managed secret services become attractive options.

AWS Secrets Manager

import boto3
import json

# Runtime secret retrieval
def get_secret(secret_name):
    client = boto3.client('secretsmanager', region_name='us-east-1')
    response = client.get_secret_value(SecretId=secret_name)
    return json.loads(response['SecretString'])

# Usage
db_secrets = get_secret('prod/database')
DATABASE_URL = db_secrets['connection_string']

Pros:

  • Managed service - no infrastructure to maintain
  • Fine-grained IAM permissions
  • Automatic rotation for supported services (RDS, Redshift)
  • Cross-service integration with other AWS services
  • Audit logging through CloudTrail

Cons:

  • Vendor lock-in to AWS ecosystem
  • Network dependency - service must reach AWS APIs
  • Cost scaling - charges per secret and API call
  • Limited rotation support for custom applications

Google Secret Manager

from google.cloud import secretmanager

client = secretmanager.SecretManagerServiceClient()
name = f"projects/{project_id}/secrets/{secret_id}/versions/latest"
response = client.access_secret_version(request={"name": name})
secret_value = response.payload.data.decode("UTF-8")

Pros:

  • Simple API and excellent client libraries
  • Automatic encryption with Google-managed keys
  • IAM integration with Google Cloud
  • Reasonable pricing model

Cons:

  • GCP lock-in
  • Limited rotation features compared to AWS
  • Fewer integrations with third-party tools

Azure Key Vault

Similar benefits and limitations, with tight integration into the Azure ecosystem.

When to Use Cloud Provider Secrets

Cloud provider secret stores work well when:

  • Already committed to a single cloud provider
  • Need automatic rotation for managed services
  • Have compliance requirements that rule out third-party tools
  • Want to minimize external dependencies

Level 4: Dedicated Secret Management Platforms

This is where specialized platforms like Doppler, 1Password Secrets Automation, and others shine.

Doppler

Doppler provides a user-friendly interface for secret management across environments:

# Install Doppler CLI
curl -Ls https://cli.doppler.com/install.sh | sh

# Configure project
doppler configure set project=myapp
doppler configure set config=production

# Run application with secrets
doppler run -- node index.js

# Or inject into deployment
doppler secrets download --no-file --format=env > .env

Pros:

  • Excellent user experience - best-in-class web interface
  • Multi-environment support (dev, staging, prod)
  • Team collaboration features
  • Integrations with major deployment platforms
  • Audit logging and access controls
  • No infrastructure to manage

Cons:

  • Another SaaS dependency and potential single point of failure
  • Pricing can scale quickly with team size
  • Network dependency for secret retrieval
  • Limited customization compared to self-hosted solutions

1Password Secrets Automation

# Using 1Password CLI
op read "op://Production/Database/password" | docker run -e DATABASE_PASSWORD myapp

# Or with environment injection
eval $(op signin)
op run --env-file=.env -- docker run myapp

Pros:

  • Familiar 1Password interface for teams already using it
  • Strong security model and reputation
  • Personal and business secret management in one platform
  • Good developer experience

Cons:

  • Newer offering with fewer enterprise features
  • 1Password ecosystem dependency
  • Limited automation compared to dedicated platforms

When to Use Dedicated Platforms

These platforms work well for:

  • Growing teams (10-100 engineers) that need collaboration
  • Multi-environment deployments (dev, staging, prod)
  • Organizations wanting best-of-breed user experience
  • Teams without dedicated platform engineering resources

Level 5: HashiCorp Vault - The Enterprise Option

Vault is the 800-pound gorilla of secret management—incredibly powerful but requires significant operational investment.

Basic Vault Usage

# Store a secret
vault kv put secret/myapp database_url="postgresql://..."

# Retrieve a secret
vault kv get -field=database_url secret/myapp

# Dynamic database credentials
vault write database/config/my-postgres-database \
    plugin_name=postgresql-database-plugin \
    connection_url="postgresql://{{username}}:{{password}}@localhost:5432/postgres" \
    allowed_roles="my-role"

vault write database/roles/my-role \
    db_name=my-postgres-database \
    creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; \
        GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
    default_ttl="1h" \
    max_ttl="24h"

# Get dynamic credentials
vault read database/creds/my-role

Pros

  • Dynamic secrets - generate credentials on-demand with automatic expiration
  • Extensive integrations - plugins for databases, cloud providers, SSH, etc.
  • Fine-grained policies - precise control over who can access what
  • Audit logging for compliance requirements
  • Encryption as a service - can encrypt data without storing it
  • Multi-cloud and hybrid environment support

Cons

  • Operational complexity - requires dedicated team to run properly
  • High availability setup is non-trivial
  • Storage backend decisions have major implications
  • Learning curve for both operators and developers
  • Network architecture requirements for secure access
  • Unsealing process must be carefully managed

Vault Deployment Complexity

Running Vault properly in production requires:

# vault.hcl - simplified production config
storage "consul" {
  address = "127.0.0.1:8500"
  path    = "vault/"
}

listener "tcp" {
  address     = "0.0.0.0:8200"
  tls_cert_file = "/path/to/cert.pem"
  tls_key_file  = "/path/to/key.pem"
}

ui = true
api_addr = "https://vault.company.com:8200"
cluster_addr = "https://vault.company.com:8201"

Plus:

  • Consul cluster for storage backend (or other HA storage)
  • Load balancing across multiple Vault instances
  • TLS certificates and PKI management
  • Unsealing automation and key management
  • Backup and recovery procedures
  • Monitoring and alerting for Vault health

When to Use Vault

Vault makes sense for:

  • Large enterprises with dedicated platform teams
  • Highly regulated industries requiring audit trails
  • Complex environments with dynamic infrastructure
  • Organizations needing dynamic credential generation
  • Multi-cloud or hybrid cloud deployments

Don't use Vault if:

  • You have fewer than 50 engineers
  • No one on your team has Vault experience
  • You can't commit to proper operational practices
  • Your secrets rarely change

The Decision Framework

Choosing the right secret management approach depends on several factors:

Team Size and Maturity

  • 1-5 engineers: .env files or CI/CD injection
  • 5-20 engineers: Cloud provider secrets or Doppler
  • 20-100 engineers: Dedicated platforms like Doppler
  • 100+ engineers: Consider Vault with dedicated platform team

Operational Maturity

  • Startup mode: Simple solutions that don't break
  • Growth stage: Managed services that scale with you
  • Enterprise: Custom solutions with dedicated teams

Compliance Requirements

  • Basic audit needs: Cloud provider solutions
  • SOC 2 compliance: Doppler or cloud provider secrets
  • FIPS/FedRAMP: Vault or specialized solutions

Secret Complexity

  • Static secrets: Any solution works
  • Frequent rotation: Managed services with rotation features
  • Dynamic secrets: Vault or custom solutions

The Anti-Patterns to Avoid

Over-Engineering Early

Don't implement Vault because it's "best practice" if your team can't operate it properly. A poorly configured Vault deployment is less secure than well-managed .env files.

Under-Engineering at Scale

Don't stick with .env files when you have 50 engineers and compliance requirements. Technical debt in secret management becomes security debt.

Ignoring the Human Factor

The best secret management system is the one your team will actually use correctly. A complex system that developers work around is worse than a simple system they follow.

Forgetting About Secrets in Code

No secret management system can save you from committing API keys to Git. Train your team and use automated scanning tools.

Migration Strategies

Moving between secret management approaches doesn't have to be all-or-nothing:

Gradual Migration

# Hybrid approach during migration
if [ -n "$VAULT_TOKEN" ]; then
    DATABASE_URL=$(vault kv get -field=url secret/database)
else
    DATABASE_URL=$DATABASE_URL_FALLBACK
fi

Service-by-Service

Migrate one service at a time rather than everything simultaneously.

Environment-by-Environment

Start with development environments before touching production.

The Verdict

The secret management landscape has matured significantly, but the best solution is still the one that fits your team and scale. Don't let perfect be the enemy of good.

For most teams:

  • Start simple with .env files or CI/CD injection
  • Graduate to cloud provider secrets when you need centralization
  • Consider dedicated platforms like Doppler when you need collaboration
  • Only implement Vault when you have specific enterprise requirements and operational capability

Remember: the goal isn't to have the most sophisticated secret management system—it's to keep your secrets actually secret while enabling your team to move fast and stay secure.

Your secrets are only as secure as your weakest operational practice. A simple system executed well beats a complex system executed poorly every time.