The Question
While building my TV Dashboard on Kubernetes project – a production-grade streaming analytics platform running on GKE – I faced an interesting architectural decision during the GitOps implementation. Should HashiCorp Vault be managed as an ArgoCD application alongside the other workloads? This post explains why I chose to keep Vault outside of the GitOps workflow.
The Temptation
At first glance, managing Vault through ArgoCD seems pretty cool:
- Consistency: Everything else is managed by ArgoCD
- Version Control: Vault configuration tracked in Git
- Declarative: Infrastructure as Code principles
- Automation: Automated deployments and updates
The Bootstrap Problem
The fundamental issue is a circular dependency:
┌─────────────┐ needs ┌──────────────────┐ needs ┌───────────┐
│ ArgoCD │ ──────────► │ External Secrets │ ──────────► │ Vault │
│ │ │ Operator │ │ │
└─────────────┘ └──────────────────┘ └───────────┘
▲ │
│ manages (if GitOps) │
└────────────────────────────────────────────────────────────┘
The problem: If ArgoCD manages Vault, then ArgoCD depends on Vault (for secrets) which depends on ArgoCD (for deployment). This creates an unresolvable bootstrap dependency.
Infrastructure Layers
I solved this by establishing clear infrastructure layers:
Layer 0: Foundation Infrastructure
- GKE Cluster (Terraform)
- Static IPs (Terraform)
- Vault (kubectl apply)
- External Secrets Operator (Helm)
Layer 1: GitOps Platform
- ArgoCD (kubectl apply)
- ClusterSecretStore (ArgoCD)
Layer 2: Applications
- tv-dashboard-dev (ArgoCD)
- tv-dashboard-prod (ArgoCD)
- monitoring-stack (ArgoCD)
Alternative Approaches Considered
1. Manual Sync Only
# vault-app.yaml
spec:
syncPolicy:
automated: null # No auto-sync to avoid bootstrap issues
Pros: Vault config in Git
Cons: Manual intervention required, defeats GitOps automation
2. App-of-Apps Pattern
# bootstrap-app.yaml - deployed manually
# Manages vault-app.yaml and argocd-apps.yaml
Pros: Complete GitOps coverage
Cons: Complex bootstrap sequence, harder to debug
3. External Vault
Use a managed secret service (Google Secret Manager, AWS Secrets Manager)
Pros: No bootstrap problem, less operational overhead
Cons: Vendor lock-in, less learning value for this project
The Decision: Keep Vault Outside GitOps
Reasons:
- Operational Simplicity: Vault is foundational infrastructure that should be stable and simple to manage
- Recovery Scenarios: If ArgoCD fails, we can still access Vault to recover secrets
- Bootstrap Clarity: Clear separation between foundation and application layers
- Production Patterns: Many organizations treat secret management as Layer 0 infrastructure
Production Considerations
In production environments, you might see:
At Large Organizations
- Dedicated Vault clusters managed by platform teams
- Vault-as-a-Service provided to application teams
- Manual deployment with infrastructure automation (Terraform)
At Cloud-Native Shops
- Managed secret services (AWS Secrets Manager, etc.)
- External Secrets Operator connecting to cloud providers
- No self-hosted Vault at all
In GitOps Purist Repos
- Everything in Git including Vault
- Complex bootstrap procedures with operator dependencies
- Acceptance of operational complexity for consistency
Key Takeaway
Not everything needs to be in GitOps. The right architectural boundary depends on:
- Operational complexity vs consistency benefits
- Bootstrap dependencies and recovery scenarios
- Team structure and operational expertise
- Compliance requirements and audit trails
For my TV Dashboard project, keeping Vault as foundational infrastructure provided the right balance of simplicity and functionality while still demonstrating modern secret management practices.
What I Learned
- Architectural boundaries matter - not every tool fits every pattern
- Bootstrap dependencies are real constraints in system design
- Operational simplicity often trumps theoretical consistency
- Production patterns should inform learning project decisions
The goal isn’t to GitOps-ify everything — it’s to build systems that are reliable, maintainable, and appropriate for their context.
This decision reflects my specific context and learning goals. Your environment may have different constraints and requirements.