Honest comparison. Some best practices we follow, some we deliberately deviate from, some we need to adopt.
Structure and Organization
| Best Practice |
Our Status |
Verdict |
| Split resources into logical files (networking.tf, compute.tf, etc.) |
Platform: 1 file (main.tf). Services: 2 files (main.tf + services.tf). |
<strong>Need to fix.</strong> Platform main.tf at 828 lines is too large. Services is fine — main.tf for static, services.tf for dynamic. |
| Use modules for repeated patterns |
No modules. The namespace→helm→funnel pattern repeats 6 times. |
<strong>Need to fix.</strong> But don't over-module. See <code>tf-modularization-roadmap</code>. |
| Separate state per environment |
One environment, one state per repo. |
<strong>Acceptable for now.</strong> Namespace-level separation is pragmatic for single-cluster. See <code>tf-environment-strategy</code>. |
| Remote state with locking |
Kubernetes backend with locking. |
<strong>Good.</strong> Unconventional (most use S3+DynamoDB) but self-contained. Correct for our "no cloud dependencies" principle. |
| Use <code>terraform_remote_state</code> for cross-state refs |
Implicit dependencies only (documented in comments). |
<strong>Acceptable.</strong> With 2 states and sequential apply, remote_state adds complexity for little benefit. Revisit if we add a third state. |
Code Quality
| Best Practice |
Our Status |
Verdict |
| Pin provider versions |
Yes — all pinned with <code>~></code> constraints. |
<strong>Excellent.</strong> |
| Pin Helm chart versions |
Yes — every chart has explicit <code>version</code>. |
<strong>Excellent.</strong> Many teams skip this and get surprised by upgrades. |
| Use <code>.terraform.lock.hcl</code> |
Yes — committed to repo. |
<strong>Good.</strong> |
| Variable validation |
3 variables have validation (harbor_admin_password, harbor_secret_key, minio_root_password). |
<strong>Good start.</strong> Could add validation to all sensitive vars (min length checks). |
| Meaningful output values |
Yes — URLs and internal endpoints. Services exports CI robot creds. |
<strong>Good.</strong> |
| Use <code>description</code> on variables/outputs |
Yes — every variable and output has descriptions. |
<strong>Excellent.</strong> |
Security
| Best Practice |
Our Status |
Verdict |
| Mark sensitive variables |
Yes — all passwords/secrets marked <code>sensitive = true</code>. |
<strong>Good.</strong> |
| Don't store secrets in state |
Partial — <code>set_sensitive</code> avoids some, but Helm release state still contains values. |
<strong>Known limitation</strong> of the Helm provider. State encryption would help. |
| Encrypt state at rest |
No — Kubernetes secrets are base64-encoded, not encrypted (unless etcd encryption is enabled). |
<strong>Should investigate.</strong> k3s may support etcd encryption at rest. |
| Use a secrets manager (Vault, SOPS) |
No — secrets in plaintext tfvars files on disk. |
<strong>Need to fix for team.</strong> SOPS + age is the lightest option. Vault is enterprise-grade but heavy. |
| Least-privilege provider credentials |
Mixed — Tailscale has scoped OAuth, but Harbor/ArgoCD use admin creds. |
<strong>Acceptable for now.</strong> Consider scoped service accounts when team grows. |
Operations
| Best Practice |
Our Status |
Verdict |
| CI/CD pipeline for plan and apply |
No pipeline. Laptop-only. |
<strong>Critical gap.</strong> See <code>tf-pipeline-design</code>. |
| Plan output on PR review |
No. |
<strong>Need.</strong> Most impactful single improvement for team safety. |
| State backup |
No automated backup. |
<strong>Need.</strong> CronJob to MinIO. See <code>tf-rollback-strategy</code>. |
| Drift detection |
No automated drift detection. |
<strong>Nice to have.</strong> Scheduled <code>tofu plan</code> that alerts on drift. |
| Import existing resources |
Documented patterns in MEMORY.md. Used during migration. |
<strong>Good institutional knowledge.</strong> |
| Use <code>moved</code> blocks for refactoring |
Not yet used, but planned for modularization. |
<strong>Ready when needed.</strong> |
Where We Are Genuinely Ahead
- The
var.services for_each pattern — Most platform teams build custom modules or use Terragrunt for per-service infra. Our flat map approach with 7 resources per service is cleaner. The tfvars file IS the API.
- Self-contained cluster — No cloud dependencies except Tailscale. State in k8s, registry in k8s, CI in k8s, GitOps in k8s. The whole platform is one
tofu apply from scratch.
- Operational comments — The TF code reads like a runbook. Two-phase apply notes, secret behavior gotchas, provider quirks. This is rare and valuable.
- Resource limits on everything — Every Helm release has explicit requests and limits. This prevents noisy-neighbor problems and makes capacity planning possible.
Where We Deliberately Deviate
- Kubernetes state backend (vs S3+DynamoDB) — Correct for "no cloud dependencies" principle. Trade-off: less battle-tested, but our state is small.
- No Terragrunt/Terramate — Overkill for 2 repos and 1 environment. Adds a tool dependency. Revisit at 5+ environments.
- No remote_state data sources — Sequential apply with implicit deps is simpler for 2 states. Revisit at 3+ states.
- No workspace-based environments — Deliberately avoided. Workspaces are footguns for environment management.
Priority Order for Improvement
- CI pipeline (plan on PR, apply on merge) — unlocks everything else
- State backup CronJob — cheap insurance
- Secrets management (SOPS or Woodpecker TF_VAR_*) — required for team
- Modularization — improves readability + enables environments
- CloudNativePG operator — enables reliable database layer
- Environment separation — namespace-level first, cluster-level later
- DORA metrics dashboard — measures the pipeline's effectiveness