TODO: Fix CNPG backup verification CronJob failure

todo-cnpg-backup-verify-failure Doc

todo done

Project

What: Daily backup verification CronJob (cnpg-backup-verify) failed. Pod was cleaned up so logs are gone. Actual CNPG backups are fine (8/8 daily backups completed Mar 9-16).

Where: pal-e-platform/terraform/main.tf line 2254 — kubernetes_cron_job_v1.cnpg_backup_verify

Likely cause: The script checks WAL files for both pal-e-postgres and woodpecker prefixes. Woodpecker CNPG cluster was recently added (PR #88). If woodpecker WAL archiving isn't configured or uses a different MinIO path, the verify script fails on the woodpecker check.

Investigate:

Check if backup/postgres-wal/woodpecker/wals/ exists in MinIO
Check woodpecker CNPG cluster backup config — does it archive WALs to the same MinIO bucket?
Increase failed_jobs_history_limit or add log persistence so we don't lose failure context again

Alert: KubeJobFailed warning — cleared by deleting the failed job (done). Will recur if tonight's run also fails.