TODO: Fix Grafana CrashLoopBackOff — duplicate default datasource

todo-fix-grafana-duplicate-default-datasource Todo

pal-e-platform todo done

Project

Problem

Grafana is in CrashLoopBackOff in the monitoring namespace. Error:

Three ConfigMaps with label grafana_datasource: "1" exist in the monitoring namespace. Two of them set isDefault: true:

kube-prometheus-stack-grafana-datasource (Helm) — Prometheus isDefault: true
loki-stack (loki-stack Helm chart auto-generated) — Loki isDefault: true
grafana-loki-datasource (custom Terraform ConfigMap) — Loki isDefault: false

ConfigMap #3 was the correct fix attempt, but #2 (auto-generated by loki-stack chart) is still present and competing.

In terraform/main.tf ~line 192, disable the loki-stack chart's built-in datasource sidecar:

Then tofu apply. The custom ConfigMap (#3) already handles the Loki datasource correctly.

This bug predates the 2026-03-06 hard shutdown. Grafana has been crash-looping for ~8 days (since the kube-prometheus-stack was last redeployed).