Network Traffic Map — pal-e Cluster

doc-network-traffic-map Doc

security doc phase-8

Project

Network Traffic Map — pal-e Cluster

Security assessment for Phase 8a (NetworkPolicy). Documents all legitimate pod-to-pod and cross-namespace traffic that must be preserved when default-deny ingress is applied.

Cluster snapshot: 2026-03-15. 25 namespaces, ~70 running pods, 19 Tailscale funnel Ingresses, 2 CNPG clusters, 29 ServiceMonitors.

Key Design Facts

Promtail collects logs via hostPath (/var/log/pods, /var/lib/docker/containers). No ingress rules needed on app pods for log collection. Promtail only needs egress to Loki (loki-stack.monitoring:3100).
Node-exporter uses hostNetwork: true. NetworkPolicy does not affect it — it runs on the host network stack.
Blackbox exporter probes external funnel URLs (13 targets). Needs egress only, no cross-namespace ingress.
DNS (kube-dns) on kube-system:53 must be allowed as egress from every pod. Forgetting this is the #1 NetworkPolicy mistake.
Tailscale funnel proxies live in the tailscale namespace. Each creates a pod that forwards external traffic to a target service in another namespace. This is the primary ingress path for all user-facing services.
CNPG operator in cnpg-system must reach Cluster CRs in postgres and woodpecker namespaces (management + webhook).

Traffic by Namespace — Platform

monitoring (9 pods)

Source	Destination	Port	Purpose
tailscale/ts-grafana-funnel	monitoring/grafana	80	Grafana UI
tailscale/ts-alertmanager-funnel	monitoring/alertmanager	9093	Alertmanager UI
monitoring/prometheus	monitoring/grafana	80	Datasource (Grafana pulls from Prometheus)
monitoring/grafana	monitoring/prometheus	9090	Query API
monitoring/grafana	monitoring/loki-stack	3100	Log queries
monitoring/prometheus	monitoring/alertmanager	9093	Alert delivery
monitoring/prometheus-operator	monitoring/prometheus	9090	Config reload
monitoring/prometheus-operator	monitoring/alertmanager	9093	Config reload
monitoring/promtail	monitoring/loki-stack	3100	Log shipping

Prometheus cross-namespace scraping (egress from monitoring, ingress in target ns):

Target Namespace	Target Service	Port	Via
kube-system	coredns	9153	ServiceMonitor
kube-system	kubelet	10250	ServiceMonitor
default	kubernetes (apiserver)	443	ServiceMonitor
basketball-api	basketball-api	8000	ServiceMonitor
gcal-scheduler	gcal-scheduler	8000	ServiceMonitor
harbor	harbor (multiple pods)	8001	ServiceMonitor
pal-e-app	pal-e-app	3000	ServiceMonitor
pal-e-docs	pal-e-docs	8000	ServiceMonitor
platform-validation	platform-validation	80	ServiceMonitor
westsidekingsandqueens	westside-app	3000	ServiceMonitor
monitoring	blackbox-exporter	9115	ServiceMonitor
monitoring	dora-exporter	8000	ServiceMonitor
monitoring	kube-state-metrics	8080	ServiceMonitor
monitoring	node-exporter	9100	ServiceMonitor (hostNetwork — bypasses policy)

Alertmanager egress (external): Slack webhook, Telegram API for notification routing.

argocd (8 pods)

Source	Destination	Port	Purpose
tailscale/ts-argocd-funnel	argocd/argocd-server	80,443	ArgoCD UI
argocd/server	argocd/repo-server	8081	Manifest generation
argocd/server	argocd/redis	6379	Cache
argocd/server	argocd/dex-server	5556,5557	SSO/OIDC
argocd/app-controller	argocd/repo-server	8081	Manifest fetch
argocd/app-controller	argocd/redis	6379	Cache
argocd/notifications	argocd/redis	6379	Cache
argocd/appset-controller	argocd/server	80	App generation
argocd/image-updater	argocd/server	80	gRPC sync trigger

ArgoCD cross-namespace: app-controller → k8s API (all namespaces for sync operations). repo-server → kube-system DNS.

ArgoCD external egress: repo-server → forgejo.tail5b443a.ts.net (git clone pal-e-deployments). image-updater → harbor.tail5b443a.ts.net (tag polling).

forgejo (1 pod)

Source	Destination	Port	Purpose
tailscale/ts-forgejo-funnel	forgejo/forgejo-http	80	UI + API + git HTTP
woodpecker/woodpecker-server	forgejo/forgejo-http	80	Clone via internal URL (WOODPECKER_FORGEJO_CLONE_URL)

Note: ArgoCD clones via external funnel URL, not internal. Woodpecker uses internal.

woodpecker (3 pods)

Source	Destination	Port	Purpose
tailscale/ts-woodpecker-funnel	woodpecker/woodpecker-server	80	UI + Forgejo webhook receiver
woodpecker/agent	woodpecker/woodpecker-server	9000	gRPC (job polling)
woodpecker/server	woodpecker/woodpecker-db-rw	5432	CNPG Postgres

Woodpecker cross-namespace egress: agent → forgejo-http.forgejo.svc:80 (clone). CI pipeline pods → harbor.tail5b443a.ts.net (kaniko push). CI pods → k8s API 10.0.0.217:6443 (kubectl deploy steps).

Note: Woodpecker CI pipeline pods are ephemeral — they run in the woodpecker namespace and need broad egress (clone repos, push images, run tests). NetworkPolicy for these must be permissive or scoped per-pipeline.

harbor (9 pods)

Source	Destination	Port	Purpose
tailscale/ts-harbor-funnel	harbor/harbor-nginx	80	UI + registry API
harbor/nginx	harbor/core	80	API proxy
harbor/nginx	harbor/portal	80	UI static assets
harbor/core	harbor/registry	5000	Image storage
harbor/core	harbor/jobservice	80	Async jobs
harbor/core	harbor/redis	6379	Cache/queue
harbor/core	harbor/database	5432	Metadata DB
harbor/registry	harbor/redis	6379	Cache
harbor/jobservice	harbor/core	80	Callback
harbor/jobservice	harbor/redis	6379	Queue
harbor/trivy	harbor/core	80	Scan results
harbor/exporter	harbor/core	8001	Metrics collection

Harbor external ingress: kubelet image pulls (hostNetwork, bypasses policy). kaniko CI pushes (from woodpecker pods, via external URL).

minio (1 pod)

Source	Destination	Port	Purpose
tailscale/ts-minio-funnel	minio/minio	9001	Console UI
tailscale/ts-minio-api-funnel	minio/minio	9000	S3 API
postgres/pal-e-postgres	minio/minio	9000	CNPG WAL archival + backups
woodpecker/woodpecker-db	minio/minio	9000	CNPG WAL archival + backups

keycloak (1 pod)

Source	Destination	Port	Purpose
tailscale/ts-keycloak-funnel	keycloak/keycloak	80	UI + OIDC endpoints

Note: basketball-api and westside-app reach Keycloak via external funnel URL, not internal service. No cross-namespace ingress needed.

tailscale (18 proxy pods + 1 operator)

Each ts-*-funnel pod proxies external Tailscale traffic to a target service. The operator manages lifecycle.

Egress pattern (cross-namespace): Each funnel proxy needs egress to its target service in the target namespace. This is the primary ingress vector for all user-facing services.

Funnel Proxy	Target Namespace	Target Service:Port
ts-grafana-funnel	monitoring	grafana:80
ts-alertmanager-funnel	monitoring	alertmanager:9093
ts-argocd-funnel	argocd	argocd-server:80
ts-forgejo-funnel	forgejo	forgejo-http:80
ts-woodpecker-funnel	woodpecker	woodpecker-server:80
ts-harbor-funnel	harbor	harbor-nginx:80
ts-minio-funnel	minio	minio:9001
ts-minio-api-funnel	minio	minio:9000
ts-keycloak-funnel	keycloak	keycloak:80
ts-pal-e-docs-funnel	pal-e-docs	pal-e-docs:8000
ts-pal-e-app-funnel	pal-e-app	pal-e-app:3000
ts-basketball-api-funnel	basketball-api	basketball-api:8000
ts-westside-app-funnel	westsidekingsandqueens	westside-app:3000
ts-gcal-scheduler-funnel	gcal-scheduler	gcal-scheduler:8000
ts-mirofish-funnel	mirofish	mirofish:3000
ts-mirofish-api-funnel	mirofish	mirofish:5001
ts-platform-validation-funnel	platform-validation	platform-validation:80
ts-playground-funnel	playground	playground:80

cnpg-system (1 pod)

Source	Destination	Port	Purpose
cnpg-system/operator	postgres/pal-e-postgres	5432	Cluster management
cnpg-system/operator	woodpecker/woodpecker-db	5432	Cluster management
cnpg-system/webhook	k8s API	443	Admission webhook

kube-system (4 pods)

coredns (port 53 UDP/TCP) — CRITICAL. Every pod in the cluster needs egress to kube-dns for name resolution. This is the single most important egress rule in any NetworkPolicy.

metrics-server (port 443) — HPA source. API server reaches it.

nvidia-device-plugin — DaemonSet, hostNetwork. Manages GPU allocation for ollama/palworld.

Traffic by Namespace — Services

pal-e-docs (2 pods)

Source	Destination	Port	Purpose
tailscale/ts-pal-e-docs-funnel	pal-e-docs/pal-e-docs	8000	API + UI
pal-e-app/pal-e-app	pal-e-docs/pal-e-docs	8000	Cross-namespace API (PAL_E_DOCS_API_URL)
monitoring/prometheus	pal-e-docs/pal-e-docs	8000	Metrics scrape
pal-e-docs/pal-e-docs	postgres/pal-e-postgres-rw	5432	Database (cross-namespace)
pal-e-docs/embedding-worker	postgres/pal-e-postgres-rw	5432	Database (cross-namespace)
pal-e-docs/embedding-worker	ollama/ollama	11434	Embedding generation (cross-namespace)

pal-e-app (1 pod)

Source	Destination	Port	Purpose
tailscale/ts-pal-e-app-funnel	pal-e-app/pal-e-app	3000	SvelteKit frontend
monitoring/prometheus	pal-e-app/pal-e-app	3000	Metrics scrape
pal-e-app/pal-e-app	pal-e-docs/pal-e-docs	8000	Backend API (SSR)

basketball-api (2 pods)

Source	Destination	Port	Purpose
tailscale/ts-basketball-api-funnel	basketball-api/basketball-api	8000	REST API
westsidekingsandqueens/westside-app	basketball-api/basketball-api	8000	Cross-namespace API
monitoring/prometheus	basketball-api/basketball-api	8000	Metrics scrape
basketball-api/basketball-api	basketball-api/postgres	5432	Database (same namespace)

westsidekingsandqueens (1 pod)

Source	Destination	Port	Purpose
tailscale/ts-westside-app-funnel	westsidekingsandqueens/westside-app	3000	SvelteKit dashboard
monitoring/prometheus	westsidekingsandqueens/westside-app	3000	Metrics scrape
westsidekingsandqueens/westside-app	basketball-api/basketball-api	8000	Backend API (SSR)

gcal-scheduler (1 pod)

Source	Destination	Port	Purpose
tailscale/ts-gcal-scheduler-funnel	gcal-scheduler/gcal-scheduler	8000	Booking UI
monitoring/prometheus	gcal-scheduler/gcal-scheduler	8000	Metrics scrape

mirofish (1 pod)

Source	Destination	Port	Purpose
tailscale/ts-mirofish-funnel	mirofish/mirofish	3000	Frontend
tailscale/ts-mirofish-api-funnel	mirofish/mirofish	5001	API

Note: VITE_API_BASE_URL uses external funnel URL — API calls originate from user browser, not server pod. No cross-namespace egress from mirofish pod.

platform-validation (1 pod)

Source	Destination	Port	Purpose
tailscale/ts-platform-validation-funnel	platform-validation/platform-validation	80	Validation UI
monitoring/prometheus	platform-validation/platform-validation	80	Metrics scrape

ollama (1 pod)

Source	Destination	Port	Purpose
pal-e-docs/embedding-worker	ollama/ollama	11434	Embedding generation

No funnel. Internal-only service.

postgres (1 CNPG pod)

Source	Destination	Port	Purpose
pal-e-docs/pal-e-docs	postgres/pal-e-postgres-rw	5432	App DB
pal-e-docs/embedding-worker	postgres/pal-e-postgres-rw	5432	App DB
cnpg-system/operator	postgres/pal-e-postgres	5432	Management
postgres/pal-e-postgres	minio/minio	9000	WAL archival + backups

palworld (1 pod + CronJobs)

Source	Destination	Port	Purpose
external (game clients)	palworld/palworld-server	8211 UDP	Game traffic
external (RCON)	palworld/palworld-server	25575	Remote console
external (Moonlight)	palworld/sunshine	47984,47990,47999,48000,48002	Game streaming

No cross-namespace traffic. Game ports likely need NodePort or hostPort — verify before applying policy.

playground (1 pod)

Source	Destination	Port	Purpose
tailscale/ts-playground-funnel	playground/playground	80	Dev playground

Cross-Namespace Traffic Summary

These are the flows that NetworkPolicy must explicitly allow (default-deny blocks everything else):

From Namespace	To Namespace	Port	Flow
tailscale	(17 namespaces)	varies	Funnel proxy → service (primary ingress)
monitoring	(10+ namespaces)	varies	Prometheus scraping ServiceMonitor targets
pal-e-app	pal-e-docs	8000	Frontend SSR → backend API
westsidekingsandqueens	basketball-api	8000	Frontend SSR → backend API
pal-e-docs	postgres	5432	App → CNPG database
pal-e-docs	ollama	11434	Embedding worker → LLM
woodpecker	forgejo	80	CI clone (internal URL)
postgres	minio	9000	CNPG backup/WAL archival
woodpecker (CNPG)	minio	9000	CNPG backup/WAL archival
cnpg-system	postgres	5432	Operator management
cnpg-system	woodpecker	5432	Operator management
ALL pods	kube-system	53	DNS resolution (CRITICAL)

NetworkPolicy Design Recommendations

Kustomize base default-deny — add to bases/standard/networkpolicy.yaml. Denies all ingress. Allows: funnel proxy (from tailscale ns), Prometheus (from monitoring ns, metrics port only). Each overlay can add service-specific rules.
Platform policies in Terraform — network-policies.tf. One resource per namespace. Deploy one at a time: monitoring → forgejo → woodpecker → harbor → minio → argocd → keycloak.
DNS egress rule — every NetworkPolicy that restricts egress MUST include kube-dns:53. Consider NOT restricting egress initially (ingress-only default-deny is safer and still high value).
Woodpecker CI pods — ephemeral pipeline pods need broad egress (clone, push, deploy). Consider labeling pipeline pods and allowing egress for that label.
CNPG operator — needs ingress to postgres pods in postgres and woodpecker namespaces. Also needs k8s API access (likely via service account, not NetworkPolicy).

Namespaces That Need NO Cross-Namespace Ingress

gcal-scheduler — only funnel + prometheus
mirofish — only funnel (no prometheus ServiceMonitor yet)
platform-validation — only funnel + prometheus
playground — only funnel
palworld — game traffic only (may need special NodePort/hostPort handling)
keycloak — only funnel (apps reach via external URL)