Wiring metrics

Every metrics panel in Astrolift — the cluster Status tab, an app's Observability tab, workload and pod resource gauges — reads a single field: the cluster's prometheus_endpoint. Until it is set, each of those panels says the endpoint isn't wired for this cluster.

What to install

Astrolift does not deploy Prometheus. Install one in the tenant cluster — kube-prometheus-stack is what the platform is developed against — and leave its default exporters on. The panels are built on series those exporters already produce:

Panel	Series	Comes from
Cluster saturation, node and pod counts	kube_node_*, kube_pod_*, kube_deployment_*	kube-state-metrics
Workload and pod CPU / memory	container_cpu_usage_seconds_total, container_memory_working_set_bytes	cAdvisor
Traffic, errors, latency	ingress-controller or application metrics	see the app side

kube-prometheus-stack's Prometheus watches PodMonitor and ServiceMonitor objects across all namespaces, which is how a workload that opts into scraping is picked up with no further configuration.

The reachability contract

The control plane queries Prometheus over HTTP, from outside the cluster. It is not a Kubernetes client for this read, so:

• a Service ClusterIP is not routable — it is an iptables construct inside the cluster's network namespace;
• the in-cluster DNS name that resolves to it (prometheus.monitoring.svc.cluster.local) is not resolvable either.

The natural first attempt is exactly that name, and it fails with a connection error that looks like Prometheus is down. Astrolift recognises an in-cluster address and says so rather than reporting it unreachable, but the fix is on your side: put an internal load balancer in front of the Prometheus service, reachable from the control plane's network, and point the endpoint at that.

Pod IPs are a different case. On EKS they are real VPC addresses and route from the control plane — which is why auto-discovery works — but a pod IP changes when the pod is rescheduled.

Setting the endpoint

Three write paths, in increasing order of durability:

Auto-discovery. Run Refresh cluster management from the cluster's Settings tab (or bringClusterIntoManagement on first adoption). The capability probe finds the Prometheus pod and records its IP in the cluster's capabilities. Nothing to configure, but it is a pod IP: it goes stale on the next reschedule and needs another refresh.

The dashboard. Set prometheus_endpoint in the cluster's provider config on the cluster Settings tab. Survives everything; set by hand.

The registration command. For an install that manages clusters declaratively:

python manage.py register_tenant_cluster \
  --slug prod-cluster \
  --prometheus-endpoint http://internal-prom-1234.us-west-2.elb.amazonaws.com:9090

or, equivalently, set ASTROLIFT_CLUSTER_PROMETHEUS_ENDPOINT in the control plane's environment — the command runs on every container start and re-asserts it. Omitting both leaves whatever the row already carries, including a probe-discovered value.

A value set in provider config wins over the probe-discovered one.

The app side

Cluster and workload panels work as soon as the endpoint is wired. Traffic, errors and latency for an individual app need a source of request metrics, which is either the cluster's ingress controller or the app's own instrumentation. That contract, and the [workloads.metrics] table that declares it, is documented in the astrolift.toml reference.