Prometheus

De facto standard for cloud native monitoring

Graduated project of CNCF

Internals

Exporters offer metrics for retrieval

Prometheus scrapes data from exporters

Metrics are stored in a time series database (TSDB)

Query language PromQL

Cluster metrics

All components provide a metrics endpoint:

kube-apiserver
coredns
kube-controller-manager
kube-proxy
kube-scheduler
kubelet

Prometheus should scrape all of them…

…but does not out-of-the-box

Scrape configuration

Configure Prometheus to scrape targets

Read from app with metrics endpoints
Read from central exporter which reads from app
Read from app with integrated exporter sidecar

scrape_configs:
- job_name: 'example-target-1'
  scrape_interval: 5s
  static_configs:
    - targets: ['localhost:8090']
      labels:
        group: 'dummy'
- job_name: 'example-target-2'
  metrics_path: /api/metrics
  static_configs:
    - targets: ['localhost:8080']
      labels:
        group: 'dummy-2'