Prometheus로 EKS Control Plane Metric 확인

Tech Note 정보
카테고리: [ Amazon Web Services ]
조회수: 37553

1. 개요

EKS + Prometheus를 하는 방법

2. 사전작업

  • eksctl 설치
  • kubectl 설치
  • eksctl로 클러스터 생성
  • helm 설치

3. Raw Metric 데이터 확인

$ kubectl get --raw /metrics |more
# HELP APIServiceOpenAPIAggregationControllerQueue1_adds (Deprecated) Total number of adds handled by workqueue: APIServiceOpenAPIAggregationControllerQueue1
# TYPE APIServiceOpenAPIAggregationControllerQueue1_adds counter
APIServiceOpenAPIAggregationControllerQueue1_adds 2850
# HELP APIServiceOpenAPIAggregationControllerQueue1_depth (Deprecated) Current depth of workqueue: APIServiceOpenAPIAggregationControllerQueue1
# TYPE APIServiceOpenAPIAggregationControllerQueue1_depth gauge
APIServiceOpenAPIAggregationControllerQueue1_depth 0
# HELP APIServiceOpenAPIAggregationControllerQueue1_longest_running_processor_microseconds (Deprecated) How many microseconds has the longest running processor for APIServiceOpenAPIAggregationControllerQueue1 bee
n running.
# TYPE APIServiceOpenAPIAggregationControllerQueue1_longest_running_processor_microseconds gauge
APIServiceOpenAPIAggregationControllerQueue1_longest_running_processor_microseconds 0
# HELP APIServiceOpenAPIAggregationControllerQueue1_queue_latency (Deprecated) How long an item stays in workqueueAPIServiceOpenAPIAggregationControllerQueue1 before being requested.
# TYPE APIServiceOpenAPIAggregationControllerQueue1_queue_latency summary
APIServiceOpenAPIAggregationControllerQueue1_queue_latency{quantile="0.5"} 68
APIServiceOpenAPIAggregationControllerQueue1_queue_latency{quantile="0.9"} 110
APIServiceOpenAPIAggregationControllerQueue1_queue_latency{quantile="0.99"} 136
APIServiceOpenAPIAggregationControllerQueue1_queue_latency_sum 160097
APIServiceOpenAPIAggregationControllerQueue1_queue_latency_count 2850

4. Prometheus 설치

$ kubectl create namespace prometheus
namespace/prometheus created
$ helm install stable/prometheus --name prometheus --namespace prometheus --set alertmanager.persistentVolume.storageClass="gp2",server.persistentVolume.storageClass="gp2",server.service.type=LoadBalancer
NAME:   prometheus
LAST DEPLOYED: Tue Nov 19 08:02:16 2019
NAMESPACE: prometheus
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME                     AGE
prometheus-alertmanager  0s
prometheus-server        0s
(생략)
$ kubectl get svc -o wide --namespace prometheus
NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE   SELECTOR
prometheus-alertmanager         ClusterIP      10.100.122.48           80/TCP         36s   app=prometheus,component=alertmanager,release=prometheus
prometheus-kube-state-metrics   ClusterIP      None                    80/TCP         36s   app=prometheus,component=kube-state-metrics,release=prometheus
prometheus-node-exporter        ClusterIP      None                    9100/TCP       36s   app=prometheus,component=node-exporter,release=prometheus
prometheus-pushgateway          ClusterIP      10.100.20.24            9091/TCP       36s   app=prometheus,component=pushgateway,release=prometheus
prometheus-server               LoadBalancer   10.100.3.139            80:30276/TCP   36s   app=prometheus,component=server,release=prometheus