Kube-state-metrics：用来监控k8s资源状态的组件

通过监听API Server生成有关资源对象的状态指标 , 比如Deployment、Node、Pod , 需要注意的是kube-state-metrics的使用常见不是用于监控对方是否存活 , 而是用于周期性获取目标对象的metrics指标数据并在web界面进行显示或被prometheus抓取(如pod的状态是running还是Terminating、pod的创建时间等)。目前的kube-state-metrics收集的指标数据可参考官方的文档：https://github.com/kubernetes/kube-state-metrics/tree/master/docs

另外 , kube-state-metrics只是简单的提供一个metrics数据 , 并不会存储这些指标数据 , 所以我们可以使用Prometheus来抓取这些数据然后存储 , 主要关注的是业务相关的一些元数据 , 比如Deployment、Pod、副本状态等 , 以及调度了多少个 replicas？现在可用的有几个？多少个Pod running/stopped/terminated 状态？Pod 重启了多少次？ 目前有多少 job 在运行中。

• github链接 :

  https://github.com/kubernetes/kube-state-metrics

• 镜像 :

  https://hub.docker.com/r/bitnami/kube-state-metrics

  https://quay.io/repository/coreos/kube-state-metrics?tag=latest&tab=tags

• 指标数据 :

  https://github.com/kubernetes/kube-state-metrics/tree/master/docs

目前kube-state-metrics的版本兼容性:

1. 部署kube-state-metrics

整体流程

1. kube-state-metrics从api-server获取k8s集群的数据 , 通过nodeport暴露端口

2. prometheus通过暴露的端从kube-state-metrics拿到监控数据

   注意 : 镜像文件在谷歌镜像仓库 , 也可以在dockerhub官网查找 , 这里使用dokerhub官网的镜像来部署

部署流程

1. 部署kube-state-metrics , 编辑yaml文件

   root@k8s-master01:/opt/k8s-data/yaml/cbear/promethues/prometheus-server# cat kube-state-metrics.yaml 
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: kube-state-metrics
     namespace: kube-system
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: kube-state-metrics
     template:
       metadata:
         labels:
           app: kube-state-metrics
       spec:
         serviceAccountName: kube-state-metrics
         containers:
         - name: kube-state-metrics
           #image: bitnami/kube-state-metrics:2.5.0
           image: harbor.cbear.top/prometheus/kube-state-metrics:v2.12.0
           ports:
           - containerPort: 8080
   ​
   #创建账号 , 授权该账号有权限访问k8s集群
   ---
   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: kube-state-metrics
     namespace: kube-system
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: kube-state-metrics
   rules:
   - apiGroups: [""]
     resources: ["nodes", "pods", "services", "resourcequotas", "replicationcontrollers", "limitranges", "persistentvolumeclaims", "persistentvolumes", "namespaces", "endpoints"]
     verbs: ["list", "watch"]
   - apiGroups: ["extensions"]
     resources: ["daemonsets", "deployments", "replicasets"]
     verbs: ["list", "watch"]
   - apiGroups: ["apps"]
     resources: ["statefulsets"]
     verbs: ["list", "watch"]
   - apiGroups: ["batch"]
     resources: ["cronjobs", "jobs"]
     verbs: ["list", "watch"]
   - apiGroups: ["autoscaling"]
     resources: ["horizontalpodautoscalers"]
     verbs: ["list", "watch"]
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: kube-state-metrics
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: kube-state-metrics
   subjects:
   - kind: ServiceAccount
     name: kube-state-metrics
     namespace: kube-system
   ​
   #创建service , 暴露端口 , 用于在k8s外部访问
   ---
   apiVersion: v1
   kind: Service
   metadata:
     annotations:
       prometheus.io/scrape: 'true'
     name: kube-state-metrics
     namespace: kube-system
     labels:
       app: kube-state-metrics
   spec:
     type: NodePort
     ports:
     - name: kube-state-metrics
       port: 8080
       targetPort: 8080
       nodePort: 31666
       protocol: TCP
     selector:
       app: kube-state-metrics
   ​

2. 启用对应部署资源

   root@k8s-master01:/opt/k8s-data/yaml/cbear/promethues/prometheus-server# kubectl apply -f kube-state-metrics.yaml
   deployment.apps/kube-state-metrics unchanged
   serviceaccount/kube-state-metrics unchanged
   clusterrole.rbac.authorization.k8s.io/kube-state-metrics unchanged
   clusterrolebinding.rbac.authorization.k8s.io/kube-state-metrics unchanged
   service/kube-state-metrics created
   ​

3. 验证: 浏览器访问：192.168.71.35:31666

   

   点击metrics按钮 , 查看数据

   

2. Prometheus配置job采集数据

1. 在prometheus监控主机上修改配置

     - job_name: "prometheus-kube-state-metrics"
       static_configs:
         - targets: ["192.168.71.35:31666"]

2. 重新加载配置

   root@prometheus01:/usr/local/src/prometheus# curl -X POST http://192.168.71.47:9090/-/reload
   ​

3. 查看监控页面

   

3. grafana查看监控数据

在官网找到合适的kube-state-metrics模板导入grafana

1. 模板id为: 13332 导入模板需要注意:

   

   

2. 模板ID: 13824

   

3. 模板ID : 14518

   

参考链接

六、kube-state-metrics 组件介绍 - 袁党生博客 (yuandangsheng.top)