搜索文章 Ctrl K

3-11-3-节点污点与pod容忍度

熊 0 阅读 容器

1. 节点污点基础

节点污点有点类似节点上的标签或注解信息 , 它们都是用来描述对应节点的元数据信息 。污点定义的格式和标签、注解的定义方式很类似 , 都是用一个kv数据来表示 。

不同于节点标签 , 污点的键值数据中包含对应污点的effect , 污点的effect是用于描述对应节点上的污点有什么作用 。 在k8s上污点有三个效用 ( effect ) , 第一个效用是NoSchedule , 表示拒绝pod调度到对应节点上运行 ; 第二个效用是PreferSchedule , 表示尽量不把pod调度到此节点上运行 ; 第三个效用是NoExecute , 表示拒绝将pod调度到此节点上运行 ; 该效用相比NoSchedule要严苛一点 ; 从上面的描述来看 , 对应污点就是来描述拒绝pod运行在对应节点的节点属性 ;

pod对节点污点的容忍度

从字面意思就能够理解 , pod要想运行在对应有污点的节点上 , 对应pod就要容忍对应节点上的污点 。 我们把这种容忍节点污点的定义叫做pod对节点污点的容忍度。 pod对节点污点的容忍度就是在对应pod中定义怎么去匹配节点污点 。通常匹配节点污点的方式有两种 , 一种是等值匹配 , 一种是存在性匹配。

  • 等值匹配表示对应pod的污点容忍度 , 必须和节点上的污点属性相等 , 所谓污点属性是指污点的key、value以及effect , 即容忍度必须满足和对应污点的key , value和effect相同 , 这样表示等值匹配关系 , 其操作符为Equal 。

  • 存在性匹配是指对应容忍度只需要匹配污点的key和effect即可 , value不纳入匹配标准 , 即容忍度只要满足和对应污点的key和effect相同就表示对应容忍度和节点污点是存在性匹配 , 其操作符为Exists ;

节点污点和pod容忍度的关系

kubernetes-kube-scheduler-pod-taints-tolerations-1

如上图所示 , 只有能够容忍对应节点污点的pod才能够被调度到对应节点运行 , 不能容忍节点污点的pod是一定不能调度到对应节点上运行 ( 除节点污点为PreferNoSchedule ) ;

2. 节点污点管理

给节点添加污点命令使用语法格式

Usage:
  kubectl taint NODE NAME KEY_1=VAL_1:TAINT_EFFECT_1 ... KEY_N=VAL_N:TAINT_EFFECT_N [options]

给节点增加污点我们可以用kubectl taint node命令来增加节点污点 , 只需要指定对应节点名称和污点即可 , 污点可以指定多个 , 用空格隔开 ;

示例 : 给node01添加一个test=test:NoSchedule的污点

[root@master01 ~]# kubectl taint node node01.k8s.org test=test:NoSchedule
node/node01.k8s.org tainted
[root@master01 ~]#

查看节点污点

[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taint
Taints:             test=test:NoSchedule
[root@master01 ~]#

删除污点

[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taint
Taints:             test=test:NoSchedule
[root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule-
node/node01.k8s.org untainted
[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taint 
Taints:             <none>
[root@master01 ~]#

删除污点可以指定对应节点上的污点的key和对应污点的effect , 也可以直接在对应污点的key后面加" -" , 表示删除对应名为对应key的所有污点 ;

3. 容忍度定义

3.1 NoSchedule

pod容忍度定义

示例 : 创建一个pod , 其容忍度为对应节点有 node-role.kubernetes.io/master:NoSchedule的污点

[root@master01 ~]# cat pod-demo-taints.yaml
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
  tolerations:
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule

定义pod对节点污点的容忍度需要用tolerations字段定义 , 该字段为一个列表对象 ;

  • key是用来指定对应污点的key , 这个key必须和对应节点污点上的key相等 ;

  • operator字段用于指定对应的操作符 , 即描述容忍度怎么匹配污点 , 这个操作符只有两个 , Equal和Exists 。

  • effect字段用于描述对应的效用 , 该字段的值通常有三个 , NoSchedule、PreferNoSchedule、NoExecute 。

这个字段的值必须和对应的污点相同 ; 上述清单表示 , redis-demo这个pod能够容忍节点上有node-role.kubernetes.io/master:NoSchedule的污点 ;

应用清单

[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo created
[root@master01 ~]# kubectl get pods -o wide
NAME         READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo   1/1     Running   0          7s    10.244.4.35   node04.k8s.org   <none>           <none>
[root@master01 ~]#

可以看到对应pod运行在node04上 。 这里需要注意 , 定义pod容忍度只是表示对应pod可以运行在对应有污点的节点上 , 并非它一定运行在对应节点上 。 它也可以运行在那些没有污点的节点上 。

验证 : 删除pod , 给node01,node02,03,04都打上test:NoSchedule的污点 , 再次应用清单 , 看看对应pod是否能够正常运行

[root@master01 ~]# kubectl delete -f pod-demo-taints.yaml
pod "redis-demo" deleted
[root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule
node/node01.k8s.org tainted
[root@master01 ~]# kubectl taint node node02.k8s.org test:NoSchedule
node/node02.k8s.org tainted
[root@master01 ~]# kubectl taint node node03.k8s.org test:NoSchedule
node/node03.k8s.org tainted
[root@master01 ~]# kubectl taint node node04.k8s.org test:NoSchedule
node/node04.k8s.org tainted
[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taints
Taints:             test:NoSchedule
[root@master01 ~]# kubectl describe node node02.k8s.org |grep Taints
Taints:             test:NoSchedule
[root@master01 ~]# kubectl describe node node03.k8s.org |grep Taints
Taints:             test:NoSchedule
[root@master01 ~]# kubectl describe node node04.k8s.org |grep Taints
Taints:             test:NoSchedule
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo created
[root@master01 ~]# kubectl get pods -o wide
NAME         READY   STATUS    RESTARTS   AGE   IP            NODE               NOMINATED NODE   READINESS GATES
redis-demo   1/1     Running   0          18s   10.244.0.14   master01.k8s.org   <none>           <none>
[root@master01 ~]#

可以看到对应pod , 被调度到master节点上运行了 ; 其原因是对应pod能够容忍master节点上的污点 ; 对应其他node节点上的污点 , 它并不能容忍 , 所以只能运行在master节点 。

删除对应pod中容忍度的定义 , 再次应用pod清单 , 看看对应pod是否会正常运行。

[root@master01 ~]# kubectl delete pod redis-demo
pod "redis-demo" deleted
[root@master01 ~]# cat pod-demo-taints.yaml
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo created
[root@master01 ~]# kubectl get pods -o wide
NAME         READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
redis-demo   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
[root@master01 ~]#

可以看到对应pod处于pending状态 ; 其原因是对应pod没法容忍对应节点污点 ; 即所有节点都排斥对应pod运行在对应节点上 ;

示例 : 定义等值匹配关系污点容忍度

[root@master01 ~]# cat pod-demo-taints.yaml
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
  tolerations:
  - key: test
    operator: Equal
    value: test
    effect: NoSchedule
 
[root@master01 ~]#

定义等值匹配关系的容忍度 , 需要指定对应污点中的value属性 ;

删除原有pod , 应用清单

[root@master01 ~]# kubectl delete pod redis-demo
pod "redis-demo" deleted
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo created
[root@master01 ~]# kubectl get pods -o wide
NAME         READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
redis-demo   0/1     Pending   0          4s    <none>   <none>   <none>           <none>
[root@master01 ~]#

可以看到应用对应清单以后 , pod处于pending状态 , 其原因是没有满足对应pod容忍度的节点 , 所以对应pod无法正常调度到节点上运行 ;

验证 : 修改node01节点的污点为test=test:NoSchedule

[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taints
Taints:             test:NoSchedule
[root@master01 ~]# kubectl taint node node01.k8s.org test=test:NoSchedule --overwrite
node/node01.k8s.org modified
[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taints                
Taints:             test=test:NoSchedule
[root@master01 ~]# kubectl get pods -o wide
NAME         READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo   1/1     Running   0          4m46s   10.244.1.44   node01.k8s.org   <none>           <none>
[root@master01 ~]#

可以看到把node01的污点修改为test=test:NoSchedule以后 , 对应pod就被调度到node01上运行 ;

验证 : 修改node01节点上的污点为test:NoSchedule , 看看对应pod是否被驱离呢

[root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule --overwrite    
node/node01.k8s.org modified
[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taints                
Taints:             test:NoSchedule
[root@master01 ~]# kubectl get pods -o wide
NAME         READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo   1/1     Running   0          7m27s   10.244.1.44   node01.k8s.org   <none>           <none>
[root@master01 ~]#

可以看到对应节点污点修改为test:NoSchedule以后 , 对应pod也不会被驱离 , 说明效用为NoSchedule的污点只是在pod调度时起作用 , 对于调度完成的pod不起作用 ;

3.2 PreferNoSchedule

示例 : 定义pod容忍度为test:PreferNoSchedule

[root@master01 ~]# cat pod-demo-taints.yaml
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo1
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
  tolerations:
  - key: test
    operator: Exists
    effect: PreferNoSchedule
 
[root@master01 ~]#

应用清单

[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo1 created
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo    1/1     Running   0          11m   10.244.1.44   node01.k8s.org   <none>           <none>
redis-demo1   0/1     Pending   0          6s    <none>        <none>           <none>           <none>
[root@master01 ~]#

可以看到对应pod处于pending状态 , 其原因是没有节点污点是test:PerferNoSchedule , 所以对应pod不能被调度运行 ;

给node02节点添加test:PreferNoSchedule污点

[root@master01 ~]# kubectl describe node node02.k8s.org |grep Taints
Taints:             test:NoSchedule
[root@master01 ~]# kubectl taint node node02.k8s.org test:PreferNoSchedule
node/node02.k8s.org tainted
[root@master01 ~]# kubectl describe node node02.k8s.org |grep -A 1 Taints
Taints:             test:NoSchedule
                    test:PreferNoSchedule
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo    1/1     Running   0          18m     10.244.1.44   node01.k8s.org   <none>           <none>
redis-demo1   0/1     Pending   0          6m21s   <none>        <none>           <none>           <none>
[root@master01 ~]#

可以看到对应node02上有两个污点 , 对应pod也没有正常运行起来 , 其原因是node02上有一个test:NoSchedule污点 , 对应pod容忍度不能容忍此类污点 ;

验证 : 修改node01,node03 , node04上的节点污点为test:PreferNoSchedule , 修改pod的容忍度为test:NoSchedule , 再次应用清单 , 看看对应pod怎么调度

[root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule-    
node/node01.k8s.org untainted
[root@master01 ~]# kubectl taint node node03.k8s.org test:NoSchedule-
node/node03.k8s.org untainted
[root@master01 ~]# kubectl taint node node04.k8s.org test:NoSchedule-
node/node04.k8s.org untainted
[root@master01 ~]# kubectl taint node node01.k8s.org test:PreferNoSchedule
node/node01.k8s.org tainted
[root@master01 ~]# kubectl taint node node03.k8s.org test:PreferNoSchedule 
node/node03.k8s.org tainted
[root@master01 ~]# kubectl taint node node04.k8s.org test:PreferNoSchedule
node/node04.k8s.org tainted
[root@master01 ~]# kubectl describe node node01.k8s.org |grep -A 1 Taints
Taints:             test:PreferNoSchedule
Unschedulable:      false
[root@master01 ~]# kubectl describe node node02.k8s.org |grep -A 1 Taints
Taints:             test:NoSchedule
                    test:PreferNoSchedule
[root@master01 ~]# kubectl describe node node03.k8s.org |grep -A 1 Taints
Taints:             test:PreferNoSchedule
Unschedulable:      false
[root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints
Taints:             test:PreferNoSchedule
Unschedulable:      false
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo    1/1     Running   0          31m   10.244.1.44   node01.k8s.org   <none>           <none>
redis-demo1   1/1     Running   0          19m   10.244.1.45   node01.k8s.org   <none>           <none>
[root@master01 ~]# kubectl delete pod --all
pod "redis-demo" deleted
pod "redis-demo1" deleted
[root@master01 ~]# cat pod-demo-taints.yaml
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo1
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
  tolerations:
  - key: test
    operator: Exists
    effect: NoSchedule
 
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo1 created
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo1   1/1     Running   0          5s    10.244.4.36   node04.k8s.org   <none>           <none>
[root@master01 ~]#

从上面的验证过程来看 , 当我们把node01 , node03 , node04节点上的污点删除以后 , 刚才创建的redis-demo1 pod被调度到node01上运行了 。 其原因是node01上的污点第一个被删除 ; 但我们把pod的容忍对修改成test:NoSchedule以后 , 再次应用清单 , 对应pod被调度到node04上运行。 这意味着NoSchedule效用污点容忍度是可以正常容忍PreferNoSchedule污点 ;

3.3 NoExecute

示例 : 定义pod容忍度为test:NoExecute

[root@master01 ~]# cat pod-demo-taints.yaml
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo2
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
  tolerations:
  - key: test
    operator: Exists
    effect: NoExecute
[root@master01 ~]#

应用清单

[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo2 created
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo1   1/1     Running   0          35m   10.244.4.36   node04.k8s.org   <none>           <none>
redis-demo2   1/1     Running   0          5s    10.244.4.38   node04.k8s.org   <none>           <none>
[root@master01 ~]#

可以看到对应pod被调度到node04上运行 , 说明容忍效用为NoExecute能够容忍污点效用为PreferNoSchedule的节点 ;

验证 : 更改所有node节点污点为test:NoSchedule , 删除原有pod , 再次应用清单 , 看看对应pod是否还会正常运行

[root@master01 ~]# kubectl taint node node01.k8s.org test-
node/node01.k8s.org untainted
[root@master01 ~]# kubectl taint node node02.k8s.org test-
node/node02.k8s.org untainted
[root@master01 ~]# kubectl taint node node03.k8s.org test-
node/node03.k8s.org untainted
[root@master01 ~]# kubectl taint node node04.k8s.org test-
node/node04.k8s.org untainted
[root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule
node/node01.k8s.org tainted
[root@master01 ~]# kubectl taint node node02.k8s.org test:NoSchedule
node/node02.k8s.org tainted
[root@master01 ~]# kubectl taint node node03.k8s.org test:NoSchedule
node/node03.k8s.org tainted
[root@master01 ~]# kubectl taint node node04.k8s.org test:NoSchedule
node/node04.k8s.org tainted
[root@master01 ~]# kubectl describe node node01.k8s.org |grep -A 1 Taints
Taints:             test:NoSchedule
Unschedulable:      false
[root@master01 ~]# kubectl describe node node02.k8s.org |grep -A 1 Taints
Taints:             test:NoSchedule
Unschedulable:      false
[root@master01 ~]# kubectl describe node node03.k8s.org |grep -A 1 Taints
Taints:             test:NoSchedule
Unschedulable:      false
[root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints
Taints:             test:NoSchedule
Unschedulable:      false
[root@master01 ~]# kubectl delete pod --all
pod "redis-demo1" deleted
pod "redis-demo2" deleted
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo2 created
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
redis-demo2   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
[root@master01 ~]#

可以看到对应pod处于pending状态 , 说明pod容忍效用为NoExecute , 并不能容忍污点效用为NoSchedule ;

删除pod , 修改所有节点污点为test:NoExecute , 把pod容忍度修改为NoScheudle , 然后应用清单 , 看看对应pod怎么调度

[root@master01 ~]# kubectl delete pod --all
pod "redis-demo2" deleted
[root@master01 ~]# kubectl taint node node01.k8s.org test-              
node/node01.k8s.org untainted
[root@master01 ~]# kubectl taint node node02.k8s.org test-
node/node02.k8s.org untainted
[root@master01 ~]# kubectl taint node node03.k8s.org test-
node/node03.k8s.org untainted
[root@master01 ~]# kubectl taint node node04.k8s.org test-
node/node04.k8s.org untainted
[root@master01 ~]# kubectl taint node node01.k8s.org test:NoExecute
node/node01.k8s.org tainted
[root@master01 ~]# kubectl taint node node02.k8s.org test:NoExecute
node/node02.k8s.org tainted
[root@master01 ~]# kubectl taint node node03.k8s.org test:NoExecute
node/node03.k8s.org tainted
[root@master01 ~]# kubectl taint node node04.k8s.org test:NoExecute
node/node04.k8s.org tainted
[root@master01 ~]# kubectl describe node node01.k8s.org |grep -A 1 Taints
Taints:             test:NoExecute
Unschedulable:      false
[root@master01 ~]# kubectl describe node node02.k8s.org |grep -A 1 Taints
Taints:             test:NoExecute
Unschedulable:      false
[root@master01 ~]# kubectl describe node node03.k8s.org |grep -A 1 Taints
Taints:             test:NoExecute
Unschedulable:      false
[root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints
Taints:             test:NoExecute
Unschedulable:      false
[root@master01 ~]# cat pod-demo-taints.yaml
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo2
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
  tolerations:
  - key: test
    operator: Exists
    effect: NoSchedule
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo2 created
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
redis-demo2   0/1     Pending   0          8s    <none>   <none>   <none>           <none>
[root@master01 ~]#

从上面的演示来看 , pod容忍度效用为NoSchedule也不能容忍污点效用为NoExecute ;

删除pod , 修改对应pod的容忍度为test:NoExecute

[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE    IP       NODE     NOMINATED NODE   READINESS GATES
redis-demo2   0/1     Pending   0          5m5s   <none>   <none>   <none>           <none>
[root@master01 ~]# kubectl delete pod --all
pod "redis-demo2" deleted
[root@master01 ~]# cat pod-demo-taints.yaml
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo2
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
  tolerations:
  - key: test
    operator: Exists
    effect: NoExecute
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo2 created
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo2   1/1     Running   0          6s    10.244.4.43   node04.k8s.org   <none>           <none>
[root@master01 ~]#

修改node04节点污点为test:NoSchedule , 看看对应pod是否可以正常运行

[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo2   1/1     Running   0          4m38s   10.244.4.43   node04.k8s.org   <none>           <none>
[root@master01 ~]# kubectl taint node node04.k8s.org test-
node/node04.k8s.org untainted
[root@master01 ~]# kubectl get pods -o wide              
NAME          READY   STATUS    RESTARTS   AGE    IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo2   1/1     Running   0          8m2s   10.244.4.43   node04.k8s.org   <none>           <none>
[root@master01 ~]# kubectl taint node node04.k8s.org test:NoSchedule
node/node04.k8s.org tainted
[root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints
Taints:             test:NoSchedule
Unschedulable:      false
[root@master01 ~]# kubectl get pods -o wide                             
NAME          READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo2   1/1     Running   0          8m25s   10.244.4.43   node04.k8s.org   <none>           <none>
[root@master01 ~]#

从NoExecute更改为NoSchedule , 对原有pod不会进行驱离 ;

修改pod的容忍度为test:NoSchedule , 再次应用清单

[root@master01 ~]# cat pod-demo-taints.yaml
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo3
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
  tolerations:
  - key: test
    operator: Exists
    effect: NoSchedule
---
apiVersion: v1
kind: Pod
metadata:
  name: redis-demo4
  labels:
    app: db
spec:
  containers:
  - name: redis
    image: redis:4-alpine
    ports:
    - name: redis
      containerPort: 6379
  tolerations:
  - key: test
    operator: Exists
    effect: NoSchedule
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml
pod/redis-demo3 created
pod/redis-demo4 created
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo2   1/1     Running   0          14m   10.244.4.43   node04.k8s.org   <none>           <none>
redis-demo3   1/1     Running   0          4s    10.244.4.45   node04.k8s.org   <none>           <none>
redis-demo4   1/1     Running   0          4s    10.244.4.46   node04.k8s.org   <none>           <none>
[root@master01 ~]#

可以看到后面两个pod都被调度node04上运行 ;其原因是对应pod的容忍度test:NoSchedule只能容忍node04上的污点test:NoSchedule ;

修改node04的污点为NoExecute , 看看对应pod是否会被驱离

[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo2   1/1     Running   0          17m     10.244.4.43   node04.k8s.org   <none>           <none>
redis-demo3   1/1     Running   0          2m32s   10.244.4.45   node04.k8s.org   <none>           <none>
redis-demo4   1/1     Running   0          2m32s   10.244.4.46   node04.k8s.org   <none>           <none>
[root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints
Taints:             test:NoSchedule
Unschedulable:      false
[root@master01 ~]# kubectl taint node node04.k8s.org test-
node/node04.k8s.org untainted
[root@master01 ~]# kubectl taint node node04.k8s.org test:NoExecute
node/node04.k8s.org tainted
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS        RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo2   1/1     Running       0          18m     10.244.4.43   node04.k8s.org   <none>           <none>
redis-demo3   0/1     Terminating   0          3m43s   10.244.4.45   node04.k8s.org   <none>           <none>
redis-demo4   0/1     Terminating   0          3m43s   10.244.4.46   node04.k8s.org   <none>           <none>
[root@master01 ~]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-demo2   1/1     Running   0          18m   10.244.4.43   node04.k8s.org   <none>           <none>
[root@master01 ~]#

可以看到修改node04的污点为test:NoExecute以后 , 对应pod容忍污点效用为不是NoExecute的pod被驱离了;说明污点效用为NoExecute , 它会驱离不能容忍该污点效用的所有pod ;

创建一个deploy , 其指定容器的容忍度为test:NoExecute , 并指定其驱离延迟施加为10秒

[root@master01 ~]# cat deploy-demo-taint.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-demo
spec:
  replicas: 3
  selector:
     matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:4-alpine
        ports:
        - name: redis
          containerPort: 6379
      tolerations:
      - key: test
        operator: Exists
        effect: NoExecute
        tolerationSeconds: 10

tolerationSeconds字段用于指定其驱离宽限其时长 ; 该字段只能用在其容忍污点效用为NoExecute的容忍度中使用 ; 其他污点效用不能使用该字段来指定其容忍宽限时长 ;

应用配置清单

[root@master01 ~]# kubectl apply -f deploy-demo-taint.yaml
deployment.apps/deploy-demo created
[root@master01 ~]# kubectl get pods -o wide -w
NAME                           READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
deploy-demo-79b89f9847-9zk8j   1/1     Running   0          7s    10.244.2.71   node02.k8s.org   <none>           <none>
deploy-demo-79b89f9847-h8zlc   1/1     Running   0          7s    10.244.3.61   node03.k8s.org   <none>           <none>
deploy-demo-79b89f9847-shscr   1/1     Running   0          7s    10.244.1.62   node01.k8s.org   <none>           <none>
redis-demo2                    1/1     Running   0          54m   10.244.4.43   node04.k8s.org   <none>           <none>
deploy-demo-79b89f9847-h8zlc   1/1     Terminating   0          10s   10.244.3.61   node03.k8s.org   <none>           <none>
deploy-demo-79b89f9847-shscr   1/1     Terminating   0          10s   10.244.1.62   node01.k8s.org   <none>           <none>
deploy-demo-79b89f9847-2x8w6   0/1     Pending       0          0s    <none>        <none>           <none>           <none>
deploy-demo-79b89f9847-2x8w6   0/1     Pending       0          0s    <none>        node03.k8s.org   <none>           <none>
deploy-demo-79b89f9847-lhltv   0/1     Pending       0          0s    <none>        <none>           <none>           <none>
deploy-demo-79b89f9847-9zk8j   1/1     Terminating   0          10s   10.244.2.71   node02.k8s.org   <none>           <none>
deploy-demo-79b89f9847-2x8w6   0/1     ContainerCreating   0          0s    <none>        node03.k8s.org   <none>           <none>
deploy-demo-79b89f9847-lhltv   0/1     Pending             0          0s    <none>        node02.k8s.org   <none>           <none>
deploy-demo-79b89f9847-lhltv   0/1     ContainerCreating   0          0s    <none>        node02.k8s.org   <none>           <none>
deploy-demo-79b89f9847-w8xjw   0/1     Pending             0          0s    <none>        <none>           <none>           <none>
deploy-demo-79b89f9847-w8xjw   0/1     Pending             0          0s    <none>        node01.k8s.org   <none>           <none>
deploy-demo-79b89f9847-w8xjw   0/1     ContainerCreating   0          0s    <none>        node01.k8s.org   <none>           <none>
deploy-demo-79b89f9847-shscr   1/1     Terminating         0          10s   10.244.1.62   node01.k8s.org   <none>           <none>
deploy-demo-79b89f9847-h8zlc   1/1     Terminating         0          10s   10.244.3.61   node03.k8s.org   <none>           <none>
deploy-demo-79b89f9847-9zk8j   1/1     Terminating         0          10s   10.244.2.71   node02.k8s.org   <none>           <none>
deploy-demo-79b89f9847-shscr   0/1     Terminating         0          11s   10.244.1.62   node01.k8s.org   <none>           <none>
deploy-demo-79b89f9847-2x8w6   0/1     ContainerCreating   0          1s    <none>        node03.k8s.org   <none>           <none>
deploy-demo-79b89f9847-lhltv   0/1     ContainerCreating   0          1s    <none>        node02.k8s.org   <none>           <none>
deploy-demo-79b89f9847-w8xjw   0/1     ContainerCreating   0          1s    <none>        node01.k8s.org   <none>           <none>
deploy-demo-79b89f9847-h8zlc   0/1     Terminating         0          11s   10.244.3.61   node03.k8s.org   <none>           <none>
deploy-demo-79b89f9847-2x8w6   1/1     Running             0          1s    10.244.3.62   node03.k8s.org   <none>           <none>
deploy-demo-79b89f9847-9zk8j   0/1     Terminating         0          11s   10.244.2.71   node02.k8s.org   <none>           <none>
deploy-demo-79b89f9847-lhltv   1/1     Running             0          1s    10.244.2.72   node02.k8s.org   <none>           <none>
deploy-demo-79b89f9847-w8xjw   1/1     Running             0          2s    10.244.1.63   node01.k8s.org   <none>           <none>
deploy-demo-79b89f9847-h8zlc   0/1     Terminating         0          15s   10.244.3.61   node03.k8s.org   <none>           <none>
deploy-demo-79b89f9847-h8zlc   0/1     Terminating         0          15s   10.244.3.61   node03.k8s.org   <none>           <none>
^C[root@master01 ~]#

可以看到对应pod只能在对应节点上运行10秒,随后就被驱离,因为我们创建的是一个deploy,对应pod被驱离以后,对应deploy又会重建;

总结 :

  • 对于污点效用为NoSchedule来说 , 它只会拒绝新建的pod , 不会对原有pod进行驱离 ; 如果对应pod能够容忍该污点 , 则对应pod就有可能运行在对应节点上 ; 如果不能容忍 , 则对应pod一定不会调度到对应节点运行 。

  • 对于污点效用为PreferNoSchedule来说 , 它也不会驱离已存在pod , 它只有在所有节点都不满足对应pod容忍度时 , 对应pod可以勉强运行在此类污点效用的节点上

  • 对于污点效用为NoExecute来说 , 默认不指定其容忍宽限时长 , 表示能够一直容忍 , 如果指定了其宽限时长 , 则到了宽限时长对应pod将会被驱离 ; 对应之前被调度到该节点上的pod , 在节点污点效用变为NoExecute后 , 该节点会立即驱离所有不能容忍污点效用为NoExecute的pod ;

参考链接

容器编排系统K8s之节点污点和pod容忍度 - Linux-1874 - 博客园 (cnblogs.com)