kube-state-metrics forbidden: User “system:serviceaccount:ops-monit:kube-state-metrics” cannot list resource “leases” in API group “coordination.k8s.io” in the namespace “kube-node-lease”

释放双眼,带上耳机,听听看~!

完整日志如下

  • kube-state-metrics版本:v2.2.1
  • kubernetes版本: 1.24
[root@k8s-01 kube-state-metrics_v2.2.1]# kubectl get pod -n ops-monit 
NAME                                  READY   STATUS    RESTARTS   AGE
kube-state-metrics-86788854d7-8f9k7   1/1     Running   0          2m46s
[root@k8s-01 kube-state-metrics_v2.2.1]# kubectl  logs -f -n ops-monit kube-state-metrics-86788854d7-8f9k7 
I0104 07:57:38.070753       1 main.go:106] Using default resources
I0104 07:57:38.070821       1 main.go:118] Using all namespace
I0104 07:57:38.070831       1 main.go:139] metric allow-denylisting: Excluding the following lists that were on denylist: 
W0104 07:57:38.070857       1 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0104 07:57:38.072007       1 main.go:241] Testing communication with server
I0104 07:57:38.079680       1 main.go:246] Running with Kubernetes cluster version: v1.23. git version: v1.23.5. git tree state: clean. commit: c285e781331a3785a7f436042c65c5641ce8a9e9. platform: linux/amd64
I0104 07:57:38.079708       1 main.go:248] Communication with server successful
I0104 07:57:38.080013       1 main.go:204] Starting metrics server: [::]:8080
I0104 07:57:38.080079       1 metrics_handler.go:96] Autosharding disabled
I0104 07:57:38.080091       1 main.go:193] Starting kube-state-metrics self metrics server: [::]:8081
I0104 07:57:38.080332       1 main.go:64] levelinfomsgTLS is disabled.http2false
I0104 07:57:38.080433       1 main.go:64] levelinfomsgTLS is disabled.http2false
I0104 07:57:38.082027       1 builder.go:190] Active resources: certificatesigningrequests,configmaps,cronjobs,daemonsets,deployments,endpoints,horizontalpodautoscalers,ingresses,jobs,leases,limitranges,mutatingwebhookconfigurations,namespaces,networkpolicies,nodes,persistentvolumeclaims,persistentvolumes,poddisruptionbudgets,pods,replicasets,replicationcontrollers,resourcequotas,secrets,services,statefulsets,storageclasses,validatingwebhookconfigurations,volumeattachments
E0104 07:57:38.086849       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Lease: failed to list *v1.Lease: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
W0104 07:57:38.086849       1 warnings.go:70] batch/v1beta1 CronJob is deprecated in v1.21+, unavailable in v1.25+; use batch/v1 CronJob
E0104 07:57:38.086877       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
W0104 07:57:38.087449       1 warnings.go:70] autoscaling/v2beta2 HorizontalPodAutoscaler is deprecated in v1.23+, unavailable in v1.26+; use autoscaling/v2 HorizontalPodAutoscaler
W0104 07:57:38.089428       1 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W0104 07:57:38.101204       1 warnings.go:70] batch/v1beta1 CronJob is deprecated in v1.21+, unavailable in v1.25+; use batch/v1 CronJob
W0104 07:57:38.101763       1 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W0104 07:57:38.101823       1 warnings.go:70] autoscaling/v2beta2 HorizontalPodAutoscaler is deprecated in v1.23+, unavailable in v1.26+; use autoscaling/v2 HorizontalPodAutoscaler
E0104 07:57:39.211317       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
E0104 07:57:39.505111       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Lease: failed to list *v1.Lease: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
E0104 07:57:41.055579       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
E0104 07:57:41.512130       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Lease: failed to list *v1.Lease: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
E0104 07:57:45.043887       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Lease: failed to list *v1.Lease: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
E0104 07:57:45.739010       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
E0104 07:57:55.278498       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
E0104 07:57:56.275135       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Lease: failed to list *v1.Lease: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
E0104 07:58:08.868356       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
E0104 07:58:21.172822       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Lease: failed to list *v1.Lease: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
E0104 07:58:41.467028       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
E0104 07:58:55.861658       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Lease: failed to list *v1.Lease: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
E0104 07:59:18.418467       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
E0104 07:59:30.452752       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Lease: failed to list *v1.Lease: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
E0104 07:59:53.679931       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
E0104 08:00:05.600141       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1.Lease: failed to list *v1.Lease: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:ops-monit:kube-state-metrics" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"

解决办法

给kube-state-metrics赋权cluster-admin

执行如下命令,给system:serviceaccount:ops-monit:kube-state-metrics做clusterrolebing

  • ops-monit 为命名空间,安装在不同namespace也记得修改
kubectl create clusterrolebinding kube-state-metrics-admin-binding \
--clusterrole=cluster-admin  \
--user=system:serviceaccount:ops-monit:kube-state-metrics

执行完最好重启下

[root@k8s-01 kube-state-metrics_v2.2.1]# kubectl get pod -n ops-monit 
NAME                                  READY   STATUS    RESTARTS   AGE
kube-state-metrics-86788854d7-8f9k7   1/1     Running   0          14m
[root@k8s-01 kube-state-metrics_v2.2.1]# kubectl delete pod -n ops-monit kube-state-metrics-86788854d7-8f9k7 
pod "kube-state-metrics-86788854d7-8f9k7" deleted

给TA打赏
共{{data.count}}人
人已打赏
报错锦集

ceph集群提示daemons have recently crashed

2023-1-3 18:18:37

KubernetesZabbix

zabbix监控Pod Restart状态

2019-1-14 18:53:05

0 条回复 A文章作者 M管理员
    暂无讨论,说说你的看法吧
个人中心
购物车
优惠劵
今日签到
有新私信 私信列表
搜索