k8s storageclass 提示External provisioner is provisioning volume for claim “prometheus/prometheus-data-db”

后端存储使用ceph，创建好storageclass。当我们创建pvc时 pvc一直处于pending状态

pvc详细信息如下

Events:
  Type     Reason                Age                  From                                                                                              Message
  ----     ------                ----                 ----                                                                                              -------
  Normal   Provisioning          54m (x8 over 55m)    rbd.csi.ceph.com_csi-rbdplugin-provisioner-7557ccdb7c-4tr8b_374c8879-defc-4de3-be7c-635cf4597fec  External provisioner is provisioning volume for claim "prometheus/prometheus-data-db"
  Warning  ProvisioningFailed    54m (x8 over 55m)    rbd.csi.ceph.com_csi-rbdplugin-provisioner-7557ccdb7c-4tr8b_374c8879-defc-4de3-be7c-635cf4597fec  failed to provision volume with StorageClass "csi-rbd-sc": rpc error: code = InvalidArgument desc = multi node access modes are only supported on rbd `block` type volumes
  Warning  ProvisioningFailed    35m (x14 over 53m)   rbd.csi.ceph.com_csi-rbdplugin-provisioner-7557ccdb7c-8wr96_339cd07c-b2a6-4252-b5c9-7944dd766784  failed to provision volume with StorageClass "csi-rbd-sc": rpc error: code = InvalidArgument desc = multi node access modes are only supported on rbd `block` type volumes
  Normal   ExternalProvisioning  38s (x222 over 55m)  persistentvolume-controller                                                                       waiting for a volume to be created, either by external provisioner "rbd.csi.ceph.com" or manually created by system administrator
  Normal   Provisioning          8s (x23 over 53m)    rbd.csi.ceph.com_csi-rbdplugin-provisioner-7557ccdb7c-8wr96_339cd07c-b2a6-4252-b5c9-7944dd766784  External provisioner is provisioning volume for claim "prometheus/prometheus-data-db"

原因:

RBD: This driver is optimized for RWO pod access where only one pod may access the storage
CephFS: This driver allows for RWX with one or more pods accessing the same storage

CephFS模式是使用RWX模式的正确选择。

我们需要将pvc的访问模式修改为ReadWriteOnce

• RWO: ReadWriteOnce，只有单个节点可以挂载这个volume，进行读写操作；
• ROX: ReadOnlyMany，多个节点可以挂载这个volume，只能进行读操作；
• RWX: ReadWriteMany，多个节点可以挂载这个volume，读写操作都是允许的。

解决方法

[root@k8s-01 prometheus]# cat prometheus-pvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: prometheus-data-db
  namespace: prometheus
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi
  storageClassName: csi-rbd-sc

#删除pending的pvc
[root@k8s-01 prometheus]# kubectl delete -f prometheus-pvc.yaml 
persistentvolumeclaim "prometheus-data-db" deleted

#创建pvc
[root@k8s-01 prometheus]# kubectl apply -f prometheus-pvc.yaml 
persistentvolumeclaim/prometheus-data-db created

#查看pvc状态
[root@k8s-01 prometheus]# kubectl get pvc -n prometheus 
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
grafana-data         Bound    pvc-dcfd7eb6-319c-4713-b27c-5410ae8b88ab   50Gi       RWO            csi-rbd-sc     2d
prometheus-data-db   Bound    pvc-af481771-d39b-4ca5-9083-ac7b4ee97e76   50Gi       RWO            csi-rbd-sc     5s