K8s CephFS: mount failed: mount failed: fork/exec /usr/bin/systemd-run: invalid argument

在使用pod挂载cephfs挂载pod错误日志如下

Events:
  Type     Reason       Age                   From               Message
  ----     ------       ----                  ----               -------
  Normal   Scheduled    13m                   default-scheduler  Successfully assigned default/nginx-pod1 to k8s-02
  Warning  FailedMount  6m25s (x2 over 10m)   kubelet            Unable to attach or mount volumes: unmounted volumes=[cephfs-pvc], unattached volumes=[kube-api-access-pqznz cephfs-pvc]: timed out waiting for the condition
  Warning  FailedMount  110s (x3 over 8m43s)  kubelet            Unable to attach or mount volumes: unmounted volumes=[cephfs-pvc], unattached volumes=[cephfs-pvc kube-api-access-pqznz]: timed out waiting for the condition
  Warning  FailedMount  42s (x14 over 13m)    kubelet            MountVolume.SetUp failed for volume "cephfs-pv" : CephFS: mount failed: mount failed: fork/exec /usr/bin/systemd-run: invalid argument
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/013c5211-74ee-4c73-b2f3-316bb9c7b80c/volumes/kubernetes.io~cephfs/cephfs-pv --scope -- mount -t ceph -o <masked>,<masked> 192.168.31.21:6789:/ /var/lib/kubelet/pods/013c5211-74ee-4c73-b2f3-316bb9c7b80c/volumes/kubernetes.io~cephfs/cephfs-pv

pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod1
  labels:
    name: nginx-pod1
spec:
  containers:
  - name: nginx-pod1
    image: nginx:alpine
    ports:
    - name: web
      containerPort: 80
    volumeMounts:
    - name: cephfs-pvc
      mountPath: /usr/share/nginx/html
  volumes:
  - name: cephfs-pvc
    persistentVolumeClaim:
      claimName: cephfs-pvc

secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: wu-ceph-secret
data:
  key: QVFCMVJ0eGhIL1llSGhBQXhJRC9KcGxUeXE3N09hMVBzbzdITlE9PQo=

pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: cephfs-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  selector:
    matchLabels:
      pv: cephfs-pv

pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: cephfs-pv
  labels:
    pv: cephfs-pv
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteMany
  cephfs:
    monitors:
      - 192.168.31.21:6789
    user: admin
    secretRef:
      name: wu-ceph-secret
    readOnly: false
  persistentVolumeReclaimPolicy: Delete

这个报错的主要原因为Secret

Kubernetes Secrets必须使用base64编码的Ceph密钥，否则使用 stringData代替 data

• 使用StringData

使用StringData则不需要base64加密，否则admin的key需要使用base64加密

apiVersion: "v1"
kind: "Secret"
metadata:
  name: "ceph-key"
stringData:  # replace `data` with `stringData`, and Kubernetes will store the secret as a literal, instead of decoding Ceph's base64
  key: "YourKey"

• Data (base64加密写法)

[root@k8s-01 home]# echo "AQB1RtxhH/YeHhAAxID/JplTyq77Oa1Pso7HNQ=="|base64
QVFCMVJ0eGhIL1llSGhBQXhJRC9KcGxUeXE3N09hMVBzbzdITlE9PQo=
[root@k8s-01 home]# cat secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: wu-ceph-secret
data:
  key: QVFCMVJ0eGhIL1llSGhBQXhJRC9KcGxUeXE3N09hMVBzbzdITlE9PQo=