[kubernetes] 쿠버네티스 볼륨(Volume) - 동적 프로비저닝(dynamic Provisioning) / nfs

쿠버네티스 동적 프로비저닝(dynamic Provisioning)

nfs dynamic provisioner 구성

https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner

git clone https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner.git

cd nfs-subdir-external-provisioner/deploy

rbac 편집

$ cat rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

kubectl create -f rbac.yaml

deployment.yaml 편집(deployment 생성)

vi deployment.yaml

$ cat deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: k8s-sigs.io/nfs-subdir-external-provisioner
            - name: NFS_SERVER
              value: 192.168.0.59
            - name: NFS_PATH
              value: /nfs-directory/shares-data1
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.0.59
            path: /nfs-directory/shares-data1

kubectl create -f deployment.yaml

storageclass 생성

$ cat class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-client
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

kubectl create -f class.yaml

storageclass 확인

kubectl get storageclasses.storage.k8s.io

$ kubectl get storageclasses.storage.k8s.io 
NAME         PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-client   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate           false                  10m

---

mypvc-dynamic.yaml 편집

vim mypvc-dynamic.yaml

cat mypvc-dynamic.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mypvc-dynamic
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1G
  storageClassName: 'nfs-client'

kubectl create -f mypvc-dynamic.yaml

persistentvolume, persistentvolumeclaims 확인

kubectl get pv,pvc

$ kubectl get pv,pvc
NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                   STORAGECLASS   REASON   AGE
persistentvolume/pvc-a4c7787a-1cd5-425b-8c3d-af91ed5ffb24   1G         RWX            Delete           Bound    default/mypvc-dynamic   nfs-client              9m57s

NAME                                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/mypvc-dynamic   Bound    pvc-a4c7787a-1cd5-425b-8c3d-af91ed5ffb24   1G         RWX            nfs-client     9m57s

pods 볼륨 정보 확인

$ kubectl describe pods nfs-client-provisioner-6866666554-zkbmj
Name:         nfs-client-provisioner-6866666554-zkbmj
Namespace:    default
Priority:     0
Node:         kube-node2/192.168.0.62
Start Time:   Fri, 21 Oct 2022 15:30:39 +0900
Labels:       app=nfs-client-provisioner
              pod-template-hash=6866666554
Annotations:  cni.projectcalico.org/containerID: bfad904743e54bd22918868878c347f705cae0c5559f2b9456910703e7250e5f
              cni.projectcalico.org/podIP: 10.233.74.2/32
              cni.projectcalico.org/podIPs: 10.233.74.2/32
Status:       Running
IP:           10.233.74.2
IPs:
  IP:           10.233.74.2
Controlled By:  ReplicaSet/nfs-client-provisioner-6866666554
Containers:
  nfs-client-provisioner:
    Container ID:   docker://0c53bbbb5af05abb57e5ba3543a160f032e61325c9a28d30ccbfa0db54808fec
    Image:          k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
    Image ID:       docker-pullable://k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner@sha256:63d5e04551ec8b5aae83b6f35938ca5ddc50a88d85492d9731810c31591fa4c9
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Fri, 21 Oct 2022 15:30:51 +0900
    Ready:          True
    Restart Count:  0
    Environment:
      PROVISIONER_NAME:  k8s-sigs.io/nfs-subdir-external-provisioner
      NFS_SERVER:        192.168.0.59
      NFS_PATH:          /nfs-directory/shares-data1
    Mounts:
      /persistentvolumes from nfs-client-root (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-pmwbc (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  nfs-client-root:
    Type:      NFS (an NFS mount that lasts the lifetime of a pod)
    Server:    192.168.0.59
    Path:      /nfs-directory/shares-data1
    ReadOnly:  false
  kube-api-access-pmwbc:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  36m   default-scheduler  Successfully assigned default/nfs-client-provisioner-6866666554-zkbmj to kube-node2
  Normal  Pulling    36m   kubelet            Pulling image "k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2"
  Normal  Pulled     36m   kubelet            Successfully pulled image "k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2" in 5.953792563s
  Normal  Created    36m   kubelet            Created container nfs-client-provisioner
  Normal  Started    36m   kubelet            Started container nfs-client-provisioner

nfs server 디렉터리 목록

- default-mypvc-dynamic-pvc-a4c7787a-1cd5-425b-8c3d-af91ed5ffb24

 

참고URL

- https://velog.io/@repush/Kubernetes-NFS%EB%A5%BC-%EC%9D%B4%EC%9A%A9%ED%95%9C-%EB%8F%99%EC%A0%81%ED%94%84%EB%A1%9C%EB%B9%84%EC%A0%80%EB%8B%9D