[kubernetes] 쿠버네티스 볼륨(Volume) - 정적 프로비저닝(Static Provisioning) / nfs

쿠버네티스 정적 프로비저닝(Static Provisioning)

nfs 서버 구성(kube-control1)

 - nfs 서버 구성 : https://sangchul.kr/432

nfs 클라이언트 구성(kube-node1, kube-node2, kube-node3)

sudo apt install -y nfs-common

showmount -e 192.168.56.11

$ showmount -e 192.168.56.11
Export list for 192.168.56.11:
/nfs-directory/shares-data1 192.168.56.0/24

PV(Persistent Volume) 생성

nfs-pv1.yaml 파일 작성

vim nfs-pv1.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mypv-data
  labels:
    name: pv-data
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteMany
  nfs:
    path: /nfs-directory/shares-data1
    server: 192.168.56.11
  storageClassName: nfs-data

$ kubectl create -f nfs-pv1.yaml
persistentvolume/mypv created

kubectl get pv

$ kubectl get persistentvolume
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                STORAGECLASS   REASON   AGE
mypv-data   1Gi        RWX            Retain           Bound    default/mypvc-data   nfs-data                64m

kubectl describe pv mypv-data

$ kubectl describe pv mypv-data
Name:            mypv-data
Labels:          name=pv-data
Annotations:     pv.kubernetes.io/bound-by-controller: yes
Finalizers:      [kubernetes.io/pv-protection]
StorageClass:    nfs-data
Status:          Bound
Claim:           default/mypvc-data
Reclaim Policy:  Retain
Access Modes:    RWX
VolumeMode:      Filesystem
Capacity:        1Gi
Node Affinity:   <none>
Message:
Source:
    Type:      NFS (an NFS mount that lasts the lifetime of a pod)
    Server:    192.168.56.11
    Path:      /nfs-directory/shares-data1
    ReadOnly:  false
Events:        <none>

PVC(Persistent Volume Claim) 생성

nfs-pvc1.yaml 파일 작성

vim nfs-pvc1.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mypvc-data
spec:
  resources:
    requests:
      storage: 1Gi
  volumeName: mypv-data
  accessModes:
    - ReadWriteMany
  storageClassName: nfs-data

$ kubectl create -f nfs-pvc1.yaml
persistentvolumeclaim/mypvc created

kubectl get pv,pvc

$ kubectl get persistentvolume,persistentvolumeclaims
NAME                         CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                STORAGECLASS   REASON   AGE
persistentvolume/mypv-data   1Gi        RWX            Retain           Bound    default/mypvc-data   nfs-data                65m

NAME                               STATUS   VOLUME      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/mypvc-data   Bound    mypv-data   1Gi        RWX            nfs-data       64m

Deployment 생성(pods)

nfs-pvc-deployment1.yaml 파일 작성

vim nfs-pvc-deployment1.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: nginx:latest
        resources:
          limits:
            memory: "128Mi"
            cpu: "500m"
        ports:
        - containerPort: 80
          protocol: TCP
        volumeMounts:
          - name: pvctest
            mountPath: /usr/share/nginx/html
      volumes:
        - name: pvctest
          persistentVolumeClaim:
            claimName: mypvc-data

kubectl create -f nfs-pvc-deployment1.yaml

$ kubectl create -f nfs-pvc-deployment1.yaml
deployment.apps/myapp created

 

kubectl get all,pv,pvc

$ kubectl get all,pv,pvc
NAME                         READY   STATUS    RESTARTS   AGE
pod/myapp-69f8fb45c5-q4sj2   1/1     Running   0          7m53s
pod/myapp-69f8fb45c5-q97zx   1/1     Running   0          7m53s
pod/myapp-69f8fb45c5-t2lxx   1/1     Running   0          7m53s

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.233.0.1   <none>        443/TCP   5d12h

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/myapp   3/3     3            3           7m53s

NAME                               DESIRED   CURRENT   READY   AGE
replicaset.apps/myapp-69f8fb45c5   3         3         3       7m53s

NAME                         CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                STORAGECLASS   REASON   AGE
persistentvolume/mypv-data   1Gi        RWX            Retain           Bound    default/mypvc-data   nfs-data                68m

NAME                               STATUS   VOLUME      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/mypvc-data   Bound    mypv-data   1Gi        RWX            nfs-data       67m

 

kubectl describe pvc mypvc-data

$ kubectl describe pvc mypvc-data
Name:          mypvc-data
Namespace:     default
StorageClass:  nfs-data
Status:        Bound
Volume:        mypv-data
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed: yes
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      1Gi
Access Modes:  RWX
VolumeMode:    Filesystem
Used By:       myapp-69f8fb45c5-q4sj2
               myapp-69f8fb45c5-q97zx
               myapp-69f8fb45c5-t2lxx
Events:        <none>

 

kubectl describe pods myapp-69f8fb45c5-q4sj2

$ kubectl describe pods myapp-69f8fb45c5-q4sj2
Name:         myapp-69f8fb45c5-q4sj2
Namespace:    default
Priority:     0
Node:         kube-node1/192.168.56.21
Start Time:   Thu, 13 Oct 2022 11:02:15 +0900
Labels:       app=myapp
              pod-template-hash=69f8fb45c5
Annotations:  cni.projectcalico.org/containerID: 3108e7abd11c0d1205aa6b6fe67b7c131cfa50179ac445a13ad0234912adbfdf
              cni.projectcalico.org/podIP: 10.233.73.73/32
              cni.projectcalico.org/podIPs: 10.233.73.73/32
Status:       Running
IP:           10.233.73.73
IPs:
  IP:           10.233.73.73
Controlled By:  ReplicaSet/myapp-69f8fb45c5
Containers:
  myapp:
    Container ID:   docker://2feda232a779adf75ad72bb8e68177cb8ea044c0d4feb8dc71d51f7dad416db9
    Image:          nginx:latest
    Image ID:       docker-pullable://nginx@sha256:2f770d2fe27bc85f68fd7fe6a63900ef7076bc703022fe81b980377fe3d27b70
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 13 Oct 2022 11:02:20 +0900
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     500m
      memory:  128Mi
    Requests:
      cpu:        500m
      memory:     128Mi
    Environment:  <none>
    Mounts:
      /usr/share/nginx/html from pvctest (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-flcrb (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  pvctest:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  mypvc-data
    ReadOnly:   false
  kube-api-access-flcrb:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Guaranteed
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  8m43s  default-scheduler  Successfully assigned default/myapp-69f8fb45c5-q4sj2 to kube-node1
  Normal  Pulling    8m41s  kubelet            Pulling image "nginx:latest"
  Normal  Pulled     8m38s  kubelet            Successfully pulled image "nginx:latest" in 2.545929469s
  Normal  Created    8m38s  kubelet            Created container myapp
  Normal  Started    8m37s  kubelet            Started container myapp

 

출처URL-http://ranjeetbadhe.com/wp-content/uploads/2021/06/word-image-61.png

 

 

퍼시스턴트볼륨(PV) 및 퍼시스턴트클레임(PVC)

출처URL-https://blog.eunsukim.me/static/media/kubernetes_volume/4.jpeg

PV 동적 프로비저닝 워크플로우

출처URL-https://blog.eunsukim.me/static/media/kubernetes_volume/7.jpeg

Reclaim Policy(반환 정책)

$ kubectl explain PersistentVolume.spec.persistentVolumeReclaimPolicy
KIND:     PersistentVolume
VERSION:  v1

FIELD:    persistentVolumeReclaimPolicy <string>

DESCRIPTION:
     persistentVolumeReclaimPolicy defines what happens to a persistent volume
     when released from its claim. Valid options are Retain (default for
     manually created PersistentVolumes), Delete (default for dynamically
     provisioned PersistentVolumes), and Recycle (deprecated). Recycle must be
     supported by the volume plugin underlying this PersistentVolume. More info:
     https://kubernetes.io/docs/concepts/storage/persistent-volumes#reclaiming

     Possible enum values:
     - `"Delete"` means the volume will be deleted from Kubernetes on release
     from its claim. The volume plugin must support Deletion.
     - `"Recycle"` means the volume will be recycled back into the pool of
     unbound persistent volumes on release from its claim. The volume plugin
     must support Recycling.
     - `"Retain"` means the volume will be left in its current phase (Released)
     for manual reclamation by the administrator. The default policy is Retain.

• Retain(데이터 유지) : 수동 반환
• Recycle(데이터 삭제 및 PV 재사용) : 기본 스크럼
• Delete(데이터 및 PV 삭제) : aws ebs, ... 등 볼륨과 같은 스토리지 자산이 삭제됨

 

Access Modes

$ kubectl explain PersistentVolumeClaim.spec.accessModes
KIND:     PersistentVolumeClaim
VERSION:  v1

FIELD:    accessModes <[]string>

DESCRIPTION:
     accessModes contains the desired access modes the volume should have. More
     info:
     https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1

• RWO - ReadWriteOnce : 노드 하나에만 볼륨을 읽기/쓰기하도록 마운트할 수 있음
• ROX - ReadOnlyMany : 여러 개 노드에서 읽기 전용으로 마운트할 수 있음
• RWX - ReadWriteMany : 여러 개 노드에서 읽기/쓰기 가능하도록 마운프할 수 있음
• RWOP - ReadWriteOncePod

 

 

 

 

참고URL

- https://kubernetes.io/ko/docs/concepts/storage/persistent-volumes/

- https://blog.eunsukim.me/posts/kubernetes-volume-overview

- https://kubernetes.io/docs/concepts/storage/persistent-volumes/#reclaiming

- https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes