Skip to content

Persistent Volumes

This guide describes the volumes required for QMigrator on Kubernetes.

QMigrator requires two types of storage:
1. Shared Storage: Shared across applications (ReadWriteMany).
2. Block Storage: Dedicated storage for the database and cache (ReadWriteOnce).

Note

  • Each manifest is tailored for its respective cloud provider's shared file system (e.g., EFS for AWS, Azure Files for Azure, GCS Bucket for GCP).
  • Review and customize the parameters as needed for your cluster setup.

Shared Persistent Volumes

This section contains example static PersistentVolume configurations for various cloud environments.
Each YAML manifest includes the StorageClass, PersistentVolume, and PersistentVolumeClaim for its cloud provider.

Azure

  • Create a Kubernetes secret for your Azure file share credentials: 
    kubectl create secret generic fileshare-secret -n <namespace> \
  --from-literal=azurestorageaccountname=<storage_account_name> \
  --from-literal=azurestorageaccountkey=<storage_key>

Note

Replace <storage_account_name> and <storage_key> with your Azure Storage account name and access key.

  • Update the parameters for the Azure file share in azure-pv.yaml: 
    csi:
  driver: file.csi.azure.com
  readOnly: false
  volumeHandle: qmig-shared-pv0
  volumeAttributes:
    resourceGroup: {{AZURE_RG}} # Resource Group
    shareName: {{AZURE_FILESHARE}} # Fileshare name
  nodeStageSecretRef:
    name: {{STG_KEY_SECRET_NAME}} # Storage account key secret name
    namespace: {{Namespace}} # Namespace of secret

AWS

Note

Only applicable to managed EKS, not EKS Auto Mode.

  • Update the EFS filesystem ID from AWS in aws-pv.yaml: 
    csi:
  driver: efs.csi.aws.com
  volumeHandle: {{FileSystemId}} # Volume ID from EFS system

Google Cloud

  • Update the GCP bucket name in gcp-pv.yaml: 
    csi:
  driver: gcsfuse.csi.storage.gke.io
  volumeHandle: {{BUCKET_NAME}} # GCP Bucket name
  readOnly: false

Tip

Ensure the service account used by Pods has read-write permission on the bucket, and update the pod annotation for each deployment, statefulset, cronjob, pod, etc.

annotations:
  gke-gcsfuse/volumes: "true"

Minikube

Note

{LOCAL_PATH} is the local drive path of your machine.

minikube start --mount --mount-string={LOCAL_PATH}:/hostpc
  • Update the local path to be mounted in minikube-pv.yaml: 
    hostPath:
  path: /hostpc

Docker Desktop

  • Update the local drive path to be mounted: 
    hostPath:
  path: {{LOCAL_PATH}} # Path to be mounted from local

Note

{LOCAL_PATH} is the local drive path of your machine.

Tip

You may need to use the WSL mounted path for your C: drive as /run/desktop/mnt/host/c/.

Create and Reference a Persistent Volume Claim

  1. Choose the appropriate YAML file for your environment (e.g., azure-pv.yaml, gcp-pv.yaml, minikube-pv.yaml, etc.).

  2. Review and update the manifest to match your environment (e.g., file system IDs, storage size, access modes).

  3. Ensure the storageClassName is consistent across the StorageClass, PV, and PVC sections.

  4. Apply the manifest:

    kubectl apply -f aws-pv.yaml
    (Replace aws-pv.yaml with the file for your provider, e.g., azure-pv.yaml, gcp-pv.yaml, minikube-pv.yaml, etc.)

  5. In your Pod or Deployment YAML, reference the PVC by name:

Example

volumes:
  - name: shared-storage
    persistentVolumeClaim:
      claimName: <your-pvc-name>

References