Production Best Practices

This guide outlines enterprise-grade security, scalability, and reliability patterns for deploying QMigrator in production environments using Helm on Kubernetes.

1. Secret Manager Synchronization

Integrate a secret manager with Kubernetes for secure, centralized credential management.

Use Official Cloud Secret Managers

Use the Secrets Store CSI Driver with the official cloud provider integration:

• Azure Key Vault provider
• AWS Secrets Manager provider
• Google Secret Manager provider

Secret Object Sync

Create a SecretProviderClass to fetch values from your cloud secret manager and sync to a Kubernetes Secret object.

  apiVersion: secrets-store.csi.x-k8s.io/v1
  kind: SecretProviderClass
  metadata:
    name: qmigrator-secrets
    namespace: qmigrator
  spec:
    provider: <azure|aws|gcp>
    parameters: # (1)!
      # Parameters go here.
      objects: | # (2)!
        # Object format goes here.
    secretObjects:
      - secretName: qmig-secret
        type: Opaque
        data:
        - key: postgres-password
          objectName: postgres-password
        - key: redis-password
          objectName: redis-password
        - key: airflow-secret-key
          objectName: airflow-secret-key
        - key: airflow-fernet-key
          objectName: airflow-fernet-key
        - key: airflow-password
          objectName: airflow-password
        - key: connection
          objectName: connection

1. Configure provider-specific fields in spec.parameters for your cloud driver.
2. Configure provider-specific object format and object names in spec.parameters.objects.

Mount the CSI volume in a workload so secret sync is triggered:

app:
  extraVolumes:
  - name: qmig-secret
    csi:
      driver: secrets-store.csi.k8s.io
      readOnly: true
      volumeAttributes:
        secretProviderClass: "qmigrator-secrets" # (1)!
  extraVolumeMounts:
    - name: qmig-secret
      mountPath: "/mnt/secrets-store"
      readOnly: true

1. Must match the SecretProviderClass name defined in secret-sync.yaml.

Update Helm Values for Synced Secrets

secret:
  create: false
  secretName: "qmig-secret"  # (1)!

airflow:
  secret:
    create: false
    secretName: "qmig-secret"

1. This secret is synced from the cloud secret manager

2. Image Security & Registry Configuration

Private Registry Authentication

Use pre-created Docker pull secrets:

kubectl create secret docker-registry qmig-docker \
  -n qmigrator \
  --docker-server=<registry-server> \
  --docker-username='<username>' \
  --docker-password='<password>'

Reference in values:

imageCredentials:
  create: false
  secretName: "qmig-docker" # (1)!

global:
  imagePullSecrets:
    - name: qmig-docker

1. Use the same pre-created registry secret name in both imageCredentials.secretName and global.imagePullSecrets.

Image Pull Policy

images:
  app:
    pullPolicy: IfNotPresent  # (1)!
  eng:
    pullPolicy: IfNotPresent

1. Prefer cached; pull if missing

3. Kubernetes Network Policies

Implement zero-trust networking by defining explicit ingress rules.

Network Policy Architecture

Enable NetworkPolicy in Helm values:

networkPolicies:
  enabled: true

NetworkPolicy Pattern

Use this sample as a base and adapt matchLabels, namespaces, and ports for your deployment:

<component>: 
  networkPolicy:
    ingress:
      from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: gateway-system # (1)!
        - podSelector:
            matchLabels:
              app.kubernetes.io/name: qmigrator # (2)!

1. Match the namespace label of your Gateway API or Ingress controller namespace.
2. Match labels used by QMigrator pods allowed to call this component.

4. Resource Requests and Limits

Define CPU and memory allocations to ensure cluster stability and prevent resource contention.

Resource Pattern

<component>:
  resources:
    requests:
      cpu: 250m
      memory: 512Mi
    limits:
      cpu: 1
      memory: 2Gi

Resource Sizing Guidance

kubectl top pods -n qmigrator

5. Pod Security Standards

Enforce container security best practices at pod creation time.

PSS (Pod Security Standards) Configuration

Add labels to enforce PSS in the namespace:

kubectl label namespace qmigrator \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/enforce-version=latest \
  pod-security.kubernetes.io/warn=restricted

Secure Pod Configuration in values.yaml

Use securityContext and podSecurityContext to run containers with reduced privileges, drop unnecessary Linux capabilities, and keep filesystem ownership consistent across mounted volumes.

<component>:
  securityContext:
    allowPrivilegeEscalation: false
    readOnlyRootFilesystem: true
    capabilities:
      drop:
        - ALL

  podSecurityContext:
    runAsNonRoot: true
    runAsUser: 5000 # (1)!
    runAsGroup: 4000 # (2)!
    fsGroup: 4000
    seccompProfile:
      type: RuntimeDefault

1. Default QMigrator runtime user ID (UID): 5000.
2. Default QMigrator runtime group ID (GID): 4000.

6. High Availability & Disaster Recovery

Pod Disruption Budgets

Protect critical components during node maintenance:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: qmigrator-app-pdb
  namespace: qmigrator
spec:
  minAvailable: 1
  selector:
    matchLabels:
      component: app

---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: qmigrator-db-pdb
  namespace: qmigrator
spec:
  minAvailable: 1
  selector:
    matchLabels:
      component: db

Pod Affinity for Resilience

Spread components across nodes:

<component>:
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions: # (1)!
                - key: component
                  operator: In
                  values:
                    - <component>
            topologyKey: kubernetes.io/hostname

1. Match component labels consistently so anti-affinity spreads replicas across nodes.

7. Cost Optimization

Resource Quotas by Namespace

apiVersion: v1
kind: ResourceQuota
metadata:
  name: qmigrator-quota
  namespace: qmigrator
spec:
  hard:
    requests.cpu: "8"
    requests.memory: "16Gi"
    limits.cpu: "16"
    limits.memory: "32Gi"
    pods: "50"
    services: "10"
    persistentvolumeclaims: "10"

Use Lower-Cost Node Pools for Non-Critical Workloads

For non-critical workloads, schedule to a dedicated lower-cost node pool:

<component>:
  nodeSelector:
    workload-tier: "batch"

8. Deployment Checklist

Before going to production, verify:

• Secrets Store CSI Driver and cloud provider plugin configured
• All secrets stored in secret manager (no plaintext in values.yaml)
• NetworkPolicies enabled and tested for all components
• Resource requests/limits configured for all pods
• Pod Security Standards enforced on namespace
• RBAC roles created with least-privilege permissions
• Image pull secrets created and referenced
• Container images scanned for vulnerabilities
• Pod Disruption Budgets created for critical components
• Pod affinity rules configured for resilience
• Ingress/Gateway API TLS certificates installed
• Cluster and node autoscaling enabled

Reference Links

• Kubernetes Network Policies
• Pod Security Standards
• Secrets Store CSI Driver