Deployment Strategies - Chiến lược triển khai

Deployment Strategies là gì?

Deployment strategies là các phương pháp khác nhau để deploy new versions của applications lên production environments. Mỗi strategy có trade-offs riêng về downtime, risk, complexity, và resource usage.

Các loại Deployment Strategies

1. Recreate Deployment

Định nghĩa: Dừng toàn bộ old version, sau đó start new version.

Đặc điểm: - Có downtime - Đơn giản nhất - Resource requirements thấp - Phù hợp cho development environments

# recreate-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service-recreate
spec:
  replicas: 3
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: user-service
  template:
    metadata:
      labels:
        app: user-service
    spec:
      containers:
      - name: user-service
        image: mycompany/user-service:2.0.0
        ports:
        - containerPort: 8080

2. Rolling Update Deployment

Định nghĩa: Gradually replace old pods với new pods, maintaining service availability.

Đặc điểm: - Zero downtime (nếu cấu hình đúng) - Default strategy trong Kubernetes - Slow rollout process - Mix của old và new versions trong quá trình deployment

# rolling-update-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service-rolling
spec:
  replicas: 6
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1        # Tối đa 1 pod unavailable
      maxSurge: 2              # Tối đa 2 pods extra
  selector:
    matchLabels:
      app: user-service
  template:
    metadata:
      labels:
        app: user-service
        version: v2.0.0
    spec:
      containers:
      - name: user-service
        image: mycompany/user-service:2.0.0
        ports:
        - containerPort: 8080
        readinessProbe:
          httpGet:
            path: /actuator/health/readiness
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        livenessProbe:
          httpGet:
            path: /actuator/health/liveness
            port: 8080
          initialDelaySeconds: 60
          periodSeconds: 30

Rolling Update với Jenkins Pipeline:

pipeline {
    agent any

    stages {
        stage('Deploy Rolling Update') {
            steps {
                script {
                    sh """
                        kubectl set image deployment/user-service \
                            user-service=mycompany/user-service:${BUILD_NUMBER} \
                            --record

                        kubectl rollout status deployment/user-service \
                            --timeout=600s

                        # Verify deployment
                        kubectl get pods -l app=user-service

                        # Health check
                        sleep 30
                        curl -f http://api.company.com/actuator/health
                    """
                }
            }
        }
    }

    post {
        failure {
            script {
                sh "kubectl rollout undo deployment/user-service"
                error "Deployment failed, rolled back to previous version"
            }
        }
    }
}

3. Blue-Green Deployment

Định nghĩa: Maintain hai identical environments (Blue và Green). Deploy to idle environment, sau đó switch traffic.

Đặc điểm: - Zero downtime - Instant rollback capability - Double resource requirements - Full environment testing trước khi switch

# blue-green-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: user-service
  labels:
    app: user-service
spec:
  selector:
    app: user-service
    version: blue  # Switch between 'blue' and 'green'
  ports:
  - port: 80
    targetPort: 8080
---
# Blue Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service-blue
  labels:
    app: user-service
    version: blue
spec:
  replicas: 3
  selector:
    matchLabels:
      app: user-service
      version: blue
  template:
    metadata:
      labels:
        app: user-service
        version: blue
    spec:
      containers:
      - name: user-service
        image: mycompany/user-service:1.0.0
        ports:
        - containerPort: 8080
---
# Green Deployment (New Version)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service-green
  labels:
    app: user-service
    version: green
spec:
  replicas: 3
  selector:
    matchLabels:
      app: user-service
      version: green
  template:
    metadata:
      labels:
        app: user-service
        version: green
    spec:
      containers:
      - name: user-service
        image: mycompany/user-service:2.0.0
        ports:
        - containerPort: 8080

Blue-Green Deployment Script:

#!/bin/bash
# blue-green-deploy.sh

NEW_VERSION=$1
CURRENT_VERSION=$(kubectl get service user-service -o jsonpath='{.spec.selector.version}')

echo "Current version: $CURRENT_VERSION"

if [ "$CURRENT_VERSION" = "blue" ]; then
    NEW_COLOR="green"
    OLD_COLOR="blue"
else
    NEW_COLOR="blue"
    OLD_COLOR="green"
fi

echo "Deploying to $NEW_COLOR environment..."

# Update the deployment with new image
kubectl set image deployment/user-service-${NEW_COLOR} \
    user-service=mycompany/user-service:${NEW_VERSION}

# Wait for rollout to complete
kubectl rollout status deployment/user-service-${NEW_COLOR} --timeout=600s

# Health check on new environment
echo "Performing health checks..."
kubectl port-forward deployment/user-service-${NEW_COLOR} 8080:8080 &
PF_PID=$!
sleep 10

if curl -f http://localhost:8080/actuator/health; then
    echo "Health check passed. Switching traffic to $NEW_COLOR..."

    # Switch service to point to new environment
    kubectl patch service user-service -p '{"spec":{"selector":{"version":"'${NEW_COLOR}'"}}}'

    echo "Traffic switched to $NEW_COLOR environment"

    # Optionally scale down old environment
    # kubectl scale deployment user-service-${OLD_COLOR} --replicas=0

else
    echo "Health check failed. Keeping traffic on $OLD_COLOR"
    exit 1
fi

kill $PF_PID

4. Canary Deployment

Định nghĩa: Deploy new version to small subset of users/traffic, gradually increase nếu metrics look good.

Đặc điểm: - Risk mitigation - Gradual rollout - Real user feedback - Complex traffic management - Requires monitoring

# canary-deployment.yaml
# Stable version (90% traffic)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service-stable
  labels:
    app: user-service
    version: stable
spec:
  replicas: 9  # 90% của traffic
  selector:
    matchLabels:
      app: user-service
      version: stable
  template:
    metadata:
      labels:
        app: user-service
        version: stable
    spec:
      containers:
      - name: user-service
        image: mycompany/user-service:1.0.0
        ports:
        - containerPort: 8080
---
# Canary version (10% traffic)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service-canary
  labels:
    app: user-service
    version: canary
spec:
  replicas: 1  # 10% của traffic
  selector:
    matchLabels:
      app: user-service
      version: canary
  template:
    metadata:
      labels:
        app: user-service
        version: canary
    spec:
      containers:
      - name: user-service
        image: mycompany/user-service:2.0.0
        ports:
        - containerPort: 8080
---
# Service pointing to both versions
apiVersion: v1
kind: Service
metadata:
  name: user-service
spec:
  selector:
    app: user-service  # Both stable and canary
  ports:
  - port: 80
    targetPort: 8080

Istio Canary với Virtual Service:

# istio-canary.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: user-service
spec:
  http:
  - match:
    - headers:
        canary:
          exact: "true"
    route:
    - destination:
        host: user-service
        subset: canary
      weight: 100
  - route:
    - destination:
        host: user-service
        subset: stable
      weight: 90
    - destination:
        host: user-service
        subset: canary
      weight: 10
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: user-service
spec:
  host: user-service
  subsets:
  - name: stable
    labels:
      version: stable
  - name: canary
    labels:
      version: canary

Automated Canary với Flagger:

# flagger-canary.yaml
apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: user-service
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: user-service
  progressDeadlineSeconds: 60
  service:
    port: 80
    targetPort: 8080
  analysis:
    interval: 1m
    threshold: 5
    maxWeight: 50
    stepWeight: 10
    metrics:
    - name: request-success-rate
      thresholdRange:
        min: 99
      interval: 1m
    - name: request-duration
      thresholdRange:
        max: 500
      interval: 30s
    webhooks:
    - name: acceptance-test
      type: pre-rollout
      url: http://flagger-loadtester.test/
      timeout: 30s
      metadata:
        type: bash
        cmd: "curl -sd 'test' http://user-service-canary/api/users/health"
    - name: load-test
      url: http://flagger-loadtester.test/
      timeout: 5s
      metadata:
        cmd: "hey -z 1m -q 10 -c 2 http://user-service-canary/"

5. A/B Testing Deployment

Định nghĩa: Deploy multiple versions simultaneously để test different features với different user groups.

# ab-testing-deployment.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: user-service-ab
spec:
  http:
  - match:
    - headers:
        user-type:
          exact: "premium"
    route:
    - destination:
        host: user-service
        subset: version-a
      weight: 100
  - match:
    - headers:
        region:
          exact: "us-west"
    route:
    - destination:
        host: user-service
        subset: version-b
      weight: 100
  - route:
    - destination:
        host: user-service
        subset: version-a
      weight: 50
    - destination:
        host: user-service
        subset: version-b
      weight: 50

Database Migration Strategies

Backward Compatible Migrations

// Version 1.0.0 - Original schema
@Entity
public class User {
    @Id
    private Long id;
    private String email;
    private String password;
    // getters/setters
}

// Version 2.0.0 - Add new field (backward compatible)
@Entity
public class User {
    @Id
    private Long id;
    private String email;
    private String password;

    @Column(nullable = true)  // Nullable cho backward compatibility
    private String firstName;

    @Column(nullable = true)
    private String lastName;
    // getters/setters
}

Database Migration with Flyway

-- V2__add_user_names.sql
ALTER TABLE users 
ADD COLUMN first_name VARCHAR(100) NULL,
ADD COLUMN last_name VARCHAR(100) NULL;

-- Update existing records with default values
UPDATE users 
SET first_name = 'Unknown', last_name = 'User' 
WHERE first_name IS NULL OR last_name IS NULL;

Zero-Downtime Database Changes

# Phase 1: Deploy app version that can handle both old and new schema
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service-migration-phase1
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: user-service
        image: mycompany/user-service:1.5.0  # Supports both schemas
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: "migration-phase1"

// Application code for migration phase
@Service
public class UserService {

    public User createUser(CreateUserRequest request) {
        User user = new User();
        user.setEmail(request.getEmail());
        user.setPassword(passwordEncoder.encode(request.getPassword()));

        // Handle both old and new schema
        if (request.getFirstName() != null) {
            user.setFirstName(request.getFirstName());
        }
        if (request.getLastName() != null) {
            user.setLastName(request.getLastName());
        }

        return userRepository.save(user);
    }

    // Gracefully handle missing fields
    public UserResponse toResponse(User user) {
        UserResponse response = new UserResponse();
        response.setId(user.getId());
        response.setEmail(user.getEmail());

        // Provide defaults if fields are null
        response.setFirstName(user.getFirstName() != null ? 
            user.getFirstName() : "Unknown");
        response.setLastName(user.getLastName() != null ? 
            user.getLastName() : "User");

        return response;
    }
}

CI/CD Pipeline với Multiple Strategies

GitHub Actions với Strategy Selection

# .github/workflows/deploy.yml
name: Deploy with Strategy

on:
  push:
    branches: [ main ]

env:
  DEPLOYMENT_STRATEGY: ${{ github.event.inputs.strategy || 'rolling' }}

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout
      uses: actions/checkout@v3

    - name: Deploy Rolling Update
      if: env.DEPLOYMENT_STRATEGY == 'rolling'
      run: |
        kubectl set image deployment/user-service \
          user-service=mycompany/user-service:${{ github.sha }}
        kubectl rollout status deployment/user-service

    - name: Deploy Blue-Green
      if: env.DEPLOYMENT_STRATEGY == 'blue-green'
      run: |
        ./scripts/blue-green-deploy.sh ${{ github.sha }}

    - name: Deploy Canary
      if: env.DEPLOYMENT_STRATEGY == 'canary'
      run: |
        kubectl apply -f k8s/canary/
        kubectl set image deployment/user-service-canary \
          user-service=mycompany/user-service:${{ github.sha }}

ArgoCD với Multiple Environments

# argocd-application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: user-service-prod
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/company/user-service
    targetRevision: HEAD
    path: k8s/overlays/production
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

Monitoring Deployments

Prometheus Metrics cho Deployments

// Java application metrics
@Component
public class DeploymentMetrics {

    private final Counter deploymentCounter;
    private final Gauge applicationVersion;
    private final Timer deploymentDuration;

    public DeploymentMetrics(MeterRegistry meterRegistry) {
        this.deploymentCounter = Counter.builder("application.deployments.total")
                .description("Total number of deployments")
                .tag("strategy", "rolling")  // or blue-green, canary
                .register(meterRegistry);

        this.applicationVersion = Gauge.builder("application.version.info")
                .description("Application version information")
                .register(meterRegistry, this, DeploymentMetrics::getVersionNumber);

        this.deploymentDuration = Timer.builder("application.deployment.duration")
                .description("Deployment duration")
                .register(meterRegistry);
    }

    @EventListener
    public void handleDeploymentStart(DeploymentStartEvent event) {
        deploymentCounter.increment();
        // Start timing
    }

    private double getVersionNumber() {
        String version = System.getenv("APP_VERSION");
        // Convert version to numeric for Prometheus
        return version != null ? parseVersion(version) : 0.0;
    }
}

Grafana Dashboard cho Deployments

{
  "dashboard": {
    "title": "Deployment Monitoring",
    "panels": [
      {
        "title": "Deployment Frequency",
        "type": "stat",
        "targets": [
          {
            "expr": "increase(application_deployments_total[24h])",
            "legendFormat": "Deployments/Day"
          }
        ]
      },
      {
        "title": "Deployment Success Rate",
        "type": "stat",
        "targets": [
          {
            "expr": "rate(application_deployments_total{status=\"success\"}[24h]) / rate(application_deployments_total[24h]) * 100",
            "legendFormat": "Success Rate %"
          }
        ]
      },
      {
        "title": "Error Rate During Deployment",
        "type": "graph",
        "targets": [
          {
            "expr": "rate(http_server_requests_seconds_count{status=~\"5..\"}[5m])",
            "legendFormat": "5xx Error Rate"
          }
        ]
      }
    ]
  }
}

Best Practices

1. Strategy Selection

• Development: Recreate (simple, cost-effective)
• Staging: Rolling update (matches production)
• Production: Blue-green hoặc canary (based on risk tolerance)

2. Health Checks

• Implement comprehensive health checks
• Use startup, liveness, và readiness probes
• Include dependency health checks

3. Rollback Strategy

• Always have rollback plan
• Automate rollback triggers
• Test rollback procedures regularly

4. Database Migrations

• Use backward-compatible changes
• Separate schema changes từ application deployment
• Test migrations thoroughly

5. Monitoring và Alerting

• Monitor key metrics during deployments
• Set up alerts cho deployment failures
• Track deployment frequency và success rate

Deployment strategies are critical cho maintaining high availability và reducing deployment risks trong production environments.