home / skills / manutej / luxor-claude-marketplace / ci-cd-pipeline-patterns

This skill guides building, testing, and deploying robust CI/CD pipelines with GitHub Actions, automating patterns, and optimizing release management.

npx playbooks add skill manutej/luxor-claude-marketplace --skill ci-cd-pipeline-patterns

Review the files below or copy the command above to add this skill to your agents.

Files (3)
SKILL.md
34.4 KB
---
name: ci-cd-pipeline-patterns
description: Comprehensive CI/CD pipeline patterns skill covering GitHub Actions, workflows, automation, testing, deployment strategies, and release management for modern software delivery
---

# CI/CD Pipeline Patterns

A comprehensive skill for designing, implementing, and optimizing CI/CD pipelines using GitHub Actions and modern DevOps practices. Master workflow automation, testing strategies, deployment patterns, and release management for continuous software delivery.

## When to Use This Skill

Use this skill when:

- Setting up continuous integration and deployment pipelines for projects
- Automating build, test, and deployment workflows
- Implementing multi-environment deployment strategies (staging, production)
- Managing release automation and versioning
- Configuring matrix builds for multi-platform testing
- Securing CI/CD pipelines with secrets and OIDC
- Optimizing pipeline performance with caching and parallelization
- Building containerized applications with Docker in CI
- Deploying to cloud platforms (AWS, Azure, GCP, Vercel, Netlify)
- Implementing infrastructure as code with Terraform/CloudFormation
- Setting up monorepo CI/CD patterns
- Creating reusable workflow templates and custom actions
- Implementing deployment strategies (blue-green, canary, rolling)
- Automating changelog generation and semantic versioning
- Integrating quality gates and code coverage checks

## Core Concepts

### CI/CD Fundamentals

**Continuous Integration (CI)**: Automatically building and testing code changes as developers commit to the repository.

**Continuous Deployment (CD)**: Automatically deploying code changes to production after passing tests.

**Continuous Delivery**: Keeping code in a deployable state, with manual approval for production deployment.

### GitHub Actions Architecture

GitHub Actions provides event-driven automation directly integrated with your repository.

#### Workflows

YAML files in `.github/workflows/` that define automated processes:

```yaml
name: CI Pipeline
on: [push, pull_request]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build project
        run: npm run build
```

**Key Components:**
- **name**: Human-readable workflow name
- **on**: Events that trigger the workflow (push, pull_request, schedule, workflow_dispatch)
- **jobs**: Collection of steps that run in sequence or parallel
- **runs-on**: The runner environment (ubuntu-latest, windows-latest, macos-latest)

#### Jobs

Groups of steps executed on the same runner:

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm test

  deploy:
    needs: test  # Runs after 'test' job completes
    runs-on: ubuntu-latest
    steps:
      - run: npm run deploy
```

**Job Features:**
- **needs**: Define job dependencies (sequential execution)
- **if**: Conditional execution based on expressions
- **strategy**: Matrix builds for multiple configurations
- **outputs**: Share data between jobs
- **environment**: Deployment environments with protection rules

#### Steps

Individual tasks within a job:

```yaml
steps:
  - name: Checkout code
    uses: actions/checkout@v4

  - name: Setup Node.js
    uses: actions/setup-node@v4
    with:
      node-version: '20'

  - name: Install dependencies
    run: npm ci

  - name: Run tests
    run: npm test
```

**Step Types:**
- **uses**: Run a pre-built action from marketplace or repository
- **run**: Execute shell commands
- **with**: Provide inputs to actions
- **env**: Set environment variables for the step

#### Actions

Reusable units of code that perform specific tasks:

**Official Actions:**
- `actions/checkout@v4`: Check out repository code
- `actions/setup-node@v4`: Setup Node.js environment
- `actions/cache@v4`: Cache dependencies
- `actions/upload-artifact@v4`: Upload build artifacts
- `actions/download-artifact@v4`: Download artifacts from previous jobs

**Marketplace Actions:**
- `docker/build-push-action@v5`: Build and push Docker images
- `aws-actions/configure-aws-credentials@v4`: Configure AWS credentials
- `codecov/codecov-action@v4`: Upload code coverage
- `google-github-actions/auth@v2`: Authenticate with Google Cloud

#### Secrets and Variables

**Secrets**: Encrypted sensitive data (API keys, credentials, tokens)

```yaml
steps:
  - name: Deploy to production
    env:
      API_KEY: ${{ secrets.API_KEY }}
      DATABASE_URL: ${{ secrets.DATABASE_URL }}
    run: npm run deploy
```

**Variables**: Non-sensitive configuration data

```yaml
env:
  NODE_ENV: ${{ vars.NODE_ENV }}
  API_ENDPOINT: ${{ vars.API_ENDPOINT }}
```

**Secret Types:**
- **Repository secrets**: Available to all workflows in a repository
- **Environment secrets**: Scoped to specific environments (production, staging)
- **Organization secrets**: Shared across repositories in an organization

#### Artifacts

Files produced by workflows that can be downloaded or used by other jobs:

```yaml
- name: Upload build artifacts
  uses: actions/upload-artifact@v4
  with:
    name: dist-files
    path: dist/
    retention-days: 7

- name: Download artifacts
  uses: actions/download-artifact@v4
  with:
    name: dist-files
    path: ./dist
```

### Workflow Triggers

#### Event Triggers

**Push Events:**
```yaml
on:
  push:
    branches:
      - main
      - develop
      - 'release/**'
    paths:
      - 'src/**'
      - 'package.json'
    tags:
      - 'v*'
```

**Pull Request Events:**
```yaml
on:
  pull_request:
    types: [opened, synchronize, reopened]
    branches:
      - main
    paths-ignore:
      - 'docs/**'
      - '**.md'
```

**Schedule (Cron):**
```yaml
on:
  schedule:
    - cron: '0 0 * * *'  # Daily at midnight UTC
    - cron: '0 */6 * * *'  # Every 6 hours
```

**Manual Triggers (workflow_dispatch):**
```yaml
on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy to'
        required: true
        type: choice
        options:
          - staging
          - production
      version:
        description: 'Version to deploy'
        required: true
        type: string
```

**Release Events:**
```yaml
on:
  release:
    types: [published, created, released]
```

**Workflow Call (Reusable Workflows):**
```yaml
on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
    secrets:
      api-key:
        required: true
```

### Matrix Builds

Run jobs across multiple configurations in parallel:

```yaml
jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node-version: [18, 20, 22]
        include:
          - os: ubuntu-latest
            node-version: 20
            coverage: true
        exclude:
          - os: macos-latest
            node-version: 18
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm test
      - if: matrix.coverage
        run: npm run coverage
```

**Matrix Features:**
- **Parallel execution**: All combinations run simultaneously
- **include**: Add specific configurations
- **exclude**: Remove specific combinations
- **fail-fast**: Stop all jobs if one fails (default: true)
- **max-parallel**: Limit concurrent jobs

### Caching Strategies

Speed up workflows by caching dependencies:

**Node.js Caching:**
```yaml
- uses: actions/setup-node@v4
  with:
    node-version: '20'
    cache: 'npm'  # Automatically caches npm dependencies
```

**Custom Caching:**
```yaml
- uses: actions/cache@v4
  with:
    path: |
      ~/.npm
      ~/.cache
      node_modules
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-
```

**Docker Layer Caching:**
```yaml
- uses: docker/build-push-action@v5
  with:
    context: .
    cache-from: type=gha
    cache-to: type=gha,mode=max
```

## Testing Strategies in CI

### Unit Testing

Fast, isolated tests for individual components:

```yaml
jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run unit tests
        run: npm run test:unit -- --coverage

      - name: Upload coverage
        uses: codecov/codecov-action@v4
        with:
          files: ./coverage/coverage-final.json
          flags: unit-tests
          token: ${{ secrets.CODECOV_TOKEN }}
```

### Integration Testing

Test interactions between components and services:

```yaml
jobs:
  integration-tests:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432

      redis:
        image: redis:7
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 6379:6379

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Run database migrations
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost:5432/testdb
        run: npm run migrate

      - name: Run integration tests
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost:5432/testdb
          REDIS_URL: redis://localhost:6379
        run: npm run test:integration
```

### End-to-End Testing

Test complete user workflows:

```yaml
jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Build application
        run: npm run build

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run E2E tests
        run: npm run test:e2e

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30
```

### Performance Testing

Benchmark and performance regression testing:

```yaml
jobs:
  performance-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Build for production
        run: npm run build

      - name: Run Lighthouse CI
        uses: treosh/lighthouse-ci-action@v11
        with:
          urls: |
            http://localhost:3000
            http://localhost:3000/dashboard
          uploadArtifacts: true
          temporaryPublicStorage: true

      - name: Run load tests
        run: npm run test:load
```

### Code Quality and Linting

Enforce code standards and quality gates:

```yaml
jobs:
  code-quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Run ESLint
        run: npm run lint

      - name: Run Prettier check
        run: npm run format:check

      - name: Run TypeScript check
        run: npm run type-check

      - name: Run security audit
        run: npm audit --audit-level=moderate

      - name: SonarCloud Scan
        uses: SonarSource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
```

## Deployment Patterns

### Blue-Green Deployment

Zero-downtime deployment by maintaining two identical environments:

```yaml
jobs:
  deploy-blue-green:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy to Green environment
        run: |
          # Deploy new version to green environment
          ./deploy.sh green

      - name: Run smoke tests on Green
        run: |
          # Verify green environment is healthy
          curl -f https://green.example.com/health

      - name: Switch traffic to Green
        run: |
          # Update load balancer to point to green
          aws elbv2 modify-rule --rule-arn $RULE_ARN \
            --actions Type=forward,TargetGroupArn=$GREEN_TG

      - name: Monitor Green environment
        run: |
          # Monitor for 5 minutes
          ./monitor.sh green 300

      - name: Rollback if needed
        if: failure()
        run: |
          # Switch back to blue
          aws elbv2 modify-rule --rule-arn $RULE_ARN \
            --actions Type=forward,TargetGroupArn=$BLUE_TG
```

### Canary Deployment

Gradual rollout to a subset of users:

```yaml
jobs:
  canary-deployment:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy canary (10% traffic)
        run: |
          kubectl set image deployment/app app=myapp:${{ github.sha }}
          kubectl scale deployment/app-canary --replicas=1
          kubectl annotate service app-service \
            traffic-split='{"canary": 10, "stable": 90}'

      - name: Monitor canary metrics
        run: |
          # Monitor error rates, latency for 15 minutes
          ./monitor-canary.sh 900

      - name: Increase canary traffic (50%)
        run: |
          kubectl annotate service app-service \
            traffic-split='{"canary": 50, "stable": 50}' --overwrite

      - name: Monitor again
        run: ./monitor-canary.sh 600

      - name: Full rollout (100%)
        run: |
          kubectl set image deployment/app-stable app=myapp:${{ github.sha }}
          kubectl scale deployment/app-canary --replicas=0

      - name: Rollback canary
        if: failure()
        run: |
          kubectl scale deployment/app-canary --replicas=0
```

### Rolling Deployment

Sequential update of instances:

```yaml
jobs:
  rolling-deployment:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy with rolling update
        run: |
          kubectl set image deployment/app \
            app=myapp:${{ github.sha }} \
            --record

      - name: Wait for rollout to complete
        run: |
          kubectl rollout status deployment/app --timeout=10m

      - name: Verify deployment
        run: |
          kubectl get pods -l app=myapp
          curl -f https://api.example.com/health

      - name: Rollback on failure
        if: failure()
        run: |
          kubectl rollout undo deployment/app
```

### Multi-Environment Deployment

Deploy to staging, then production with approvals:

```yaml
jobs:
  deploy-staging:
    runs-on: ubuntu-latest
    environment:
      name: staging
      url: https://staging.example.com
    steps:
      - uses: actions/checkout@v4
      - name: Deploy to staging
        run: ./deploy.sh staging

  deploy-production:
    needs: deploy-staging
    runs-on: ubuntu-latest
    environment:
      name: production
      url: https://example.com
    steps:
      - uses: actions/checkout@v4
      - name: Deploy to production
        run: ./deploy.sh production
```

## Security Best Practices

### Secret Management

**Using GitHub Secrets:**
```yaml
steps:
  - name: Configure AWS credentials
    uses: aws-actions/configure-aws-credentials@v4
    with:
      aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
      aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      aws-region: us-east-1
```

**Environment-Scoped Secrets:**
```yaml
jobs:
  deploy:
    environment: production  # Uses production-scoped secrets
    steps:
      - name: Deploy
        env:
          API_KEY: ${{ secrets.PRODUCTION_API_KEY }}
        run: ./deploy.sh
```

### OIDC (OpenID Connect)

Authenticate without long-lived credentials:

```yaml
jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials (OIDC)
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
          aws-region: us-east-1

      - name: Deploy to AWS
        run: aws s3 sync ./dist s3://my-bucket
```

**Google Cloud OIDC:**
```yaml
- name: Authenticate to Google Cloud
  uses: google-github-actions/auth@v2
  with:
    workload_identity_provider: 'projects/123/locations/global/workloadIdentityPools/pool/providers/provider'
    service_account: '[email protected]'
```

### Secure Workflows

**Restrict permissions:**
```yaml
permissions:
  contents: read      # Read repository contents
  pull-requests: write # Comment on PRs
  id-token: write     # OIDC token generation
  actions: read       # Read workflow runs
```

**Pin action versions to SHA:**
```yaml
# Less secure (tag can be moved)
- uses: actions/checkout@v4

# More secure (immutable SHA)
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
```

**Prevent script injection:**
```yaml
# Vulnerable to injection
- run: echo "Hello ${{ github.event.issue.title }}"

# Safe approach
- run: echo "Hello $TITLE"
  env:
    TITLE: ${{ github.event.issue.title }}
```

## Docker in CI/CD

### Building Docker Images

```yaml
jobs:
  build-docker:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: myorg/myapp
          tags: |
            type=ref,event=branch
            type=ref,event=pr
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}
            type=sha,prefix={{branch}}-

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
```

### Multi-Stage Docker Builds

```dockerfile
# Build stage
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Production stage
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package*.json ./
EXPOSE 3000
CMD ["npm", "start"]
```

### Container Scanning

```yaml
- name: Run Trivy vulnerability scanner
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: 'myorg/myapp:${{ github.sha }}'
    format: 'sarif'
    output: 'trivy-results.sarif'

- name: Upload Trivy results to GitHub Security
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: 'trivy-results.sarif'
```

## Release Automation

### Semantic Versioning

Automatically version releases based on commit messages:

```yaml
jobs:
  release:
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Semantic Release
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
        run: npx semantic-release
```

**Configuration (.releaserc.json):**
```json
{
  "branches": ["main"],
  "plugins": [
    "@semantic-release/commit-analyzer",
    "@semantic-release/release-notes-generator",
    "@semantic-release/changelog",
    "@semantic-release/npm",
    "@semantic-release/github",
    ["@semantic-release/git", {
      "assets": ["CHANGELOG.md", "package.json"],
      "message": "chore(release): ${nextRelease.version} [skip ci]\n\n${nextRelease.notes}"
    }]
  ]
}
```

### Changelog Generation

```yaml
- name: Generate changelog
  uses: mikepenz/release-changelog-builder-action@v4
  with:
    configuration: '.github/changelog-config.json'
    outputFile: 'CHANGELOG.md'
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Create GitHub Release
  uses: ncipollo/release-action@v1
  with:
    tag: ${{ steps.version.outputs.tag }}
    name: Release ${{ steps.version.outputs.tag }}
    bodyFile: 'CHANGELOG.md'
    artifacts: 'dist/*'
```

### Release Notes Automation

```yaml
- name: Build Release Notes
  id: release_notes
  uses: mikepenz/release-changelog-builder-action@v4
  with:
    configurationJson: |
      {
        "categories": [
          {
            "title": "## 🚀 Features",
            "labels": ["feature", "enhancement"]
          },
          {
            "title": "## 🐛 Fixes",
            "labels": ["bug", "fix"]
          },
          {
            "title": "## 📝 Documentation",
            "labels": ["documentation"]
          }
        ]
      }
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```

## Monorepo CI/CD Patterns

### Path-Based Triggers

Run workflows only when specific packages change:

```yaml
name: Frontend CI
on:
  push:
    paths:
      - 'packages/frontend/**'
      - 'package.json'
      - 'pnpm-lock.yaml'

jobs:
  test-frontend:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Test frontend
        run: pnpm --filter frontend test
```

### Affected Package Detection

```yaml
jobs:
  detect-changes:
    runs-on: ubuntu-latest
    outputs:
      affected: ${{ steps.affected.outputs.packages }}
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Detect affected packages
        id: affected
        run: |
          # Use tools like Nx or Turborepo to detect changes
          AFFECTED=$(npx nx affected:apps --base=origin/main --plain)
          echo "packages=$AFFECTED" >> $GITHUB_OUTPUT

  test-affected:
    needs: detect-changes
    runs-on: ubuntu-latest
    strategy:
      matrix:
        package: ${{ fromJson(needs.detect-changes.outputs.affected) }}
    steps:
      - uses: actions/checkout@v4
      - name: Test ${{ matrix.package }}
        run: npm run test --workspace=${{ matrix.package }}
```

### Turborepo CI

```yaml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Build with Turborepo
        run: npx turbo build --cache-dir=.turbo

      - name: Cache Turbo
        uses: actions/cache@v4
        with:
          path: .turbo
          key: ${{ runner.os }}-turbo-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-turbo-
```

## Performance Optimization

### Parallel Job Execution

```yaml
jobs:
  # These jobs run in parallel
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run lint

  unit-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run test:unit

  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run build

  # This job waits for all above to complete
  deploy:
    needs: [lint, unit-test, build]
    runs-on: ubuntu-latest
    steps:
      - run: npm run deploy
```

### Conditional Job Execution

```yaml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run build

  deploy-staging:
    needs: build
    if: github.ref == 'refs/heads/develop'
    runs-on: ubuntu-latest
    steps:
      - run: ./deploy.sh staging

  deploy-production:
    needs: build
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - run: ./deploy.sh production
```

### Dependency Caching

```yaml
steps:
  # Node.js with npm
  - uses: actions/setup-node@v4
    with:
      node-version: '20'
      cache: 'npm'

  # Python with pip
  - uses: actions/setup-python@v5
    with:
      python-version: '3.11'
      cache: 'pip'

  # Ruby with bundler
  - uses: ruby/setup-ruby@v1
    with:
      ruby-version: '3.2'
      bundler-cache: true

  # Go modules
  - uses: actions/setup-go@v5
    with:
      go-version: '1.21'
      cache: true
```

## Reusable Workflows

### Creating Reusable Workflows

```yaml
# .github/workflows/reusable-deploy.yml
name: Reusable Deploy Workflow

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
      version:
        required: false
        type: string
        default: 'latest'
    secrets:
      deploy-key:
        required: true
    outputs:
      deployment-url:
        description: "URL of the deployment"
        value: ${{ jobs.deploy.outputs.url }}

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}
    outputs:
      url: ${{ steps.deploy.outputs.url }}
    steps:
      - uses: actions/checkout@v4

      - name: Deploy
        id: deploy
        env:
          DEPLOY_KEY: ${{ secrets.deploy-key }}
        run: |
          ./deploy.sh ${{ inputs.environment }} ${{ inputs.version }}
          echo "url=https://${{ inputs.environment }}.example.com" >> $GITHUB_OUTPUT
```

### Calling Reusable Workflows

```yaml
# .github/workflows/main.yml
name: Main Pipeline

on: [push]

jobs:
  deploy-staging:
    uses: ./.github/workflows/reusable-deploy.yml
    with:
      environment: staging
      version: ${{ github.sha }}
    secrets:
      deploy-key: ${{ secrets.STAGING_DEPLOY_KEY }}

  deploy-production:
    needs: deploy-staging
    uses: ./.github/workflows/reusable-deploy.yml
    with:
      environment: production
      version: ${{ github.sha }}
    secrets:
      deploy-key: ${{ secrets.PRODUCTION_DEPLOY_KEY }}
```

## Infrastructure as Code

### Terraform Deployment

```yaml
jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.7.0

      - name: Terraform Format
        run: terraform fmt -check

      - name: Terraform Init
        run: terraform init
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

      - name: Terraform Validate
        run: terraform validate

      - name: Terraform Plan
        run: terraform plan -out=tfplan
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve tfplan
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
```

### AWS CloudFormation

```yaml
jobs:
  deploy-cloudformation:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: us-east-1

      - name: Deploy CloudFormation stack
        run: |
          aws cloudformation deploy \
            --template-file infrastructure/template.yml \
            --stack-name my-app-stack \
            --parameter-overrides \
              Environment=production \
              Version=${{ github.sha }} \
            --capabilities CAPABILITY_IAM
```

## Best Practices

### Workflow Organization

1. **Separate concerns**: Different workflows for CI, CD, and scheduled tasks
2. **Use descriptive names**: Clear workflow and job names
3. **Organize with directories**: Group related workflows
4. **Version control**: Track workflow changes like code

### Efficiency

1. **Cache dependencies**: Reduce build times significantly
2. **Parallel execution**: Run independent jobs simultaneously
3. **Conditional runs**: Skip unnecessary jobs
4. **Matrix strategies**: Test multiple configurations efficiently
5. **Artifact reuse**: Share build outputs between jobs

### Security

1. **Minimize permissions**: Use least-privilege principle
2. **Use OIDC**: Avoid long-lived credentials
3. **Secret rotation**: Regularly update secrets
4. **Pin dependencies**: Use specific versions or SHAs
5. **Scan for vulnerabilities**: Automated security checks

### Reliability

1. **Timeout settings**: Prevent hanging jobs
2. **Retry logic**: Handle transient failures
3. **Failure notifications**: Alert on critical failures
4. **Rollback mechanisms**: Quick recovery from failed deployments
5. **Health checks**: Verify deployments before marking complete

### Observability

1. **Detailed logging**: Clear, actionable logs
2. **Status checks**: Prevent merging failing builds
3. **Deployment tracking**: Know what's deployed where
4. **Metrics collection**: Track pipeline performance
5. **Audit trails**: Track who deployed what and when

## Failure Handling

### Retry Failed Steps

```yaml
steps:
  - name: Deploy with retry
    uses: nick-fields/retry-action@v2
    with:
      timeout_minutes: 10
      max_attempts: 3
      retry_wait_seconds: 30
      command: npm run deploy
```

### Continue on Error

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Run optional check
        continue-on-error: true
        run: npm run optional-check

      - name: Run required tests
        run: npm test
```

### Conditional Cleanup

```yaml
steps:
  - name: Deploy
    id: deploy
    run: ./deploy.sh

  - name: Rollback on failure
    if: failure() && steps.deploy.conclusion == 'failure'
    run: ./rollback.sh

  - name: Cleanup
    if: always()
    run: ./cleanup.sh
```

## Advanced Patterns

### Dynamic Matrix Generation

```yaml
jobs:
  generate-matrix:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.set-matrix.outputs.matrix }}
    steps:
      - uses: actions/checkout@v4
      - id: set-matrix
        run: |
          # Generate matrix based on project structure
          MATRIX=$(find packages -maxdepth 1 -type d -not -name packages | \
            jq -R -s -c 'split("\n")[:-1]')
          echo "matrix=$MATRIX" >> $GITHUB_OUTPUT

  test:
    needs: generate-matrix
    runs-on: ubuntu-latest
    strategy:
      matrix:
        package: ${{ fromJson(needs.generate-matrix.outputs.matrix) }}
    steps:
      - uses: actions/checkout@v4
      - run: npm test --workspace=${{ matrix.package }}
```

### Composite Actions

Create reusable action combinations:

```yaml
# .github/actions/setup-project/action.yml
name: 'Setup Project'
description: 'Setup Node.js and install dependencies'
inputs:
  node-version:
    description: 'Node.js version'
    required: false
    default: '20'
runs:
  using: 'composite'
  steps:
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ inputs.node-version }}
        cache: 'npm'

    - run: npm ci
      shell: bash

    - run: npm run build
      shell: bash
```

**Usage:**
```yaml
steps:
  - uses: actions/checkout@v4
  - uses: ./.github/actions/setup-project
    with:
      node-version: '20'
```

### Self-Hosted Runners

```yaml
jobs:
  deploy:
    runs-on: [self-hosted, linux, production]
    steps:
      - uses: actions/checkout@v4
      - name: Deploy to production
        run: ./deploy.sh
```

**Benefits:**
- Custom hardware/software requirements
- Faster builds (pre-cached dependencies)
- Access to internal networks
- Cost savings for high-volume CI/CD

## Platform-Specific Deployments

### Vercel Deployment

```yaml
jobs:
  deploy-vercel:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy to Vercel
        uses: amondnet/vercel-action@v25
        with:
          vercel-token: ${{ secrets.VERCEL_TOKEN }}
          vercel-org-id: ${{ secrets.VERCEL_ORG_ID }}
          vercel-project-id: ${{ secrets.VERCEL_PROJECT_ID }}
          vercel-args: '--prod'
```

### Netlify Deployment

```yaml
jobs:
  deploy-netlify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build
        run: npm run build

      - name: Deploy to Netlify
        uses: nwtgck/actions-netlify@v3
        with:
          publish-dir: './dist'
          production-branch: main
          github-token: ${{ secrets.GITHUB_TOKEN }}
          deploy-message: 'Deploy from GitHub Actions'
        env:
          NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
          NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}
```

### AWS ECS Deployment

```yaml
jobs:
  deploy-ecs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: us-east-1

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Build and push Docker image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: my-app
          IMAGE_TAG: ${{ github.sha }}
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG

      - name: Update ECS service
        run: |
          aws ecs update-service \
            --cluster my-cluster \
            --service my-service \
            --force-new-deployment
```

### Kubernetes Deployment

```yaml
jobs:
  deploy-k8s:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup kubectl
        uses: azure/setup-kubectl@v3
        with:
          version: 'v1.28.0'

      - name: Configure kubeconfig
        run: |
          echo "${{ secrets.KUBE_CONFIG }}" | base64 -d > kubeconfig.yml
          echo "KUBECONFIG=$(pwd)/kubeconfig.yml" >> $GITHUB_ENV

      - name: Deploy to Kubernetes
        run: |
          kubectl set image deployment/myapp \
            myapp=myregistry/myapp:${{ github.sha }}
          kubectl rollout status deployment/myapp
```

---

**Skill Version**: 1.0.0
**Last Updated**: October 2025
**Skill Category**: DevOps, CI/CD, Automation, Deployment
**Compatible With**: GitHub Actions, Docker, Kubernetes, AWS, Azure, GCP, Vercel, Netlify

Overview

This skill provides a practical catalog of CI/CD pipeline patterns focused on GitHub Actions, workflow design, automation, testing, deployment strategies, and release management. It distills patterns and examples for building reliable, secure, and fast delivery pipelines for modern applications. Use it to design end-to-end pipelines, optimize runtime, and standardize release processes across teams.

How this skill works

The skill inspects and codifies common CI/CD building blocks: workflows, jobs, steps, actions, secrets, artifacts, and triggers. It maps testing strategies (unit, integration, e2e, performance) to pipeline jobs, shows matrix and caching patterns for speed, and documents deployment strategies (blue-green, canary, rolling) with rollout and rollback steps. It also covers best practices for secrets, environments, reusable workflows, and multi-platform matrix builds.

When to use it

  • Setting up CI/CD for a new project using GitHub Actions
  • Automating builds, tests, and deployments across staging and production
  • Implementing multi-platform matrix builds or monorepo pipelines
  • Securing pipelines with scoped secrets, OIDC, and environment protections
  • Optimizing pipeline performance with caching, artifacts, and parallel jobs
  • Designing safe rollouts using blue-green, canary, or rolling deployments

Best practices

  • Keep workflows small and single-purpose; use reusable workflow_call for shared logic
  • Scope secrets to environments and prefer OIDC or short-lived credentials
  • Use matrix builds with fail-fast disabled for broader test feedback when needed
  • Cache dependencies and Docker layers to reduce build time and runner costs
  • Publish artifacts and test reports for traceability and post-failure analysis
  • Automate semantic versioning, changelog generation, and gated releases

Example use cases

  • A Node.js project: CI pipeline with lint, unit tests, coverage upload, and Docker image build
  • A microservices platform: integration tests using service containers and contract checks
  • A web app: E2E Playwright tests and Lighthouse performance checks before deployment
  • Progressive delivery: canary rollout to 10% traffic with metric-based promotion to 100%
  • Monorepo: targeted builds and caching per package with reusable workflows for each team

FAQ

How do I share common steps across repositories?

Create reusable workflows (workflow_call) or publish custom actions to the marketplace and call them from repository workflows.

When should I use environment secrets versus repository secrets?

Use environment secrets when you need scoped access controls and approval gates for staging or production; use repository secrets for general CI credentials.