Sandbox testing in a GitHub Actions pipeline is straightforward to set up and surprisingly easy to get wrong.

The setup part is a few workflow steps. The getting-wrong part is what happens after the first few upstream service deployments, when the sandbox environment starts validating against mock files that no longer accurately represent the services they describe. The tests keep passing. The sandbox keeps reporting green. The production failures that follow look like deployment issues when they are actually sandbox currency issues.

This guide covers both parts: how to structure a GitHub Actions workflow for sandbox testing, and how to keep the sandbox environment current as the distributed system it validates against continues to change.

What Sandbox Testing Requires Before the Pipeline

Before writing any GitHub Actions YAML, the sandbox environment needs three things defined.

The service boundary being tested. Sandbox testing validates how a service behaves when interacting with its dependencies under conditions that approximate production. Define which service is under test and which dependencies it needs to interact with - upstream APIs, downstream services, external providers.

The behavioral representations of each dependency. In a GitHub Actions context, these are typically mock files or recorded traffic fixtures that represent how each dependency responds. These can be VCR cassettes, WireMock stubs, or any format the testing framework supports. The key property is that they need to reflect current dependency behavior - not historical behavior from when the fixtures were first written.

The test scenarios to run. The tests that exercise the service against its dependency representations. These validate whether the service handles current dependency behavior correctly.

With those three elements defined, the pipeline structure is straightforward.

Basic GitHub Actions Workflow Structure

A sandbox testing workflow in GitHub Actions runs after unit tests and before deployment to any shared environment. This placement catches integration failures at the point where they are cheapest to fix before any downstream impact.

name: Sandbox Testing

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up environment
        uses: actions/setup-node@v4
        with:
          node-version: '20'
      - name: Install dependencies
        run: npm ci
      - name: Run unit tests
        run: npm test

  sandbox-tests:
    runs-on: ubuntu-latest
    needs: unit-tests
    steps:
      - uses: actions/checkout@v4
      - name: Set up environment
        uses: actions/setup-node@v4
        with:
          node-version: '20'
      - name: Install dependencies
        run: npm ci
      - name: Start dependent services
        run: docker-compose -f docker-compose.sandbox.yml up -d
      - name: Wait for services to be ready
        run: |
          timeout 60 bash -c 'until curl -s http://localhost:8080/health; do sleep 2; done'
      - name: Run sandbox tests
        run: npm run test:sandbox
      - name: Stop dependent services
        if: always()
        run: docker-compose -f docker-compose.sandbox.yml down

  deploy:
    runs-on: ubuntu-latest
    needs: sandbox-tests
    if: github.ref == 'refs/heads/main'
    steps:
      - name: Deploy to staging
        run: echo "Deploy after sandbox tests pass"

The needs directive enforces the order. Unit tests run first. Sandbox tests run only when unit tests pass. Deployment runs only when sandbox tests pass. A failure at any stage stops the pipeline before the next stage begins.

Setting Up the Docker Compose Sandbox Environment

The docker-compose.sandbox.yml referenced in the workflow defines the services and dependency representations the sandbox tests run against.

version: '3.8'

services:
  app:
    build: .
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=sandbox
      - PAYMENT_API_URL=http://payment-mock:4000
      - NOTIFICATION_API_URL=http://notification-mock:4001
    depends_on:
      - payment-mock
      - notification-mock

  payment-mock:
    image: wiremock/wiremock:latest
    ports:
      - "4000:8080"
    volumes:
      - ./sandbox/fixtures/payment:/home/wiremock/mappings

  notification-mock:
    image: wiremock/wiremock:latest
    ports:
      - "4001:8080"
    volumes:
      - ./sandbox/fixtures/notification:/home/wiremock/mappings

The application service points to mock services rather than real upstream APIs. The mock services load fixture files from the sandbox/fixtures directory. These fixtures are what the sandbox tests validate against.

This setup works correctly when the fixtures are current. The challenge begins when the real payment API or notification API deploys a change that the fixture files do not reflect.

The Sandbox Currency Problem in GitHub Actions Pipelines

In the workflow above, the fixture files in sandbox/fixtures/payment and sandbox/fixtures/notification were written at a point in time. They represent how the payment and notification services responded when the fixtures were created.

Those services keep deploying. The payment service may update its error response format. The notification service may add a required field. The fixture files do not update automatically when this happens. Someone on the team has to notice the upstream change, update the relevant fixture files, commit them, and push. Until that happens, the sandbox is validating against stale representations.

In a GitHub Actions context, this creates a specific failure mode: the sandbox tests pass in CI because they are running against the fixture files, which reflect old behavior. The deployment goes out. Production encounters the new behavior the fixture files did not capture. The production failure looks like a deployment issue. The root cause is a sandbox currency issue that the pipeline had no mechanism to surface.

The rate at which this problem compounds is proportional to how many upstream services the application integrates with and how frequently those services deploy. Two upstream services each deploying twice a week generate four potential fixture update events per week. Ten upstream services generate twenty. Manual fixture maintenance cannot scale to this rate reliably under delivery pressure.

Adding Traffic Capture to Keep Sandbox Fixtures Current

The structural fix for sandbox currency in a GitHub Actions pipeline is changing how fixture files are generated from manually authored specifications to recordings of real service behavior.

Rather than writing fixture files by hand and updating them after each upstream change, the pipeline captures real traffic between the service and its dependencies and generates fixtures from those observations. When an upstream service changes, the next capture reflects the updated behavior automatically.

Keploy supports this approach for API-driven services. It records real HTTP traffic between services during a capture run and generates test fixtures from those actual interactions. These fixtures can be committed to the repository and used in subsequent sandbox test runs, or regenerated on each pipeline run from fresh traffic observations.

Here is how this integrates into a GitHub Actions workflow:

name: Sandbox Testing With Traffic Capture

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 6 * * *'  # Daily fixture refresh at 6am UTC

jobs:
  refresh-fixtures:
    runs-on: ubuntu-latest
    if: github.event_name == 'schedule'
    steps:
      - uses: actions/checkout@v4
        with:
          token: ${{ secrets.GITHUB_TOKEN }}

      - name: Set up environment
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Install Keploy
        run: |
          curl --silent --location "https://github.com/keploy/keploy/releases/latest/download/keploy_linux_amd64.tar.gz" | tar xz -C /tmp
          sudo mv /tmp/keploy /usr/local/bin

      - name: Start real upstream services
        run: docker-compose -f docker-compose.integration.yml up -d
        env:
          PAYMENT_API_URL: ${{ secrets.PAYMENT_API_URL }}
          NOTIFICATION_API_URL: ${{ secrets.NOTIFICATION_API_URL }}

      - name: Capture real service traffic
        run: |
          keploy record --app="npm start" \
            --delay=5 \
            --path=./sandbox/fixtures
        env:
          NODE_ENV: capture

      - name: Run traffic generation script
        run: npm run generate:sandbox-traffic

      - name: Stop capture
        run: keploy stop

      - name: Commit updated fixtures
        run: |
          git config --local user.email "github-actions@github.com"
          git config --local user.name "GitHub Actions"
          git add sandbox/fixtures/
          git diff --quiet && git diff --staged --quiet || git commit -m "chore: refresh sandbox fixtures from upstream traffic [skip ci]"
          git push

  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up environment
        uses: actions/setup-node@v4
        with:
          node-version: '20'
      - name: Install dependencies
        run: npm ci
      - name: Run unit tests
        run: npm test

  sandbox-tests:
    runs-on: ubuntu-latest
    needs: unit-tests
    steps:
      - uses: actions/checkout@v4

      - name: Set up environment
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Install Keploy
        run: |
          curl --silent --location "https://github.com/keploy/keploy/releases/latest/download/keploy_linux_amd64.tar.gz" | tar xz -C /tmp
          sudo mv /tmp/keploy /usr/local/bin

      - name: Run sandbox tests with captured fixtures
        run: |
          keploy test --app="npm start" \
            --delay=5 \
            --path=./sandbox/fixtures \
            --coverage
        env:
          NODE_ENV: sandbox

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: sandbox-coverage
          path: coverage/

  deploy:
    runs-on: ubuntu-latest
    needs: sandbox-tests
    if: github.ref == 'refs/heads/main'
    steps:
      - name: Deploy to staging
        run: echo "Deploy after sandbox tests pass"

The refresh-fixtures job runs on a schedule - daily at 6am UTC in this example. It connects to real upstream services, captures actual traffic, generates updated fixture files, and commits them back to the repository. The sandbox-tests job then runs against the most recently captured fixtures rather than against fixtures that may be weeks outdated.

This workflow separates the two concerns that manual fixture maintenance conflates: when to update fixtures and whether current fixtures produce passing tests. The scheduled refresh handles the first concern automatically. The sandbox test run handles the second.

Handling Non-Deterministic Fields

Real traffic captures include non-deterministic fields - timestamps, request IDs, session tokens, generated identifiers - that change with every request. If the sandbox tests assert against these fields, they will fail on every run regardless of whether the underlying behavior has changed.

Keploy handles this through automatic noise detection: it identifies fields that vary across multiple observations of the same interaction and excludes those fields from test assertions automatically. This means the sandbox tests assert against the behavioral properties that matter - response structure, status codes, business logic outputs - rather than against the ephemeral values that differ between requests.

Explicit configuration is available for cases where automatic detection needs guidance:

# keploy.yml
noiseConfig:
  global:
    header:
      - "x-request-id"
      - "x-correlation-id"
    body:
      - "timestamp"
      - "createdAt"
      - "requestId"

This configuration tells Keploy to exclude these fields from sandbox test assertions across all captured interactions, preventing false positives from non-deterministic values while still validating the behavioral properties that indicate whether the integration is working correctly.

Managing Fixture Freshness Visibility

The scheduled refresh approach solves the currency problem but introduces a visibility gap: how do developers know how current the fixtures are?

Adding fixture timestamp tracking to the workflow surfaces this information explicitly:

- name: Record fixture refresh timestamp
  run: |
    echo "last_refreshed: $(date -u +%Y-%m-%dT%H:%M:%SZ)" > sandbox/fixtures/metadata.yml
    echo "upstream_services:" >> sandbox/fixtures/metadata.yml
    echo "  payment: ${{ env.PAYMENT_SERVICE_VERSION }}" >> sandbox/fixtures/metadata.yml
    echo "  notification: ${{ env.NOTIFICATION_SERVICE_VERSION }}" >> sandbox/fixtures/metadata.yml
    git add sandbox/fixtures/metadata.yml

The metadata.yml file committed alongside the fixtures records when they were last refreshed and which upstream service versions they were captured from. This gives developers a clear answer to the question "how current are these fixtures" without requiring investigation of git history.

A check in the sandbox test job can surface a warning when fixtures exceed a defined staleness threshold:

- name: Check fixture freshness
  run: |
    LAST_REFRESHED=$(grep 'last_refreshed' sandbox/fixtures/metadata.yml | cut -d' ' -f2)
    HOURS_SINCE=$((( $(date -u +%s) - $(date -u -d "$LAST_REFRESHED" +%s) ) / 3600))
    if [ $HOURS_SINCE -gt 48 ]; then
      echo "::warning::Sandbox fixtures are ${HOURS_SINCE} hours old. Consider triggering a manual refresh."
    fi

This check does not fail the pipeline - stale fixtures that still pass tests are better than no sandbox layer. It surfaces the staleness as a warning that developers can act on rather than discovering it through a production failure.

What This Pipeline Achieves

The complete workflow described above integrates sandbox testing as a required validation layer between unit tests and deployment, with automated fixture currency maintenance running on a daily schedule.

The sandbox tests catch the specific category of failure that unit tests cannot: failures that originate from changes in upstream service behavior that the unit tests had no knowledge of. The automated fixture refresh ensures that the sandbox environment reflects current upstream behavior rather than accumulating drift as services deploy independently.

The combination produces a pipeline where green means something more specific than it did with unit tests alone. A passing sandbox test run means the service works correctly against how its dependencies are actually currently behaving - not just against how a developer described them in a mock file at some earlier point.