serverless-integration-test-builder
Overview
Serverless integration tests are awkward because the test
surface is multi-tool: each platform has its own local
emulator, event-source binding model, and cold-start
characteristic. This skill produces a coherent test directory
from the IaC definition.
The output: a test directory layout + per-function test files
When to use
- New serverless project; need integration test coverage.
- Inherited a serverless project with no tests; need to build
coverage.
- Migrating between platforms (e.g., Lambda → Workers); need to
port test discipline.
Step 1 - Inventory functions from IaC
Per platform:
| Platform | IaC source | Where functions live |
|---|
| AWS SAM | template.yaml AWS::Serverless::Function | Resources.<name> |
| Serverless Framework | serverless.yml functions: | each entry |
| Cloudflare Workers | wrangler.toml + script bindings | [[durable_objects.bindings]], scripts |
| Vercel | pages/api/ + app/api/ + middleware | filesystem-derived |
| Netlify | netlify/functions/, netlify/edge-functions/ | filesystem-derived |
# Example: SAM
yq '.Resources | to_entries | map(select(.value.Type == "AWS::Serverless::Function")) | .[].key' template.yaml
# Example: Serverless Framework
yq '.functions | keys' serverless.yml
Output: a function inventory:
functions:
- name: get-user
platform: aws-sam
runtime: nodejs20.x
handler: src/get-user.handler
timeout: 10
memory: 512
events:
- type: api
path: /users/{id}
method: GET
- name: process-upload
platform: aws-sam
runtime: python3.12
timeout: 300
events:
- type: s3
bucket: uploads
events: ['s3:ObjectCreated:*']
Step 2 - Pick the emulator per function
Default: pick one emulator per platform detected in Step 1's inventory -
the per-platform skill below is the canonical driver for that runtime. Use
multiple emulators only when the project mixes platforms (e.g., Vercel app
| Platform | Test path | Skill |
|---|
| AWS Lambda (Node/Python/etc.) | sam local invoke or direct handler | aws-sam-local-testing |
| AWS Lambda (.NET) | Amazon.Lambda.TestUtilities | aws-sam-local-testing references/dotnet.md |
| Cloudflare Workers | vitest-pool-workers / Miniflare | cloudflare-workers-miniflare |
| Azure Functions | Core Tools func start + Azurite | azure-functions-tests |
| Vercel Edge / Netlify / Serverless Framework | Lambda-under-the-hood; vendor dev CLI + direct handler | (no dedicated skill; same patterns as SAM) |
Step 3 - Generate test events per event source
| Event source | Tool |
|---|
| API Gateway request | sam local generate-event apigateway aws-proxy |
| S3 object created | sam local generate-event s3 put |
| SQS message | sam local generate-event sqs receive-message |
| DynamoDB Streams | sam local generate-event dynamodb update |
| EventBridge | sam local generate-event events |
Cloudflare Workers fetch | new Request('https://...') directly |
| Vercel/Netlify | construct Request object |
Commit events to tests/fixtures/events/. Reference in tests.
Step 4 - Emit test scaffold per function
tests/
integration/
get-user.test.ts
process-upload.test.ts
fixtures/
events/
api-get-users-id.json
s3-object-created.json
payloads/
user-1.json
conftest.py # Or jest setup
README.md # How to run + matrix
Per-function test:
// tests/integration/get-user.test.ts
import { handler } from '../../src/get-user';
import event from '../fixtures/events/api-get-users-id.json';
describe('get-user', () => {
test('returns 200 for valid id', async () => {
const response = await handler({ ...event, pathParameters: { id: '1' } } as any, {} as any);
expect(response.statusCode).toBe(200);
});
test('returns 404 for missing id', async () => {
const response = await handler({ ...event, pathParameters: { id: 'missing' } } as any, {} as any);
expect(response.statusCode).toBe(404);
});
test('completes within timeout budget', async () => {
// Per cold-start-budget-reference references/timeout-budgets.md: timeout=10s; want headroom
const start = Date.now();
await handler({ ...event, pathParameters: { id: '1' } } as any, {} as any);
const elapsed = Date.now() - start;
expect(elapsed).toBeLessThan(5000); // 50% of timeout
});
});
Step 5 - Cold-start + timeout budget assertions
For each function, derive a budget from
cold-start-budget-reference (cold-start tables + timeout budgets in
its references/timeout-budgets.md).
Typical cold starts per runtime (per AWS / Cloudflare / Vercel
docs; bigger packages and memory classes skew higher):
| Runtime | Typical cold start |
|---|
| AWS Lambda Node.js (256MB) | 200-700ms |
| AWS Lambda Python (256MB) | 250-800ms |
| AWS Lambda Java 11 (512MB, no SnapStart) | 1.5-6s |
| AWS Lambda Java 11 (512MB, SnapStart) | 100-300ms |
| AWS Lambda .NET (1GB) | 1-3s |
| AWS Lambda Go (256MB) | 100-300ms |
| AWS Lambda Rust (256MB) | 50-200ms |
| Cloudflare Workers | 0-5ms (V8 isolate spawn) |
| Vercel Edge Runtime | 5-30ms |
| Vercel Node.js Functions | 200-500ms (small) to 2-3s (large) |
| Netlify Functions | 300ms-2s |
The upstream integration - not the function's own timeout - is
usually the operational ceiling (per
docs.aws.amazon.com/lambda;
Lambda's hard maximum is 900s):
| Integration | Timeout ceiling |
|---|
| API Gateway (REST + HTTP API) | 29s hard - function timeout must be < 29s |
| Application Load Balancer | 4s default; configurable to 4000s |
| CloudFront Lambda@Edge | 5s viewer functions; 30s origin functions |
| SQS event source | Per-queue visibility timeout (default 30s); function timeout must be below it |
| DynamoDB Streams | 6-hour batch window; per-batch function limit |
| EventBridge (async) | Async with retry on timeout - requires idempotency |
| Step Functions | Per-task timeout, default 60s |
| Cloudflare Workers | 10ms CPU (free) / 50ms CPU (paid); 30s wall-clock |
| Vercel Edge Functions | 30s wall-clock max |
Set the assertion threshold at roughly 50% of the binding ceiling
so a regression trips the test before it trips production:
budgets:
get-user:
cold_start_p99_ms: 800 # AWS Lambda Node@512MB typical
warm_p99_ms: 200
timeout_s: 10
integration_timeout: api-gateway-29s # API GW < 29s
process-upload:
cold_start_p99_ms: 1500
warm_p99_ms: 2000
timeout_s: 300
integration_timeout: s3-event-async # No upstream limit
These become assertion thresholds in staging-smoke tests (run
against deployed functions, not local).
Step 6 - CI integration
# .github/workflows/serverless-tests.yml
jobs:
local-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v4
- uses: aws-actions/setup-sam@v2
- run: sam build --use-container
- run: npm ci && npx jest tests/integration/
staging-deploy-and-smoke:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
needs: local-tests
steps:
- uses: actions/checkout@v5
- run: sam deploy --stack-name staging --parameter-overrides Stage=staging
- run: npx jest tests/staging-smoke/
Two-tier: local-fast tests on every PR; deploy + smoke on main.
Step 7 - Document coverage matrix
# Serverless Integration Test Matrix
## Functions covered
| Function | Local test | Cold-start test | Timeout test | Event sources |
|---|---|---|---|---|
| get-user | ✅ tests/integration/get-user.test.ts | staging only | ✅ <5s | API GW GET /users/{id} |
| process-upload | ✅ | staging only | ✅ <150s of 300s | S3 ObjectCreated |
| ... | | | | |
## Gaps
- `legacy-export` - no integration tests; on backlog.
- Cold-start tests not feasible locally for any function (per `cold-start-budget-reference`).
## How to add a new function
1. Add to IaC.
2. Generate event fixture.
3. Add test file following the template.
4. Update this matrix.
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|
| One emulator for all functions | Mixed-platform tests are awkward; some emulators miss platform-specific behaviour | Per-platform emulator |
| No staging deploy + smoke tests | Local-pass + prod-fail gap | Two-tier CI |
| Hand-rolled event payloads | Schema drift; missed fields | Use sam local generate-event or commit fixtures |
| Skip cold-start budget assertions | p99 spikes go unnoticed | Per-function budget in staging smoke |
| Test directory mixed with prod code | tests/ dir not in deploy artifact | Keep separate |
| No README documenting matrix | Future maintainers can't add functions consistently | Step 7 README |
| Inflate jest test timeouts to mask slow handlers | Hides performance regression | Match jest timeout to function timeout |
| Mock AWS SDK in all tests | Tests pass; IAM permission bugs hide | Use LocalStack or real test account |
Output
This skill produces:
- A function inventory (Step 1).
- Per-function test files using the right emulator (Steps 2-4).
- A budget matrix (Step 5).
- A two-tier CI config (Step 6).
- A coverage matrix README (Step 7).
References
- Companion catalog:
cold-start-budget-reference (cold-start + timeout budgets).
- Per-platform tools:
aws-sam-local-testing (incl. .NET in references/dotnet.md),
cloudflare-workers-miniflare,
azure-functions-tests.
- Cross-plugin:
testcontainers (in the qa-test-environment plugin)
- LocalStack alternative for AWS-service emulation.