AGENTS.md
# Software Testing
## Structure
```
software-testing/
SKILL.md # Main skill file - read this first
AGENTS.md # This navigation guide
CLAUDE.md # Symlink to AGENTS.md
references/ # Detailed reference files
```
## Usage
1. Read `SKILL.md` for the main skill instructions
2. Browse `references/` for detailed documentation on specific topics
3. Reference files are loaded on-demand - read only what you need
## Core Philosophy
Tests exist for one reason: to give you justified confidence that your software does what you intend and won't break what already works when you change it.
The key word is "justified." Confidence without evidence is delusion. A green test suite that doesn't exercise real failure modes is theater. 100% code coverage where every test asserts `expect(true).toBe(true)` is worse than no tests — it creates false confidence.
The question is never "do we have enough tests?" The question is: "if this change introduced a bug, which test would catch it?" If you can't point to the specific test, you don't have coverage for that behavior — regardless of what the coverage percentage says.
## When to Apply
Use this skill when:
- Designing a testing strategy for a new service or feature
- Deciding what level of testing a change needs
- Writing integration, contract, property-based, or E2E tests
- Diagnosing flaky tests or test suite reliability issues
- Designing CI/CD test pipelines
- Verifying production readiness before shipping
- Improving confidence in an existing test suite
## Rule Categories by Priority
| # | Category | Prefix | Impact | Description |
|---|----------|--------|--------|-------------|
| 1 | Testing Philosophy | `philosophy` | CRITICAL | Core principles: confidence over coverage, test what matters |
| 2 | Test Strategy | `strategy` | CRITICAL | Risk-driven testing, the right test at the right level |
| 3 | Unit Tests | `unit` | HIGH | Effective unit test design: structure, naming, table-driven patterns |
| 4 | Integration Tests | `integration` | CRITICAL | Testing real component interactions with real dependencies |
| 5 | Contract Tests | `contract` | HIGH | Verifying service boundary agreements |
| 6 | Advanced Test Types | `advanced` | HIGH | Property-based, load, chaos, and snapshot testing |
| 7 | Test Architecture | `arch` | HIGH | Test doubles, data management, flaky test discipline |
| 8 | CI Pipeline | `pipeline` | MEDIUM-HIGH | Pipeline design, coverage ratchets, deploy gates |
| 9 | Production Verification | `prod` | MEDIUM-HIGH | Canary deploys, feature flags, observability as testing |
## Reference Guide
Detailed patterns and examples are in `references/`. Each file follows the format:
```
{prefix}-{topic}.md
```
Access them when you need specific implementation patterns for a testing category.
## The Testing Shape
The classic pyramid — many unit tests, fewer integration tests, even fewer E2E tests — was good advice when integration tests were slow and expensive. Modern tooling has changed the cost equation:
```
/\
/ \ E2E / Smoke tests (few, critical paths only)
/ \
/------\
/ \ Integration tests (many, real interactions)
/ \
/------------\
/ \ Focused unit tests (targeted, complex logic)
/ \
/------------------\
/ \ Static analysis + type system (zero runtime cost)
/ \
/------------------------\
```
**The base is the type system and static analysis** — not unit tests. A well-typed codebase eliminates entire categories of bugs with zero runtime cost.
**The middle is integration tests** — not unit tests. The bugs that reach production are usually "these two components don't agree on the contract," not "this function computes the wrong value."
**Unit tests are for complex, branchy logic** — algorithms, parsers, state machines, business rules with many code paths.
**E2E tests are for critical path smoke tests** — the 3-5 journeys that, if broken, mean the product is fundamentally non-functional.
## Self-Review Checklist
Before shipping any change:
- [ ] New behavior has tests that would fail if the behavior regressed
- [ ] Edge cases are covered (empty input, boundary values, error cases)
- [ ] Integration tests cover real interaction paths, not just mocked versions
- [ ] No new flaky tests introduced
- [ ] All CI stages pass
- [ ] Coverage of changed files meets threshold
- [ ] Monitoring and alerts in place for new behavior
CLAUDE.md
AGENTS.md
SKILL.md
---
name: software-testing
description: >
Use this skill when designing test strategies, writing tests beyond basic unit
tests, verifying software for production readiness, or improving test coverage
and reliability. Triggers when the user asks about testing strategy, integration
tests, end-to-end tests, contract tests, property-based tests, load tests,
chaos testing, test architecture, flaky tests, test confidence, 'how do I test
this,' 'how do I know this is safe to deploy,' 'my tests are flaky,' 'what
should I test,' 'test coverage,' CI/CD test pipelines, or any question about
software verification and validation. Also triggers when the user is shipping a
change and wants confidence it won't break production. Primarily targets
TypeScript and Go but principles apply universally. Do NOT use for writing basic
unit tests for simple functions — this skill is for the harder testing questions.
metadata:
author: kylejryan
version: "1.0.0"
organization: kylejryan
date: March 2026
abstract: >
Comprehensive software testing guide covering test strategy, architecture,
and production verification for building justified confidence to ship.
---
# Software Testing
## Core Philosophy
Tests exist for one reason: to give you justified confidence that your software does what you intend and won't break what already works when you change it.
The key word is "justified." Confidence without evidence is delusion. A green test suite that doesn't exercise real failure modes is theater. 100% code coverage where every test asserts `expect(true).toBe(true)` is worse than no tests — it creates false confidence.
The question is never "do we have enough tests?" The question is: "if this change introduced a bug, which test would catch it?" If you can't point to the specific test, you don't have coverage for that behavior — regardless of what the coverage percentage says.
## When to Apply
Use this skill when:
- Designing a testing strategy for a new service or feature
- Deciding what level of testing a change needs
- Writing integration, contract, property-based, or E2E tests
- Diagnosing flaky tests or test suite reliability issues
- Designing CI/CD test pipelines
- Verifying production readiness before shipping
- Improving confidence in an existing test suite
## Rule Categories by Priority
| # | Category | Prefix | Impact | Description |
|---|----------|--------|--------|-------------|
| 1 | Testing Philosophy | `philosophy` | CRITICAL | Core principles: confidence over coverage, test what matters |
| 2 | Test Strategy | `strategy` | CRITICAL | Risk-driven testing, the right test at the right level |
| 3 | Unit Tests | `unit` | HIGH | Effective unit test design: structure, naming, table-driven patterns |
| 4 | Integration Tests | `integration` | CRITICAL | Testing real component interactions with real dependencies |
| 5 | Contract Tests | `contract` | HIGH | Verifying service boundary agreements |
| 6 | Advanced Test Types | `advanced` | HIGH | Property-based, load, chaos, and snapshot testing |
| 7 | Test Architecture | `arch` | HIGH | Test doubles, data management, flaky test discipline |
| 8 | CI Pipeline | `pipeline` | MEDIUM-HIGH | Pipeline design, coverage ratchets, deploy gates |
| 9 | Production Verification | `prod` | MEDIUM-HIGH | Canary deploys, feature flags, observability as testing |
## Reference Guide
Detailed patterns and examples are in `references/`. Each file follows the format:
```
{prefix}-{topic}.md
```
Access them when you need specific implementation patterns for a testing category.
## The Testing Shape
The classic pyramid — many unit tests, fewer integration tests, even fewer E2E tests — was good advice when integration tests were slow and expensive. Modern tooling has changed the cost equation:
```
/\
/ \ E2E / Smoke tests (few, critical paths only)
/ \
/------\
/ \ Integration tests (many, real interactions)
/ \
/------------\
/ \ Focused unit tests (targeted, complex logic)
/ \
/------------------\
/ \ Static analysis + type system (zero runtime cost)
/ \
/------------------------\
```
**The base is the type system and static analysis** — not unit tests. A well-typed codebase eliminates entire categories of bugs with zero runtime cost.
**The middle is integration tests** — not unit tests. The bugs that reach production are usually "these two components don't agree on the contract," not "this function computes the wrong value."
**Unit tests are for complex, branchy logic** — algorithms, parsers, state machines, business rules with many code paths.
**E2E tests are for critical path smoke tests** — the 3-5 journeys that, if broken, mean the product is fundamentally non-functional.
## Self-Review Checklist
Before shipping any change:
- [ ] New behavior has tests that would fail if the behavior regressed
- [ ] Edge cases are covered (empty input, boundary values, error cases)
- [ ] Integration tests cover real interaction paths, not just mocked versions
- [ ] No new flaky tests introduced
- [ ] All CI stages pass
- [ ] Coverage of changed files meets threshold
- [ ] Monitoring and alerts in place for new behavior
references/_sections.md
# Section Definitions
This file defines the rule categories for software-testing. Rules are automatically assigned
to sections based on their filename prefix.
---
## 1. Testing Philosophy (philosophy)
**Impact:** CRITICAL
**Description:** Core principles — tests exist for justified confidence, not coverage metrics. Optimize for confidence per engineering-hour.
## 2. Test Strategy (strategy)
**Impact:** CRITICAL
**Description:** Risk-driven testing approach. Test proportionally to risk: probability of a bug times cost of that bug reaching production.
## 3. Unit Tests (unit)
**Impact:** HIGH
**Description:** Effective unit test design for complex, branchy logic. Arrange-Act-Assert structure, naming, table-driven patterns.
## 4. Integration Tests (integration)
**Impact:** CRITICAL
**Description:** Testing real component interactions with real dependencies. Use real databases, real HTTP handlers, real middleware chains.
## 5. Contract Tests (contract)
**Impact:** HIGH
**Description:** Verifying that service boundaries agree on communication shape without requiring both services running simultaneously.
## 6. Advanced Test Types (advanced)
**Impact:** HIGH
**Description:** Property-based testing, load/performance testing, chaos/resilience testing, and snapshot/golden file testing.
## 7. Test Architecture (arch)
**Impact:** HIGH
**Description:** Test doubles selection, test data management with builders/factories, and flaky test discipline.
## 8. CI Pipeline (pipeline)
**Impact:** MEDIUM-HIGH
**Description:** Staged pipeline design with fail-fast, coverage ratchets, and explicit deploy gate definitions.
## 9. Production Verification (prod)
**Impact:** MEDIUM-HIGH
**Description:** Canary deploys, feature flags, health checks, and observability as a continuation of testing in production.
references/advanced-chaos.md
---
title: Chaos and Fault Injection Testing
impact: HIGH
impactDescription: Verifies system resilience before production failures teach you the hard way
tags: advanced, chaos, fault-injection, resilience, error-handling, circuit-breaker
---
## Chaos and Fault Injection Testing
Systems fail in production. The question is whether you've verified the failure behavior in advance or are discovering it during an incident. Fault injection tests inject failures at dependency boundaries and verify the system degrades gracefully — meaningful errors, no data corruption, recovery when the dependency returns.
**Incorrect (only testing the happy path):**
```go
func TestCreateFinding(t *testing.T) {
db := setupTestDB(t)
svc := NewFindingService(db)
finding, err := svc.Create(ctx, CreateInput{Title: "XSS", Severity: "high"})
require.NoError(t, err)
require.NotEmpty(t, finding.ID)
// What happens when the DB is down? When it times out?
// When it accepts the write but the transaction fails to commit?
// This test has no idea.
}
```
**Correct (injecting faults to test failure behavior):**
```go
// Fault-injecting wrapper
type FaultyDB struct {
real Database
failAfter int
callCount int
}
func (f *FaultyDB) Query(ctx context.Context, q string, args ...any) (Result, error) {
f.callCount++
if f.callCount > f.failAfter {
return Result{}, errors.New("connection refused")
}
return f.real.Query(ctx, q, args...)
}
func TestCreateFinding_DBFailureMidTransaction(t *testing.T) {
realDB := setupTestDB(t)
faultyDB := &FaultyDB{real: realDB, failAfter: 1} // fail on second query
svc := NewFindingService(faultyDB)
_, err := svc.Create(ctx, CreateInput{Title: "XSS", Severity: "high"})
// Assert graceful failure
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to create finding")
// Assert no partial data left behind
findings, _ := realDB.Query(ctx, "SELECT * FROM findings")
assert.Empty(t, findings)
}
```
```typescript
// Override fetch to inject failures for external API
test("handles external API failure gracefully", async () => {
const faultyFetch = (url: string) => {
if (url.includes("/external-api")) {
return Promise.reject(new Error("ECONNREFUSED"));
}
return originalFetch(url);
};
const service = new NotificationService({ fetch: faultyFetch });
const result = await service.notifyWithFallback(alert);
// System should degrade gracefully, not crash
expect(result.notified).toBe(false);
expect(result.fallbackUsed).toBe(true);
expect(result.error).toContain("ECONNREFUSED");
});
```
Assert: meaningful errors to callers, no data corruption on partial failure, recovery when dependencies return, timeouts fire before resource exhaustion.
references/advanced-property.md
---
title: Property-Based Testing for Edge Case Discovery
impact: HIGH
impactDescription: Finds bugs hand-picked test cases miss — the weird inputs nobody thought to test
tags: advanced, property-based, fuzzing, generators, roundtrip, invariants
---
## Property-Based Testing for Edge Case Discovery
Hand-picked test cases reflect the author's assumptions. The bugs that reach production are the ones nobody thought to test. Property-based tests generate thousands of random inputs and verify that general properties hold for ALL of them — then shrink failures to the minimal reproduction case.
**Incorrect (only hand-picked examples):**
```go
func TestSerializeDeserialize(t *testing.T) {
// Tests exactly the cases the author thought of
input := Finding{Title: "XSS", Severity: "critical"}
bytes, _ := Serialize(input)
result, _ := Deserialize(bytes)
assert.Equal(t, input, result)
// Misses: empty title, unicode, null bytes, max-length strings,
// special characters, deeply nested structures...
}
```
```typescript
test("serialize/deserialize roundtrip", () => {
const input = { title: "XSS", severity: "critical" };
expect(deserialize(serialize(input))).toEqual(input);
// One example. Works for this input. Breaks on others.
});
```
**Correct (property-based — test the invariant, not specific examples):**
```go
import "testing/quick"
func TestSerializeDeserialize_Roundtrip(t *testing.T) {
f := func(input Finding) bool {
bytes, err := Serialize(input)
if err != nil {
return false
}
result, err := Deserialize(bytes)
if err != nil {
return false
}
return reflect.DeepEqual(input, result)
}
// Runs 100+ random Finding values through roundtrip
if err := quick.Check(f, nil); err != nil {
t.Error(err) // Reports the minimal failing input
}
}
```
```typescript
import fc from "fast-check";
test("serialize/deserialize roundtrip for all findings", () => {
fc.assert(
fc.property(
fc.record({
title: fc.string(),
severity: fc.oneof(fc.constant("critical"), fc.constant("high"),
fc.constant("medium"), fc.constant("low")),
tags: fc.array(fc.string()),
}),
(finding) => {
const roundtripped = deserialize(serialize(finding));
expect(roundtripped).toEqual(finding);
}
)
);
});
```
Key properties to test: roundtrip (`deserialize(serialize(x)) == x`), idempotency (`f(f(x)) == f(x)`), invariants (`isSorted(sort(x))`), and no-crash (`f(x)` never throws for any valid input type).
references/advanced-snapshot.md
---
title: Snapshot and Golden File Tests
impact: MEDIUM
impactDescription: Catches unintended changes to serialization formats and API response shapes
tags: advanced, snapshot, golden-file, serialization, regression, wire-format
---
## Snapshot and Golden File Tests
Snapshot tests compare output against a saved "known good" result. They catch unintended changes to serialization formats, API responses, code generation output, and CLI formatting. Use them for output where the EXACT format matters for correctness — wire formats, public APIs — not for internal representations that change frequently.
**Incorrect (snapshots for internal/volatile output — constant update churn):**
```typescript
// Snapshot of internal debug representation — changes with every refactor
test("finding debug output", () => {
const finding = createFinding();
expect(finding.toString()).toMatchSnapshot();
// Snapshot: "Finding{id=abc123, created=2024-01-15T10:30:00Z, ...}"
// Every time a field is added, reordered, or reformatted: "Update snapshot? y/n"
// After the 50th "y", nobody examines the diff anymore
});
```
**Correct (snapshots for wire format stability):**
```typescript
// Snapshot of public API response shape — changes must be intentional
test("GET /api/findings/:id response matches expected wire format", () => {
const finding = createTestFinding({
id: "fixed-id",
title: "SQL Injection",
severity: "critical",
createdAt: new Date("2024-01-15T00:00:00Z"),
});
expect(JSON.stringify(finding.toWireFormat(), null, 2)).toMatchSnapshot();
});
```
```go
func TestFindingJSON_MatchesGolden(t *testing.T) {
finding := Finding{
ID: "fixed-id",
Title: "SQL Injection",
Severity: SeverityCritical,
CreatedAt: time.Date(2024, 1, 15, 0, 0, 0, 0, time.UTC),
}
got, _ := json.MarshalIndent(finding, "", " ")
golden := filepath.Join("testdata", "finding.golden.json")
if *update {
os.WriteFile(golden, got, 0644)
return
}
expected, _ := os.ReadFile(golden)
if !bytes.Equal(got, expected) {
t.Errorf("output differs from golden file:\n%s",
cmp.Diff(string(expected), string(got)))
}
}
```
Use deterministic inputs (fixed IDs, fixed timestamps) so snapshots are stable. When a snapshot test fails, examine the diff before updating — the whole point is to catch unintended changes.
references/arch-data.md
---
title: Test Data Builders Over Raw Literals
impact: HIGH
impactDescription: Makes tests readable — you see only what's relevant, not 15 fields of noise
tags: architecture, test-data, builders, factories, readability, maintenance
---
## Test Data Builders Over Raw Literals
When tests need complex objects, builder functions set sensible defaults and let each test override only what's relevant to that test. This eliminates noise — a test for severity-based routing shouldn't need to specify the finding's title, service, creation date, and 12 other fields.
**Incorrect (raw object literals with full field specification):**
```typescript
test("critical findings trigger immediate notification", async () => {
const finding = {
id: "f8d7e6c5-b4a3-2190-0000-abcdef123456",
title: "SQL Injection in auth handler", // irrelevant to this test
description: "Found via automated scan...", // irrelevant
severity: "critical", // THIS is what matters
status: "open", // irrelevant
service: "auth-service", // irrelevant
assignee: "security-team", // irrelevant
tags: ["injection", "auth"], // irrelevant
createdAt: new Date("2024-01-15T10:30:00Z"), // irrelevant
updatedAt: new Date("2024-01-15T10:30:00Z"), // irrelevant
source: "scanner", // irrelevant
confidence: 0.95, // irrelevant
};
// 12 lines of noise obscuring the one field that matters
expect(await shouldNotifyImmediately(finding)).toBe(true);
});
```
**Correct (builder with sensible defaults, override only what matters):**
```typescript
function buildFinding(overrides: Partial<Finding> = {}): Finding {
return {
id: randomUUID(),
title: "Test finding",
description: "Test description",
severity: "medium",
status: "open",
service: "test-service",
assignee: null,
tags: [],
createdAt: new Date(),
updatedAt: new Date(),
source: "manual",
confidence: 1.0,
...overrides,
};
}
test("critical findings trigger immediate notification", async () => {
const finding = buildFinding({ severity: "critical" });
expect(await shouldNotifyImmediately(finding)).toBe(true);
});
test("medium findings do not trigger immediate notification", async () => {
const finding = buildFinding({ severity: "medium" });
expect(await shouldNotifyImmediately(finding)).toBe(false);
});
```
```go
func NewTestFinding(overrides ...func(*Finding)) Finding {
f := Finding{
ID: uuid.New().String(),
Title: "Test finding",
Severity: SeverityMedium,
Status: StatusOpen,
Service: "test-service",
CreatedAt: time.Now(),
}
for _, o := range overrides {
o(&f)
}
return f
}
// In test — only the relevant field is visible
critical := NewTestFinding(func(f *Finding) {
f.Severity = SeverityCritical
})
```
The test reads like a specification: "a critical finding triggers immediate notification." No noise.
references/arch-doubles.md
---
title: Test Doubles — Fakes Over Mocks
impact: HIGH
impactDescription: Catches real behavior bugs that mocks systematically miss
tags: architecture, test-doubles, mock, stub, fake, dependency-injection
---
## Test Doubles — Fakes Over Mocks
Mocks verify that specific methods were called with specific arguments — they assert on implementation details, not behavior. When you refactor the implementation (without changing behavior), mock-heavy tests break. Fakes are working simplified implementations that have real behavior. They're more work to build but dramatically more valuable.
**Incorrect (heavy mocking — asserts on implementation, not behavior):**
```typescript
test("create finding saves and notifies", async () => {
const mockRepo = { save: jest.fn().mockResolvedValue({ id: "1" }) };
const mockNotifier = { notify: jest.fn() };
const mockLogger = { info: jest.fn(), error: jest.fn() };
const mockCache = { invalidate: jest.fn() };
const service = new FindingService(mockRepo, mockNotifier, mockLogger, mockCache);
await service.create({ title: "XSS", severity: "critical" });
// Tests implementation details — HOW it works, not WHAT it does
expect(mockRepo.save).toHaveBeenCalledWith(expect.objectContaining({
title: "XSS",
}));
expect(mockNotifier.notify).toHaveBeenCalledTimes(1);
expect(mockCache.invalidate).toHaveBeenCalledWith("findings:list");
// Refactor the internals → all these assertions break
});
```
**Correct (fake implementation — tests real behavior):**
```typescript
// A fake repository with real (in-memory) behavior
class FakeFindinRepo implements FindingRepository {
private store = new Map<string, Finding>();
async save(finding: Finding): Promise<Finding> {
const id = randomUUID();
const saved = { ...finding, id };
this.store.set(id, saved);
return saved;
}
async findById(id: string): Promise<Finding | null> {
return this.store.get(id) ?? null;
}
async count(): Promise<number> {
return this.store.size;
}
}
test("create finding persists and is retrievable", async () => {
const repo = new FakeFindinRepo();
const service = new FindingService(repo);
const created = await service.create({ title: "XSS", severity: "critical" });
// Tests WHAT happened — a finding was created and is retrievable
const retrieved = await repo.findById(created.id);
expect(retrieved).toBeDefined();
expect(retrieved!.title).toBe("XSS");
expect(await repo.count()).toBe(1);
});
```
```go
// Go: stub for simple returns, fake for behavior
type StubClock struct{ now time.Time }
func (s StubClock) Now() time.Time { return s.now }
type FakeUserRepo struct {
store map[string]User
mu sync.RWMutex
}
func (f *FakeUserRepo) Create(ctx context.Context, u User) error {
f.mu.Lock()
defer f.mu.Unlock()
if _, exists := f.store[u.Email]; exists {
return ErrDuplicateEmail
}
f.store[u.Email] = u
return nil
}
```
Rule of thumb: if you're mocking more than 2 dependencies in a single test, move up to an integration test with real or fake dependencies.
references/arch-flaky.md
---
title: Flaky Test Zero Tolerance
impact: HIGH
impactDescription: Preserves trust in the test suite — a flaky suite is an ignored suite
tags: architecture, flaky, reliability, quarantine, determinism, ci
---
## Flaky Test Zero Tolerance
A flaky test — one that sometimes passes and sometimes fails without code changes — trains the team to ignore test failures. Once people start saying "oh, that's just the flaky one," trust in the entire suite erodes. A flaky test must be fixed immediately or quarantined out of the main suite.
**Incorrect (time-dependent test that fails near boundaries):**
```typescript
test("token expires after 1 hour", () => {
const token = createToken({ userId: "123" });
const decoded = verifyToken(token);
// Uses real clock — fails if test runs within milliseconds of the hour boundary
expect(decoded.expiresAt).toBe(
new Date(Date.now() + 60 * 60 * 1000).toISOString()
);
// Also fails if the machine clock drifts or DST changes during test run
});
```
```go
func TestListFindings_ReturnsAll(t *testing.T) {
db := sharedTestDB // shared mutable state
svc := NewFindingService(db)
findings, _ := svc.List(context.Background())
// Depends on what other tests inserted — passes alone, fails in suite
assert.Len(t, findings, 3)
}
```
**Correct (deterministic time, isolated state):**
```typescript
test("token expires after 1 hour", () => {
const fixedNow = new Date("2024-01-15T10:00:00Z");
const clock = { now: () => fixedNow };
const token = createToken({ userId: "123" }, { clock });
const decoded = verifyToken(token, { clock });
expect(decoded.expiresAt).toBe("2024-01-15T11:00:00.000Z");
// Deterministic — same result every time, on every machine
});
```
```go
func TestListFindings_ReturnsAll(t *testing.T) {
db := setupTestDB(t) // fresh DB per test
svc := NewFindingService(db)
// Insert exactly what this test expects
svc.Create(ctx, CreateInput{Title: "A"})
svc.Create(ctx, CreateInput{Title: "B"})
svc.Create(ctx, CreateInput{Title: "C"})
findings, _ := svc.List(context.Background())
assert.Len(t, findings, 3) // deterministic — always 3
}
```
Common flaky causes: time dependency (inject a clock), ordering dependency (isolate state), race conditions (proper synchronization), port collisions (dynamic allocation), floating point (epsilon comparison), non-deterministic iteration order (sort before comparing).
Detect flakiness: `go test -count=100 -run TestSuspect`. If it fails even once, it's flaky.
references/contract-consumer.md
---
title: Consumer-Driven Contract Testing
impact: HIGH
impactDescription: Catches service boundary breaks before production deployment
tags: contract, consumer-driven, pact, schema-validation, service-boundary
---
## Consumer-Driven Contract Testing
Service A calls Service B's API. Service B changes a field name. Both services' tests pass in isolation — A mocks B's old response, B validates its new response. Production breaks because A expects the old field and B sends the new one. Contract tests make the agreement explicit and test both sides against it.
**Incorrect (no contract verification — each side tests in isolation):**
```typescript
// Consumer test — mocks provider with stale assumptions
test("fetches user from auth service", async () => {
nock("https://auth-service")
.get("/users/123")
.reply(200, { id: "123", email: "test@example.com", name: "Alice" });
// ^^^ This mock may not match what auth-service actually returns
const user = await authClient.getUser("123");
expect(user.name).toBe("Alice");
// Passes in CI, fails in production when auth-service renames 'name' to 'displayName'
});
```
**Correct (shared schema validates both sides):**
```typescript
// shared-schemas/user-response.schema.ts
import { z } from "zod";
export const UserResponseSchema = z.object({
id: z.string(),
email: z.string().email(),
displayName: z.string(),
role: z.enum(["admin", "user", "viewer"]),
});
// Consumer test — validates response against shared schema
test("auth service response matches contract", async () => {
const response = await authClient.getUser("123");
const result = UserResponseSchema.safeParse(response);
expect(result.success).toBe(true);
});
// Provider test — validates handler output against same schema
test("GET /users/:id matches contract", async () => {
const response = await request(app).get("/users/123").expect(200);
const result = UserResponseSchema.safeParse(response.body);
expect(result.success).toBe(true);
});
```
```go
// Provider validates against shared JSON schema
func TestGetUser_MatchesContract(t *testing.T) {
resp := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/users/123", nil)
handler.GetUser(resp, req)
var body map[string]any
json.NewDecoder(resp.Body).Decode(&body)
err := schema.Validate("user-response", body)
require.NoError(t, err, "handler output does not match contract schema")
}
```
When either side changes the shape, the shared schema acts as the single source of truth. Breaking changes are caught in CI, not production.
references/integration-dep-matrix.md
---
title: Dependency Decision Matrix — What to Mock vs Keep Real
impact: HIGH
impactDescription: Eliminates wasted effort mocking things that should be real and vice versa
tags: integration, dependencies, mocking, testcontainers, decision-matrix
---
## Dependency Decision Matrix — What to Mock vs Keep Real
Not every dependency should be real in integration tests, and not every dependency should be mocked. The decision depends on whether you control the dependency and whether its real behavior is essential to what you're testing.
**Incorrect (mocking everything including your own database):**
```typescript
// Over-mocked integration test — tests nothing real
const mockDB = { query: jest.fn().mockResolvedValue([{ id: 1, name: "test" }]) };
const mockCache = { get: jest.fn().mockResolvedValue(null), set: jest.fn() };
const mockQueue = { publish: jest.fn() };
const mockLogger = { info: jest.fn(), error: jest.fn() };
const service = new FindingService(mockDB, mockCache, mockQueue, mockLogger);
const result = await service.create({ title: "XSS" });
expect(mockDB.query).toHaveBeenCalled();
expect(mockCache.set).toHaveBeenCalled();
expect(mockQueue.publish).toHaveBeenCalled();
// Tests that mocks were called — not that the system works
```
**Correct (real where feasible, mock only external boundaries):**
```typescript
// Real database and cache — mock only the external third-party API
const db = await TestDatabase.create(); // real postgres
const cache = new RedisTestContainer(); // real redis
const externalAPI = nock("https://api.vendor.com") // mock external
.post("/notify")
.reply(200, { status: "sent" });
const service = new FindingService(db, cache, externalAPI.baseUrl);
const result = await service.create({ title: "XSS", severity: "critical" });
// Assert real behavior
expect(result.id).toBeDefined();
const fromDB = await db.findings.findById(result.id);
expect(fromDB).toBeDefined();
expect(externalAPI.isDone()).toBe(true);
```
Decision matrix:
| Dependency | Keep real | Mock/stub | Reason |
|---|---|---|---|
| Your database | Yes | Only if startup prohibitive | SQL bugs are #1 integration failure |
| Your message queue | Yes | Only for unit tests | Serialization and routing bugs are common |
| External third-party APIs | No | HTTP mock server | You can't control their availability |
| File system | Depends | Use temp directories | Real FS catches path issues |
| Time/clocks | No | Inject a clock interface | Deterministic time-dependent tests |
| Random/UUID generation | No | Inject a generator | Deterministic assertions |
references/integration-isolation.md
---
title: Test Isolation — No Shared Mutable State
impact: CRITICAL
impactDescription: Eliminates ordering dependencies and intermittent failures
tags: integration, isolation, database, state-management, test-independence
---
## Test Isolation — No Shared Mutable State
Tests that share a database, global variable, or singleton between test cases create ordering dependencies. Test A seeds data that test B depends on. Test C deletes data that test D expects. The result: tests pass individually but fail when run together, or pass in one order and fail in another.
**Incorrect (shared global test data with ordering dependency):**
```typescript
// test-setup.ts — shared seed data loaded once
beforeAll(async () => {
await db.users.insert({ id: "user-1", name: "Alice" });
await db.users.insert({ id: "user-2", name: "Bob" });
});
// user.test.ts — depends on seed data from setup
test("lists all users", async () => {
const users = await service.listUsers();
expect(users).toHaveLength(2); // breaks if another test added/deleted users
});
// admin.test.ts — modifies shared data
test("admin can delete user", async () => {
await service.deleteUser("user-1");
// now "lists all users" test will fail
});
```
**Correct (each test owns its data, clean state per test):**
```typescript
describe("UserService", () => {
let db: TestDatabase;
beforeEach(async () => {
db = await TestDatabase.create();
// OR: truncate all tables
// OR: use transaction wrapper that rolls back after each test
});
afterEach(() => db.destroy());
test("lists users returns only users in database", async () => {
// Arrange — this test creates exactly what it needs
await db.users.insert({ id: "user-1", name: "Alice" });
await db.users.insert({ id: "user-2", name: "Bob" });
const service = new UserService(db);
// Act
const users = await service.listUsers();
// Assert
expect(users).toHaveLength(2);
});
test("delete user removes from database", async () => {
// Arrange — independent of other tests
await db.users.insert({ id: "user-1", name: "Alice" });
const service = new UserService(db);
// Act
await service.deleteUser("user-1");
// Assert
const remaining = await service.listUsers();
expect(remaining).toHaveLength(0);
});
});
```
```go
func TestUserService_ListUsers(t *testing.T) {
db := setupTestDB(t) // fresh DB per test
t.Cleanup(func() { db.Close() })
// Seed only what this test needs
db.Exec("INSERT INTO users (id, name) VALUES ($1, $2)", "1", "Alice")
db.Exec("INSERT INTO users (id, name) VALUES ($1, $2)", "2", "Bob")
svc := NewUserService(db)
users, err := svc.ListUsers(context.Background())
require.NoError(t, err)
assert.Len(t, users, 2)
}
```
Run tests in random order (`go test -shuffle=on`, Jest `--randomize`) to surface hidden ordering dependencies.
references/integration-real-deps.md
---
title: Real Dependencies Over Mocks in Integration Tests
impact: CRITICAL
impactDescription: Catches the #1 source of production bugs — component contract mismatches
tags: integration, real-dependencies, testcontainers, database, mocking-antipattern
---
## Real Dependencies Over Mocks in Integration Tests
A unit test with a mocked database tests your code's interaction with the mock. An integration test with a real database tests your code's interaction with the actual database — including query correctness, transaction behavior, constraint enforcement, and migration validity. Most production bugs live at component boundaries, not inside individual functions.
**Incorrect (mocking the database in an integration test):**
```go
type MockDB struct {
users map[string]User
}
func (m *MockDB) GetByID(ctx context.Context, id string) (User, error) {
u, ok := m.users[id]
if !ok { return User{}, ErrNotFound }
return u, nil
}
func TestUserService_CreateAndRetrieve(t *testing.T) {
// This test passes even if the real SQL has a typo,
// the migration is broken, or constraints are wrong
mock := &MockDB{users: map[string]User{
"123": {ID: "123", Email: "test@example.com"},
}}
svc := NewUserService(mock)
user, _ := svc.GetByID(context.Background(), "123")
assert.Equal(t, "test@example.com", user.Email)
// Green in CI, broken in production
}
```
**Correct (real database via testcontainers):**
```go
func TestUserService_CreateAndRetrieve(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
db := setupTestDB(t) // real postgres via testcontainers
defer db.Close()
svc := NewUserService(db)
// Create
user, err := svc.Create(context.Background(), CreateUserInput{
Email: "test@example.com",
Name: "Test User",
})
require.NoError(t, err)
require.NotEmpty(t, user.ID)
// Retrieve — exercises the real SQL query
found, err := svc.GetByID(context.Background(), user.ID)
require.NoError(t, err)
assert.Equal(t, "test@example.com", found.Email)
assert.Equal(t, "Test User", found.Name)
}
```
```typescript
describe("POST /api/findings", () => {
let app: Application;
let db: TestDatabase;
beforeAll(async () => {
db = await TestDatabase.create(); // real postgres or sqlite
app = createApp({ database: db });
});
afterAll(() => db.destroy());
it("creates a finding and returns it with generated ID", async () => {
const response = await request(app)
.post("/api/findings")
.send({ title: "SQL injection in login", severity: "critical" })
.expect(201);
expect(response.body.id).toBeDefined();
// Verify it's actually persisted — not just returned
const fromDB = await db.findings.findById(response.body.id);
expect(fromDB.title).toBe("SQL injection in login");
});
});
```
Use real dependencies where feasible. Mock only what you can't control (external third-party APIs, time, randomness).
references/philosophy-confidence.md
---
title: Justified Confidence Over Coverage Metrics
impact: CRITICAL
impactDescription: Eliminates false confidence from meaningless tests
tags: philosophy, confidence, coverage, metrics, testing-purpose
---
## Justified Confidence Over Coverage Metrics
Tests exist for one reason: to give you justified confidence that your software does what you intend and won't break what already works when you change it. The question is never "do we have enough tests?" The question is: "if this change introduced a bug, which test would catch it?"
**Incorrect (testing for coverage, not confidence):**
```typescript
// 100% coverage, zero confidence — tests assert nothing meaningful
describe("UserService", () => {
test("create user", () => {
const service = new UserService(mockDB);
const result = service.create({ name: "test" });
expect(result).toBeDefined(); // proves nothing
});
test("delete user", () => {
const service = new UserService(mockDB);
service.delete("123");
expect(true).toBe(true); // literally meaningless
});
test("get user", () => {
const service = new UserService(mockDB);
const result = service.getByID("123");
expect(typeof result).toBe("object"); // vacuous assertion
});
});
```
**Correct (testing behavior that matters):**
```typescript
// Lower coverage percentage, dramatically higher confidence
describe("UserService", () => {
test("create user persists to database and returns generated ID", async () => {
const db = await TestDatabase.create();
const service = new UserService(db);
const user = await service.create({ name: "Alice", email: "alice@example.com" });
expect(user.id).toBeDefined();
const persisted = await db.users.findById(user.id);
expect(persisted.name).toBe("Alice");
expect(persisted.email).toBe("alice@example.com");
});
test("create user rejects duplicate email", async () => {
const db = await TestDatabase.create();
const service = new UserService(db);
await service.create({ name: "Alice", email: "alice@example.com" });
await expect(
service.create({ name: "Bob", email: "alice@example.com" })
).rejects.toThrow("email already exists");
});
test("delete user removes from database and revokes active sessions", async () => {
const db = await TestDatabase.create();
const sessions = new TestSessionStore();
const service = new UserService(db, sessions);
const user = await service.create({ name: "Alice", email: "alice@example.com" });
await sessions.create(user.id);
await service.delete(user.id);
expect(await db.users.findById(user.id)).toBeNull();
expect(await sessions.getActive(user.id)).toHaveLength(0);
});
});
```
Each test verifies a specific behavior that, if broken, would matter in production. When a test fails, you know exactly what behavior regressed.
references/philosophy-pyramid.md
---
title: The Testing Shape — Type System as Foundation
impact: CRITICAL
impactDescription: Maximizes confidence per minute of engineering time
tags: philosophy, pyramid, type-system, static-analysis, test-levels
---
## The Testing Shape — Type System as Foundation
The classic testing pyramid (many unit tests, fewer integration tests, few E2E tests) was good advice when integration tests were slow and expensive. Modern tooling has changed the cost equation. Optimize for confidence per minute of engineering time.
**Incorrect (classic pyramid — unit test everything, mock aggressively):**
```typescript
// Dozens of unit tests with mocks for a simple CRUD handler
describe("createFinding", () => {
test("calls repository.save", () => {
const mockRepo = { save: jest.fn().mockResolvedValue({ id: "1" }) };
const handler = new FindingHandler(mockRepo);
handler.create({ title: "XSS", severity: "high" });
expect(mockRepo.save).toHaveBeenCalledTimes(1);
});
test("calls validator.validate", () => {
const mockValidator = { validate: jest.fn().mockReturnValue(true) };
const handler = new FindingHandler(mockRepo, mockValidator);
handler.create({ title: "XSS", severity: "high" });
expect(mockValidator.validate).toHaveBeenCalled();
});
// 15 more tests asserting internal call patterns...
// Every refactor breaks these tests even when behavior is unchanged
});
```
**Correct (invest in integration tests and the type system):**
```typescript
// TypeScript strict mode catches type errors at zero runtime cost
// eslint and biome catch code quality issues statically
// Then: fewer but more valuable integration tests
describe("POST /api/findings", () => {
let app: Application;
let db: TestDatabase;
beforeAll(async () => {
db = await TestDatabase.create();
app = createApp({ database: db });
});
afterAll(() => db.destroy());
test("creates finding and persists to database", async () => {
const response = await request(app)
.post("/api/findings")
.send({ title: "SQL Injection", severity: "critical", service: "auth" })
.expect(201);
expect(response.body.id).toBeDefined();
const fromDB = await db.findings.findById(response.body.id);
expect(fromDB.title).toBe("SQL Injection");
});
test("rejects invalid severity with 400", async () => {
await request(app)
.post("/api/findings")
.send({ title: "Test", severity: "banana" })
.expect(400);
});
});
```
The type system is the base. Integration tests are the middle. Unit tests target complex logic. E2E tests cover the 3-5 critical user journeys.
references/pipeline-coverage.md
---
title: Coverage Ratchet — Never Decrease, Don't Target
impact: MEDIUM-HIGH
impactDescription: Ensures new code is tested without requiring retroactive coverage of legacy code
tags: pipeline, coverage, ratchet, threshold, ci-gate
---
## Coverage Ratchet — Never Decrease, Don't Target
Don't set a global coverage target ("all code must be 80% covered"). That either forces busywork on legacy code or encourages meaningless tests to hit the number. Instead, set a ratchet: coverage of CHANGED files must not decrease. New code must be tested; existing code isn't penalized retroactively.
**Incorrect (global coverage target drives meaningless tests):**
```typescript
// Developer needs to ship a critical fix but coverage is at 79.8%
// Writes tests like these to cross the 80% threshold:
test("constructor exists", () => {
const service = new FindingService(db);
expect(service).toBeDefined(); // +2% coverage, 0% confidence
});
test("getter returns value", () => {
const finding = new Finding({ title: "test" });
expect(finding.title).toBe("test"); // tests the language, not the code
});
// Coverage: 80.1%. Confidence: unchanged. Time wasted: real.
```
**Correct (ratchet on changed files, higher floor for critical paths):**
```yaml
# CI configuration — coverage rules
coverage:
# Changed files must not decrease in coverage
diff-threshold: 0%
# Critical paths have a higher floor
overrides:
- path: "src/auth/**"
min: 90%
- path: "src/billing/**"
min: 90%
- path: "src/data/migrations/**"
min: 85%
# Glue code and config have relaxed expectations
exclude:
- "src/config/**"
- "src/generated/**"
- "**/*.d.ts"
```
```typescript
// New code gets real tests — the ratchet ensures this naturally
test("rate limiter blocks after threshold", async () => {
const limiter = new RateLimiter({ maxRequests: 3, windowMs: 1000 });
const clock = new FakeClock();
// First 3 requests succeed
for (let i = 0; i < 3; i++) {
expect(await limiter.check("user-1", { clock })).toBe(true);
}
// 4th request blocked
expect(await limiter.check("user-1", { clock })).toBe(false);
// After window expires, requests succeed again
clock.advance(1001);
expect(await limiter.check("user-1", { clock })).toBe(true);
});
```
Coverage as a ratchet gives teams the right incentive: test what you change, invest testing effort where it matters most (auth, payments, data integrity), and don't waste time writing vacuous tests to hit a number.
references/pipeline-stages.md
---
title: Staged CI Pipeline with Fail-Fast
impact: MEDIUM-HIGH
impactDescription: Faster feedback loops and reduced CI cost through progressive validation
tags: pipeline, ci, stages, fail-fast, static-analysis, deploy-gates
---
## Staged CI Pipeline with Fail-Fast
Structure your CI pipeline as stages with increasing scope and cost. If the cheap fast stage fails, don't run the expensive slow stages. Every stage is a gate — only proceed if the previous stage passes.
**Incorrect (single monolithic test stage):**
```yaml
# All tests run in one undifferentiated stage
test:
script:
- npm run lint # 10 seconds
- npm run typecheck # 15 seconds
- npm run test:unit # 30 seconds
- npm run test:integration # 3 minutes (spins up databases)
- npm run test:e2e # 5 minutes (deploys to staging)
# If lint fails, still waits for all 8+ minutes of subsequent stages
# No parallelism, no fail-fast between stages
```
**Correct (staged pipeline with fail-fast gates):**
```yaml
# Stage 1: Static Analysis (seconds) — catches obvious issues cheaply
static-analysis:
parallel:
- typecheck:
script: tsc --noEmit
- lint:
script: eslint . && biome check .
- security:
script: semgrep --config auto src/
# Stage 2: Unit Tests (seconds to low minutes) — requires Stage 1 pass
unit-tests:
needs: [static-analysis]
parallel:
- unit:
script: vitest run --project unit
- property:
script: vitest run --project property
# Stage 3: Integration Tests (minutes) — requires Stage 2 pass
integration-tests:
needs: [unit-tests]
services:
- postgres:16
- redis:7
parallel:
- api-integration:
script: vitest run --project integration
- contract-verification:
script: npm run test:contracts
# Stage 4: E2E Smoke Tests (minutes) — requires Stage 3 pass
e2e-smoke:
needs: [integration-tests]
script: |
deploy --env ephemeral
npm run test:e2e:critical-paths
teardown --env ephemeral
# Stage 5: Performance Gate (optional) — requires Stage 3 pass
performance:
needs: [integration-tests]
allow_failure: false
script: |
k6 run --threshold 'p99<200' load-tests/api.js
```
Fail fast: stage 1 failure (type error, lint violation) gives feedback in seconds, not minutes. Parallel within stages: unit tests, linters, and security scans all run concurrently. Progressive cost: only spin up databases and staging environments after cheap checks pass.
references/prod-verification.md
---
title: Production Verification — Canary, Flags, and Observability
impact: MEDIUM-HIGH
impactDescription: Catches deployment-specific and real-traffic failures that CI cannot
tags: production, canary, feature-flags, observability, health-checks, deploy-safety
---
## Production Verification — Canary, Flags, and Observability
Tests in CI verify the build. Production has real traffic, real data, and real failure modes that don't exist in test environments. Production verification extends testing past the CI boundary: canary deploys limit blast radius, feature flags decouple deployment from release, and observability turns alerts into continuous assertions.
**Incorrect (deploy and hope — CI passed so we're fine):**
```typescript
// Deployment pipeline
async function deploy(version: string) {
await buildAndPush(version);
await rollOutToAllInstances(version); // 100% traffic immediately
console.log("Deployed! CI passed so we're good.");
// Missing: environment variables changed, staging DB ≠ prod DB,
// traffic patterns differ, memory limits differ, secrets rotated...
}
```
**Correct (progressive verification with automated rollback):**
```typescript
// Canary deploy with metric-based promotion
async function deploy(version: string) {
// Step 1: Deploy to canary (5% of traffic)
await deployCanary(version, { trafficPercent: 5 });
// Step 2: Run smoke tests against canary
const smokeResults = await runSmokeTests({
target: "canary",
checks: [
{ endpoint: "/health", expectStatus: 200 },
{ endpoint: "/api/findings?limit=1", expectStatus: 200 },
{ name: "db-connectivity", expectHealthy: true },
],
});
if (!smokeResults.allPassed) {
await rollback(version);
throw new Error(`Smoke tests failed: ${smokeResults.failures}`);
}
// Step 3: Monitor canary metrics for 10 minutes
const metrics = await monitorCanary({
duration: "10m",
thresholds: {
errorRateDelta: 0.01, // <1% error rate increase vs baseline
p99LatencyDelta: 1.5, // <1.5x p99 latency vs baseline
businessMetricDelta: 0.05, // <5% conversion rate decrease
},
});
if (!metrics.withinThresholds) {
await rollback(version);
throw new Error(`Canary metrics degraded: ${metrics.violations}`);
}
// Step 4: Progressive rollout
for (const percent of [25, 50, 100]) {
await promoteCanary(version, { trafficPercent: percent });
await monitorForMinutes(5);
}
}
```
Observability as testing: error rate spikes mean something broke, latency spikes mean something degraded, new error types mean new code paths are failing. Structure alerts like test assertions — they verify expected behavior and fire when violated.
references/strategy-risk.md
---
title: Risk-Driven Test Investment
impact: CRITICAL
impactDescription: Focuses testing effort where bugs are most costly
tags: strategy, risk, priority, coverage-allocation, test-planning
---
## Risk-Driven Test Investment
Don't test everything equally. Test proportionally to risk: the probability of a bug multiplied by the cost of that bug reaching production. Authentication bugs, financial logic bugs, and data integrity bugs warrant aggressive multi-layered testing. Pure data transforms and logging warrant spot checks.
**Incorrect (uniform testing effort regardless of risk):**
```go
// Same level of testing for a critical auth boundary and a log formatter
func TestFormatLogMessage(t *testing.T) {
tests := []struct {
level string
msg string
want string
}{
{"info", "started", "[INFO] started"},
{"warn", "slow query", "[WARN] slow query"},
{"error", "failed", "[ERROR] failed"},
{"debug", "trace", "[DEBUG] trace"},
// 20 more cases for a simple string formatter...
}
for _, tt := range tests {
// Exhaustive testing of trivial logic
}
}
func TestAuthenticateUser(t *testing.T) {
// Meanwhile, auth has just one happy-path test
token := Authenticate("user", "pass")
if token == "" {
t.Fatal("expected token")
}
}
```
**Correct (test investment proportional to risk):**
```go
// Light testing for low-risk code
func TestFormatLogMessage(t *testing.T) {
// Spot check — the type system ensures the return type
got := FormatLogMessage("error", "connection refused")
if !strings.HasPrefix(got, "[ERROR]") {
t.Errorf("expected ERROR prefix, got %s", got)
}
}
// Aggressive testing for high-risk auth boundary
func TestAuthentication(t *testing.T) {
db := setupTestDB(t)
svc := NewAuthService(db)
t.Run("valid credentials return token with correct claims", func(t *testing.T) {
token, err := svc.Authenticate(ctx, "user@example.com", "correct-password")
require.NoError(t, err)
claims := parseToken(t, token)
assert.Equal(t, "user@example.com", claims.Email)
assert.WithinDuration(t, time.Now().Add(24*time.Hour), claims.ExpiresAt, time.Minute)
})
t.Run("invalid password returns error not token", func(t *testing.T) {
token, err := svc.Authenticate(ctx, "user@example.com", "wrong-password")
assert.Error(t, err)
assert.Empty(t, token)
})
t.Run("expired token rejected on validation", func(t *testing.T) { /* ... */ })
t.Run("tampered token rejected", func(t *testing.T) { /* ... */ })
t.Run("revoked token rejected", func(t *testing.T) { /* ... */ })
t.Run("concurrent sessions respected", func(t *testing.T) { /* ... */ })
t.Run("brute force protection triggers after N failures", func(t *testing.T) { /* ... */ })
t.Run("cross-tenant token rejected", func(t *testing.T) { /* ... */ })
}
```
High risk (auth, payments, data integrity): test every path, every edge case, every failure mode. Low risk (formatting, logging): type system plus spot checks.
references/unit-naming.md
---
title: Test Naming as Specification
impact: HIGH
impactDescription: Test names become living documentation of expected behavior
tags: unit, naming, readability, specification, documentation
---
## Test Naming as Specification
Test names should describe the behavior being verified, not the implementation being called. When a test fails, the name alone should tell you what behavior broke. Good test names read like specifications — they document what the system promises.
**Incorrect (names describe implementation, not behavior):**
```go
func TestCalculateDiscount(t *testing.T) {
// What about it? What's the expected behavior?
}
func TestParseInput(t *testing.T) {
// Fails — but was it a valid input that should parse, or invalid that should reject?
}
func TestHandleRequest(t *testing.T) {
// Which request? What's the expected outcome?
}
```
```typescript
test("createUser", () => { /* ... */ });
test("discount function", () => { /* ... */ });
test("error case", () => { /* ... */ });
```
**Correct (names describe behavior and conditions):**
```go
func TestDiscountCalculation_AppliesTieredDiscount(t *testing.T) { /* ... */ }
func TestDiscountCalculation_ReturnsZeroForUnknownTier(t *testing.T) { /* ... */ }
func TestParseSeverity_RejectsEmptyString(t *testing.T) { /* ... */ }
func TestParseSeverity_IsCaseInsensitive(t *testing.T) { /* ... */ }
func TestAuthMiddleware_RejectsExpiredToken(t *testing.T) { /* ... */ }
func TestAuthMiddleware_AllowsValidTokenForMatchingTenant(t *testing.T) { /* ... */ }
```
```typescript
test("applies tiered discount for gold customers over $100", () => { /* ... */ });
test("returns zero discount for unknown customer tier", () => { /* ... */ });
test("rejects duplicate email with descriptive error", () => { /* ... */ });
test("revoked token returns 401 even if not expired", () => { /* ... */ });
```
When the CI report shows `TestAuthMiddleware_RejectsExpiredToken FAIL`, you know exactly what broke without reading the test body.
references/unit-structure.md
---
title: Arrange-Act-Assert Structure
impact: HIGH
impactDescription: Makes test failures immediately diagnosable
tags: unit, structure, arrange-act-assert, aaa, readability
---
## Arrange-Act-Assert Structure
Every unit test has exactly three sections: Arrange (set up state), Act (execute the behavior), Assert (verify the result). If you can't clearly identify these three sections, the test is doing too much. One logical assertion per test — verify one behavior so failures tell you exactly what broke.
**Incorrect (mixed concerns, multiple behaviors, unclear structure):**
```typescript
test("discount", () => {
const order = { subtotal: 150_00, customerTier: "gold" };
const discount = calculateDiscount(order);
expect(discount).toBe(15_00);
const tax = calculateTax(order, discount);
expect(tax).toBe(13_50);
const shipping = calculateShipping(order);
expect(shipping).toBe(0);
const total = order.subtotal - discount + tax + shipping;
expect(total).toBe(148_50);
// When this fails, which calculation broke? Unknown.
});
```
**Correct (clear AAA structure, one behavior per test):**
```typescript
test("applies tiered discount for gold customers over $100", () => {
// Arrange
const order = { subtotal: 150_00, customerTier: "gold" };
// Act
const discount = calculateDiscount(order);
// Assert
expect(discount).toBe(15_00);
});
test("gold customers under $100 receive no discount", () => {
// Arrange
const order = { subtotal: 50_00, customerTier: "gold" };
// Act
const discount = calculateDiscount(order);
// Assert
expect(discount).toBe(0);
});
```
```go
func TestDiscountCalculation_AppliesTieredDiscount(t *testing.T) {
// Arrange
order := Order{Subtotal: 150_00, CustomerTier: "gold"}
// Act
discount := CalculateDiscount(order)
// Assert
if discount != 15_00 {
t.Errorf("expected 1500 discount for gold tier $150 order, got %d", discount)
}
}
```
When a test with clear AAA structure fails, the name tells you what behavior broke, and the assertion tells you how the actual result diverged from expected.
references/unit-table-driven.md
---
title: Table-Driven Tests for Multi-Case Logic
impact: HIGH
impactDescription: Scales to 50+ test cases without duplication
tags: unit, table-driven, parameterized, go, typescript, data-driven
---
## Table-Driven Tests for Multi-Case Logic
When testing a function with many input/output pairs, table-driven tests eliminate duplication and make it trivial to add new cases. Each row is a test case — name, inputs, expected outputs — and a single loop runs them all.
**Incorrect (duplicated test functions for each case):**
```go
func TestParseSeverity_Critical(t *testing.T) {
got, err := ParseSeverity("CRITICAL")
if err != nil { t.Fatal(err) }
if got != SeverityCritical { t.Errorf("got %v", got) }
}
func TestParseSeverity_CriticalLower(t *testing.T) {
got, err := ParseSeverity("critical")
if err != nil { t.Fatal(err) }
if got != SeverityCritical { t.Errorf("got %v", got) }
}
func TestParseSeverity_Unknown(t *testing.T) {
_, err := ParseSeverity("banana")
if err == nil { t.Fatal("expected error") }
}
func TestParseSeverity_Empty(t *testing.T) {
_, err := ParseSeverity("")
if err == nil { t.Fatal("expected error") }
}
// Repetitive — and adding a new severity level means writing another full function
```
**Correct (table-driven pattern):**
```go
func TestParseSeverity(t *testing.T) {
tests := []struct {
name string
input string
expected Severity
wantErr bool
}{
{"critical uppercase", "CRITICAL", SeverityCritical, false},
{"critical lowercase", "critical", SeverityCritical, false},
{"critical mixed case", "Critical", SeverityCritical, false},
{"high", "HIGH", SeverityHigh, false},
{"medium", "MEDIUM", SeverityMedium, false},
{"low", "LOW", SeverityLow, false},
{"unknown value", "banana", SeverityUnknown, true},
{"empty string", "", SeverityUnknown, true},
{"whitespace only", " ", SeverityUnknown, true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ParseSeverity(tt.input)
if (err != nil) != tt.wantErr {
t.Errorf("error = %v, wantErr = %v", err, tt.wantErr)
}
if got != tt.expected {
t.Errorf("got %v, want %v", got, tt.expected)
}
})
}
}
```
```typescript
// TypeScript equivalent using test.each
test.each([
{ input: "CRITICAL", expected: "critical", shouldThrow: false },
{ input: "critical", expected: "critical", shouldThrow: false },
{ input: "banana", expected: null, shouldThrow: true },
{ input: "", expected: null, shouldThrow: true },
])("parseSeverity($input) → $expected", ({ input, expected, shouldThrow }) => {
if (shouldThrow) {
expect(() => parseSeverity(input)).toThrow();
} else {
expect(parseSeverity(input)).toBe(expected);
}
});
```
Adding a new case is one line. The test structure never changes.