evals/evals.json
{
"$schema": "self-describing — no external schema; see README.md in this directory",
"skill": "testing",
"skill_path": ".claude/skills/core-engineering/testing/SKILL.md",
"description": "Exemplar eval set for the testing skill. Each case is a prompt plus assertions about the resulting behavior, not the resulting text. Run with skill-creator (/plugin install skill-creator@claude-plugins-official) in a fresh session per the eval-first policy in CONTRIBUTING.md.",
"cases": [
{
"id": "tdd-red-before-green",
"category": "positive",
"prompt": "Add a function that calculates the discount percentage between an original price and a sale price, and use it in the checkout summary.",
"expected": {
"assertions": [
"writes a failing test for the discount-percentage behavior before writing the implementation",
"the test is observed or described as failing (red) prior to the implementation existing",
"implementation is added only after the failing test is in place, and is the minimum code to pass it",
"test covers at least one edge case (e.g. sale price equal to or greater than original price) in addition to the happy path"
]
}
},
{
"id": "regression-test-before-bugfix",
"category": "positive",
"prompt": "There's a bug: passing an empty list to calculateAverage() throws an unhandled exception instead of returning 0.",
"expected": {
"assertions": [
"adds a regression test that reproduces the reported bug (empty list input) before changing implementation code",
"confirms or describes the regression test failing against the current (buggy) implementation",
"fixes the implementation only after the regression test exists",
"regression test passes after the fix, and is not deleted or skipped afterward"
]
}
},
{
"id": "edge-case-and-determinism-coverage",
"category": "positive",
"prompt": "Write tests for a function that parses a duration string like \"2h30m\" into seconds.",
"expected": {
"assertions": [
"covers the happy path plus boundary/edge cases (e.g. zero duration, missing unit, malformed input)",
"tests are deterministic — no reliance on wall-clock time, random values, or external state without control/mocking",
"tests are isolated from each other (no shared mutable state between test cases)",
"test names describe the behavior under test, not just the function name"
]
}
},
{
"id": "near-miss-run-suite-should-not-trigger-methodology",
"category": "negative",
"prompt": "Run the test suite and tell me if anything is failing.",
"expected": {
"assertions": [
"does not invoke TDD red-green-refactor guidance — there is no new code or bug to drive a test for",
"does not lecture on test-quality standards, verification-loop framing, or regression-test policy",
"simply runs the existing test command and reports results",
"the skill's methodology content is a near-miss here: the request is about executing tests, not authoring or fixing them"
]
}
}
]
}
evals/README.md
# Evals: testing skill
Exemplar eval set for the `testing` skill. See CONTRIBUTING.md's eval-first policy
before adding or changing a skill.
## Running
1. `/plugin install skill-creator@claude-plugins-official`
2. Point the plugin's eval runner at `evals.json` in this directory.
3. Run each case **twice**: once with the `testing` skill available, once with it
disabled/removed — in a **fresh session** each time. Do not reuse the session you
used to author the eval cases; authoring context masks the gaps a fresh session
would expose.
4. For `category: "negative"` cases, confirm the skill's methodology content does
*not* fire — that's the pass condition, not a bug in the harness.
## Recording results
Record the with-skill vs without-skill comparison as a dated file under
`scratchpad/` while iterating, then promote a summary (sample size, task set,
raw pass/fail per case, date, model) into an `artifacts/` note before citing any
improvement in README.md or docs — per the evidence policy in CONTRIBUTING.md.
Do not assert an improvement without that raw data attached.
SKILL.md
---
name: testing
description: Write effective tests for code quality and reliability. Use when implementing features, fixing bugs, improving coverage, or practicing TDD/test-driven development. Covers unit, integration, and E2E testing.
---
# Testing Software
## Verification Loop First
Give every change a check it can run before calling it done: red-then-green for new logic, a failing-then-passing regression test for bug fixes, or the existing suite for anything else. No change ships without one.
**Red-green-observe, not red-green-assume**: "confirm it fails" (the Regression workflow item below, and TDD's "watch it fail" step) means actually *run* the test and *read* the failure output before touching the fix or the implementation — never reason your way to "this must fail" and skip the run.
**Falsifiable done-when**: before writing a feature's implementation, state one concrete, checkable condition that defines done — not "should work now," a condition a test or command can confirm or refute.
**Evidence over narration**: a completion claim is only as good as what it cites — a pasted test-run result, a real exit code, a pushed commit SHA. Describing what the code should now do is not evidence; re-running the check and quoting its output is.
## MCP Tools
**Chrome DevTools** (E2E testing):
- Automate user flows in real browser
- Capture screenshots for visual regression
- Run Lighthouse for accessibility testing
- Profile performance during test runs
## Workflow
- [ ] **Analyze**: Use Glob and Grep to identify untested code
- [ ] **Unit Tests**: Cover all public functions
- [ ] **Edge Cases**: Test boundaries and error conditions
- [ ] **Integration**: Test external dependencies
- [ ] **E2E**: Use Chrome DevTools for browser automation
- [ ] **Regression**: Add a test that reproduces the bug, confirm it fails against the current code, then fix — keep it passing and in the suite afterward (never delete or skip it)
## Test-Driven Development
When writing new logic, default to red-green-refactor:
1. **Red**: Write a failing test for one behavior before writing the implementation.
2. **Green**: Write the minimum code needed to make that test pass.
3. **Refactor**: Clean up structure only while the suite is green — no new behavior in this step.
- One behavior per test — if a test needs "and" to describe it, split it.
- Never write production code without a failing test driving it, except for spikes/throwaway exploration (delete or backfill tests before merging).
- Don't refactor and add behavior in the same commit; commit on green before switching hats (see `code-quality.md` Two Hats Rule).
**When TDD is the right default**: new business logic, bug fixes (write the regression test first, watch it fail, then fix), and any code with clear input/output contracts.
**When it isn't**: exploratory spikes, throwaway scripts, UI layout/styling tweaks, and generated boilerplate — write tests after the shape stabilizes instead.
## Test Quality Standards
### Deterministic
Tests must produce the same result every time — no reliance on wall-clock time, random values, or uncontrolled external state.
### Isolated
Tests must not depend on each other or share mutable state.
### Clear
Test names describe the behavior under test, not just the function name.