examples/before-after.md
# Worked example — 5 runs of history → one self-improvement suggestion
## The history (`.refactor-chain/history.jsonl`, 5 web-lane runs)
```jsonl
{"lane":"web","case":"refactor-web","confidence":0.55,"clarified":true,"steps":[{"skill":"refactor-web-01-structure","attempts":1,"healed":false,"status":"done"},{"skill":"refactor-web-03-components","attempts":2,"healed":true,"status":"done"}],"outcome":"done","at":"2026-06-02T…"}
{"lane":"web","case":"refactor-web","confidence":0.58,"clarified":true,"steps":[{"skill":"refactor-web-03-components","attempts":2,"healed":true,"status":"done"}],"outcome":"done","at":"2026-06-11T…"}
{"lane":"web","case":"refactor-web","confidence":0.61,"clarified":false,"steps":[{"skill":"refactor-web-03-components","attempts":1,"healed":false,"status":"done"}],"outcome":"done","at":"2026-06-20T…"}
{"lane":"web","case":"refactor-web","confidence":0.66,"clarified":false,"steps":[{"skill":"refactor-web-03-components","attempts":2,"healed":true,"status":"done"}],"outcome":"done","at":"2026-06-27T…"}
{"lane":"web","case":"refactor-web","confidence":0.70,"clarified":false,"steps":[{"skill":"refactor-web-03-components","attempts":2,"healed":true,"status":"done"}],"outcome":"done","at":"2026-07-01T…"}
```
## Analysis
```
$ node scripts/checklist.mjs --history /repo
{
"skill": "refactor-improve",
"runs": 5,
"troubleSteps": [
{ "skill": "refactor-web-03-components", "multiAttempt": 4, "seen": 5, "rate": 0.8 }
],
"cleanLanes": [ { "lane": "web", "runs": 5 } ],
"frictionLanes": [],
"oneImprovement": {
"kind": "pre-step-safety",
"target": "refactor-web-03-components",
"why": "needed >1 attempt in 4/5 runs",
"suggestion": "Add a characterization test / checkpoint before refactor-web-03-components so it stops drifting here."
}
}
```
## Patterns found (surfaced, not all acted on)
- **Trouble step:** `refactor-web-03-components` needed a second attempt in **4 of 5** runs (80%).
That's a systemic drift point in this repo's component-tier split — not bad luck.
- **Clean lane:** the web lane is `done` in all 5 runs → the confidence prior has been climbing
(0.55 → 0.70) and diagnosis stopped asking to clarify after run 2. Trust it more.
## The ONE improvement
> Add a characterization test that pins the rendered output of the components being re-tiered,
> and take a checkpoint, **before** `refactor-web-03-components` runs. That step drifts here 4 out
> of 5 times; pinning its behavior up front turns the recurring self-heal into a clean pass.
*(Deliberately not also suggesting: reordering the lane, extra logging, a naming-convention lint —
those are lower-leverage. Self-improvement is one step.)*
## Confidence prior picture
- Web lane: 5 confirmed `done` runs → `historyPrior` adds `+min(2,5)*0.5 = +1.0` to the web vote on
the next run. Next diagnosis will classify this repo's web refactors with high confidence and
won't ask to clarify. (This skill confirms the mechanism; it does not hand-edit the number.)
## Plain-language close
> The work shipped. Looking back at your last 5 refactors here, one step —
> splitting components into tiers — keeps needing a second try. My one suggestion: pin that step's
> behavior with a quick test before it runs next time, and it'll go clean. Everything else is in
> good shape; the web lane is now high-confidence for this repo. Want me to note that suggestion
> for next time?
references/method.md
# refactor-improve — full method: the improvement retro & the confidence prior
Self-improvement = small, steady improvement. This skill closes each run by learning from this repo's own
history and proposing **one** thing to sharpen — no more. It reads; it never edits code.
## The history source
`<target>/.refactor-chain/history.jsonl` is append-only. The harness writes one retro line per run
on `orchestrate.mjs reset` (via `diagnose.mjs learn`). Each line:
```json
{
"lane": "web",
"case": "refactor-web",
"mode": "ask",
"confidence": 0.62,
"clarified": true,
"steps": [ { "skill": "refactor-web-03-components", "attempts": 2, "healed": true, "status": "done" }, ... ],
"outcome": "done",
"at": "ISO-8601"
}
```
Parse it with `node scripts/checklist.mjs --history <target>`.
## Make history complete first
If this run's retro isn't in the file yet (`reset` hasn't run), record it before analyzing —
otherwise the newest, most relevant data point is missing. The retro shape mirrors what
`orchestrate.mjs reset` builds.
## Pattern detection
| Pattern | Signal | Meaning |
|---|---|---|
| **Trouble step** | a step needs `attempts > 1` (or `healed`) in ≥2 runs and ≥50% of its appearances | this step reliably drifts here — pre-empt it |
| **Diagnosis friction** | a lane is `clarified:true` in ≥50% of its runs | the harness can't tell what this repo needs — add a signal |
| **Low-confidence lane** | recurring `confidence` below ~0.5 for a lane | classification is shaky — a repo cue would help |
| **Clean streak** | a lane has ≥3 runs all `outcome:"done"` | trust it more — the prior already nudges it up |
| **Recurring fail reason** | the same fail note across runs (from step notes) | a systemic cause worth naming |
## The one-improvement rule
Surfacing several patterns is fine. Recommending several changes is not. Rank candidates by
**leverage = how often it bites × how cheap the fix**, and propose the single top one. Typical shapes:
- **Pre-step safety:** "Add a characterization test / checkpoint before `<trouble step>`" — the most
common and highest-leverage fix for a step that keeps drifting.
- **Diagnosis cue:** "Add a repo signal so the `<lane>` lane auto-detects without a clarify."
- **Plan tweak:** "The plan should checkpoint before `<step>`" / "run `<conditional guard>` earlier."
If there's no recurring pain point, say so — "the chain is running clean here" is a valid retro.
If there's only one run, say "not enough history for a pattern yet" rather than over-fitting.
## The confidence prior (already automatic — just confirm)
`diagnose.mjs historyPrior` reads `history.jsonl` on the *next* run and, for each lane with
confirmed `outcome:"done"` runs, nudges that lane's vote up (`+min(2,n)*0.5`). So recording clean
outcomes literally makes the next diagnosis more confident for this repo. This skill does **not**
hand-edit confidence — it confirms the mechanism will pick up the newly-recorded outcome and reports
the resulting picture (e.g. "web lane now has 5 clean runs → high prior").
## Optional: plugin refresh
If the one improvement is a change to the chain/config itself (not just a note for next time),
mention a `/update`-style plugin-refresh so the change is actually loaded — but only propose it;
the human approves and applies.
## Output
Fill `templates/output.md`: the patterns found, the single proposed improvement (with its
supporting count), and the updated per-repo confidence picture.
scripts/checklist.mjs
#!/usr/bin/env node
/**
* refactor-improve — checklist + history analyzer (self-improvement). Zero deps.
* node checklist.mjs -> prints the step checklist as JSON
* node checklist.mjs --history <dir> -> parses .refactor-chain/history.jsonl and surfaces
* recurring patterns + a one-improvement candidate.
*
* History line schema (written by orchestrate.mjs reset -> diagnose.mjs learn):
* { lane, case, mode, confidence, clarified, steps:[{skill,attempts,healed,status}], outcome, at }
*/
import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path";
const SKILL = "refactor-improve";
const PHASE = "improve";
const STEPS = [
{ id: 1, step: "read-history", detail: "Load .refactor-chain/history.jsonl (append-only retros the harness writes on reset)." },
{ id: 2, step: "confirm-recorded", detail: "Ensure this run's retro is appended; record it first if reset hasn't run." },
{ id: 3, step: "find-patterns", detail: "Steps repeatedly needing >1 attempt / self-heal; often-clarified lanes; recurring fail reasons; clean streaks." },
{ id: 4, step: "pick-one", detail: "Rank by leverage (frequency x cheapness); propose EXACTLY ONE improvement." },
{ id: 5, step: "strengthen-prior", detail: "Confirm harness historyPrior will read updated history; note /update if it's a chain/config change." },
{ id: 6, step: "report", detail: "Fill templates/output.md into a Self-improvement Retro: patterns + the one improvement + confidence picture." },
];
function analyze(dir) {
const hf = join(dir, ".refactor-chain", "history.jsonl");
if (!existsSync(hf)) return { runs: 0, note: "no history yet — not enough for a pattern" };
const runs = readFileSync(hf, "utf8").trim().split("\n").filter(Boolean).map((l) => { try { return JSON.parse(l); } catch { return null; } }).filter(Boolean);
if (runs.length < 2) return { runs: runs.length, note: "only one run — record more before over-fitting a pattern", lastOutcome: runs[0]?.outcome || null };
const stepStats = {}; // skill -> { seen, multiAttempt, healed }
const laneStats = {}; // lane -> { runs, done, clarified }
for (const r of runs) {
const L = (laneStats[r.lane] ||= { runs: 0, done: 0, clarified: 0 });
L.runs++; if (r.outcome === "done") L.done++; if (r.clarified) L.clarified++;
for (const s of r.steps || []) {
const S = (stepStats[s.skill] ||= { seen: 0, multiAttempt: 0, healed: 0 });
S.seen++; if ((s.attempts || 1) > 1) S.multiAttempt++; if (s.healed) S.healed++;
}
}
// recurring trouble steps: needed >1 attempt in a majority of appearances (min 2)
const troubleSteps = Object.entries(stepStats)
.filter(([, v]) => v.seen >= 2 && v.multiAttempt >= 2 && v.multiAttempt / v.seen >= 0.5)
.map(([skill, v]) => ({ skill, multiAttempt: v.multiAttempt, seen: v.seen, rate: +(v.multiAttempt / v.seen).toFixed(2) }))
.sort((a, b) => b.multiAttempt - a.multiAttempt);
// lanes with a clean streak -> trust more; lanes often clarified -> diagnosis friction
const cleanLanes = Object.entries(laneStats).filter(([, v]) => v.runs >= 3 && v.done === v.runs).map(([lane, v]) => ({ lane, runs: v.runs }));
const frictionLanes = Object.entries(laneStats).filter(([, v]) => v.runs >= 2 && v.clarified / v.runs >= 0.5).map(([lane, v]) => ({ lane, clarified: v.clarified, runs: v.runs }));
// one-improvement candidate = the single highest-frequency trouble step, else a friction lane
const candidate = troubleSteps[0]
? { kind: "pre-step-safety", target: troubleSteps[0].skill, why: `needed >1 attempt in ${troubleSteps[0].multiAttempt}/${troubleSteps[0].seen} runs`, suggestion: `Add a characterization test / checkpoint before ${troubleSteps[0].skill} so it stops drifting here.` }
: frictionLanes[0]
? { kind: "diagnosis", target: frictionLanes[0].lane, why: `clarified in ${frictionLanes[0].clarified}/${frictionLanes[0].runs} runs`, suggestion: `Add a repo signal so the ${frictionLanes[0].lane} lane is auto-detected without a clarify.` }
: { kind: "none", suggestion: "No recurring pain point — the chain is running clean here." };
return { runs: runs.length, troubleSteps, cleanLanes, frictionLanes, oneImprovement: candidate };
}
const argv = process.argv.slice(2);
const hi = argv.indexOf("--history");
if (hi >= 0) {
process.stdout.write(JSON.stringify({ skill: SKILL, ...analyze(argv[hi + 1] || process.cwd()) }, null, 2) + "\n");
} else {
process.stdout.write(JSON.stringify({ skill: SKILL, phase: PHASE, steps: STEPS }, null, 2) + "\n");
}
SKILL.md
---
name: refactor-improve
description: "Use this skill at the very end of a refactor-chain run for the improvement retro — it reads the project's own run history (.refactor-chain/history.jsonl), surfaces recurring failure reasons and patterns across past runs, and proposes exactly ONE concrete improvement for next time, feeding the per-repo confidence prior the diagnostic harness already uses. Trigger phrases: \"what can we do better next time\", \"run the retro\", \"self-improvement\", \"why does this keep failing\", \"learn from this run\", \"improve the chain for this repo\", \"update the pipeline\". This is the improve phase of the refactor-chain pipeline; it runs after ship, closing the loop. Advisory — it reflects and recommends; it never edits product code, and it only proposes plugin/config changes for the human to approve."
---
# Self-improvement Improve — refactor-chain · improve
**Bundle:** refactor-chain (self-diagnosing, self-healing fix-it pipeline).
**Phase:** improve · **Prerequisite:** at least one recorded run (this one) in `history.jsonl` · **Next:** none — this closes the loop.
**Adaptivity / conditional:** repo-agnostic and advisory. It learns per-repo from `history.jsonl` and strengthens the confidence prior the harness already reads; it proposes one improvement and never changes code.
## Purpose
A pipeline that never reflects can't get better. At the end of a run, this skill does the self-improvement
retro: it reads this project's accumulated run history — every past chain's lane, confidence,
per-step attempts, self-heals, and outcome — finds the patterns that keep recurring (a step that
always needs two attempts here, a lane that's often misclassified, a fail reason that shows up
again and again), and proposes **exactly one** concrete, high-leverage improvement for next time.
It also confirms the run's outcome is recorded so the harness's per-repo confidence prior gets
stronger. One improvement, not twenty — self-improvement is small, steady, and actually adopted.
## When to use
- The run has shipped (or been parked) and you want the "what did we learn?" pass.
- Someone says "run the retro", "self-improvement", "what can we do better", "why does this keep failing here?".
- The orchestrator reaches the improve phase after ship.
- Any time you want to review this repo's refactor-chain track record and pick one thing to sharpen.
## What I'll tell you (plain-language / ADHD-friendly)
- "The work is done — now a quick look-back so next time goes smoother. I'm only reading history and suggesting one thing; I won't change anything."
- "Across your last 5 runs, the `web-03-components` step needed a second attempt 4 times. That's a pattern, not bad luck."
- "My one suggestion: add a characterization test for the component-tier split before that step runs, so it doesn't drift. Want me to note that for next time?"
- "Good news too — the web lane is now high-confidence for this repo (5 clean runs), so the chain will stop second-guessing it."
- "Just one improvement on purpose. Piling on ten changes is how retros get ignored. Say 'show technical details' for the full history breakdown."
- "You corrected me twice this run — 'keep the barrel exports' and 'this is a web repo'. I've turned both into proposals for next time; nothing changes unless you approve them."
## Method
1. **Read the history.** Load `<target>/.refactor-chain/history.jsonl` (append-only retros written
by the harness on each `reset`). Run `node scripts/checklist.mjs --history <target>` to parse and
summarize it. Each line has `{lane, case, confidence, clarified, steps:[{skill,attempts,healed,status}], outcome, at}`.
2. **Confirm this run is recorded.** Ensure the just-finished run appended its retro. If not
(e.g. `reset` hasn't run), record it before analyzing, so history is complete.
3. **Find the recurring patterns:** steps that repeatedly needed >1 attempt or self-heal; lanes
often clarified/misdiagnosed (low confidence, or corrected); fail reasons that recur across runs;
lanes with a clean streak (candidates to trust more).
4. **Pick ONE improvement.** Rank candidates by leverage (how often it bites × how cheap the fix)
and propose the single highest-leverage one — e.g. "add a pre-step characterization test for X",
"raise the confidence prior for lane Y", "the plan should checkpoint before step Z". Resist listing more.
5. **Strengthen the prior (already automatic).** Confirm the harness's `historyPrior` will read the
updated history (confirmed `outcome:"done"` runs nudge that lane's confidence up next time).
Optionally note a `/update`-style plugin-refresh if the improvement is a chain/config change.
6. **Corrections capture.** Sweep this run's state notes for user corrections and judgment
calls recorded mid-run — moments where the human overrode the chain ("no, keep the barrel
exports", "that lane guess was wrong, this is a web repo", "don't touch generated files") or
settled an ambiguity the pipeline couldn't. Each becomes a concrete improvement **proposal**
at retro time: what was corrected, what the chain would do differently next time (a scope
default, a lane hint, a plan-gate question, a guideline tweak), stated in one sentence and
offered for the human to approve. **Human-approved only — never auto-applied.** These
proposals sit alongside (and may become) the one self-improvement improvement in step 4; when a
correction-derived proposal and a history-derived one compete, the same leverage ranking
picks the single winner, and the rest are recorded as candidates for future retros.
7. **Automation recommendations (adopted from claude-automation-recommender).** When a pattern in
step 3 or a correction in step 6 is *mechanizable* — a class of issue a standing automation would
prevent — surface it as a concrete recommendation the human can accept: a git pre-commit/pre-push
hook, a CI check (a new `.github/workflows` step or the refactor-chain Action), a formatter/linter
config, a small script, or a scheduled task. Frame each as "this kept biting → here's the automation
that ends it", with the exact file to add. **Recommendation only — never installed without approval;**
nothing that changes standing project configuration is applied silently (same contract as decision
checkpoints). Recorded in the retro under "Automations worth adding".
8. Fill `templates/output.md` into a Self-improvement Retro (patterns + the one improvement + the
corrections-capture proposals + automation recommendations + the updated per-repo confidence
picture). See `references/method.md` for the pattern-detection rules.
## Guardrails
- **Advisory only. Never edits product code.** It reads history and proposes; any plugin/config
change is offered for the human to approve, not applied.
- **Exactly one improvement.** Self-improvement is one small step. Surfacing multiple patterns is fine;
recommending multiple changes is not — pick the highest-leverage one.
- Only reason from real history — if there's just one run, say "not enough history for a pattern
yet" rather than over-fitting a single data point.
- Never inflate the confidence prior manually; let the harness derive it from recorded outcomes.
- Corrections capture is proposal-only: a user correction becomes a written proposal for the
human to approve at retro — never a silent change to the chain's behavior, config, or plugin.
Unapproved proposals are kept as candidates, not applied "provisionally".
## Verify
- Plain: "I can name the one thing most worth improving for this repo's next refactor, and it's backed by a pattern across runs — not a hunch from a single run."
- Technical: `history.jsonl` parses; this run's retro is present; the proposed improvement cites the
recurring pattern (step/lane + count); the confidence-prior effect is stated and matches how
`diagnose.mjs historyPrior` reads confirmed `done` outcomes; every user correction found in this
run's state notes appears in the retro as a proposal (or an explicit "not actionable" line), each
marked approved / declined / deferred by the human — none silently applied.
### Contract
- **Inputs:** the finished run's retro (written by `orchestrate.mjs reset`), `history.jsonl`, corrections captured during the run.
- **Outputs:** one concrete improvement proposal per run, plus any automation recommendations (skill/ruleset/threshold), human-approved only; validated retro appended via `diagnose.mjs learn` (malformed retros rejected).
- **Failure modes:** no history → says so and skips (never invents lessons); rejected proposal → recorded and not re-proposed verbatim.
## Resources
- `references/method.md` — the history schema, pattern-detection rules, and the one-improvement ranking.
- `examples/before-after.md` — worked example: 5 runs of history → the single self-improvement suggestion.
- `scripts/checklist.mjs` — prints this skill's steps as JSON and summarizes `history.jsonl` (`--history`).
- `templates/output.md` — the Self-improvement Retro scaffold.
## Chain position
Runs in the **improve** phase, after `refactor-ship`, closing the loop. It consumes
`.refactor-chain/history.jsonl` (which the harness appends to on `reset`) and feeds the per-repo
confidence prior that `diagnose.mjs` reads on the *next* run — so each refactor makes the next one
a little more sure-footed. Nothing runs after it.
templates/output.md
# Self-improvement Retro — <project name>
> Refactor-chain · improve phase · run <date> · history: <N> runs analyzed
> Advisory. One improvement, on purpose. Nothing changed automatically.
## This run at a glance
- Lane: `<lane>` · outcome: `<done/aborted>` · confidence: `<x>` · clarified: `<yes/no>`
- Self-heals this run: <list, or "none">
## Patterns across history (surfaced — not all acted on)
- **Trouble steps:** `<step>` needed >1 attempt in `<k>/<n>` runs (<rate>). <or "none">
- **Diagnosis friction:** lane `<lane>` clarified in `<k>/<n>` runs. <or "none">
- **Clean streaks:** lane `<lane>` — `<n>` consecutive `done` runs (trust more). <or "none">
- **Recurring fail reasons:** <reason × count>, <or "none">
## The ONE improvement
> <The single highest-leverage change, in plain language.>
- **Why (evidence):** <the recurring pattern + its count>
- **Kind:** pre-step-safety | diagnosis-cue | plan-tweak
- **How to adopt:** <smallest concrete action>
- **Deliberately NOT also doing:** <lower-leverage candidates set aside — self-improvement is one step>
## Confidence prior (per-repo)
- `<lane>`: `<n>` confirmed `done` runs → next diagnosis nudged `+<amount>` (auto, via `historyPrior`).
- Effect next run: <"high-confidence, no clarify" / "still shaky — this is why the cue above helps">.
## Optional
- Plugin/config change? <if the improvement edits the chain itself, note a `/update` refresh — human approves>.