assets/methods.csv
num,direction,method_name,description,form,tier,aliases
1,wrong,Critique and Refine,Find the strengths to keep and the weaknesses to fix,"Strengths in two lines, then each weakness with its fix beside it",more,
2,wrong,Edge Case Sweep,"Try the extremes: empty, zero, maximum, malformed, the inputs happy-path thinking skips","A table of boundary inputs and what the draft does with each, failures marked",core,Boundary & Edge Case Sweep
3,wrong,Failure Mode Analysis,"For each part, how it fails and what that takes down","One line per component: failure, effect, how it is detected, how it is prevented",core,
4,wrong,Consistency Check,Restate the draft's claims and find where it contradicts itself or its own numbers,"Pairs of claims that conflict, with the one to keep and why",core,
5,missing,Stakeholder Lens,Each person the draft affects says what it owes them,"One stakeholder speaking in their own voice for a few lines, then what the draft owes them, located in the draft",core,Stakeholder Round Table; Stakeholder Lens Rotation
6,missing,Second-Order Thinking,Follow each consequence to the consequence it causes,"For each action: the first-order effect, what that effect causes next, what to do about it",core,
7,missing,Assumption Audit,"List every assumption, rate it, stress the weakest","A table of assumptions rated for confidence and impact, then the three weakest stress-tested",core,
8,missing,Expert Panel,"Several experts read it, each from their own field, without seeing each other","One expert's reading from their own field: what they see, what concerns them, and the change each concern calls for, located in the draft",core,Expert Panel Review
9,missing,Six Thinking Hats,"Facts, feelings, caution, optimism, creativity, process, one at a time","Six short sections, one per hat",more,
10,premise,First Principles Analysis,Strip the assumptions and rebuild from what is known to be true,"The assumptions stripped, the truths that remain, the approach rebuilt from them",core,
11,premise,5 Whys,"Ask why until the root cause, not the symptom, is in hand","The chain of whys, five deep, ending at the root cause and what changes because of it",more,5 Whys Deep Dive
12,future,Inversion Analysis,"Ask what would guarantee failure, then avoid it","A recipe for guaranteeing failure, then what the draft must do to avoid each step",more,
13,premise,Question the Question,Ask whether the stated problem is the real problem,"The stated problem, the better problem, and what the draft becomes if it solves that one",core,Reframe the Question
14,premise,Abstraction Laddering,"Move up for why and down for how, then pick the altitude","The draft restated one level up and one level down, and which altitude it should sit at",core,
15,simplify,Subtraction,"Improve by removing, against the habit of adding","What to remove, one line each with what is lost and gained, then the leaner draft",core,
16,simplify,Occam's Razor,Keep the simplest version that is still sufficient,"The competing versions, and the simplest one that still does the job",more,Occam's Razor Application
17,simplify,Adjust for Audience,Match depth and detail to who will read it,"Who reads it, what they need, and each passage that is too much or too little for them",core,Expand or Contract for Audience
18,simplify,Feynman Technique,"Explain it simply, every stall is a gap","The draft explained to a smart twelve-year-old, every place the explanation stalls marked as a gap",core,
19,simplify,Problem Decomposition,Split the whole into parts that can be solved alone,"The whole split into independent parts, each solvable on its own, the order to reassemble, and where the draft's split differs",core,
20,future,Pre-mortem Analysis,"Assume it failed, explain why","Written from after the failure: what went wrong, in order, and what in the draft made it likely",core,
21,future,Hindsight Reflection,Look back from after it worked,"Written from a year after success: what mattered, what did not, and what the draft spent effort on that never counted",core,
22,future,Cascading Failure Simulation,Break one thing on purpose and trace where the break travels,"One failure triggered, its path through the dependencies traced, the amplifiers and single points named, and what hardens each",core,Chaos Monkey Scenarios
23,future,What If Scenarios,Put the draft in situations it did not plan for,"Three scenarios the draft did not plan for, and how it fares in each",core,
24,other-side,Steelmanning,Build the opposing case at its strongest before answering it,"The opposing case argued as its best advocate would, then only what honestly survives",core,
25,other-side,Red Team,"Attack it as an adversary would, then harden it","An attacker's plan against the draft, step by step, then the hardening for each step",core,Red Team vs Blue Team
26,other-side,Devil's Advocate,"Press the case against, hardest where the draft is weakest","The case against, pressed hard, naming the point where the draft is weakest",more,Challenge from Critical Perspective
27,other-side,Socratic Questioning,Question until the draft cannot answer,"A chain of questions, each answered from the draft, until one it cannot answer",core,
28,wrong,Explain Reasoning,Show the steps from premise to conclusion and where one does not follow,"The reasoning laid out step by step, each step marked as given, inferred or asserted",core,
29,wrong,Code Review Gauntlet,Reviewers with different philosophies read the same code,"One reviewer's findings in their own voice, from their philosophy, each located in the code with its fix",more,
30,wrong,Performance Profiler Panel,"Database, frontend and operations specialists hunt the bottleneck","One specialist's suspects from their own layer, ranked, each with its evidence in the draft and the fix",more,
31,wrong,Source Triangulation,Demand three independent kinds of evidence before a claim stands,"Each claim with the evidence the draft gives for it by kind, a confidence rating, and the claims that stand on one kind alone",more,
32,premise,Comparative Analysis Matrix,Score the options against weighted criteria,"Options against criteria with scores, and the recommendation that falls out",more,
33,wrong,Map Is Not the Territory,Check where the model or diagram parts from the real system,"Each point where the representation and reality differ, and which to trust",more,
34,missing,Delphi Method,"Independent estimates with their reasoning, for the merge to compare","One estimator's figures with the reasoning behind each, and where the draft's own numbers differ",more,
35,missing,User Focus Group,The product's users react to it,"One user's reaction in their own words: what they would use, what puts them off, and what would change their mind",more,User Persona Focus Group
36,missing,Cross-Functional War Room,"Product, engineering and design settle the trade-offs together","One function's constraints, the trade-offs it can and cannot accept, and the changes it asks of the draft",more,
37,premise,Morphological Analysis,"List the parameters, the options for each, and the combinations nobody tried","A grid of parameters and options, then the untried combinations worth a look",more,
38,premise,Tree of Thoughts,"Follow several reasoning paths, then judge them","Each path to its conclusion, the judgment of each, and the one chosen",more,
39,premise,SCAMPER Method,"Substitute, combine, adapt, modify, put to other use, eliminate, reverse","One line per lens, with what it would change",more,
40,premise,Analogy Mapping,Borrow the structure of a problem already solved elsewhere,"The parallel domain, the mapping, and what it says about this draft",more,
41,premise,Reverse Engineering,Start from the end state and work back to the path,"The end state, the steps backward, the path forward, and where the draft's path differs",more,
42,premise,Constraint Injection,Add a hard limit and see what it forces,"The constraint added, what it forces, and what survives when it is lifted",more,
43,premise,Architecture Decision Record,"Options with their trade-offs, then the decision and its reason","Options, trade-offs, decision, rationale, and where the draft's choice or its stated reasons differ",more,Architecture Decision Records
44,future,Identify Potential Risks,List what could go wrong by category,"Risks by category, each with its mitigation",more,
47,other-side,Debate,"Each side argues alone at full strength, for the merge to weigh","One side's case argued at full strength against the draft's position, and the changes it would force",more,Debate Club Showdown
48,other-side,Investor Pitch,Pitch it to investors who ask about the moat and the numbers,"The pitch, the investors' hardest questions, and the answers the draft can and cannot give",more,Shark Tank Pitch
49,other-side,Thesis Defense,Defend it before a committee with different concerns,"Each committee member's challenge, the defense, what changes",more,Thesis Defense Simulation
50,other-side,Security Audit Personas,"Attacker, defender and auditor examine it from their threat models","Vulnerabilities, defenses, compliance gaps",more,
bmod.toml
[skill]
bmod = "bmod-core-tools"
source = "github:bmad-code-org/BMAD-METHOD/skills"
scripts = ["scripts/pick_methods.py"]
customize.toml
# DO NOT EDIT -- overwritten on every update.
#
# Workflow customization surface for bmad-advanced-elicitation.
#
# Override files (not edited here):
# {project-root}/_bmad/custom/bmad-advanced-elicitation.toml (team)
# {project-root}/_bmad/custom/bmad-advanced-elicitation.user.toml (personal)
[workflow]
# --- Configurable below. Overrides merge per BMad structural rules: ---
# scalars: override wins • plain arrays: append
# arrays of tables keyed by `code`: matching key replaces, new keys append
# The method catalog served by `uv run {skill-root}/scripts/pick_methods.py`. Columns:
# num, direction, method_name, description, form, tier, aliases.
# direction: wrong | missing | premise | simplify | future | other-side
# (find what is wrong, find what is missing, challenge the premise,
# simplify or re-pitch, stress against the future, argue the other side)
# form: one line saying what the method's output looks like; quoted into the brief
# tier: core (recommended) | more (kept so a method asked for by name resolves)
# aliases: earlier names of the method, separated by semicolons; optional
# Swap the path in team/user TOML to ship a different catalog. Use `{skill-root}`,
# `{project-root}` or an absolute path; a relative path resolves from the
# working directory.
methods_file = "{skill-root}/assets/methods.csv"
# Session output. The staged target, each push's return and the report land in
# `{output_dir}/{output_folder_name}/`; `{topic_slug}` is a kebab-case name for
# the target, filled at run time. The report is `{output_folder_name}.md`.
output_dir = "{output_folder}/{active_initiative}"
output_folder_name = "elicitation-{topic_slug}"
# Methods to run when the call names none, by method_name, shipped or from
# additional_methods, for example ["Pre-mortem Analysis", "Red Team"]. Empty: the
# skill chooses two or three against where the target looks weak. Team and user
# lists append, so both sets run.
default_methods = []
# Steps run before and after the shipped activation steps. Literal instructions;
# append-merges.
activation_steps_prepend = []
activation_steps_append = []
# Facts held for every session: methods or directions to favor or avoid, how
# pushback should land, house rules for applying changes. Literal sentences, or
# `file:` followed by a path or glob under {project-root} to read. Append-merges,
# so team and personal entries both apply.
#
# Examples (set in team/user override TOML):
# persistent_facts = [
# "Lead with a stress-against-the-future method for anything touching production.",
# "Never offer methods with several voices.",
# ]
persistent_facts = []
# Instructions run once the accepted version has been handed back.
on_complete = []
# Extra methods merged into the catalog without editing the shipped CSV. Add one
# when the team has a critique method of its own, or to retune a shipped one by
# reusing its method_name. Every entry needs a `code`, the merge key across
# override layers; `direction` takes one of the six keys above. The merge rules are in
# `uv run {skill-root}/scripts/pick_methods.py --help`.
#
# Example (set in team/user override TOML):
# [[workflow.additional_methods]]
# code = "regulatory-inversion"
# direction = "premise"
# method_name = "Regulatory Inversion"
# description = "Start from the compliance constraint and ask what it makes possible"
# form = "The constraint, the possibilities only it allows, and the design that uses them"
additional_methods = []
evals/cases.json
[
{
"id": "settled-plan",
"state_prefix": "[Earlier in this session you and the user wrote files/rollout-plan.md together. Along the way the user settled three things and you agreed: migrating tenant by tenant is final, because a single big-bang cutover was considered and rejected as too risky; Keyring is the vendor, chosen after comparing three providers, and is not up for debate; the budget is two engineers for six weeks and cannot grow.]",
"input": "ok now run advanced elicitation on the rollout plan. go with the methods you'd pick, don't wait for me to choose. show me the proposed revision and stop there, don't apply anything yet",
"rubric": [
"The final message presents a proposed revision as changed passages, each change tagged with the method or push that drove it.",
"The findings or revision identify that deleting the legacy password hashes on cutover day leaves no way to roll a tenant back, and propose a rollback path or delaying the deletion.",
"The findings or revision identify that 40 enterprise tenants, each needing its own SAML setup, in one week does not fit two engineers, and propose a change within the fixed budget (spreading the phase, reordering, or cutting scope) rather than adding engineers.",
"The findings or revision identify at least one of: users get no notice before their sessions are invalidated; the success criterion ('goes smoothly and customers are happy') cannot be measured.",
"No change in the proposed revision replaces tenant-by-tenant migration with a big-bang cutover, replaces Keyring with another vendor, or adds engineers or weeks; any objection to those appears only as a decision for the user, not as a change.",
"No Write or Edit in the transcript targets files/rollout-plan.md, and its content is unchanged."
],
"files": [
"files/rollout-plan.md",
"_bmad/config.toml",
"_bmad/scripts/resolve_customization.py",
"_bmad/scripts/resolve_config.py",
"_bmad/scripts/roster.py",
"_bmad/scripts/config_utils.py",
"_bmad/scripts/knowledge.py"
]
},
{
"id": "user-onepager",
"input": "I wrote this one-pager myself for a pitch next week: files/pantry-pal.md. advanced elicitation please, pick whatever methods you think fit and run them without asking me. just show the proposed revision, I'll decide what to take",
"rubric": [
"The final message presents a proposed revision as changed passages, each change tagged with the method or push that drove it.",
"The findings or revision identify that the stated user (busy parents who want dinner fast) conflicts with the feature set (chef-curated world recipes, cuisine explorer, videos, wine pairings).",
"The findings or revision identify at least two of: the 30% first-month conversion has no basis and is far above typical freemium rates; 'nothing else like Pantry Pal' is false because meal-planning apps already exist; twelve MVP features do not fit six weeks with one developer.",
"At least three findings name the specific section or line of the one-pager they refer to.",
"No Write or Edit in the transcript targets files/pantry-pal.md, and its content is unchanged."
],
"files": [
"files/pantry-pal.md",
"_bmad/config.toml",
"_bmad/scripts/resolve_customization.py",
"_bmad/scripts/resolve_config.py",
"_bmad/scripts/roster.py",
"_bmad/scripts/config_utils.py",
"_bmad/scripts/knowledge.py"
]
},
{
"id": "long-prd",
"input": "run advanced elicitation on files/prd-notifications.md, section 4 of our PRD. your call on methods, go straight to it. show the proposed revision only, nothing applied",
"rubric": [
"The final message presents a proposed revision as changed passages, each change tagged with the method or push that drove it.",
"The findings or revision identify that NFR-1 (delivery within 1 second at p99) contradicts the architecture note that batches sends every 5 minutes, and propose resolving it.",
"The findings or revision identify that FR-6 evaluates quiet hours in UTC instead of each user's own time zone.",
"The findings or revision identify at least one of: FR-4 ('must feel timely') cannot be tested; users have no way to turn off a channel or unsubscribe from email although all three are on by default; at-least-once delivery (NFR-3) conflicts with no duplicate emails.",
"The revision keeps the section's existing structure (4.1 to 4.8 headings and FR/NFR numbering) rather than rewriting it into a new layout.",
"No Write or Edit in the transcript targets files/prd-notifications.md, and its content is unchanged."
],
"files": [
"files/prd-notifications.md",
"_bmad/config.toml",
"_bmad/scripts/resolve_customization.py",
"_bmad/scripts/resolve_config.py",
"_bmad/scripts/roster.py",
"_bmad/scripts/config_utils.py",
"_bmad/scripts/knowledge.py"
]
},
{
"id": "named-premortem",
"input": "advanced elicitation, pre-mortem files/rollout-plan.md. show me what you'd change but don't apply it",
"rubric": [
"The transcript shows no message asking the user to choose methods or directions before the pre-mortem runs.",
"The pre-mortem output is written from after a failure: it describes the rollout as having already failed and explains why.",
"The findings or revision identify at least two of: deleting password hashes on cutover day removes any rollback; 40 enterprise SAML setups in one week does not fit two engineers; users get no notice before sessions are invalidated; Keyring outage handling relies only on the vendor SLA.",
"The final message presents a proposed revision as changed passages tagged with the pre-mortem.",
"No Write or Edit in the transcript targets files/rollout-plan.md, and its content is unchanged."
],
"files": [
"files/rollout-plan.md",
"_bmad/config.toml",
"_bmad/scripts/resolve_customization.py",
"_bmad/scripts/resolve_config.py",
"_bmad/scripts/roster.py",
"_bmad/scripts/config_utils.py",
"_bmad/scripts/knowledge.py"
]
}
]
evals/files/pantry-pal.md
# Pantry Pal — one-pager
## The problem
Busy parents with two jobs and kids at home do not have time to plan dinners. They end up buying takeout four nights a week, which costs money and isn't healthy.
## The product
Pantry Pal is a mobile app that plans your week of dinners. It shows you beautiful chef-curated recipes from around the world, lets you explore new cuisines, and saves the dishes you love. Every recipe has a step-by-step video and a wine pairing.
## Who it is for
Busy parents who want to get dinner on the table fast.
## MVP (6 weeks, one developer)
1. Recipe discovery feed
2. Cuisine explorer
3. Chef profiles
4. Step-by-step recipe videos
5. Wine pairings
6. Weekly meal plan
7. Shopping list
8. Pantry inventory with barcode scanning
9. Family member profiles with allergies
10. Social sharing
11. Grocery delivery integration
12. Nutrition dashboard
## Business model
Free tier with ads. Premium at $4.99/month removes ads and unlocks the meal planner. We expect 30% of free users to convert to premium in the first month.
## Competition
There is nothing else like Pantry Pal on the market today.
## Ask
$250k pre-seed to build the MVP and reach 10,000 users.
evals/files/prd-notifications.md
# PRD — Section 4: Notifications
## 4.1 Purpose
Teams using Tasklane miss changes to the tasks they care about. Today the only signal is the activity feed, which users check about twice a day according to our analytics. This section defines a notification system that tells users about the changes that matter to them, through the channel they prefer, without adding noise.
## 4.2 Users
- **Contributors** own tasks and need to know when someone comments on, reassigns or changes the due date of a task they own.
- **Leads** watch a project and need to know when a task in it becomes blocked or overdue.
- **Admins** manage the workspace and need to know about billing and security events.
## 4.3 Channels
Notifications go out in three channels:
- In-app: the bell icon in the top bar, with an unread count.
- Email: one message per notification, sent from `notifications@tasklane.io`.
- Mobile push: through our iOS and Android apps.
Every user gets all three channels by default.
## 4.4 Functional requirements
**FR-1 Task events.** The system sends a notification to a task's owner when another user comments on the task, reassigns it, or changes its due date.
**FR-2 Project events.** The system sends a notification to every lead watching a project when a task in that project is marked blocked or passes its due date.
**FR-3 Admin events.** The system sends a notification to every admin when a payment fails or a new admin is added.
**FR-4 Timeliness.** Notifications must feel timely to the user.
**FR-5 Grouping.** When several events about the same task happen within a short time, the system groups them into one notification that lists each event.
**FR-6 Quiet hours.** A user can set quiet hours. During quiet hours no push notifications are sent; they are delivered when quiet hours end. Quiet hours are evaluated in the server's time zone (UTC).
**FR-7 Read state.** Reading a notification in any channel marks it read in every channel.
## 4.5 Non-functional requirements
**NFR-1 Latency.** A notification is delivered to every channel within 1 second of the triggering event at p99.
**NFR-2 Volume.** The system handles 2 million notifications per day at launch and 10 million within a year.
**NFR-3 Reliability.** No notification is lost. Delivery is at least once; duplicates are acceptable for in-app but not for email.
## 4.6 Architecture notes
The notification service consumes task events from the existing event bus. To keep email and push costs down, the service batches pending notifications and sends each batch every 5 minutes. In-app notifications are written to the notifications table at the same time as the batch send.
## 4.7 Acceptance criteria
- A contributor who owns a task receives a notification in all three channels when another user comments on it.
- A lead receives a notification when a watched task becomes blocked.
- An admin receives a notification when a payment fails.
- Grouped notifications list every grouped event in order.
- Reading a notification on mobile marks it read on the web within 10 seconds.
## 4.8 Out of scope
- Slack and Microsoft Teams integrations.
- Digest emails (daily or weekly summaries).
- Per-project notification settings.
evals/files/rollout-plan.md
# Keyring SSO Rollout Plan
## Goal
Move every customer tenant from our in-house login to Keyring SSO by the end of Q1, so we can retire the legacy auth service and its two open security findings.
## Approach
We migrate tenant by tenant. For each tenant we import its users into Keyring, switch the tenant's login route to Keyring, and turn off the legacy password form for that tenant. Users keep their email addresses as identifiers, so no one needs a new account.
## Team
Two backend engineers, full time, for six weeks. Support is briefed before each phase.
## Phases
| Phase | Week | Tenants | Notes |
|---|---|---|---|
| 1 | 1–2 | 5 internal and friendly tenants | Shake out the import script |
| 2 | 3–4 | 60 self-serve tenants | Automated import, no manual review |
| 3 | 5 | 40 enterprise tenants | Each needs its own SAML setup with the customer's IT team |
| 4 | 6 | Cleanup | Remove the legacy login code |
## Cutover
At cutover for a tenant, all existing sessions are invalidated so that every user logs in once through Keyring. The legacy password hashes are deleted the same day.
## Risks
- Import script bugs. Mitigated by phase 1.
- Keyring outage. Keyring's SLA is 99.95%.
## Success
The migration is a success if it goes smoothly and customers are happy.
evals/triggers.json
[
{
"query": "run advanced elicitation on that section",
"should_trigger": true
},
{
"query": "advanced elicitation on the plan you just wrote",
"should_trigger": true
},
{
"query": "use bmad-advanced-elicitation on this draft",
"should_trigger": true
},
{
"query": "bmad-advanced-elicitation: steelman the other option before we commit",
"should_trigger": true
},
{
"query": "let's do some advanced elicitation on the pricing section before we move on",
"should_trigger": true
},
{
"query": "can you run an advanced elicitation round with a pre-mortem on this rollout plan",
"should_trigger": true
},
{
"query": "Advanced Elicitation please, this PRD section feels thin",
"should_trigger": true
},
{
"query": "I want to try the advanced elicitation skill on my cover letter",
"should_trigger": true
},
{
"query": "push this harder",
"should_trigger": false
},
{
"query": "challenge this draft, I think it's too safe",
"should_trigger": false
},
{
"query": "poke holes in the plan you just wrote",
"should_trigger": false
},
{
"query": "do a pre-mortem on this rollout plan",
"should_trigger": false
},
{
"query": "steelman the other option before we commit",
"should_trigger": false
},
{
"query": "ok the outline is decent but make it stronger, first principles it",
"should_trigger": false
},
{
"query": "red team the pricing section",
"should_trigger": false
},
{
"query": "can you socratic this decision memo I pasted",
"should_trigger": false
},
{
"query": "tear this apart",
"should_trigger": false
},
{
"query": "critique this draft before I send it",
"should_trigger": false
},
{
"query": "devil's advocate this plan",
"should_trigger": false
},
{
"query": "run six thinking hats on this spec",
"should_trigger": false
},
{
"query": "assumption audit on the proposal please",
"should_trigger": false
},
{
"query": "pre-mortem my startup idea",
"should_trigger": false
},
{
"query": "get a few experts to weigh in on this design",
"should_trigger": false
},
{
"query": "use SCAMPER to generate feature ideas",
"should_trigger": false
},
{
"query": "elicit the requirements for the login feature from me",
"should_trigger": false
},
{
"query": "review this PR",
"should_trigger": false
},
{
"query": "review the architecture doc and report what you find",
"should_trigger": false
},
{
"query": "I have a half-formed idea for a side business, pressure-test it with me",
"should_trigger": false
},
{
"query": "help me brainstorm names for the product",
"should_trigger": false
},
{
"query": "party mode, get the architect and the PM in here",
"should_trigger": false
},
{
"query": "rewrite this paragraph to be shorter",
"should_trigger": false
},
{
"query": "what does first principles thinking mean?",
"should_trigger": false
},
{
"query": "push this branch to origin",
"should_trigger": false
}
]
scripts/pick_methods.py
#!/usr/bin/env python3
# /// script
# requires-python = ">=3.11"
# ///
"""Serve the elicitation method catalog without loading it all into context.
The catalog is a CSV (num, direction, method_name, description, form, tier, aliases).
`direction` is what the push does to the draft, one of six keys (see DIRECTIONS);
`description` is a one-line gist; `form` says what the method's output looks like
and is quoted verbatim into a push's brief; `tier` is `core` (recommended) or
`more` (kept so a method asked for by name resolves); `aliases` lists earlier names
of the method, separated by semicolons, so an old name still finds it.
Commands:
directions the six directions with core/more counts, the cheap map
list [--direction D ...] [--tier T] [--all]
num, direction, name and gist; --all for the whole catalog
show NAME_OR_NUM [...] full rows, form included, for a brief
random [-n N] [--direction D ...] [--tier T] [--exclude NAME ...] [--spread]
a random draw; --spread takes at most one per direction
`list` refuses to run with neither --direction, --tier nor --all so dumping the
whole catalog into context is always deliberate.
`--customization SPEC` takes the merged customization as resolve_customization.py
prints it (`-` for stdin, or a path): `methods_file` becomes the catalog, with
`{skill-root}` and `{project-root}` (from --project-root) bound by this script,
and `additional_methods` is merged in; each of those needs a `code`.
`--extra SPEC` merges a bare JSON array of methods (a literal starting with `[`,
or a path). Each method carries method_name, description and form, plus direction
and tier when known. A missing direction files under `custom`; an unknown
direction or tier is an error. Older rows are accepted: `output_pattern` stands
in for a missing form, and unknown keys, `category` among them, are ignored. An extra whose method_name matches a
catalog row or one of its aliases (case-insensitive) REPLACES it and keeps that
row's num; others append and get the next free nums.
Default output is lean tab-separated text for an LLM to read; --json for structured.
"""
import argparse
import csv
import json
import random
import sys
from pathlib import Path
DEFAULT_FILE = Path(__file__).resolve().parent.parent / "assets" / "methods.csv"
FIELDS = ("num", "direction", "method_name", "description", "form", "tier", "aliases")
REQUIRED_FIELDS = ("method_name", "description", "form")
# Ordered: the order the directions are presented in. `custom` holds rows whose
# direction is not one of the six.
DIRECTIONS = {
"wrong": "Find what is wrong",
"missing": "Find what is missing",
"premise": "Challenge the premise",
"simplify": "Simplify or re-pitch",
"future": "Stress against the future",
"other-side": "Argue the other side",
"custom": "Custom",
}
TIERS = ("core", "more")
def normalize(row: dict) -> dict:
"""Coerce any row, shipped or extra, old columns or new, into FIELDS. An
unknown direction or tier is an error; a missing tier is core."""
get = lambda k: str(row.get(k) or "").strip() # noqa: E731
out = {k: get(k) for k in FIELDS}
name = out["method_name"] or get("code") or "unnamed"
given = out["direction"]
out["direction"] = given.lower()
if given and out["direction"] not in DIRECTIONS:
raise ValueError(f"{name}: direction {given!r} is not one of {'/'.join(DIRECTIONS)}")
if not given:
# An older row carries `category`, which is not a direction: it files under custom.
out["direction"] = "custom"
out["form"] = out["form"] or get("output_pattern")
tier = out["tier"].lower()
if tier and tier not in TIERS:
raise ValueError(f"{name}: tier {tier!r} is not one of {'/'.join(TIERS)}")
out["tier"] = tier or "core"
out["code"] = get("code") # kept for traceability
return out
def load(file: Path) -> list[dict]:
# utf-8-sig: tolerate BOM-prefixed catalogs (Excel "CSV UTF-8", Notepad)
with open(file, newline="", encoding="utf-8-sig") as f:
rows = [normalize(r) for r in csv.DictReader(f)]
for n, row in enumerate(rows, 2):
for field in REQUIRED_FIELDS:
if not row[field]:
raise ValueError(f"{file} line {n} ({row['method_name'] or 'unnamed'}) is missing {field}")
return rows
def read_spec(spec: str):
"""A JSON literal, `-` for stdin, or a path to a JSON file."""
if spec == "-":
text = sys.stdin.read()
if not text.strip():
raise ValueError("nothing arrived on stdin; did the resolver fail?")
return json.loads(text)
if spec.lstrip().startswith(("[", "{")):
return json.loads(spec)
return json.loads(Path(spec).read_text(encoding="utf-8-sig"))
def load_customization(spec: str, skill_root: Path, project_root: Path | None = None) -> tuple[Path | None, list[dict]]:
"""Read resolve_customization.py output: (methods_file, additional_methods).
The `workflow` wrapper is optional; `{skill-root}` and `{project-root}` in
methods_file are bound here. Every additional method needs a `code`, the key
the override layers merge on; without one the whole array degrades to append."""
data = read_spec(spec)
if not isinstance(data, dict):
raise ValueError("--customization must be a JSON object")
table = data.get("workflow", data)
if not isinstance(table, dict):
raise ValueError("--customization: `workflow` must be a JSON object")
methods_file = str(table.get("methods_file") or "").strip()
if methods_file and "{project-root}" in methods_file:
if project_root is None:
raise ValueError("methods_file uses {project-root}; pass --project-root")
methods_file = methods_file.replace("{project-root}", str(project_root))
path = Path(methods_file.replace("{skill-root}", str(skill_root))) if methods_file else None
extras = table.get("additional_methods") or []
rows = load_extra(json.dumps(extras))
for r in rows:
if not r["code"]:
raise ValueError(f"additional method {r['method_name']} has no code; every entry needs one")
return path, rows
def load_extra(spec: str) -> list[dict]:
"""Parse the --extra overlay: a JSON array literal, `-` for stdin, or a path."""
data = read_spec(spec)
if not isinstance(data, list):
raise ValueError("--extra must be a JSON array of objects")
rows = []
for n, item in enumerate(data, 1):
if not isinstance(item, dict):
raise ValueError(f"each --extra entry must be a JSON object, got: {item!r}")
row = normalize(item)
for field in REQUIRED_FIELDS:
if not row[field]:
name = row["method_name"] or row["code"] or "unnamed"
raise ValueError(f"--extra entry {n} ({name}) is missing {field}")
rows.append(row)
return rows
def merge_extra(rows: list[dict], extras: list[dict]) -> list[dict]:
"""Extras replace a catalog row with the same method_name or alias (case-insensitive),
otherwise append. A replacement inherits the shipped row's num and keeps its
names as aliases, so a request by the current or an old name still finds it;
appended extras get the next free nums, so every method stays addressable by number."""
merged = list(rows)
index = {r["method_name"].lower(): i for i, r in enumerate(merged)}
for i, r in enumerate(rows):
for a in names_of(r)[1:]:
index.setdefault(a, i)
for e in extras:
key = e["method_name"].lower()
if key in index:
old = merged[index[key]]
e = dict(e)
e["num"] = e["num"] or old["num"]
kept = dict.fromkeys(n for n in names_of(old) + names_of(e)[1:] if n != key)
e["aliases"] = "; ".join(kept)
for n in kept:
index[n] = index[key]
merged[index[key]] = e
else:
index[key] = len(merged)
merged.append(dict(e))
next_num = max((int(r["num"]) for r in merged if r["num"].isdigit()), default=0) + 1
seen = {}
for r in merged:
if not r["num"]:
r["num"] = str(next_num)
next_num += 1
if r["num"] in seen:
raise ValueError(f"num {r['num']} is used by both {seen[r['num']]} and {r['method_name']}")
seen[r["num"]] = r["method_name"]
return merged
def directions(rows: list[dict]) -> list[tuple[str, str, int, int]]:
"""(key, label, core count, more count) in presentation order; empty ones left out."""
counts: dict[str, list[int]] = {}
for r in rows:
c = counts.setdefault(r["direction"], [0, 0])
c[0 if r["tier"] == "core" else 1] += 1
return [(k, label, *counts[k]) for k, label in DIRECTIONS.items() if k in counts]
def filter_rows(rows: list[dict], dirs: list[str] | None = None, tier: str | None = None) -> list[dict]:
out = rows
if dirs:
wanted = {d.lower() for d in dirs}
out = [r for r in out if r["direction"] in wanted]
if tier:
out = [r for r in out if r["tier"] == tier.lower()]
return out
def names_of(row: dict) -> list[str]:
"""The method_name, then each alias, lowercased."""
return [row["method_name"].lower()] + [a.strip().lower() for a in row.get("aliases", "").split(";") if a.strip()]
def find(rows: list[dict], names: list[str]) -> tuple[list[dict], list[str]]:
"""Match each query by method_name, alias or num, case-insensitively; a current name wins over an alias."""
by_key: dict[str, dict] = {}
for r in rows:
by_key[r["method_name"].lower()] = r
for r in rows:
for a in names_of(r)[1:]:
by_key.setdefault(a, r)
if r["num"]:
by_key.setdefault(r["num"], r)
found, missing = [], []
for n in names:
r = by_key.get(n.strip().lower())
(found if r else missing).append(r if r else n)
return found, missing
def exclude(rows: list[dict], names: list[str] | None) -> list[dict]:
if not names:
return rows
skip = {n.strip().lower() for n in names}
return [r for r in rows if r["method_name"].lower() not in skip]
def spread_sample(rows: list[dict], n: int, rng: random.Random | None = None) -> list[dict]:
"""Draw n methods with maximum direction diversity: shuffle the directions,
take one random method per direction round-robin, wrapping only when there
are fewer directions than picks."""
rng = rng or random
by_dir: dict[str, list[dict]] = {}
for r in rows:
by_dir.setdefault(r["direction"], []).append(r)
buckets = list(by_dir.values())
rng.shuffle(buckets)
for b in buckets:
rng.shuffle(b)
out: list[dict] = []
while buckets and len(out) < n:
exhausted = []
for b in buckets:
if len(out) >= n:
break
out.append(b.pop())
if not b:
exhausted.append(b)
buckets = [b for b in buckets if b not in exhausted]
return out
def fmt_directions(dirs: list[tuple[str, str, int, int]], as_json: bool) -> str:
if as_json:
return json.dumps([{"direction": k, "label": label, "core": c, "more": m} for k, label, c, m in dirs])
return "\n".join(f"{k}\t{label}\t{c} core\t{m} more" for k, label, c, m in dirs)
def fmt_rows(rows: list[dict], as_json: bool, full: bool = False) -> str:
if as_json:
return json.dumps([{k: r[k] for k in FIELDS} for r in rows])
if full:
return "\n".join(
f"{r['num']}\t{r['direction']}\t{r['method_name']}\t{r['description']}\tform: {r['form']}\t{r['tier']}"
for r in rows
)
return "\n".join(f"{r['num']}\t{r['direction']}\t{r['method_name']}\t{r['description']}\t{r['tier']}" for r in rows)
def main(argv: list[str] | None = None) -> int:
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8") # don't die on locale code pages
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
p.add_argument(
"--file", type=Path, help="method CSV (default: the customization's methods_file, else assets/methods.csv)"
)
p.add_argument("--customization", help="resolve_customization.py output: `-` for stdin, or a path")
p.add_argument("--project-root", type=Path, help="binds {project-root} in the customization's methods_file")
p.add_argument("--extra", help="additional methods: a JSON array literal, `-` for stdin, or a path")
p.add_argument("--json", action="store_true", help="emit structured JSON instead of lean text")
sub = p.add_subparsers(dest="cmd", required=True)
sub.add_parser("directions", help="the directions with core/more counts")
pl = sub.add_parser("list", help="the index for chosen directions (needs --direction, --tier or --all)")
pl.add_argument(
"--direction",
action="append",
type=str.lower,
choices=list(DIRECTIONS),
help="filter to a direction (repeatable)",
)
pl.add_argument("--tier", choices=TIERS, help="filter to core or more")
pl.add_argument("--all", action="store_true", help="dump the entire catalog (deliberate; large)")
ps = sub.add_parser("show", help="full row, form included, for each named method")
ps.add_argument("names", nargs="+", help="method names or nums")
pr = sub.add_parser("random", help="draw methods at random")
pr.add_argument("-n", type=int, default=1, help="how many (default 1)")
pr.add_argument(
"--direction",
action="append",
type=str.lower,
choices=list(DIRECTIONS),
help="restrict to a direction (repeatable)",
)
pr.add_argument("--tier", choices=TIERS, help="restrict to core or more")
pr.add_argument("--exclude", action="append", help="method name to skip (repeatable), e.g. already offered")
pr.add_argument("--spread", action="store_true", help="at most one per direction")
args = p.parse_args(argv)
extras: list[dict] = []
file = args.file
try:
if args.customization:
custom_file, extras = load_customization(
args.customization, Path(__file__).resolve().parent.parent, args.project_root
)
file = file or custom_file
if args.extra:
extras += load_extra(args.extra)
except (OSError, ValueError) as e:
print(f"error: could not read customization: {e}", file=sys.stderr)
return 2
file = file or DEFAULT_FILE
if not file.is_file():
print(f"error: method file not found: {file}", file=sys.stderr)
return 2
try:
rows = merge_extra(load(file), extras)
except ValueError as e:
print(f"error: bad catalog: {e}", file=sys.stderr)
return 2
if args.cmd == "directions":
print(fmt_directions(directions(rows), args.json))
elif args.cmd == "list":
if not args.direction and not args.tier and not args.all:
print(
"error: `list` needs --direction or --tier, or --all to dump the whole catalog on purpose. "
"Use `directions` for the cheap map, or `random` to draw blind.",
file=sys.stderr,
)
return 2
print(fmt_rows(filter_rows(rows, args.direction, args.tier), args.json))
elif args.cmd == "show":
found, missing = find(rows, args.names)
for m in missing:
print(f"# not found: {m}", file=sys.stderr)
if not found:
return 1
print(fmt_rows(found, args.json, full=True))
elif args.cmd == "random":
pool = exclude(filter_rows(rows, args.direction, args.tier), args.exclude)
if not pool:
print("# no methods match", file=sys.stderr)
return 1
n = max(0, min(args.n, len(pool))) # clamp: never crash on a negative or oversized -n
picks = spread_sample(pool, n) if args.spread else random.sample(pool, n)
print(fmt_rows(picks, args.json, full=True))
return 0
if __name__ == "__main__":
if sys.platform == "win32":
# Piped output on Windows defaults to a legacy code page, not UTF-8.
sys.stdout.reconfigure(encoding="utf-8")
sys.stderr.reconfigure(encoding="utf-8")
sys.exit(main())
scripts/tests/test_pick_methods.py
# /// script
# requires-python = ">=3.11"
# dependencies = ["pytest>=8.0"]
# ///
"""Tests for pick_methods.py.
Run from the repo root: uv run --frozen python -B -m pytest skills/bmad-advanced-elicitation/scripts/tests/
"""
import json
import random
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
import pick_methods # noqa: E402
CSV = """num,direction,method_name,description,form,tier
1,future,Pre-mortem Analysis,Assume it failed,Written from after the failure,core
2,missing,Assumption Audit,List and stress-test every assumption,A rated table then the weakest stress-tested,core
3,premise,First Principles Analysis,Rebuild from fundamental truths,Assumptions stripped then rebuilt,core
4,other-side,Socratic Questioning,Question until it cannot answer,A chain of questions,core
5,premise,SCAMPER Method,Seven creativity lenses,One line per lens,more
"""
EXTRA = [
{
"code": "reg-inversion",
"category": "domain",
"method_name": "Regulatory Inversion",
"description": "Start from the compliance constraint",
"output_pattern": "constraint → possibility",
},
{
"code": "premortem-lite",
"direction": "future",
"method_name": "Pre-mortem Analysis",
"description": "RETUNED pre-mortem",
"form": "failure → prevention",
},
]
@pytest.fixture
def lib(tmp_path):
csv_path = tmp_path / "methods.csv"
csv_path.write_text(CSV, encoding="utf-8")
return csv_path
def rows(lib):
return pick_methods.load(lib)
# --- load / normalize / merge ------------------------------------------------
def test_load_all_fields_present(lib):
r = rows(lib)
assert len(r) == 5
assert r[0]["method_name"] == "Pre-mortem Analysis"
assert all(set(pick_methods.FIELDS) <= set(row) for row in r)
assert r[4]["tier"] == "more"
def test_old_columns_are_accepted():
row = pick_methods.normalize(
{"category": "risk", "method_name": "X", "description": "d", "output_pattern": "a → b", "bogus": "ignored"}
)
assert row["direction"] == "custom" # an old category is not a direction
assert row["form"] == "a → b" # output_pattern stands in for form
assert row["tier"] == "core" # tier defaults to core
assert "bogus" not in row
def test_unknown_direction_or_tier_is_an_error():
with pytest.raises(ValueError, match="direction 'premsie'"):
pick_methods.normalize({"direction": "premsie", "method_name": "X", "description": "d", "form": "f"})
with pytest.raises(ValueError, match="tier 'x'"):
pick_methods.normalize({"direction": "wrong", "method_name": "X", "description": "d", "form": "f", "tier": "x"})
def test_shipped_style_catalog_missing_a_field_is_rejected(tmp_path):
bad = tmp_path / "bad.csv"
bad.write_text("num,direction,method_name,description\n1,wrong,X,d\n", encoding="utf-8")
with pytest.raises(ValueError, match="line 2 \\(X\\) is missing form"):
pick_methods.load(bad)
assert pick_methods.main(["--file", str(bad), "directions"]) == 2
def test_cli_empty_stdin_names_the_resolver(monkeypatch, capsys):
import io
monkeypatch.setattr(sys, "stdin", io.StringIO(""))
assert pick_methods.main(["--customization", "-", "directions"]) == 2
assert "resolver" in capsys.readouterr().err
def test_customization_object_binds_methods_file_and_extras(tmp_path, lib):
spec = {"workflow": {"methods_file": str(lib), "additional_methods": EXTRA}}
path, extras = pick_methods.load_customization(json.dumps(spec), tmp_path)
assert path == lib and len(extras) == 2
# {skill-root} is bound to the given root; an empty additional_methods is a no-op
path, extras = pick_methods.load_customization(
'{"methods_file": "{skill-root}/assets/m.csv", "additional_methods": []}', tmp_path
)
assert path == tmp_path / "assets" / "m.csv" and extras == []
path, _ = pick_methods.load_customization(
'{"methods_file": "{project-root}/_bmad/custom/m.csv"}', tmp_path, tmp_path / "p"
)
assert path == tmp_path / "p" / "_bmad" / "custom" / "m.csv"
with pytest.raises(ValueError, match="project-root"):
pick_methods.load_customization('{"methods_file": "{project-root}/m.csv"}', tmp_path)
with pytest.raises(ValueError, match="code"):
pick_methods.load_customization(
'{"additional_methods": [{"method_name": "X", "description": "d", "form": "f"}]}', tmp_path
)
with pytest.raises(ValueError):
pick_methods.load_customization("[1]", tmp_path)
def test_load_extra_json_literal_and_file(tmp_path, lib):
literal = pick_methods.load_extra(json.dumps(EXTRA))
f = tmp_path / "extra.json"
f.write_text(json.dumps(EXTRA), encoding="utf-8")
from_file = pick_methods.load_extra(str(f))
assert literal == from_file
assert literal[0]["method_name"] == "Regulatory Inversion"
assert literal[0]["num"] == "" # missing fields normalize to empty
assert literal[0]["code"] == "reg-inversion" # code survives loading
def test_merge_extra_replaces_by_name_and_appends(lib):
merged = pick_methods.merge_extra(rows(lib), pick_methods.load_extra(json.dumps(EXTRA)))
assert len(merged) == 6 # 5 shipped, 1 replaced in place, 1 appended
premortem = next(r for r in merged if r["method_name"] == "Pre-mortem Analysis")
assert premortem["description"] == "RETUNED pre-mortem"
assert premortem["num"] == "1" # replacement inherits the shipped num
appended = next(r for r in merged if r["method_name"] == "Regulatory Inversion")
assert appended["num"] == "6" # appended extras get the next free num
assert appended["direction"] == "custom"
assert [d[0] for d in pick_methods.directions(merged)][-1] == "custom" # custom is listed last
def test_extras_are_addressable_by_num(lib):
merged = pick_methods.merge_extra(rows(lib), pick_methods.load_extra(json.dumps(EXTRA)))
found, missing = pick_methods.find(merged, ["6", "1"])
assert [r["method_name"] for r in found] == ["Regulatory Inversion", "Pre-mortem Analysis"]
assert missing == []
# --- directions / filter / find / exclude ------------------------------------
def test_directions_in_presentation_order_with_counts(lib):
got = pick_methods.directions(rows(lib))
assert [d[0] for d in got] == ["missing", "premise", "future", "other-side"]
assert got[1] == ("premise", "Challenge the premise", 1, 1)
def test_filter_direction_is_case_insensitive(lib):
got = pick_methods.filter_rows(rows(lib), ["PREMISE"])
assert {r["method_name"] for r in got} == {"First Principles Analysis", "SCAMPER Method"}
def test_filter_tier(lib):
assert {r["method_name"] for r in pick_methods.filter_rows(rows(lib), tier="more")} == {"SCAMPER Method"}
assert len(pick_methods.filter_rows(rows(lib), ["premise"], "core")) == 1
def test_filter_none_returns_all(lib):
assert len(pick_methods.filter_rows(rows(lib))) == 5
def test_find_by_name_num_and_missing(lib):
found, missing = pick_methods.find(rows(lib), ["scamper method", "3", "Nope"])
assert [r["method_name"] for r in found] == ["SCAMPER Method", "First Principles Analysis"]
assert missing == ["Nope"]
def test_exclude_skips_named(lib):
got = pick_methods.exclude(rows(lib), ["pre-mortem analysis", "SCAMPER Method"])
assert {r["method_name"] for r in got} == {
"Assumption Audit",
"First Principles Analysis",
"Socratic Questioning",
}
# --- spread sampling ---------------------------------------------------------
def test_spread_hits_distinct_directions(lib):
for seed in range(20):
picks = pick_methods.spread_sample(rows(lib), 3, random.Random(seed))
assert len({r["direction"] for r in picks}) == 3
def test_spread_wraps_when_directions_run_out(lib):
picks = pick_methods.spread_sample(rows(lib), 5, random.Random(0))
assert len(picks) == 5
assert len({r["method_name"] for r in picks}) == 5 # no duplicates
def test_spread_clamps_to_pool(lib):
assert len(pick_methods.spread_sample(rows(lib), 99, random.Random(0))) == 5
# --- CLI ---------------------------------------------------------------------
def run(args, lib, capsys):
code = pick_methods.main(["--file", str(lib), *args])
captured = capsys.readouterr()
return code, captured.out, captured.err
def test_cli_directions(lib, capsys):
code, out, _ = run(["directions"], lib, capsys)
assert code == 0
assert "premise\tChallenge the premise\t1 core\t1 more" in out
def test_cli_list_requires_scope(lib, capsys):
code, _, err = run(["list"], lib, capsys)
assert code == 2
assert "--direction" in err
def test_cli_list_direction_tier_and_all(lib, capsys):
code, out, _ = run(["list", "--direction", "premise"], lib, capsys)
assert code == 0 and len(out.strip().splitlines()) == 2
assert "SCAMPER" in out and "Socratic" not in out
assert "form:" not in out # the index leaves the form line out
code, out, _ = run(["list", "--tier", "core"], lib, capsys)
assert code == 0 and len(out.strip().splitlines()) == 4
code, out, _ = run(["list", "--all"], lib, capsys)
assert code == 0 and len(out.strip().splitlines()) == 5
def test_cli_show_found_and_missing(lib, capsys):
code, out, err = run(["show", "Assumption Audit", "Ghost"], lib, capsys)
assert code == 0
assert "form: A rated table" in out
assert "not found: Ghost" in err
code, _, _ = run(["show", "Ghost"], lib, capsys)
assert code == 1
def test_cli_random_spread_exclude(lib, capsys):
code, out, _ = run(["random", "-n", "3", "--spread", "--exclude", "SCAMPER Method"], lib, capsys)
assert code == 0
lines = [ln for ln in out.strip().splitlines() if ln]
assert len(lines) == 3
assert "SCAMPER" not in out and "form:" in out
def test_cli_random_clamps_and_empty_pool(lib, capsys):
code, out, _ = run(["random", "-n", "99"], lib, capsys)
assert code == 0 and len(out.strip().splitlines()) == 5
code, _, err = run(["random", "--direction", "custom"], lib, capsys)
assert code == 1 and "no methods match" in err
def test_cli_unknown_direction_is_rejected(lib, capsys):
for cmd in ("list", "random"):
with pytest.raises(SystemExit) as e:
run([cmd, "--direction", "other side"], lib, capsys)
assert e.value.code == 2 and "invalid choice" in capsys.readouterr().err
def test_cli_random_tier(lib, capsys):
code, out, _ = run(["random", "-n", "9", "--tier", "more"], lib, capsys)
assert code == 0 and out.strip().splitlines() == [ln for ln in out.strip().splitlines() if ln.endswith("more")]
def test_cli_customization_from_file_and_stdin(tmp_path, lib, capsys, monkeypatch):
spec = {"workflow": {"methods_file": str(lib), "additional_methods": EXTRA}}
f = tmp_path / "customization.json"
f.write_text(json.dumps(spec), encoding="utf-8")
code = pick_methods.main(["--customization", str(f), "list", "--direction", "custom"])
out = capsys.readouterr().out
assert code == 0 and "Regulatory Inversion" in out
import io
monkeypatch.setattr(sys, "stdin", io.StringIO(json.dumps(spec)))
code = pick_methods.main(["--customization", "-", "directions"])
out = capsys.readouterr().out
assert code == 0 and "custom\tCustom\t1 core" in out
code = pick_methods.main(["--customization", "[]", "directions"])
assert code == 2 and "customization" in capsys.readouterr().err
def test_cli_extra_inline_json(lib, capsys):
code, out, _ = run(["--extra", json.dumps(EXTRA), "list", "--direction", "custom"], lib, capsys)
assert code == 0 and "Regulatory Inversion" in out
def test_cli_bad_extra_and_missing_file(tmp_path, lib, capsys):
code, _, err = run(["--extra", str(tmp_path / "gone.json"), "directions"], lib, capsys)
assert code == 2 and "customization" in err
code = pick_methods.main(["--file", str(tmp_path / "gone.csv"), "directions"])
assert code == 2
def test_cli_extra_row_missing_a_required_field_exits_2(lib, capsys):
partial = [{"direction": "future", "method_name": "Pre-mortem Analysis", "description": "No form."}]
code, out, err = run(["--extra", json.dumps(partial), "list", "--all"], lib, capsys)
assert code == 2 and out == ""
assert "Pre-mortem Analysis" in err and "form" in err
assert "customization" in err
def test_cli_extra_duplicate_num_exits_2_naming_the_num(lib, capsys):
clash = [{**EXTRA[0], "num": "3"}]
code, out, err = run(["--extra", json.dumps(clash), "list", "--all"], lib, capsys)
assert code == 2 and out == ""
assert "num 3" in err
def test_cli_json_output(lib, capsys):
code, out, _ = run(["--json", "show", "1"], lib, capsys)
assert code == 0
data = json.loads(out)
assert data[0]["method_name"] == "Pre-mortem Analysis" and data[0]["form"]
# --- shipped catalog integration ----------------------------------------------
def test_shipped_catalog_loads_clean():
shipped = pick_methods.DEFAULT_FILE
assert shipped.is_file(), f"shipped catalog missing: {shipped}"
r = pick_methods.load(shipped)
assert len(r) >= 40
for row in r:
assert row["direction"] != "custom", row # every shipped row sits in one of the six
assert row["method_name"] and row["description"] and row["form"], row
core = [row for row in r if row["tier"] == "core"]
assert 20 <= len(core) <= 35
assert len({row["direction"] for row in core}) == 6 # every direction has a core method
nums = [row["num"] for row in r]
assert len(nums) == len(set(nums))
def test_old_names_resolve_through_aliases(tmp_path):
cat = tmp_path / "m.csv"
cat.write_text(
"num,direction,method_name,description,form,tier,aliases\n"
"1,future,Cascading Failure Simulation,Break one thing,A trace,core,Chaos Monkey Scenarios\n"
"2,missing,Expert Panel,Experts read it,One view,core,Expert Panel Review; Panel\n",
encoding="utf-8",
)
rows = pick_methods.load(cat)
found, missing = pick_methods.find(rows, ["chaos monkey scenarios", "Panel", "Nope"])
assert [r["method_name"] for r in found] == ["Cascading Failure Simulation", "Expert Panel"]
assert missing == ["Nope"]
retuned = {"method_name": "Expert Panel Review", "direction": "missing", "description": "d", "form": "f"}
merged = pick_methods.merge_extra(rows, pick_methods.load_extra(json.dumps([retuned])))
assert len(merged) == 2 and merged[1]["method_name"] == "Expert Panel Review" and merged[1]["num"] == "2"
found, missing = pick_methods.find(merged, ["Expert Panel", "Panel", "expert panel review"])
assert [r["method_name"] for r in found] == ["Expert Panel Review"] * 3 and not missing
SKILL.md
---
name: bmad-advanced-elicitation
description: 'Pushes a draft off its first answer with critique methods and proposes one revision. Use when the user or another skill names bmad-advanced-elicitation, or the user says "advanced elicitation". Not for a critique or a named method such as pre-mortem without that name.'
---
# Advanced Elicitation
A first draft is the most likely draft: centered, agreeable with its own prompt, biased toward adding and toward the first idea. You push it off center in chosen directions through cold readers, and propose a revision the user accepts or rejects. The target is a piece of work in the conversation, whoever wrote it. Work in the surrounding session's language.
## Setup
- Resolve customization: `uv run {project-root}/_bmad/scripts/resolve_customization.py --skill {skill-root} --project-root {project-root} --key workflow`; `{workflow.<key>}` below is that table. Script not found means BMad is not set up here: offer the `bmad` skill's setup (`npx skills add bmad-code-org/BMAD-METHOD --skill bmad` first if needed), then run it again. Run `{workflow.activation_steps_prepend}` before anything else and `{workflow.activation_steps_append}` after setup. Read `{workflow.persistent_facts}`: `file:` entries are paths or globs under `{project-root}` to load, the rest are facts as written.
- Resolve config: `uv run {project-root}/_bmad/scripts/resolve_config.py --project-root {project-root} --key core.output_folder --key core.active_initiative`. The session folder is `{workflow.output_dir}/{workflow.output_folder_name}/`, `/{active_initiative}` dropped when unset, `{topic_slug}` a kebab-case name for the target. When the folder already exists, continue the round numbering from the files there and append to its report.
- Catalog calls are `uv run {project-root}/_bmad/scripts/resolve_customization.py --skill {skill-root} --project-root {project-root} --key workflow | uv run {skill-root}/scripts/pick_methods.py --customization - --project-root {project-root} <command>`. `list --tier core` is the index, `show <name-or-num>` the row with its form line; a method outside the index still resolves by name through `show`, and `list --all` runs only when the user asks to see everything.
## The Target and the Start
The target is what the user or calling skill points at, at whatever size they name. With no pointer, it is what the last exchange was about, at its own size: the answer, choice or change you just gave, or the one the user just made. A one-line suggestion is a one-line target, not the plan or document around it; the rest of the context informs the pushes but is not pushed. A target that exists only in chat is staged to `target.md` in the session folder; a file target is used in place.
Choose the pushes and start them without waiting:
- Methods named in the call run as named, and a named direction gets its best-fitting method from `list`; say in one line if one is an odd fit for the target, then run it anyway. A call that asks for a surprise draws with `random --tier core --spread`.
- Otherwise `{workflow.default_methods}` run when set. When it is empty, run three pushes, each method from a different direction: the method from `list` that best fits where the target is probably weak, a wildcard from any tier (`list --tier more` adds the rest) chosen as the method whose answer is hardest to predict for this target, and a third push. For a single choice or a target of a few lines, the third is an open push, which asks to push harder and consider alternatives with no method; for a section or document, it is a second method against another weak spot.
One message names the target by path or phrase, says in a line or two where it is probably weak, names each method with its reason, and the round starts. Stop to ask first only when there is no clear target.
## The Brief
Each push gets a brief of exactly these parts and nothing of the conversation beyond them; a calling skill passes them when it invokes you, so use them without re-asking. The author holds every reason for every choice and cannot critique them cold; what a cold reader is not told cannot anchor it.
1. The target: its path, the section if partial, and that its structure (headings, template slots) is a constraint, not content to push. A target of a few lines is quoted in the brief instead.
2. Purpose and consumer, in a line or two.
3. What is settled: the constraints the author worked under, decisions already made, alternatives already rejected, one line each, so the push does not reopen them. Take them from the calling skill's memlog when it keeps one; the memlog itself stays out of the brief.
4. The method: its name, gist and form line from `show`, quoted.
5. The return, sized to the target, a few lines of form and at most five findings for a single choice: the form as the body, then a `## Findings` list, each with location, what is wrong or missing, and the proposed change; only what is specific to this draft, nothing that would be true of any draft of its kind.
Close with: return only that as text, invoke no skill, spawn nothing, edit nothing, read only the files named. A method with several voices (expert panel, stakeholder lens, Delphi) gets one brief per voice so the voices cannot see each other; a voice that should be an installed agent takes its persona from `uv run {project-root}/_bmad/scripts/roster.py --skill {skill-root} --project-root {project-root}`.
## The Round
Run every method push of the round at once, each in its own fresh subagent, all against the same version of the target. An open push is yours: while the readers run, write it from everything the conversation holds, in the same return shape. Save each return as `round-<r>-push-<n>-<method>.md` in the session folder, the open push as `open`.
Without subagents, run at most two pushes yourself, each read fresh from the target file and written in full before you read the other; say so in one line, and offer the brief as a prompt for a fresh session.
## Merge
Compose one proposed revision from the findings. A finding is set aside only on a stated ground and listed in one line so the user can rescue it. A finding that rejects the premise of the target is shown as a decision for the user, not folded into the revision. A push's narrative is color; only its located findings and their changes carry weight. Nothing in the target changes until the user accepts.
## Present and Choose
Append the round as a section of the report, `{workflow.output_folder_name}.md` in the session folder: the revision, the set-aside list, and the push files by name. Then one chat message:
- One line per push: its headline, in the method's voice.
- The revision as a numbered list of changes, each with its location, a short before and after, and the push that drove it. For a staged target, the revised text follows in full, since the user has no file to read it in; for a file target, the full passages go only in the report, named by path.
- Decisions, if any, and the set-aside list.
- The choice: apply all, apply by number, reject, rerun push N with direction, or direction.
- One line for next time: methods can be named in the call, set as defaults with `bmad-customize`, or listed on request.
Under the round in the report, record each change as applied or rejected. Apply by origin: a file target is edited in place and the changed passages shown; a staged target has `target.md` updated and the changed passages shown. Between rounds only the target and the settled list advance: an applied change is settled, a rejected one is a rejected alternative, and the findings themselves stay out of the next briefs so the readers stay cold. Then offer another round on the accepted version, suggesting a direction not yet pushed, or done.
## Done
Done hands the accepted version back in the caller's structure: the file path, or for a staged target its full text, printed here once, and signals completion so a calling skill resumes where it paused. Findings go back only on request. Then run each `{workflow.on_complete}` entry.
Teams change the catalog, facts and output location with the `bmad-customize` skill (`npx skills add bmad-code-org/BMAD-METHOD --skill bmad-customize` if it is not installed).