references/anti-patterns.md
# Anti-patterns — UX Research Discovery Testing
Prioritize these because AI agents commonly produce them when asked for UX, UI, frontend, or research help.
## Anti-pattern: Asking users what they want
What it looks like:
- “What feature do you want?”
- “Would you use this?”
- “Do you like this design?”
- Turning feature requests directly into roadmap items.
Why it fails:
- People often cannot accurately predict future behavior or explain hidden motivations.
- Preference claims are weak evidence and can hide the underlying need.
Better approach:
- Ask for recent specific stories and observed behavior.
- Ask what happened, what triggered it, what tools were used, what got in the way, and what workaround happened.
- If a user proposes a solution, interpret it as evidence of an underlying need.
Ask the user when:
- The product decision depends on whether to discover needs or evaluate a specific solution.
## Anti-pattern: Starting with a solution
What it looks like:
- Jumping straight to features, UI components, or visual redesign.
- Treating stakeholder requests as the problem statement.
- Testing a fully formed solution when a risky assumption could be tested faster.
Why it fails:
- The team may build the wrong thing efficiently.
- Early solution fixation narrows learning and hides better alternatives.
Better approach:
- Frame the business outcome, user behavior, opportunity, assumptions, and smallest test.
- Compare multiple opportunities or solution directions before choosing.
Ask the user when:
- The user asks for a redesign or feature but has not identified the user problem or outcome.
## Anti-pattern: Research theater
What it looks like:
- Running research because it “looks research-y.”
- Using a lab, survey, or focus group because stakeholders expect it.
- Producing a polished report that does not change a decision.
Why it fails:
- It wastes time and can legitimize weak evidence.
- It turns research into politics or decoration instead of decision support.
Better approach:
- Choose methods based on the decision and evidence needed.
- Use concise outputs that show evidence, confidence, recommended action, and next test.
Ask the user when:
- Stakeholder expectations conflict with the most appropriate lightweight method.
## Anti-pattern: One-person research silo
What it looks like:
- The researcher disappears, returns with a report, and expects the team to comply.
- Designers and engineers do not observe sessions or join synthesis.
- Findings are treated as the researcher’s opinion.
Why it fails:
- Teams are less likely to act on findings they did not experience.
- Critical domain knowledge from engineering, content, support, and design is lost.
Better approach:
- Involve the team in planning, observing, note-taking, and synthesis.
- Use visual artifacts and debriefs to create shared ownership.
Ask the user when:
- The recommendation must persuade a team or stakeholder group.
## Anti-pattern: Recruiting by demographics alone
What it looks like:
- “We need women age 25-34.”
- Recruiting whoever is easiest internally.
- Ignoring non-users, competitor users, experts, novices, admins, support, or other affected roles.
Why it fails:
- Demographics often do not predict task behavior, mental model, or context.
- Typical users alone may not reveal the forces shaping behavior.
Better approach:
- Recruit by recent behavior, task, context, relationship to the product, and contrast.
- Include the whole system of roles when the product affects more than one person.
Ask the user when:
- Participant selection is unclear or multiple user roles may matter.
## Anti-pattern: Leading questions and over-talking
What it looks like:
- Asking questions that imply the desired answer.
- Filling silence.
- Talking about yourself.
- Explaining the product during a test.
- Asking participants to validate the team’s opinion.
Why it fails:
- It contaminates evidence and suppresses real stories.
- Participants may try to please the moderator or avoid conflict.
Better approach:
- Ask neutral questions, stay silent, use follow-ups, and let participants struggle during tests.
- Use natural language and participant vocabulary.
Ask the user when:
- A script or guide contains leading wording.
## Anti-pattern: Fixing the participant’s problem during the session
What it looks like:
- Helping participants complete a usability task.
- Taking product support questions during an interview.
- Teaching the intended interaction before observing.
Why it fails:
- It prevents the team from seeing where the design fails.
- It changes task behavior and makes the evidence unreliable.
Better approach:
- Remind yourself that the goal is to learn.
- Save help, answers, or troubleshooting for the end unless participant welfare requires intervention.
Ask the user when:
- The study mixes support, training, sales, and research goals.
## Anti-pattern: Treating small qualitative studies as statistical proof
What it looks like:
- Claiming “80% of users want X” from five interviews.
- Prioritizing solely by count of mentions.
- Reporting percentages without sample and method limitations.
Why it fails:
- Small qualitative samples reveal patterns and causes, not population prevalence.
- Counts can hide severity, context, and task importance.
Better approach:
- Report patterns, examples, severity, confidence, and next evidence needed.
- Use quantitative methods when prevalence or precise comparison matters.
Ask the user when:
- They need prevalence, benchmarking, or statistically defensible prioritization.
## Anti-pattern: Analytics without why
What it looks like:
- Treating funnels, clicks, bounce rate, or conversion as self-explanatory.
- Optimizing a metric without understanding the task or context.
- Running A/B tests before knowing what problem is being solved.
Why it fails:
- Quantitative data shows what happened, not necessarily why.
- Local optimization can miss larger product opportunities.
Better approach:
- Pair analytics with qualitative research, usability testing, support evidence, or session review.
- Use A/B testing for specific measurable changes after the problem is understood.
Ask the user when:
- The user asks to optimize a metric but the cause of behavior is unknown.
## Anti-pattern: Opportunity backlog as feature backlog
What it looks like:
- Listing feature ideas under “opportunities.”
- Prioritizing by stakeholder excitement or implementation ease.
- Moving from one quote to one feature.
Why it fails:
- It skips the user need and makes the team overconfident in solutions.
- It prevents comparing opportunities and assumptions.
Better approach:
- Write opportunities as user needs, pain points, desires, barriers, or unmet jobs.
- Map candidate solutions and assumptions beneath each opportunity.
Ask the user when:
- The user asks for roadmap prioritization but the inputs are feature ideas.
## Anti-pattern: Verbose report that arrives too late
What it looks like:
- A long report delivered after the sprint/decision has moved on.
- Findings without recommended actions.
- Dense data with no visual synthesis.
Why it fails:
- Teams do not have time or context to act.
- The output becomes an archive instead of a decision tool.
Better approach:
- Use a concise decision memo, annotated screenshots, severity list, journey map, opportunity tree, or dashboard.
- Involve the team before the report is written.
Ask the user when:
- Stakeholders require a formal report, but immediate action is also needed.
## Anti-pattern: Polished personas without evidence
What it looks like:
- Personas invented from stereotypes.
- Persona details that do not affect design decisions.
- Demographic-heavy profiles with no goals, behaviors, environment, or constraints.
Why it fails:
- Fictional artifacts can create false confidence.
- Teams may optimize for personality details rather than task needs.
Better approach:
- Build personas from research patterns, goals, behavior, environment, and mental models.
- Use as few personas as possible and keep them decision-relevant.
Ask the user when:
- The user asks for personas but no research evidence exists.
## Anti-pattern: Accessibility as post-hoc compliance
What it looks like:
- Adding ARIA or color contrast after design is complete.
- Excluding disabled users from research.
- Treating assistive technology as an edge case.
Why it fails:
- Accessibility affects task context, component choice, content, interaction, and implementation.
- Retrofits are more expensive and less effective.
Better approach:
- Include accessibility needs in participant criteria, task design, prototype fidelity, and implementation guidance from the start.
- Test keyboard, focus, labels, dynamic updates, contrast, motion, and error recovery.
Ask the user when:
- Known accessibility needs or assistive technology contexts may change recruiting or implementation.
## Anti-pattern: Testing only your own product
What it looks like:
- Ignoring competitors or workaround tools.
- Assuming competition is limited to direct market rivals.
- Redesigning without seeing how users solve the problem today.
Why it fails:
- The strongest competitor may be an existing workaround or habit.
- Competitive testing reveals conventions, expectations, and opportunities.
Better approach:
- Test competitor or analogous flows for the same task when relevant.
- Compare strengths, weaknesses, conventions, and unmet needs.
Ask the user when:
- The product category has strong competitors or users already have established workarounds.
## Anti-pattern: Treating the experience map as final truth
What it looks like:
- A beautiful journey map is created and then frozen.
- The map represents internal opinion instead of evolving evidence.
- Teams debate wording instead of learning from users.
Why it fails:
- Maps are working models, not reality.
- Over-polishing can discourage updates and hide uncertainty.
Better approach:
- Start with rough individual maps, merge them, identify gaps, use interviews to update, and treat the map as provisional.
Ask the user when:
- The team asks for a journey map but has limited evidence.
references/checklists.md
# Checklists — UX Research Discovery Testing
Use these checklists before finalizing a research plan, interview guide, usability test, UX critique, or recommendation.
## Discovery/context checklist
- The product or feature area is named.
- The decision the research should inform is explicit.
- The desired user behavior or business outcome is stated.
- The primary user group is defined by behavior, task, and context.
- Adjacent or affected roles are considered: chooser, approver, admin, support, operator, influencer, non-user, competitor user.
- Existing evidence is listed: prior research, analytics, support tickets, sales calls, session recordings, stakeholder knowledge, screenshots, prototypes.
- Known assumptions are separated from facts.
- The riskiest assumptions are identified.
- Accessibility, language, device, connectivity, environment, and assistive-technology contexts are considered.
- Business, technical, legal, brand, design-system, and timeline constraints are listed.
- The proposed research method directly supports the decision.
- The expected output is clear: opportunity map, test findings, prioritized issues, decision memo, journey map, task flow, persona, or implementation guidance.
## Research plan checklist
- Problem statement uses an outcome verb such as identify, describe, evaluate, compare, or prioritize.
- Research questions are answerable through the chosen method.
- The plan avoids trying to answer prevalence, causality, and motivation with one lightweight method.
- Participant criteria are based on recent behavior or relevant context, not demographics alone.
- Recruiting channel and screener are specified.
- Screener questions do not reveal the “right” answers too obviously.
- Incentive, scheduling, reminders, and no-show handling are planned.
- Roles are assigned: lead, moderator, note-taker, recruiter/scheduler, observer, analyst/documenter.
- Consent, recording, privacy, and data handling are addressed.
- A pilot or pre-pilot is planned when the script, prototype, tasks, or logistics are uncertain.
- Synthesis is scheduled before data collection starts.
- Stakeholders who need to act on findings have an observation or debrief role.
## Interview guide checklist
- The guide starts with purpose, timing, consent, recording, confidentiality, and permission to skip questions.
- The opening questions build rapport without collecting irrelevant biography.
- Questions are neutral and do not contain the desired answer.
- Questions ask about recent specific experiences before asking general opinions.
- The guide includes prompts for triggers, goals, steps, tools, workarounds, people involved, environment, constraints, interruptions, emotions, and outcomes.
- The guide asks participants to show artifacts or demonstrate behavior when possible.
- The guide uses participant-friendly language rather than internal terminology.
- The guide includes follow-up prompts such as “Tell me more,” “What happened next,” and “Why was that important?”
- Hypothetical or future-state questions are clearly treated as exploratory, not reliable predictions.
- The moderator is instructed not to pitch, teach, fix, or defend the design.
- The wrap-up asks what important topic was missed.
- Post-session note cleanup and immediate synthesis are planned.
## Usability test checklist
- The artifact under test is identified: sketch, prototype, live UI, competitor flow, or component.
- The test objective is tied to a decision.
- Participants match the task context.
- Tasks are realistic scenarios, not instructions to click specific UI elements.
- The most critical task is tested first or protected from fatigue.
- Success criteria are defined: completion, major errors, assistance, time/effort, confidence, comprehension, accessibility blockers.
- The script reminds participants the design is being tested, not them.
- The moderator avoids leading, rescuing, or explaining during the task.
- The note template captures observed behavior, quotes, errors, workarounds, confusion points, and recovery paths.
- Accessibility checks include keyboard, focus order, labels, error announcements, contrast, motion, touch target size, and screen-reader-relevant behavior where applicable.
- Observers know when to stay silent and how to record notes.
- The severity scale is defined before the test.
- Findings will be prioritized by severity, frequency/confidence, task importance, and implementation effort.
- The plan includes a follow-up test or validation path for major fixes.
## Synthesis checklist
- Notes, recordings, screenshots, and artifacts are organized with consistent names.
- The team reviews data while it is fresh.
- Observations are separated from interpretations.
- Exact participant language is captured where it affects labels, content, or mental models.
- Findings are traceable to source sessions or artifacts.
- Patterns are grouped by behavior, motivation, barrier, trigger, workaround, tool, relationship, environment, and task.
- Outliers are considered rather than discarded automatically.
- The synthesis distinguishes user needs from requested solutions.
- Opportunities are framed from the user's perspective.
- Assumptions behind solution ideas are identified.
- Evidence strength and confidence are stated.
- Accessibility and inclusion findings are not buried under “edge cases.”
- Recommendations are prioritized and connected to the product decision.
## Opportunity/discovery checklist
- The business outcome is stated.
- The user behavior that drives the outcome is stated.
- The opportunity is expressed as a user need, pain point, desire, barrier, or unmet job—not as a feature.
- The opportunity map/tree separates outcome, opportunities, solutions, assumptions, and tests.
- The team has considered multiple opportunities before choosing one.
- The team has considered multiple solution directions before choosing one.
- The selected opportunity has evidence, not just executive enthusiasm.
- Prioritization considers customer value, business value, risk, reversibility, accessibility impact, and effort.
- The next test targets an assumption, not the entire solution when a smaller test would suffice.
- Stakeholders can understand the path from evidence to decision.
## UX/design quality checklist
- The primary user goal is clear.
- The primary action is visually and semantically stronger than secondary actions.
- The UI follows the user's task flow rather than the organization's internal structure.
- Content and labels use user language.
- Required information appears before it is needed.
- Feedback is timely and specific.
- Errors help users prevent, recognize, and recover from problems.
- Empty, loading, error, offline, and success states are designed.
- Navigation and wayfinding match user expectations.
- Grouping, spacing, and hierarchy reduce cognitive load.
- The design does not rely on color alone.
- The design avoids unnecessary steps, choices, and fields.
- The design supports novice users without blocking efficient expert use.
- The critique separates evidence-backed issues from taste preferences.
## Accessibility checklist
- Semantic HTML or platform-native semantics can represent the interaction.
- All interactive elements are keyboard reachable and operable.
- Focus order matches visual and task order.
- Visible focus indicators are present and high contrast.
- Form fields have programmatic labels, helper text, and error associations.
- Error messages state what happened and how to fix it.
- Dynamic updates are announced appropriately.
- Color contrast supports text and important graphical information.
- The design does not require color alone, hover alone, pointer precision, sound alone, or motion.
- Motion can be reduced or avoided for users who prefer reduced motion.
- Touch targets and spacing support mobile and motor constraints.
- Content is written in clear language and does not depend on insider vocabulary.
- Screen reader, magnification, zoom, and reflow implications are considered.
- Research participants with relevant accessibility needs are included or the limitation is explicitly flagged.
## Implementation feasibility checklist
- The recommendation can be built with semantic HTML or native controls where possible.
- Custom components have defined roles, states, properties, keyboard behavior, focus management, and announcements.
- Existing design-system components/tokens are reused unless there is a reason not to.
- The responsive behavior is specified for key breakpoints and content lengths.
- Loading, error, empty, success, disabled, and permission states are specified.
- Content requirements are included: labels, helper text, validation, confirmation, microcopy.
- Performance risks are identified for flows where delay affects trust or completion.
- Analytics/instrumentation events match the target behavior and avoid collecting unnecessary personal data.
- The recommendation avoids brittle one-off implementation.
- QA can test the critical task, accessibility behaviors, and edge states.
## Final response checklist
- The answer states assumptions and limitations.
- The recommendation is concise and action-oriented.
- The user value and business value are both clear.
- Evidence strength and confidence are clear.
- Tradeoffs are acknowledged.
- The next step is the smallest useful action.
- The response does not overclaim from small qualitative samples.
- The response does not present user preferences as facts.
- The response does not paste unnecessary long source text.
- Accessibility and frontend implications are included when relevant.
references/decision-prompts.md
# Decision Prompts — UX Research Discovery Testing
Use these prompts only when the missing context would materially change the research plan, critique, or recommendation. Do not ask the user to decide routine best practices.
## How to use
- Ask at most one or two questions before making progress.
- Always include a recommended default.
- If the user does not answer, proceed with the default and label it as an assumption.
- Prefer asking about decisions, outcomes, audience, constraints, and access—not methodology jargon.
### Decision: Product or design decision
When to ask:
- The request says “do research,” “critique this,” “improve UX,” or “test this” but does not identify what decision the work should inform.
Do not ask when:
- The user already names a concrete decision, such as choosing between two onboarding concepts or finding checkout usability issues.
Recommended default:
- Focus the work on the next product/design decision that could improve the primary user task and business outcome.
Reason:
- Research is only useful if it changes a decision.
Question tool pattern:
```js
question({
question: "What decision should this research or UX critique help you make?",
recommended_default: "Decide the next product/design change that best improves the user's primary task while supporting the business outcome.",
options: [
"Decide what problem or opportunity to pursue",
"Decide which solution or concept to build",
"Find usability issues in an existing UI/prototype",
"Prioritize improvements for a shipped product",
"Other / custom"
]
})
```
### Decision: Target outcome
When to ask:
- The user asks for product recommendations, prioritization, or design critique without naming the behavior or business result that matters.
Do not ask when:
- The outcome is implied by the artifact, such as checkout completion, account creation, task completion, or reduced support contact.
Recommended default:
- Use a user behavior metric that plausibly drives the business outcome.
Reason:
- Outcomes keep the agent from optimizing for outputs or taste.
Question tool pattern:
```js
question({
question: "What user behavior or business result should improve if this work succeeds?",
recommended_default: "Use one behavior metric tied to business value, such as task completion, activation, retention, conversion, successful self-service, or reduced support contact.",
options: [
"Complete a critical task with fewer errors",
"Activate or onboard successfully",
"Convert or purchase",
"Retain or repeat use",
"Reduce support/operations burden",
"Other / custom"
]
})
```
### Decision: Research stage
When to ask:
- It is unclear whether the team needs discovery, concept evaluation, usability testing, or post-launch measurement.
Do not ask when:
- The user explicitly asks for interviews, a usability test, analytics interpretation, or a critique of an existing UI.
Recommended default:
- If no solution exists, use discovery. If a solution exists, use evaluative testing. If live behavior data exists, combine metrics with qualitative follow-up.
Reason:
- The research method should match the maturity of the product decision.
Question tool pattern:
```js
question({
question: "What stage is the work in right now?",
recommended_default: "Use discovery if the problem is unclear; use usability testing if a design exists; use analytics plus qualitative follow-up if the product is live.",
options: [
"Exploring a problem/opportunity",
"Comparing concepts or solution directions",
"Testing a prototype or existing UI",
"Understanding live product behavior",
"Other / custom"
]
})
```
### Decision: Primary participant group
When to ask:
- The user has not identified who the research should include.
Do not ask when:
- The artifact is clearly for a known user role already described in the prompt.
Recommended default:
- Recruit people who recently performed or attempted the target task.
Reason:
- Recent behavior provides richer, less speculative evidence.
Question tool pattern:
```js
question({
question: "Who should we learn from first?",
recommended_default: "Start with people who recently tried to complete the target task, then add one contrasting group if it may reveal hidden constraints.",
options: [
"Current users performing the target task",
"Prospective users or non-users",
"Competitor users / recent switchers",
"Internal operators, admins, support, or sales",
"Other / custom"
]
})
```
### Decision: Participant contrast
When to ask:
- The product affects multiple roles, has a marketplace/workflow ecosystem, or the user asks who to recruit.
Do not ask when:
- A quick usability test on one critical flow is sufficient.
Recommended default:
- Include the primary user plus one contrast group: non-user, competitor loyalist, expert, novice, affected stakeholder, or extreme user.
Reason:
- Contrast reveals assumptions and influencing factors that typical users alone may hide.
Question tool pattern:
```js
question({
question: "Should we include a contrasting participant group to reveal hidden constraints?",
recommended_default: "Include one contrast group when the product has multiple roles, strong competitors, or high uncertainty.",
options: [
"No, focus only on primary users",
"Add non-users or prospective users",
"Add competitor loyalists/recent switchers",
"Add extreme/expert/novice users",
"Add affected stakeholders such as admins/support/approvers",
"Other / custom"
]
})
```
### Decision: Research access
When to ask:
- The user asks for a research plan but access to users, analytics, artifacts, or prototype is unknown.
Do not ask when:
- The task is to draft a generic guide/script using stated assumptions.
Recommended default:
- Use the strongest available evidence now, label confidence, and recommend the next smallest research loop.
Reason:
- Research plans must be realistic under access constraints.
Question tool pattern:
```js
question({
question: "What access do we currently have to users, artifacts, analytics, or prototypes?",
recommended_default: "Use the strongest available evidence now, flag confidence, and propose the smallest next research loop.",
options: [
"Direct access to users",
"Prototype or live product available for testing",
"Analytics/support tickets/session recordings available",
"Only stakeholders/internal knowledge available",
"Other / custom"
]
})
```
### Decision: Method depth and timeline
When to ask:
- The user asks for a plan but timeline, rigor, or budget may change method choice.
Do not ask when:
- The user asks for a lightweight plan or immediate critique.
Recommended default:
- Use a one-week lightweight loop: plan, recruit, 5 sessions, same-day synthesis, decision memo.
Reason:
- A fast loop is practical for product teams and prevents research from becoming a blocker.
Question tool pattern:
```js
question({
question: "How much time and rigor should this research use?",
recommended_default: "Run a lightweight one-week loop with 5 sessions, collaborative synthesis, and a decision memo.",
options: [
"Very fast: 1-2 days, directional",
"Lightweight: about 1 week",
"Moderate: 2-4 weeks with stronger recruiting/synthesis",
"High rigor: formal study, larger sample, or statistics",
"Other / custom"
]
})
```
### Decision: Artifact fidelity
When to ask:
- The user wants usability testing or critique but it is unclear what artifact exists.
Do not ask when:
- The user uploaded or described the artifact.
Recommended default:
- Test the lowest fidelity artifact that can reveal the target risk.
Reason:
- Early, cheap tests avoid expensive rework.
Question tool pattern:
```js
question({
question: "What artifact should we evaluate?",
recommended_default: "Test the lowest fidelity artifact that can reveal the key risk: sketch for concept, prototype for flow, live UI for real behavior.",
options: [
"Sketch/wireframe",
"Clickable prototype",
"Live UI",
"Competitor or analogous product",
"No artifact yet",
"Other / custom"
]
})
```
### Decision: Critical tasks
When to ask:
- Usability testing is requested but critical tasks are not specified.
Do not ask when:
- The artifact has an obvious primary task and the user asked for quick critique.
Recommended default:
- Test the highest-risk task that must succeed for the product to deliver value.
Reason:
- Not all tasks are equal; high-impact failures should drive test design.
Question tool pattern:
```js
question({
question: "Which task is most important to test first?",
recommended_default: "Start with the task whose failure would block user value or business value.",
options: [
"Find/understand information",
"Sign up/onboard/configure",
"Complete a transaction/submission",
"Recover from an error or edge case",
"Use a repeated workflow",
"Other / custom"
]
})
```
### Decision: Success criteria
When to ask:
- The user needs a usability test, benchmark, or post-launch measurement but has no success criteria.
Do not ask when:
- A qualitative directional test is enough and you can use task completion/errors/confidence by default.
Recommended default:
- Track task completion, major errors, time/effort, confidence, and qualitative cause of failure.
Reason:
- Metrics make findings actionable and comparable without pretending small samples are statistically representative.
Question tool pattern:
```js
question({
question: "What should count as success for the test or recommendation?",
recommended_default: "Track task completion, severe errors, time/effort, confidence, and the reason behind failures.",
options: [
"Task completion and severe errors",
"Speed/efficiency",
"Comprehension and confidence",
"Conversion or activation metric",
"Accessibility task success",
"Other / custom"
]
})
```
### Decision: Accessibility and inclusion coverage
When to ask:
- The product is broadly used, the user asks for a test plan, or accessibility needs may change recruiting, prototype requirements, or implementation guidance.
Do not ask when:
- The user explicitly asks for an accessibility review; proceed with accessibility as the core focus.
Recommended default:
- Include baseline accessibility coverage: keyboard, screen reader, contrast, reduced motion, cognitive load, language clarity, mobile constraints.
Reason:
- Accessibility is a user context and product quality requirement, not a final checklist.
Question tool pattern:
```js
question({
question: "Are there known accessibility needs, assistive technologies, language needs, or situational constraints to include?",
recommended_default: "Include baseline coverage for keyboard, screen reader, magnification, contrast, reduced motion, cognitive load, language clarity, and mobile constraints.",
options: [
"Known assistive technology users",
"Known cognitive/language/access constraints",
"No known data; include baseline accessibility coverage",
"This is an accessibility-specific study",
"Other / custom"
]
})
```
### Decision: Stakeholder involvement
When to ask:
- Research findings need to influence a team, roadmap, executive decision, or implementation.
Do not ask when:
- The user only needs a personal critique or quick script.
Recommended default:
- Invite product, design, engineering, and one decision-maker to observe or synthesize.
Reason:
- Direct exposure increases action and reduces report-only handoff.
Question tool pattern:
```js
question({
question: "Who needs to observe or participate so the findings lead to action?",
recommended_default: "Include product, design, engineering, and one decision-maker in observation or synthesis.",
options: [
"Product/design/engineering trio",
"Include executives or roadmap owners",
"Include support/sales/operations",
"Keep research contained for now",
"Other / custom"
]
})
```
### Decision: Privacy, sensitivity, and consent
When to ask:
- The study involves personal data, minors, health/finance/legal topics, workplace surveillance, screenshots, recordings, or sensitive contexts.
Do not ask when:
- The plan can safely use anonymous low-risk feedback and no recording.
Recommended default:
- Use informed consent, minimal data collection, anonymized reporting, and separate consent from NDA/incentive terms.
Reason:
- Ethical failures damage participants, product trust, and research validity.
Question tool pattern:
```js
question({
question: "Does this study involve sensitive data, vulnerable participants, recordings, minors, or regulated domains?",
recommended_default: "Use informed consent, minimal data collection, anonymized reporting, and separate consent from NDA/incentive terms.",
options: [
"No sensitive data expected",
"Recordings or screenshots involved",
"Personal, financial, health, legal, or workplace data involved",
"Minors or vulnerable participants involved",
"Other / custom"
]
})
```
### Decision: Frontend/design-system constraints
When to ask:
- Recommendations will become frontend implementation or design-system work and constraints are unknown.
Do not ask when:
- The user only asks for a research plan.
Recommended default:
- Prefer semantic, accessible, responsive, design-system-aligned components and identify instrumentation for the target outcome.
Reason:
- UX recommendations that cannot be implemented accessibly and maintainably are not good product decisions.
Question tool pattern:
```js
question({
question: "What frontend or design-system constraints should the recommendation respect?",
recommended_default: "Use existing accessible components and tokens where possible, preserve semantic HTML, and add instrumentation for the target behavior.",
options: [
"Existing design system/components",
"Specific frontend framework constraints",
"Performance or mobile constraints",
"Analytics/instrumentation requirements",
"No known constraints",
"Other / custom"
]
})
```
references/principle-cards.md
# Principle Cards — UX Research Discovery Testing
These cards consolidate the reusable lessons from the uploaded sources into agent behavior.
## Principle: Frame the decision before the method
Rule:
- Start by identifying the decision, research problem, target user/context, and expected output. Then choose the method.
Why it matters:
- Method-first research wastes time and can produce findings that do not change a product decision.
Use when:
- Planning research, critiquing a UI, choosing between concepts, or recommending next steps.
Do not use when:
- The user only asks for a narrowly specified deliverable and already provides the decision context.
Default recommendation:
- Write a one-sentence decision statement and one primary research question.
Ask the user when:
- The decision is missing and cannot be safely inferred.
Question prompt:
- `question({ question: "What decision should this research or UX critique help you make?", recommended_default: "Decide the next product/design change that best improves the user's primary task while supporting the business outcome.", options: ["Decide what problem/opportunity to pursue", "Decide which solution to build", "Find usability issues", "Prioritize shipped-product improvements", "Other / custom"] })`
Agent behavior:
- Do not draft a method-heavy plan until the decision and question are clear. If missing, ask once or proceed with a stated assumption.
## Principle: Separate business questions from research questions
Rule:
- Translate broad business questions into researchable questions about people, tasks, motivations, constraints, and context.
Why it matters:
- “What should we build?” is not directly answerable by users; “How do people currently solve this and where do they struggle?” is.
Use when:
- The user asks about roadmap, features, redesign, product-market fit, or opportunity discovery.
Do not use when:
- The request is purely evaluative and the test task is already clear.
Default recommendation:
- Convert the business question into 3-5 research questions before writing interview or test materials.
Ask the user when:
- Business outcome or intended user behavior is unclear.
Question prompt:
- `question({ question: "What user behavior or business result should improve if this work succeeds?", recommended_default: "Use one behavior metric tied to business value, such as completion, activation, retention, conversion, self-service, or support reduction.", options: ["Task completion", "Activation/onboarding", "Conversion/revenue", "Retention/repeat use", "Support reduction", "Other / custom"] })`
Agent behavior:
- Show the translation from business decision to research question so the user can see why the method fits.
## Principle: Prefer observed behavior over preferences
Rule:
- Treat behavior, context, artifacts, task performance, and recent stories as stronger evidence than likes, hypotheticals, and generic feature requests.
Why it matters:
- People often report idealized or socially acceptable answers; behavior reveals constraints and workarounds.
Use when:
- Writing interview questions, interpreting survey results, critiquing findings, or designing tests.
Do not use when:
- The research question is genuinely about preference, brand association, or sentiment; even then, label evidence appropriately.
Default recommendation:
- Ask “Tell me about the last time…” and “Show me how…” before asking opinions.
Ask the user when:
- They want preference data to drive a high-risk product decision.
Question prompt:
- `question({ question: "Do you need to understand actual behavior or stated preference for this decision?", recommended_default: "Study actual behavior first; use stated preference only as secondary evidence.", options: ["Actual behavior/task success", "Motivations and context", "Preference/sentiment", "Both, triangulated", "Other / custom"] })`
Agent behavior:
- Rewrite leading or preference questions into behavior-based prompts.
## Principle: Use continuous, small learning loops
Rule:
- Prefer frequent lightweight customer contact and iterative tests over one large, late research phase.
Why it matters:
- Product decisions happen continuously; stale or one-time research decays quickly.
Use when:
- The team owns an ongoing product, roadmap, or design system.
Do not use when:
- The domain requires formal longitudinal research, regulatory review, or high statistical confidence before action.
Default recommendation:
- For product teams, schedule weekly customer contact and synthesize continuously. For one-off projects, run the smallest credible loop.
Ask the user when:
- User access, cadence, or team availability is unknown.
Question prompt:
- `question({ question: "What research cadence is realistic for this team?", recommended_default: "Use weekly customer contact for ongoing product teams; use a one-week lightweight loop for one-off decisions.", options: ["Weekly continuous interviews/tests", "One-week lightweight loop", "One-time study", "No user access yet", "Other / custom"] })`
Agent behavior:
- Recommend a cadence and recruiting mechanism, not just a one-time plan.
## Principle: Recruit for behavior, context, and contrast
Rule:
- Recruit participants based on what they do, recently did, are trying to do, or are affected by—not demographics alone.
Why it matters:
- Behavioral contrast reveals hidden influences, roles, and constraints that typical-user sampling can miss.
Use when:
- Planning interviews, usability tests, diary studies, or discovery.
Do not use when:
- Demographic criteria directly affect access, legal requirements, safety, culture, or equity; then combine demographics with behavior.
Default recommendation:
- Start with people who recently attempted the target task; add one contrasting group if uncertainty is high.
Ask the user when:
- The target user or participant criteria are not specified.
Question prompt:
- `question({ question: "Who should we learn from first?", recommended_default: "Start with people who recently tried to complete the target task, then add one contrasting group if it may reveal hidden constraints.", options: ["Current task performers", "Prospective/non-users", "Competitor users/switchers", "Internal operators/admins/support", "Other / custom"] })`
Agent behavior:
- Draft screeners around behaviors, recency, tools, contexts, and role in the decision system.
## Principle: Use a guide, then listen beyond it
Rule:
- Prepare a field guide to align the team, but follow the participant’s story when it reveals useful evidence.
Why it matters:
- Rigid scripts miss the unexpected; unplanned conversations drift. Good interviews balance structure and curiosity.
Use when:
- Conducting interviews, contextual inquiry, stakeholder interviews, or concept exploration.
Do not use when:
- The method requires standardized task metrics or survey consistency.
Default recommendation:
- Use a semi-structured guide with intro, main topics, probes, artifacts/demonstrations, and wrap-up.
Ask the user when:
- The study requires comparability versus exploration.
Question prompt:
- `question({ question: "Should this session optimize for comparability or exploration?", recommended_default: "Use a semi-structured guide: consistent core questions with flexible follow-ups.", options: ["Comparable metrics across participants", "Exploratory discovery", "Hybrid semi-structured", "Other / custom"] })`
Agent behavior:
- Include follow-up prompts and instructions for silence, transitions, and non-leading language.
## Principle: Test tasks, not opinions
Rule:
- Usability tests should ask participants to accomplish realistic goals and observe whether they can complete them.
Why it matters:
- A user may like a design that still fails the task; a confusing flow may not be obvious until use.
Use when:
- Evaluating sketches, prototypes, live UIs, competitors, flows, forms, navigation, dashboards, or components.
Do not use when:
- The question is about brand perception or desirability; then use concept exploration and label it separately.
Default recommendation:
- Test 3-5 critical tasks with clear success criteria and severity scoring.
Ask the user when:
- The critical task is not obvious.
Question prompt:
- `question({ question: "Which task is most important to test first?", recommended_default: "Start with the task whose failure would block user value or business value.", options: ["Find/understand information", "Sign up/onboard/configure", "Complete a transaction/submission", "Recover from an error", "Use a repeated workflow", "Other / custom"] })`
Agent behavior:
- Write task scenarios in user-goal language and instruct the moderator not to help unless participant welfare requires it.
## Principle: Synthesize collaboratively and visually
Rule:
- Turn raw notes into visible patterns, maps, issue lists, opportunity trees, or annotated screenshots with the team.
Why it matters:
- Shared synthesis builds shared understanding and action. Visual artifacts make evidence easier to inspect and challenge.
Use when:
- Translating interviews/tests into recommendations or persuading stakeholders.
Do not use when:
- Confidentiality limits who can see raw data; then anonymize and synthesize with a smaller approved group.
Default recommendation:
- Run a same-day synthesis session after each batch of sessions.
Ask the user when:
- Stakeholders need to act but have not been included.
Question prompt:
- `question({ question: "Who needs to observe or join synthesis so the findings lead to action?", recommended_default: "Include product, design, engineering, and one decision-maker in observation or synthesis.", options: ["Product/design/engineering trio", "Executives/roadmap owners", "Support/sales/operations", "Keep research contained", "Other / custom"] })`
Agent behavior:
- Produce concise visual structures and trace findings back to evidence.
## Principle: Triangulate qualitative and quantitative evidence
Rule:
- Use qualitative methods to understand context and causes; use quantitative methods to monitor scale, change, and outcomes.
Why it matters:
- Numbers without stories lack explanation; stories without metrics lack scale.
Use when:
- The product is live, analytics exist, or stakeholders ask “how many” or “how much.”
Do not use when:
- There is no traffic or metric yet; use qualitative discovery or concept tests first.
Default recommendation:
- Pair analytics or survey signals with interviews/usability tests before recommending major changes.
Ask the user when:
- The required confidence level or metric is unclear.
Question prompt:
- `question({ question: "Do you need directional insight or quantified confidence for this decision?", recommended_default: "Use directional qualitative evidence for early decisions; use quantitative methods for prevalence, benchmarks, and post-launch impact.", options: ["Directional insight", "Quantified confidence", "Both, triangulated", "Other / custom"] })`
Agent behavior:
- State what each evidence source can and cannot prove.
## Principle: Prioritize opportunities before solutions
Rule:
- Express opportunities as user needs, barriers, desires, or unmet jobs; map multiple solutions beneath them.
Why it matters:
- Prioritizing feature ideas too early produces roadmaps disconnected from user and business value.
Use when:
- Creating roadmaps, redesign priorities, opportunity maps, or product recommendations.
Do not use when:
- The user only needs a narrow defect fix; then still identify the user problem briefly.
Default recommendation:
- Use an opportunity solution tree or equivalent structure: outcome → opportunities → solutions → assumptions/tests.
Ask the user when:
- The target outcome is missing.
Question prompt:
- `question({ question: "What outcome should the opportunity map support?", recommended_default: "Use one measurable user behavior tied to business value.", options: ["Activation", "Task completion", "Conversion", "Retention", "Support reduction", "Other / custom"] })`
Agent behavior:
- Rewrite feature requests into opportunity statements before prioritizing.
## Principle: Make recommendations action-ready
Rule:
- Findings should culminate in prioritized actions, confidence, risks, and next tests.
Why it matters:
- Reports that do not drive action fail even if the research was good.
Use when:
- Delivering critique, research synthesis, findings, or roadmap recommendations.
Do not use when:
- The user explicitly asks only for raw notes or transcript cleanup.
Default recommendation:
- Provide a decision memo with top findings, action, evidence, confidence, tradeoffs, and next step.
Ask the user when:
- The audience for the recommendation affects format or detail.
Question prompt:
- `question({ question: "Who will use these findings and what format will help them act?", recommended_default: "Use a concise decision memo plus annotated screenshots or a prioritized issue list.", options: ["Product/design/engineering team", "Executives/roadmap decision-makers", "Frontend implementers", "Formal research archive", "Other / custom"] })`
Agent behavior:
- Do not stop at “users struggled.” Say what to change, why, how confident to be, and how to validate.
## Principle: Treat accessibility as user context
- Skill author synthesis from source emphasis on context, behavior, and task success.
Rule:
- Include accessibility needs, assistive technology, device constraints, language, and situational limitations in research and recommendations from the beginning.
Why it matters:
- Accessibility affects task performance, content, interaction patterns, frontend implementation, and product inclusion.
Use when:
- Reviewing or creating any UI, test plan, prototype, form, flow, or component.
Do not use when:
- Never omit accessibility entirely; scale the depth to the risk and scope.
Default recommendation:
- Include baseline keyboard, screen reader, contrast, focus, motion, error recovery, and language clarity coverage.
Ask the user when:
- Known accessibility groups, assistive technologies, or compliance constraints may alter the plan.
Question prompt:
- `question({ question: "Are there known accessibility needs, assistive technologies, language needs, or situational constraints to include?", recommended_default: "Include baseline coverage for keyboard, screen reader, magnification, contrast, reduced motion, cognitive load, language clarity, and mobile constraints.", options: ["Known assistive technology users", "Known cognitive/language/access constraints", "Baseline coverage", "Accessibility-specific study", "Other / custom"] })`
Agent behavior:
- Flag evidence gaps when research excludes relevant accessibility contexts.
SKILL.md
---
name: ux-research-discovery-testing
description: "Use when planning UX research, discovery interviews, usability tests, synthesis, or evidence-backed product recommendations."
license: MIT
metadata:
author: hueyexe
---
# UX Research Discovery Testing
## Purpose
Help an agent make better product, UX, UI, and frontend recommendations by grounding them in user goals, observed behavior, business outcomes, and testable evidence. Use this skill to choose just-enough research methods, write interview and usability-test plans, synthesize findings, identify opportunities, and explain evidence-backed product decisions.
It is an operating manual for doing practical research under real product constraints.
## When to use this skill
Use this skill when the user asks you to:
- Plan discovery, customer interviews, field visits, contextual inquiry, or stakeholder interviews.
- Review, critique, redesign, or generate a UI where user behavior, task success, accessibility, or product outcomes matter.
- Create an interview guide, screener, usability-test script, research plan, synthesis framework, opportunity map, journey map, persona, task flow, or evidence-backed recommendation.
- Decide what to build, improve, remove, or test next.
- Evaluate a prototype, live UI, design system component, onboarding flow, form, navigation model, dashboard, empty state, error flow, or content hierarchy.
- Turn qualitative notes, analytics, survey findings, support tickets, screenshots, or observations into product decisions.
## When not to use this skill
Do not use this skill as the primary tool when the task is only:
- Visual UI styling with no user goal, task, behavior, or product decision involved.
- Brand identity, illustration, or aesthetic exploration unrelated to use.
- Statistical analysis requiring rigorous sampling, inferential statistics, or experimental design beyond lightweight product research.
- Legal, medical, or regulated human-subjects research advice. In those cases, recommend qualified review and use only general UX planning guidance.
- Production frontend coding where the user has already specified the user problem, behavior, accessibility requirements, and interaction design.
## Core principles
1. **Start with the decision, not the method.** Identify the product, design, or frontend decision the research must inform. Choose only the research activities that reduce uncertainty for that decision.
2. **Separate business questions from research questions.** Convert “What should we build?” into researchable questions about people, tasks, motivations, contexts, constraints, and current workarounds.
3. **Prefer behavior and context over preference claims.** Treat “Do you like it?” and “What do you want?” as weak evidence. Ask for recent, specific stories; observe work; test tasks; and inspect real artifacts.
4. **Balance customer value and business value.** Do not optimize only for shipped features, executive preferences, or isolated user requests. Frame work around outcomes: a behavior change that creates customer value and supports the business.
5. **Use the smallest credible research loop.** Default to lightweight, iterative research that can influence the next decision. Research should accelerate learning, not become theater.
6. **Make assumptions explicit.** Before research or critique, list assumptions about users, contexts, motivations, constraints, accessibility needs, and business goals. Convert risky assumptions into testable questions.
7. **Use mixed evidence.** Qualitative work explains why and how; quantitative evidence shows what, where, how often, and whether a change moved a metric. Do not ask one method to answer every kind of question.
8. **Recruit for behavior, role, and contrast.** Recruit people who have relevant experience with the task or context. Include adjacent roles, non-users, recent defectors, competitor loyalists, extreme users, and affected stakeholders when they can reveal constraints or opportunities.
9. **Research is a team sport.** Involve product, design, engineering, content, accessibility, support, sales, and other stakeholders where useful. Direct exposure to users is more persuasive than a long report.
10. **Synthesize visibly.** Use maps, affinity clusters, opportunity trees, task flows, journey maps, screenshots, and short evidence notes to create shared understanding.
11. **Protect participants.** Explain the purpose, consent, recording, confidentiality, incentives, and use of findings. Separate consent from NDAs and incentives. Respect participant time and welfare.
12. **Recommend action, not just findings.** Findings should lead to prioritized decisions, risks, next tests, and product changes.
See [references/principle-cards.md](references/principle-cards.md) for each principle as a reusable card.
## Default recommendations
Use these defaults unless the user provides stronger context.
| Area | Default recommendation | Why this is usually best | Override when |
|---|---|---|---|
| Research goal | Define one decision and one primary research question before choosing methods. | Prevents unfocused research and over-asking. | The user is explicitly exploring a broad product area. |
| Discovery method | Start with 5-8 semi-structured interviews or contextual sessions anchored in recent real experiences. | Fast enough for product work and rich enough to reveal behaviors, context, language, and assumptions. | The task is safety-critical, highly regulated, or needs statistical confidence. |
| Interview cadence | For ongoing product teams, schedule weekly customer contact. For one-off work, run the smallest batch that can inform the next decision. | Continuous exposure prevents stale assumptions. | The team has no participant access; then use secondary research, support logs, analytics, or internal experts while flagging lower confidence. |
| Interview style | Use a guide, but keep it flexible. Ask about recent stories, examples, artifacts, workarounds, and context. | Specific episodes beat abstract opinions. | The study requires comparable metrics across participants. |
| Usability testing | Test realistic tasks with representative or high-learning participants; ask participants to think aloud only when it does not distort the task. | Task performance reveals usability issues better than opinions. | The product context makes think-aloud unsafe, unrealistic, or too disruptive. |
| Sample size | Start small, iterate, and keep recruiting. Use small tests to find issues, not to estimate population prevalence. | Small batches expose high-impact issues quickly. | The user needs prevalence, segmentation, or statistical confidence. |
| Participant criteria | Recruit by behavior, task, context, and relationship to the product, not demographics alone. | Demographics rarely describe the job-to-be-done or mental model by themselves. | Demographics directly affect access, safety, needs, culture, or equity. |
| Synthesis | Analyze immediately after sessions with multiple team members. Separate observations from interpretations and recommendations. | Reduces memory loss, bias, and report-only handoff. | Confidentiality or team structure prevents broad participation. |
| Opportunity framing | Map findings into outcomes, opportunities, candidate solutions, assumptions, and tests. | Keeps teams from jumping from one quote to a feature. | The user only needs a quick usability defect list. |
| Evidence strength | Treat observed behavior, task success, analytics, and controlled tests as stronger evidence than preference surveys. | Reduces credulity and self-report bias. | The research question is about awareness, sentiment, brand perception, or stated expectations. |
| Reporting | Produce a concise decision memo: decision, evidence, confidence, recommended action, alternatives, risks, and next test. | Stakeholders need action, not a research archive. | The user requests a formal report or compliance artifact. |
| Accessibility | Include accessibility needs as research criteria and testable requirements from the start. | Accessibility changes task context, interaction cost, and implementation choices. | Never fully omit; only scale depth to project scope. |
## Required user questions
Ask only when the answer materially changes the research plan, critique, or recommendation. Use the recommended default first; do not ask routine best-practice questions.
### Ask when the product decision is unclear
```js
question({
question: "What decision should this research or UX critique help you make?",
recommended_default: "Decide the next product/design change that best improves the user's primary task while supporting the business outcome.",
options: [
"Decide what problem or opportunity to pursue",
"Decide which solution or concept to build",
"Find usability issues in an existing UI/prototype",
"Prioritize improvements for a shipped product",
"Other / custom"
]
})
```
### Ask when the target user or context is unclear
```js
question({
question: "Who is the primary user or participant group, and what real-world context should we design or test for?",
recommended_default: "Recruit people who recently tried to complete the target task, plus one adjacent or contrasting group if it may reveal hidden constraints.",
options: [
"Current users performing the target task",
"Prospective users or non-users",
"Competitor users / recent switchers",
"Internal users, operators, support, or admins",
"Other / custom"
]
})
```
### Ask when the success outcome is unclear
```js
question({
question: "What user behavior or business outcome should improve if this work succeeds?",
recommended_default: "Use one user behavior metric tied to a business outcome, such as task completion, activation, retention, conversion, reduced support contact, or successful self-service.",
options: [
"Task completion / fewer errors",
"Activation or onboarding success",
"Conversion or revenue action",
"Retention or repeat use",
"Support reduction or operational efficiency",
"Other / custom"
]
})
```
### Ask when participant access or research constraints are unclear
```js
question({
question: "What access do we have to users, artifacts, analytics, and the product/prototype?",
recommended_default: "Use the best available evidence now, flag confidence, and propose the next smallest research loop.",
options: [
"Can interview or observe users directly",
"Can run moderated or unmoderated usability tests",
"Have analytics/support tickets/session recordings",
"Only have stakeholder knowledge right now",
"Other / custom"
]
})
```
### Ask when accessibility needs may change the plan
```js
question({
question: "Are there known accessibility needs, assistive technologies, language needs, or situational constraints that must be included?",
recommended_default: "Plan for keyboard, screen reader, magnification, color contrast, reduced motion, cognitive load, mobile constraints, and diverse language proficiency unless the product scope clearly narrows this.",
options: [
"Known assistive technology users",
"Known cognitive/language/access constraints",
"No known data; include baseline accessibility coverage",
"This is an accessibility-specific study",
"Other / custom"
]
})
```
Use the question-tool-ready prompts in [references/decision-prompts.md](references/decision-prompts.md) for the full decision set.
## Workflow
### 1. Triage the request
Identify:
- Product or feature area.
- Stage: discovery, concept, prototype, shipped UI, redesign, or ongoing optimization.
- Decision to be made.
- Target users, affected roles, and context.
- Existing evidence and artifacts.
- Risks: accessibility, privacy, safety, trust, business impact, technical constraints.
- Deadline and research access.
If any of these are missing but essential, ask one focused question. Otherwise proceed with assumptions and label them.
### 2. Choose the research mode
Use the research mode that matches the decision:
- **Generative / exploratory:** Use when the team does not yet know the right problem or opportunity. Use interviews, field visits, diary/logging, stakeholder interviews, secondary research, support-ticket review, and competitive observation.
- **Descriptive / explanatory:** Use when the problem exists but the team needs to understand the workflow, context, user groups, mental models, tasks, or constraints.
- **Evaluative:** Use when there is a concept, prototype, UI, flow, or competitor experience to test. Use task-based usability testing, heuristic review, accessibility review, cognitive walkthroughs, and prototype tests.
- **Causal / quantitative:** Use when a shipped product has measurable behavior and the team needs to know whether a change affects a metric. Use analytics, A/B tests, funnel analysis, and task metrics, while using qualitative work to explain why.
### 3. Frame the decision and assumptions
Before writing a plan or recommendation:
1. State the business outcome.
2. State the user behavior that would drive it.
3. State the target user/context.
4. List known facts.
5. List assumptions.
6. Identify the riskiest assumptions.
7. Choose the smallest method that can reduce those risks.
### 4. Plan lightweight discovery
For discovery or interviews, produce:
- Research objective.
- Research questions.
- Participant criteria based on behavior/context.
- Recruiting channels and screener questions.
- Interview/session guide.
- Consent and recording plan.
- Roles: moderator, note-taker, observer, recruiter, analyst.
- Schedule and synthesis plan.
- Decision output: opportunity map, findings memo, journey/task map, or recommendations.
### 5. Conduct interviews or contextual sessions
Use these moderation rules:
- Start with consent, purpose, timing, confidentiality, recording, and permission to skip questions.
- Build rapport with just enough small talk.
- Ask for recent, concrete stories: “Tell me about the last time…”
- Ask follow-ups before moving on.
- Use silence after asking and after answers.
- Use the participant’s language.
- Ask about artifacts, tools, environment, interruptions, workarounds, triggers, relationships, and constraints.
- Avoid putting answers in the question.
- Avoid teaching, fixing, selling, or defending the design during the session.
- Save participant questions or troubleshooting for the end.
- Capture exact phrases where useful for interface language.
### 6. Plan usability testing
For usability tests, produce:
- Test objective and product decision.
- Artifact under test: sketch, prototype, live UI, competitor flow, or component.
- Participants and why they are high-learning.
- Critical tasks in realistic wording.
- Success criteria: completion, errors, time/effort, confidence, comprehension, accessibility, and severe friction.
- Moderator script.
- Data capture plan.
- Severity rubric.
- Post-test synthesis and next action.
Task wording should describe the participant’s goal, not the UI steps. Do not ask participants to find a specific button unless the button is the object being tested.
### 7. Synthesize evidence
During synthesis:
1. Review notes and recordings quickly.
2. Capture observations, not interpretations, first.
3. Cluster behaviors, quotes, obstacles, motivations, workarounds, tools, triggers, environments, and relationships.
4. Distinguish what happened, what it means, and what to do.
5. Map findings to opportunities, assumptions, and candidate tests.
6. Prioritize by customer impact, business impact, frequency/confidence, risk, accessibility impact, and implementation effort.
7. State confidence level and what would change your mind.
Use visual artifacts when they help: affinity diagram, journey map, task flow, opportunity solution tree, screenshot forensics, mental model map, workflow diagram, or severity matrix.
### 8. Make research-backed recommendations
Every recommendation should include:
- Product decision or change.
- Evidence used.
- User problem or opportunity.
- Expected behavior change.
- Business value.
- Accessibility and inclusion implications.
- Frontend/design-system implications.
- Tradeoffs and risks.
- Confidence level.
- Next test or metric.
Prefer “Based on the evidence, do X next because…” over “Users said they want X.”
## Decision framework
### Evidence ladder
Use this ladder to communicate confidence:
1. **Observed behavior in context:** field visits, contextual inquiry, usability tests, support/session evidence.
2. **Behavioral product data:** analytics, funnels, search logs, task success, A/B tests.
3. **Artifacts and workarounds:** screenshots, spreadsheets, notes, tickets, forms, tools people actually use.
4. **Structured self-report:** interviews about recent behavior, diary studies, well-designed surveys.
5. **Weak preference claims:** likes, hypotheticals, feature requests, generic survey responses.
Do not discard weaker evidence; label it appropriately and triangulate it.
### Method selection
- Need to know **who users are and what they do**: interview, observe, contextual inquiry, stakeholder/support review.
- Need to know **whether a UI works**: task-based usability test, accessibility review, heuristic review.
- Need to know **what changed after launch**: analytics, funnels, A/B test, support-volume change, task metric.
- Need to know **why a metric changed**: combine analytics with interviews, session review, usability testing, or support analysis.
- Need to know **what to build next**: opportunity discovery, experience map, opportunity solution tree, assumption mapping, concept tests.
- Need to know **how to prioritize**: compare opportunities, not just solutions; use customer impact, business outcome, reversibility, risk, and effort.
### Prioritization rules
- Prioritize opportunities before solutions.
- Treat early product decisions as reversible when possible.
- Compare sets of opportunities or solutions instead of evaluating one idea in isolation.
- Test assumptions, not whole ideas, when a smaller test can reduce the risk.
- Start with high-impact usability and accessibility blockers.
- Do not delay all action while waiting for perfect data.
## Practical rules
### Research planning
- Write a one-paragraph problem statement before any guide or test script.
- Include only methods that support the decision.
- Start with a pilot or pre-pilot when the script, task, prototype, or logistics are uncertain.
- Record known assumptions and potential biases.
- Plan synthesis before collecting data.
### Interview guides
A good guide includes:
1. Intro, consent, purpose, timing.
2. Participant background relevant to the task.
3. Main body organized by research question.
4. Specific recent stories and demonstrations.
5. Artifact/environment probes.
6. Optional projection or ideal-experience prompts, clearly treated as exploratory.
7. Wrap-up: missed topics, participant questions, incentive/next steps.
### Usability test scripts
A good script includes:
1. Intro and consent.
2. Reminder that the design is being tested, not the participant.
3. Scenario setup.
4. Realistic tasks.
5. Success criteria and note-taking fields.
6. Non-leading probes.
7. Post-task questions about confidence and expectations.
8. Wrap-up.
### Synthesis
- Analyze as a group when possible.
- Separate observations from interpretations.
- Keep traceability back to source notes.
- Turn repeated patterns into insights.
- Turn insights into product opportunities, design principles, and testable changes.
- Avoid solving too early during analysis.
### Reporting
Default to a concise memo or annotated artifact:
- Summary.
- Top findings.
- Evidence and confidence.
- Recommended decisions.
- Prioritized fixes or opportunities.
- Risks and tradeoffs.
- Next research/test.
- Appendix only when useful.
## Accessibility and inclusion requirements
Always treat accessibility as part of user context, not as a final compliance pass.
- Include participants with relevant disabilities, assistive technologies, device constraints, language needs, and situational limitations when the product is broadly used.
- Ask about actual tools and settings: keyboard use, screen reader, magnification, voice input, reduced motion, captions, contrast, translation, mobile constraints, and cognitive load.
- Test critical tasks with keyboard-only interaction when UI behavior is relevant.
- Inspect focus order, visible focus states, labels, roles, error messages, dynamic updates, reduced-motion behavior, and touch target size.
- Avoid research plans that exclude people because recruiting them is inconvenient.
- Provide accommodations, accessible prototypes, clear consent, and flexible session logistics.
- Flag when research evidence lacks accessibility coverage and propose a follow-up.
## Frontend implementation guidance
When recommendations affect frontend work, include implementation implications:
- Use semantic HTML for controls, navigation, headings, forms, lists, tables, and landmarks.
- Preserve native behavior unless a custom component is necessary.
- Specify keyboard interaction, focus management, ARIA only where needed, and screen-reader announcements for dynamic states.
- Design and test empty, loading, error, offline, success, permission, and partial-data states.
- Align recommendations with design tokens, component APIs, responsive breakpoints, and existing design-system patterns.
- Add analytics or event instrumentation for the target behavior, but do not instrument without a decision-relevant metric.
- Consider performance and latency as UX variables, especially for task success and perceived trust.
- Include content requirements: labels, helper text, error copy, confirmation messages, and progressive disclosure.
- Avoid recommendations that require brittle one-off UI if a reusable component or pattern is appropriate.
## Critique workflow
When reviewing an existing UI, prototype, or frontend implementation, inspect in this order:
1. **User goal and product outcome:** What task is this supposed to support and what behavior should change?
2. **Context and audience fit:** Who is using it, where, under what constraints, with what prior knowledge?
3. **Evidence:** What is known from research, analytics, support, stakeholder knowledge, or observed behavior?
4. **Task flow:** Can users start, progress, recover, and finish?
5. **Information hierarchy:** Are the most important decisions, content, and actions obvious?
6. **Interaction clarity:** Are controls, states, validation, navigation, and feedback understandable?
7. **Accessibility and inclusion:** Can users operate and understand it across input modes, assistive tech, language needs, and cognitive load?
8. **Content and terminology:** Does the UI use language users understand from their context?
9. **Visual design only as it supports use:** Does spacing, contrast, grouping, typography, and emphasis clarify the task?
10. **Frontend feasibility:** Can this be built maintainably with semantic, responsive, performant, design-system-aligned components?
11. **Risks and next test:** What evidence is missing, and what is the smallest credible test?
## Creation workflow
When creating a research plan, UX recommendation, or design improvement:
1. Clarify the decision and desired outcome.
2. Identify users, contexts, and constraints.
3. List assumptions and unknowns.
4. Choose the smallest research or testing method that can reduce risk.
5. Create the guide, script, task list, or analysis structure.
6. Include recruiting, consent, roles, and logistics.
7. Define evidence to collect and success criteria.
8. Include accessibility coverage.
9. Plan synthesis and reporting.
10. Recommend next actions and how to measure impact.
## Quality checklist
Before finalizing, verify:
- The product/design decision is explicit.
- The research question is answerable.
- The method matches the decision.
- Participant criteria are behavior/context-based.
- Questions are neutral and anchored in real experiences.
- Usability tasks are realistic and not step-by-step instructions.
- Consent, recording, privacy, incentives, and accessibility accommodations are addressed.
- Synthesis separates observations, interpretations, and recommendations.
- Recommendations are prioritized by impact, confidence, risk, and effort.
- Business value and user value are both represented.
- Frontend implications are practical and accessible.
- Limitations and confidence are stated.
Use the full checklists in [references/checklists.md](references/checklists.md).
## Common mistakes to avoid
- Asking users what they want and treating the answer as a roadmap.
- Asking whether users like a design.
- Starting with a feature idea instead of a decision or opportunity.
- Using research to validate a foregone conclusion.
- Recruiting only easy internal participants.
- Confusing demographics with behavior.
- Running usability tests without realistic tasks.
- Helping participants during the test.
- Producing long reports that arrive too late to affect decisions.
- Treating analytics as self-explanatory.
- Treating interviews as statistically representative.
- Optimizing the current UI into a local maximum while ignoring larger opportunities.
- Forgetting accessibility until after design or implementation.
See [references/anti-patterns.md](references/anti-patterns.md) for the full anti-pattern list.
- Over-asking the user for context when a safe default is available.
## How to explain recommendations to the user
Use this structure:
1. **Recommendation:** State the action clearly.
2. **Why:** Tie it to user behavior and business outcome.
3. **Evidence:** Identify the source and confidence level.
4. **Tradeoff:** Explain what is gained and what risk remains.
5. **Next step:** Propose the smallest test, implementation step, or metric.
Example:
> Recommend testing the checkout error flow before redesigning the entire checkout. The likely risk is not visual polish but task recovery: users need to understand what failed, how to fix it, and whether their payment was charged. Start with five task-based sessions on the current flow and prototype the top two fixes. Measure completion rate, error recovery, and support contacts after launch.