references/code-analyzer-violations.md
# Code Analyzer Violations
For each Code Analyzer violation, always include:
- The rule name translated to plain English
- The exact line number
- The fix direction (without writing code)
## Rule-name translations (examples)
| Rule name | Plain-language meaning | Fix direction |
|---|---|---|
| `ApexCRUDViolation` | A SOQL query or DML was made without checking object-level permissions first | Add a `Schema.sObjectType.<Object>.isAccessible()` (or `isCreateable`/`isUpdateable`) check before the operation |
| `ApexSharingViolations` | A class that performs data access does not declare sharing enforcement | Add `with sharing` (or an explicit `without sharing` with justification) to the class declaration |
| `ApexDangerousMethods` | A dangerous or disallowed method was used | Replace with the safe, supported alternative |
| `EmptyCatchBlock` | An exception is being swallowed silently | Log or handle the exception rather than leaving the catch block empty |
When a rule isn't in this table, translate it from its name and message into a plain-language description of what was violated, then give the fix direction.
## Plain-language rule
Never paste raw stack traces, JSON payloads, or internal Salesforce error codes into the output. Always translate to file name, method, line, and plain description.
## Fix location for violations
Code Analyzer violations almost always indicate a **production-code** fix — the static-analysis rule is flagging the code under test, not the test itself. Flag these as **Fix location: Production code** and track them separately from test-quality issues.
references/failure-categories.md
# Failure Categories & Improvement Mapping
## Inputs required per failed test
- Test method name
- Failure message (the assertion error or exception text)
- Failure category (assertion failure, unhandled exception, timeout, compile error)
## Category table
| Category | Description |
|---|---|
| **Assertion failure** | A test assertion failed (expected vs actual mismatch) |
| **Exception** | An unhandled exception was thrown |
| **Code Analyzer violation** | A static analysis rule was violated (e.g. `ApexCRUDViolation`, `ApexSharingViolations`) |
| **Timeout** | Test exceeded execution time limit |
| **Compile error** | Class failed to compile |
## Per-failure extraction (Part 1)
For each failure, extract and translate to plain language:
- Offending file and class name
- Method name
- Line number
- What rule or assertion was violated, in plain language
- Suggested fix direction (without writing code)
Group failures by category if more than one.
## Failure-pattern → improvement suggestion (Part 2)
Reason over the failure message to identify the root-cause pattern:
| Failure pattern | Improvement suggestion |
|---|---|
| `NullPointerException` | The test is not handling null input — add a null check or a test setup that ensures the data exists |
| `Assertion failed: expected X but was Y` | The expected value in the assertion is wrong or the test data setup does not produce the right state |
| `List has no rows for assignment` | The test is querying for data that doesn't exist — test setup is incomplete |
| `System.LimitException: Too many SOQL queries` | The test is hitting governor limits — the code under test or test setup is making too many queries |
| `Insufficient access rights on cross-reference id` | The test user lacks permissions — run as a user with the appropriate profile/permission set |
| `DML currently not allowed` | The test is performing DML inside a method called from a context that doesn't allow it |
| Code Analyzer violation message | The production code violates a specific rule — the test exposed it, but the fix is in production code, not the test |
## Producing actionable suggestions
For each failure describe, in plain language:
- What the failure reveals about what the test is not handling
- What specifically should be added or changed to make the test robust
- Whether the fix is in the **test** (assertion, setup, permissions) or in **production code** (test is correct, code under test is broken)
Do not rewrite the test — only describe what needs to change and why.
## Test fix vs. production-code fix
- **Fix location: Production code** — a code defect exposed by a sound test. Should NOT block suite promotion on test-quality grounds; track separately as a production defect.
- **Fix location: Test** — the test needs hardening: missing setup, wrong assertions, inadequate coverage of edge cases (null inputs, bulk record volumes, mixed permission contexts, governor-limit boundaries).
## Output formats
**Part 1 — failure summary:**
```text
Test failure summary:
<N> failure(s) found:
1. [<Category>] `<ClassName>.cls` — `<methodName>()` at line <N>
What happened: <plain-language description>
Rule violated: <ruleName or assertion description>
Fix direction: <plain-language suggestion>
```
**Part 2 — improvement suggestions:**
```text
Test improvement suggestions based on execution results:
`<testMethodName>()` — [Assertion Failure / Exception / etc.]
Failure: "<failure message>"
What this reveals: <plain-language explanation>
Suggestion: <specific, actionable recommendation>
Fix location: Test | Production code
Overall: <N> improvement(s) across <M> failed test(s).
```
references/prerequisite-checks.md
# Prerequisite Checks — Shared DevOps Center Gate
Shared environment gate for every DevOps Center testing skill. Run these checks **before** any query or system call. On any failure, surface the plain-language message and stop until the user resolves it — never proceed to a write with an unverified environment.
> **API version:** All DevOps testing system calls target Salesforce API **v67.0** (minimum required).
**Important:** All DevOps Center data (pipelines, stages, test suites, executions) lives in the Salesforce org — NOT in the local repository. Never search the filesystem for pipeline configuration. Always query the org using `sf data query` or `sf api request rest`.
**Object model — use the STANDARD objects, never the `sf_devops__` managed package:** DevOps Center testing data lives in standard platform objects — `DevopsPipeline`, `DevopsPipelineStage`, `DevopsPipelineStageTrigger`, `DevopsTestSuite`, `DevopsTestSuiteStage`, `DevopsTestSuiteExecution`, `DevopsProject`, `WorkItem`. Do **NOT** query the legacy managed-package objects (`sf_devops__Pipeline__c`, `sf_devops__Pipeline_Stage__c`, etc.) and do **NOT** gate on a `PackageLicense` / namespace check for `sf_devops`. "DevOps Center installed" is determined **solely** by whether `DevopsPipeline` is queryable and returns records (Prerequisite 4) — never by the presence of the `sf_devops__` namespace. If a `sf_devops__*` object is missing, that is expected and is NOT evidence that DevOps Center is uninstalled.
## How skills use this
Run Prerequisites 1–4 in order. Prerequisite 5 (stage) is run **only** when the operation targets a specific pipeline stage (configuring a gate, running/retriggering a suite, mapping a suite). Carry forward the resolved `doce-org-alias`, `pipelineId`, and (when applicable) `stageId`.
Resolve the **DevOps Center org alias** without asking the user unless genuinely ambiguous:
1. If the user named an org alias in their message, use it.
2. Otherwise, use the default org (`sf org display --json`, no `--target-org`).
3. Only if the default org has no `DevopsPipeline` records (Prereq 4 fails), ask: "Which org alias is your DevOps Center org?"
---
## Prerequisite 1 — Salesforce org: active login
```bash
sf org list --json
```
Look for at least one entry in `result.nonScratchOrgs`, `result.scratchOrgs`, or `result.sandboxes` with `"connectedStatus": "Connected"`.
- **Pass:** at least one org is Connected
- **Fail:** no orgs listed, or all show a non-connected status
**On fail:** "No authenticated Salesforce org found. Run `sf org login web --alias <your-alias>` in your terminal, then come back."
---
## Prerequisite 2 — Agentforce DX plugin installed
```bash
sf plugins --json
```
Look for a plugin entry whose `name` contains `plugin-agent`, `agentforce`, or `einstein` (case-insensitive).
- **Pass:** plugin found
- **Fail:** no matching entry
**On fail:** "The Agentforce DX Plugin is not installed. Run `sf plugins install @salesforce/plugin-agent`, then restart the IDE and try again."
---
## Prerequisite 3 — DevOps Center org authenticated
```bash
sf org display --target-org <doce-org-alias> --json
```
Check that `"connectedStatus"` is `"Connected"`.
- **Pass:** Connected
- **Fail:** expired session or error
**On fail:** "Your DevOps Center org session has expired. Run `sf org login web --alias <doce-org-alias>` to re-authenticate."
---
## Prerequisite 4 — Pipeline identified
```bash
sf data query \
--query "SELECT Id, Name, CreatedDate FROM DevopsPipeline ORDER BY Name ASC" \
--target-org <doce-org-alias> \
--json
```
- **Pass (exactly one):** use it automatically — do NOT ask.
- **Pass (multiple):** if the user already named a pipeline, match by name. Otherwise display a numbered list and ask:
```text
Found <N> pipelines:
1. <Name>
2. <Name>
Which pipeline would you like to work with?
```
- **Fail (no records):** "No DevOps Center pipeline found. Create a project and pipeline in DevOps Center before using the DevOps Testing Skills."
- **Fail (unsupported object):** "DevOps Center does not appear to be installed on `<doce-org-alias>`. Check that you're pointing at the correct org."
---
## Prerequisite 5 — Pipeline stage identified (conditional)
Run **only** when the operation targets a specific stage (e.g. configuring a quality gate).
If the user's message already names a stage (e.g. "Integration", "Staging", "Production"), use that name directly — do NOT ask again. Look up its Id:
```bash
sf data query \
--query "SELECT Id, Name FROM DevopsPipelineStage WHERE DevopsPipelineId = '<pipelineId>' AND Name = '<stageName>'" \
--target-org <doce-org-alias> --json
```
Only if no stage is mentioned, fetch the full list and ask which one:
```bash
sf data query \
--query "SELECT Id, Name FROM DevopsPipelineStage WHERE DevopsPipelineId = '<pipelineId>' ORDER BY Name ASC" \
--target-org <doce-org-alias> --json
```
Then ask: "Which pipeline stage are we working with?" — do NOT ask for an org alias; stages are resolved by name from the pipeline.
- **Pass:** stage Id and Name confirmed
- **Deferred:** not required until the operation needs it
references/work-item-creation.md
# Part 3 — Create a Fix Work Item
Creates a DevOps Center `WorkItem` to track a fix for a test failure or Code Analyzer violation. This is an **optional write**, triggered only when the user asks to create a fix work item, log a remediation, or assign a failure to a developer.
## Prerequisites
Run Prerequisites 1–4 (`references/prerequisite-checks.md`). You need `doce-org-alias`, a `DevopsProjectId` to file under, and an `OwnerId` (assignee). If no `DevopsProject` exists, surface that the work item cannot be created until a project exists — do NOT fabricate a project or work item.
## Inputs required before creating
| Input | How to obtain |
|---|---|
| `DevopsProjectId` | From the pipeline's associated project — query `DevopsProject WHERE Name = '<projectName>'` on the doce org if not already known |
| `Subject` | Derived from failure analysis — e.g. "Fix: Missing code-analyzer-v5.yml workflow in blitz-10-06 repository" |
| `OwnerId` | User ID of the developer to assign to — query `SELECT Id, Name FROM User WHERE Username = '<username>'` on the doce org if not known. Default to the requesting user when no assignee is specified; ask only if the username is unknown and no default applies. |
| `doce-org-alias` | Established in Prerequisites |
## Confirmation gate
Before creating the work item, show a summary and wait for explicit confirmation:
> "I'll create a fix work item with the following details:
> - **Subject:** `<subject>`
> - **Assigned to:** `<assigneeName>`
> - **Project:** `<projectName>`
>
> Shall I create it?"
Do not proceed until the user confirms.
## Creating the work item
```bash
sf data create record \
--sobject WorkItem \
--values "Subject='<subject>' DevopsProjectId='<DevopsProjectId>' OwnerId='<OwnerId>'" \
--target-org <doce-org-alias> \
--json
```
> **Important:** Use `WorkItem` (no namespace) — `DevopsWorkItem` is not a supported sObject in this org version.
## On success
Parse the returned `id` and confirm:
> "Fix work item created (`<id>`): `<subject>`. Assigned to `<assigneeName>` in the `<projectName>` project."
## Error handling
Never expose raw API error messages. Map errors to plain-language responses:
| Error | Response |
|---|---|
| `FIELD_INTEGRITY_EXCEPTION` | "The assignee ID is invalid. Let me look up the correct user ID — what's the developer's username?" |
| `REQUIRED_FIELD_MISSING` | "A required field is missing. Check that `Subject` and `DevopsProjectId` are both provided." |
| `INSUFFICIENT_ACCESS` | "Your user doesn't have permission to create work items in this project." |
| Any other error | "The work item could not be created. Error: `<plain summary>`. Try again or create it manually in DevOps Center." |
## No-project case
If the `DevopsProject` query returns 0 records, report clearly that no DevOps Center project exists and the work item cannot be created until one is set up. Do NOT fabricate a project name/ID, do NOT proceed to the confirmation gate or the create command.
SKILL.md
---
name: dx-devops-test-failures-analyze
description: "Analyzes DevOps Center test failures and Code Analyzer violations in plain language — failure category, offending file/class/method/line, rule violated, fix direction, and prioritized improvement suggestions (test-code vs production-code) — then optionally creates a tracked fix WorkItem on explicit request. Analysis is pure reasoning; work-item creation is a confirmation-gated write. Use this skill to explain failures or improvement suggestions, translate Code Analyzer violations, or track a fix as a work item. TRIGGER when: a run failed and the user wants root cause; a quality gate failure needs explaining; violations need translating; the user shares a failure payload and asks how to address it; wants to strengthen tests; or wants to create a fix work item, log a remediation, or assign a failure. DO NOT TRIGGER when: the user wants fix code written (use platform-apex-generate) or new test classes authored (use platform-apex-test-generate)."
metadata:
version: "1.0"
domains: ["Developer Experience"]
minApiVersion: "67.0"
relatedSkills:
- "dx-devops-test-suite-assignments-configure"
- "dx-devops-test-suite-run"
- "platform-apex-generate"
- "platform-apex-test-generate"
cliTools:
- tool: ["sf"]
semver: ">=2.67.0"
---
# Analyze DevOps Center Test Failures
Parses a test failure or Code Analyzer violation payload, explains it in plain language, produces prioritized improvement suggestions, and — only on explicit user request — creates a tracked fix work item. Parts 1–2 are pure reasoning (no writes); Part 3 is an optional, confirmation-gated write.
**Never expose raw JSON, stack traces, or internal Salesforce error codes to the user.** Always translate to file name, method, line, and plain description.
---
## Prerequisites
- **Parts 1–2 (analysis):** If the failure payload is already in context, no prerequisites are needed — this is pure reasoning. If you must fetch the payload yourself, run prerequisites (`references/prerequisite-checks.md`, Prereqs 1–4) and obtain the execution result via `dx-devops-test-suite-run` (its polling step).
- **Part 3 (work item):** Run Prerequisites 1–4. You also need a `DevopsProjectId` to file under and an `OwnerId` (assignee). See `references/work-item-creation.md`.
---
## Part 1 — Classify and explain each failure
Determine the failure category, then for each failure extract and translate to plain language: offending file/class, method, line number, the rule or assertion violated, and a fix direction (without writing code). Group failures by category if more than one.
| Category | Description |
|---|---|
| Assertion failure | A test assertion failed (expected vs actual mismatch) |
| Exception | An unhandled exception was thrown |
| Code Analyzer violation | A static-analysis rule was violated (e.g. `ApexCRUDViolation`) |
| Timeout | Test exceeded execution time limit |
| Compile error | Class failed to compile |
**Output format:**
```text
Test failure summary:
<N> failure(s) found:
1. [<Category>] `<ClassName>.cls` — `<methodName>()` at line <N>
What happened: <plain-language description>
Rule violated: <ruleName or assertion description>
Fix direction: <plain-language suggestion>
```
Full category/pattern tables and Code Analyzer rule translations: `references/failure-categories.md` and `references/code-analyzer-violations.md`.
**Empty / no-data case:** If the payload contains no failures or violations, report that clearly (e.g. "No failures found in the provided execution results.") and stop. Do NOT fabricate failures or suggestions.
---
## Part 2 — Improvement suggestions
Run this **after execution completes with failures**, not on static source. For each failed test, reason over the failure message (the primary signal) to identify what the test is not handling, then produce a specific, actionable suggestion and a **fix location** (Test vs Production code). The full failure-pattern → suggestion mapping is in `references/failure-categories.md`.
```text
Test improvement suggestions based on execution results:
`<testMethodName>()` — [Assertion Failure / Exception / etc.]
Failure: "<failure message>"
What this reveals: <plain-language explanation>
Suggestion: <specific, actionable recommendation>
Fix location: Test | Production code
Overall: <N> improvement(s) across <M> failed test(s).
```
Do not rewrite the test — only describe what needs to change and why. **Fix location: Production code** indicates a code defect exposed by a sound test (track separately, not a test-quality blocker). **Fix location: Test** indicates the test needs hardening (setup, assertions, edge cases).
---
## Part 3 — Create a fix work item (optional, on request only)
Trigger only when the user wants to create a fix work item, log a remediation, or assign a failure to a developer. This is a **write** operation with a mandatory confirmation gate. Follow `references/work-item-creation.md` for inputs, the subject/assignee/project confirmation gate, the `sf data create record --sobject WorkItem` call, and error handling.
> Use `WorkItem` (no namespace) — `DevopsWorkItem` is not a supported sObject in this org version.
If no `DevopsProject` exists in the org, report that the work item cannot be created until a project is set up — do NOT fabricate a project or proceed.
---
## Related skills
- **`dx-devops-test-suite-run`** — produces the failure payload (via its polling step) that feeds this skill.
- **`dx-devops-test-suite-assignments-configure`** — assign/strengthen the suites whose tests are failing.
- **`platform-apex-generate` / `platform-apex-test-generate`** — to actually write fix code or new test classes (out of scope here).