evals/evals.json
{
"skill_name": "coding-agent-docs-optimization",
"evals": [
{
"id": 1,
"prompt": "I run docs at Halyard, a payments infrastructure company. Two months ago we moved docs.halyard.io behind a new CDN and whoever did the migration left the managed bot-protection ruleset switched on. Since then our Discord has maybe fifteen people saying their AI coding tools \"can't find\" our docs, but the pages load fine for me and analytics traffic looks normal. Our CEO wants an AI-readiness score for the docs by Friday. Can you give me one and tell me what to fix on the pages?",
"expected_output": "A refusal to score before an access check and a cold run exist, followed by an empirical per-user-agent fetch protocol naming the CDN bot ruleset as the prime suspect, and framing crawler admission as a policy decision outside the docs team.",
"files": [],
"expectations": [
"Refuses to produce an AI-readiness score or a yes/no verdict before an access check and a cold run exist",
"Places the crawler access check before any page-content audit, stating content findings are meaningless behind a blocked fetch",
"Specifies fetching both a docs page and the index with each agent user-agent string of interest, recording the HTTP status code per user-agent",
"Specifies running those fetches from outside Halyard's own network",
"States the failure is silent: the audit sees a well-built site while the agent sees a challenge page",
"Distinguishes signalling (per-agent robots.txt rules) from enforcement (WAF, bot management, paid-crawl gate), noting signalling is honour-system and only edge controls actually stop a fetch",
"Names the CDN's default bot-protection setting as the prime suspect, i.e. a dashboard change rather than a documentation change",
"States that every surface scores zero until a fetch returns 200, so the audit cannot tell a good corpus from a bad one",
"Treats admitting a crawler as a policy decision with sign-off outside the docs team rather than a docs-team edit",
"Notes the asymmetry that the dashboard setting is reversible while the crawling that already happened is not",
"Does not accept \"pages load fine for me\" or \"analytics traffic looks normal\" as evidence that agents can fetch",
"Distinguishes training crawlers from search-and-fetch agents as separately nameable user-agents, so one can be admitted while the other is denied",
"Delivers the access check as a named report section listing which user-agents can fetch which surfaces with status codes, not as prose reassurance"
]
},
{
"id": 2,
"prompt": "My manager handed me a one-liner: \"Write an llms.txt for docs.brightloom.com and ship it this week.\" Brightloom is a product-analytics SDK company. We have 340 docs pages on a Next.js site, HTML only, no markdown versions of anything. We do serve an OpenAPI file at /openapi.json but it was hand-written about two years ago and roughly half the operation descriptions are empty strings. Can you draft the llms.txt?",
"expected_output": "An inventory-first response that produces the index but states outright it changes nothing measurable alone, ranks markdown endpoints and annotated spec above it, applies the index format rules, and requires advertising in three places.",
"files": [],
"expectations": [
"States plainly that publishing the index file alone changes nothing measurable until the surfaces it points at exist",
"Does not stop at emitting the file: performs or specifies a surface inventory first",
"Ranks markdown endpoints and annotated spec files above the index file on efficiency, and says the ranking out loud",
"Identifies that index entries pointing at HTML-only pages is the defect, since entries should point at markdown URLs so following a link needs no HTML stripping",
"Requires the index be generated from the navigation source at build time rather than hand-maintained",
"States that a hand-maintained index rots within about two releases",
"Specifies the index structure: a single H1 project name, a blockquote summary of what the project is, then H2 sections whose items are links optionally followed by a routing note",
"Notes that an H2 section literally titled \"Optional\" is reserved for links an agent may skip when working in a shorter context",
"Orders index sections by task frequency rather than by site hierarchy",
"Writes each entry note as a routing signal describing what the reader will find, not as marketing copy",
"Requires advertising the index in at least three places: the repository README, the docs navigation, and response headers or link relations",
"Names the empty description fields in /openapi.json as a finding, since an empty description is where invented parameters come from",
"Flags the hand-written spec as needing generation from the implementation rather than hand-editing"
]
},
{
"id": 3,
"prompt": "Corvid Data here, we sell a streaming data platform. Our VP of Engineering wants a number in the OKR doc: \"coding agents complete 95% of integration tasks from our docs on the first try by Q1.\" Separately I want to open the budget slide with the stat that AI agents are now 66% of documentation web traffic, to justify the headcount ask. Our task set is 9 integration jobs: 3 are basic REST CRUD, the other 6 are deep things like our streaming replay API, custom partitioner, and schema-registry failover. What number should I actually put in, and does the traffic stat hold up?",
"expected_output": "A rejection of the 95% target calibrated against published benchmark bands, a warning that 80% is a stretch goal and best-of-three not first-attempt, category scoring given the depth-heavy mix, and a refusal to present the traffic figure as an industry fact.",
"files": [],
"expectations": [
"Rejects 95% first-attempt unattended as a target and says so directly",
"Cites the published benchmark bands: backend-only roughly 73-85%, full-stack roughly 77-92%, depth-of-feature roughly 61-73%",
"States those published figures were scored best-of-three rather than first-attempt, so they are not directly comparable to an unattended single-attempt target",
"Presents 80% first-attempt as a stretch goal rather than a floor or minimum acceptable standard",
"Warns explicitly against letting a stakeholder hear the stretch goal as a minimum",
"Requires scoring by category rather than reporting one blended average across the 9 tasks",
"States that depth-of-feature tasks land roughly 15-25 points below breadth tasks for every model",
"Concludes that Corvid Data's 6-deep / 3-basic mix scores lower by construction and that this is not a regression",
"States that Corvid Data's own first baseline run is the only baseline that means anything, with iteration-over-iteration delta on a frozen set as the comparison that survives",
"Refuses to present the 66% agent-traffic figure as an industry or population statistic",
"Identifies that figure as self-reported by a single docs-platform vendor and measured across sites that had already invested in agent surfaces",
"Permits using it as motivation while stating it is not evidence",
"Requires recording the model identifier and the date alongside any score, since a score is comparable only within one generation of agents"
]
},
{
"id": 4,
"prompt": "Meridian Freight, logistics API. Here is my plan for testing our docs with an agent, tell me if it's sound. I'll open one long session and work through all eight tasks back to back so it keeps the context about our OAuth setup instead of re-learning it every time. I'll give it full web access so it can also check our GitHub issues and Stack Overflow. The first three tasks I wrote: (1) \"Call meridian.shipments.track() with a tracking number and print the status\", (2) \"Explain how our OAuth refresh flow works\", (3) \"Add webhook signature verification to an existing Express server.\" At the end I record pass/fail per task and report the percentage to the team. Anything I'm missing?",
"expected_output": "A rewrite of the protocol: fresh context per task, sources restricted to published surfaces, two of the three sample tasks rejected on shape, the frozen set preserved, and five metrics rather than a pass percentage.",
"files": [],
"expectations": [
"Rejects running all eight tasks in one session and requires a fresh context per task",
"States the reason: knowledge leaks between tasks and inflates every score after the first",
"Offers one sub-agent per task where parallel sub-agents are available, and separate sequential sessions otherwise",
"Rejects full web access and restricts sources to Meridian's own published agent-facing surfaces",
"States that if the agent needs something Meridian does not publish, that absence is the finding and the run should be allowed to fail",
"Rejects task 1 as a weak task because it hands the agent the exact method name it was supposed to discover",
"Rejects task 2 as ungradeable because it has no runnable or verifiable end state",
"Endorses the webhook verification task shape because it names an outcome and leaves the path open",
"Rejects a pass/fail percentage as the sole reported metric",
"Names invented-surface rate, corrective turns per task, context cost per task, and fetch-path length alongside first-attempt success",
"States that success rate alone hides a corpus that only works after several expensive retries",
"Requires freezing the task set, because a benchmark that changes between runs measures nothing",
"Requires capturing the full transcript, since the corrective turns are the data and the final file is only the receipt",
"Limits intervention to non-documentation obstacles such as a missing sandbox credential or a network block, and requires logging every intervention"
]
},
{
"id": 5,
"prompt": "Larkspur, HR and payroll API. We have a 140-operation OpenAPI 3.1 document that is generated from our Go handlers on every build. I've booked the whole next sprint for two technical writers to go through and write a rich multi-paragraph description on every single operation, on the theory that the description is what the model actually reads. Does that sound like the right use of the sprint? For context: about 90 of the 140 operations have no operationId at all, and most of our query parameters are plain `type: string` with no constraints.",
"expected_output": "A re-sequenced plan putting operationId first, value-space constraints second, curation third, and the description prose fourth, with the reasoning for each position and the operationId naming rules.",
"files": [],
"expectations": [
"Re-sequences the work rather than endorsing descriptions-first",
"Puts operationId first and gives the reason: generators derive the agent-facing tool name from it",
"Describes the operationId work as near-zero effort and mechanical",
"States the auto-generated fallback produces names like `get_api_v1_users_id` that models handle badly",
"Specifies verb-noun camelCase for operationId, with concrete examples such as `listCustomers`, `getUserById` or `createRefund`",
"Caps operationId at roughly 30 characters and drops articles and conjunctions such as \"a\", \"the\", \"and\", \"for\"",
"Requires operationId uniqueness and stability across releases, noting that a renamed operationId silently renames a tool",
"Recommends enforcing operationId uniqueness in CI with a spec linter",
"Puts closing the value space second, naming enums, explicit `required`, `default` values, `format` keywords, and `oneOf` with a discriminator",
"States that an unconstrained field is where an invented value goes, and that each constraint is a one-line edit removing a whole class of them",
"Ranks the summary/description split below those cheaper edits and states it costs on the order of a day or two on a real spec",
"Splits summary and description by job: summary is one imperative line for choosing the call",
"Assigns description the operational contract: preconditions, side effects, idempotency, required auth scope, rate limits, and a pointer to the sibling operation that might fit better",
"Places per-language runnable samples attached to the operation last in the default order, while promoting them to first for the two or three operations cold runs fail on repeatedly"
]
},
{
"id": 6,
"prompt": "Quillon, we sell a deployment platform. We want agents to be able to call our API directly. The plan is to run our whole OpenAPI file, 162 operations, through an openapi-to-mcp converter, host the resulting MCP server on our own infra behind an API key header, and point customers at it. Can you build the conversion and set up the server for us? For reference our operationIds look like `GetV2ProjectsProjectIdDeploymentsDeploymentIdLogs`.",
"expected_output": "A refusal to build or operate the server, a handoff for that part, and spec curation work done in scope: cut the exposed surface well under 162, fix the operationId shape, and cover generator naming behaviour.",
"files": [],
"expectations": [
"Declines to build or operate the MCP server itself, naming that as product work rather than documentation work",
"Hands the server build and hosting off to the platform-side skill rather than attempting it",
"Keeps curating the spec before conversion in scope and performs that work",
"Rejects converting all 162 operations wholesale into an exposed tool surface",
"Cites the guidance of fewer than 20 functions available at the start of a turn",
"Recommends tag filtering or vendor extensions to expose only what agents need past roughly 20-30 operations",
"States that curated tool surfaces outperform auto-converted OpenAPI servers, especially for complex APIs with many endpoints",
"Identifies `GetV2ProjectsProjectIdDeploymentsDeploymentIdLogs` as too long and not verb-noun, and proposes a shorter verb-noun replacement",
"Notes the converter name-resolution order: `x-mcp-tool-name`, then `operationId`, then a `{method}_{path}` fallback",
"Notes that a tool whose name fails the `^[a-zA-Z0-9_-]{1,64}$` pattern is discarded",
"Notes operationId truncation at the first `__` and a 56-character cap in at least one named generator",
"Recommends `x-mcp-tool-name` or an equivalent vendor extension when the SDK wants a terse method name and the agent wants a descriptive one, rather than compromising on a single string",
"Warns that renaming an operationId silently renames the exposed tool for existing consumers"
]
},
{
"id": 7,
"prompt": "I maintain ferroquant, a Rust crate with about 4,100 stars, in my evenings. Documentation is docs.rs plus a hand-written book published on GitHub Pages. I don't own a CDN account, I can't set response headers, and there is no docs team or build pipeline I control beyond a CI job that runs mdbook. Three people have now opened issues saying ChatGPT wrote them code calling `ferroquant::Engine::new_with_config`, which we removed back in 0.9. What do I actually do here?",
"expected_output": "A plan led by repo-root and source-level artefacts, explicitly ruling out header and CDN surfaces, plus in-page deprecation labelling at the top of the page and Rust-native doc comments with doctests.",
"files": [],
"expectations": [
"Leads with repo-root artefacts rather than response headers or CDN controls",
"Explicitly rules out the surfaces requiring header or CDN control given GitHub Pages, rather than recommending them and leaving the maintainer to discover they are impossible",
"Recommends generated spec or published type artefacts and doc comments in the SDK source as the reachable surfaces",
"Recommends a versioned index committed alongside releases rather than one generated by a hosted docs build",
"Recommends encoding the page rules in the contribution guide so incoming pull requests do not undo them",
"States that the cold-run protocol, the task set, and the page rules are identical regardless of ownership model, and does not re-derive them for the open-source case",
"Addresses the removed `new_with_config` API with an in-page deprecation label naming the replacement call",
"Places that label at the top of the page rather than in a footnote, because an agent that got its answer in the first paragraph never reaches the footnote",
"States that models carry confident memories of removed APIs and that only an in-page label overrides that prior",
"Recommends Rust `///` doc comments as the language's native convention rather than a generic prose format",
"Recommends doctests, since a doc-comment example the toolchain compiles and runs cannot rot without failing the build",
"Recommends keeping previous major versions reachable at distinct URLs rather than overwriting them"
]
},
{
"id": 8,
"prompt": "Osprey Analytics. A big partner integration goes live on the 14th of next month and Northwind's engineering team starts wiring our SDK into their stack in nine days. Their engineers use Cursor for basically everything. I'm the only technical writer here, I get zero engineering time this quarter, and our docs build is owned by the web team, who currently have a six-week ticket queue. Give me the plan.",
"expected_output": "A plan re-ranked against the deadline and the effort ceiling, naming which constraint moved which surface, deleting the surfaces the constraints rule out rather than demoting them, and recording the deletions.",
"files": [],
"expectations": [
"Produces a plan re-ranked against Osprey's constraints rather than reciting the default surface order",
"Names explicitly which constraint moved which surface",
"Drops every surface needing a docs build change, given the nine-day deadline against the web team's six-week queue",
"Drops every surface needing another team's release cycle",
"Deletes SDK doc comments from the plan outright rather than parking them at the bottom, because the writer has no engineering time",
"Deletes terminal and package-manager paths from the plan on the same grounds",
"Records which surfaces were deleted and why, rather than leaving them silently absent",
"States the reason for deletion rather than demotion: a ruled-out surface parked at the bottom of a plan silently reappears as scope",
"Runs the crawler access check first regardless of deadline pressure",
"Builds the frozen task set around the specific integration jobs Northwind's team will actually perform",
"Keeps the deliverable an audit report and names which report sections will be empty and why, given the reduced scope",
"States that this plan buys one release's worth of fixes rather than a standing guarantee, and names what a standing mandate would have promoted instead",
"Prioritizes page-level edits reachable through a content commit, since those ship without the web team"
]
},
{
"id": 9,
"prompt": "Read through our docs myself over two days and found 12 problems at Thistlewood (we sell a billing API). Here they are ranked by how fast I can fix each one: (1) inconsistent code fence languages on the CLI pages, (2) our Concepts page uses `sdk` as the variable name while every other page uses `client`, (3) the changelog page is three releases out of date, (4) the billing-webhook page's only example is a fragment with no imports and no client construction, (5) three pages still say v2 in the title, (6) our errors page lists error names but not the exact strings we emit, (7) the enterprise SSO setup page is dashboard clicks only, (8) two broken links in the footer, (9) the pagination page explains cursor design for four paragraphs before showing a call, (10) the glossary isn't alphabetized, (11) there's no Java quickstart, (12) the team page photos are stale. I also ran a cold run over 6 tasks and 4 failed: billing webhook, pagination, enterprise SSO, and rate-limit retry. Which do I do first?",
"expected_output": "A findings list admitted by task impact rather than ranked by fix speed, with the cosmetic items excluded outright, the admitted ones ordered by fixes-per-hour, concrete rewrites, and the unmatched failing task flagged.",
"files": [],
"expectations": [
"Admits findings by task impact first: only findings touching a task that actually failed enter the list",
"Excludes items 1, 3, 8, 10 and 12 from the findings list entirely rather than merely ranking them last",
"States the exclusion rule explicitly: a finding on a task no cold run failed does not make the list at all, however easy it would be to fix",
"Admits the billing-webhook fragment (4), the pagination ordering (9), and the dashboard-clicks SSO page (7) because each maps to a failed task",
"Orders the admitted findings by fixes-per-hour rather than by the user's speed-of-fix ranking",
"States that a page breaking a top revenue-driving task outranks ten cosmetic fixes because its value is higher, not because effort stopped counting",
"Fixes item 4 with a complete snippet carrying imports, client construction, the call, the awaited result, and the error branch",
"Fixes item 9 by reordering the page to the working call, then what it returns, then the parameters, then the rationale",
"Fixes item 7 by adding the equivalent command or API call while keeping the dashboard click path as the human alternative",
"Treats item 2 as a naming-discipline issue (one identifier per concept corpus-wide) and admits or excludes it on whether it touched a failed task",
"Fixes item 6 by mapping the exact error text that reaches stderr to a cause, a fix, and a verification step",
"Flags that the rate-limit retry task failed but has no corresponding entry in the user's list, so a finding still needs to be written for it",
"Writes each admitted finding naming the failing task, the page or spec field involved, and what the agent did instead"
]
},
{
"id": 10,
"prompt": "Vantage Grid, industrial IoT platform. Our docs are about 1,400 pages spread across four product lines (Edge, Fleet, Historian, Studio). Someone on the team generated an llms-full.txt over the weekend and it came out at 6.2 MB. Our llms.txt is 180,000 characters. I want to ship both this week and link them from the docs footer so people can find them. Good plan?",
"expected_output": "A refusal to ship the bundle, with deletion or per-product-line splitting rather than backlogging it, the oversized index split into child indexes, and advertising moved beyond the footer to three surfaces.",
"files": [],
"expectations": [
"Refuses to ship the 6.2 MB full-corpus bundle as it stands",
"States that a truncated bundle is worse than no bundle, because the agent believes it read everything",
"States that past the working context budget the bundle crowds out the user's own code",
"Recommends deleting the bundle outright, or splitting it per product line, rather than ranking it last or moving it to a backlog",
"States the reason for deleting rather than deferring: a bundle parked at the bottom of a plan is a defect waiting for someone with spare time",
"Flags the 180,000-character index as past the split threshold of roughly 100,000 characters",
"Recommends splitting the index into child indexes and linking those, so an agent can descend instead of truncating",
"Rejects the docs footer as sufficient advertising and requires the README, the docs navigation, and response headers or link relations",
"Notes that an agent typically fetches these surfaces because a human, a rules file, or a tool pointed at them, not through passive discovery",
"Ranks markdown endpoints above both the index file and the full-corpus bundle on efficiency",
"Notes the index and the full-corpus bundle tie on effort because both generate from the navigation source in one build step, while tying on no other axis",
"States that the index changes nothing measurable until the surfaces it points at exist"
]
},
{
"id": 11,
"prompt": "Kestrel Streams, we do managed event streaming. We ship a first-party TypeScript SDK with full .d.ts type declarations published to npm. I ran a baseline cold run over 7 tasks and the interesting part is that in 5 of the 7 the agent never fetched a single docs page - it read our types straight out of node_modules and worked from those, then got the consumer-group option wrong because the JSDoc on `subscribe()` is a single line saying \"Subscribes to a topic.\" We also publish an AsyncAPI 3.0 document for our event payloads that nobody has ever annotated. Where should the effort go?",
"expected_output": "SDK doc comments promoted above every other surface with the promoting condition named, TypeScript-native conventions applied, and the AsyncAPI annotation offered with an explicit analogy caveat.",
"files": [],
"expectations": [
"Promotes SDK doc comments above every other surface for Kestrel rather than applying the default efficiency order",
"Names the promoting condition explicitly: the product ships typed SDKs and the cold runs show the agent reading local code instead of fetching pages",
"States that doc comments are otherwise what the default order starves, because every pull request can drift them so they never win on ratio",
"States that doc comments are the only surface reaching an agent working offline against installed types",
"Notes that web docs and SDK source are two different surfaces reaching two different agents",
"States that the signature and its adjacent doc comment are what an in-editor agent most reliably sees, while a reference page three hops away is invisible to it",
"Prescribes JSDoc/TSDoc as the native convention for TypeScript rather than a generic comment format",
"Requires fully typed signatures and descriptive names, not only added prose",
"Applies the OpenAPI-derived annotation rules to the AsyncAPI document while stating plainly that AsyncAPI guidance is inferred from the OpenAPI case by analogy rather than separately sourced",
"Presents the AsyncAPI recommendation with lower confidence than the OpenAPI ones rather than with equal confidence",
"Warns that `$comment` is non-rendered and intended for schema maintainers, so anything an agent must see belongs in `description` or `examples`",
"Notes that terminal and package-local paths tie with doc comments on both effort and value, since each ships on the SDK release cycle and each reaches only the agent already inside the user's codebase",
"Requires re-running the identical frozen 7-task set and comparing corrective turns against the baseline rather than declaring the fix successful"
]
}
],
"trigger_queries": [
{ "query": "Our developer docs need to be AI-ready before the next release.", "should_trigger": true },
{ "query": "can a coding agent actually integrate our SDK from the docs alone?", "should_trigger": true },
{ "query": "write an llms.txt for docs.northwind.dev", "should_trigger": true },
{ "query": "what belongs in llms-full.txt and is it worth publishing?", "should_trigger": true },
{ "query": "users keep telling us Cursor hallucinates methods that don't exist in our Python SDK", "should_trigger": true },
{ "query": "I'd like an agent experience audit of our API documentation.", "should_trigger": true },
{ "query": "how do we improve AX on our developer platform?", "should_trigger": true },
{ "query": "should we be serving a markdown version of every docs page?", "should_trigger": true },
{ "query": "our OpenAPI file has no operationIds anywhere. does that actually matter?", "should_trigger": true },
{ "query": "Claude keeps inventing query parameters on our /v1/search endpoint", "should_trigger": true },
{ "query": "how would I measure whether an LLM can wire up our API without a human helping?", "should_trigger": true },
{ "query": "set up a benchmark where a fresh model integrates our product using only our published docs", "should_trigger": true },
{ "query": "customers' AI assistants keep writing code against an SDK version we removed two years ago", "should_trigger": true },
{ "query": "documentation for coding agents", "should_trigger": true },
{ "query": "make our documentation machine-readable", "should_trigger": true },
{ "query": "is publishing an llms.txt file actually worth the effort?", "should_trigger": true },
{ "query": "I think Cloudflare is blocking AI crawlers on our docs subdomain, how do I confirm that?", "should_trigger": true },
{ "query": "which AI user-agents should our docs site allow through?", "should_trigger": true },
{ "query": "why do agents only ever get our auth setup half right?", "should_trigger": true },
{ "query": "we want to start measuring first-attempt integration success", "should_trigger": true },
{ "query": "our snippets are fine for humans but agents paste them and they crash immediately", "should_trigger": true },
{ "query": "should we annotate our OpenAPI descriptions specifically for LLMs?", "should_trigger": true },
{ "query": "give me an agent-readiness score for our docs site", "should_trigger": true },
{ "query": "how are other companies making their docs work well with Copilot?", "should_trigger": true },
{ "query": "I want ours to be the SDK the model picks and then gets right", "should_trigger": true },
{ "query": "put together something so every release keeps our docs agent-readable, not just this one", "should_trigger": true },
{ "query": "we shipped llms.txt six weeks ago and nothing measurable changed", "should_trigger": true },
{ "query": "our support queue is full of tickets where an AI wrote the integration wrong", "should_trigger": true },
{ "query": "audit our docs from the perspective of a bot rather than a person", "should_trigger": true },
{ "query": "highest-leverage thing we can do so models stop guessing our API surface?", "should_trigger": true },
{ "query": "should we add x-codeSamples to our spec?", "should_trigger": true },
{ "query": "does it matter that our getting-started page tells you to click through the dashboard?", "should_trigger": true },
{ "query": "our docs are entirely client-rendered React. can an agent even read them?", "should_trigger": true },
{ "query": "how should we label deprecated endpoints so models stop recommending them?", "should_trigger": true },
{ "query": "we need a repeatable harness that proves the docs work before each release ships", "should_trigger": true },
{ "query": "review our developer docs for anything an unattended integration would trip over", "should_trigger": true },
{ "query": "how many pages should an agent have to read to finish a routine task?", "should_trigger": true },
{ "query": "what should our spec's description fields say that they currently don't?", "should_trigger": true },
{ "query": "our SDK is generated from the spec and the generated method names are awful", "should_trigger": true },
{ "query": "we're a volunteer-run open source project with no CDN. how do we do this?", "should_trigger": true },
{ "query": "should we ship the docs inside the npm package itself?", "should_trigger": true },
{ "query": "our CLI ought to be able to print the relevant doc page to stdout, right?", "should_trigger": true },
{ "query": "help me set a realistic target for unattended integration success", "should_trigger": true },
{ "query": "the CTO is asking whether 90% first-attempt is realistic for us", "should_trigger": true },
{ "query": "someone told me bots are two-thirds of docs traffic now. can I put that in the budget deck?", "should_trigger": true },
{ "query": "how do I prove the docs rewrite actually helped?", "should_trigger": true },
{ "query": "we rewrote 40 pages and the agent still fails the webhook task", "should_trigger": true },
{ "query": "our docs are good but the bot never actually sees them", "should_trigger": true },
{ "query": "make the docs work for the robot reading them, not the human", "should_trigger": true },
{ "query": "our onboarding is broken for anyone using an AI coding tool", "should_trigger": true },
{ "query": "what do we need to publish so an integration can self-serve end to end?", "should_trigger": true },
{ "query": "our webhook verification page has no runnable example and it shows", "should_trigger": true },
{ "query": "rank the work for me: markdown endpoints, llms.txt, doc comments, or spec descriptions?", "should_trigger": true },
{ "query": "our Rust crate's doc comments are thin. does that affect AI completions?", "should_trigger": true },
{ "query": "does an agent care about the JSON Schema files we publish?", "should_trigger": true },
{ "query": "we need a docs strategy for the era where the reader is a model", "should_trigger": true },
{ "query": "how do we stop agents inventing endpoint names that were never in our API?", "should_trigger": true },
{ "query": "score our documentation the way an agent would experience it", "should_trigger": true },
{ "query": "what goes wrong when a model reads our docs?", "should_trigger": true },
{ "query": "design a test where a fresh model tries to integrate us and we count the retries", "should_trigger": true },
{ "query": "a partner's integration was written entirely by an agent and took four days. how do we fix that?", "should_trigger": true },
{ "query": "which parts of our docs should we fix first for automated consumers?", "should_trigger": true },
{ "query": "our AsyncAPI file: same annotation rules as OpenAPI or not?", "should_trigger": true },
{ "query": "where exactly should we advertise our markdown endpoints?", "should_trigger": true },
{ "query": "our index file points at pages that have no markdown version", "should_trigger": true },
{ "query": "the agent read our docs and still called a function that doesn't exist", "should_trigger": true },
{ "query": "audit our docs structure against Diátaxis and tell me which pages are mixed-mode", "should_trigger": false },
{ "query": "our how-to guides and reference pages are all jumbled together in the nav", "should_trigger": false },
{ "query": "write a quickstart that gets a new developer to first success in five minutes", "should_trigger": false },
{ "query": "nobody finishes our getting-started page, they drop off at step four", "should_trigger": false },
{ "query": "write a tutorial that teaches Kafka consumers by building a small pipeline", "should_trigger": false },
{ "query": "our tutorial introduces too many concepts per step and learners get lost", "should_trigger": false },
{ "query": "set a policy for code samples in our docs and audit the existing corpus against it", "should_trigger": false },
{ "query": "our docs snippets stopped compiling after the v5 release", "should_trigger": false },
{ "query": "how do we test our documentation examples in CI?", "should_trigger": false },
{ "query": "turn our support tickets into proper troubleshooting pages", "should_trigger": false },
{ "query": "build an error-code catalog from our error telemetry", "should_trigger": false },
{ "query": "write the migration guide for our v3 to v4 breaking changes", "should_trigger": false },
{ "query": "what codemod coverage should we promise for the upgrade path?", "should_trigger": false },
{ "query": "our docs don't rank on Google for \"kafka consumer group lag\"", "should_trigger": false },
{ "query": "the v2 docs outrank our current version in search results", "should_trigger": false },
{ "query": "fix canonicals and hreflang across our versioned and translated docs", "should_trigger": false },
{ "query": "how do we get ChatGPT to cite our blog posts in its answers?", "should_trigger": false },
{ "query": "we want to show up in Google AI Overviews for our category", "should_trigger": false },
{ "query": "improve our brand's share of voice in Perplexity answers", "should_trigger": false },
{ "query": "write an AGENTS.md so Claude Code knows how to run our test suite", "should_trigger": false },
{ "query": "our repo's contributor-facing agent instructions file is badly out of date", "should_trigger": false },
{ "query": "set up CLAUDE.md conventions across our monorepo packages", "should_trigger": false },
{ "query": "deploy and host an MCP server for us with OAuth and per-tenant rate limiting", "should_trigger": false },
{ "query": "which integration surface should we build first, GraphQL or gRPC?", "should_trigger": false },
{ "query": "should we offer a bulk data sharing surface to enterprise customers?", "should_trigger": false },
{ "query": "audit every endpoint in our API reference for missing response codes", "should_trigger": false },
{ "query": "our try-it console is broken on about half the endpoints", "should_trigger": false },
{ "query": "design the error response format for our public API", "should_trigger": false },
{ "query": "what should our rate limit policy be for the free tier?", "should_trigger": false },
{ "query": "how should we version our public API going forward?", "should_trigger": false },
{ "query": "design idempotency keys for our payments endpoint", "should_trigger": false },
{ "query": "how should we handle API key rotation for customers?", "should_trigger": false },
{ "query": "write our release notes for v4.2", "should_trigger": false },
{ "query": "our CHANGELOG.md is just raw commit messages, clean it up", "should_trigger": false },
{ "query": "optimize our GitHub README for first-time visitors", "should_trigger": false },
{ "query": "write an engineering blog post about our storage engine rewrite", "should_trigger": false },
{ "query": "find the keywords developers actually search for in our category", "should_trigger": false },
{ "query": "which devrel metrics should we report to the board this quarter?", "should_trigger": false },
{ "query": "plan our developer community launch on Discord", "should_trigger": false },
{ "query": "help me submit a CFP for KubeCon", "should_trigger": false },
{ "query": "our npm package needs a better publishing pipeline", "should_trigger": false },
{ "query": "pick a license for our new open-source project", "should_trigger": false },
{ "query": "evaluate our own agent's tool-calling accuracy on a benchmark", "should_trigger": false },
{ "query": "refactor our agent markdown files into a cleaner directory structure", "should_trigger": false },
{ "query": "build a RAG pipeline over our documentation to power a support bot", "should_trigger": false },
{ "query": "add a chat widget to our docs site so users can ask questions", "should_trigger": false },
{ "query": "our internal engineering wiki is a mess, help me reorganize it", "should_trigger": false },
{ "query": "write API documentation for the six endpoints we shipped last week", "should_trigger": false },
{ "query": "translate our documentation into Japanese and Korean", "should_trigger": false },
{ "query": "our docs site takes four seconds to load, make it faster", "should_trigger": false },
{ "query": "should we move our docs from Docusaurus to Mintlify?", "should_trigger": false },
{ "query": "improve screen reader accessibility on our documentation site", "should_trigger": false },
{ "query": "our robots.txt is accidentally blocking Googlebot from the docs subdirectory", "should_trigger": false },
{ "query": "add structured data markup so our pages get rich results in search", "should_trigger": false },
{ "query": "design the information architecture for our new marketing site", "should_trigger": false },
{ "query": "our OpenAPI document doesn't validate against the 3.1 schema", "should_trigger": false },
{ "query": "generate TypeScript types from our OpenAPI spec", "should_trigger": false },
{ "query": "our SDK release process is manual and error-prone, automate it", "should_trigger": false },
{ "query": "write a customer case study about how Acme integrated us", "should_trigger": false },
{ "query": "how do we measure time to first hello world for human developers?", "should_trigger": false },
{ "query": "our developer survey says docs are the top complaint. what now?", "should_trigger": false },
{ "query": "segment our developer audience for next quarter's campaign", "should_trigger": false },
{ "query": "plan the content calendar for our devrel team next quarter", "should_trigger": false },
{ "query": "review this pull request that adds a new API endpoint", "should_trigger": false },
{ "query": "our docs search returns irrelevant results, fix the search index", "should_trigger": false },
{ "query": "set up analytics so we can see which docs pages people actually read", "should_trigger": false }
]
}
SKILL.md
---
name: coding-agent-docs-optimization
description: Makes SDK, API or protocol documentation something a coding agent can integrate from unattended - crawler access, machine-readable entry points (llms.txt, markdown endpoints, OpenAPI and JSON Schema files), self-contained copy-paste-safe pages, and a measured first-attempt agent success rate. Use whenever the user mentions agent-readable or AI-ready docs, agent experience or AX, llms.txt or llms-full.txt, docs for coding agents, agents inventing API calls that do not exist, or asks whether an agent could wire up their SDK from the docs alone - even if they never say "agent". Not for AI-search citation (samber/developer-relations-skills@developer-docs-ai-visibility) or a repo's contributor agent instructions. For human docs architecture use samber/developer-relations-skills@developer-docs-structure-audit.
license: MIT
metadata:
author: Samuel Berthe
version: "1.0.3"
---
# Coding Agent Docs Optimization
You are a documentation engineer whose reader is a coding agent integrating someone else's SDK, API, or protocol. Make that integration succeed on the first attempt, unattended, and prove it with a repeatable run.
This is the documentation half of **agent experience (AX)** - the term Mathias Biilmann (Netlify) coined in January 2025 for "the holistic experience AI agents will have as the user of a product or platform". His three pillars:
- Simple access and permissions.
- Clean APIs.
- Machine-optimized documentation - this skill owns that one.
The first two are product and API design. Pointing at them as the real cause of a failure is a legitimate finding, not a deflection.
Biilmann again, on why this is a distinct discipline: "LLMs understand your product differently than human developers. And writing documentation for LLMs is not the same as for humans."
Treat the agent as a real user persona with fixed traits:
- No memory of the previous page.
- No eyes for screenshots.
- A hard context budget.
- A strong habit of inventing plausible API surface when the real one is unstated.
## Scope
Cover the vendor side: the surfaces a developer's agent reads while wiring your technology into their codebase.
- Docs pages.
- Machine-readable specs.
- SDK source-level documentation.
Leave out:
- Getting cited by AI search engines. That is a visibility problem with different metrics: samber/developer-relations-skills@developer-docs-ai-visibility.
- The project's own contributor-facing agent instructions - build, test, lint conventions for people hacking on the repo. That file has its own open specification and its own well-served tooling.
- Building a callable tool surface for agents (tool server, function-calling API). Operating the server is product work. Curating a spec down before conversion stays in scope.
- General documentation architecture and content-type placement (see References).
## Interview
Follow these interview rules:
- Ask one question at a time.
- Prefer multiple choice.
- Stop once you can name the task set.
Skip anything the conversation already answered.
**Memory: Store a few info about the current task or project context. When memory lives in a file, use `devrel-context.md`; if a different memory system is in use, rely on that instead. Separate task info in different sections. Remove finished tasks. Add a date to a task; no date for general project context. Some interview responses may differ between 2 tasks.**
1. What is being documented - client SDKs, a hosted API, a protocol or spec, a CLI, or several of these?
2. Is there a machine-readable spec (OpenAPI, AsyncAPI, GraphQL SDL, protobuf, published types), is it generated from the implementation, and is it served at a public URL?
3. Where do the docs live: a generated site, markdown in the repo, or both? Can you change the build output, the server headers, and the CDN's bot policy?
4. Which 5-10 integration jobs matter most? Name them the way a developer would phrase the request.
5. What already goes wrong: invented method names, stale versions, snippets that do not run, missing auth setup, silent breaking changes?
6. Who maintains this - a volunteer maintainer team merging contributor PRs, or a vendor with a docs team and CDN control?
7. Which languages must reach parity, and how is the product versioned?
8. Is there a date this has to be working by - a launch, a partner integration, a customer escalation - or is it ongoing quality work?
9. Do you want one release's worth of fixes, or the surfaces and the cold-run harness standing so every release stays agent-readable?
10. What is your ceiling - your own hours, a docs build change, engineering time inside the SDK release cycle, a bot-policy decision someone else signs off?
Re-rank the surface order below against 8-10, and name which answer moved which surface.
- A hard date drops every surface that needs a build change or another team's release cycle.
- A standing mandate promotes SDK doc comments and the cold-run harness, which pay back per release instead of once.
- A ceiling of your own hours deletes doc comments and terminal paths from the plan rather than parking them at the bottom.
## Workflow
1. Verify an agent can fetch the docs at all before auditing what they say. If you can make outbound HTTP requests, fetch a page and the index with each agent user-agent string you care about and record the status codes; otherwise have the docs owner run those requests and report them. A CDN that blocks AI crawlers by default fails every later step silently. See [./references/agent-surfaces.md](./references/agent-surfaces.md).
2. Run a baseline cold run. A fresh agent, no prior knowledge, sources restricted to the published surfaces, one run per task in the frozen set. Log every corrective turn. See [./references/cold-run-protocol.md](./references/cold-run-protocol.md).
3. Inventory what an agent can already reach: index file, markdown versions of pages, machine-readable specs, versioned entry point, SDK doc comments, terminal or package-manager paths. Mark each present, partial, or missing, then order the missing ones with Which surface to build first.
4. Let task impact decide what enters the findings list, then order that list by what each fix returns per hour spent. A finding on a task no cold run failed does not make the list at all, however easy it would be to fix; among the findings that do qualify, a page breaking the top revenue-driving task outranks ten cosmetic fixes because its value is higher, not because effort stopped counting.
5. Rewrite the pages the failed runs actually touched, applying the page rules below.
6. Annotate the spec where an invented parameter or a wrong call traces back to it. See [./references/spec-annotation-rules.md](./references/spec-annotation-rules.md).
7. Publish the missing entry points and advertise each one in the README, the docs navigation, and the response headers. An agent fetches an index because a human, a rules file, or a tool pointed at it - publishing without advertising changes nothing.
8. Re-run the identical task set and compare. Iterate until the exit criteria below hold.
If your environment has persistent memory, store the task set, the surface inventory, and the last scores, so the next run measures against the same baseline instead of a new one.
## Which surface to build first
Crawler access is not on this list. It is the gate: until a fetch returns 200 for the agents you care about, every surface below scores zero, and the audit cannot tell the difference.
Treat opening that gate as its own decision with its own review. Admitting or refusing a given crawler is a policy call about who may read and train on the corpus. It is usually signed off outside the docs team, and reversible from a dashboard in a way the crawling that already happened is not.
Everything after the gate is a real menu, and the axes disagree:
- effort, least first: `index file == full-corpus bundle > version labels > markdown endpoints > spec files annotated > SDK doc comments == terminal and package paths`
- value, most first: `spec files annotated > markdown endpoints > version labels > SDK doc comments > terminal and package paths > index file > full-corpus bundle`
- efficiency, do first: `markdown endpoints > spec files annotated > version labels > index file > SDK doc comments == terminal and package paths > full-corpus bundle`
The index file and the full-corpus bundle tie on effort. Both are generated from the navigation source in one build step, and neither requires a decision about content. They do not tie anywhere else.
The index's position is the whole point of ranking this menu:
- It is the cheapest surface.
- It is the one users ask for by name.
- It changes nothing measurable until the surfaces it points at exist.
Leading with it produces a tidy file above a corpus an agent still cannot use.
SDK doc comments and terminal or package-local access tie on both effort and value. Each ships on the SDK release cycle rather than the docs build. Each reaches only the agent already working inside the user's codebase, never the one fetching from the web.
Default to markdown endpoints and annotated spec files. Move down to version labels once a cold run stops failing on invented calls and starts failing on version mismatch, which is the usual second failure.
SDK doc comments are what this order starves. They are a standing job: every pull request can drift them, so they never win on ratio. They are also the only surface that reaches an agent working offline against installed types.
Promote them above everything when the product ships typed SDKs and the cold runs show the agent reading local code instead of fetching pages.
Delete the full-corpus bundle rather than ranking it last when the corpus does not fit a working context budget alongside the user's own code. A truncated bundle is worse than none, because the agent believes it read everything. A bundle parked at the bottom of a plan is a defect waiting for someone with spare time.
This order is a default, not a law: it shifts with who executes it, since ownership shapes exactly which rows are reachable at all.
Re-rank it against what you already know:
- A vendor whose docs build already emits markdown gets that rung free and should start at the spec.
- A project whose spec is generated from the implementation gets annotation far cheaper than one hand-maintaining it.
Delete every surface answer 10's ceiling rules out, and record which one you removed.
## Exit criteria
Hold the run open until all four hold. The first is calibrated against published data. The other three are this skill's own baselines: replace them the moment the team has data of its own.
- First-attempt task success reaches the agreed target, unattended. Calibrated: set the target from [./references/cold-run-protocol.md](./references/cold-run-protocol.md) § Calibrating the target rather than picking a round number - 80% first-attempt is a stretch goal there, not a floor.
- Zero invented API surface on the three highest-traffic tasks. Self-set, and a policy rather than a measurement: on a top task, an invented call reaches production.
- No more than three documents fetched for a routine task. Self-set: raise it if your corpus is legitimately layered, but raise it deliberately.
- Corrective turns per task lower than the previous iteration. Self-set: direction matters more than the absolute number.
Fix the documentation, never the prompt. A task that only passes with a hand-tuned prompt has not passed - a real developer will not write that prompt.
## Page rules
Apply these to every page a failed run touched.
All ten cost the same thing: one editing pass over a page you are already opening. Effort is flat across the list, so frequency alone decides the efficiency order. That is the one place in this skill where a single axis is honest rather than lazy - there is no cheap-versus-valuable trade to split out.
The order reflects how often each failure shows up in cold runs. It is a judgement call, not a measured ranking. Start at the top when time is short, and reorder against what your own runs keep failing on.
Full before/after pairs live in [./references/page-rewrite-patterns.md](./references/page-rewrite-patterns.md).
- Make each page standalone: restate the prerequisite instead of pointing back to an earlier section.
- Pin the package name and version range inside the snippet, not on a distant setup page.
- Replace UI click paths with the equivalent command or API call; keep the click path as the human alternative.
- Ship complete snippets: imports, client construction, the call, the awaited result, the error branch.
- State what the snippet returns or prints, so the agent can verify rather than assume.
- Label illustrative fragments as non-runnable - an unmarked fragment gets pasted verbatim.
- Use one name per concept across the whole corpus, and one placeholder convention.
- Map each error string the product emits to a cause and a fix, using the exact text that reaches stderr.
- Put the version the page describes in the page, machine-readably, and label deprecated surface at the top with its replacement.
- Put the working call first, the explanation after.
## Spec rules
A handful of spec fields decide whether an agent picks the right call and fills it correctly, and a spec written for SDK generation leaves most of them empty. Work them in the order below - ranked by wrong calls prevented per hour of annotation - and read [./references/spec-annotation-rules.md](./references/spec-annotation-rules.md) before editing:
1. Give every operation a stable, verb-noun `operationId`. Near-zero effort, mechanical, and generators derive the agent-facing tool name from it - the auto-generated fallback produces names models handle badly.
2. Close the value space - prefer these over free-form strings:
- Enums.
- Explicit `required`.
- `default` values.
- `format` keywords.
- `oneOf` with a discriminator.
An unconstrained field is where an invented value goes, and each constraint is a one-line edit that removes a whole class of them.
3. Curate the spec before converting it into tools, when a conversion is planned. Cheap, and skipped entirely otherwise. OpenAI's function-calling guidance: fewer than 20 functions available at the start of a turn - past that, filter by tag or vendor extension instead of exposing everything.
4. Split `summary` and `description` by job:
- `summary`: one imperative line for choosing the call.
- `description`: preconditions, side effects, idempotency, auth scope, rate limits, and a pointer to the sibling operation that might fit better.
This is per-operation prose, so it costs a day or two on a real spec, most of it checking each claim against the implementation, and lands below three cheaper edits despite being what an agent reads to choose at all.
5. Attach per-language runnable samples to the operation itself, so the contract and the call live in one fetch. This is what the order starves: a sample per language per operation, kept running, is a standing job that never wins on ratio. Promote it to first for the two or three operations your cold runs fail on repeatedly - there, one worked sample outperforms every description you could write.
One caveat before applying these beyond OpenAPI: guidance for AsyncAPI and bare JSON Schema is inferred from the OpenAPI case, not separately sourced. Say so when you apply it.
## Ownership shapes the plan
The failures are the same; the achievable fixes are not.
- Maintainer-run open-source project: you control repo files, not headers or a docs CDN. Lead with repo-root artefacts, generated spec files, doc comments in the SDK source, and a versioned index committed alongside releases. Encode the page rules in the contribution guide so PRs do not undo them.
- Commercial vendor with a docs platform: you control build output, headers, redirects, and the bot policy. Lead with the access check, markdown endpoints, and an index generated by the docs build, so the surfaces cannot drift from the pages.
- Both: the cold-run protocol, the task set, and the page rules are identical. Do not re-derive them per ownership model.
## Invocation examples
Typical requests, and what each one should produce:
- "Our users keep telling us their coding agent hallucinates methods that don't exist in our Python SDK." → Run the interview, build a task set around the surfaces being invented, cold-run it, and return the audit report. Expect the root cause in the spec's `description` fields or in prose-only helper documentation, not in the pages that look worst.
- "Write an llms.txt for docs.example.com." → Do not just emit the file. Inventory first, generate the index from the navigation source, advertise it in three places, and say plainly that the file alone changes nothing measurable.
- "Are our docs AI-ready?" → An agent-readiness audit: baseline scores, surface inventory table, ranked findings, fix plan. Refuse to answer with a yes or a score before a cold run exists.
- "Make our OpenAPI spec work better with MCP servers." → Apply the spec rules and the curation step. Building or hosting the server itself is out of scope; hand that off (see References).
The deliverable is always the audit report below, plus the page and spec edits it justifies. When only part of the workflow is wanted, say which sections of the report will be empty and why.
## Output shape
Produce one audit report, not a pile of page edits:
1. Task set and baseline scores, with the model and date recorded.
2. Access check: which agent user-agents can fetch which surfaces, with status codes.
3. Surface inventory table: surface, status, effort, what it unblocks.
4. Findings, admitted by task impact and ordered by fixes-per-hour, each naming the failing task, the page or spec field, and what the agent did instead.
5. Fix plan split into what ships this week and what needs a build change, naming which of the owner's answers to questions 8-10 moved which item and which surfaces were deleted from the plan outright.
6. Re-run scores against the baseline, plus what remains failing and why.
## Failure modes
- Auditing content on a site that blocks the agent at the edge. Every score is zero for a reason no page rewrite can fix. Check access first.
- Publishing an index file and stopping. Advertise it in the README, the docs navigation, and the server response, or it never gets fetched.
- Publishing a spec with empty `description` fields and calling the spec surface done. The agent still guesses - now inside a schema-valid call.
- Dumping the entire corpus into one giant context file. Past the agent's budget it crowds out the user's own code and gets truncated mid-page.
- Optimizing pages that no task in the set touches. Traffic and task impact, not tidiness, decide the queue.
- Restating the public specification in your own docs instead of linking it. It goes stale and doubles the maintenance surface.
- Grading a re-run on a task the same session already solved. Cached knowledge inflates the score; start clean every time.
- Treating a single model's behaviour as the population's. Vary the agent between runs where you can, and record which one produced each score.
- Quoting a vendor's own traffic statistic as an industry fact to justify the budget. The widely-shared figure that agents are now the majority of docs traffic is self-reported by one docs platform, measured across sites that already invested in agent surfaces. It is a motivation, not evidence.
## References
- `samber/developer-relations-skills@docs-code-sample-standards` - runnable-sample policy, language parity, and sample testing
- `samber/developer-relations-skills@developer-docs-structure-audit` - content-type placement and site information architecture
- `samber/developer-relations-skills@developer-troubleshooting-docs` - building the error-to-fix pages this skill assumes exist
- `samber/developer-platform-skills@agent-skill-offering` - see also: shipping an Agent Skill inside the SDK or CLI repo, versioned with each release
- `samber/developer-relations-skills@version-migration-guide` - the upgrade guides an agent needs when a major version lands
- `samber/developer-platform-skills@api-reference-quality` - endpoint-level reference completeness audited against the spec surface
- `samber/developer-platform-skills@agent-integration-surfaces` - see also: agent access policy beyond docs, agent identity and signup, agent usage measurement; other API design and agent tool-surface questions live in the same collection
See [./references/agent-surfaces.md](./references/agent-surfaces.md) for the access layer, each machine-readable surface, its format, and when it earns its keep.
See [./references/cold-run-protocol.md](./references/cold-run-protocol.md) for task set design, run rules, metric definitions, target calibration, and a worked finding entry.
See [./references/page-rewrite-patterns.md](./references/page-rewrite-patterns.md) for before/after page rewrites, including what not to do.
See [./references/spec-annotation-rules.md](./references/spec-annotation-rules.md) for the spec fields that decide tool naming, selection, and parameter validity, with a negative example.
See [./references/published-findings.md](./references/published-findings.md) for every figure and threshold this skill uses, its source, and which targets are self-set baselines.