pubmed-database

어떤 스킬인가요?

>-

설치 방법

프로젝트 폴더에서 아래 명령어를 실행하고, 설치할 에이전트를 선택하세요.

설치 명령을 직접 실행해야 적용됩니다. 지원 에이전트와 필요한 권한·라이선스는 제작자의 안내를 확인하세요.

스킬 원문 파일

# Advanced Biological Database Linking (ELink)

## 1. Core Concepts: Database vs. Linkname

A query requires both a `target_database` and a `linkname`:

*   Target Database: The destination repository (e.g., `gene`, `nuccore`,
    `pccompound`).
*   Linkname: The specific "semantic bridge" defined by NCBI. Linknames follow
    the naming convention: `dbfrom_db_subset`. One database pair can have
    multiple linknames representing different relationships.

### Common Database and Linkname Pairs

*   **pubmed**, `pubmed_pubmed_citedin`: **Forward Citations** — Find papers
    that cite the source paper.
*   **pubmed**, `pubmed_pubmed_refs`: **Backward Citations** — Extract the
    source paper's bibliography.
*   **pubmed**, `pubmed_pubmed`: **Similar Articles** — Papers sharing MeSH
    terms or keywords.
*   **pmc**, `pubmed_pmc`: **Full Text** — Resolve a PMID to a PMCID (required
    for BioC API).
*   **pccompound**, `pubmed_pccompound`: **Chemicals** — Find specific chemicals
    or drugs mentioned (CIDs).
*   **pcassay**, `pubmed_pcassay`: **PubChem BioAssays** — Link to experimental
    results and screening data.
*   **gene**, `pubmed_gene`: **Genetics** — Identify specific NCBI Gene records
    discussed.
*   **nuccore**, `pubmed_nuccore`: **Sequence Data** — Link to
    GenBank/nucleotide sequences.
*   **protein**, `pubmed_protein`: **Proteins** — Link to RefSeq or GenPept
    protein records.
*   **clinvar**, `pubmed_clinvar`: **Clinical Variants** — Find links to the
    ClinVar database (mutations).
*   **snp**, `pubmed_snp`: **SNPs** — Find specific Single Nucleotide
    Polymorphisms.
*   **sra**, `pubmed_sra`: **Raw Data** — Find raw datasets in the Sequence Read
    Archive.
*   **structure**, `pubmed_structure`: **3D Structures** — Find molecular
    structures (PDB) for proteins/ligands.

This list covers the most common `pubmed → X` links. For the full list of all
ELink linknames across all NCBI databases, see the
[NCBI Entrez Links catalog](https://eutils.ncbi.nlm.nih.gov/entrez/query/static/entrezlinks.html).

--------------------------------------------------------------------------------

## 2. Procedural Wisdom: Handling Failure Modes

### The "Indexing Lag" Problem (Recent Papers)

NCBI links are not created instantly. There is a human-in-the-loop and automated
indexing process that results in a **4-8 week delay** for cross-database links.

**Symptom**: `find_linked_biological_data` returns `[]` for a paper published 2
weeks ago.

**Strategy**: If the paper is very recent, **pivot immediately** to semantic
search or full-text extraction. Use `get_full_text_pmc` and search for primary
identifiers in the prose (e.g., by searching for specific identifiers or
nomenclature manually).

### The "High-Citation" Timeout

For foundational papers with >10,000 citations, `pubmed_pubmed_citedin` may fail
or timeout.

**Strategy**: Instead of linking, use `search_pubmed` with the title of the
paper in quotes or a specific query like `"citations for PMID [SOURCE_PMID]"`

### Verifying Open Access Availability

Before calling `get_full_text_pmc`, it is more reliable to check the link first

**Workflow**: Call `find_linked_biological_data` with `target_database="pmc"`
and `linkname="pubmed_pmc"`. If it returns a result, the paper is definitely in
the PMC BioC database. If not, don't waste time on a full-text fetch; use
`fetch_article_abstracts` instead.

--------------------------------------------------------------------------------

## 3. Category-Specific Tips

### Chemical Entities (`pccompound`, `pcassay`)

Links to chemical databases typically return **internal identifiers** (e.g.,
PubChem CIDs) rather than common names.

**Example**: A link to a drug study might return `["4091"]`.

**Note**: To resolve these to names, you must use a separate metadata lookup or
search strategy, as the linking tool only provides the relationship, not the
entity details.

### Sequence and Genomic Data (`nuccore`, `protein`, `gene`)

These links represent formal submissions to NCBI repositories (like GenBank).

**Strategy**: If a link search returns empty for recent research, search for the
paper's title or key findings in PubMed to find the abstract. Authors often
include primary identifiers in the text before the database cross-references are
finalized.

--------------------------------------------------------------------------------

## 4. Troubleshooting Empty Results

If `find_linked_biological_data` returns `[]`:

1.  **Check Date**: Is it a recent paper? (Indexing lag).
2.  **Check Scope**: Is the topic niche? (Authors may not have submitted data to
    NCBI).
3.  **Check Database Pair**: Ensure you are using the correct `linkname` for the
    `target_database`. Using `pubmed_gene` with `target_database="nuccore"` will
    fail.

--------------------------------------------------------------------------------

## 5. Advanced Features

*   **Reverse lookups**: `find_linked_biological_data` accepts a `dbfrom`
    parameter (default: `"pubmed"`). Set it to another database to traverse
    links in the opposite direction (e.g., `gene → pubmed`).
*   **Date filtering**: Pass `mindate` and `maxdate` (YYYY/MM/DD format) to
    filter linked results by publication date. Only works when `dbfrom` and
    `target_database` are both `"pubmed"`.
*   **Link discovery**: Use `discover_available_links` with a record ID to list
    all available linknames before calling `find_linked_biological_data`.

# PubMed Advanced Search Reference

This guide provides a comprehensive reference for advanced PubMed querying,
specifically for scientific research, high-precision filtering, and niche
metadata analysis.

## 1. Complete Field Qualifier Reference (Search Tags)

*   **[sh]**, MeSH Subheadings: Limits a MeSH term to a specific aspect. E.g.,
    `"Parkinson disease/genetics"[sh]`. Common: `/therapy`, `/toxicity`,
    `/drug effects`.
*   **[majr]**, MeSH Major Topic: Limits results to citations where the MeSH
    term is the **primary focus** of the article.
*   **[crdt]**, Create Date: The date the record was first added to PubMed. Best
    for finding "what's new" in the database.
*   **[edat]**, Entry Date: Used for sorting by "Most Recent." Set to
    publication date for older papers newly added.
*   **[sb]**, Subset: Pre-built high-quality filters. E.g., `systematic[sb]`,
    `"free full text"[sb]`, `cancer[sb]`.
*   **[nm]**, Supplementary Concept: For specific chemicals, drugs, or rare
    substances not yet in MeSH (formerly Substance Name).
*   **[ad]**, Affiliation: Search for authors at specific institutions, cities,
    or countries. E.g., `Stanford[ad]`.
*   **[auid]**, Author Identifier: Search by unique identifiers like ORCID.
    E.g., `0000-0002-1234-5678[auid]`.
*   **[la]**, Language: Restrict results by language. E.g., `eng[la]`.
*   **[pt]**, Publication Type: Filter by study design: `Meta-Analysis`,
    `Systematic Review`, `Randomized Controlled Trial`.
*   **[tw]**, Text Word: Includes Title, Abstract, MeSH, Subheadings, and other
    indexing fields. Very broad.
*   **[ta]**, Journal: Search by NLM Title Abbreviation or ISSN. E.g.,
    `Nature[ta]`.

## 2. Advanced Syntax & Logic

### Proximity Searching

Finds terms within a specific distance of each other. Available in `[ti]`,
`[tiab]`, and `[ad]`.

**Format:** `"term1 term2"[field:~N]`

**Example:** `"gut microbiome"[tiab:~2]` finds "gut" and "microbiome" with up
to 2 words between them.

### Boolean Nesting

PubMed processes logic from left to right. Use parentheses to control the order
of operations.

**Example:**

`(Parkinson OR "Dopamine deficiency") AND (Microbiome OR "Gut Flora")`

### Truncation

Use `*` at the end of a word to search for all terms that begin with that root.

**Example:** `patholog*` finds pathology, pathologist, pathological.

**Note:** Truncation turns off **Automatic Term Mapping (ATM)**.

## 3. Specialized Research Strategies

### Evidence-Based Medicine (EBM) Filters

To find the highest quality clinical evidence, append these filters:

-   **Systematic Reviews:** `AND systematic[sb]`
-   **Clinical Trials:** `AND "Clinical Trial"[pt]`
-   **Meta-Analyses:** `AND "Meta-Analysis"[pt]`

### The "Expert Query" Construction

For maximum coverage and precision, combine MeSH terms with Title/Abstract
keywords:

```
("Parkinson Disease"[mesh] OR "Parkinson's"[tiab])
    AND ("Gastrointestinal Microbiome"[mesh] OR "gut microbiome"[tiab:~2])
```

### Tracking Literature Updates

To find only the papers added to the database since your last visit (e.g., March
1st):

-   `Parkinson[tiab] AND 2026/03/01:2026/03/13[crdt]`

## 4. Troubleshooting & Nuances

-   **ATM (Automatic Term Mapping):** If you search `Parkinson`, PubMed
    automatically adds `"Parkinson Disease"[mesh]`. If you use quotes
    (`"Parkinson"`) or truncation (`Parkinson*`), ATM is disabled.
-   **Interchangeable Tags:** `[dp]` and `[pdat]` are identical. `[ta]` and
    `[journal]` are identical.
-   **Language Bias:** PubMed is predominantly English-language. Use `eng[la]`
    if the agent must perform text analysis on the results.

# Bulk Workflows

Argument specifications for `cache_results_history` and common multi-step batch
patterns for search, fetch, linking, and large-scale retrieval.

## `cache_results_history` — Batch-upload PMIDs for bulk retrieval

Uploads a batch of PMIDs to the NCBI History Server and returns a `webenv` and
`query_key` session handle. Use this when working with **more than ~10 PMIDs**
or when the same batch of IDs will be passed to multiple subsequent calls (e.g.
fetch abstracts *and* link to genes). For small batches (≤10 IDs), pass them
inline instead — it saves a round-trip. The returned handles can be passed to
`fetch_article_abstracts` or `find_linked_biological_data`.

```bash
uv run scripts/pubmed_api.py ./batch_session.json cache_results_history "35113657,31234568,29474920"
```

**Arguments:**

-   `pmids` (list[str], required) – comma-separated PMIDs to upload.

**Output:**

```json
{
  "webenv": "NCID_1_123456789_130.14.18.97_9001",
  "query_key": "1"
}
```

--------------------------------------------------------------------------------

## Workflow Recipes

### Search → batch fetch abstracts → summarize

```bash
uv run scripts/pubmed_api.py ./mrna_melanoma_results.json search_pubmed "mRNA vaccines melanoma" --max_results 5 --sort_by relevance
uv run scripts/pubmed_api.py ./mrna_melanoma_abstracts.json fetch_article_abstracts "PMID1,PMID2,PMID3,PMID4,PMID5"
```

### Search → full text (open-access only)

```bash
uv run scripts/pubmed_api.py ./psilocybin_depression_results.json search_pubmed "psilocybin depression" --max_results 5 --sort_by relevance
uv run scripts/pubmed_api.py ./psilocybin_depression_full_text.json get_full_text_pmc "PMID1"
```

If `get_full_text_pmc` returns an error, the paper is not open-access; use
`fetch_article_abstracts` instead.

### Fuzzy citation → fetch

```bash
uv run scripts/pubmed_api.py ./nature2006_pmids.json match_raw_citations "nature|2006||||takahashi k|key0|"
uv run scripts/pubmed_api.py ./abstracts.json fetch_article_abstracts "RESOLVED_PMID"
```

### Cross-database exploration

```bash
uv run scripts/pubmed_api.py ./35113657_links.json discover_available_links "35113657"
uv run scripts/pubmed_api.py ./35113657_gene_links.json find_linked_biological_data "35113657" gene pubmed_gene
uv run scripts/pubmed_api.py ./35113657_compound_links.json find_linked_biological_data "35113657" pccompound pubmed_pccompound
uv run scripts/pubmed_api.py ./35113657_nuccore_links.json find_linked_biological_data "35113657" nuccore pubmed_nuccore
uv run scripts/pubmed_api.py ./35113657_citing_papers.json find_linked_biological_data "35113657" pubmed pubmed_pubmed_citedin
```

### Bulk Retrieval and Data Slimming (N > 10)

For large result batches, avoid processing IDs iteratively. Chain
`cache_results_history` and **shell pipelines** to perform batch retrieval and
slimming in one turn.

```bash
# 1. Search and batch-upload to get a session handle
PMIDS=$(uv run scripts/pubmed_api.py ./s.json search_pubmed "CRISPR" --max_results 50 && cat ./s.json | jq -r 'join(",")')
uv run scripts/pubmed_api.py ./session.json cache_results_history "$PMIDS"
WEBENV=$(cat ./session.json | jq -r '.webenv')

# 2. Bulk fetch AND slim to only relevant fields in ONE turn
# Use brackets in jq [ ... ] to ensure the output is a valid JSON array.
uv run scripts/pubmed_api.py ./full.json fetch_article_abstracts "" --webenv "$WEBENV" --query_key 1
cat ./full.json | jq '[.[] | {pmid: .pmid, title: .title, abstract: .abstract}]' > ./slim.json
```

When using `--webenv`/`--query_key`, pass `""` for the `pmids` argument.

# Citation Matching Guide (ecitmatch)

The `match_raw_citations` tool uses NCBI's ecitmatch endpoint, which predates
modern APIs. This guide covers the pipe format, field selection strategies, and
what to do when matching fails.

--------------------------------------------------------------------------------

## 1. The Pipe Format

Each citation is a single string with 7 pipe-separated fields and a trailing
pipe:

```
journal|year|volume|first_page|author_name|key|
```

All fields except `journal` are optional — empty segments are valid. The `key`
is an arbitrary label you choose to track which citation matched which PMID in
the results.

**Examples:** `nature|2006||||takahashi k|ref1| proc natl acad
sci|2007|104|11760|takahashi k|ref2| cell|2020|183||doe j|ref3|`

Assemble the pipe-delimited string directly — all fields except `journal` are
optional, so leave unwanted segments empty.

--------------------------------------------------------------------------------

## 2. Which Fields Matter Most

Not all fields contribute equally. ecitmatch uses fuzzy matching, but some
combinations are far more reliable than others.

**High-value combinations (use these first):** - `journal` + `author` + `year` —
resolves most citations - `journal` + `volume` + `first_page` — uniquely
identifies articles even without author/year

**Low-value on their own:** - `year` alone — too broad - `first_page` alone —
page numbers repeat across volumes - `volume` alone — meaningless without
journal

**Strategy:** Start with whatever fields you have. If it misses, drop the most
uncertain field (often `first_page` or `volume`) and retry — ecitmatch sometimes
does worse with partly-wrong data than with missing data.

--------------------------------------------------------------------------------

## 3. Journal Name Pitfalls

The `journal` field is the most critical and the most error-prone.

**Abbreviation vs. full name:** ecitmatch accepts both, but they are not
interchangeable for all journals. When in doubt, use the NLM abbreviated form.

Common mappings: - "The New England Journal of Medicine" → `n engl j med` -
"Proceedings of the National Academy of Sciences" → `proc natl acad sci` - "The
Journal of Biological Chemistry" → `j biol chem` - "JAMA" → `jama` (already
abbreviated)

**Case:** ecitmatch is case-insensitive. `Nature` and `nature` are equivalent.

**Punctuation:** Strip periods from abbreviations. Use `j biol chem` not `J.
Biol. Chem.`

--------------------------------------------------------------------------------

## 4. Author Name Format

ecitmatch expects **last name followed by first initial**, lowercase, no
periods, no commas.

Source format      | ecitmatch format
:----------------- | :---------------
Takahashi, K.      | `takahashi k`
John A. Smith      | `smith j`
María García-López | `garcia-lopez m`
van der Berg, P.J. | `van der berg p`

Only provide the **first author**. ecitmatch ignores additional authors.

--------------------------------------------------------------------------------

## 5. Batching Multiple Citations

Pass multiple citations as a comma-separated list to `match_raw_citations`:

```bash
uv run scripts/pubmed_api.py /tmp/pubmed_results.json match_raw_citations \
  "nature|2006||||takahashi k|ref1|,cell|2020|183||doe j|ref2|"
```

The response returns PMIDs for matched citations only. Unmatched citations are
silently dropped. Match the `key` field in the response to track which citations
resolved.

--------------------------------------------------------------------------------

## 6. When ecitmatch Fails

ecitmatch has a ~70-80% hit rate on well-formed citations and drops sharply with
messy input. When it returns empty:

### Fallback 1: Search PubMed directly

Construct a query from the citation fields:

```bash
uv run scripts/pubmed_api.py /tmp/pubmed_results.json search_pubmed \
  '"Takahashi" AND "2006" AND "Nature"' 5
```

### Fallback 2: Title search

If you have the paper title (even partial):

```bash
uv run scripts/pubmed_api.py /tmp/pubmed_results.json search_pubmed \
  '"Induction of Pluripotent Stem Cells"[ti]' 5
```

### Fallback 3: Strip fields and retry

Remove the least-reliable field and resubmit:

```bash
uv run scripts/pubmed_api.py /tmp/pubmed_results.json match_raw_citations \
  "nature|2006||||takahashi k|ref1|"
uv run scripts/pubmed_api.py /tmp/pubmed_results.json match_raw_citations \
  "nature|||||takahashi k|ref1|"
```

### Fallback 4: DOI or known identifier

If the source has a DOI, skip ecitmatch entirely:

```bash
uv run scripts/pubmed_api.py /tmp/pubmed_results.json search_pubmed \
  "10.1016/j.cell.2006.07.024[doi]" 1
```

--------------------------------------------------------------------------------

## 7. Common Failure Patterns

-   **Returns empty for a known paper** — Wrong journal abbreviation. Try full
    name or NLM abbreviation.
-   **Returns wrong PMID** — Author name mismatch. Check last-name + initial
    format.
-   **Returns empty for recent paper** — Not yet indexed by ecitmatch. Use
    `search_pubmed` title search.
-   **Returns empty for old paper (pre-1966)** — Pre-MEDLINE era. These papers
    may lack structured metadata.
-   **Multiple citations, partial hits** — One citation has bad data. Check
    returned keys to isolate the miss.

@ARTICLE{Sayers2025,
  ABSTRACT = {The National Center for Biotechnology Information (NCBI) provides online information resources for biology, including the GenBank® nucleic acid sequence repository and the PubMed® repository of citations and abstracts published in life science journals. NCBI provides search and retrieval operations for most of these data from 31 distinct repositories and knowledgebases. The E-utilities serve as the programming interface for most of these. Resources receiving significant updates in the past year include PubMed, PubMed Central, Bookshelf, the NIH Comparative Genomics Resource, BLAST, Sequence Read Archive, Taxonomy, iCn3D, Conserved Domain Database, Pathogen Detection, antimicrobial resistance resources and PubChem. These resources can be accessed through the NCBI home page at https://www.ncbi.nlm.nih.gov.},
  AUTHOR = {Sayers, Eric W and Beck, Jeffrey and Bolton, Evan E and Brister, J Rodney and Chan, Jessica and Connor, Ryan and Feldgarden, Michael and Fine, Anna M and Funk, Kathryn and Hoffman, Jinna and Kannan, Sivakumar and Kelly, Christopher and Klimke, William and Kim, Sunghwan and Lathrop, Stacy and Marchler-Bauer, Aron and Murphy, Terence D and O’Sullivan, Chris and Schmieder, Erin and Skripchenko, Yuriy and Stine, Adam and Thibaud-Nissen, Francoise and Wang, Jiyao and Ye, Jian and Zellers, Erin and Schneider, Valerie A and Pruitt, Kim D},
  URL = {https://doi.org/10.1093/nar/gkae979},
  DATE = {2025-01},
  DOI = {10.1093/nar/gkae979},
  EPRINT = {https://academic.oup.com/nar/article-pdf/53/D1/D20/60586411/gkae979.pdf},
  ISSN = {1362-4962},
  JOURNALTITLE = {Nucleic Acids Research},
  NUMBER = {D1},
  PAGES = {D20--D29},
  TITLE = {Database resources of the National Center for Biotechnology Information in 2025},
  VOLUME = {53},
}

# Cross-Database Linking Functions

Detailed argument specifications, output schemas, and strategies for
`find_linked_biological_data` and `discover_available_links`.

## 1. `find_linked_biological_data` — Cross-database linking

Finds records in other NCBI databases linked to a source record. Use this to
identify genes, proteins, or chemicals discussed in a paper without parsing the
text. Common target databases: `gene`, `protein`, `nuccore`, `pccompound`,
`pubmed`. Supports reverse lookups (e.g., `gene → pubmed`) via the `dbfrom`
parameter. This is the required path for entity identification in large batches.

For a complete map of link types and advanced strategies, refer to

-   [Advanced Biological Database Linking](advanced-linking.md).
-   [NCBI ELink Reference](https://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ELink)

```bash
python3 scripts/pubmed_api.py ./gene_links.json find_linked_biological_data "35113657" gene pubmed_gene
```

**Arguments:**

-   `source_pmid` (str, required) – source record ID (PMID when dbfrom is
    pubmed)
-   `target_database` (str, required) – target NCBI database name
-   `linkname` (str, required) – elink link name
-   `dbfrom` (str, default "pubmed") – source database
-   `mindate` (str, default "") – minimum date filter (YYYY/MM/DD),
    pubmed→pubmed only
-   `maxdate` (str, default "") – maximum date filter (YYYY/MM/DD),
    pubmed→pubmed only
-   `webenv` (str, default "") – WebEnv from `cache_results_history`
-   `query_key` (str, default "") – query_key from `cache_results_history`

**Output:** `["123456", "789101"]` (target database record IDs)

Returns `[]` if no links exist. Use `fetch_database_summary` to resolve these
UIDs into accession numbers, gene names, or other metadata.

### Cross-Database Linking Strategy

Unless you are performing a standard citation traversal
(`pubmed_pubmed_citedin`) or gene lookup (`pubmed_gene`), follow the **Discover
→ Filter → Merge** workflow for complete results. **Preference**: When asked to
identify linked entities (genes, compounds, etc.), PREFER using
`find_linked_biological_data` as the primary, most efficient path, rather than
searching in external databases by name unless direct links are known to be
missing.

1.  **Discover**: Call `discover_available_links` to see which connections exist
    for the specific record.
2.  **Filter**: Identify all linkname entries that point to your target database
    (e.g., all links where `db` is `pccompound`).
3.  **Merge**: If multiple linknames point to the same database (e.g.,
    `pubmed_pccompound` and `pubmed_pccompound_mesh`), fetch from **all** of
    them and merge the results. Different linknames often represent different
    indexing methods (manual curation vs automated mapping), and using only one
    may result in missing data.

**Warning: Data Freshness.** Cross-database links (`elink`) typically lag behind
publication by weeks to months. For papers published in the last year, expect
`find_linked_biological_data` to return `[]` and pivot to searching for
accession numbers or CIDs directly in the abstract text. Does not apply to
papers older than 2 years, which should have complete links.

--------------------------------------------------------------------------------

## 2. `discover_available_links` — List available linknames

Lists all available ELink linknames for a given record. Use this when you don't
know which `linkname` to pass to `find_linked_biological_data`.

```bash
python3 scripts/pubmed_api.py ./available_links.json discover_available_links "35113657"
python3 scripts/pubmed_api.py ./available_gene_links.json discover_available_links "93986" --dbfrom gene
```

**Arguments:**

-   `source_id` (str, required) – source record ID (e.g. a PMID)
-   `dbfrom` (str, default "pubmed") – source database

**Output:**

```json
[
  {"linkname": "pubmed_gene", "db": "gene"},
  {"linkname": "pubmed_nuccore", "db": "nuccore"},
  {"linkname": "pubmed_pmc", "db": "pmc"}
]
```

# Fetch & Resolve Functions

Detailed argument specifications, output schemas, and usage guidance for
`fetch_article_abstracts`, `get_full_text_pmc`, and `fetch_database_summary`.

## 1. `fetch_article_abstracts` — Get metadata + abstracts

Retrieves title, authors, journal, publication date, DOI, and abstract text for
a batch of PMIDs via a single efetch XML call. Structured abstracts (BACKGROUND,
METHODS, RESULTS, CONCLUSION) are concatenated with labels.

```bash
uv run scripts/pubmed_api.py ./abstracts.json fetch_article_abstracts "35113657,31234568"
```

**Arguments:**

-   `pmids` (list[str], required) – comma-separated PMIDs
-   `webenv` (str, default "") – WebEnv from `cache_results_history`
-   `query_key` (str, default "") – query_key from `cache_results_history`

**Output:** `list[object]` — one object per PMID:

-   `pmid` (str) — always present
-   `title` (str | null) — `null` for dead/unpopulated records
-   `authors` (list[str]) — `"LastName Initials"` format; may be empty
-   `journal` (str | null) — full journal name
-   `pubdate` (str | null) — `"YYYY Mon DD"` or `MedlineDate` fallback
-   `doi` (str | null) — `null` if no DOI on record
-   `abstract` (str | null) — plain text, or `"LABEL: text\nLABEL: text"` for
    structured abstracts

**Important**: If both the `title` and `abstract` are `null`, the PMID is an
unpopulated or "dead" database record. If this occurs, report that the paper
does not exist. **DO NOT** attempt to summarize neighboring PMIDs or guess the
intended paper unless explicitly asked to disambiguate.

--------------------------------------------------------------------------------

## 2. `get_full_text_pmc` — Open-access full text

Retrieves full text of an open-access article from PMC. **Important** Only
returns articles in the **PMC Open Access Subset** — with licenses that permit
text mining and redistribution. Being "in PMC" is not sufficient; many PMC
articles have restrictive licenses that exclude them from the OA subset.

```bash
uv run scripts/pubmed_api.py ./full_text_35113657.json get_full_text_pmc "35113657"
```

**Arguments:**

-   `pmid` (str, required) – PMID of the article

**Output (success):** `{"pmid": str, "full_text": str}`

**Output (error):** `{"error": str, "endpoint": str}` — article is paywalled,
embargoed, or not in the OA subset.

**Important**: If you plan to use `get_full_text_pmc`, pre-filter your search
with `AND "pmc open access"[filter]` to only return papers that are in the OA
subset. This avoids wasting calls on papers that will inevitably fail. *Note*:
The `"pmc open access"[filter]` is highly restrictive and may return zero
results for some topics. If so, you can fall back to `"free full text"[Filter]`,
but be prepared to handle failures in `get_full_text_pmc` as not all free papers
are in the BioC OA subset. Example:

```bash
uv run scripts/pubmed_api.py ./oa_results.json search_pubmed \
  "psilocybin depression AND \"pmc open access\"[filter]" --max_results 5
```

If retrieval fails, use `fetch_article_abstracts` to fetch the abstract only.

--------------------------------------------------------------------------------

## 3. `fetch_database_summary` — Resolve UIDs from any NCBI database

Retrieves summary metadata for records in any NCBI database. Use this to resolve
the opaque UIDs returned by `find_linked_biological_data` into human-readable
data (accession numbers, gene names, descriptions, etc.).

```bash
uv run scripts/pubmed_api.py ./nuccore_summary.json fetch_database_summary nuccore "1798174254,1798172431"
uv run scripts/pubmed_api.py ./gene_summary.json fetch_database_summary gene "43740568"
uv run scripts/pubmed_api.py ./compound_summary.json fetch_database_summary pccompound "2244"
```

**Arguments:**

-   `database` (str, required) – target NCBI database (e.g. `nuccore`, `gene`,
    `protein`, `pccompound`)
-   `id_list` (list[str], required) – comma-separated UIDs

**Output:** `list[object]` — one object per UID with database-specific fields.
Key fields by database:

-   **nuccore/protein**:
    -   `uid` (str) — NCBI unique identifier for the sequence record
    -   `title` (str) — sequence definition line (e.g. `"Homo sapiens BRCA1
        mRNA, complete cds"`)
    -   `accessionversion` (str) — versioned accession number (e.g.
        `"NM_007294.4"`)
    -   `organism` (str) — source organism (e.g. `"Homo sapiens"`)
    -   `slen` (int) — sequence length in base pairs or amino acids
    -   `moltype` (str) — molecule type (`"dna"`, `"rna"`, `"aa"`)
    -   `sourcedb` (str) — originating database (`"refseq"`, `"insd"`, etc.)
-   **gene**:
    -   `uid` (str) — NCBI Gene ID
    -   `name?` (str) — official gene symbol (e.g. `"BRCA1"`)
    -   `description` (str) — full gene name (e.g. `"BRCA1 DNA repair
        associated"`)
    -   `summary` (str) — functional summary paragraph from RefSeq
    -   `organism` (object) — `{"scientificname": str, "commonname": str,
        "taxid": int}`
    -   `otheraliases` (str) — comma-separated alternative gene symbols
    -   `genomicinfo` (list[object]) — chromosomal location(s), each with
        `chrloc`, `chrstart`, `chrstop`, `exoncount`
-   **pccompound**:
    -   `uid` (str) — NCBI record UID
    -   `cid` (int) — PubChem Compound ID
    -   `synonymlist` (list[str]) — common names and identifiers (e.g.
        `["Aspirin", "Acetylsalicylic acid", ...]`)
    -   `sourcecategorylist` (list[str]) — data source categories (e.g.
        `["Deposited Substances", "Chemical Vendors", ...]`)

# Search Functions

Detailed argument specifications, output schemas, search strategies, and
troubleshooting for `search_pubmed` and `global_database_discovery`.

## 1. `search_pubmed` — Find PMIDs by query

Returns a list of PubMed IDs matching a free-text query. Supports full NCBI
query syntax: Boolean operators (`AND`, `OR`, `NOT`), MeSH terms and tags.

1.  Syntax: Use term[tag](e.g., Parkinson[ti]). Tags are NOT case-sensitive.
2.  Most common tags:
    -   [tiab]: Title & Abstract (Best for keyword precision).
    -   [mesh]: Medical Subject Headings (Best for conceptual accuracy).
    -   [pt]: Publication Type (Filter by Review, Clinical Trial, Case Reports).
    -   [dp]: Publication Date (Interchangeable with [pdat]).
3.  Modern Power Moves:
    -   Relative Dates: Use `"last X months"[dp]` or `"last X years"[dp]` (e.g.,
        `"last 1 months"[dp]`).
    -   Proximity Search: `"word1 word2"[tiab:~N]` finds words within N tokens.
        Use this instead of AND for multi-word concepts (e.g., `"gut
        microbiome"[tiab:~2]`).
4.  Reliability: Use YYYY/MM/DD:YYYY/MM/DD[dp] for custom ranges. Avoid dptr.

### Search Strategies & Troubleshooting

If searching returns few or no results, try at most 3 queries before changing
strategy. After 3 failed searches, the query is flawed. Broaden to core entities
(gene + disease), fetch 5 abstracts, and adopt the authors' vocabulary.
Relaxation order:

1.  Drop Field Restrictors: If you used [tiab]/[ti]/[pt] in the query and got no
    results, remove them: key terms often appear in body text, not titles.
    -   Too restrictive: `"CRISPR"[ti] AND "indirect cardiovascular side
        effects"[tiab]`
    -   Better: `"CRISPR" AND "cardiovascular" AND ("side effects" OR "toxicity"
        OR "unintended")`
2.  Use Broad Synonyms: Group standardized synonyms with OR.
    -   Example: `("cardiovascular" OR "cardiac" OR "heart") AND ("toxicity" OR
        "adverse effects" OR "side effects" OR "off-target")`
3.  Use MeSH Terms: If keywords fail, use MeSH indexed terms.
    -   Example: `"CRISPR-Cas Systems"[mesh] AND "Cardiovascular
        Diseases/chemically induced"[mesh]`
4.  Remove Granular Constraints: For highly specific intersections, drop the
    weakest constraint first (e.g., expand date range to 2 years, or drop
    "liver" and filter broader results manually).
5.  Avoid Over-Quoting: Double quotes force **exact phrase matching**, with
    quoted words contiguous in that exact order. E.g., `"dopaminergic neurons"`
    excludes papers with "dopaminergic projection neurons", "dopamine neurons",
    "DA neurons", or with "dopaminergic" and "neurons" in separate sentences.
    Only quote multi-word terms where word order matters (anatomical regions,
    compound names). Use unquoted terms or proximity search (`[tiab:~N]`) for
    conceptual matching.
6.  Finding Primary Data: If you need raw identifiers (Sequence IDs, Chemical
    CIDs), append data-source terms to your query, e.g. `AND (accession[tiab] OR
    "GenBank"[tiab] OR "supplementary"[tiab])`. This filters out review articles
    that discuss concepts but omit the raw data.

For advanced search tags and strategies refer to

-   [PubMed Advanced Search Reference](advanced-search.md).
-   https://pubmed.ncbi.nlm.nih.gov/help/

```bash
uv run scripts/pubmed_api.py ./search_results.json search_pubmed "BRCA1 cancer" --max_results 5 --sort_by relevance
```

**Arguments:**

-   `query` (str, required) – free-text or structured NCBI query
-   `max_results` (int, default 10) – maximum PMIDs to return
-   `sort_by` (str, default "relevance") – `relevance`, `pub_date`, `Author`,
    `JournalName`, or `Title`

**Output:** `["35113657", "31234568"]`

**Filtering tips:**

Prefer restrictors in initial queries to reduce noise. If a restricted query
returns 0 results, see Troubleshooting above.

-   Publication type: append `AND "systematic review"[pt]` or `AND
    "meta-analysis"[pt]` or `AND "clinical trial"[pt]`
-   Date range: append `AND 2023/01:2023/12[dp]`
-   Title/abstract only: use `[tiab]` tag, e.g. `"CRISPR off-target"[tiab]`
-   Exclude noise: append `NOT "comment"[pt] NOT "editorial"[pt]` if getting too
    many non-original research results.

--------------------------------------------------------------------------------

## 2. `global_database_discovery` — Count hits across all NCBI databases

Reports how many records match a query across every NCBI database at once.
Useful for deciding which databases are worth querying for a given topic.

```bash
uv run scripts/pubmed_api.py ./crispr_counts.json global_database_discovery "CRISPR"
```

**Arguments:**

-   `query` (str, required) – free-text query

**Output:** `dict[str, int]` — keys are NCBI database names (e.g. `pubmed`,
`pmc`, `gene`), values are hit counts. Only databases with ≥1 hit are included.
Example: `json {"pubmed": 14500, "pmc": 4200, "gene": 12, "protein": 105}`

# Utilities

## `verify_medical_spelling` — Spell-check biomedical terms

Suggests spelling corrections for biomedical terms using NCBI's dictionary.
Useful for normalizing user-provided terminology before searching.

```bash
python3 scripts/pubmed_api.py ./spelling.json verify_medical_spelling "rhuematoid arthritus"
```

**Arguments:**

-   `term` (str, required) – term to spell-check

**Output:** `json {"original": "rhuematoid arthritus", "corrected": "rheumatoid
arthritis"}`

Returns the original term unchanged if the spelling is already correct.

--------------------------------------------------------------------------------

## `match_raw_citations` — Resolve messy citations to PMIDs

Resolves incomplete or messy bibliographic citations to PMIDs via the ecitmatch
endpoint. Each citation must be a pipe-delimited string in the format
`journal|year|volume|first_page|author_name|key|`. Empty fields are valid. The
trailing pipe is required.

For field selection strategies, journal name pitfalls, and fallback chains,
refer to [Citation Matching Guide](citation-matching.md).

```bash
uv run scripts/pubmed_api.py ./nature2006_pmids.json match_raw_citations "nature|2006||||takahashi k|key0|"
```

**Arguments:**

-   `citation_strings` (list[str], required) – comma-separated pipe-delimited
    citations

**Output:** `["16904174"]`

# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

r"""PubMed API CLI.

Provides command-line access to NCBI E-utilities and PMC BioC endpoints.
Outputs JSON to stdout; all diagnostics go to stderr.

Usage:
  uv run pubmed_api.py brca1_search.json \
    search_pubmed "BRCA1 cancer" 5 relevance
  uv run pubmed_api.py abstract_35113657_31234568.json \
    fetch_article_abstracts "35113657,31234568"
"""

# /// script
# requires-python = ">=3.10"
# dependencies = [
#   "polite-http",
#   "python-dotenv",
# ]
# ///

import inspect
import json
import os
import sys
import urllib.parse
import xml.etree.ElementTree as ET

import dotenv
from polite_http import http_client

EUTILS_BASE = "https://eutils.ncbi.nlm.nih.gov"
PMC_BIOC_BASE = (
    "https://www.ncbi.nlm.nih.gov/research/bionlp/RESTful/pmcoa.cgi/BioC_json"
)

_EUTILS_CLIENT = None
_PMC_CLIENT = None


def get_eutils_client():
  """Returns the lazily initialized E-utilities HttpClient."""
  global _EUTILS_CLIENT
  if _EUTILS_CLIENT is None:
    qps = 10 if os.environ.get("NCBI_API_KEY") else 3
    _EUTILS_CLIENT = http_client.HttpClient(EUTILS_BASE, qps=qps)
  return _EUTILS_CLIENT


def get_pmc_client():
  """Returns the lazily initialized PMC HttpClient."""
  global _PMC_CLIENT
  if _PMC_CLIENT is None:
    qps = 10 if os.environ.get("NCBI_API_KEY") else 3
    _PMC_CLIENT = http_client.HttpClient(PMC_BIOC_BASE, qps=qps)
  return _PMC_CLIENT


_MAX_JSON_ERROR_SNIPPET_LENGTH = 500


def _env_params():
  """Returns a dictionary of parameters from the environment and API key."""

  params = {}
  email = os.environ.get("USER_EMAIL")
  tool = os.environ.get("NCBI_TOOL")
  api_key = os.environ.get("NCBI_API_KEY")
  if email:
    params["email"] = email
  if tool:
    params["tool"] = tool
  if api_key:
    params["api_key"] = api_key

  return params


def _get(url, params=None, *, raw=False, client=None):
  """GET request with retry logic and rate limiting."""
  if client is None:
    client = get_eutils_client()

  if params:
    url = url + "?" + urllib.parse.urlencode(params)

  try:
    if raw:
      return client.fetch_text(url)
    return client.fetch_json(url)
  except http_client.HttpError as e:
    if e.status_code == 404:
      return {
          "error": "Record not found (HTTP 404).",
          "endpoint": url.split("?")[0],
      }
    else:
      return {
          "error": f"HTTP Error {e.status_code or 'Error'}: {str(e)}",
          "endpoint": url.split("?")[0],
      }
  except json.JSONDecodeError as e:
    body_snippet = e.doc
    if len(body_snippet) > _MAX_JSON_ERROR_SNIPPET_LENGTH:
      body_snippet = f"{body_snippet[:_MAX_JSON_ERROR_SNIPPET_LENGTH]}..."
    return {
        "error": f"Failed to parse JSON response. Body: {body_snippet}",
        "endpoint": url.split("?")[0],
    }


def _post(url, params):
  """POST request with retry logic and rate limiting."""
  try:
    data_bytes = urllib.parse.urlencode(params).encode("utf-8")
    resp = get_eutils_client().fetch(url, method="POST", data=data_bytes)
    return resp.text
  except http_client.HttpError as e:
    return {"error": f"HTTP Error {e.status_code or 'Error'}", "endpoint": url}


def search_pubmed(
    query: str,
    max_results: int = 10,
    sort_by: str = "relevance",
) -> list[str]:
  """Returns a list of PubMed IDs (PMIDs) matching a free-text query.

  Supports the full NCBI query syntax including Boolean operators, MeSH terms,
  field tags (e.g. [tiab], [au]), and date ranges. The sort_by parameter accepts
  'relevance', 'pub_date', 'Author', 'JournalName', or 'Title'.

  Args:
    query: Free-text or structured NCBI query
    max_results: Maximum PMIDs to return
    sort_by: 'relevance', 'pub_date', 'Author', 'JournalName', or 'Title'

  Returns:
    List of PMIDs
  """
  params = _env_params() | {
      "db": "pubmed",
      "term": query,
      "retmax": max_results,
      "sort": sort_by,
      "retmode": "json",
  }
  data = _get(f"{EUTILS_BASE}/entrez/eutils/esearch.fcgi", params)
  if isinstance(data, dict) and "error" in data:
    return data
  try:
    return data["esearchresult"]["idlist"]
  except (KeyError, TypeError):
    return {
        "error": "Unexpected esearch response structure",
        "endpoint": "esearch.fcgi",
    }


def _id_params(ids, webenv, query_key):
  if webenv:
    return {"WebEnv": webenv, "query_key": query_key}
  return {"id": ",".join(ids) if isinstance(ids, list) else ids}


def fetch_article_abstracts(
    pmids: list[str],
    webenv: str = "",
    query_key: str = "",
) -> list[dict[str, str | list[str] | None]]:
  """Retrieves article metadata and abstracts for a batch of PMIDs.

  Uses a single efetch XML call to extract all fields from PubmedArticle
  elements: title, authors, journal, date, abstract, and DOI.
  Structured abstracts (BACKGROUND, METHODS, RESULTS, CONCLUSION) are
  concatenated with their section labels.

  Can accept either explicit pmids or a webenv/query_key pair from
  cache_results_history to reference a previously uploaded set.

  Args:
    pmids: List of PMIDs
    webenv: WebEnv from cache_results_history
    query_key: query_key from cache_results_history

  Returns:
    List of dicts with article metadata and abstracts
  """
  id_p = _id_params(pmids, webenv, query_key)
  params = (
      _env_params()
      | id_p
      | {"db": "pubmed", "rettype": "abstract", "retmode": "xml"}
  )
  xml_data = _get(f"{EUTILS_BASE}/entrez/eutils/efetch.fcgi", params, raw=True)
  if isinstance(xml_data, dict) and "error" in xml_data:
    return xml_data

  try:
    root = ET.fromstring(xml_data)
  except ET.ParseError:
    return {"error": "Failed to parse efetch XML", "endpoint": "efetch.fcgi"}

  results = []
  for article in root.iter("PubmedArticle"):
    pmid_elem = article.find(".//PMID")
    if pmid_elem is None:
      continue

    art = article.find(".//Article")
    if art is None:
      continue

    authors = []
    for author in art.findall(".//AuthorList/Author"):
      last = author.findtext("LastName") or ""
      init = author.findtext("Initials") or ""
      name = (
          f"{last} {init}".strip()
          if last
          else author.findtext("CollectiveName") or ""
      )
      if name:
        authors.append(name)

    abstract_parts = []
    for at in art.findall(".//Abstract/AbstractText"):
      label = at.get("Label")
      text = "".join(at.itertext())
      if label:
        abstract_parts.append(f"{label}: {text}")
      else:
        abstract_parts.append(text)
    abstract = "\n".join(abstract_parts) if abstract_parts else None

    doi = None
    for eid in art.findall("ELocationID"):
      if eid.get("EIdType") == "doi":
        doi = eid.text
        break

    journal_elem = art.find(".//Journal")
    journal = None
    pubdate = None
    if journal_elem is not None:
      journal = journal_elem.findtext("Title")
      pd = journal_elem.find(".//PubDate")
      if pd is not None:
        year = pd.findtext("Year") or ""
        month = pd.findtext("Month") or ""
        day = pd.findtext("Day") or ""
        medline = pd.findtext("MedlineDate") or ""
        pubdate = f"{year} {month} {day}".strip() if year else medline

    results.append({
        "pmid": pmid_elem.text,
        "title": art.findtext("ArticleTitle"),
        "authors": authors,
        "journal": journal,
        "pubdate": pubdate,
        "doi": doi,
        "abstract": abstract,
    })

  return results


def find_linked_biological_data(
    source_pmid: str,
    target_database: str,
    linkname: str,
    dbfrom: str = "pubmed",
    mindate: str = "",
    maxdate: str = "",
    webenv: str = "",
    query_key: str = "",
) -> list[str]:
  """Finds records in another NCBI database linked to a source record.

  NCBI maintains cross-references between databases (e.g. pubmed -> gene,
  pubmed -> nuccore, pubmed -> pccompound). This traverses those links and
  returns the target database record IDs. The elink response structure is
  deeply nested (linksets -> linksetdbs -> links), so this flattens it.

  Can accept either an explicit source_pmid or a webenv/query_key pair from
  cache_results_history to link all IDs in a cached set at once.

  Args:
    source_pmid: Source record ID (PMID when dbfrom is pubmed)
    target_database: Target database
    linkname: Link name
    dbfrom: Source database (default: pubmed)
    mindate: Minimum date filter (YYYY/MM/DD), pubmed->pubmed only
    maxdate: Maximum date filter (YYYY/MM/DD), pubmed->pubmed only
    webenv: WebEnv from cache_results_history
    query_key: query_key from cache_results_history

  Returns:
    List of target database record IDs
  """
  id_p = _id_params(source_pmid, webenv, query_key)
  params = (
      _env_params()
      | id_p
      | {
          "dbfrom": dbfrom,
          "db": target_database,
          "linkname": linkname,
          "retmode": "json",
      }
  )
  if mindate:
    params["mindate"] = mindate
    params["datetype"] = "pdat"
  if maxdate:
    params["maxdate"] = maxdate
    params["datetype"] = "pdat"
  data = _get(f"{EUTILS_BASE}/entrez/eutils/elink.fcgi", params)
  if isinstance(data, dict):
    if "error" in data:
      return data
    if "ERROR" in data:
      return {"error": data["ERROR"], "endpoint": "elink.fcgi"}

  linksets = data.get("linksets", [])
  if not linksets:
    print(
        f"Got empty linksets for {source_pmid}, {target_database},"
        f" {linkname}, {dbfrom}",
        file=sys.stderr,
    )
    return []
  linksetdbs = linksets[0].get("linksetdbs", [])
  if not linksetdbs:
    print(
        f"Got empty linksetdbs for {source_pmid}, {target_database},"
        f" {linkname}, {dbfrom}",
        file=sys.stderr,
    )
    return []
  return [str(link_id) for link_id in linksetdbs[0].get("links", [])]


def discover_available_links(
    source_id: str,
    dbfrom: str = "pubmed",
) -> list[dict[str, str]]:
  """Lists all available ELink linknames for a given record.

  Uses cmd=acheck to ask NCBI which cross-database links exist for the
  source record. Returns a list of dicts with linkname and target database.

  Args:
    source_id: Source record ID (e.g. a PMID)
    dbfrom: Source database (default: pubmed)

  Returns:
    List of dicts with linkname and db keys
  """
  params = _env_params() | {
      "dbfrom": dbfrom,
      "id": source_id,
      "cmd": "acheck",
      "retmode": "json",
  }
  data = _get(f"{EUTILS_BASE}/entrez/eutils/elink.fcgi", params)
  if isinstance(data, dict):
    if "error" in data:
      return data
    if "ERROR" in data:
      return {"error": data["ERROR"], "endpoint": "elink.fcgi"}

  linksets = data.get("linksets", [])
  if not linksets:
    print(f"Got empty linksets for {source_id}, {dbfrom}", file=sys.stderr)
    return []

  idchecklist = linksets[0].get("idchecklist", {})
  if not idchecklist:
    print(f"Got empty idchecklist for {source_id}, {dbfrom}", file=sys.stderr)
    return []

  idlinksets = idchecklist.get("idlinksets", [])
  if not idlinksets:
    print(f"Got empty idlinksets for {source_id}, {dbfrom}", file=sys.stderr)
    return []

  results = []
  for idcheck in idlinksets:
    for linkinfo in idcheck.get("linkinfos", []):
      results.append({
          "linkname": linkinfo.get("linkname", ""),
          "db": linkinfo.get("dbto", ""),
      })
  return results


def get_full_text_pmc(pmid: str) -> dict[str, str]:
  """Retrieves the full text of an open-access article from PubMed Central.

  Uses the PMC BioC API (not E-utilities) which returns structured JSON with
  passage-level annotations. The passages are concatenated into a single string.
  Returns an error dict if the article is paywalled, embargoed, or not in PMC.

  Args:
    pmid: PMID of the article

  Returns:
    Dict with pmid and full_text or error message
  """
  url = f"{PMC_BIOC_BASE}/{pmid}/unicode"
  data = _get(url, client=get_pmc_client())
  if isinstance(data, dict) and "error" in data:
    return data
  try:
    passages = []
    for doc in data if isinstance(data, list) else [data]:
      for document in doc.get("documents", []):
        for passage in document.get("passages", []):
          text = passage.get("text", "")
          if text:
            passages.append(text)
    return {"pmid": pmid, "full_text": "\n".join(passages)}
  except (KeyError, TypeError):
    return {"error": "Unexpected BioC response structure", "endpoint": url}


def verify_medical_spelling(term: str) -> dict[str, str]:
  """Suggests spelling corrections for biomedical terms using NCBI's dictionary.

  Useful for normalizing user-provided medical terminology before searching.
  Returns the original term unchanged if NCBI considers the spelling correct
  (i.e. the CorrectedQuery field in the XML response is empty).

  Args:
    term: Term to correct

  Returns:
    Dict with original and corrected term or error message
  """
  params = _env_params() | {"db": "pubmed", "term": term}
  data = _get(f"{EUTILS_BASE}/entrez/eutils/espell.fcgi", params, raw=True)
  if isinstance(data, dict) and "error" in data:
    return data
  try:
    root = ET.fromstring(data)
    corrected = root.findtext(".//CorrectedQuery") or ""
    return {"original": term, "corrected": corrected if corrected else term}
  except ET.ParseError:
    return {"error": "Failed to parse espell XML", "endpoint": "espell.fcgi"}


def global_database_discovery(query: str) -> dict[str, int]:
  """Reports how many records match a query across all NCBI databases at once.

  Useful for deciding which databases (pubmed, gene, protein, nuccore, etc.)
  are worth querying for a given topic. This endpoint only returns XML, so
  responses are parsed with ElementTree rather than JSON.

  Args:
    query: Query to search for

  Returns:
    Dict with database names and counts or error message
  """
  params = _env_params() | {"term": query, "retmode": "xml"}
  data = _get(f"{EUTILS_BASE}/gquery", params, raw=True)
  if isinstance(data, dict) and "error" in data:
    return data
  try:
    root = ET.fromstring(data)
    result = {}
    for item in root.iter("ResultItem"):
      db_name = item.findtext("DbName")
      count = item.findtext("Count")
      if db_name and count:
        try:
          result[db_name] = int(count)
        except ValueError:
          result[db_name] = count
    return result
  except ET.ParseError:
    return {
        "error": "Failed to parse egquery XML",
        "endpoint": "egquery.fcgi",
    }


def match_raw_citations(citation_strings: list[str]) -> list[str]:
  """Resolves messy or incomplete bibliographic citations to PMIDs.

  Each citation string should be pipe-delimited in the format
  'journal|year|volume|first_page|author_name|your_key|'. The ecitmatch
  endpoint is archaic and returns pipe-delimited plain text (not JSON/XML),
  so responses are split by line and the PMID extracted from the last field.
  Unmatched citations are silently omitted from the result.

  Args:
    citation_strings: List of citation strings

  Returns:
    List of PMIDs
  """
  params = _env_params() | {
      "db": "pubmed",
      "retmode": "xml",
      "bdata": "\r".join(citation_strings),
  }
  data = _get(f"{EUTILS_BASE}/entrez/eutils/ecitmatch.cgi", params, raw=True)
  if isinstance(data, dict) and "error" in data:
    return data
  pmids = []
  for line in data.strip().split("\n"):
    parts = line.strip().rstrip("|").split("|")
    pmid = parts[-1].strip() if parts else ""
    if pmid and pmid.lower() != "not found":
      pmids.append(pmid)
  return pmids


def cache_results_history(pmids: list[str]) -> dict[str, str]:
  """Uploads PMIDs to the NCBI History Server and returns a session handle.

  The returned WebEnv and query_key can be passed to subsequent E-utility calls
  to reference the stored set, avoiding repeated transmission of large ID lists.
  Uses POST because GET would exceed the URL length limit for large batches.
  The endpoint returns XML despite accepting retmode=json.

  Args:
    pmids: List of PMIDs

  Returns:
    Dict with webenv and query_key or error message
  """
  params = _env_params() | {"db": "pubmed", "id": ",".join(pmids)}
  data = _post(f"{EUTILS_BASE}/entrez/eutils/epost.fcgi", params)
  if isinstance(data, dict) and "error" in data:
    return data
  try:
    root = ET.fromstring(data)
    webenv = root.findtext(".//WebEnv") or ""
    query_key = root.findtext(".//QueryKey") or ""
    return {"webenv": webenv, "query_key": query_key}
  except ET.ParseError:
    return {"error": "Failed to parse epost XML", "endpoint": "epost.fcgi"}


def fetch_database_summary(
    database: str,
    id_list: list[str],
) -> list[dict[str, str | list[str] | None]]:
  """Retrieves summary metadata for records in any NCBI database.

  Wraps the esummary endpoint to resolve opaque UIDs (returned by
  find_linked_biological_data) into human-readable metadata such as
  accession numbers, gene names, organism, and descriptions.

  Args:
    database: Target NCBI database (e.g. nuccore, gene, protein, pccompound)
    id_list: List of UIDs to summarize

  Returns:
    List of dicts, one per UID, with database-specific metadata fields.
  """
  params = _env_params() | {
      "db": database,
      "id": ",".join(id_list),
      "retmode": "json",
  }
  data = _get(f"{EUTILS_BASE}/entrez/eutils/esummary.fcgi", params)
  if isinstance(data, dict) and "error" in data:
    return data
  try:
    result_block = data.get("result", {})
    uids = result_block.get("uids", [])
    summaries = []
    for uid in uids:
      doc = result_block.get(uid, {})
      if doc:
        summaries.append(doc)
    return summaries
  except (KeyError, TypeError, AttributeError):
    return {
        "error": "Unexpected esummary response",
        "endpoint": "esummary.fcgi",
    }


# ---------------------------------------------------------------------------
# CLI dispatch — inferred from type hints via inspect
# ---------------------------------------------------------------------------
FUNCTIONS = {
    fn.__name__: fn
    for fn in [
        search_pubmed,
        fetch_article_abstracts,
        find_linked_biological_data,
        discover_available_links,
        get_full_text_pmc,
        verify_medical_spelling,
        global_database_discovery,
        match_raw_citations,
        cache_results_history,
        fetch_database_summary,
    ]
}


def _is_list_type(annotation):
  origin = getattr(annotation, "__origin__", None)
  return origin is list


def _coerce_arg(value: str, annotation):
  if _is_list_type(annotation):
    return value.split(",")
  if annotation is int:
    return int(value)
  return value


def main():
  dotenv.load_dotenv(os.path.expanduser("~/.env"))
  if len(sys.argv) < 3:
    print("Usage: pubmed_api.py <output_file> <func> [--flag val]")
    print(f"Available: {', '.join(FUNCTIONS.keys())}")
    sys.exit(1)

  output_file = sys.argv[1]
  func_name = sys.argv[2]
  if func_name not in FUNCTIONS:
    print(f"Error: Unknown function: {func_name}")
    sys.exit(1)

  if os.path.exists(output_file):
    print(f"Error: Output file {output_file} already exists")
    sys.exit(1)

  fn = FUNCTIONS[func_name]
  sig = inspect.signature(fn)

  positional = []
  flags = {}
  raw_args = sys.argv[3:]
  i = 0
  while i < len(raw_args):
    if raw_args[i].startswith("--"):
      key = raw_args[i][2:]
      if i + 1 < len(raw_args):
        flags[key] = raw_args[i + 1]
        i += 2
      else:
        print(f"Error: Missing value for flag --{key}")
        sys.exit(1)
    else:
      positional.append(raw_args[i])
      i += 1

  kwargs = {}
  pos_idx = 0
  for name, param in sig.parameters.items():
    if name in flags:
      kwargs[name] = _coerce_arg(flags[name], param.annotation)
    elif pos_idx < len(positional):
      kwargs[name] = _coerce_arg(positional[pos_idx], param.annotation)
      pos_idx += 1
    elif param.default is not inspect.Parameter.empty:
      kwargs[name] = param.default
    else:
      print(f"Error: Missing required argument: {name}")
      sys.exit(1)

  try:
    result = fn(**kwargs)
  except Exception as e:
    print(f"Internal error in {func_name}: {e}")
    sys.exit(2)

  if isinstance(result, dict) and "error" in result:
    msg = result["error"]
    endpoint = result.get("endpoint", "")
    if endpoint:
      print(f"API error ({endpoint}): {msg}")
    else:
      print(f"API error: {msg}")
    sys.exit(1)

  with open(output_file, "w", encoding="utf-8") as f:
    json.dump(result, f, indent=2)
    print(file=f)

  if isinstance(result, list):
    print(f"API call OK: {len(result)} results json written to {output_file}")
  elif isinstance(result, dict):
    keys = ", ".join(sorted(result.keys()))
    print(f"API call OK: result ({keys}) json written to {output_file}")
  else:
    print(f"API call OK: result json written to {output_file}")


if __name__ == "__main__":
  main()

---
name: pubmed-database
description: >-
  Search PubMed for scientific literature, including published clinical trials.
  Fetch abstracts and full text. Link published research to biological databases
  (gene, protein, nucleotide, PubChem) to discover associations between papers
  and specific compounds or genes. Verify medical spelling, match raw citations,
  and cache result sets for bulk processing. Interfaces NCBI E-utilities and PMC
  BioC APIs.
---

# PubMed API

## Prerequisites

1.  **`uv`**: Read the `uv` skill and follow its Setup instructions to ensure
    `uv` is installed and on PATH.
2.  **User Notification**: If .licenses/pubmed_database_LICENSE.txt does not
    already exist in the workspace root directory then (1) prominently notify
    the user to check the terms at https://pubmed.ncbi.nlm.nih.gov/disclaimer/
    and https://www.ncbi.nlm.nih.gov/home/about/policies/ and to always check
    the license of the papers retrieved by the skill for any restrictions, then
    (2) create the file recording the notification text and timestamp.
3.  **`.env` file**: Make sure the `.env` file exists in your home directory.
    Create one if it does not exist.
4.  **`NCBI_API_KEY`** (optional): Raises the NCBI E-utilities rate limit from 3
    to 10 requests/second. The skill works without it, but a key is recommended
    if the user plans many queries or encounters a 429 error. You can register
    for a key for free at https://www.ncbi.nlm.nih.gov/account/settings/. You
    **MUST** use the safe credentials protocol in the `credentials` skill to
    check for and request this key if this skill looks relevant to the user's
    request.
5.  **`USER_EMAIL`** (optional): Identifies the caller to NCBI (recommended by
    their Terms of Use). You **MUST** use the safe credentials protocol in the
    `credentials` skill to check for and request this credential if this skill
    looks relevant to the user's request.

This skill provides CLI access to the NCBI PubMed and PubMed Central APIs via
`scripts/pubmed_api.py` — a single CLI with 10 functions covering search, fetch,
linking, full text, spelling, discovery, citation matching, and caching.

## Core Rules

-   **API Use**: Always use the provided wrapper `scripts/pubmed_api.py` which
    manages rate limits automatically and prevents API abuse. Setting the
    `NCBI_API_KEY` environment variable raises the rate limit from 3 to 10
    requests/second. Querying the API any other way (e.g. via curl, wget, or
    hand-written code) is strictly forbidden.
-   **JSON Processing**: Use `jq` to filter and transform JSON output (or python
    equivalents if `jq` is not available) to prevent hallucinations and context
    overflow.
-   **Temporary Files**: To avoid polluting the working directory with JSON
    files, use a temporary directory inside the current directory. When running
    multiple agents or tasks in parallel, ensure each uses a unique subdirectory
    name (e.g., `tmp_$TASK_ID/`) to avoid file collisions.
-   **Notification**: If this skill is used, ensure this is mentioned in the
    output AND list the URLs of all papers that were used in producing the
    output.

## Structure of the skill folder

-   `SKILL.md` - This file
-   `scripts/pubmed_api.py` - The skill CLI
-   `references/` - Directory with detailed function specifications
    -   `advanced-linking.md`
    -   `advanced-search.md`
    -   `bulk-workflows.md`
    -   `citation-matching.md`
    -   `cross-database-linking.md`
    -   `fetch-and-resolve.md`
    -   `search-and-discovery.md`
    -   `utilities.md`

## CLI Usage

```bash
uv run scripts/pubmed_api.py <output_file> <function_name> <required_args> [--flag value ...]
```

-   **Positional Arguments**: Arguments are positional; list arguments are
    passed as comma-separated strings without spaces (e.g.
    `"35113657,31234568"`).
-   **Flag Options**: Optional arguments can be passed as `--flag value` instead
    of positional args.
-   **Output Handling**: On success, JSON is written to `output_file`. On error,
    the process exits with a non-zero code and no output file is written.

### Example Usage

```bash
uv run scripts/pubmed_api.py ./search_results.json search_pubmed "BRCA1" --max_results 5
cat ./search_results.json | jq '.[]' -r
uv run scripts/pubmed_api.py ./abstracts.json fetch_article_abstracts "35113657"
cat ./abstracts.json | jq '.[0].title' -r
```

## Essential Recipes

**Join PMIDs for the next call (most common chaining pattern):**

```bash
cat ./search_results.json | jq -r 'join(",")'
```

**Slim abstracts to essential fields and truncate long abstracts:**

```bash
cat ./abstracts.json | jq '[.[] | {pmid, title, snippet: (.abstract // "")[:500]}]'
```

**Filter by keyword (null-safe):**

```bash
cat ./abstracts.json | jq '[.[] | select((.title // "") | contains("Review"))]'
```

### Context Management & Accuracy

When processing larger result sets (>10 abstracts):

1.  **Filter Early**: Use `jq` to verify keywords in abstracts *before* reading
    the full JSON into context.
2.  **Slimming**: Extract only `title` and `abstract` fields unless explicitly
    instructed otherwise. Author lists and metadata contribute to noise.
3.  **Bulk Operations (N > 10)**: Avoid fetching or processing IDs one-by-one.
    The API and History Server are designed for bulk retrieval. Fetch all data
    in a **single turn** and use shell pipelines to slim the results before
    reading into context. This prevents turn exhaustion and context overflow.
4.  **Grounding**: Never use internal knowledge to provide specific identifiers
    (PMIDs, CIDs, Gene IDs) if no results are found. Report the tool's output
    accurately to ensure results are grounded in the current database state.
5.  **Search Termination**: When asked to find papers that may not exist, limit
    exploration to 3–5 high-quality, varied search queries. If no results match
    after these attempts, conclude that no papers meet the criteria rather than
    continuing to iterate — unless explicitly instructed to be thorough.

## Functions

> **⚠️ MANDATORY**: You **MUST** read the linked reference file for a function
> group **before calling any function** in that group. The tables below only
> describe *what* each function does — not *how* to call it. Argument names,
> argument order, flags, and output schemas are **only** documented in the
> reference files. **Do NOT guess or infer arguments from function names.** If
> you call a function without first reading its reference, you **will** produce
> incorrect invocations.

### [Search](references/search-and-discovery.md)

-   `search_pubmed`: Find PMIDs matching a free-text or structured NCBI query.
-   `global_database_discovery`: Count how many records match a query across
    every NCBI database.

### [Fetch & Resolve](references/fetch-and-resolve.md)

-   `fetch_article_abstracts`: Retrieve metadata and abstracts for a batch of
    PMIDs.
-   `get_full_text_pmc`: Retrieve open-access full text from PMC.
-   `fetch_database_summary`: Resolve opaque UIDs from any NCBI database into
    human-readable metadata.

### [Cross-Database Linking](references/cross-database-linking.md)

-   `find_linked_biological_data`: Find records in other NCBI databases linked
    to a source record.
-   `discover_available_links`: List all available ELink linknames for a given
    record.

### [Bulk Workflows](references/bulk-workflows.md)

When working with **more than ~10 PMIDs**, avoid processing IDs one-by-one.
Upload them to the NCBI History Server via `cache_results_history` to get a
session handle (`webenv` + `query_key`), then pass that handle to
`fetch_article_abstracts` or `find_linked_biological_data` for a single bulk
call. Chain with `jq` shell pipelines to slim results before reading into
context. This prevents turn exhaustion and context overflow. See the reference
for complete workflow recipes (search→fetch, cross-db exploration, citation
resolution, and bulk retrieval with data slimming).

-   `cache_results_history`: Upload PMIDs to the NCBI History Server for bulk
    retrieval.

### [Utilities](references/utilities.md)

-   `verify_medical_spelling`: Spell-check biomedical terms before searching.
-   `match_raw_citations`: Resolve incomplete bibliographic citations to PMIDs.