twitter-reader

어떤 스킬인가요?

Fetch Twitter/X post content including long-form Articles with full images and metadata. Use when Claude needs to retrieve tweet/article content, author info, engagement metrics, and embedded media. Supports individual posts and X Articles (long-form content). Automatically downloads all images to local attachments folder and generates complete Markdown with proper image references. Preferred over Jina for X Articles with images.

설치 방법

프로젝트 폴더에서 아래 명령어를 실행하고, 설치할 에이전트를 선택하세요.

설치 명령을 직접 실행해야 적용됩니다. 지원 에이전트와 필요한 권한·라이선스는 제작자의 안내를 확인하세요.

스킬 원문 파일

#!/usr/bin/env python3
"""
Fetch Twitter/X Article with images using twitter-cli.

Usage:
    python fetch_article.py <article_url> [output_dir]

Example:
    python fetch_article.py https://x.com/HiTw93/status/2040047268221608281 ./Clippings

Features:
    - Fetches structured data via twitter-cli
    - Downloads all images to attachments folder
    - Generates Markdown with embedded image references
"""

import sys
import os
import re
import subprocess
import argparse
from pathlib import Path
from datetime import datetime


def run_twitter_cli(url: str) -> dict:
    """Fetch article data using twitter-cli via uv run."""
    cmd = ["uv", "run", "--with", "twitter-cli", "twitter", "article", url]

    result = subprocess.run(cmd, capture_output=True, text=True)

    if result.returncode != 0:
        print(f"Error fetching article: {result.stderr}", file=sys.stderr)
        sys.exit(1)

    return parse_yaml_output(result.stdout)


def run_jina_api(url: str) -> str:
    """Fetch article text with images using Jina API."""
    api_key = os.getenv("JINA_API_KEY", "")
    jina_url = f"https://r.jina.ai/{url}"

    cmd = ["curl", "-s", jina_url]
    if api_key:
        cmd.extend(["-H", f"Authorization: Bearer {api_key}"])

    result = subprocess.run(cmd, capture_output=True, text=True)

    if result.returncode != 0:
        print(f"Warning: Jina API failed: {result.stderr}", file=sys.stderr)
        return ""

    return result.stdout


def parse_yaml_output(output: str) -> dict:
    """Parse twitter-cli YAML output into dict."""
    try:
        import yaml
        data = yaml.safe_load(output)
        if data.get("ok") and "data" in data:
            return data["data"]
        return data
    except ImportError:
        print("Error: PyYAML required. Install with: uv pip install pyyaml", file=sys.stderr)
        sys.exit(1)
    except Exception as e:
        print(f"Error parsing YAML: {e}", file=sys.stderr)
        sys.exit(1)


def extract_image_urls(text: str) -> list:
    """Extract image URLs from markdown text."""
    # Extract all pbs.twimg.com URLs (note: twimg not twitter)
    pattern = r'https://pbs\.twimg\.com/media/[^\s\)"\']+'
    matches = re.findall(pattern, text)

    # Deduplicate and normalize to large size
    seen = set()
    urls = []
    for url in matches:
        base_url = url.split('?')[0]
        if base_url not in seen:
            seen.add(base_url)
            urls.append(f"{base_url}?format=jpg&name=large")

    return urls


def download_images(image_urls: list, attachments_dir: Path) -> list:
    """Download images and return list of local paths."""
    attachments_dir.mkdir(parents=True, exist_ok=True)
    local_paths = []

    for i, url in enumerate(image_urls, 1):
        filename = f"{i:02d}-image.jpg"
        filepath = attachments_dir / filename

        cmd = ["curl", "-sL", url, "-o", str(filepath)]
        result = subprocess.run(cmd, capture_output=True)

        if result.returncode == 0 and filepath.exists() and filepath.stat().st_size > 0:
            local_paths.append(f"attachments/{attachments_dir.name}/{filename}")
            print(f"  ✓ {filename}")
        else:
            print(f"  ✗ Failed: {filename}")

    return local_paths


def replace_image_urls(text: str, image_urls: list, local_paths: list) -> str:
    """Replace remote image URLs with local paths in markdown text."""
    for remote_url, local_path in zip(image_urls, local_paths):
        # Extract base URL pattern
        base_url = remote_url.split('?')[0].replace('?format=jpg&name=large', '')
        # Replace all variations of this URL
        pattern = re.escape(base_url) + r'(\?[^\)]*)?'
        text = re.sub(pattern, local_path, text)
    return text


def sanitize_filename(name: str) -> str:
    """Sanitize string for use in filename."""
    # Remove special chars, keep alphanumeric, CJK, and some safe chars
    name = re.sub(r'[^\w\s\-\u4e00-\u9fff]', '', name)
    name = re.sub(r'\s+', '-', name.strip())
    return name[:60]  # Limit length


def generate_markdown(data: dict, text: str, image_urls: list, local_paths: list, source_url: str) -> str:
    """Generate complete Markdown document."""
    # Parse date
    created = data.get("createdAtLocal", "")
    if created:
        date_str = created[:10]
    else:
        date_str = datetime.now().strftime("%Y-%m-%d")

    author = data.get("author", {})
    metrics = data.get("metrics", {})
    title = data.get("articleTitle", "Untitled")

    # Build frontmatter
    md = f"""---
source: {source_url}
author: {author.get("name", "")}
date: {date_str}
likes: {metrics.get("likes", 0)}
retweets: {metrics.get("retweets", 0)}
bookmarks: {metrics.get("bookmarks", 0)}
---

# {title}

"""

    # Replace image URLs with local paths
    if image_urls and local_paths:
        text = replace_image_urls(text, image_urls, local_paths)

    md += text
    return md


def main():
    parser = argparse.ArgumentParser(description="Fetch Twitter/X Article with images")
    parser.add_argument("url", help="Twitter/X article URL")
    parser.add_argument("output_dir", nargs="?", default=".", help="Output directory (default: current)")
    args = parser.parse_args()

    if not args.url.startswith(("https://x.com/", "https://twitter.com/")):
        print("Error: URL must be from x.com or twitter.com", file=sys.stderr)
        sys.exit(1)

    print(f"Fetching: {args.url}")
    print("-" * 50)

    # Fetch metadata from twitter-cli
    print("Getting metadata...")
    data = run_twitter_cli(args.url)

    title = data.get("articleTitle", "")
    if not title:
        print("Error: Could not fetch article data", file=sys.stderr)
        sys.exit(1)

    author = data.get("author", {})

    print(f"Title: {title}")
    print(f"Author: {author.get('name', 'Unknown')}")
    print(f"Likes: {data.get('metrics', {}).get('likes', 0)}")

    # Fetch content with images from Jina API
    print("\nGetting content and images...")
    jina_content = run_jina_api(args.url)

    # Use Jina content if available, otherwise fall back to twitter-cli text
    if jina_content:
        text = jina_content
        # Remove Jina header lines to get clean markdown
        # Find "Markdown Content:" and keep everything after it
        marker = "Markdown Content:"
        idx = text.find(marker)
        if idx != -1:
            text = text[idx + len(marker):].lstrip()
    else:
        text = data.get("articleText", "")

    # Extract image URLs
    image_urls = extract_image_urls(text)
    print(f"Images: {len(image_urls)}")

    # Setup output paths
    output_dir = Path(args.output_dir)
    output_dir.mkdir(parents=True, exist_ok=True)

    # Create attachments folder
    date_str = data.get("createdAtLocal", "")[:10] if data.get("createdAtLocal") else datetime.now().strftime("%Y-%m-%d")
    safe_author = sanitize_filename(author.get("screenName", "unknown"))
    safe_title = sanitize_filename(title)
    attachments_name = f"{date_str}-{safe_author}-{safe_title[:30]}"
    attachments_dir = output_dir / "attachments" / attachments_name

    # Download images
    local_paths = []
    if image_urls:
        print(f"\nDownloading {len(image_urls)} images...")
        local_paths = download_images(image_urls, attachments_dir)

    # Generate Markdown
    md_content = generate_markdown(data, text, image_urls, local_paths, args.url)

    # Save Markdown
    md_filename = f"{date_str}-{safe_title}.md"
    md_path = output_dir / md_filename
    md_path.write_text(md_content, encoding="utf-8")

    print(f"\n✓ Saved: {md_path}")
    if local_paths:
        print(f"✓ Images: {attachments_dir} ({len(local_paths)} downloaded)")

    return md_path


if __name__ == "__main__":
    main()

#!/usr/bin/env python3
"""
Fetch Twitter/X post content using jina.ai API.

Usage:
    python fetch_tweet.py <tweet_url> [output_file]

Example:
    python fetch_tweet.py https://x.com/dabit3/status/2009131298250428923 tweet.md

Requires:
    JINA_API_KEY environment variable set with your Jina.ai API key
"""

import argparse
import os
import subprocess
import sys
from pathlib import Path


def fetch_tweet(url: str, output_file: str = None) -> str:
    """Fetch tweet content using jina.ai API via curl."""
    if not url.startswith(("https://x.com/", "https://twitter.com/")):
        raise ValueError("URL must be from x.com or twitter.com (HTTPS only)")

    api_key = os.getenv("JINA_API_KEY")
    if not api_key:
        raise RuntimeError(
            "JINA_API_KEY environment variable is not set. "
            "Get your API key from https://jina.ai/ and set: "
            "export JINA_API_KEY='your_api_key_here'"
        )

    jina_api_url = f"https://r.jina.ai/{url}"

    cmd = [
        "curl", "-s", jina_api_url,
        "-H", f"Authorization: Bearer {api_key}"
    ]

    result = subprocess.run(cmd, capture_output=True, text=True)

    if result.returncode != 0:
        detail = result.stderr.strip() or f"curl exited with status {result.returncode}"
        raise RuntimeError(f"Failed to fetch tweet: {detail}")

    content = result.stdout

    if output_file:
        output_path = Path(output_file)
        output_path.write_text(content, encoding="utf-8")
        print(f"Saved to {output_file}")

    return content


def main() -> int:
    """Run the command-line interface and return its process exit code."""
    parser = argparse.ArgumentParser(description="Fetch Twitter/X post content")
    parser.add_argument("url", help="Twitter/X post URL")
    parser.add_argument("output", nargs="?", help="Optional output file path")
    args = parser.parse_args()

    try:
        content = fetch_tweet(args.url, args.output)
    except (OSError, RuntimeError, ValueError) as exc:
        print(f"Error: {exc}", file=sys.stderr)
        return 1

    if not args.output:
        print(content)

    return 0


if __name__ == "__main__":
    sys.exit(main())

#!/bin/bash
# Fetch Twitter/X post content using jina.ai API
# Usage: ./fetch_tweets.sh <url1> [url2] ...
# Requires: JINA_API_KEY environment variable

if [ -z "$JINA_API_KEY" ]; then
    echo "Error: JINA_API_KEY environment variable is not set" >&2
    echo "Get your API key from https://jina.ai/ and set:" >&2
    echo "  export JINA_API_KEY='your_api_key_here'" >&2
    exit 1
fi

if [ $# -eq 0 ]; then
    echo "Usage: $0 <tweet_url> [tweet_url2] ..."
    echo "Example: $0 https://x.com/dabit3/status/2009131298250428923"
    exit 1
fi

for url in "$@"; do
    if [[ ! "$url" =~ ^https?://(x\.com|twitter\.com)/ ]]; then
        echo "Skipping invalid URL: $url" >&2
        continue
    fi

    echo "Fetching: $url"
    curl -s "https://r.jina.ai/${url}" \
        -H "Authorization: Bearer ${JINA_API_KEY}"
    echo ""
    echo "---"
    echo ""
done

---
name: twitter-reader
description: Fetch Twitter/X post content including long-form Articles with full images and metadata. Use when Claude needs to retrieve tweet/article content, author info, engagement metrics, and embedded media. Supports individual posts and X Articles (long-form content). Automatically downloads all images to local attachments folder and generates complete Markdown with proper image references. Preferred over Jina for X Articles with images.
---

# Twitter Reader

Fetch Twitter/X post and article content with full media support.

## Reading a single post's text: fxtwitter first (2026-08-30)

For plain post text, prefer the fxtwitter mirror API — login-free, key-free,
works direct, and returns the **full note_tweet body in `tweet.text`** (the
`full_text` key does not exist; a 2,324-char long-form announcement came back
complete):

```bash
curl -sS --max-time 20 "https://api.fxtwitter.com/<user>/status/<id>" \
  | python3 -c "import json,sys; t=json.load(sys.stdin)['tweet']; print(t['created_at']); print(t['text'])"
```

`replies` is a count, not the reply thread. For X Articles with images, use
`fetch_article.py` below — fxtwitter does not carry article bodies.

## Quick Start (Recommended)

For X Articles with images, use the new fetch_article.py script:

```bash
uv run --with pyyaml python scripts/fetch_article.py <article_url> [output_dir]
```

Example:
```bash
uv run --with pyyaml python scripts/fetch_article.py \
  https://x.com/HiTw93/status/2040047268221608281 \
  ./Clippings
```

This will:
- Fetch structured data via `twitter-cli` (likes, retweets, bookmarks)
- Fetch content with images via `jina.ai` API
- Download all images to `attachments/YYYY-MM-DD-AUTHOR-TITLE/`
- Generate complete Markdown with embedded image references
- Include YAML frontmatter with metadata

### Example Output

```
Fetching: https://x.com/HiTw93/status/2040047268221608281
--------------------------------------------------
Getting metadata...
Title: 你不知道的大模型训练：原理、路径与新实践
Author: Tw93
Likes: 1648

Getting content and images...
Images: 15

Downloading 15 images...
  ✓ 01-image.jpg
  ✓ 02-image.jpg
  ...

✓ Saved: ./Clippings/2026-04-03-文章标题.md
✓ Images: ./Clippings/attachments/2026-04-03-HiTw93-.../ (15 downloaded)
```

## Alternative: Jina API (Text-only)

⚠️ **Known reliability risk (2026-08-30 live tests)**: anonymous `r.jina.ai`
access to x.com gets **403-globally-banned for hours** when *third-party*
users abuse the domain — the ban blocks every anonymous caller, then
expires. Verified working again after expiry (anonymous fetch then returns
post text), so treat Jina as **intermittent**, never a load-bearing path.
The shared key in this repo is also currently out of balance (402
InsufficientBalanceError), which makes `fetch_tweets.sh` — it hard-requires
`JINA_API_KEY` — unusable until recharged.

For simple text-only fetching:

```bash
# Single tweet
curl "https://r.jina.ai/https://x.com/USER/status/TWEET_ID" \
  -H "Authorization: Bearer ${JINA_API_KEY}"

# Batch fetching
scripts/fetch_tweets.sh url1 url2 url3
```

## Features

### Full Article Mode (fetch_article.py)
- ✅ Structured metadata (author, date, engagement metrics)
- ✅ Automatic image download (all embedded media)
- ✅ Complete Markdown with local image references
- ✅ YAML frontmatter for PKM systems
- ✅ Handles X Articles (long-form content)

### Simple Mode (Jina API)
- Text-only content
- Intermittent availability (see risk note above); batch script hard-requires `JINA_API_KEY`
- Good for quick text extraction when it's up

## Prerequisites

### For Full Article Mode
- `uv` (Python package manager)
- No additional setup (twitter-cli auto-installed)

### For Simple Mode (Jina)
```bash
export JINA_API_KEY="your_api_key_here"
# Get from https://jina.ai/
```

## Output Structure

```
output_dir/
├── YYYY-MM-DD-article-title.md       # Main Markdown file
└── attachments/
    └── YYYY-MM-DD-author-title/
        ├── 01-image.jpg
        ├── 02-image.jpg
        └── ...
```

## What Gets Returned

### Full Article Mode
- **YAML Frontmatter**: source, author, date, likes, retweets, bookmarks
- **Markdown Content**: Full article text with local image references
- **Attachments**: All downloaded images in dedicated folder

### Simple Mode
- **Title**: Post author and content preview
- **URL Source**: Original tweet link
- **Published Time**: GMT timestamp
- **Markdown Content**: Text with remote media URLs

## URL Formats Supported

- `https://x.com/USER/status/ID` (posts)
- `https://x.com/USER/article/ID` (long-form articles)
- `https://twitter.com/USER/status/ID` (legacy)

## Scripts

### fetch_article.py
Full-featured article fetcher with image download:
```bash
uv run --with pyyaml python scripts/fetch_article.py <url> [output_dir]
```

### fetch_tweet.py
Simple text-only fetcher using Jina API:
```bash
python scripts/fetch_tweet.py <tweet_url> [output_file]
```

### fetch_tweets.sh
Batch fetch multiple tweets (Jina API):
```bash
scripts/fetch_tweets.sh <url1> <url2> ...
```

## Migration from Jina API

Old workflow:
```bash
curl "https://r.jina.ai/https://x.com/..."
# Manual image extraction and download
```

New workflow:
```bash
uv run --with pyyaml python scripts/fetch_article.py <url>
# Automatic image download, complete Markdown
```