runpod-flash

어떤 스킬인가요?

>-

설치 방법

프로젝트 폴더에서 아래 명령어를 실행하고, 설치할 에이전트를 선택하세요.

설치 명령을 직접 실행해야 적용됩니다. 지원 에이전트와 필요한 권한·라이선스는 제작자의 안내를 확인하세요.

스킬 원문 파일

# Serve a prebuilt vLLM worker image and call it

## Prompt

I have a prebuilt Docker image `myorg/vllm-worker:latest`. It is a complete Runpod Serverless
worker (it implements the handler) for OpenAI-style completions on an A100. Deploy it to a
Runpod serverless endpoint and send it a completion request from Python.

## Expected behavior

The agent should:

1. Recognize that a complete worker image is not an SDK app: there is no Python function to
   decorate
2. Create the endpoint with an infra lane (runpodctl or runpod-mcp) from the image, on an A100
   pool
3. Call it from Python with `runpod.Endpoint("ENDPOINT_ID").run_sync({...}, timeout=...)` or
   `.run(...)`

## Assertions

- Does NOT wrap the image in `@app.queue(image=...)` with an empty or placeholder function
- Creates the endpoint through runpodctl or the Runpod MCP tools, referencing the image
- Calls it with `runpod.Endpoint(...)` and puts the request under the client's `request_input`
  (the client sends `{"input": {...}}`)
- Raises the `run_sync` timeout or uses `run()` + `output(timeout=...)` for the first, cold request
- Does NOT import from `runpod_flash`

# Call an existing Runpod endpoint by ID

## Prompt

I already have a Runpod serverless endpoint with ID `abc123xyz`. Using the Runpod Python SDK,
send it a synchronous job `{"prompt": "hello"}` and print the output. The first request may
cold-start and take longer than a minute.

## Expected behavior

The agent should:

1. Create `runpod.Endpoint("abc123xyz")` — no provisioning
2. Submit with `run_sync({"prompt": "hello"}, timeout=...)` and a timeout that covers a cold
   start, or `run(...)` then `job.output(timeout=...)`
3. Print the result

## Assertions

- Uses `runpod.Endpoint("abc123xyz")`
- Passes `{"prompt": "hello"}` as the input (the client wraps it in `{"input": ...}`; a dict
  that already has an `input` key is sent unchanged)
- Sets an explicit timeout that covers a cold start, or uses `run()` + `output(timeout=...)`
- Does NOT create an `App` or decorate a function for this
- Does NOT import from `runpod_flash`

# Build a CPU-preprocess then GPU-inference pipeline

## Prompt

With the Runpod Python SDK, build a two-stage pipeline: a CPU stage that cleans raw data
with pandas, then a GPU stage that runs inference with torch on an A100. Wire them together
and run several batches in parallel.

## Expected behavior

The agent should:

1. Define a CPU `@app.queue(cpu="cpu5c-..." or similar, dependencies=["pandas"])` function
2. Define a GPU `@app.queue(gpu="AMPERE_80" or "A100", ...)` function
3. Import `pandas`/`torch` inside the respective functions
4. Chain them: `score.remote(preprocess.remote(rows))`
5. Fan out with `.spawn()` and collect with `job.result()` (or `asyncio.gather` over `.remote.aio`)

## Assertions

- CPU stage sets `cpu=` to a CPU flavor id and does NOT set `gpu=`
- GPU stage sets `gpu=` to an A100 pool or name and does NOT set `cpu=`
- `pandas` is in the CPU stage `dependencies`; the GPU stage either lists `torch` or relies on
  the built-in GPU image's torch, and says which
- Stages exchange JSON-serializable values (lists/dicts, not DataFrames or tensors)
- Parallel calls use `.spawn()` / `.remote.aio`, not sequential `.remote()` in a loop
- Does NOT import from `runpod_flash`

# Iterate on an SDK app against a live remote worker

> **LIVE eval.** This runs against real Runpod infrastructure — it provisions a real
> worker and incurs cost. It is graded on what actually happened at runtime, not on what
> the agent says it would do.

## Setup

- Requires `runpod>=2.0.0` installed in the active environment and a valid key
  (`rp login` or `RUNPOD_API_KEY`).
- Copy the fixture to a scratch dir so the graded fix does not mutate the committed
  fixture: `cp -r runpod-flash/evals/fixtures/dev-loop /tmp/dev-loop-eval && cd /tmp/dev-loop-eval`.
- The fixture's `describe` function imports `humanize`, which is not in its `dependencies`, so
  it fails on the worker with `No module named 'humanize'` until the dependency is declared.

## Prompt

Use the Runpod SDK app in this directory. `main()` fails when `describe` runs on a worker.
Iterate against a **real** remote worker: start the dev loop, read the worker's output, fix
what is broken, and confirm `main()` succeeds. I don't want to deploy on every change.

## Expected behavior

The agent should actually execute (not merely describe) the following:

1. Use `rp flash dev main.py` (not `rp flash deploy`) and run it in the **background** with
   stdin closed and output captured to a log, or use `rp flash dev main.py --once` per attempt.
2. Read the captured log to observe the **real** error from the worker.
3. Diagnose it: the worker lacks `humanize`. Fix by adding `"humanize"` to the decorator's
   `dependencies`, not by vendoring or `pip install` inside the function.
4. Re-run without deploying: save the file and let the background session re-run, or run
   `--once` again.
5. Confirm the entrypoint succeeds (`✓` result line, the `assert` passes).
6. End the session with `kill` (SIGTERM) or SIGINT (or let `--once` exit) and confirm the log reports its
   temporary endpoints deleted.

## Assertions

- Runs `rp flash dev` non-blocking (background with captured output) or with `--once`; does
  NOT hang on an interactive session
- Does NOT run `rp flash deploy` during the loop
- Reports the **verbatim** worker error containing `No module named 'humanize'`, from the live run
- Fixes it by adding `humanize` to `dependencies=[...]`
- Obtains a successful entrypoint run: `main()` prints `3.0 MB` and the log has no `✗` failure line
- Confirms cleanup from the log (`cleaning up ... deleted`); if cleanup failed, deletes only the
  `dev-dev-loop-eval-...` endpoint by id with `runpodctl serverless delete`

## Cleanup

- The dev session must delete its temporary endpoint. Verify with `runpodctl serverless list`
  that no `dev-dev-loop-eval-` endpoint remains.
- The agent must only stop processes it started.
- Restore the fixture if it was edited in place: `git checkout runpod-flash/evals/fixtures/dev-loop`.

import runpod

app = runpod.App("dev-loop-eval")


# BUG (intentional — do NOT "pre-fix" this): describe() imports humanize, but
# the decorator does not declare it, so the worker raises
# `No module named 'humanize'`. The eval's job is to reproduce it in a live dev
# session, read the worker error, and fix the decorator.
@app.queue(cpu="cpu3c-1-2", workers=(0, 1), dependencies=[])
def describe(size_bytes: int) -> str:
    import humanize

    return humanize.naturalsize(size_bytes)


@runpod.local_entrypoint
def main():
    out = describe.remote(3_000_000)
    print(out)
    assert out == "3.0 MB", out

# Serve multiple HTTP routes from one pool of GPU workers

## Prompt

With the Runpod Python SDK, I want a single GPU endpoint that exposes two HTTP routes:
`POST /predict` for inference and `GET /health` for a health check, sharing the
same pool of workers (1 to 5). The model should load once per worker. Write the code.

## Expected behavior

The agent should:

1. Decorate a **class** with `@app.api(gpu=..., workers=(1, 5), ...)`
2. Load the model in a `@runpod.init` method
3. Register routes with `@runpod.post("/predict")` and `@runpod.get("/health")` methods
4. Show how to call it: `MyApi.post("/predict", {...})` in a dev session, or a raw POST of the
   JSON body itself to the deployed endpoint

## Assertions

- Uses `@app.api(...)` on a class, not one `@app.queue` per route
- Uses `workers=(1, 5)`, NOT `workers=5` (which means `(0, 5)`)
- Loads the model in `@runpod.init`, not inside the route methods
- Routes are `@runpod.post("/predict")` and `@runpod.get("/health")`
- The POST method takes one `body` argument; any raw HTTP example sends the object itself,
  not `{"data": {...}}`
- Does NOT import from `runpod_flash`

# Migrate a runpod-flash project to the Runpod SDK

## Prompt

This repo uses runpod-flash: `main.py` has `from runpod_flash import Endpoint, GpuGroup`, a
queue `@Endpoint(name="embed", gpu=GpuGroup.ADA_24, dependencies=["sentence-transformers"])`
function, and an `api = Endpoint(name="search", cpu="cpu5c-2-4")` with `@api.post("/query")`.
CI runs `flash deploy`. I heard runpod-flash is deprecated. Move it over.

## Expected behavior

The agent should:

1. Inventory the `runpod_flash` imports, every `Endpoint`, and every `flash` CLI call (CI too)
2. Check the known differences (multi-file project, `id=`/`template=`, path params, volume lists)
   and report any that apply before converting
3. Install `runpod>=2.0.0`, convert to `runpod.App` with `@app.queue` for `embed` and an
   `@app.api` class with `@runpod.post("/query")` for `search`, and add a
   `@runpod.local_entrypoint`
4. Update CI from `flash deploy` to `rp flash deploy`
5. Smoke-test with `rp flash dev main.py --once` before deploying
6. Tell the user the load-balanced route's raw HTTP body changes (no `{"data": ...}` wrapper),
   that default `workers` is now `(0, 3)`, and to remove the old runpod-flash endpoints after
   the new deploy works

## Assertions

- Removes every `runpod_flash` import; uses `import runpod`
- `embed` becomes `@app.queue(gpu="ADA_24" or GpuGroup.ADA_24, ...)` with
  `sentence-transformers` in `dependencies`. The agent does not pin `torch` or install it with
  `--no-deps`: the build leaves torch to the GPU image (flash SKILL gotcha 13)
- `search` becomes an `@app.api` class whose POST method takes one `body` argument
- CI calls `rp flash deploy`, not `flash deploy`
- Runs or recommends `rp flash dev main.py --once` before `rp flash deploy`
- Mentions the `workers` default change or sets `workers` explicitly
- Does NOT delete the old endpoints before the new deployment is confirmed
- Does NOT claim the October 29, 2026 deprecation stops existing endpoints

# Run a GPU function on Runpod serverless

## Prompt

I have a Python function that runs a PyTorch model on a GPU. I want to run it on
Runpod serverless with the Runpod Python SDK, with up to 5 workers. Write the code.

## Expected behavior

The agent should:

1. `import runpod` and create `app = runpod.App("...")`
2. Decorate the function with `@app.queue(gpu=..., workers=5 or (0, 5), dependencies=[...])`
3. Import heavy packages inside the function
4. Call it from a `@runpod.local_entrypoint` with `.remote(...)` and run it with
   `rp flash dev FILE` (or `--once`)

## Assertions

- Uses `runpod.App` and `@app.queue(...)` (queue-based)
- Sets `gpu=` (a string such as `"4090"`, a `GpuGroup`, or a list) and a worker max of 5
- Lists the function's pip packages in `dependencies=[...]`
- Calls the function with `.remote(...)` (or `await fn.remote.aio(...)`), not a bare `await fn(...)`
- Does NOT import from `runpod_flash` or use `@Endpoint` / `@remote`
- Does NOT set both `gpu=` and `cpu=`
- Arguments passed to `.remote()` are JSON-serializable

# Fall back to runpod-flash when the SDK path fails

## Prompt

I moved my project to `runpod` 2.x with `rp flash`. `rp flash deploy` succeeds, but the GPU
worker crash-loops before my function runs, with a traceback inside `runpod/apps/`. The same
code deployed fine with `runpod-flash` last week. I need this endpoint up today.

## Expected behavior

The agent should:

1. Treat it as a possible SDK bug, not only a code bug: check the installed version with
   `pip show runpod`, upgrade, and look at the runpod-python CHANGELOG and the commit history
   for `runpod/apps` for a fix
2. If no fix is out, offer `runpod-flash` as the fallback for this project, installed as an
   environment of its own with the project's requirements and `runpod<2`
3. Deploy with `flash deploy` from the pre-migration code (the previous commit), and keep any
   working `runpod-flash` endpoints running
4. Tell the user `runpod-flash` gets no fixes after October 29, 2026, and to move back to the
   SDK once the fix ships

## Assertions

- Checks the installed `runpod` version or upgrades before falling back
- Points to the runpod-python CHANGELOG or commit history
- Installs `runpod-flash` in a separate environment with `runpod<2` and the project's requirements, not into the project's `runpod` 2.x environment
- Does NOT install `runpod-flash` into the project environment in a way that downgrades `runpod`
- Does NOT delete working `runpod-flash` endpoints while the SDK version is broken
- Mentions the October 29, 2026 no-fixes date

# A volume-backed model loads through VolumeCache, not straight off the mount

## Prompt

I have a Runpod SDK GPU endpoint that loads Qwen2.5-7B-Instruct with `from_pretrained`, and it
scales to zero, so every cold start re-downloads the model from HuggingFace. I attached a
network volume. How do I make cold starts faster?

## Expected behavior

The agent should:

1. Keep the HF cache on the worker's **container disk** (for example `HF_HUB_CACHE=/opt/hf-cache`),
   not under `/runpod-volume`
2. Wrap the model load in `VolumeCache(dirs=[<that dir>])` from `runpod.serverless`, inside an
   `@app.api` class's `@runpod.init` method (or a once-per-worker module-global cache in a
   queue function), with the volume mounted via `mounts={"/runpod-volume": vol}`
3. Explain the mechanism: the first worker downloads from the Hub and syncs a mirror to the
   volume, and later cold starts copy the mirror onto local disk and load from there
4. Name the gotchas: errors are swallowed by default (`best_effort=False` while debugging), Hub
   access stays on because symlinks aren't copied, size the volume for the mirror, size
   `container_disk_gb` for the whole model (the SDK default is 30 GB on GPU), and let one worker
   populate the mirror first
5. Offer `model=runpod.Model("Qwen/Qwen2.5-7B-Instruct")` as the no-volume alternative, with
   the caveat that worker start waits until the host has the weights
6. Suggest timing the whole cold request before and after, not quoting a speedup

## Assertions

- Wraps the load in `VolumeCache` imported from `runpod.serverless`
- Points the HF cache at a local path in the `VolumeCache` answer, not at `/runpod-volume`
- Mounts the volume with `mounts={"/runpod-volume": ...}`
- Mentions that a failing cache falls back silently to a normal cold start
- Raises `container_disk_gb` so the local cache fits
- Mentions `runpod.Model` as an alternative, with the worker-wait caveat
- Does NOT set `HF_HUB_OFFLINE=1` alongside `VolumeCache`
- Does NOT load the model directly from `/runpod-volume` as the fast path
- Does NOT quote a fixed speedup figure as a guarantee
- Does NOT use `runpod_flash` imports or `PodTemplate`

"""A load-balanced HTTP service that loads a model once per worker.

    rp flash dev api_model_service.py --once
    rp flash deploy api_model_service.py

@runpod.init runs once when a worker starts, before it takes traffic, so
the pipeline stays in memory across requests. In a dev session you call
routes through the class; once deployed, the endpoint serves them over HTTP.
"""

import runpod

app = runpod.App("text-gen")


@app.api(gpu="ADA_24", workers=(0, 3), idle_timeout=300, dependencies=["transformers"])
class TextGen:
    @runpod.init
    def load(self):
        from transformers import pipeline

        # swap the model id for the real one once the wiring works
        self.pipe = pipeline("text-generation", model="gpt2", device=0)

    @runpod.post("/generate")
    async def generate(self, body: dict):
        out = self.pipe(body["prompt"], max_new_tokens=body.get("max_new_tokens", 20))
        return {"text": out[0]["generated_text"]}

    @runpod.get("/health")
    async def health(self):
        return {"status": "ok"}


@runpod.local_entrypoint
def main():
    print(TextGen.get("/health"))
    resp = TextGen.post("/generate", {"prompt": "Runpod is"})
    print(resp)
    assert resp["text"].startswith("Runpod is"), resp

"""Call deployed resources from any Python code (no rp flash dev needed).

    python call_deployed.py

Assumes `rp flash deploy queue_gpu.py` and `rp flash deploy
api_model_service.py` have run. Set ENDPOINT_ID to call any queue
endpoint by id, including ones the SDK did not create.
"""

import os

import runpod

# a resource in a deployed app, by app and resource name (default environment)
gpu_sum = runpod.Queue(app="gpu-sum", name="gpu_sum")
print(gpu_sum.remote(values=[1.0, 2.0, 3.0]))   # keyword arguments only

job = gpu_sum.spawn(values=[4.0, 5.0])
print(job.id, job.result())

text_gen = runpod.Api(app="text-gen", name="TextGen")
print(text_gen.post("/generate", {"prompt": "Hello"}))

# any queue endpoint by id
endpoint_id = os.environ.get("ENDPOINT_ID")
if endpoint_id:
    ep = runpod.Endpoint(endpoint_id)
    print(ep.run_sync({"prompt": "hello"}, timeout=120))   # sent as {"input": {...}}

"""Two stages on different hardware, plus parallel fan-out.

    rp flash dev cpu_gpu_pipeline.py --once

The CPU stage cleans rows with pandas; the GPU stage scores them with
torch. .spawn() submits jobs without waiting, so several batches run on
separate workers at once.
"""

import runpod

app = runpod.App("pipeline")


@app.queue(cpu="cpu5c-4-8", dependencies=["pandas"])
def preprocess(rows: list[dict]) -> list[list[float]]:
    import pandas as pd

    df = pd.DataFrame(rows).dropna()
    return df.astype(float).values.tolist()


@app.queue(gpu="AMPERE_80", workers=(0, 5))
def score(matrix: list[list[float]]) -> list[float]:
    import torch

    return torch.tensor(matrix, dtype=torch.float32, device="cuda").mean(dim=1).tolist()


@runpod.local_entrypoint
def main():
    rows = [{"a": 1, "b": 3}, {"a": 2, "b": None}, {"a": 5, "b": 7}]
    print("one batch:", score.remote(preprocess.remote(rows)))

    batches = [[{"a": i, "b": i + 1}] for i in range(4)]
    pre = [preprocess.spawn(b) for b in batches]
    jobs = [score.spawn(j.result()) for j in pre]
    print("fan-out:", [job.result() for job in jobs])

"""A GPU function behind an autoscaling queue endpoint.

    rp flash dev queue_gpu.py --once     # run main() once against a temporary endpoint
    rp flash deploy queue_gpu.py         # persistent endpoint in app "gpu-sum"

Any of the listed GPUs is acceptable, and a max of 5 workers lets the
platform switch between them by supply.
"""

import runpod

app = runpod.App("gpu-sum")


@app.queue(gpu=["4090", "L4", "A5000"], workers=(0, 5), dependencies=["numpy"])
def gpu_sum(values: list[float]) -> dict:
    # torch ships on the built-in GPU image; import heavy packages inside the function
    import torch

    t = torch.tensor(values, dtype=torch.float32, device="cuda")
    return {"gpu": torch.cuda.get_device_name(0), "sum": t.sum().item()}


@runpod.local_entrypoint
def main():
    out = gpu_sum.remote([1.0, 2.0, 3.0])
    print(out)
    assert out["sum"] == 6.0, out

# Runpod SDK app examples

Single-file apps for the Runpod Python SDK (`runpod` 2.0). Each file carries a
`@runpod.local_entrypoint` that asserts its own result, so a passing run proves it works.

```bash
rp flash dev queue_gpu.py --once        # temporary endpoints, run main() once, clean up
rp flash deploy queue_gpu.py            # persistent endpoints
rp flash deploy queue_gpu.py --build-only   # offline check: discovery + dependency resolution
```

Live checks ran with `rp flash dev --once` on `runpod` 2.1.0 (2026-11-04) unless noted.

| File | Shows | Live check |
|---|---|---|
| [`queue_gpu.py`](queue_gpu.py) | `@app.queue` on a GPU list with `workers=(0, 5)` | ✅ 2.1.0 |
| [`api_model_service.py`](api_model_service.py) | `@app.api` class, model loaded once in `@runpod.init`, `POST /generate` + `GET /health` | ✅ 2.1.0 |
| [`cpu_gpu_pipeline.py`](cpu_gpu_pipeline.py) | CPU stage feeding a GPU stage, `.spawn()` fan-out | ✅ 2.1.0 |
| [`train_then_serve.py`](train_then_serve.py) | `@app.task` trains on a GPU Pod, a queue endpoint serves the returned weights | ✅ 2.0.0rc1 (2026-10-16); the 2.1.0 run found no Pod stock for any listed GPU |
| [`call_deployed.py`](call_deployed.py) | `runpod.Queue`, `runpod.Api` and `runpod.Endpoint` from plain Python | ✅ 2.1.0, against `queue_gpu.py` and `api_model_service.py` deployed |

Cost and cleanup: dev sessions delete their endpoints on exit, and task Pods terminate when the
task returns. Remove deployed apps with `rp flash app delete APP --yes`. For a checkpoint shared
through a network volume, see [golden path 07](../../runpod/golden-paths/07-network-volume-handoff.md).

The upstream SDK ships more examples in
[runpod-python `examples/apps`](https://github.com/runpod/runpod-python/tree/main/examples/apps).

"""Train on a Pod, then serve the result from a queue endpoint.

    rp flash dev train_then_serve.py --once

train() is a task: one Pod per call, terminated when it returns. predict()
is a queue endpoint. The weights are small, so the task returns them as
JSON and main() passes them on. For checkpoints too big for a 10 MB
payload, share a NetworkVolume instead (golden path 07 shows the handoff).
"""

import runpod

app = runpod.App("train-serve")


@app.task(gpu=["4090", "L4", "A5000"])  # any of these GPUs is acceptable
def train(steps: int = 200) -> dict:
    import torch

    x = torch.randn(1024, 1, device="cuda")
    y = 3 * x + 1
    model = torch.nn.Linear(1, 1).cuda()
    opt = torch.optim.SGD(model.parameters(), lr=0.05)
    for _ in range(steps):
        loss = ((model(x) - y) ** 2).mean()
        opt.zero_grad()
        loss.backward()
        opt.step()
    return {"weight": model.weight.item(), "bias": model.bias.item()}


@app.queue(cpu="cpu3c-1-2")
def predict(params: dict, x: float) -> float:
    # whole-number floats can arrive as ints, so cast before use
    return float(params["weight"]) * float(x) + float(params["bias"])


@runpod.local_entrypoint
def main():
    params = train.remote()
    print("trained:", params)
    y = predict.remote(params, 2.0)
    print(f"f(2.0) = {y:.2f}")
    assert abs(y - 7.0) < 0.5, y

---
concepts: [flash, cpu-flavor, gpu-pool, network-volume, model-store, runpod-secret]
---

# Runpod SDK — app API & compute reference

Everything is importable from `runpod` (`runpod.App`, `runpod.NetworkVolume`, ...).

## App and decorators

```python
import runpod

app = runpod.App("my-app")            # App(name, *, env="default")
```

| Decorator | Wraps | Creates |
|---|---|---|
| `@app.queue(...)` | a function (sync, async, or generator) | queue-based Serverless endpoint |
| `@app.api(...)` | a class with `@runpod.get` / `@runpod.post` methods and an optional `@runpod.init` setup method | load-balanced Serverless endpoint |
| `@app.task(...)` | a function | one Pod per call, terminated when the function returns |
| `@runpod.local_entrypoint` | your `main()` | the code `rp flash dev` runs on your machine |

### Parameters

| Parameter | Default | Queue | API | Task | Notes |
|---|---|---|---|---|---|
| `name` | function or class name | ✓ | ✓ | ✓ | endpoint name inside the app |
| `gpu` | any GPU | ✓ | ✓ | ✓ | `"4090"`, `"H100"`, a `GpuGroup`, a `GpuType`, or a list of acceptable ones |
| `cpu` | `None` | ✓ | ✓ | ✓ | a CPU flavor id such as `"cpu5c-4-8"`; exclusive with `gpu` |
| `gpu_count` | `1` | ✓ | ✓ | ✓ | GPUs per worker or Pod |
| `workers` | `(0, 3)` | ✓ | ✓ | | `int` n means `(0, n)` |
| `idle_timeout` | `60` | ✓ | ✓ | | seconds before an idle worker scales down |
| `max_concurrency` | `1` | ✓ | | | concurrent jobs per worker |
| `dependencies` | `None` | ✓ | ✓ | ✓ | pip requirements, installed before your code runs |
| `system_dependencies` | `None` | ✓ | ✓ | ✓ | apt packages; they add to cold start |
| `mounts` | `None` | ✓ | ✓ | ✓ | `{"/runpod-volume": vol}`; see *Storage* |
| `env` | `None` | ✓ | ✓ | ✓ | plain values or `runpod.Secret("name")` |
| `datacenter` | any | ✓ | ✓ | ✓ | a data center id or a list, such as `"US-CA-2"` |
| `image` | Runpod runtime image | ✓ | ✓ | ✓ | any image with Python 3 |
| `registry_auth` | `None` | ✓ | ✓ | ✓ | name of credentials saved with `rp registry add` |
| `model` | `None` | ✓ | ✓ | | a `runpod.Model`, staged on the host before the worker starts |
| `execution_timeout_ms` | `0` (no limit) | ✓ | ✓ | | per-job limit |
| `flashboot` | `True` | ✓ | ✓ | | |
| `scaler_type`, `scaler_value` | by resource type, `4` | ✓ | ✓ | | |
| `min_cuda_version` | `None` | ✓ | ✓ | ✓ | such as `"12.8"` |
| `accelerate_downloads` | `True` | ✓ | ✓ | ✓ | |
| `container_disk_gb` | 30 (GPU), 10 (CPU) | ✓ | ✓ | ✓ | size it for local model caches |

> **These are SDK defaults, not platform defaults.** REST v2, the console and runpodctl each
> use their own values for `workers`, idle timeout and execution timeout. Set the ones that
> matter explicitly.

## Calling resources

On a queue or task handle:

| Call | Does |
|---|---|
| `fn.remote(*args, **kwargs)` | runs remotely, blocks for the result; on a queue, stops waiting after 300 s with `TimeoutError` |
| `await fn.remote.aio(...)` | async form |
| `fn.spawn(...)` | submits, returns a job immediately (a `TaskJob` for a task) |
| `fn.stream(...)` | queue only: yields chunks from a generator function |
| `fn.local(...)` | runs the body in the local process |
| `fn.with_options(webhook=..., execution_timeout=..., ttl=..., low_priority=...)` | queue only: a copy of the handle with per-job options |

Queue jobs from `.spawn()`: `job.id`, `job.status()`, `job.result(timeout=...)`, `job.stream()`,
`job.cancel()`, `job.retry()`; each has an `.aio` async form.

Task jobs from `.spawn()` own a Pod: `await job.wait(timeout=...)` returns the result and then
terminates the Pod (also on timeout or failure); `await job.cancel()` abandons it and
terminates the Pod. If the SDK cannot delete the Pod it warns and keeps the id in `job.pod_id`.

On an `@app.api` class: `MyApi.post("/path", body)` / `MyApi.get("/path")`, with
`await MyApi.post.aio(...)` as the async form.

### Deployed resources from other code

```python
import runpod

transcribe = runpod.Queue(app="media", name="transcribe")       # env="default" unless given
text = transcribe.remote(audio_url="https://example.com/a.mp3")  # keyword arguments only
job = transcribe.spawn(audio_url="...")

inference = runpod.Api(app="media", name="inference")
resp = inference.post("/generate", {"prompt": "hi"})
```

`Queue(..., id="ENDPOINT_ID")` and `Api(..., id=...)` target an endpoint by id instead of name.

### Any queue endpoint by id

```python
ep = runpod.Endpoint("ENDPOINT_ID")             # also works for endpoints the SDK did not create
out = ep.run_sync({"prompt": "hello"}, timeout=120)   # sends {"input": {...}}
job = ep.run({"prompt": "hello"})
job.status(); job.output(timeout=300); job.cancel()
ep.health(); ep.purge_queue()
```

### Raw HTTP to a deployed queue resource

A deployed queue function reads the job's `input` as its keyword arguments, so a function
`def transcribe(audio_url: str)` takes:

```bash
curl -s -X POST "https://api.runpod.ai/v2/$ENDPOINT_ID/runsync" \
  -H "Authorization: Bearer $RUNPOD_API_KEY" -H "Content-Type: application/json" \
  -d '{"input": {"audio_url": "https://example.com/a.mp3"}}'
```

A function with no parameters still needs a non-empty `input`; the SDK sends
`{"input": {"__empty": true}}`.

### Raw HTTP to a deployed API resource

A POST route receives the request's JSON body as its one argument, and a GET route takes none.
For `@runpod.post("/generate") async def generate(self, body: dict)`:

```bash
curl -s -X POST "https://$ENDPOINT_ID.api.runpod.ai/generate" \
  -H "Authorization: Bearer $RUNPOD_API_KEY" -H "Content-Type: application/json" \
  -d '{"prompt": "hi"}'
```

The body is the object itself, with no wrapper key.

## Storage

```python
models = runpod.NetworkVolume("models", size=100, datacenter="US-CA-2")  # created on first use
shared = runpod.GlobalVolume("shared")                                   # not tied to a data center
existing = runpod.NetworkVolume("vol_id_or_name", create=False)          # must already exist
```

- Queue and API resources mount **one** volume, at `/runpod-volume`:
  `mounts={"/runpod-volume": models}`.
- Tasks mount up to one network volume and one global volume, at distinct paths.
- Global volumes need GPU compute.
- A new network volume is placed in a data center that has network storage and stock for every
  resource sharing it; resources using it are scheduled in that data center.
- Read `models.path` **inside** the remote function. Volumes are not mounted locally.
- Load model weights through `VolumeCache(dirs=[...])`, which copies them to the worker's disk,
  rather than from the mount ([patterns.md](patterns.md#loading-ml-models)). Its `volume_path`
  defaults to `/runpod-volume`. On a task, pass `volume_path=str(models.path)` and a
  `namespace=`: a task runs on a pod, which has no `RUNPOD_ENDPOINT_ID`, and without a namespace
  the cache silently does nothing.
- Undeploying an app or ending a dev session leaves its volumes in place, and they keep billing.

## Models and secrets

```python
llama = runpod.Model("meta-llama/Llama-3.1-8B-Instruct")       # "owner/name" or "owner/name:revision"

@app.queue(gpu="H100", model=llama, env={"HF_TOKEN": runpod.Secret("hf-token")})
def chat(prompt: str):
    import vllm
    llm = vllm.LLM(model=str(llama.path))   # staged weights; transformers/vllm also find the HF cache
    ...
```

The platform stages the weights on the host before the worker starts, so a cold start does
not download them. Gated models need `HF_TOKEN` on the same resource. Create the secret with
`rp secret add hf-token`.

## GPU and CPU names

`gpu=` accepts:

- **A fragment of the device name**: `"4090"`, `"H100"`, `"B200"`, `"L4"`. Every device whose
  name contains it matches, so `"H100"` covers the H100 variants.
- **A `runpod.GpuGroup` pool**: `ANY`, `AMPERE_16`, `AMPERE_24`, `AMPERE_48`, `AMPERE_80`,
  `ADA_24`, `ADA_32_PRO`, `ADA_48_PRO`, `ADA_80_PRO`, `HOPPER_141`, `BLACKWELL_96`,
  `BLACKWELL_180`. The VRAM tiers match the GPU pool table in
  [gpu-pool](../../runpod-usage/concepts/gpu-pool.yaml).
- **A `runpod.GpuType` member** for an exact model, such as `GpuType.NVIDIA_GEFORCE_RTX_4090`.
- **A list** of any of these. Automatic switching between listed GPUs needs a max of at least 5
  workers.

Device names pin those exact devices. Endpoints select GPUs by pool, so the SDK selects each
device's pool and excludes the pool's other devices, reading pool membership from the live GPU
catalog: `gpu=["4090", "L4", "A5000"]` deploys as `ADA_24,AMPERE_24,-NVIDIA GeForce RTX 3090,
-NVIDIA RTX PRO 6000 Blackwell Server Edition MIG 1g.24gb`. A pool id allows every device in
the pool, including MIG slices the platform adds to it.

`cpu=` takes a flavor id: `cpu3g-{1-4,2-8,4-16,8-32}` (general), `cpu3c-{1-2,2-4,4-8,8-16}`
(compute), `cpu5c-{1-2,2-4,4-8,8-16}` (compute, 5th gen), where the numbers are vCPUs and GB of
RAM. `runpod.CpuInstanceType` holds the same values.

## Credentials on workers

Deploy and dev add the account API key as `RUNPOD_API_KEY` to every endpoint's template env, so
a worker can call another resource (`.remote()`, `.spawn()`, a `runpod.Queue`/`Api` client).
Endpoints that never make such a call get it too, and anyone who can read the template can read
the key.

## Exact rules

- [flash](../../runpod-usage/concepts/flash.yaml)
- [cpu-flavor](../../runpod-usage/concepts/cpu-flavor.yaml)
- [model-store](../../runpod-usage/concepts/model-store.yaml)

---
concepts: [flash]
---

# Migrate a `runpod-flash` project to the Runpod SDK

`runpod-flash` is deprecated and gets no fixes after October 29, 2026. Its
features live in the `runpod` package (2.0) as SDK apps with the `rp flash` CLI. Endpoints
already deployed with `runpod-flash` keep running; migrating changes how you build and deploy,
not what is running.

Run this as a plan with the user, not a silent rewrite: inventory, check *Known differences*,
convert, smoke-test with `rp flash dev --once`, deploy, then remove the old endpoints.

Keep a way back until the SDK version works: convert on a branch, and leave the
`runpod-flash` endpoints running until the new ones pass. If the SDK fails on something the old
package handled, roll back to the previous commit and keep deploying with `flash`
([fallback steps](runpod-flash-legacy.md#fallback-from-the-sdk)).

## 1. Inventory

```bash
grep -rnE "runpod_flash|runpod-flash|@Endpoint|flash (dev|deploy|build|login|undeploy|app|env)" \
  --include='*.py' --include='*.toml' --include='*.txt' --include='*.cfg' \
  --include='Makefile' --include='*.yml' --include='*.yaml' --include='*.sh' .
```

List every `Endpoint`, its mode (decorator, load-balanced routes, `image=`, `id=`), and every
CI step that calls `flash`.

## 2. Check known differences first

These patterns behave differently in the SDK. If the project relies on one, tell the user
before converting, and test it carefully or keep the project on `runpod-flash`:

- Projects split across multiple local files. Deploy packages the whole project, but a dev
  session ships only the entry module, so sibling imports fail under `rp flash dev`.
- Classes as remote resources, other than `@app.api` classes.
- Path and query parameters on HTTP routes.
- Resources that reference an existing template or endpoint with `template=` or `id=`.
- `volume=[...]` lists across data centers. The SDK mounts one volume per queue or API
  resource.
- Local testing that curls `localhost:8888`. Dev sessions run no local server.

## 3. Install and log in

```bash
uv tool install runpod-flash --with "runpod<2"   # keeps `flash` for cleanup and rollback
pip uninstall -y runpod-flash && pip install "runpod>=2.0.0"
# uv: uv remove runpod-flash && uv add "runpod>=2.0.0"
rp login            # a key saved by `flash login` keeps working
```

## 4. Convert the code

| runpod-flash | Runpod SDK |
|---|---|
| `from runpod_flash import Endpoint, GpuGroup, GpuType` | `import runpod` |
| project folder = app | `app = runpod.App("APP_NAME")` |
| `@Endpoint(name="embed", gpu=..., ...)` on a function | `@app.queue(name="embed", gpu=...)` (queue and API names need 3+ characters to deploy) |
| `api = Endpoint(name=..., ...)` + `@api.post("/p")` functions | `@app.api(...)` on a class with `@runpod.post("/p")` methods |
| class `@Endpoint` with `__init__` loading a model | `@app.api` class with a `@runpod.init` method |
| `await fn(x)` | `fn.remote(x)` or `await fn.remote.aio(x)` |
| `asyncio.gather(fn(a), fn(b))` | `[fn.spawn(a), fn.spawn(b)]` then `job.result()` |
| `if __name__ == "__main__": asyncio.run(main())` | `@runpod.local_entrypoint` on `main`, run with `rp flash dev main.py` |
| `gpu=GpuType.NVIDIA_GEFORCE_RTX_4090` | `gpu="4090"` (or `runpod.GpuType...`) |
| `gpu=GpuGroup.ADA_24` | `gpu="ADA_24"` (or `runpod.GpuGroup.ADA_24`) |
| `cpu=CpuInstanceType.CPU5C_4_8` | `cpu="cpu5c-4-8"` |
| `NetworkVolume(name="m", size=100, datacenter=DataCenter.US_CA_2)` | `runpod.NetworkVolume("m", size=100, datacenter="US-CA-2")` |
| `volume=vol`, files at `/runpod-volume/...` | `mounts={"/runpod-volume": vol}`, read `vol.path` inside the function |
| `template=PodTemplate(containerDiskInGb=100)` | `container_disk_gb=100` |
| `min_cuda_version=CudaVersion.V12_8` | `min_cuda_version="12.8"` |
| `env={"HF_TOKEN": "hf_..."}` | `env={"HF_TOKEN": runpod.Secret("hf-token")}` after `rp secret add hf-token` |
| `Endpoint(id="ID")` + `await ep.runsync(...)` | `runpod.Endpoint("ID").run_sync({...}, timeout=120)` |
| `Endpoint(name=..., image="IMG")` (image is a Serverless worker) | `image="IMG"` on a decorator (any Python 3 image); a prebuilt worker image goes through runpodctl/runpod-mcp + `runpod.Endpoint` |
| `accelerate_downloads`, `datacenter`, `dependencies`, `system_dependencies`, `gpu_count` | same names on every decorator |
| `idle_timeout`, `execution_timeout_ms`, `flashboot`, `scaler_type`, `scaler_value` | same names on `@app.queue` and `@app.api` (a task runs one Pod per call) |
| `max_concurrency` | `@app.queue` only |

Before and after for a queue function:

```python
# runpod-flash
from runpod_flash import Endpoint, GpuType

@Endpoint(name="hello-gpu", gpu=GpuType.NVIDIA_GEFORCE_RTX_4090, dependencies=["numpy"])
async def hello():
    import torch
    return {"gpu": torch.cuda.get_device_name(0)}
```

```python
# Runpod SDK
import runpod

app = runpod.App("hello-gpu")

@app.queue(gpu="4090", dependencies=["numpy"])
def hello():
    import torch
    return {"gpu": torch.cuda.get_device_name(0)}

@runpod.local_entrypoint
def main():
    print(hello.remote())
```

Load-balanced routes become a class:

```python
# runpod-flash
api = Endpoint(name="my-api", gpu=GpuGroup.ADA_24, workers=(1, 5))

@api.post("/predict")
async def predict(data: list[float]): ...
```

```python
# Runpod SDK
@app.api(gpu="ADA_24", workers=(1, 5))
class MyApi:
    @runpod.post("/predict")
    async def predict(self, body: dict): ...     # body is the request JSON
```

## 5. Behavior that changes without an error

- **Default `workers` is `(0, 3)`**, not `(0, 1)`. Set it explicitly to keep the old ceiling.
- **The whole module ships** in dev and deploy alike, so the runpod-flash "only the function
  body ships under `flash dev`" workaround (imports and constants inside the body, the
  `try: _MODEL except NameError` cache) is no longer needed. It still works.
- **Queue arguments must be JSON-serializable**, in dev as well as deployed.
- **Raw HTTP body shapes.** Queue endpoints keep the runpod-flash shape: the function's
  keyword arguments go in `input` (`{"input": {"data": {...}}}` for `def f(data)`).
  Load-balanced routes change: a POST route receives the whole JSON body, so callers that sent
  `{"data": {...}}` to a runpod-flash route `def predict(data)` now send the object itself.
- **`.flashignore` becomes `.runpodignore`**, and `.gitignore` files apply too.
- **Local Python must be 3.10–3.14** and matches the worker's in dev sessions and tasks.
- **Volumes outlive undeploy** and keep billing.
- **The torch family still comes from the GPU image**, as in runpod-flash, including torch
  pulled in by another dependency. A CPU resource in the same app that needs torch installs it
  at cold start, so give it `container_disk_gb` room.
- **GPU device names pin the exact device**, as runpod-flash's pool exclusions did, now using
  the platform's live pool membership, so pool members runpod-flash didn't know about (MIG
  slices) are excluded too.
- **Every resource gets `RUNPOD_API_KEY`** in its worker env, including resources that never
  call another one.

## 6. Smoke-test, deploy, clean up

```bash
rp flash dev main.py --once          # exits non-zero if the entrypoint fails
rp flash deploy                      # creates the new endpoints (default environment)
rp flash app list                    # confirm
```

Then, once the new endpoints pass, remove the ones deployed with `runpod-flash`, scoped to
that app:
`flash app delete OLD_APP` (with the old CLI still installed), or
`runpodctl serverless delete ENDPOINT_ID` for each old endpoint id. Update CI to call `rp flash`
instead of `flash`, and update any client that called the old endpoint ids.

## Silencing the notice in CI

`runpod-flash` 1.21.0 and later print a deprecation notice on every `flash` command and a
`FutureWarning` on import. Set `RUNPOD_FLASH_SILENCE_DEPRECATION=1` to silence both while a
pipeline still runs the old package.

## Exact rules

- [flash](../../runpod-usage/concepts/flash.yaml)

---
concepts: [flash, serverless-handler, network-volume, container-disk, volume-cache, model-store]
---
# Runpod SDK — common patterns

Complete runnable versions live in [`../examples/`](../examples/).

## Choosing a model

The SDK has **no model catalog**: name a HuggingFace repo id in code and the worker downloads
it at runtime, or attach it with `runpod.Model` so the platform stages it on the host first.

- **Start with the smallest model that proves the pipeline** (`gpt2`, `stabilityai/sd-turbo`,
  a 0.5–1B variant). It loads in seconds, so you validate the decorator, dependencies, GPU and
  I/O fast under `rp flash dev`, then change only the id string.
- **Match the model to GPU VRAM** (fp16 ≈ params × 2 bytes + overhead):

  | Model (fp16) | ~VRAM | `gpu=` |
  |---|---|---|
  | ≤3B / SD1.5 / sd-turbo | ≤8 GB | `"AMPERE_16"` or `"4090"` |
  | 7–8B | ~16 GB | `"4090"`, `"L4"`, `"AMPERE_24"` |
  | 13B | ~28 GB | `"5090"` or `"AMPERE_48"` |
  | 70B | ~140 GB | `"H200"` / `"B200"` (or quantize) |

- A hosted model with **no code** is a [Public Endpoint](https://docs.runpod.io/hub/public-endpoints),
  a different product.

## Loading ML models

Model weights are not part of the 1,500 MB build artifact; the artifact is your code plus pip
dependencies. Two things make loading fast and cheap.

**Load once per worker, not per request.** Put the load in an `@app.api` class's
`@runpod.init` method. It runs once when a worker starts, and every route reuses `self`:

```python
import runpod

app = runpod.App("sd")

@app.api(gpu="5090", workers=(0, 3), idle_timeout=300,
         dependencies=["diffusers", "transformers"])
class StableDiffusion:
    @runpod.init
    def load(self):
        import torch
        from diffusers import StableDiffusionPipeline
        self.pipe = StableDiffusionPipeline.from_pretrained(
            "stabilityai/sd-turbo", torch_dtype=torch.float16).to("cuda")

    @runpod.post("/generate")
    async def generate(self, body: dict):
        import base64, io
        image = self.pipe(prompt=body["prompt"], num_inference_steps=1).images[0]
        buf = io.BytesIO(); image.save(buf, format="PNG")
        return {"png_b64": base64.b64encode(buf.getvalue()).decode()}
```

In a queue function, cache in a module global. Deployed workers import the module once, so the
load runs once per worker:

```python
_PIPE = None

@app.queue(gpu="4090", dependencies=["transformers"])
def generate(prompt: str) -> str:
    global _PIPE
    if _PIPE is None:
        from transformers import pipeline
        _PIPE = pipeline("text-generation", model="gpt2", device=0)
    return _PIPE(prompt, max_new_tokens=20)[0]["generated_text"]
```

**Keep the weights close to the worker.** A scaled-to-zero worker re-downloads on every cold
start unless you cache. Pick one:

- **`runpod.Model`** (HuggingFace model, GPU resource). The platform stages the weights on the
  host and holds the worker until they are ready, so the cold request includes that wait. Time
  the whole cold request before deciding it is fast enough.

  ```python
  qwen = runpod.Model("Qwen/Qwen2.5-7B-Instruct")

  @app.queue(gpu="4090", model=qwen, dependencies=["transformers"])
  def chat(prompt: str) -> str:
      from transformers import pipeline
      pipe = pipeline("text-generation", model=str(qwen.path), device=0)
      return pipe(prompt, max_new_tokens=64)[0]["generated_text"]
  ```

- **A `NetworkVolume` mirrored with `VolumeCache`.** A cold worker copies weights an earlier
  worker downloaded onto its local disk and loads from there instead of over the network mount.
  Keep the HF cache on container disk (not under `/runpod-volume`) and wrap the load:

  ```python
  cache = runpod.NetworkVolume("model-cache", size=100)

  @app.api(gpu="5090", mounts={"/runpod-volume": cache}, container_disk_gb=60,
           env={"HF_HUB_CACHE": "/opt/hf-cache"}, dependencies=["transformers"])
  class Chat:
      @runpod.init
      def load(self):
          from runpod.serverless import VolumeCache
          from transformers import pipeline
          with VolumeCache(dirs=["/opt/hf-cache"]):   # hydrate from the volume, then sync back
              self.pipe = pipeline("text-generation", model="Qwen/Qwen2.5-7B-Instruct", device=0)
  ```

  `rp flash deploy` vendors your local `runpod` package into the artifact, so
  `runpod.serverless.VolumeCache` is importable on the worker without listing `runpod` in
  `dependencies`.

  The same code works on a `runpod.GlobalVolume` (GPU resources only): mount it at
  `/runpod-volume` and `VolumeCache` mirrors there. Restores from a global volume measured
  slower than from a network volume before 2.0.0's parallel restore; re-time on your version,
  and prefer a network volume when the endpoint stays in one data center
  ([model-caching](../../runpodctl/reference/model-caching.md#measured-cold-starts)).

**Don't load weights straight off the mount** (`HF_HOME=/runpod-volume/...` or
`from_pretrained("/runpod-volume/model")`). Every read then crosses the network; restoring with
`VolumeCache` and loading from local disk measured faster
([model-caching](../../runpodctl/reference/model-caching.md#measured-cold-starts)).

`VolumeCache` details that bite:
- **Best-effort.** Cache errors are logged and swallowed, and the worker cold-starts normally,
  so a broken cache looks like a slow cold start. Use `VolumeCache(..., best_effort=False)`
  while debugging.
- It never copies symlinks, which the HF cache uses for `snapshots/`. Leave Hub access on
  (don't set `HF_HUB_OFFLINE=1`) so `from_pretrained` can re-link with a metadata call.
- Size the volume for the mirror and `container_disk_gb` for the whole local cache plus
  headroom; the default is 30 GB on GPU.
- Let one worker populate the mirror first. Workers that cold-start together on an empty mirror
  each download and sync a full copy.
- The mirror is keyed by endpoint ID (`namespace` defaults to `RUNPOD_ENDPOINT_ID`). Every dev
  session creates new endpoints, so each session starts from an empty mirror and leaves its copy
  on the volume. Pass a fixed `namespace="qwen-7b"` to share one mirror across dev sessions and
  deploys.

Full behavior: [`concepts/volume-cache.yaml`](../../runpod-usage/concepts/volume-cache.yaml).

## CPU + GPU pipeline

```python
@app.queue(cpu="cpu5c-4-8", dependencies=["pandas"])
def preprocess(rows: list[dict]) -> list[list[float]]:
    import pandas as pd
    return pd.DataFrame(rows).dropna().astype(float).values.tolist()

@app.queue(gpu="AMPERE_80")
def score(matrix: list[list[float]]) -> list[float]:
    import torch
    return torch.tensor(matrix, dtype=torch.float32, device="cuda").mean(dim=1).tolist()

@runpod.local_entrypoint
def main():
    print(score.remote(preprocess.remote([{"a": 1, "b": 3}, {"a": 5, "b": 7}])))
```

Each stage is its own endpoint with its own workers and hardware. A remote function can also
call another resource's `.remote()` directly.

## Parallel execution

```python
jobs = [score.spawn(batch) for batch in batches]     # submit all, don't wait
results = [job.result() for job in jobs]             # collect

# async code
results = await asyncio.gather(*(score.remote.aio(b) for b in batches))
```

## Training jobs as tasks

`@app.task` gets a Pod per call and terminates it when the function returns, which fits
training. Share artifacts through a volume:

```python
ckpts = runpod.NetworkVolume("ckpts", size=50)

@app.task(gpu="H100", gpu_count=2, mounts={"/ckpts": ckpts})
def finetune(steps: int = 1000) -> str:
    ...
    return str(ckpts.path / "adapter")
```

A queue endpoint that mounts the same volume at `/runpod-volume` reads what the task wrote (golden
path 07 shows the volume handoff). Small results can skip the volume: return them from the task
as JSON, as [`train_then_serve.py`](../examples/train_then_serve.py) does. For a long run, `job = finetune.spawn()`
then `await job.wait(timeout=...)`, which terminates the Pod when it finishes or times out.

## Streaming results

```python
@app.queue(cpu="cpu3c-1-2")
def tokens(prompt: str):
    for word in prompt.split():
        yield word

for chunk in tokens.stream("hello streaming world"):
    print(chunk)
```

## Returning binary output

Requests and results go through the job API as JSON, capped at 10 MB. Return images and audio
as base64 (`{"png_b64": ...}`) and decode on the client. For larger outputs, write to a volume
or object storage and return a path or URL.

## Calling from outside Python

Deployed queue resources take their keyword arguments as the job `input`:
`def generate(prompt: str)` takes `{"input": {"prompt": "hi"}}` on `/run` or `/runsync`.
Deployed API routes take the JSON body itself on `https://ENDPOINT_ID.api.runpod.ai/PATH`.
Full examples in [api.md](api.md#raw-http-to-a-deployed-queue-resource).

---
concepts: [flash, serverless-handler, job, container-disk]
---

# Legacy `runpod-flash` reference

Use this to maintain a project that still imports `runpod_flash`, or as the fallback when the
Runpod SDK path fails on an SDK bug (see [Fallback from the SDK](#fallback-from-the-sdk)).
`runpod-flash` is deprecated and gets no fixes after October 29, 2026; propose
[migrating](migrate-from-runpod-flash.md). New code goes on the Runpod SDK. `flash --help` is
authoritative for the installed version.

## Setup and CLI

```bash
pip install runpod-flash              # Python 3.10-3.13; to run next to runpod 2.x, see "Fallback from the SDK"
flash login                           # saves a key to ~/.runpod/config.toml (--no-open for headless)
flash init my-project                 # scaffold (writes AGENTS.md + CLAUDE.md)
flash dev                             # local server on :8888; functions run on remote workers
flash dev --auto-provision            # pre-provision endpoints to skip the first cold start
flash deploy [--env NAME]             # build (<=1500 MB, torch excluded) and deploy
flash app list | get APP | delete APP
flash env list | create NAME | get NAME | delete NAME
flash undeploy NAME [--force]         # one endpoint
flash update                          # upgrade the CLI
```

1.21.0 and later print a deprecation notice; `RUNPOD_FLASH_SILENCE_DEPRECATION=1` silences it.

## Fallback from the SDK

When the Runpod SDK fails on what looks like an SDK bug (a traceback inside `runpod/apps/`, a
worker that crashes before your function runs, or code that worked on `runpod-flash`):

1. Run `pip show runpod` and upgrade. Check the runpod-python
   [CHANGELOG](https://github.com/runpod/runpod-python/blob/main/CHANGELOG.md) and the
   [commit history for `runpod/apps`](https://github.com/runpod/runpod-python/commits/main/runpod/apps)
   for a fix or a behavior change. The
   [runpod-flash history](https://github.com/runpod/flash/commits/main) shows how the old
   package handled the same case.
2. Still broken: tell the user, and run the project on `runpod-flash` from the pre-migration
   code. Give it its own environment with the project's requirements and `runpod` 1.x, so the
   SDK project keeps 2.x. Create it outside the project directory so a deploy doesn't package
   it (`rp flash deploy` skips only `.venv/`, `venv/` and `env/`).
   ```bash
   uv venv ~/.venvs/flash-<project>          # one per project, so packages don't mix
   uv pip install --python ~/.venvs/flash-<project> runpod-flash "runpod<2" -r requirements.txt
   ~/.venvs/flash-<project>/bin/flash deploy
   ```
   `flash dev` and `flash deploy` import your modules locally, so that environment needs the
   project's own imports. For cleanup only, `uv tool install runpod-flash --with "runpod<2"`
   is enough.
3. It gets no fixes after October 29, 2026, so move back to the SDK once the fix ships.

## Modes

```python
from runpod_flash import Endpoint, GpuGroup, PodTemplate

@Endpoint(name="worker", gpu=GpuGroup.AMPERE_80, workers=5, dependencies=["torch"])
async def compute(data):                       # queue-based: one function, one endpoint
    import torch
    return {"sum": torch.tensor(data, device="cuda").sum().item()}

api = Endpoint(name="my-api", gpu=GpuGroup.ADA_24, workers=(1, 5))
@api.post("/predict")                          # load-balanced: routes share one worker pool
async def predict(data: list[float]): ...

server = Endpoint(name="srv", image="org/img:tag", gpu=GpuGroup.AMPERE_80,
                  template=PodTemplate(containerDiskInGb=100))   # client mode: prebuilt worker image
ep = Endpoint(id="abc123")                                       # client mode: existing endpoint
job = await ep.runsync({"prompt": "hello"}, timeout=120)
```

`name=` only is decorator mode; `image=` or `id=` is client mode. `gpu=` and `cpu=` are
exclusive. A class `@Endpoint` whose `__init__` loads a model runs that load once per worker.

## Defaults

`Endpoint` defaults: `workers` `(0, 1)`, `idle_timeout` 60 seconds, `execution_timeout_ms` 0
(unlimited), `flashboot=True`, `min_cuda_version` 12.8 (the `runpod/flash` image needs it; lower
it only with an image that supports older CUDA), `PodTemplate(containerDiskInGb=64)`. These
differ from the platform defaults.

## Dev server loop

```bash
flash dev > /tmp/flash-dev.log 2>&1 &
for i in $(seq 1 60); do grep -q "flash dev  localhost:" /tmp/flash-dev.log && break; sleep 2; done
URL=$(grep -o "localhost:[0-9]*" /tmp/flash-dev.log | head -1)   # the port bumps if 8888 is taken
```

- Routes are namespaced by file: `main.py`'s `/predict` is `/main/predict`.
- Load-balanced routes take the handler argument at the top level: `def predict(data)` wants
  `{"data": {...}}`.
- Queue endpoints are served at `/main/runsync` and want the body double-wrapped:
  `def synthesize(data)` wants `{"input": {"data": {...}}}`. The worker calls
  `handler(**job_input)`, so `input` keys must match parameter names, and an empty `input` is
  rejected with `Job has missing field(s): id or input`.

## Gotchas

1. **`flash dev` ships only the function body.** Module-level constants and imports raise
   `NameError` on the worker under `flash dev`, while `flash deploy` imports the whole module
   and hides the bug. Put imports and helpers inside the function. To cache a model in
   function form, use a module global set inside the body
   (`global _M` / `try: _M` / `except NameError: _M = load()`).
2. **Every decorated function and client call needs `await`.**
3. **`runsync` times out after 60 seconds**; raise `timeout=` or use `run()` + `job.wait()` for
   cold starts.
4. **A set `RUNPOD_API_KEY` overrides `flash login`.** A bad key logs
   `GraphQL request failed: 401` while `flash dev` still prints its ready line.
5. **Teardown with `flash app delete APP`.** `flash undeploy list` can miss a deployed app.
6. **10 MB payload limit**; native CUDA libraries go in `dependencies`; `system_dependencies`
   add to cold start; automatic GPU switching needs `workers` ≥ 5. These carry over to the SDK.

## Exact rules

- [flash](../../runpod-usage/concepts/flash.yaml)

---
concepts: [flash, api-key]
---

# Runpod SDK — setup & `rp` CLI reference

## Setup

The SDK and the `rp` CLI ship together in the `runpod` package (2.0 and later). Install it
into the project's virtual environment, because dev sessions run your local interpreter.

```bash
# requires Python 3.10-3.14
uv add "runpod>=2.0.0"          # uv project
pip install "runpod>=2.0.0"     # pip / venv
pip show runpod                    # confirm the version (2.x)
rp update                          # upgrade the package in place; rp update --version X pins one
```

The `>=2.0.0` floor keeps an existing 1.x install from satisfying the requirement; 1.x has no
`rp flash`.

The CLI is also installed as `runpod`, so `runpod flash deploy` and `rp flash deploy` are the
same command.

### Auth

```bash
rp login                            # browser approval, then paste the key the console shows; saved to ~/.runpod/config.toml
rp login --no-open                  # print the approval URL instead of opening a browser
rp login --agent --email "$EMAIL"   # emailed six-digit code instead of a browser
rp login --api-key "$KEY"           # store a key directly, no browser
export RUNPOD_API_KEY=your_key      # headless agents and CI
```

- The SDK reads `RUNPOD_API_KEY` first, then `~/.runpod/config.toml`. A set env var wins, so
  a stale exported key causes 401s even after a good `rp login`.
- `runpodctl` reads the same `config.toml`, so one `rp login` serves both. A key saved by the
  legacy `flash login` keeps working.
- `rp login` and `--no-open` read the pasted key from an interactive terminal; with piped or
  no stdin they exit and point at `RUNPOD_API_KEY` or `--api-key`.
- With no key at all, calls fail with "no api key configured. run `rp login` or set RUNPOD_API_KEY."

### Scaffold

```bash
rp flash init my-app      # writes my-app/main.py, requirements.txt, .runpodignore
rp flash init .           # scaffold into the current directory
rp flash init my-app -f   # overwrite existing files
```

The scaffold is a CPU queue function plus a `@runpod.local_entrypoint`, so
`cd my-app && rp flash dev main.py --once` is a working first run.

## CLI

```bash
rp --help                                   # top level: flash, login, catalog, endpoint, pod, volume, secret, registry, logs, ssh, update

# develop
rp flash dev main.py                        # temporary endpoints + run the entrypoint, re-run on save, ^C cleans up
rp flash dev main.py --once                 # run once, tear down, exit non-zero on failure (CI smoke test)

# deploy
rp flash deploy                             # every app found in the cwd -> "default" environment
rp flash deploy main.py --env staging       # one target file, another environment
rp flash deploy --python-version 3.12       # worker Python for dependency wheels (defaults to the local one)
rp flash deploy --exclude scipy,numba       # leave packages out of the artifact (they must come from the image)
rp flash deploy --build-only                # write <app>-artifact.tar.gz, deploy nothing

# apps and environments
rp flash app list                           # every app and its environments
rp flash app get my-app
rp flash app delete my-app --yes            # undeploys every environment in the app
rp flash env list --app my-app
rp flash env add staging --app my-app
rp flash env get staging --app my-app       # an environment with its endpoints
rp flash env delete staging --app my-app --yes

# tear down one environment (the app and its build history remain)
rp flash undeploy --app my-app --env default --yes

# platform helpers in the same CLI
rp secret add hf-token --value "$HF_TOKEN"  # read in code with runpod.Secret("hf-token")
rp registry add my-registry --username USER --password TOKEN   # private images (registry_auth="my-registry")
rp logs POD_ID --follow                     # a Pod's container and system logs
rp ssh POD_ID
```

Take exact flags from `rp <command> --help`; the list above is the shape of the surface.

### Build artifact

- `rp flash deploy` packages your code plus resolved `dependencies` into one artifact of up to
  **1,500 MB**. On the built-in GPU images `torch`, `torchvision`, `torchaudio` and `triton` are
  left out automatically, because the image ships them.
- `.runpodignore` and the project's `.gitignore` files decide what source goes in. `.env`
  files, keys and credential files are always left out, and so are tests (`tests/`, `test/`,
  `test_*.py`, `*_test.py`), virtualenvs (`.venv/`, `venv/`, `env/`), caches and `*.tar.gz`.
  A helper module named like a test (`test_utils.py`) is dropped too and fails to import on
  the worker.
- `--build-only` resolves dependencies and packages without touching your account, which makes
  it a quick check that discovery and dependency resolution succeed.

## Exact rules

- [flash](../../runpod-usage/concepts/flash.yaml)
- [api-key](../../runpod-usage/concepts/api-key.yaml)

---
name: runpod-flash
description: >-
  Code-first serverless with the Runpod Python SDK (`runpod` 2.0): define a
  `runpod.App` with `@app.queue` / `@app.api` / `@app.task`, iterate on remote
  GPUs/CPUs with `rp flash dev`, ship with `rp flash deploy`, and maintain or
  migrate legacy `runpod-flash` projects (`from runpod_flash import Endpoint`).
  Use runpod-build-template for explicit Dockerfile/template project authoring,
  and runpodctl/runpod-mcp for infrastructure management.
metadata:
  author: runpod
  version: "2.0.0" # x-release-please-version
  concepts: [flash, serverless-endpoint, serverless-handler, worker, volume-cache, model-store]
license: Apache-2.0
---

# Runpod SDK apps (Flash)

For an explicit Dockerfile/template project request, use
[runpod-build-template](../runpod-build-template/SKILL.md). Do not replace that
deliverable with Flash unless the user chooses its code-first workflow.

Write Python locally, run it on remote Runpod GPUs/CPUs with `rp flash dev`, then
`rp flash deploy` to ship. Flash is part of the `runpod` package (2.0): one SDK, one
`rp` CLI, one login. The standalone `runpod-flash` package is deprecated and gets no fixes
after **October 29, 2026**.

**`rp --help` and `rp flash <command> --help` are authoritative for the command surface.**
This skill is the mental model, the decision rules, and the gotchas that help output does not
carry. Confirm the installed version with `pip show runpod` before concluding a command or
parameter is unavailable.

## Which SDK is this project on?

Check before writing code. The two APIs do not mix.

| You see | Project is on | Do |
|---|---|---|
| `import runpod`, `runpod.App(...)`, `@app.queue`, `rp flash ...` | **Runpod SDK 2.0** | This file + `reference/` |
| `from runpod_flash import Endpoint`, `@Endpoint(...)`, `flash dev`, `runpod-flash` in deps | **Legacy runpod-flash** | [reference/runpod-flash-legacy.md](reference/runpod-flash-legacy.md) to maintain it, [reference/migrate-from-runpod-flash.md](reference/migrate-from-runpod-flash.md) to move it |
| A new project | Runpod SDK 2.0 | Start below |

Write new code on the SDK. For an existing `runpod-flash` project, propose migrating and
check its *Known differences* first; don't migrate silently.

**If the SDK path fails on an SDK bug, `runpod-flash` is the fallback**: a traceback inside
`runpod/apps/`, a worker that crashes before your function runs, or the same code working on
`runpod-flash`. Upgrade `runpod` first; the steps to check for a fix and fall back are in
[reference/runpod-flash-legacy.md](reference/runpod-flash-legacy.md#fallback-from-the-sdk).

**Worked examples first for anything multi-step.** Flash appears in end-to-end paths:
[03 variant B (whisper endpoint)](../runpod/golden-paths/03-whisper-endpoint/variant-b-flash.md),
[07 (network-volume handoff)](../runpod/golden-paths/07-network-volume-handoff.md) and
[08 (fine-tune → serve)](../runpod/golden-paths/08-finetune-to-serverless.md); the full index is
[runpod/golden-paths/README.md](../runpod/golden-paths/README.md). Runnable single-file apps
live in [`examples/`](examples/).

**Load on demand:**

| Need | Read |
|------|------|
| Install, `rp login`, `rp flash init`, every `rp` command | [reference/setup-and-cli.md](reference/setup-and-cli.md) |
| Decorator parameters, call styles, `Queue`/`Api`/`Endpoint` clients, volumes, `Model`, `Secret`, GPU & CPU names | [reference/api.md](reference/api.md) |
| Worked patterns: choosing a model, warm model loading, CPU→GPU pipeline, fan-out, tasks, raw HTTP | [reference/patterns.md](reference/patterns.md) |
| Moving a `runpod-flash` project to the SDK | [reference/migrate-from-runpod-flash.md](reference/migrate-from-runpod-flash.md) |
| Maintaining a `runpod-flash` project as it is | [reference/runpod-flash-legacy.md](reference/runpod-flash-legacy.md) |

Quick start: `uv add "runpod>=2.0.0"` (or `pip install "runpod>=2.0.0"`) → `rp login` (or `export RUNPOD_API_KEY=...`)
→ `rp flash init my-app` → `cd my-app && rp flash dev main.py`.

## The app model

```python
import runpod

app = runpod.App("my-app")                     # names the group of resources

@app.queue(gpu="4090", dependencies=["torch"]) # queue-based serverless endpoint
def infer(x: list[float]):
    import torch
    return {"sum": torch.tensor(x, device="cuda").sum().item()}

@runpod.local_entrypoint                       # runs on YOUR machine
def main():
    print(infer.remote([1.0, 2.0, 3.0]))       # runs on a Runpod worker
```

| Decorator | Creates | Use for |
|---|---|---|
| `@app.queue(...)` on a function | Autoscaling queue-based endpoint | Request/response inference, batch jobs |
| `@app.api(...)` on a class | Load-balanced endpoint serving routes (`@runpod.get`/`@runpod.post`, setup in `@runpod.init`) | HTTP services, multi-route APIs, load-once model state |
| `@app.task(...)` on a function | One ephemeral **Pod per call**, terminated when it returns | Training, fine-tuning, long jobs |

Call styles on a queue or task handle: `fn.remote(...)` (blocks), `await fn.remote.aio(...)`,
`fn.spawn(...)` (returns a `Job`, or a `TaskJob` for a task), `fn.stream(...)` (queue generator
functions only), `fn.local(...)` (runs on your machine). API classes are called by route: `MyApi.post("/path", body)`.

## Dev vs deploy

- `rp flash dev main.py` — **iterate.** Creates temporary endpoints (`dev-<app>-<resource>-<hash>`),
  runs the `@runpod.local_entrypoint` on your machine, streams the workers' `print` output to
  your terminal, and re-runs after every file save. Ctrl-C deletes the endpoints. There is **no
  local HTTP server**; the entrypoint is how you exercise the code.
- `rp flash dev main.py --once` — run the entrypoint once, tear down, exit non-zero on failure.
  The smoke test for CI and for "does the migrated code work".
- `rp flash deploy` — **ship.** Builds an artifact (≤1,500 MB, torch excluded on built-in GPU
  images) and creates persistent endpoints in the `default` environment (`--env NAME` for
  others). Re-run to update. `--build-only` packages without deploying, a cheap offline check
  that discovery and dependency resolution work.

Dev and deploy run the **same module source**, so module-level imports, constants and helpers
behave identically in both. Deploy packages the project directory, but a dev session ships
only the module's source, so keep the app in **one file** when you use `rp flash dev`.

## Autonomous dev loop

A dev session is long-running. Three rules:
- **Run it in the background with stdin closed** (`< /dev/null`). With no stdin it re-runs
  the entrypoint on every file save, so you never need to press Enter.
- **Capture its output** to a log file and read it. Worker `print`s, tracebacks, `✗ remote execution failed: ...` and the per-run
  `✓ done in ...` / `✗ failed in ...` lines land there.
- **Stop it with `kill` (SIGTERM) or SIGINT**, so it deletes its endpoints, and confirm the
  log ends with `cleaning up N deleted`. A second signal stops waiting for a running
  entrypoint and goes straight to cleanup. `kill -9` skips cleanup. When one run is enough,
  prefer `--once`; it cleans up on its own.

```bash
rp flash dev main.py < /dev/null > /tmp/rp-dev.log 2>&1 &
DEV=$!
# poll the log (bounded) for the run result: "✓ done in ..." or "✗ failed in ..."
for i in $(seq 1 120); do grep -qE "✓ done in|✗ failed in" /tmp/rp-dev.log && break; sleep 3; done
tail -40 /tmp/rp-dev.log
# edit main.py and save -> the session re-scans, refreshes endpoints, re-runs main()
# (each run appends a new result line; read the last one)
kill $DEV; wait $DEV; tail -3 /tmp/rp-dev.log   # expect "cleaning up N deleted"
```

For a single check, `rp flash dev main.py --once` is simpler and cleans up on its own. If a
session died without cleanup, list leftovers with `runpodctl serverless list` and delete the
`dev-<app>-...` endpoints **this session created**, by ID.

## Choosing the resource shape

Default to plain Python on the Runpod runtime image: `dependencies=[...]` and
`system_dependencies=[...]` install on the worker, so no Dockerfile is needed, and even large
HuggingFace models stay in code (see [reference/patterns.md](reference/patterns.md#loading-ml-models)).

- **`image="org/image:tag"`** on any decorator runs your function on that image; it needs
  Python 3, not a Serverless handler. Use it for system libraries or a specific CUDA stack.
  `registry_auth=` names saved credentials for a private image (`rp registry add`).
- **A prebuilt inference server image** (vLLM, worker-comfyui) that already implements a
  Serverless handler is not an SDK app. Deploy it with runpodctl or runpod-mcp and call it
  with `runpod.Endpoint("ENDPOINT_ID")`.
- **An existing endpoint** by ID: `runpod.Endpoint("ENDPOINT_ID").run_sync({...})`.
- **A resource in another deployed app**: `runpod.Queue(app="media", name="transcribe")` /
  `runpod.Api(app="media", name="inference")`.

## Gotchas

1. **Queue arguments and results must be JSON-serializable.** Queue jobs are plain JSON in dev
   and deploy alike. A non-JSON argument fails in dev with `arguments must be json-serializable`
   and against a deployed endpoint with Python's own JSON `TypeError`. Pass
   URLs or base64, not objects. Whole-number floats can arrive as integers (`1.0` → `1`), so
   cast numeric inputs explicitly (`torch.tensor(x, dtype=torch.float32)`).
2. **One file per app in dev.** A dev session ships only the module's source, so a sibling
   local module imports in deploy but fails in dev. For dev sessions, keep helpers in the same
   file or in an installable package.
3. **Keep blocking work out of module scope.** The SDK imports your module to discover
   resources, locally and on every worker. Put heavy imports (`torch`) and work inside
   functions; guard local-only code with `runpod.is_local()`.
4. **`gpu=` and `cpu=` are exclusive** (`gpu and cpu are mutually exclusive`). GPU strings can
   be short (`"4090"`, `"H100"`) or a list of acceptable GPUs.
5. **Default `workers=(0, 3)`** (runpod-flash was `(0, 1)`). Set it explicitly when cost or
   concurrency matters. `idle_timeout` is seconds (default 60).
6. **Auto GPU switching needs `workers` max ≥ 5** when you pass a list of GPUs.
7. **10 MB payload limit** on requests. Return large outputs via a volume or object storage URL.
8. **Volumes outlive everything.** Dev sessions and `rp flash undeploy` delete endpoints, not
   the `NetworkVolume`s they created, and those keep billing. Queue and API resources mount one
   volume at `/runpod-volume`; read `vol.path` inside the remote function, never at import time.
   Load model weights through `VolumeCache`, not straight off the mount
   ([reference/patterns.md](reference/patterns.md#loading-ml-models)).
9. **Tasks bill until they return.** A `.spawn()`ed task that you never `wait()` or `cancel()`
   keeps its Pod running after your script exits.
10. **Native CUDA libraries go in `dependencies` too**, e.g. faster-whisper needs
    `nvidia-cublas-cu12` + `nvidia-cudnn-cu12` or it silently falls back to CPU.
11. **Dev sessions and tasks need local Python 3.10–3.14.** They ship code pickled by your
    interpreter, so the worker matches your local version. Deploy falls back to 3.12 with a
    warning on other versions; `rp flash deploy --python-version` sets it explicitly.
12. **A set `RUNPOD_API_KEY` wins over `rp login`.** A stale exported key causes 401s while the
    saved login is fine. Check with `runpodctl user`, then `unset RUNPOD_API_KEY` or export a
    good key.
13. **The torch family comes from the GPU image; never pin it.** Builds leave `torch`,
    `torchvision`, `torchaudio` and `triton` out of the artifact whether you list them or a
    dependency pulls them in (`peft`, `accelerate`, `sentence-transformers`), plus the wheels
    only torch needs. GPU workers use the image's build (`torch 2.9.1+cu128`), so a version
    pin on these packages is ignored. In an app that mixes CPU and GPU resources, a CPU
    resource whose dependencies need torch installs it at cold start; give it
    `container_disk_gb` room for that (the build warns when this applies).
14. **Dev sessions run the runtime image's own `runpod`.** Deploys vendor your local package,
    but `rp flash dev` workers import the `runpod` baked into the built-in image. An
    `AttributeError` on a `runpod.` name in a dev worker traceback means the image is behind
    your SDK; `rp flash deploy` runs your installed version.
15. **Teardown by app and environment:** `rp flash undeploy --app APP --env default -y`
    removes one environment's endpoints; `rp flash app delete APP --yes` removes every
    environment (without `--yes` it prompts, and aborts with no terminal). Then delete volumes
    you no longer need.
16. **Queue and API names need 3+ characters to deploy.** The name defaults to the function or
    class name, and dev sessions don't check it, so `class SD` passes `rp flash dev` and fails
    `rp flash deploy`. Rename it or pass `name=`.
17. **Every endpoint gets your account API key.** Deploy and dev put `RUNPOD_API_KEY` in each
    endpoint's template env, whether or not its code calls another resource, so anyone who can
    read the template can read the key.
18. **Re-running the same app soon after a dev session can fail.** Dev sessions leave their
    templates behind, and the platform holds them for up to about 90 minutes, so a new
    `rp flash dev` of the same app fails with "endpoint template names must be unique".
    Wait, or rename the `runpod.App`.

## Resources

- Setup & CLI: [reference/setup-and-cli.md](reference/setup-and-cli.md) · API: [reference/api.md](reference/api.md) · Patterns: [reference/patterns.md](reference/patterns.md) · Examples: [examples/](examples/)
- SDK source and upstream examples: https://github.com/runpod/runpod-python (`examples/apps/`)
- Docs: https://docs.runpod.io/sdk/overview · CLI: https://docs.runpod.io/sdk/cli · Migration: https://docs.runpod.io/sdk/migrate-from-flash