threejs-audio-generator

어떤 스킬인가요?

Generate, convert, clean, and integrate audio for Three.js browser games with ElevenLabs: sound effects, looping ambience, music tracks and loops (menu, gameplay, boss, victory/defeat stingers), UI sounds, impact/weapon/vehicle audio, creature and boss stingers, announcer and dialogue TTS, voice conversion from a scratch performance, voice cleanup, audio manifests, and Web Audio integration.

설치 방법

프로젝트 폴더에서 아래 명령어를 실행하고, 설치할 에이전트를 선택하세요.

설치 명령을 직접 실행해야 적용됩니다. 지원 에이전트와 필요한 권한·라이선스는 제작자의 안내를 확인하세요.

스킬 원문 파일

interface:
  display_name: "Three.js Audio Generator"
  short_description: "Generate SFX, ambience, and voices for Three.js games"
  default_prompt: "Use $threejs-audio-generator to generate and integrate audio for this game's actual events and runtime constraints."

# Three.js Audio Workflows

Use this reference before generating or integrating audio for a game.

## Audio Planning

Create an audio matrix before generating files:

| Category | Required events | Asset count | Loop? | Runtime group |
| --- | --- | ---: | --- | --- |
| UI | hover, confirm, cancel, pause, fail | 3-8 | no | ui |
| Movement | jump, dash, boost, landing, drift | 3-10 | sometimes | sfx |
| Interaction | pickup, hit, shield, score, checkpoint | 4-12 | no | sfx |
| Threat | enemy attack, warning, impact, boss cue | 4-12 | no | sfx |
| Ambience | room tone, wind, engines, crowd, weather | 1-4 | yes | ambience |
| Music | menu, gameplay, boss, win/lose stingers | 1-5 | tracks yes, stingers no | music |
| Voice | announcer, boss, tutorial, combat barks | optional | no | voice |

For a first premium pass, fill the rows the game's events actually produce — typically an ambience loop, the core UI actions, and each primary gameplay event.

Music and voice are optional; add a gameplay loop when the genre and pacing benefit from a score, and voice when the design benefits from dialogue or callouts. An explicit no-music or silent brief wins.

## Prompting

Good prompts specify source, transient, tail, mix density, genre, and gameplay use:

```text
short [event] sound for [game genre], [material/source], clear transient, [tail length], no music, no voice, readable under gameplay mix
```

Examples:

- `short shield absorb impact for sci-fi boss fight, glassy plasma hit, bright transient, low sub thump, 0.8s tail, no music, no voice`
- `looping abandoned cathedral ambience for dark fantasy arena, distant wind through stone arches, subtle torch crackle, no melody, seamless loop`
- `tiny premium menu confirm click, soft mechanical latch, warm sparkle tail, no harsh beep`

Avoid prompts that are only mood words (`epic`, `AAA`, `cool`). Name the gameplay event.

## Generation Strategy

- Generate short SFX individually instead of one long mixed file.
- Make loops deliberately with `--loop`; test them looping in the game.
- Avoid music inside SFX prompts unless the user asked for music.
- Keep UI sounds quieter and shorter than gameplay SFX.
- Generate variants for high-frequency events to avoid repetition.
- Normalize in the game through volume groups, not by editing every file manually during early iteration.

## Music Strategy

Music prompts name genre, tempo (BPM), key instrumentation, energy curve, and the game moment: `tense orchestral boss theme, 140 BPM, low brass ostinato, taiko hits, relentless energy, no intro, loopable, instrumental`. Use `--instrumental` unless the user wants vocals, since vocals compete with SFX and dialogue.

- Gameplay and menu loops: 30–90s. Ask for constant energy with no intro, outro, or fade so the ends meet; the API has no loop flag.
- Stingers (victory, defeat, level-up, boss reveal): 3–8s, not looped.
- Sectioned tracks (intro → loop → outro, calm → combat): write a `composition_plan` JSON with global styles and sections, and pass it with `--plan`. Plans also accept `--seed` for reproducible regeneration.
- Intensity layers (explore → combat): generate matching variants from the same prompt with one changed energy term, then crossfade them from game state.
- Keep the music group below SFX in the mix; duck it under dialogue, and during hitstop or big impacts.

Listen to the loop seam in game before generating variants. If it clicks or jumps, set `loopStart`/`loopEnd` on beat boundaries, or crossfade two sources, rather than regenerating blindly.

## Voice Strategy

Use TTS when the line can be generated from text and exact acting is less important.

Use voice-change when:

- The user or agent can record a scratch performance.
- Timing, breaths, laughter, anger, fear, or delivery need to be preserved.
- A boss, announcer, narrator, or character line needs stronger acting than text prompt alone.

Clean noisy scratch recordings with `isolate` before conversion unless `voice-change --remove-background-noise` is sufficient.

Do not generate or convert voices that imply impersonation of a real private person. For characters, describe a fictional voice style or use a licensed/available voice ID.

## Runtime Integration

Use a small audio manager instead of ad hoc `new Audio()` calls once a game has more than a few sounds:

- Load sounds after user gesture unlock.
- Maintain groups: `master`, `sfx`, `ui`, `ambience`, `voice`, `music`.
- Expose mute and per-group volume.
- Loop ambience and music through `AudioBufferSourceNode` (`loop = true`, with `loopStart`/`loopEnd` for music) or a library wrapper that handles loop restarts. Crossfade music tracks with two gains rather than hard-cutting.
- Stop/dispose old sources when restarting scenes.
- Do not trigger the same high-volume SFX every frame; add cooldowns or variant pools.
- Pause/resume audio with the game pause state and page visibility.

Minimal Web Audio shape:

```ts
class GameAudio {
  private ctx = new AudioContext();
  private buffers = new Map<string, AudioBuffer>();
  private gains = new Map<string, GainNode>();

  async unlock() {
    if (this.ctx.state !== 'running') await this.ctx.resume();
  }

  async load(id: string, url: string) {
    const data = await fetch(url).then(r => r.arrayBuffer());
    this.buffers.set(id, await this.ctx.decodeAudioData(data));
  }

  play(id: string, group = 'sfx', volume = 1) {
    const buffer = this.buffers.get(id);
    if (!buffer) return;
    const source = this.ctx.createBufferSource();
    const gain = this.ctx.createGain();
    gain.gain.value = volume;
    source.buffer = buffer;
    source.connect(gain).connect(this.gains.get(group) ?? this.ctx.destination);
    source.start();
  }
}
```

## Runtime failure modes

Ambience or music loops stacking after a pause or restart · audible clicks at a music loop seam · music masking gameplay SFX · autoplay blocked because nothing unlocked the context from a user gesture · mute or volume reaching only some groups · decode/load errors swallowed silently · mobile Safari needing its own unlock path.

#!/usr/bin/env python3
"""Generate and process Three.js game audio assets (SFX, music, voice) with ElevenLabs."""

from __future__ import annotations

import argparse
import json
import mimetypes
import os
import sys
import urllib.error
import urllib.parse
import urllib.request
import uuid
from pathlib import Path
from typing import Any


BASE_URL = "https://api.elevenlabs.io/v1"
DEFAULT_OUTPUT_FORMAT = "mp3_44100_128"
DEFAULT_TTS_VOICE_ID = "JBFqnCBsd6RMkjVDRZzb"


class AudioGeneratorError(RuntimeError):
    pass


def eprint(message: str) -> None:
    print(message, file=sys.stderr)


def api_key(args: argparse.Namespace) -> str:
    key = getattr(args, "api_key", None) or os.environ.get("ELEVENLABS_API_KEY")
    if not key:
        raise AudioGeneratorError("Missing API key. Set ELEVENLABS_API_KEY or pass --api-key.")
    return key


def build_url(path: str, query: dict[str, Any] | None = None) -> str:
    url = f"{BASE_URL}{path}"
    clean = {key: value for key, value in (query or {}).items() if value is not None}
    if clean:
        url = f"{url}?{urllib.parse.urlencode(clean)}"
    return url


def request_bytes(
    method: str,
    path: str,
    key: str,
    body: bytes | None = None,
    headers: dict[str, str] | None = None,
    query: dict[str, Any] | None = None,
    timeout: int = 300,
) -> bytes:
    req = urllib.request.Request(build_url(path, query), data=body, method=method)
    req.add_header("xi-api-key", key)
    for name, value in (headers or {}).items():
        req.add_header(name, value)
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            return resp.read()
    except urllib.error.HTTPError as exc:
        detail = exc.read().decode("utf-8", errors="replace")
        raise AudioGeneratorError(f"HTTP {exc.code}: {detail}") from exc
    except urllib.error.URLError as exc:
        raise AudioGeneratorError(f"Network error: {exc.reason}") from exc


def post_json_audio(
    args: argparse.Namespace, path: str, payload: dict[str, Any], out: Path, timeout: int = 300
) -> None:
    body = json.dumps(payload).encode("utf-8")
    data = request_bytes(
        "POST",
        path,
        api_key(args),
        body=body,
        headers={"Content-Type": "application/json", "Accept": "audio/mpeg"},
        query={"output_format": args.output_format},
        timeout=timeout,
    )
    write_file(out, data)


def multipart_body(fields: dict[str, Any], files: dict[str, Path]) -> tuple[bytes, str]:
    boundary = f"----threejs-audio-{uuid.uuid4().hex}"
    chunks: list[bytes] = []

    for name, value in fields.items():
        if value is None:
            continue
        if isinstance(value, bool):
            value = "true" if value else "false"
        chunks.append(f"--{boundary}\r\n".encode())
        chunks.append(f'Content-Disposition: form-data; name="{name}"\r\n\r\n'.encode())
        chunks.append(str(value).encode())
        chunks.append(b"\r\n")

    for name, path in files.items():
        if not path.exists():
            raise AudioGeneratorError(f"Input file not found: {path}")
        mime = mimetypes.guess_type(path.name)[0] or "application/octet-stream"
        chunks.append(f"--{boundary}\r\n".encode())
        chunks.append(
            f'Content-Disposition: form-data; name="{name}"; filename="{path.name}"\r\n'.encode()
        )
        chunks.append(f"Content-Type: {mime}\r\n\r\n".encode())
        chunks.append(path.read_bytes())
        chunks.append(b"\r\n")

    chunks.append(f"--{boundary}--\r\n".encode())
    return b"".join(chunks), boundary


def post_multipart_audio(
    args: argparse.Namespace,
    path: str,
    fields: dict[str, Any],
    files: dict[str, Path],
    out: Path,
    query: dict[str, Any] | None = None,
) -> None:
    body, boundary = multipart_body(fields, files)
    data = request_bytes(
        "POST",
        path,
        api_key(args),
        body=body,
        headers={"Content-Type": f"multipart/form-data; boundary={boundary}", "Accept": "audio/mpeg"},
        query=query,
    )
    write_file(out, data)


def write_file(path: Path, data: bytes) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
    print(f"Audio saved: {path.resolve()}")


def voice_settings(args: argparse.Namespace) -> dict[str, Any] | None:
    settings: dict[str, Any] = {}
    for field in ("stability", "similarity_boost", "style"):
        value = getattr(args, field, None)
        if value is not None:
            settings[field] = value
    if getattr(args, "speaker_boost", False):
        settings["use_speaker_boost"] = True
    return settings or None


def cmd_probe(args: argparse.Namespace) -> int:
    marker = "SET" if (args.api_key or os.environ.get("ELEVENLABS_API_KEY")) else "MISSING"
    print(f"ELEVENLABS_API_KEY={marker}")
    if args.validate and marker == "SET":
        data = request_bytes("GET", "/user", api_key(args))
        user = json.loads(data.decode("utf-8"))
        print(f"VALID_USER={user.get('email') or user.get('user_id') or 'ok'}")
    return 0


def cmd_sfx(args: argparse.Namespace) -> int:
    payload: dict[str, Any] = {
        "text": args.prompt,
        "model_id": args.model_id,
        "prompt_influence": args.prompt_influence,
        "loop": args.loop,
    }
    if args.duration is not None:
        payload["duration_seconds"] = args.duration
    post_json_audio(args, "/sound-generation", payload, Path(args.out))
    return 0


def cmd_music(args: argparse.Namespace) -> int:
    if bool(args.prompt) == bool(args.plan):
        raise AudioGeneratorError("Pass exactly one of --prompt or --plan.")
    payload: dict[str, Any] = {}
    if args.model_id:
        payload["model_id"] = args.model_id
    if args.prompt:
        if args.seed is not None:
            raise AudioGeneratorError("--seed only applies with --plan; the API rejects it with --prompt.")
        payload["prompt"] = args.prompt
        if args.duration is not None:
            if not 3 <= args.duration <= 600:
                raise AudioGeneratorError("--duration must be between 3 and 600 seconds.")
            payload["music_length_ms"] = int(args.duration * 1000)
        if args.instrumental:
            payload["force_instrumental"] = True
    else:
        if args.duration is not None or args.instrumental:
            raise AudioGeneratorError("--duration and --instrumental only apply with --prompt; set section lengths in the plan.")
        plan_path = Path(args.plan)
        if not plan_path.exists():
            raise AudioGeneratorError(f"Composition plan not found: {plan_path}")
        try:
            payload["composition_plan"] = json.loads(plan_path.read_text(encoding="utf-8"))
        except json.JSONDecodeError as exc:
            raise AudioGeneratorError(f"Composition plan is not valid JSON: {exc}") from exc
        if args.seed is not None:
            payload["seed"] = args.seed
    post_json_audio(args, "/music", payload, Path(args.out), timeout=900)
    return 0


def cmd_tts(args: argparse.Namespace) -> int:
    payload: dict[str, Any] = {
        "text": args.text,
        "model_id": args.model_id,
    }
    settings = voice_settings(args)
    if settings:
        payload["voice_settings"] = settings
    post_json_audio(args, f"/text-to-speech/{urllib.parse.quote(args.voice_id)}", payload, Path(args.out))
    return 0


def cmd_isolate(args: argparse.Namespace) -> int:
    fields = {"file_format": args.file_format}
    post_multipart_audio(
        args,
        "/audio-isolation",
        fields,
        {"audio": Path(args.input)},
        Path(args.out),
        query={"output_format": args.output_format},
    )
    return 0


def cmd_voice_change(args: argparse.Namespace) -> int:
    fields: dict[str, Any] = {
        "model_id": args.model_id,
        "remove_background_noise": args.remove_background_noise,
        "file_format": args.file_format,
    }
    settings = voice_settings(args)
    if settings:
        fields["voice_settings"] = json.dumps(settings)
    if args.seed is not None:
        fields["seed"] = args.seed
    post_multipart_audio(
        args,
        f"/speech-to-speech/{urllib.parse.quote(args.voice_id)}",
        fields,
        {"audio": Path(args.input)},
        Path(args.out),
        query={
            "output_format": args.output_format,
            "optimize_streaming_latency": args.optimize_streaming_latency,
        },
    )
    return 0


def add_common(parser: argparse.ArgumentParser) -> None:
    parser.add_argument("--api-key", help="ElevenLabs API key; defaults to ELEVENLABS_API_KEY")


def add_output_format(parser: argparse.ArgumentParser) -> None:
    parser.add_argument("--output-format", default=DEFAULT_OUTPUT_FORMAT)


def add_voice_settings(parser: argparse.ArgumentParser) -> None:
    parser.add_argument("--stability", type=float)
    parser.add_argument("--similarity-boost", type=float)
    parser.add_argument("--style", type=float)
    parser.add_argument("--speaker-boost", action="store_true")


def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(description="Generate and process Three.js game audio assets.")
    sub = parser.add_subparsers(dest="command", required=True)

    probe = sub.add_parser("probe", help="Report whether ELEVENLABS_API_KEY is available.")
    add_common(probe)
    probe.add_argument("--validate", action="store_true", help="Call /user to validate the key.")
    probe.set_defaults(func=cmd_probe)

    sfx = sub.add_parser("sfx", help="Generate sound effects or ambience from a prompt.")
    add_common(sfx)
    add_output_format(sfx)
    sfx.add_argument("--prompt", required=True)
    sfx.add_argument("--out", required=True)
    sfx.add_argument("--duration", type=float, help="Duration in seconds, typically 0.5-30.")
    sfx.add_argument("--prompt-influence", type=float, default=0.55)
    sfx.add_argument("--loop", action="store_true")
    sfx.add_argument("--model-id", default="eleven_text_to_sound_v2")
    sfx.set_defaults(func=cmd_sfx)

    music = sub.add_parser("music", help="Compose a music track from a prompt or a composition plan.")
    add_common(music)
    add_output_format(music)
    music.add_argument("--prompt", help="Style, instrumentation, tempo, mood, and game use.")
    music.add_argument("--plan", help="Path to a composition_plan JSON file (instead of --prompt).")
    music.add_argument("--out", required=True)
    music.add_argument("--duration", type=float, help="Length in seconds, 3-600 (prompt mode only).")
    music.add_argument("--instrumental", action="store_true", help="Guarantee no vocals (prompt mode only).")
    music.add_argument("--seed", type=int, help="Seed for reproducible output (plan mode only).")
    music.add_argument("--model-id", help="Music model, e.g. music_v1; defaults to the API default.")
    music.set_defaults(func=cmd_music)

    tts = sub.add_parser("tts", help="Generate a spoken line from text.")
    add_common(tts)
    add_output_format(tts)
    add_voice_settings(tts)
    tts.add_argument("--text", required=True)
    tts.add_argument("--out", required=True)
    tts.add_argument("--voice-id", default=DEFAULT_TTS_VOICE_ID)
    tts.add_argument("--model-id", default="eleven_multilingual_v2")
    tts.set_defaults(func=cmd_tts)

    isolate = sub.add_parser("isolate", help="Clean or isolate speech from a source audio file.")
    add_common(isolate)
    add_output_format(isolate)
    isolate.add_argument("--input", required=True)
    isolate.add_argument("--out", required=True)
    isolate.add_argument("--file-format", default="other")
    isolate.set_defaults(func=cmd_isolate)

    voice = sub.add_parser("voice-change", help="Convert source performance to a target voice.")
    add_common(voice)
    add_output_format(voice)
    add_voice_settings(voice)
    voice.add_argument("--input", required=True)
    voice.add_argument("--out", required=True)
    voice.add_argument("--voice-id", default=DEFAULT_TTS_VOICE_ID)
    voice.add_argument("--model-id", default="eleven_multilingual_sts_v2")
    voice.add_argument("--file-format", default="other")
    voice.add_argument("--seed", type=int)
    voice.add_argument("--remove-background-noise", action="store_true")
    voice.add_argument("--optimize-streaming-latency", type=int, choices=range(0, 5))
    voice.set_defaults(func=cmd_voice_change)

    return parser


def main() -> int:
    parser = build_parser()
    args = parser.parse_args()
    try:
        return args.func(args)
    except AudioGeneratorError as exc:
        eprint(f"threejs_audio_asset.py: {exc}")
        return 1


if __name__ == "__main__":
    raise SystemExit(main())

---
name: threejs-audio-generator
description: "Generate, convert, clean, and integrate audio for Three.js browser games with ElevenLabs: sound effects, looping ambience, music tracks and loops (menu, gameplay, boss, victory/defeat stingers), UI sounds, impact/weapon/vehicle audio, creature and boss stingers, announcer and dialogue TTS, voice conversion from a scratch performance, voice cleanup, audio manifests, and Web Audio integration."
---

# Three.js Audio Generator

Game-ready audio for Three.js projects: SFX, music, voice work, cleanup, and runtime integration. Provider: ElevenLabs.

Resolve `<this-skill-dir>` from the actual loaded skill file. Resolve sibling skills beside it first, then use the runner's discovered paths. Do not mix installed versions or assume a particular home directory.

## Reference

`references/audio-workflows.md` — the audio matrix, prompt patterns, generation, music, and voice strategy, Web Audio manager shape, and runtime failure modes. Read it before planning a game's audio, generating a batch, wiring runtime playback, or converting voices.

## When to use

SFX (jumps, hits, weapons, explosions, pickups, collisions, UI clicks) · music (menu and gameplay loops, boss themes, intensity layers, win/lose stingers) · ambience (wind, rain, city bed, engine hum, room tone, arena beds) · voice (announcer barks, boss lines, tutorial prompts, menu narration) · voice conversion from a scratch performance when timing and acting matter · cleanup and isolation before conversion or final use · Web Audio integration with loading, looping, manifests, volume groups, pause/resume, and gesture unlock.

Audio is not cosmetic for a premium game. Build an audio matrix from the events the game actually has; do not add dialogue, weapons, music, or ambience layers just to fill categories. Respect explicit silent, procedural-audio, accessibility, and external-service constraints. A narrow sound fix does not need a new soundtrack.

## API key

The script reads `--api-key` or `ELEVENLABS_API_KEY`. Keys never go in skill files, game code, or reports.

```bash
python3 <this-skill-dir>/scripts/threejs_audio_asset.py probe   # ELEVENLABS_API_KEY=SET|MISSING
```

Keys defined only in a shell profile can be absent from the process env; `threejs-game-director/scripts/probe_asset_credentials.sh` sources the profile and probes all three providers.

Add `--validate` to call `GET /user` and confirm the key actually works (prints `VALID_USER=...`) when a key is present but generation fails. A valid key can still be blocked by credit or plan limits, which surface as an `HTTP 4xx` from a real generation attempt — report that as a plan blocker rather than a missing key.

## Commands

Run from the game project directory:

```bash
python3 <this-skill-dir>/scripts/threejs_audio_asset.py sfx \
  --prompt "tight futuristic boost pickup, bright transient, short sparkling tail, arcade racing game" \
  --duration 1.2 --prompt-influence 0.65 --out assets/audio/sfx/boost-pickup.mp3

python3 <this-skill-dir>/scripts/threejs_audio_asset.py sfx \
  --prompt "seamless cyber resort ambience, distant surf, soft neon transformer hum, gentle crowd bed" \
  --duration 12 --loop --prompt-influence 0.45 --out assets/audio/ambience/cyber-resort-loop.mp3

python3 <this-skill-dir>/scripts/threejs_audio_asset.py music \
  --prompt "driving synthwave race loop, 128 BPM, punchy analog bass, bright arpeggios, steady energy with no intro or fade-out, designed to loop" \
  --duration 60 --instrumental --out assets/audio/music/race-loop.mp3

python3 <this-skill-dir>/scripts/threejs_audio_asset.py tts \
  --text "Perfect shot." --voice-id JBFqnCBsd6RMkjVDRZzb --out assets/audio/voice/perfect-shot.mp3

python3 <this-skill-dir>/scripts/threejs_audio_asset.py isolate \
  --input assets/audio/source/noisy-boss-line.wav --out assets/audio/voice/boss-line-clean.mp3

python3 <this-skill-dir>/scripts/threejs_audio_asset.py voice-change \
  --input assets/audio/source/scratch-boss-line.wav --voice-id JBFqnCBsd6RMkjVDRZzb \
  --remove-background-noise --out assets/audio/voice/boss-line-final.mp3
```

## Defaults

- SFX: `mp3_44100_128`, 0.5–2.5s, prompt influence 0.55–0.8.
- UI: 0.15–0.8s, high prompt influence, transients kept clear.
- Ambience: 8–30s with `--loop`, prompt influence 0.3–0.55.
- Music: `music --prompt` (`--duration` 3–600s, `--instrumental`) or `--plan FILE` with a `composition_plan` JSON for sectioned tracks (`--seed` works only with plans). The API has no loop flag; lengths, looping, and layering are in the reference's Music Strategy.
- Voice: TTS for clean generated lines; `voice-change` when timing and acting from a scratch performance matter. Isolate noisy speech first.
- Runtime: generate into the game project and load through Web Audio. Follow the project's asset/version-control policy; do not commit or publish unless asked. No API keys in browser code.

## Recovery and Coordination

For coordinated games use the director's `references/asset-recovery.md`. Preserve outputs and the audio trigger mapping before retrying work. Distinguish missing credentials, permissions, exhausted credits, invalid input, and transient service failures. Reconcile an uncertain paid request before resubmitting; these one-shot commands do not implement Tripo task resume. Keep independent gameplay work moving and provide a local/synthesized fallback with honest limitations when genuinely blocked.

Listen to a representative effect or line before generating a batch. Test it through its real game event, then give the lead paths and local playback findings for one consolidated QA pass.

## Report

Generated and processed file paths, the prompts, composition plans, text, source files, voice IDs, durations, loop flags and loop points and formats behind them, the runtime trigger mapping and audio groups, and any remaining gaps or plan limits.