references/design-format.md
# Design format: the JSON that mix.py renders
The soundtrack is data. One top-level design (`audio/sounddesign.json`) includes one fragment per category
(`audio/design/<category>.json`); `mix.py` places real recordings on the picture's cue events, mixes three buses (sfx,
amb, music) and masters. Nothing is synthesized: the only processing is trim, fade, gain, pan, EQ, resampling (pitch),
a gain envelope, Rubber Band tempo stretch of the music, summing and mastering.
Templates: `templates/sounddesign.json` (top level), `templates/design_fragment.json` (every feature; its file names
are placeholders, so it renders with `templates/cues.json` once each one is replaced by a real recording from your
manifests), `templates/cues.json` (the cue file). `python` below = the interpreter `{{PY}}` described in SKILL.md.
```
python -I <skill>/scripts/mix.py --root . --design audio/sounddesign.json --cues out/cues.json --trace out/qa/adds.json
python -I <skill>/scripts/mix.py --only keyboard # solo render of the fragments whose path contains "keyboard"
python -I <skill>/scripts/mix.py --only keyboard --dump out/audio-review/keyboard/dump # every sample as a WAV
```
Every relative path (design, cues, outputs, each sample's `file`) resolves against `--root`. Keys starting with `_` are
comments (write the reason for every non-obvious number there: `_doc`, `_about`, `_buses_doc`); `mix.py` ignores
unknown keys everywhere except inside `buses` (each key there must be `sfx`, `amb` or `music`).
## Contents
1. Top level and includes
2. The cue file
3. samples
4. slices
5. pools
6. rules (cue-driven placements)
7. place (free placements)
8. ambience
9. music
10. buses and master
11. What mix.py writes
12. qa_ok: acknowledging a deliberate exception
13. Worked examples from a shipped film
## 1. Top level and includes
| key | meaning |
|---|---|
| `duration` | film length in seconds (default: cue `total` / `fps`) |
| `cues` | default cue file (the `--cues` flag wins; default `out/cues.json`) |
| `include` | fragment paths; their `samples`, `slices`, `pools` are merged (a name defined twice is an error), `rules`, `place`, `ambience` are concatenated, `music` is taken when the top level has none |
| `samples`, `slices`, `pools`, `rules`, `place`, `ambience`, `music` | as below (allowed at the top level too) |
| `buses`, `master` | read from the top-level file only |
Name everything with the fragment's prefix (`kb.`, `ui.`, `mo.`, `am.`, `mu.`): fragments are built by different agents
in parallel and must never collide. Variation (cycle order, jitter, random pitch) is keyed on rule names, so a solo
render (`--only`) plays exactly the same sounds as the full mix.
## 2. The cue file
`{fps, total, events: [{f, kind, id?, weight?, pan?, variant?, dur?, apexFrac?, pitch?}], acts?, cue?}` or a list whose
first element is that object. This is Cinetic's `out/cues.json` format (`scripts/export-cues.ts`), so a Cinetic film
plugs in directly (`docs/INTEGRATION.md`). `f` is the absolute frame the sound belongs on (fractional is fine).
Compute cue frames from the picture code (contact frames, 50 % pop frames, velocity peaks); typed-in sync points land
8-11 frames off in practice. A design with no rules needs no cue file.
## 3. samples
`name -> {file, start?, end?, gain_db?, norm_peak_db?, mono?, fade_in?, fade_out?, eq?, pitch?, glide?, env?}`:
a trimmed piece of a recording.
| key | default | meaning |
|---|---|---|
| `file` | required | the recording (any format ffmpeg reads; decoded to 48 kHz float stereo, cached) |
| `start`, `end` | 0, end of file | the trim in source seconds. Start 3-5 ms before the attack; end after the decay, before the next event |
| `mono` | false | fold to mono (point sources: keys, clicks, steps). Keep stereo for whooshes, beds, wide impacts |
| `eq` | none | `{hp, hp_order (2), lp, lp_order (2), peq: [[f0, q, gain_db], ...], lowshelf: [f0, gain_db], highshelf: [f0, gain_db]}` (Butterworth HP/LP, RBJ biquads) |
| `pitch` | 0 | semitones by resampling (tape-style: pitch AND length change together). +2..+6 makes a hand-sized sound miniature; keep tuned material within +-2 |
| `glide` | none | `[from, to]` semitones: a varispeed glide across the sample (band-limited, no aliasing) |
| `env` | none | `[[t_from_sample_start, dB], ...]` piecewise-linear gain inside the sample (shape a riser, duck a tail) |
| `norm_peak_db` | none | normalise the processed sample's peak to this level (makes a pool even before rule gains) |
| `fade_in`, `fade_out` | 0.002, 0.012 | raised-cosine fades in seconds, applied after normalisation |
| `gain_db` | 0 | static gain after everything |
Processing order: mono -> eq -> pitch -> glide -> env -> norm_peak_db -> fades -> gain_db. `reverse` is refused (a
reversed recording is still real, but the design keeps to what the recording is).
## 4. slices
`name -> {file, start?, end?, delta?, min_gap?, gate_db?, pre?, len?, stop_before_next?, pick?, max?, ...sample opts}`:
cuts a recording of many hits (typing, clicks, steps, dominoes) into single hits at its detected onsets. The slices are
named `name#0`, `name#1` ... in the order they are kept.
| key | default | meaning |
|---|---|---|
| `delta` | 0.07 | onset detector sensitivity (librosa, hop 128, backtracked); lower finds softer hits |
| `min_gap` | 0.06 | seconds between onsets |
| `gate_db` | -18 | drop onsets whose first 30 ms peak is this far below the loudest (handling noise, far keys) |
| `pre` | 0.004 | seconds kept before each onset |
| `len` | 0.16 | maximum slice length |
| `stop_before_next` | true | end each slice `pre` before the next onset (never two hits in one slice) |
| `pick` | all | list of onset indices to keep, in this order (after `pick`, `name#0` is the first picked) |
| `max` | all | keep at most this many |
| sample opts | | `mono`, `eq`, `pitch`, `glide`, `env`, `norm_peak_db`, `gain_db`, `fade_in` (0.002), `fade_out` (0.02) |
Always `--dump` a slice group and read `_samples.json` (duration, peak, attack, head/tail level per slice) before
using it; qa.py's edges section flags slices that start mid-sound or are cut mid-decay.
## 5. pools
`name -> [sample names]`; `"group#*"` expands to every slice of a slice group (sorted by index). An unknown name or an
empty pool is an error. A rule plays from a pool: one name = always that sound; several = variation.
## 6. rules (cue-driven placements)
`[{name, match, pool, mode?, gain_db?, weight_db?, weight_ref?, jitter_db?, pitch?, pitch_ramp?, gain_ramp?, pan?,
pan_scale?, align?, offset?, bus?, no_cue?, cycle_start?}]`. Every cue event that matches a rule gets a sound from the
rule's pool; several rules matching one event LAYER it (a key: thock + bottom-out + tiny motif note).
| key | default | meaning |
|---|---|---|
| `name` | rule index | stable name: the hash key for jitter / pitch / hash picks (renaming a rule changes its variation) |
| `match` | {} | `kind` (regex, full match), `id` (regex, search), `from` / `to` (film seconds, `[from, to)`), `variant` (exact) |
| `pool` | required | the pool to play |
| `mode` | cycle | `cycle`: round-robin, never the same sound twice in a row (`cycle_start` = first index); `hash`: picked per event by a hash of id + frame + rule name; `fixed`: always the first |
| `gain_db` | 0 | level |
| `weight_db` | 0 | dB per doubling of the event's `weight` vs `weight_ref` (default 1.0): bigger events play louder |
| `jitter_db` | 0 | +- random level per event (hashed: deterministic) |
| `pitch` | [0, 0] | `[lo, hi]` random semitones per event (hashed) |
| `pitch_ramp`, `gain_ramp` | none | `[a, b]` added across the matched events in time order: a climbing build, a decrescendo |
| `pan` | "cue" | `"cue"` = the event's `pan` x `pan_scale`, or a number -1..1 |
| `align` | attack | which point of the sound lands on the cue: `attack` (first ms within 20 dB of the peak; impacts, clicks, keys), `peak` (5 ms envelope maximum; whooshes), `end` (risers into a hit), `start` |
| `offset` | 0 | seconds added to the cue time (a release 85 ms after a press; a tail that trails the move) |
| `bus` | sfx | `sfx`, `amb` or `music` |
| `no_cue` | false | the placement does not mark the event as sounded (sweeteners; the A/V audit ignores it) |
Cue events no rule sounds are listed as `unsounded_events` in the report.
## 7. place (free placements)
`[{sample, at, gain_db?, pan?, align?, offset?, pitch?, bus?}]`: sounds with no cue event (a crowd bed, a lift before
a logo, a one-off sweetener). `at` is film seconds; `align` as in rules.
## 8. ambience
`[{name?, file, src_start?, src_end?, at, until, fade_in?, fade_out?, gain_db?, eq?, width?, auto?}]`: beds on the amb
bus. `src_start..src_end` is the clean window of the recording (no voices, dogs, bumps); if the bed needs more than the
window it is looped with a 1.5 s equal-power crossfade (the report says `looped`). `fade_in` / `fade_out` default 1 s.
`width` scales the side signal (0 = mono, 1 = as recorded, 1.15 = a little wider). `auto` = `[[film t, dB], ...]` rides
inside the bed (duck birds under a run, open the room on a cut).
## 9. music
`{file, src_bpm, target_bpm?, stretch?, gain_db?, eq?, segments: [{src_from, src_to, at, fade_in?, fade_out?,
fade_in_at_start?, tail_fade?, gain_db?, label?}]}`
| key | default | meaning |
|---|---|---|
| `src_bpm` | required | the track's measured tempo (arrange.py) |
| `target_bpm` | 120 | the picture's tempo |
| `stretch` | rubberband | `none` plays the source untouched (use it when the track is within ~0.2 % of the film tempo); otherwise the whole file is time-stretched by target/src with Rubber Band (keep the ratio within about +-6 %) |
| `segments[].src_from`, `src_to` | | ORIGINAL source seconds (before any stretch), on the track's downbeats |
| `segments[].at` | | film seconds where `src_from` lands; nudge by a few ms so the measured downbeat transient sits on the picture's downbeat |
| `fade_in`, `fade_out` | 0.008, 0.06 | the segment starts `fade_in` before `src_from` (a pre-roll, so the downbeat's own attack is kept) and fades out over `fade_out` after `src_to` |
| `fade_in_at_start` | true | false = no fade at the head (a hard cut on purpose) |
| `tail_fade` | none | seconds of a longer musical fade over the end of the segment (the last chord rings out) |
| `gain_db`, `label` | | per-segment level; a label that says which bars and why |
Edit at bar lines: a splice from bar n's end to bar m's start works when the harmony continues (V -> I, the same
chord) and the energy of the next bar is what the picture wants. `arrange.py` drafts the segment list; check every
splice (qa.py edges section, earshot timeline) and write the bars and the reason in each `label`.
## 10. buses and master
`buses: {sfx|amb|music: {gain_db?, eq?, auto?}}`: after everything is placed, each bus gets eq, then the `auto` ride
(`[[film t, dB], ...]`, piecewise linear), then `gain_db` (the music bus first gets `music.eq` and `music.gain_db`).
Rides shape the arc: the music dips under typing, lifts into the drop, settles under the logo. Put each step on a
musical splice or under a transient, never in the middle of a sustained note.
`master: {lufs?, ceiling?, no_glue?, no_comp?, mono_hz?}` (defaults -14, 0.77, true, false, 120): `master.py` high-passes
at 28 Hz, folds below `mono_hz` to mono, compresses gently (2:1 above -14 dBFS, unless `no_comp`), then runs loudness
passes with a 4x-oversampled true-peak limiter to `ceiling` (linear), a 3 ms fade-in at sample 0, a fade onto the last
sample and 480 zero samples at the end. The gate passes at |LUFS - target| <= 0.5 and true peak <= -1.5 dBTP.
## 11. What mix.py writes
- `--out` (default `out/audio/soundtrack.wav`): the master, 48 kHz 24-bit; the unmastered sum `premaster.wav` next to
it. With `--no-master` or `--only`, `--out` is the unmastered sum itself (solo default
`out/audio-review/solo/<name>.wav`).
- `--stems` (default `out/audio/stems`): `sfx.wav`, `amb.wav`, `music.wav` (pre-master, they sum to the premaster).
- `--report` (default `out/qa/mix_report.json`): `placements` (t, event, kind, rule, sample, gain_db, pitch, pan,
align), `unsounded_events`, `buses` (peak / RMS), `ambience` (looped, body level), `music_segments`, `files_used`
(every recording that is audible: the input of `credits.py` and `audit.py --report`), `premaster_peak_dbfs`;
`master.json` (loudness gate) next to it.
- `--cues-out` (default `out/qa/cues_sounded.json`): the cue events that received an sfx-bus sound (for an A/V audit).
- `--trace FILE`: every `Mix.add()` (bus, start sample, length, gain, pan, sample / name / label, rule, kind): qa.py
rebuilds every placement from it to attribute clicks, masking and harshness to a rule and a sample.
- `--dump DIR`: every sample and slice as WAV + `_samples.json` (duration, peak, RMS, attack, head/tail level), then exits.
## 12. qa_ok: acknowledging a deliberate exception
A sample, rule, free placement or music segment may carry `"qa_ok": "reason"` (mix.py ignores it). qa.py then lists
that item's edge / sync / audibility / harshness finding as acknowledged instead of a warning: e.g. a riser choked
25 ms before a pop on purpose, a chime that rings past its cue, a hard music cut into a quiet breakdown. Use it only for
decisions a critic has already confirmed, and say why.
## 13. Worked examples from a shipped film
From the Clawd film (60 s, 264 placements, real recordings only), trimmed for reading.
A key thock from a recording of single presses, shaped for a deep "thocky" board:
```json
"kb.thock.1": {"file": "audio/assets/keyboard/fs-789628_keychron-linear-individual-keys.ogg", "start": 1.168,
"end": 1.598, "mono": true, "eq": {"hp": 115, "hp_order": 6, "peq": [[190, 0.8, 3.0], [380, 0.9, 3.0],
[3300, 1.3, -1.5]], "highshelf": [5000, -4.5]}, "norm_peak_db": -10.0, "fade_in": 0.004, "fade_out": 0.16}
```
Typing on 16th notes: one key per typed pixel, climbing a semitone across the run, never the same key twice:
```json
{"name": "kb.type", "match": {"kind": "pop", "id": "^genesis:type"}, "pool": "kb.type", "mode": "cycle",
"jitter_db": 1.5, "pitch": [1.0, 2.0], "pitch_ramp": [0.0, 1.0], "pan": 0.0, "align": "attack"}
```
Dominoes sliced into single clacks, 13 picked after measuring every slice (--dump):
```json
"ty.dom": {"file": "audio/assets/toy/fs-329098_domino-pieces-hit.ogg", "start": 0.2, "end": 16.4, "pre": 0.004,
"len": 0.3, "min_gap": 0.2, "gate_db": -12, "pick": [2, 10, 11, 3, 17, 22, 23, 21, 24, 9, 16, 19, 4], "mono": true,
"norm_peak_db": -3, "fade_out": 0.04, "eq": {"hp": 250, "hp_order": 4}}
```
A stacking flurry that ends ON each row's lock (aligned by its end):
```json
{"name": "ty.row.flurry", "match": {"kind": "pop", "id": "^swarm:row:"}, "pool": "ty.p.row.flurry", "mode": "cycle",
"gain_db": -20.5, "gain_ramp": [-3, 2], "pitch_ramp": [0, 4], "jitter_db": 1.5, "pan": 0.12, "align": "end", "offset": 0.004}
```
Birdsong ducked under the hero's run (an `auto` ride inside the bed):
```json
{"name": "am.meadow_birds", "file": "audio/assets/ambience/fs-677535_woodlark-field-spring.ogg", "src_start": 15.5,
"at": 28.15, "until": 52.2, "fade_in": 1.05, "fade_out": 0.8, "gain_db": 2.4, "width": 0.38,
"eq": {"hp": 300, "lp": 13000, "lowshelf": [700, -5.0]},
"auto": [[28.15, 0.0], [31.7, 0.0], [32.15, -2.5], [32.32, -10.0], [32.78, -10.0], [32.95, -2.5]]}
```
A music edit at bar lines (a 120 BPM track, no stretch), each splice explained:
```json
"segments": [
{"label": "open bars 1-6 (E -> B), film = src - 0.5", "src_from": 0.5, "src_to": 12.5, "at": 0.0, "fade_in": 0.01, "fade_out": 0.05},
{"label": "theme bars 17-24 (entrance on E: V -> I across the splice)", "src_from": 32.5, "src_to": 48.455, "at": 11.995, "fade_in": 0.01, "fade_out": 0.045},
{"label": "drop bars 67-72 (E-major tutti) on the hero reveal", "src_from": 132.489, "src_to": 143.465, "at": 39.995, "fade_in": 0.008, "fade_out": 0.01},
{"label": "coda bars 83-87 (final E on 59.0)", "src_from": 164.489, "src_to": 173.498, "at": 50.981, "fade_in": 0.01, "fade_out": 0.01, "tail_fade": 0.7}
]
```
Bus rides with their reasons (the drop hits, then a diminuendo lets the story read over the tutti):
```json
"_buses_doc": "the opening is lifted (+4.5 music/sfx) so it is soft but present; the DROP hits at -1.5 for its first beat, then -3.5 by 42.45 and -4.5 by 44.3 so the unfold/crowd/stacking story reads; a 2.5 dB duck 47.9-48.7 under the giant lock",
"music": {"auto": [[0, 0], [1.9, 0], [2.1, 4.5], [11.9, 4.5], [11.99, 0], [27.95, 0], [28.0, 5.5], [39.99, 5.5], [40.0, -1.5], [40.5, -1.5], [42.45, -3.5], [44.3, -4.5], [47.9, -4.5], [47.98, -7.0], [48.3, -7.0], [48.7, -4.5]]}
```
references/licences.md
# Licences: what may go into a film, how to verify it, how to credit it
A launch video is commercial use, it is edited (trimmed, layered, synced to picture: legally a derivative), and it is
published on platforms that scan audio. Every recording in the film needs a licence that allows all three, read on the
item's own page, recorded in its manifest, confirmed by the licence audit and credited. When in doubt, leave it out:
there is always another recording.
## Allowed / forbidden
| licence | status | why | credit |
|---|---|---|---|
| CC0 1.0, Public Domain Mark | allowed | no conditions | courtesy only |
| CC BY 4.0 / 3.0 / 2.5 / 2.0 | allowed | commercial use and edits allowed with attribution | REQUIRED (title, author, source link, licence link, "edited") |
| Mixkit Sound Effects Free License / Stock Music Free License | allowed | free for commercial video, no attribution; read the current text at https://mixkit.co/license/ | courtesy only; never redistribute the raw files, never register the film with Content ID |
| Pixabay Content License | allowed | free for commercial use, no attribution (Pixabay blocks scripted access: the user downloads by hand) | courtesy only |
| CC BY-NC (any version) | forbidden | NonCommercial: a launch video is commercial | |
| CC BY-ND | forbidden | NoDerivatives: trimming and syncing to picture are derivatives | |
| CC BY-SA | forbidden | ShareAlike: the whole film would have to be released under CC BY-SA | |
| Sampling+, Sampling | forbidden | retired, restrictive | |
| "personal use", "editorial", "custom", "free with credit on social media", all rights reserved | forbidden | not a standard licence the audit can trust | |
| two sources disagree ("DISPUTED") | forbidden | the strictest reading wins: reject it | |
| "royalty free" with no licence text, YouTube rips, anything behind a login or paywall you would bypass, AI-generated or code-synthesized audio | not used | unknown terms, or not a real recording | |
`scripts/licences.py` encodes this table (`classify()`); audit.py and credits.py use it.
## Verify on the item's own page (never from a search result, an aggregator or memory)
**Freesound** (most foley, ambiences, whooshes):
- The page is `https://freesound.org/people/<user>/sounds/<id>/`; the licence is in the sidebar of THAT page ("Creative
Commons 0", "Attribution 4.0", "Attribution NonCommercial 4.0" ...). Licences are per sound, not per user.
- Record: licence text as shown, page author, title, the date you read it (`licence_seen` in audit.json).
- Originals need a login; the public high-quality preview is `https://cdn.freesound.org/previews/<id // 1000>/<id>_<user_id>-hq.ogg`
(Ogg Vorbis, about 192 kbps; `-hq.mp3` also exists). `<user_id>` is the uploader's number: it must belong to the same
page. audit.py checks that the id in the entry, the page and the preview URL agree.
- Read the description: some authors ask for a specific credit form (e.g. "<title> by <name> (www.site) of
Freesound.org"), or forbid putting raw files online. Copy the requested form into the manifest's `attribution` and
any restriction into the audit's `agent_notes`.
**Mixkit** (lossless WAV effects, some music):
- No per-item page exists for sound effects: record the category page that lists the item and its title there.
- WAV download: `https://assets.mixkit.co/active_storage/sfx/<id>/<id>.wav`. Read the licence text itself
(https://mixkit.co/license/) once per session and quote what it allows in `licence_seen`.
- Do not publish the raw assets folder; do not register the film's audio with Content ID or a rights-management service.
**Music sites** (one licensed track carries the film):
- Composer libraries that release CC BY music (e.g. Scott Buckley, scottbuckley.com.au): the licence line is on the
track's page; many ask for a specific credit line: use it verbatim.
- Aggregators (chosic.com and similar) are not authoritative: confirm on the artist's own page. A real case: an
aggregator listed a track as CC BY 3.0 while the artist's page said "free with credit on social media" (copyrighted):
rejected as disputed.
- Free Music Archive, Internet Archive, SoundCloud: the licence is per track and often NC or ND: read it every time.
- YouTube Audio Library, platform libraries: terms tied to one platform: ask the user before using.
**Pixabay**: scripted downloads are blocked (bot protection). If a Pixabay item is the best choice, give the user the
page link and ask them to download it into the assets folder; then audit it like any other file.
## Attribution (CC BY)
A complete credit names the Title, the Author, the Source (link to the item page), the Licence (name + link) and says
it was modified. The manifest's `attribution` holds the line the author asked for (or this default):
```
"Rain on Windows, Interior, A.wav" by InspectorJ (www.jshaw.co.uk) of Freesound.org
(https://freesound.org/people/InspectorJ/sounds/346642/), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
```
`credits.py` writes `CREDITS.md` (music + every sound, grouped by licence, with the sentence saying how the recordings
were edited) and `out/CREDITS_description.txt` (paste it into the video description). It completes a requested line
with the source link and the licence link when the author's form leaves them out. Credits never need to be on screen.
## Content ID and CC BY music
Some composers register their CC BY tracks with YouTube Content ID through a distributor. A video using the track
correctly can still get an automatic claim. The licence still allows the use: put the credit in the description, and
if a claim arrives, dispute it citing the licence and the credit (many composers also clear claims on request). Choose
a track whose composer explains this on their site, and tell the user before publishing. Mixkit and Pixabay forbid the
reverse: never register a film that contains their items with Content ID.
## The audit (scripts/audit.py, prompts/licence_audit.md)
1. The licence auditor opens every in-scope item's page, reads the licence and writes `licence_seen` (quote + URL +
date + page author/title) and `licence_confirmed: true` into `audio/assets/manifests/audit.json` (format:
`templates/audit_entry.json`). Never confirm a page you did not read in this session.
2. `audit.py --report out/qa/mix_report.json` classifies every manifest entry, checks ids and URLs, checks every file
(exists, codec, rate, channels, duration vs the manifest, decodes without errors), merges the auditor's fields and
writes audit.json + a Markdown table. In-scope = audible in the film. Exit 1 while any in-scope recording is blocked.
3. `--fetch` compares the local bytes with `download_url` (sequential, 1.5 s apart, a clear User-Agent, in memory,
nothing executed; never Pixabay). Re-encoded or edited files show up here.
4. `credits.py --audit audio/assets/manifests/audit.json` refuses to call the credits clean while a used recording is
unconfirmed.
On the Clawd film this caught four used recordings (celesta notes added after the hand audit) with no audit record.
Their manifest notes do say the licence was read on the Freesound page, so it was a bookkeeping gap, not an unverified
licence: audit.py quotes such a note and asks the auditor to copy it into `licence_seen`.
references/process.md
# The process: how a real-recordings soundtrack gets built by agents who cannot hear
This is the process that built the Clawd film's soundtrack (60 s, 83 real recordings, 264 placements, reproduced
bit-identically by `scripts/mix.py`), generalised. Agents cannot listen, so every decision is measured, every claim is
backed by a command, and the work is split so that parallel agents never edit the same file.
## Contents
1. Roles and files
2. The steps
3. What each critic measures (and the qa.py section that automates it)
4. Thresholds and where they come from
5. Variants: how critics test a fix without touching the design
6. Rounds, stopping, reporting
## 1. Roles and files
| role | prompt | owns (writes) | reads |
|---|---|---|---|
| lead | SKILL.md | `audio/SOUND_BRIEF.md`, `audio/sounddesign.json` (top level: include list, buses, master), final reports | everything |
| sourcing agent (one per category) | `templates/prompts/sourcing.md` | `audio/assets/<category>/`, `audio/assets/manifests/<category>.json`, `out/audio-review/<category>/` | brief |
| licence auditor | `templates/prompts/licence_audit.md` | agent fields of `audio/assets/manifests/audit.json` | manifests, item pages |
| music supervisor | `templates/prompts/music_supervisor.md` | `audio/assets/manifests/music_choice.json`, `audio/design/music.json` | music candidates, picture map |
| designer (one per category) | `templates/prompts/designer.md` | `audio/design/<category>.json` | its manifest, cues, solo renders |
| critic (one per lens) | `templates/prompts/critic_*.md` | only `out/audio-review/critic_<lens>/` (variants, measurements) | the whole render, design, trace |
| fixer (exactly one) | `templates/prompts/fixer.md` | every fragment + top-level buses/master during the fix round | all critic reports |
**Filling a prompt.** The lead replaces every `{{NAME}}` before handing a prompt to a sub-agent; an unfilled
placeholder must never reach one. The common ones: `{{PY}}` = the full path of the Python that has numpy, scipy,
soundfile and earshot (see SKILL.md, e.g. `<earshot repo>/.venv/Scripts/python.exe`); `{{SKILL}}` = this skill's folder;
`{{ROOT}}` = the film project root; `{{CATEGORY}}` = the sound category the agent owns; `{{FILM}}` = the film's name;
`{{ROUND}}` = the critique round number; `{{RENDER}}` = the folder of the render under review (soundtrack.wav,
stems); `{{REPORT}}` / `{{TRACE}}` / `{{QA_JSON}}` = its mix report, placement trace and qa.py report; `{{VIDEO}}` =
the rendered film; `{{CUES}}` = the cue file. The rest (`{{NEED_DESCRIPTION}}`, `{{KEY}}`, `{{BPM}}`, `{{DROP_T}}` ...)
are explained in the prompt that uses them.
## 2. The steps
1. **Cue list from the picture.** `out/cues.json` (format in `templates/cues.json`; Cinetic writes it with
`export-cues.ts`). Sync frames are computed from the picture code (contact frames, 50 % pops, velocity peaks), never
typed. Check the event kinds and ids are stable enough for rules to match with a regex (`act:thing:n`).
2. **Brief.** `audio/SOUND_BRIEF.md` from `templates/SOUND_BRIEF.md`: concept, picture map (every cue with what is on
screen and the sound need), ambience zones, licences, quality bar, file layout, manifest format. The brief is the
contract every sub-agent receives verbatim.
3. **Sourcing, per category, in parallel.** 15-40 candidates per category, licences read on each item's page,
downloads polite (sequential, a pause between requests), each file measured (`analyze.py`, spectrogram PNG) and
heard through earshot (`realness` gate, `describe` / `match` / `rank` against the need, `find` for the best take),
ranked per need in the manifest with notes saying exactly what to use and what to avoid.
4. **Licence audit.** The auditor reads every candidate's page and records `licence_seen`; `audit.py` checks the rest.
Nothing unconfirmed reaches the design.
5. **Music supervisor.** Measures every music candidate (`analyze.py --music`, `arrange.py`), verifies the beat grid
and the downbeats against transients (the beat tracker lags attacks by 20-50 ms; arrange.py corrects the phase),
picks the winner with reasons, and writes the edit: segments at bar lines, the drop's downbeat on the hero frame, each
splice labelled with its bars and why it works.
6. **Design fragments, per category, in parallel.** Each designer trims samples (earshot `find` gives trims; `--dump`
checks them), builds pools and rules on the cue events, and verifies the fragment alone: `mix.py --only <category>`
(solo render, same variation as the full mix), `qa.py --only <category>`, `earshot timeline` of the solo with the
cues. A designer never touches another fragment or the top level.
7. **Integrate.** The lead runs the full mix with `--trace`, then `qa.py` and earshot (`timeline`, `sheet`, `critique`
against the launch-film profile). Fix blockers (missing files, failed gate) before critique.
8. **Critique, in parallel, read-only.** Five lenses (sync, edges, masking, tonal, director), each with its own prompt.
Each critic measures, tests its proposed fixes as variants in its own folder (section 5) and returns ranked issues
with exact JSON operations. Critics never edit the design.
9. **One fixer.** Merges the critics' operations, resolves conflicts (masking wants a hit louder, tonal wants its
2-6 kHz softer: the fixer finds the change that satisfies both, e.g. louder low body + softer top), applies them,
re-renders, re-runs qa.py and compares with the previous round (nothing that passed may now fail). One writer
avoids lost edits and contradictory fixes.
10. **Master and gate.** `mix.py` masters (`master.py`): integrated LUFS within 0.5 of the target, true peak at most
-1.5 dBTP on the WAV; then check again after the AAC encode (about -1 dBTP).
11. **Credits.** `audit.py --report out/qa/mix_report.json` (exit 0) then `credits.py --audit ...` (exit 0): CREDITS.md
and the video-description text.
12. **Mux.** `mux.mjs` (stream copy of the picture, AAC 320k, cut to the picture's duration); verify the audio track of
the MP4 against the WAV (lag 0 samples) and run `earshot timeline film.mp4 --cues out/qa/cues_sounded.json` on the
finished file.
## 3. What each critic measures
| lens | question | measurements (Clawd critic scripts, now qa.py) | typical fixes |
|---|---|---|---|
| **sync** | does every sound land on its cause? | per event: attack, 5 ms peak and 50 ms apex of the rebuilt event vs its cue (qa.py sync); risers by their end; per-frame picture motion around the event (frame differences of the film, earshot timeline cut/cue sync); envelope shape of each dumped sample (attack, -3/-6 dB span); solo renders of one rule | `align` (attack/peak/end), `offset`, a trim that starts closer to the attack, a different take whose peak is earlier |
| **edges** (clicks) | is anything cut, clicked or truncated? | > 14 kHz bursts concentrated in 1 ms in the master and every stem, attributed to the layer carrying the burst (qa.py clicks); unfaded head / tail of each used sample vs its peak and the source just outside the trim; second hits inside a sample; slices; placements past the film ends; music splices (the source level just after a segment end vs what enters) (qa.py edges) | longer `fade_in`/`fade_out`, move `start` 3-5 ms before the attack, end after the decay, `stop_before_next`, a different splice bar, `tail_fade` |
| **masking** (audibility, density) | can every effect be heard, and is the effect layer dense or repetitive? | per event: 1/3-octave margin vs the bed and vs everything else over its loudest 60 ms and its first 20 ms (`read_db`), per layer (dead layers); onsets per second; sample and take reuse; per-rule regularity (inter-onset CV, gain and pitch spread, distinct samples) (qa.py audibility, density); A/B variants of crowd beds | gain, EQ that carves the effect's band (or the bed's), a brighter take, fewer layers, a bus ride under the moment, more takes in a pool, `jitter_db` |
| **tonal** | is it pleasant on phones and laptops? | per event 2-6 kHz 5 ms peak vs the music's loudest (qa.py harshness) and its centroid; band levels over time of master vs stems (2-6k, 6-10k, 10-16k, >16k); low end under 120 Hz (mono, the impact's low partials vs the music's at the drop: beating); stereo width per section; tuned samples vs the key; limiter gain reduction at candidate (lufs, ceiling) settings | `highshelf` / `peq` cuts, `norm_peak_db` down, `mono`, `lp`, a pitch to the key, `master.lufs` / `ceiling` |
| **director** | does the soundtrack tell the film's story? | momentary / short-term loudness of the master and each stem across the picture map's beats, LRA, section medians, the loudest moment's position, the drop vs the logo, every beat's sfx and music peaks, ambience sources per zone (qa.py loudness + earshot timeline + sheet + critique) | bus rides, the music edit, which events are sounded, the ending |
## 4. Thresholds and where they come from
| check | threshold | source |
|---|---|---|
| click | > 14 kHz sample > 9x its 20 ms RMS and > 6x its 4 ms RMS, >= 80 % of the burst's energy in 1 ms | Cinetic av-audit's detector, ported (the same 7 click times as av-audit on the Clawd master: a check of the port, not independent validation) |
| click at an edge = a cut | the carrying layer's start or end within 2 ms | qa.py (fail); inside a recording = warn (the recording's own crisp transient, or a defect in the source) |
| abrupt entry | source 20 ms before the trim within 20 dB of the trim's peak, fade-in < 5 ms, and no attack within 10 ms to cover it (or the source within 10 dB) | Clawd critic_clicks edges.py (-30 dB listing), tightened so attacks that follow at once are not flagged |
| chopped decay | unfaded tail > -35 dB re peak, the source after the trim > -35 dB, fade-out < 30 ms | Clawd critic_clicks edges.py |
| chopped music | the source keeps sounding after `src_to` (within 6 dB) and is 6 dB above what enters, fade-out < 100 ms, no `tail_fade` | qa.py (generalises the supervisor's splice listings) |
| audibility | read = max(60 ms margin vs everything, 20 ms margin vs everything) over bands within 30 dB of the effect's strongest: >= 6 clear, 3-6 ok, 0-3 barely, < 0 masked | Clawd critic_masking measure.py |
| harshness | an event's 2-6 kHz 5 ms RMS more than 6 dB over the music's loudest 2-6 kHz 5 ms RMS | qa.py (the critic listed both levels; the 6 dB margin is a judgement); centroid > 6.5 kHz noted (norm) |
| sync | impacts: attack >= -45 ms and peak <= +75 ms from the cue; whooshes and peak-aligned swells: apex within +-45 ms; risers and end- / attack-aligned swells: audible end (last sample above -20 dB re its peak) -250..+30 ms (checked at -300..+60); a deviation that one layer's rule offset removes is listed as deliberate, naming that rule | measured norms of pro launch films (Cinetic sound.md §13) |
| machine gun | a rule with >= 4 events, one sample, pitch sd < 0.1 st, gain sd < 0.5 dB | Clawd critic_density metrics.py (regularity table) |
| loudness | integrated within 0.5 LU of the target (fail), true peak <= -1.0 dBTP (fail), last 50 ms <= -45 dBFS RMS and last 10 ms <= -60 dBFS peak (fail), first 5 ms <= -40 dBFS (warn), LRA 4-10 LU (warn), loudest moment before 85 % of the runtime (warn) | master.py gate, Cinetic av-audit head/tail, norms |
What qa.py flagged on the finished Clawd mix (which had been through five critics and a supervisor): 7 clicks in the
master, all inside recordings (the detector is av-audit's, so these are the transients the critics had already heard
about and accepted as crisp), 5 of 153 events masked (busy stacking moments), 3 events hot at 2-6 kHz, 2 edge defects
(one a deliberate 25 ms choke, one a deliberate music cut into the meadow), no sync defect (4 events sit off their cue
by exactly a rule offset, listed as deliberate with the rule that explains each), 2 cue events unsounded. Treat findings as questions
for a critic, not as orders; acknowledge confirmed exceptions with `qa_ok` (design-format.md §12).
## 5. Variants: how critics test a fix without touching the design
A critic writes its proposed operations to a JSON list and lets `variant.py` copy the design (top level + fragments)
into its own folder, apply them, render with every output inside that folder and run qa.py on the result:
```
python -I <skill>/scripts/variant.py --ops out/audio-review/critic_tonal/ops_v1.json \
--out out/audio-review/critic_tonal/v1 --render --qa [--cues out/cues.json]
```
It prints every change as old -> new (an operation that matches nothing writes nothing), then the mix's gate line and
the qa digest; `v1/qa.json` sits next to the base render's `out/qa/qa.json` for a before / after comparison. The
critic reports those numbers next to each proposed operation. Operations are exact and small:
`{"f": "keyboard", "sample": "kb.thock.1", "set": {"norm_peak_db": -10}}`,
`{"f": "motion", "rule": "mo.leap.air", "set": {"offset": 0.30}}`,
`{"sd": "buses.music.auto", "value": [[0, 0], ...]}` (all forms in `templates/prompts/fixer.md`). The fixer applies
the merged list to the real files with `variant.py --apply` (originals copied to `--backup` first).
Variants catch interactions nobody predicts by reading JSON. A real case on the Clawd design: a 4 dB high-shelf cut on
a plastic "toy open" sample made its event LOUDER at 2-6 kHz (-15.8 -> -15.2 dBFS), because `norm_peak_db`
renormalises the sample after EQ (design-format.md §3, processing order); the fix is a lower `norm_peak_db` with the cut.
## 6. Rounds, stopping, reporting
- One critique round + one fix round is the minimum; a second round when the first fix round changed levels or the
music edit. Stop when qa.py shows no fail, every remaining warn is acknowledged with a reason, and the director lens
has no P0.
- Every number in a report comes from a command that was run; the report lists the commands.
- Keep `out/audio-review/` (critic folders, variants, dumps): it is the audit trail of why each number is what it is.
references/sound-design.md
# Sound design rules: launch videos and code-made films
The rules the designers, critics and fixer work from. Numbers marked *(norm)* are measured norms of top-tier launch
films (from the Cinetic skill's measurements, `references/sound.md` §13 there); the rest is practice that held up on
shipped films. The earshot repo's `docs/launch-video-sound.md` profiles 7 professional reference launch films in depth
(earshot's bundled launch profile); use it for the targets of a specific film, this file stands alone for the craft.
## 1. The bar
- **Every sound has a visible cause**, at a loudness that matches the cause: a caret blink whispers, the hero reveal
hits. Not every picture event gets a sound; once a class of action is sounded, every instance is, at one level.
- **Real and close.** Professional recordings: close-miked, dry, clean (floor under -55 dBFS between events), full
bandwidth (95 % rolloff above 12 kHz), no clipping, no room of their own, no background events.
- **One palette.** One material family per film (felt + wood, glass + soft plastic, metal + air), one click voice, one
keyboard, one music track. A different material for every sound is the fastest way to sound cheap.
- **The music carries the film.** Effects punctuate; the music's own events (drop, stop, re-entry) carry the big
moments and most cuts.
- **Restraint.** About 5 designed effects per 10 s *(norm; 0-28)*, clustered where the music is sparse (cold open,
typing, logo build) and thinning out after the drop. Good sound design mostly disappears.
## 2. Launch-video grammar (picture event -> sound)
| picture | sound (real source) | where it sits | level / notes |
|---|---|---|---|
| cursor moves, hovers, scrolls, progress bars, streaming text, rolling counters | nothing | | the music carries continuous motion |
| cursor click / tap / press | one real click voice for the whole film: mouse, trackpad, phone tap on glass; press + release 60-110 ms later (about 85, release 0-3 dB quieter) *(norm)* | the press frame (clicks land 10-30 ms before the visible press in pro films) | about 3 dB under the music's short-term level, 12-20 dB over it above 3 kHz *(norm)*; bright but short (-20 dB within 6-10 ms); centred; the same level every time |
| typing | sliced real keyboard (mechanical or laptop, one board per film); space / return deeper and longer | each sounded key on its character frame; at most ~12 keys/s; above ~25 chars/s one key per word or silence *(norm)* | music dips 6-10 dB under the run; vary the key sample and level (sigma ~2 dB, no jump over 6 dB), not random pitch; most on-screen typing in pro films is silent *(norm: ~75 %)* |
| a card / chip / toast / badge appears | a soft real pop or tick: cork, mouth pop, plastic click, wood block, pitched up for size | 50 % of its travel | a run climbs (`pitch_ramp` +1..+3 st); +2..+9 dB over the music for word-like pops *(norm)* |
| toggle, switch, lock-in, snap into place | a real latch, switch, magnet clack, Lego click | the contact frame | short; in sync within a frame |
| a counter or metric locks on its value | one crisp tick on the final value only | the lock frame | about 8 dB under the music *(norm)* |
| a camera-scale move, whip, zoom into the product, logo zoom | a real swish (bamboo, stick, cloth, cape, air past the mic) | apex on the move's velocity peak, onset 100-450 ms ahead *(norm)* | about 2 per film, never on every cut; may be wide; travels with the motion (pan) |
| hard cuts | nothing | | about 85 % of cuts in pro films have no sound of their own *(norm)* |
| tension into the reveal | a real riser: cymbal roll (yarn mallets), orchestral swell, the music's own build | ends 30-250 ms before the hit (a beat of air), or stops on it | in about 40 % of films *(norm)*; never a crash cymbal on the hit |
| the hero reveal / the turn | the music's DROP lands on it (edit the track so its downbeat is on the frame); optionally one real impact: drum, taiko, kick, felt sub hit, with a short top transient | the contact / cut frame | the loudest moment of the film, at 30-60 % of the runtime, +2..+7 LU over integrated *(norm)* |
| the logo / lockup | the music stops on a bar line or filters down, 0.2-0.7 s of near-silence, then ONE soft resolved element: a bell, celesta or piano note in the track's key, a soft swell | the frame the wordmark becomes readable (its settle), never later | 8-20 dB under the film's peak *(norm)*; a 0.5-2.5 s tail to below -60 dBFS, then silence |
| product objects (a phone, a card, a box) | real foley of the object's material | contacts | dry, centred |
## 3. Music
- **One licensed track** at 110-130 BPM (or half-time 55-65) whose structure fits the film: soft intro, groove, a
breakdown or stop at 40-80 % of the runtime, a drop for the hero reveal, a resolve. Steady tempo, 4/4,
professionally mastered, full bandwidth, instrumental.
- **Avoid:** vocals, corporate ukulele + whistling, EDM brostep, lo-fi hiss, trailer braams on a product film,
dark/tense moods for a friendly product, famous overused stock tracks.
- **Match the brand's personality:** calm/literary -> soft piano, felt, strings, no booms; fast/exact -> tight drums,
plucks, a drop on true silence; bright/friendly -> marimba, claps, pops, major key; precise/expensive -> sparse piano,
low strings, air; raw/mechanical -> ticks, latches, metallic hits; warm/kind -> plucked strings, soft pads.
- **Edit at bar lines** (`arrange.py`): the picture is cut on the grid (120 BPM = 2 s bars at 4/4), the music is cut on
its downbeats, splices continue the harmony (V -> I, or the same chord), and the drop's downbeat lands on the hero
frame. Stretch only when the tempo is off by more than ~0.2 %, and by at most ~6 %.
- **Shape with rides, not with new music:** dip under typing, lift into the drop, settle under the logo (`buses.music.auto`).
## 4. Levels, arc and mix
- -14 LUFS integrated for web and social; true peak at most -1 dBTP after the AAC encode (`master.ceiling` 0.77 gives
that). -15/-16 only for a stated reason (a dynamic orchestral track the limiter would squash).
- LRA 5-9 LU for a film with an arc *(norm: 1-2 sounds crushed)*; the intro 5-12 LU under the body for its first
2-3 s, entering with intent on frame 0 (no fade over the first bar, no click on sample 0).
- The loudest momentary sits on the drop / hero reveal, not on the logo, not in the last 15 %.
- Effects are dry, mono and centred (+-0.15 pan); only whooshes, risers, ambiences and the music are wide. Low end
below ~120 Hz is mono (master.py folds it).
- Every effect at least +3 dB over the bed in its own best band (qa.py `read_db`; >= 6 is clear); a layer that adds
nothing measurable is a wasted placement (qa.py `--layers`).
- Nothing in the 2-6 kHz band much louder than the music's own loudest presence peaks (qa.py harshness): that band
hurts first on phones and laptops. Effect centroids at or below 6.5 kHz *(norm)*; pings at 9-14 kHz fatigue.
## 5. Sync
- Impacts, clicks, keys, pops: `align: attack` on the contact / press / 50 % frame. Whooshes: `align: peak` on the
velocity peak. Risers: `align: end` on (or 30-250 ms before) the hit. Beds and crowds: `align: start`.
- A sound that leads its picture by more than ~45 ms reads as "sound first"; one that peaks more than ~75 ms late reads
as late *(norm: accents land -45..+75 ms, median +8)*. Move the picture to the beat, not the sound off its cause.
- Check slow-attack samples: an attack-aligned sample whose peak comes 80 ms after its first transient lands late
(qa.py sync reports attack and peak per event).
- Cue frames are computed from the picture code; typed-in times land 8-11 frames off *(Cinetic, in practice)*.
## 6. Variation without randomness
- A repeated action draws from a pool of 4+ takes (`mode: cycle`: never the same take twice in a row), +-1-2 dB jitter,
+-0.3-0.5 st pitch at most (real keyboards do not change pitch per key: vary the take, not the pitch).
- The UI click is the exception: one voice, one level, every time.
- A progression climbs: `pitch_ramp` across a run of pops or stacking blocks, `gain_ramp` into a build.
- Machine-gun (one sample, no variation, regular rhythm) is the second-fastest cheap tell; qa.py density flags it.
## 7. Tuning
- Tuned elements (chimes, bells, celesta, a motif) play in the track's key (`analyze.py --music` and `arrange.py`
estimate it); tune a recording by resampling (`pitch`) within +-2 semitones; choose a recording near the right note
rather than shifting far.
- Booms and subs on the track's root, or neutral (unpitched) at least; an untuned boom under a resolving chord sounds
cheap even to people who cannot say why.
## 8. Film foley (character pieces, miniatures, worlds)
- **Scale through pitch and closeness:** a 1 cm character's steps are fingertip taps pitched +2..+6 st, close and dry;
a giant's are slowed (-3..-6 st) heavy foley with a longer tail.
- **Layer by role:** body (the weight, 100-400 Hz) + transient (the crisp contact, 2-6 kHz, short) + detail (material:
rattle, creak, rustle). Each layer is one rule on the same cue; check each adds something (`--layers`).
- **Ambience zones:** one bed per location, from clean windows of real field recordings (no voices, traffic, planes,
dogs unless the story wants them), cut with the picture's cuts, ducked under key actions with `auto`.
- **Materials stay consistent:** the same key always sounds like the same key; the same surface like the same surface.
## 9. Choosing a recording (agents cannot listen: measure)
- `analyze.py` (format, floor, bandwidth, clipping, onsets, spikes in an ambience, tempo/key for music) and the
spectrogram PNG; `earshot describe / match` (does it sound like the description? use earshot's verdict words:
strong z >= 3, plausible 2.5-3, weak 2-2.5, no match below 2),
`earshot rank` (sort candidates), `earshot find` (the best take inside a long recording: ready-made trims),
`earshot compare --refs` / `earshot ab` (closer to approved references?), `earshot realness` (a real recording, not
code-synthesized: require "real or pro-produced" on the trim you use; an "uncertain" needs its evidence read and
written down).
- Prefer: lossless or HQ previews (Freesound -hq.ogg, Mixkit WAV), 44.1/48 kHz or higher, many takes of the same object
(variation for free), a clean floor between takes.
- Reject: clipped, low-bitrate (a cliff above 16 kHz), roomy foley, anything with voices or bumps inside the window you
need, anything whose licence you cannot read on its own page.
## 10. The cheap tells (what makes a soundtrack sound like random stock effects)
Code-synthesized beeps, sine pings and noise sweeps; one sample machine-gunned; a whoosh on every cut; a boom on every
logo; sparkle / shimmer sweeps standing in for "magic"; reverb on tiny sounds; random pitch on typing; partial
coverage (some presses heard, others silent); clicks at cuts; music chopped mid-note at a splice; an audible ambience
loop; a different room on every sound; the logo louder than the drop; an ending that stops mid-note instead of a tail
into silence.
references/troubleshooting.md
# Troubleshooting
## mix.py
| symptom | cause | fix |
|---|---|---|
| `cue file out/cues.json not found (the design has cue rules)` | no cue file at the default path | pass `--cues path` or set `"cues"` in the design |
| `set "duration" (seconds) in the design or "total" (frames) in the cue file` | neither is known | add `"duration"` to the top-level design |
| `KeyError: 'pool X: unknown sample Y'` | a typo, or a slice group without `#*` | fix the name; `"kb.type#*"` expands every slice |
| `ValueError: pool X is empty` / `slices F: no onsets found` | the slice window has no onsets above the gate | widen `start`/`end`, lower `delta` (0.05), lower `gate_db` (-24), check the file with `analyze.py` |
| `KeyError: 'fragment.json: samples "x" is defined twice'` | two fragments use one name | prefix every name with its category (`kb.`, `mo.`) |
| `KeyError: '_doc'` in the bus loop | a comment key inside `buses` | move comments to `_buses_doc` at the top level |
| `ValueError: reverse is not used` | a sample asks for `reverse` | find a recording that already has the shape (a real swell, a reverse cymbal recorded as such) |
| `rubberband failed` | the ffmpeg build has no librubberband | use `"stretch": "none"` when the track is within ~0.2 % of the film tempo, else ask the user for an ffmpeg with rubberband (`ffmpeg -filters | grep rubberband`) |
| `master gate FAILED` | integrated loudness or true peak missed | read `master.json` next to the report: lower `master.lufs` by 1 for a dynamic track, or trim the loudest peaks (often one impact or the drop's first beat) with a bus ride; `ceiling` 0.77 leaves AAC headroom |
| the first run is slow | every recording is decoded once (and the music stretched once) | later runs read `out/audio/cache` (keyed on path, size, mtime); `--cache DIR` moves it |
| a solo render "has no file" | older habit: looking for premaster.wav | `--only NAME` writes the unmastered mix to `--out` (default `out/audio-review/solo/<NAME>.wav`) |
| the solo sounds different from the full mix | a rule without `name` (variation keyed on its index) | name every rule; then solo and full mix play identical placements |
| an event is in `unsounded_events` | no rule matches its kind / id / time | add a rule, or accept it on purpose (qa.py coverage lists partial classes, which read as mistakes) |
## qa.py
| symptom | cause | fix |
|---|---|---|
| sections "skipped: needs the trace" | the mix ran without `--trace` | re-run `mix.py ... --trace out/qa/adds.json` |
| `trace (N placements) and report (M) disagree` | the trace and the report come from different runs or a different `--only` | re-render with both from one run; pass the same `--only` to qa.py |
| `placement(s) could not be rebuilt` | the design changed after the mix | re-render before qa |
| a click "inside a recording" | the recording itself has a crisp transient (a tick, a pop) or a defect (a digital click, a bump) | `--dump` the sample and look at its spectrogram (`analyze.py --png`); if it is a defect, trim around it or use another take; if it is the sound itself, acknowledge with `qa_ok` |
| a music splice "chops the music" | the outgoing segment ends while its note still sounds and the next one is quieter | end the segment where the chord has decayed, splice into a louder bar, add `tail_fade`, or acknowledge a deliberate hard cut with `qa_ok` |
| many "bright" notes | a tiny-object palette (plastic, glass, bells) is bright by nature | fine when it is the film's palette; otherwise a `highshelf` cut or a darker take |
| `--listen` says earshot is not importable | earshot is not installed in this Python | `pip install -e <earshot repo>`, or run qa.py without `--listen` and call `python -m earshot ...` separately |
## audit.py / credits.py
| symptom | fix |
|---|---|
| every recording "licence not verified" | the auditor has not recorded `licence_seen` / `licence_confirmed` in `audio/assets/manifests/audit.json` yet: run `templates/prompts/licence_audit.md` |
| `licence_seen reads as X but the manifest says Y` | the page and the manifest disagree: re-read the page, fix the manifest's `license`, or reject the item |
| `used in the film but in no manifest` | a file was added to a fragment without a manifest entry: add the entry (and audit it) or remove the sound |
| `entry id ... but the Freesound page is sound ...` | the entry, the page and the download URL describe different sounds: re-download from the right page |
| credits list "PROBLEMS" | a used recording is forbidden, unknown, unconfirmed or lacks author / title / page_url: fix before publishing |
## Sources and downloads
| symptom | fix |
|---|---|
| Freesound preview 404 | the preview folder is `id // 1000` and the number after `_` is the uploader's user id (take it from the page's player) |
| Pixabay returns 403 / a challenge page | Pixabay blocks scripted downloads: give the user the link and ask them to download the file |
| a Mixkit item has no page | Mixkit SFX have none: record the category page and the listed title |
| a file decodes with errors | re-download; prefer the lossless (Mixkit WAV) or HQ (Freesound -hq.ogg) version |
## Mux and delivery
| symptom | fix |
|---|---|
| true peak over -1 dBTP after AAC | lower `master.ceiling` (0.77 -> 0.72) and re-render; measure the MP4's decoded audio with `python -m earshot measure film.mp4` (`master.py --measure` reads WAV / FLAC only) |
| the audio ends early | the soundtrack is shorter than the picture: set `duration` to the picture's length (mux.mjs reports the mismatch) |
| sync looks off | `mux.mjs` stream-copies the picture and cuts the audio to the picture's duration; cross-correlate the MP4's decoded audio with the WAV (lag must be 0) and check the cue frames; never re-time the audio to fix a picture-side offset |
## Platform
- Run the scripts with `python -I` (isolated: downloaded files can never shadow a module); the scripts add their own
folder to the path themselves.
- Console output is ASCII (Windows cp1252 consoles); quote paths with spaces (`--root "/path/to/my film"`).
- `mix.py`'s audio maths are frozen: the Clawd soundtrack reproduces bit-identically
(`sha1 b48c5e8872fe96765624f88db18a0b49a0a05d68`). Change the DSP only with a new version and a new reference.
scripts/analyze.py
#!/usr/bin/env python
"""Objective analysis of downloaded audio assets (we cannot listen, so we measure).
python -I <skill>/scripts/analyze.py FILE [FILE ...] [--music] [--png DIR] [--out OUT.json] [--json]
Prints one plain-language line per file (then a summary line; exit code 1 when a file is missing or does not decode);
--out writes every measurement as a JSON list, --json prints that list instead of the lines.
Every file is decoded with ffmpeg to 48 kHz float stereo (a mono source is duplicated at unity gain). Reports per file:
format (codec, source rate, channels, bitrate), duration, peak / RMS / integrated LUFS, noise floor (10th percentile of
20 ms RMS), clipping, DC, the first transient (attack) time, onset times, spectral centroid, bandwidth (95% rolloff),
and "spikes" (sudden events far above the median level: useful to catch dogs, voices, bumps inside an ambience).
With --music also: tempo + beat times (librosa), tempo stability, key estimate (Krumhansl), loudness per 2 s and per
bar, and novelty-based section boundaries.
With --png: a spectrogram + waveform image per file (ffmpeg showspectrumpic), to look at with an image viewer.
Reading the numbers (rules of thumb for picking recordings):
foley / SFX noise_floor_dbfs < -55 between events, clipped_samples 0, rolloff95_hz > 12 kHz (not dull), a
band_db_rel_1_4k['16-19k'] cliff far below '12-16k' means a low-bitrate encode; spikes_over_12db inside an
ambience = single events (dog, voice, bump) to avoid or cut around.
ambience level_std_db_100ms small (steady bed), side_to_mid_db > -12 (real stereo; null = mono), no spikes,
>= 40 s usable.
music tempo + ibi_std_ms small (steady grid), key, rms_db_per_bar (the arc), sections_at (edit points).
"""
import argparse, json, os, subprocess, sys, math, warnings
import numpy as np
SR = 48000
def probe(path):
try:
out = subprocess.run(['ffprobe', '-v', 'error', '-select_streams', 'a:0', '-show_entries',
'stream=codec_name,sample_rate,channels,bit_rate:format=bit_rate,duration',
'-of', 'json', path], capture_output=True, text=True, timeout=60).stdout
j = json.loads(out)
s = (j.get('streams') or [{}])[0]
f = j.get('format') or {}
brs = [int(b) for b in (s.get('bit_rate'), f.get('bit_rate')) if str(b).isdigit() and 0 < int(b) < 10 ** 7]
return {'codec': s.get('codec_name'), 'src_rate': int(s.get('sample_rate', 0) or 0),
'channels': int(s.get('channels', 0) or 0), 'kbps': round(brs[0] / 1000) if brs else None}
except Exception as e: # noqa
return {'probe_error': str(e)}
def decode(path, channels=2):
"""(n, 2) float32 at SR; a mono source is duplicated at unity gain (ffmpeg's own upmix is 3 dB down)"""
ch = 1 if channels == 1 else 2
p = subprocess.run(['ffmpeg', '-v', 'error', '-i', path, '-map', '0:a:0', '-f', 'f32le', '-acodec', 'pcm_f32le',
'-ac', str(ch), '-ar', str(SR), '-'], capture_output=True, timeout=600)
if p.returncode != 0 or not p.stdout:
raise RuntimeError(p.stderr.decode(errors='replace').strip()[:400] or 'no audio decoded')
x = np.frombuffer(p.stdout, dtype=np.float32).reshape(-1, ch)
return np.repeat(x, 2, axis=1) if ch == 1 else x.copy()
def db(x):
return float(20 * np.log10(max(float(x), 1e-12)))
def frame_rms(mono, win):
n = len(mono) // win
if n == 0:
return np.array([np.sqrt(np.mean(mono ** 2) + 1e-20)])
f = mono[: n * win].reshape(n, win)
return np.sqrt(np.mean(f ** 2, axis=1) + 1e-20)
KK_MAJ = np.array([6.35, 2.23, 3.48, 2.33, 4.38, 4.09, 2.52, 5.19, 2.39, 3.66, 2.29, 2.88])
KK_MIN = np.array([6.33, 2.68, 3.52, 5.38, 2.60, 3.53, 2.54, 4.75, 3.98, 2.69, 3.34, 3.17])
NAMES = ['C', 'C#', 'D', 'Eb', 'E', 'F', 'F#', 'G', 'Ab', 'A', 'Bb', 'B']
def key_estimate(chroma_mean):
best = []
for mode, prof in (('major', KK_MAJ), ('minor', KK_MIN)):
for k in range(12):
r = np.corrcoef(chroma_mean, np.roll(prof, k))[0, 1]
best.append((float(r), f'{NAMES[k]} {mode}'))
best.sort(reverse=True)
return [{'key': b[1], 'r': round(b[0], 3)} for b in best[:3]]
def analyze(path, music=False):
import librosa
warnings.filterwarnings('ignore', message='n_fft=') # very short sounds: librosa pads, the numbers stay usable
info = {'file': path.replace('\\', '/'), **probe(path)}
y = decode(path, info.get('channels'))
mono = y.mean(axis=1)
n = len(mono)
info['duration'] = round(n / SR, 3)
if n < SR * 0.02:
info['error'] = 'too short'
return info
peak = float(np.max(np.abs(y)))
info['peak_dbfs'] = round(db(peak), 2)
info['rms_dbfs'] = round(db(np.sqrt(np.mean(y ** 2))), 2)
info['clipped_samples'] = int(np.sum(np.abs(y) >= 0.999))
info['dc'] = round(float(np.mean(mono)), 5)
try:
import pyloudnorm as pyln
if n > SR * 0.45:
v = float(pyln.Meter(SR).integrated_loudness(y.astype(np.float64)))
info['lufs'] = round(v, 2) if math.isfinite(v) else None # None: silence
except Exception:
pass
r20 = frame_rms(mono, int(0.02 * SR))
info['noise_floor_dbfs'] = round(db(np.percentile(r20, 10)), 1)
info['loudest_20ms_dbfs'] = round(db(np.max(r20)), 1)
info['dynamic_range_db'] = round(info['loudest_20ms_dbfs'] - info['noise_floor_dbfs'], 1)
# stereo width: side/mid energy
mid, side = (y[:, 0] + y[:, 1]) / 2, (y[:, 0] - y[:, 1]) / 2
info['side_to_mid_db'] = (round(db(np.sqrt(np.mean(side ** 2))) - db(np.sqrt(np.mean(mid ** 2))), 1)
if np.any(side) else None) # None: both channels identical (mono)
# first transient: first 1 ms window whose envelope gets within 20 dB of the peak envelope
r1 = frame_rms(mono, int(0.001 * SR))
thr = np.max(r1) * 0.1
idx = int(np.argmax(r1 >= thr))
info['attack_at'] = round(idx * 0.001, 4)
pk = int(np.argmax(r1))
info['peak_at'] = round(pk * 0.001, 4)
# spectral shape (on a 22.05k copy for speed)
ys = librosa.resample(mono, orig_sr=SR, target_sr=22050) if n < SR * 400 else mono[:: 2]
sr2 = 22050
S = np.abs(librosa.stft(ys, n_fft=2048, hop_length=512))
cen = librosa.feature.spectral_centroid(S=S, sr=sr2)[0]
roll = librosa.feature.spectral_rolloff(S=S, sr=sr2, roll_percent=0.95)[0]
act = r20[: len(cen)] if len(r20) else None
info['centroid_hz'] = int(np.median(cen))
info['rolloff95_hz'] = int(np.percentile(roll, 90))
# high-band levels relative to 1-4 kHz (an encoder lowpass shows up as a cliff, e.g. 16-19k far below 12-16k)
m30 = mono[: min(n, SR * 60)]
Sf = np.abs(np.fft.rfft(m30 * np.hanning(len(m30)))) ** 2
fr = np.fft.rfftfreq(len(m30), 1 / SR)
band = lambda a, b: float(np.mean(Sf[(fr >= a) & (fr < b)]) + 1e-30)
ref = band(1000, 4000)
info['band_db_rel_1_4k'] = {k: round(10 * np.log10(band(a, b) / ref), 1) for k, (a, b) in
{'40-120': (40, 120), '8-12k': (8000, 12000), '12-16k': (12000, 16000),
'16-19k': (16000, 19000), '19-22k': (19000, 22000)}.items()}
onsets = librosa.onset.onset_detect(y=ys, sr=sr2, units='time', backtrack=False)
info['onset_count'] = int(len(onsets))
info['onsets'] = [round(float(o), 3) for o in onsets[:40]]
# spikes: 100 ms windows more than 12 dB above the median level
r100 = frame_rms(mono, int(0.1 * SR))
med = np.median(r100)
sp = np.where(r100 > med * 4)[0]
info['spikes_over_12db'] = [round(i * 0.1, 1) for i in sp[:30]]
info['level_std_db_100ms'] = round(float(np.std(20 * np.log10(r100 + 1e-12))), 2)
if music:
tempo, beats = librosa.beat.beat_track(y=ys, sr=sr2, start_bpm=120, units='time', tightness=200)
tempo = float(np.atleast_1d(tempo)[0])
info['tempo'] = round(tempo, 2)
if len(beats) > 4:
ibi = np.diff(beats)
info['tempo_from_ibi'] = round(60 / float(np.median(ibi)), 2)
info['ibi_std_ms'] = round(float(np.std(ibi)) * 1000, 1)
info['beats'] = [round(float(b), 3) for b in beats]
chroma = librosa.feature.chroma_cqt(y=ys, sr=sr2, hop_length=512)
info['key'] = key_estimate(chroma.mean(axis=1))
r2 = frame_rms(mono, SR * 2)
info['rms_db_per_2s'] = [round(db(v), 1) for v in r2]
# loudness per bar (assuming 4/4 on the tracked beats)
if len(beats) > 8:
bars = beats[::4]
pb = []
for a, b in zip(bars[:-1], bars[1:]):
seg = mono[int(a * SR): int(b * SR)]
pb.append(round(db(np.sqrt(np.mean(seg ** 2) + 1e-20)), 1))
info['bar_starts_if_beat0_is_downbeat'] = [round(float(b), 3) for b in bars]
info['rms_db_per_bar'] = pb
# sections: novelty on stacked chroma + mfcc
try:
mf = librosa.feature.mfcc(y=ys, sr=sr2, n_mfcc=13, hop_length=512)
F = np.vstack([librosa.util.normalize(chroma, axis=1), librosa.util.normalize(mf, axis=1)])
k = max(4, min(12, int(info['duration'] // 15)))
bounds = librosa.segment.agglomerative(F, k)
info['sections_at'] = [round(float(t), 2) for t in librosa.frames_to_time(bounds, sr=sr2, hop_length=512)]
except Exception as e:
info['sections_error'] = str(e)[:200]
return info
def png(path, outdir):
os.makedirs(outdir, exist_ok=True)
base = os.path.splitext(os.path.basename(path))[0]
out = os.path.join(outdir, base + '_spec.png')
p = subprocess.run(['ffmpeg', '-v', 'error', '-y', '-i', path, '-lavfi',
'showspectrumpic=s=1400x420:mode=combined:color=intensity:scale=log:legend=1', out],
capture_output=True, timeout=300)
if p.returncode or not os.path.exists(out):
raise RuntimeError(p.stderr.decode(errors='replace').strip()[:200] or 'no image written')
return out.replace('\\', '/')
def describe(r):
"""one plain-language line: format, level, noise, timing, tone, and what looks wrong"""
if r.get('error'):
return f"{os.path.basename(r['file'])}: ERROR {r['error']}"
fmt = f"{r.get('codec')} {r.get('kbps') or '?'} kbps {(r.get('src_rate') or 0) / 1000:g} kHz {r.get('channels')}ch"
lufs = f"{r['lufs']} LUFS" if r.get('lufs') is not None else 'LUFS n.a.'
s = (f"{os.path.basename(r['file'])}: {fmt}, {r['duration']} s, peak {r['peak_dbfs']} dBFS, {lufs}, noise floor "
f"{r['noise_floor_dbfs']} dBFS, attack {r['attack_at']} s, centroid {r['centroid_hz']} Hz, rolloff95 "
f"{r['rolloff95_hz']} Hz, " + ('mono' if r.get('side_to_mid_db') is None else f"side/mid {r['side_to_mid_db']} dB"))
if r.get('tempo'):
s += f", {r.get('tempo_from_ibi', r['tempo'])} BPM (ibi std {r.get('ibi_std_ms')} ms), key {r['key'][0]['key']}"
warn = []
if r['peak_dbfs'] <= -90:
warn.append('digital silence')
if r.get('clipped_samples'):
warn.append(f"{r['clipped_samples']} clipped samples")
if r.get('kbps') and r.get('codec') == 'mp3' and r['kbps'] < 192:
warn.append('low-bitrate mp3')
if r.get('src_rate') and r['src_rate'] < 44100:
warn.append(f"sample rate {r['src_rate']} Hz")
if r['rolloff95_hz'] < 8000 and r['peak_dbfs'] > -90:
warn.append('dull (little above 8 kHz)')
if r.get('spikes_over_12db'):
warn.append(f"{len(r['spikes_over_12db'])} spike(s) > 12 dB over the median level")
return s + ('; CHECK: ' + ', '.join(warn) if warn else '')
def clean(o):
if isinstance(o, dict):
return {k: clean(v) for k, v in o.items()}
if isinstance(o, list):
return [clean(v) for v in o]
return None if isinstance(o, float) and not math.isfinite(o) else o
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('files', nargs='+')
ap.add_argument('--music', action='store_true', help='also tempo, beats, key, loudness per bar, sections')
ap.add_argument('--png', metavar='DIR', help='write a spectrogram image per file here')
ap.add_argument('--out', metavar='OUT.json', help='write every measurement as a JSON list')
ap.add_argument('--json', action='store_true', help='print the JSON list instead of one line per file')
a = ap.parse_args()
res = []
for f in a.files:
if not os.path.isfile(f):
r = {'file': f, 'error': 'file not found'}
else:
try:
r = analyze(f, a.music)
except Exception as e: # noqa: BLE001
r = {'file': f, 'error': str(e)[:400]}
if a.png:
try:
r['png'] = png(f, a.png)
except Exception as e: # noqa: BLE001
r['png_error'] = str(e)[:200]
r = clean(r)
r['summary'] = describe(r)
res.append(r)
if not a.json:
print(r['summary'].encode('ascii', 'replace').decode())
bad = [os.path.basename(r['file']) for r in res if r.get('error')]
summary = f"{len(res)} file(s) analysed, {len(bad)} failed" + (f" ({', '.join(bad[:6])})" if bad else '')
if a.out:
os.makedirs(os.path.dirname(os.path.abspath(a.out)), exist_ok=True)
with open(a.out, 'w', encoding='utf-8') as fh:
json.dump(res, fh, indent=1)
summary += f'; wrote {a.out}'
print(json.dumps(res, indent=1, ensure_ascii=True) if a.json else summary)
return 1 if bad else 0
if __name__ == '__main__':
sys.exit(main())
scripts/arrange.py
#!/usr/bin/env python
"""Fit a music track to the film's bar structure by editing it at bar lines (no generation).
python -I <skill>/scripts/arrange.py TRACK [--bpm 123.97] [--phase auto|0..3]
[--film-bpm 120] [--beats-per-bar 4] [--bars 30 | --arc arc.json | --arc "-14,-12,...,0,0,-7"]
[--drop-film-bar 21] [--drop auto|none|SRC_BAR] [--start-bar SRC_BAR] [--end-bar SRC_BAR] [--phrase 2]
[--film-offset 0] [--file audio/assets/music/track.mp3] [--out out/audio-review/arrange.json] [--json]
The film's arc is a target loudness per film bar in dB relative to the track's loud bars (0 = the loudest sections,
-14 = a soft intro). Give it as a comma list, or a JSON file (templates/arc.json: {"arc": [...], "film_bpm": 120,
"beats_per_bar": 4, "drop_film_bar": 21}), or let the script build a generic one from --bars and --drop-film-bar (soft
open, groove, breakdown, build, big section from the drop, resolve). Write the arc from the picture map in SOUND_BRIEF.md.
1. Beat grid: librosa beats, a robust linear fit beats ~ t0 + k*T over the whole track (precise BPM), then the phase is
moved onto the measured attacks (a 23 ms-window spectral flux at 2.9 ms resolution, folded at the beat period): the
beat tracker's onset envelope lags real attacks by 20-50 ms, and a splice cut on a lagging grid chops the downbeat's
own attack.
2. Downbeat: which beat phase starts bars (low-band onsets + harmonic change), with a confidence (z-score gap) and a
local check in 8-bar windows (sections where another phase clearly wins are listed). Measured on the two
hand-labelled tracks of the Clawd project (both seen while building this): Born Of The Sky correct (confidence 0.93);
Bring Me The Sky half a bar off (confidence 0.15, flagged LOW: its bass plays on beats 1 and 3 and its accents move
by half a bar later in the track). Under LOW confidence the downbeat is a guess between the two printed grids:
decide from the music (the first note, section entrances, chord changes in the bar table) and pass --phase N.
3. Bar features: chroma, MFCC, loudness, low-band energy, onset density; novelty = change from the previous bar.
4. Arrangement by dynamic programming over the film's bars (film bar k spans bar_len*(k-1)..bar_len*k s):
cost = energy-curve mismatch (the film's arc) + jump costs (a jump must land on a bar that sounds like the one that
would naturally follow: chroma/MFCC/loudness distance) + phrase penalties (jumps prefer phrase starts) +
hard anchors (--drop-film-bar = --drop source bar, the last film bar = --end-bar, film bar 1 = --start-bar). By
default (--drop auto) the drop film bar is anchored on the strongest drop candidate (a bar within 3 dB of the loud
sections that stays loud, scored by its lift over the 4 bars before + novelty); --drop none lets the arc decide.
On the two Clawd tracks the top candidate is a section start the music supervisor marked by hand (Born Of The Sky
bar 50 = the crash into big section 1; Bring Me The Sky the re-entry at 127.8 s); developed on those same tracks.
Prints the summary, the bar table, the chosen map and a ready-to-paste "music" block for the design (source seconds,
film seconds); --json prints the full result as JSON instead, --out writes it to a file.
It is a first draft: check every splice (qa.py edges, earshot timeline) and the bar table, then fine-tune "at" by a few
ms so the measured downbeat transients land on the picture's downbeats.
"""
import argparse, json, os, subprocess, sys
import numpy as np
SR = 22050
def default_arc(n, drop):
"""a generic short-film arc over n film bars with the big section starting at film bar `drop` (1-based):
soft open -> groove -> breakdown -> build -> big section -> resolve"""
arc = []
big_until = drop + max(2, int(round((n - drop) * 0.6)))
for k in range(1, n + 1):
if k < drop:
x = (k - 1) / max(1, drop - 2) # 0..1 up to the bar before the drop
v = -14 + 10 * min(1.0, x / 0.45) # opening rises to the groove (-4)
if 0.55 <= x < 0.8:
v = -9.5 # breakdown before the build
elif x >= 0.8:
v = -9 + 4 * (x - 0.8) / 0.2 # build
elif k < big_until:
v = 0.0 # the big section
else:
v = -13 + 2 * min(3, n - k) # resolve to the end
arc.append(round(v, 1))
return arc
def decode(path):
p = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-i', path, '-f', 'f32le', '-ac', '1', '-ar', str(SR), '-'],
capture_output=True)
if p.returncode != 0 or not p.stdout:
sys.exit(f'cannot decode {path}: {p.stderr.decode(errors="replace")[:300]}')
return np.frombuffer(p.stdout, np.float32).copy()
def fit_grid(beats, T0):
"""robust fit of beat times to t0 + k*T, starting from period T0"""
k = np.r_[0, np.cumsum(np.maximum(1, np.round(np.diff(beats) / T0)))]
A = np.vstack([k, np.ones_like(k)]).T
inl = np.ones(len(beats), bool)
T, t0 = T0, float(beats[0])
for _ in range(5):
T, t0 = np.linalg.lstsq(A[inl], beats[inl], rcond=None)[0]
res = beats - (t0 + k * T)
inl = np.abs(res) < 0.035
if inl.sum() < 8:
inl[:] = True
res = beats - (t0 + k * T)
return float(T), float(t0), float(np.median(np.abs(res))), float(np.mean(np.abs(res) < 0.035))
def flux(y, n_fft=512, hop=64, block=4096):
"""log-compressed spectral flux with a short (23 ms) window at 2.9 ms resolution, computed in blocks of frames
(bounded memory); returns (flux, frame centre times)"""
win = np.hanning(n_fft).astype(np.float32)
nfr = max(0, (len(y) - n_fft) // hop + 1)
out, prev = np.zeros(nfr, np.float32), None
for f0 in range(0, nfr, block):
f = np.arange(f0, min(nfr, f0 + block))
L = np.log1p(100 * np.abs(np.fft.rfft(y[f[:, None] * hop + np.arange(n_fft)] * win, axis=1)))
d = np.diff(L, axis=0, prepend=L[:1] if prev is None else prev[None])
out[f] = np.maximum(0, d).sum(axis=1)
prev = L[-1]
return out, (np.arange(nfr) * hop + n_fft / 2) / SR
def transient_phase(y, T, t0):
"""shift (s) that puts the grid on the measured attacks: the short-window flux folded at the beat period over the
whole track, searched within +-min(100 ms, 0.3 beat); returns (shift, score ratio best / median)"""
fl, tt = flux(y)
hop_s = tt[1] - tt[0]
search = min(0.1, 0.3 * T)
offs = np.arange(-search, search + 1e-9, 0.001)
k = np.arange(int(np.ceil((tt[0] - t0 + search) / T)), int((tt[-1] - t0 - search) / T))
score = np.array([fl[np.round((t0 + o + k * T - tt[0]) / hop_s).astype(int)].mean() for o in offs])
b = int(np.argmax(score))
return float(offs[b]), float(score[b] / (np.median(score) + 1e-9))
def local_phases(score, grid, bpb, phase, win=8, hop=4, min_gap=0.5):
"""stretches where another downbeat phase clearly wins (by >= min_gap, the global LOW threshold) in windows of
`win` bars: [{'from', 'to', 'phase', 'gap'}]"""
out, W = [], win * bpb
for s in range(0, len(grid) - W + 1, hop * bpb):
seg = score[s:s + W]
sc = [float(np.mean(seg[(p - s) % bpb::bpb])) for p in range(bpb)]
o = np.argsort(sc)[::-1]
gap = sc[o[0]] - sc[o[1]]
if o[0] == phase or gap < min_gap:
continue
t0, t1 = round(float(grid[s]), 2), round(float(grid[min(len(grid) - 1, s + W)]), 2)
if out and out[-1]['phase'] == int(o[0]) and t0 <= out[-1]['to']:
out[-1].update({'to': t1, 'gap': max(out[-1]['gap'], round(gap, 2))})
else:
out.append({'from': t0, 'to': t1, 'phase': int(o[0]), 'gap': round(gap, 2)})
return out
def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__.splitlines()[1])
ap.add_argument('track')
ap.add_argument('--bpm', type=float, help='the track\'s tempo if you know it (the phase is still fitted)')
ap.add_argument('--phase', default='auto', help='downbeat phase 0..beats-per-bar-1, or auto')
ap.add_argument('--film-bpm', type=float, default=120.0, help='tempo the picture is cut to (after any stretch)')
ap.add_argument('--beats-per-bar', type=int, default=4)
ap.add_argument('--bars', type=int, default=30, help='film bars (ignored when --arc is given)')
ap.add_argument('--arc', help='target dB per film bar: comma list or JSON file')
ap.add_argument('--drop-film-bar', type=int, help='1-based film bar of the drop / big section')
ap.add_argument('--drop', default='auto', help='source bar for the drop film bar: auto (the top drop candidate), '
'none, or a 0-based bar index')
ap.add_argument('--end-bar', type=int, help='source bar index for the film\'s last bar')
ap.add_argument('--start-bar', type=int, help='source bar index for film bar 1')
ap.add_argument('--phrase', type=int, default=2, help='jumps prefer film bars that start a phrase of this length')
ap.add_argument('--film-offset', type=float, default=0.0, help='film time of film bar 1 (s)')
ap.add_argument('--jump', type=float, default=3.0, help='base cost per jump')
ap.add_argument('--no-snap', action='store_true', help='keep the beat tracker\'s phase (skip the transient fit)')
ap.add_argument('--file', help='the track path as the design should reference it (default: TRACK as given)')
ap.add_argument('--out', help='write the full result as JSON here')
ap.add_argument('--json', action='store_true', help='print the full result as JSON instead of the table')
a = ap.parse_args(argv)
if a.drop not in ('auto', 'none') and not a.drop.lstrip('-').isdigit():
sys.exit(f'--drop must be auto, none or a source bar index, not {a.drop!r}')
if not os.path.isfile(a.track):
sys.exit(f'track not found: {a.track}')
cfg = {}
if a.arc and a.arc.strip().endswith('.json'):
cfg = json.load(open(a.arc, encoding='utf-8'))
arc_list = [float(v) for v in cfg['arc']]
elif a.arc:
arc_list = [float(v) for v in a.arc.split(',') if v.strip()]
else:
arc_list = None
film_bpm = float(cfg.get('film_bpm', a.film_bpm))
bpb = int(cfg.get('beats_per_bar', a.beats_per_bar))
FILM_BARS = len(arc_list) if arc_list else a.bars
drop_k = int(a.drop_film_bar or cfg.get('drop_film_bar') or round(FILM_BARS * 0.7))
ARC = arc_list or default_arc(FILM_BARS, drop_k)
bar_len = bpb * 60.0 / film_bpm
import librosa
y = decode(a.track)
dur = len(y) / SR
hop = 256
oenv = librosa.onset.onset_strength(y=y, sr=SR, hop_length=hop)
tempo, beats = librosa.beat.beat_track(onset_envelope=oenv, sr=SR, hop_length=hop, start_bpm=a.bpm or 120,
tightness=400, units='time')
beats = np.asarray(beats, float)
if len(beats) < 8:
sys.exit(f'only {len(beats)} beats tracked in {a.track}: not a steady-tempo track, or too short')
T0 = 60.0 / a.bpm if a.bpm else float(np.median(np.diff(beats)))
T, t0, med_res, inlier = fit_grid(beats, T0)
if a.bpm: # trust a given tempo but keep the fitted phase
T = 60.0 / a.bpm
shift, snap_ratio = (0.0, None) if a.no_snap else transient_phase(y, T, t0)
t0 += shift
bpm = 60.0 / T
first = t0 - np.floor(t0 / T) * T
grid = np.arange(first, dur, T)
# downbeat phase: low-band onset strength + chroma change at each beat
S = np.abs(librosa.stft(y, n_fft=2048, hop_length=hop))
freqs = librosa.fft_frequencies(sr=SR, n_fft=2048)
low = S[freqs < 150].sum(axis=0)
low_on = np.maximum(0, np.diff(low, prepend=low[0]))
chroma = librosa.feature.chroma_stft(S=S ** 2, sr=SR, hop_length=hop)
fr = lambda t: min(int(t * SR / hop), S.shape[1] - 1)
beat_low = np.array([low_on[max(0, fr(t) - 2): fr(t) + 4].max() for t in grid])
beat_chg = []
for t in grid:
a0, a1, b1 = fr(max(0, t - T)), fr(t), fr(min(dur, t + T))
pa = chroma[:, a0:a1].mean(axis=1) if a1 > a0 else chroma[:, a1]
pb = chroma[:, a1:b1].mean(axis=1) if b1 > a1 else chroma[:, a1]
beat_chg.append(1 - float(np.dot(pa, pb) / (np.linalg.norm(pa) * np.linalg.norm(pb) + 1e-9)))
beat_chg = np.array(beat_chg)
zl = (beat_low - beat_low.mean()) / (beat_low.std() + 1e-9)
zc = (beat_chg - beat_chg.mean()) / (beat_chg.std() + 1e-9)
ph_score = [float(np.mean(zl[p::bpb]) + np.mean(zc[p::bpb])) for p in range(bpb)]
order = np.argsort(ph_score)[::-1]
phase = int(order[0]) if a.phase == 'auto' else int(a.phase)
conf = ph_score[order[0]] - ph_score[order[1]]
alt = int(order[1]) if order[0] == phase else int(order[0])
local = local_phases(zl + zc, grid, bpb, phase)
bars = grid[phase::bpb]
bars = bars[bars + bpb * T <= dur + 1e-3]
nb = len(bars)
if nb < 2:
sys.exit('the track is shorter than two bars')
# bar features
mf = librosa.feature.mfcc(S=librosa.power_to_db(S ** 2), sr=SR, n_mfcc=13)
feats, loud, dens = [], [], []
for b in bars:
s0, s1 = int(b * SR), int((b + bpb * T) * SR)
f0, f1 = fr(b), fr(b + bpb * T)
loud.append(20 * np.log10(np.sqrt(np.mean(y[s0:s1] ** 2)) + 1e-9))
dens.append(float((oenv[f0:f1] > np.median(oenv) * 1.5).mean()))
c = chroma[:, f0:f1].mean(axis=1); c = c / (np.linalg.norm(c) + 1e-9)
m = mf[1:, f0:f1].mean(axis=1); m = m / (np.linalg.norm(m) + 1e-9)
feats.append(np.concatenate([c, m]))
feats, loud = np.array(feats), np.array(loud)
rel = loud - np.percentile(loud, 90)
nov = np.r_[0, np.linalg.norm(np.diff(feats, axis=0), axis=1)]
rise = np.r_[0, np.diff(loud)]
# a drop: a loud bar (within 3 dB of the loud sections) that stays loud for 4 bars, lifted over the 4 bars before it
lift = np.array([rel[i] - np.median(rel[max(0, i - 4):i]) if i else 0.0 for i in range(nb)])
loud_now = (rel > -3) & (np.array([rel[i:i + 4].mean() for i in range(nb)]) > -4)
drop_score = np.where(loud_now if loud_now.any() else rel > np.percentile(rel, 75), lift / 3 + nov * 2, -np.inf)
drop_cands = [int(i) for i in np.argsort(drop_score)[::-1][:6] if np.isfinite(drop_score[i])]
drop_src = drop_cands[0] if a.drop == 'auto' else None if a.drop == 'none' else int(a.drop)
for what, v in (('--drop', drop_src), ('--start-bar', a.start_bar), ('--end-bar', a.end_bar)):
if v is not None and not 0 <= v < nb:
sys.exit(f'{what} {v} is outside the source bars 0..{nb - 1}')
if not 1 <= drop_k <= FILM_BARS:
sys.exit(f'--drop-film-bar {drop_k} is outside the film bars 1..{FILM_BARS}')
# how different bar j sounds from bar i
D = np.linalg.norm(feats[:, None] - feats[None], axis=2) + np.abs(loud[:, None] - loud[None]) / 6
nxt = np.minimum(np.arange(nb) + 1, nb - 1)
J = a.jump + 2.5 * D[nxt] # J[i, j]: jump from source bar i to j
J[np.arange(nb - 1), np.arange(1, nb)] = 0.0 # the natural continuation is free
J[np.arange(nb), np.arange(nb)] += 1.5 # repeating the same bar
cont = np.zeros((nb, nb), bool)
cont[np.arange(nb - 1), np.arange(1, nb)] = True
INF = 1e18
arc = np.array(ARC, float)
cost = np.full((FILM_BARS, nb), INF)
back = np.zeros((FILM_BARS, nb), int)
cost[0] = np.abs(rel - arc[0]) / 3.0 + 0.4 * np.arange(nb) # prefer starting near the top
if a.start_bar is not None:
keep = cost[0, a.start_bar] - 0.4 * a.start_bar
cost[0] = INF
cost[0, a.start_bar] = keep
for k in range(1, FILM_BARS):
Ck = J + np.where(cont, 0.0, 1.0 if k % a.phrase else 0.0) # film bar k+1 not on a phrase start: jumps cost more
tot = cost[k - 1][:, None] + Ck
back[k] = np.argmin(tot, axis=0)
cost[k] = tot[back[k], np.arange(nb)] + np.abs(rel - arc[k]) / 3.0
if k == drop_k - 1 and drop_src is not None:
cost[k, np.arange(nb) != drop_src] = INF
if k == FILM_BARS - 1 and a.end_bar is not None:
cost[k, np.arange(nb) != a.end_bar] = INF
jend = int(np.argmin(cost[-1]))
path = [jend]
for k in range(FILM_BARS - 1, 0, -1):
path.append(int(back[k, path[-1]]))
path = path[::-1]
segs = []
for k, j in enumerate(path):
if segs and j == segs[-1]['last'] + 1:
segs[-1]['last'] = j
segs[-1]['bars'] += 1
else:
segs.append({'first': j, 'last': j, 'film_bar': k + 1, 'bars': 1})
out_segs = []
for s in segs:
src_from = float(bars[s['first']])
out_segs.append({'src_from': round(src_from, 4), 'src_to': round(src_from + bpb * T * s['bars'], 4),
'at': round(a.film_offset + bar_len * (s['film_bar'] - 1), 4),
'src_bars': [s['first'], s['last']], 'film_bars': [s['film_bar'], s['film_bar'] + s['bars'] - 1]})
ratio = film_bpm / bpm
music = {'file': (a.file or a.track).replace('\\', '/'), 'src_bpm': round(bpm, 3), 'target_bpm': film_bpm,
'stretch': 'none' if abs(ratio - 1) < 0.002 else 'rubberband', 'gain_db': 0.0,
'segments': [{'label': f"src bars {s['src_bars'][0]}-{s['src_bars'][1]} -> film bars "
f"{s['film_bars'][0]}-{s['film_bars'][1]}",
'src_from': s['src_from'], 'src_to': s['src_to'], 'at': s['at'],
'fade_in': 0.008, 'fade_out': 0.06} for s in out_segs]}
if music['stretch'] == 'none':
music['src_bpm'] = film_bpm # within 0.2 %: play it unstretched on the film grid
low = conf < 0.5 and a.phase == 'auto'
summary = (f'{bpm:.3f} BPM (grid residual {med_res * 1000:.1f} ms, {inlier:.0%} of beats on it), downbeats at '
f'{bars[0]:.3f} + {bpb * T:.4f}*n s (phase {phase}' + (
f', confidence {conf:.2f}' + (f': LOW, the other candidate is {grid[alt]:.3f} + {bpb * T:.4f}*n s '
f'(--phase {alt})' if low else '') if a.phase == 'auto' else
f' as given; the automatic choice is phase {int(order[0])}, confidence {conf:.2f}')
+ f'); {nb} source bars -> {FILM_BARS} film bars in {len(out_segs)} segment(s), drop film bar {drop_k} = '
+ (f'source bar {drop_src}' + (' (top drop candidate)' if a.drop == 'auto' else '') if drop_src is not None
else 'chosen by the arc') + f'; stretch ratio {ratio:.4f}')
if local:
summary += ('; another downbeat phase wins locally at ' + ', '.join(
f"{x['from']:.1f}-{x['to']:.1f} s (phase {x['phase']})" for x in local[:4]) + ': check the bar lines there')
res = {'summary': summary, 'track': a.track, 'duration': round(dur, 3), 'bpm': round(bpm, 4), 'beat_period': round(T, 6),
'film_bpm': film_bpm, 'beats_per_bar': bpb, 'film_bars': FILM_BARS, 'drop_film_bar': drop_k, 'arc': ARC,
'stretch_ratio_to_film': round(ratio, 6), 't0': round(t0, 4), 'transient_shift_ms': round(shift * 1000, 1),
'transient_fit_ratio': round(snap_ratio, 2) if snap_ratio else None,
'grid_median_residual_ms': round(med_res * 1000, 1), 'grid_inlier_frac': round(inlier, 3),
'downbeat_phase': phase, 'phase_scores': [round(s, 3) for s in ph_score], 'phase_confidence': round(conf, 3),
'phase_low_confidence': low, 'phase_alternative': {'phase': alt, 'first_downbeat': round(float(grid[alt]), 3)},
'local_phase_disagreements': local, 'drop_source_bar': drop_src,
'bars': [{'i': i, 't': round(float(bars[i]), 3), 'rel_db': round(float(rel[i]), 1), 'novelty': round(float(nov[i]), 2),
'rise_db': round(float(rise[i]), 1), 'density': round(dens[i], 2)} for i in range(nb)],
'drop_candidates': drop_cands, 'map': path, 'segments': out_segs, 'music': music,
'total_cost': round(float(cost[-1, jend]), 2)}
if a.out:
os.makedirs(os.path.dirname(os.path.abspath(a.out)), exist_ok=True)
json.dump(res, open(a.out, 'w'), indent=1)
if a.json:
print(json.dumps(res, indent=1))
return res
print(summary)
print(f'transient fit moved the grid {shift * 1000:+.0f} ms (peak/median {snap_ratio or 0:.2f}); '
f'phase scores {[round(s, 2) for s in ph_score]}; drop candidates (source bars) {drop_cands}')
if low:
print(f'LOW downbeat confidence: bars may start at {grid[alt]:.3f} + {bpb * T:.4f}*n s instead. Decide from the '
'music (first note, section entrances, chord changes) and re-run with --phase N.')
print('bar t rel_dB nov rise dens')
for b in res['bars']:
mark = ' <drop?' if b['i'] in drop_cands[:3] else ''
print(f"{b['i']:3d} {b['t']:7.2f} {b['rel_db']:6.1f} {b['novelty']:5.2f} {b['rise_db']:5.1f} {b['density']:5.2f}{mark}")
print('map (film bar -> source bar):', ' '.join(f'{k + 1}:{j}' for k, j in enumerate(path)))
print('design "music" block (check every splice, then fine-tune "at" by a few ms):')
print(json.dumps(music, indent=1))
if a.out:
print(f'wrote {a.out}')
return res
if __name__ == '__main__':
main()
scripts/audit.py
#!/usr/bin/env python
"""Licence + file audit of every recording in the manifests: is each one allowed, verified, credited and intact?
python -I <skill>/scripts/audit.py [--root .] [--manifests audio/assets/manifests] [--report out/qa/mix_report.json]
[--out audio/assets/manifests/audit.json] [--table out/qa/audit.md]
[--fetch] [--fetch-all] [--decode scope|all|none] [--json]
Per manifest entry (references/licences.md has the policy; templates/manifest_entry.json the fields):
licence classified from the manifest's "license": allowed (CC0, public domain, CC BY 2.0-4.0, Mixkit free
licences, Pixabay) / forbidden (NC, ND, SA, Sampling+, personal, custom, editorial, disputed) / unknown
verified the agent's own reading of the licence on the item's page: "licence_seen" (+ "licence_confirmed") merged
from an existing audit.json (the --out file); the script never verifies a licence itself, it only checks
that what the agent saw classifies the same as the manifest
attribution CC BY items need a ready-to-paste "attribution" credit line (and author, title, page_url)
urls Freesound / Mixkit ids agree between the entry id, the page and the download URL
file exists, ffprobe codec / rate / channels / duration, decodes without errors (ffmpeg -f null; in-scope
recordings unless --decode all), duration matches the manifest's analysis; with --fetch the bytes are
compared with download_url (sequential, 1.5 s apart, in memory, never executed; only for recordings
used in the film unless --fetch-all)
in_scope = used in the film (report.files_used) when --report is given, else the existing audit's value, else true.
Severity: blocker (in scope and forbidden / unknown / unverified licence, missing or broken file, CC BY without a
credit line), warning, ok. Exit code 1 when an in-scope recording has a blocker.
Writes audit.json (a list, one record per entry; agent notes kept under "agent_notes") and a Markdown table.
"""
import argparse, hashlib, json, os, re, shutil, subprocess, sys, time
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import licences as L # noqa: E402
TOOL = 'real-sound audit.py v2'
NEEDED = ('id', 'file', 'title', 'author', 'page_url', 'license')
NO_SCRIPTS = ('pixabay.com',) # sites that block scripted downloads: never fetch
UA = 'real-sound-licence-audit/1.0 (polite single-file check; contact: project owner)'
def say(s):
print(str(s).encode('ascii', 'replace').decode())
COMPUTED = ('path', 'exists', 'codec', 'sample_rate', 'channels', 'kbps', 'duration_s', 'video', 'decodes_clean',
'max_volume_db', 'bytes_match_download_url', 'fetch_note')
AUDIT_KEYS = ('id', 'manifest', 'need', 'rank', 'in_scope', 'severity', 'licence_in_manifest', 'licence', 'licence_status',
'licence_seen', 'licence_confirmed', 'attribution_required', 'attribution_ok', 'attribution_in_manifest',
'file_ok', 'file_check', 'earlier_file_check', 'problems', 'warnings', 'agent_notes', 'audit_tool', 'problem',
'fix', 'duplicate_of')
SENT = r'(?:[^.]|\.(?=\d))' # inside one sentence ("4.0" does not end it)
READ_NOTE = re.compile(SENT + r'*\blicen[cs]e' + SENT + r'{0,40}\b(read|verified|checked|seen|confirmed)\b' + SENT + '*',
re.I)
def ffprobe(path):
"""first audio stream's codec / rate / channels / bit rate, the duration, and any real video stream (not cover art)"""
p = subprocess.run(['ffprobe', '-v', 'error', '-show_entries', 'stream=codec_type,codec_name,sample_rate,channels,'
'bit_rate:stream_disposition=attached_pic:format=duration,bit_rate', '-of', 'json', path],
capture_output=True, text=True, timeout=120)
j = json.loads(p.stdout or '{}')
st = j.get('streams') or []
s = next((x for x in st if x.get('codec_type') == 'audio'), {})
f = j.get('format') or {}
brs = [int(b) for b in (s.get('bit_rate'), f.get('bit_rate')) if str(b).isdigit() and 0 < int(b) < 10 ** 7]
video = [x.get('codec_name') for x in st if x.get('codec_type') == 'video'
and not (x.get('disposition') or {}).get('attached_pic')]
return {'codec': s.get('codec_name'), 'sample_rate': int(s.get('sample_rate') or 0), 'channels': int(s.get('channels') or 0),
'kbps': round(brs[0] / 1000) if brs else None, 'duration_s': round(float(f.get('duration') or 0), 3),
**({'video': video[0]} if video else {})}
def decodes_clean(path):
"""(decodes without errors, first error line, max volume dBFS) from one full ffmpeg decode of the audio stream"""
p = subprocess.run(['ffmpeg', '-hide_banner', '-nostdin', '-loglevel', 'level+info', '-i', path, '-map', '0:a:0',
'-af', 'volumedetect', '-f', 'null', '-'], capture_output=True, timeout=600)
lines = p.stderr.decode(errors='replace').splitlines()
err = [ln for ln in lines if '[error]' in ln or '[fatal]' in ln]
vol = [float(m.group(1)) for ln in lines for m in [re.search(r'max_volume: (-?[\d.]+|-inf) dB', ln)] if m
and m.group(1) != '-inf'] or [None]
return p.returncode == 0 and not err, err[0][:200] if err else None, vol[-1]
def url_checks(e):
"""id consistency for the two sites whose URLs encode it"""
out = []
page, dl, eid = e.get('page_url') or '', e.get('download_url') or '', str(e.get('id') or '')
m = re.search(r'freesound\.org/people/([^/]+)/sounds/(\d+)', page)
if m:
sid = m.group(2)
if re.match(r'fs-\d+', eid) and eid[3:] != sid:
out.append(f'entry id {eid} but the Freesound page is sound {sid}')
d = re.search(r'previews/(\d+)/(\d+)_(\d+)-(hq|lq)\.(ogg|mp3)', dl)
if d:
if d.group(2) != sid:
out.append(f'download_url is sound {d.group(2)}, the page is sound {sid}')
if d.group(1) != str(int(sid) // 1000):
out.append(f'preview folder {d.group(1)} should be {int(sid) // 1000}')
if d.group(4) == 'lq':
out.append('low-quality Freesound preview (-lq): use the -hq preview')
if e.get('author') and m.group(1).lower() != str(e['author']).lower():
out.append(f'author "{e["author"]}" differs from the page owner "{m.group(1)}"')
d = re.search(r'assets\.mixkit\.co/active_storage/sfx/(\d+)/(\d+)\.(wav|mp3)', dl)
if d:
if d.group(1) != d.group(2):
out.append('Mixkit download_url ids disagree')
if re.match(r'mk-\d+', eid) and eid[3:] != d.group(1):
out.append(f'entry id {eid} but the Mixkit download is item {d.group(1)}')
return out
def fetch_match(url, path):
"""True / False when the bytes at url equal the local file; None + reason when not checked"""
host = re.sub(r'^[a-z]+://', '', url.lower()).split('/')[0]
if any(host.endswith(h) for h in NO_SCRIPTS):
return None, f'{host} blocks scripted downloads: compare by hand'
import urllib.request
try:
req = urllib.request.Request(url, headers={'User-Agent': UA})
with urllib.request.urlopen(req, timeout=60) as r:
data = r.read(200 * 1024 * 1024 + 1)
except Exception as ex: # noqa: BLE001
return None, f'fetch failed: {str(ex)[:160]}'
local = open(path, 'rb').read()
return hashlib.sha1(data).hexdigest() == hashlib.sha1(local).hexdigest(), None
def agent_fields(old):
"""what the agent wrote into an earlier audit record (licence reading, notes, hand-recorded file checks such as a
byte comparison); script findings are recomputed, earlier evidence is kept under earlier_file_check"""
if not old:
return {}
keep = {k: old[k] for k in ('licence_seen', 'licence_confirmed') if k in old}
fc = old.get('file_check') or {}
earlier = dict(old.get('earlier_file_check') or {})
if old.get('audit_tool'):
notes = old.get('agent_notes') or []
earlier.update({k: fc[k] for k in ('bytes_match_download_url', 'fetch_note') if k in fc})
else: # a hand-written audit: its warnings / problem / fix are the agent's notes, its file checks earlier evidence
notes = list(old.get('warnings') or []) + [old[k] for k in ('problem', 'fix') if old.get(k)]
earlier.update({k: v for k, v in fc.items() if k not in COMPUTED or k in ('bytes_match_download_url', 'fetch_note')})
keep.update({k: v for k, v in old.items() if k not in AUDIT_KEYS})
if earlier:
keep['earlier_file_check'] = earlier
if notes:
keep['agent_notes'] = notes
if 'in_scope' in old:
keep['_in_scope'] = old['in_scope']
return keep
def audit_entry(e, root, used, old, decode='scope'):
path = os.path.join(root, e['file']) if not os.path.isabs(e['file']) else e['file']
f = L.rel(e['file'], root)
lic = L.classify(e.get('license'))
ag = agent_fields(old)
in_scope = (f.lower() in used) if used is not None else bool(ag.get('_in_scope', True))
problems, warnings = [], []
for k in NEEDED:
if not e.get(k):
(problems if k in ('file', 'license') else warnings).append(f'missing "{k}"')
if lic['status'] == 'forbidden':
problems.append(f'licence forbidden: {lic["reason"]}')
elif lic['status'] == 'unknown':
problems.append(f'licence unknown: {lic["reason"]}')
seen = ag.get('licence_seen')
confirmed = bool(ag.get('licence_confirmed')) and bool(seen)
if not seen:
m = READ_NOTE.search(str(e.get('notes') or ''))
problems.append('no licence record in audit.json' + (
f' (the manifest notes say: "{m.group(0).strip()[:90]}"): copy that reading into "licence_seen" and set '
'"licence_confirmed"' if m else ': read the licence on the item page and record "licence_seen"'))
else:
sk = L.classify(seen)['key']
if sk not in ('unknown', 'none', lic['key']):
warnings.append(f'licence_seen reads as {sk} but the manifest says {lic["key"]}')
if not ag.get('licence_confirmed'):
problems.append('licence_seen recorded but licence_confirmed is not true')
attr_ok = True
if lic['attribution']:
attr_ok = bool(e.get('attribution')) and bool(e.get('author')) and bool(e.get('page_url'))
if not attr_ok:
problems.append('CC BY needs "attribution" (a ready-to-paste credit line), "author" and "page_url"')
else:
at = e['attribution']
if str(e['author']).split(' (')[0].lower() not in at.lower():
warnings.append('the attribution line does not name the author')
if not re.search(r'cc[ -]?by|attribution|creativecommons\.org', at, re.I):
warnings.append('the attribution line does not name the licence (credits.py adds the licence link)')
warnings += url_checks(e)
fc = {'path': f, 'exists': os.path.isfile(path)}
if not fc['exists']:
problems.append('file missing')
else:
try:
fc.update(ffprobe(path))
except Exception as ex: # noqa: BLE001
problems.append(f'ffprobe failed: {str(ex)[:120]}')
if fc.get('video'):
warnings.append(f"the file carries a {fc['video']} video stream: extract the audio to its own file and make "
'sure the licence covers the audio')
if decode == 'all' or (decode == 'scope' and in_scope):
ok, err, vol = decodes_clean(path)
fc['decodes_clean'], fc['max_volume_db'] = ok, vol
if not ok:
problems.append(f'decode errors: {err}')
if vol is not None and vol <= -90:
problems.append(f'the recording is digital silence (max {vol} dB)')
elif vol is not None and vol < -60:
warnings.append(f'the recording is near-silent (max {vol} dB): the wrong file?')
an = (e.get('analysis') or {}).get('duration')
if an and fc.get('duration_s') and abs(float(an) - fc['duration_s']) > 0.05:
warnings.append(f'duration {fc["duration_s"]} s differs from the manifest analysis ({an} s): file replaced?')
if fc.get('sample_rate') and fc['sample_rate'] < 44100:
warnings.append(f'sample rate {fc["sample_rate"]} Hz (dull above {fc["sample_rate"] // 2000} kHz)')
if fc.get('codec') == 'mp3' and fc.get('kbps') and fc['kbps'] < 192:
warnings.append(f'MP3 at {fc["kbps"]} kbps: look for a lossless or higher-rate source')
r = {'id': e.get('id'), 'manifest': e['_manifest'], 'need': e.get('need'), 'rank': e.get('rank'),
'in_scope': in_scope, 'severity': None,
'licence_in_manifest': e.get('license'), 'licence': lic['key'], 'licence_status': lic['status'],
'licence_seen': seen, 'licence_confirmed': confirmed, 'attribution_required': lic['attribution'],
'attribution_ok': attr_ok, 'attribution_in_manifest': e.get('attribution'),
'file_ok': fc['exists'] and fc.get('decodes_clean', True) is not False, 'file_check': fc,
**({'earlier_file_check': ag['earlier_file_check']} if ag.get('earlier_file_check') else {}),
'problems': problems, 'warnings': warnings, 'agent_notes': ag.get('agent_notes', []), 'audit_tool': TOOL,
**{k: v for k, v in ag.items() if k not in AUDIT_KEYS and not k.startswith('_')}}
r['severity'] = severity(r)
return r
def severity(r):
sev = 'blocker' if r['problems'] else 'warning' if r['warnings'] else 'ok'
return sev if r['in_scope'] else 'info' if sev != 'ok' else 'ok'
def audit(root='.', manifests='audio/assets/manifests', report=None, existing=None, fetch=False, fetch_all=False,
decode='scope'):
"""audit every manifest entry; returns {'summary', 'counts', 'entries', 'notes'}"""
mdir = manifests if os.path.isabs(manifests) else os.path.join(root, manifests)
if not os.path.isdir(mdir):
raise FileNotFoundError(f'manifest folder not found: {mdir} (pass --manifests)')
entries, notes = L.load_manifests(mdir)
if not entries:
raise FileNotFoundError(f'no manifest entries in {mdir}' + (f' ({"; ".join(notes)})' if notes else ''))
used = None
if report:
rp = report if os.path.isabs(report) else os.path.join(root, report)
used = {L.rel(u, root).lower() for u in json.load(open(rp, encoding='utf-8')).get('files_used', [])}
old = {}
existing = existing and (existing if os.path.isabs(existing) else os.path.join(root, existing))
if existing and os.path.exists(existing):
for r in json.load(open(existing, encoding='utf-8')):
p = L.rel(((r.get('file_check') or {}).get('path') or ''), root).lower()
old[(str(r.get('id')), p)] = old[(str(r.get('id')), '')] = r
rows, first = [], {}
for e in entries:
f = L.rel(e['file'], root).lower()
o = old.get((str(e.get('id')), f)) or old.get((str(e.get('id')), ''))
if f in first: # the same recording in another manifest: audited once, under its first entry
r0, e0 = first[f]
note = f'duplicate of {e0.get("id")} in {os.path.basename(e0["_manifest"])}'
lic = L.classify(e.get('license'))
if lic['status'] != 'allowed':
r0['problems'].append(f'{os.path.basename(e["_manifest"])} ({e.get("id")}) lists this file under a '
f'{lic["status"]} licence "{e.get("license")}" ({lic["reason"]}): one recording has '
'one licence; fix the manifest or reject the file')
elif lic['key'] != r0['licence']:
r0['warnings'].append(f'{os.path.basename(e["_manifest"])} lists this file as {lic["key"]}, not '
f'{r0["licence"]}')
r0['severity'] = severity(r0)
rows.append({**r0, 'id': e.get('id'), 'manifest': e['_manifest'], 'need': e.get('need'), 'rank': e.get('rank'),
'in_scope': False, 'severity': 'info', 'duplicate_of': e0.get('id'), 'problems': [],
'warnings': [note]})
continue
r = audit_entry(e, root, used, o, decode)
first[f] = (r, e)
rows.append(r)
if fetch or fetch_all:
n = 0
for r, e in zip(rows, entries):
if not (fetch_all or r['in_scope']) or not e.get('download_url') or not r['file_check']['exists']:
continue
if n:
time.sleep(1.5)
n += 1
ok, why = fetch_match(e['download_url'], os.path.join(root, e['file']))
r['file_check']['bytes_match_download_url'] = ok
if why:
r['file_check']['fetch_note'] = why
if ok is False:
r['warnings'].append('local bytes differ from download_url (re-encoded or edited file?)')
r['severity'] = severity(r)
if used is not None:
listed = {L.rel(e['file'], root).lower() for e in entries}
for u in sorted(used - listed):
rows.append({'id': None, 'manifest': None, 'in_scope': True, 'severity': 'blocker', 'licence': 'none',
'licence_status': 'unknown', 'file_check': {'path': u}, 'problems': ['used in the film but in no manifest'],
'warnings': [], 'agent_notes': [], 'audit_tool': TOOL})
sc = [r for r in rows if r['in_scope']]
cnt = {k: sum(1 for r in sc if r['severity'] == k) for k in ('ok', 'warning', 'blocker')}
by_lic = {}
for r in sc:
by_lic[r['licence']] = by_lic.get(r['licence'], 0) + 1
scope = 'used in the film' if used is not None else 'in scope'
summary = (f'{len(rows)} manifest entries, {len(sc)} {scope}: {cnt["ok"]} ok, {cnt["warning"]} with warnings, '
f'{cnt["blocker"]} blocked. Licences {scope}: ' + ', '.join(f'{k} {v}' for k, v in sorted(by_lic.items())) + '.')
n_notes = sum(1 for r in sc if r.get('agent_notes'))
if n_notes:
summary += f' {n_notes} of them carry auditor notes (credit forms, restrictions): read agent_notes.'
now = sum(1 for r in sc if (r.get('file_check') or {}).get('bytes_match_download_url'))
before = sum(1 for r in sc if not (r.get('file_check') or {}).get('bytes_match_download_url')
and (r.get('earlier_file_check') or {}).get('bytes_match_download_url'))
if now:
summary += (f' Bytes match download_url: {now} checked in this run' + (f', {before} more recorded by an earlier '
'audit (earlier_file_check)' if before else '') + '.')
elif before:
summary += f' Bytes match download_url for {before} according to an earlier audit (not re-checked: --fetch).'
if cnt['blocker']:
summary += ' Fix every blocker before publishing (see the table).'
return {'summary': summary, 'counts': {**cnt, 'entries': len(rows), 'in_scope': len(sc)}, 'licences_in_scope': by_lic,
'notes': notes, 'entries': rows}
def table(rows):
order = {'blocker': 0, 'warning': 1, 'info': 2, 'ok': 3}
out = ['| id | in film | severity | licence | verified | credit | file | issues |', '|---|---|---|---|---|---|---|---|']
for r in sorted(rows, key=lambda r: (not r['in_scope'], order.get(r['severity'], 9), str(r.get('id')))):
fc = r.get('file_check') or {}
fdesc = (f"{fc.get('codec')} {fc.get('sample_rate', 0) // 1000 if fc.get('sample_rate') else '?'}k "
f"{fc.get('channels')}ch {fc.get('duration_s')}s") if fc.get('exists') else 'MISSING'
issues = '; '.join(r.get('problems', []) + r.get('warnings', []))[:300].replace('|', '/')
out.append(f"| {r.get('id')} | {'yes' if r['in_scope'] else 'no'} | {r['severity']} | {r.get('licence')} | "
f"{'yes' if r.get('licence_confirmed') else 'NO'} | "
f"{'n/a' if not r.get('attribution_required') else 'ok' if r.get('attribution_ok') else 'MISSING'} | "
f"{fdesc} | {issues} |")
return '\n'.join(out) + '\n'
def backup(out):
"""before overwriting an audit: keep the previous one as <name>.prev.json, and a hand-written one (no audit_tool
field) once as <name>.handwritten.json; returns the paths written"""
if not os.path.exists(out):
return []
stem = os.path.splitext(out)[0]
done = [stem + '.prev.json']
shutil.copy(out, done[0])
try:
rows = json.load(open(out, encoding='utf-8'))
hand = isinstance(rows, list) and rows and not any(isinstance(r, dict) and r.get('audit_tool') for r in rows)
except ValueError:
hand = True
if hand and not os.path.exists(stem + '.handwritten.json'):
shutil.copy(out, stem + '.handwritten.json')
done.append(stem + '.handwritten.json')
return done
def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--root', default='.', help='project root: relative paths resolve against it')
ap.add_argument('--manifests', default='audio/assets/manifests')
ap.add_argument('--report', help='mix_report.json: in_scope = the files the film uses')
ap.add_argument('--out', help='audit JSON to write (default <manifests>/audit.json); agent fields are merged from it')
ap.add_argument('--existing', help='read agent fields from this audit instead of --out')
ap.add_argument('--table', help='Markdown table (default: --out with .md)')
ap.add_argument('--fetch', action='store_true', help='compare bytes with download_url for recordings in the film')
ap.add_argument('--fetch-all', action='store_true', help='... for every entry')
ap.add_argument('--decode', choices=('scope', 'all', 'none'), default='scope',
help='full decode check for in-scope recordings (default), all, or none')
ap.add_argument('--json', action='store_true', help='print the full result as JSON')
a = ap.parse_args(argv)
root = os.path.abspath(a.root)
mdir = a.manifests if os.path.isabs(a.manifests) else os.path.join(root, a.manifests)
out = a.out or os.path.join(mdir, 'audit.json')
out = os.path.normpath(out if os.path.isabs(out) else os.path.join(root, out))
try:
res = audit(root, a.manifests, a.report, a.existing or out, a.fetch, a.fetch_all, a.decode)
except FileNotFoundError as e:
say(f'audit.py: {e}')
return 2
os.makedirs(os.path.dirname(out), exist_ok=True)
backups = backup(out)
json.dump(res['entries'], open(out, 'w', encoding='utf-8'), indent=1, ensure_ascii=False)
tab = a.table or os.path.splitext(out)[0] + '.md'
tab = os.path.normpath(tab if os.path.isabs(tab) else os.path.join(root, tab))
open(tab, 'w', encoding='utf-8').write('# Licence and file audit\n\n' + res['summary'] + '\n\n' + table(res['entries']))
if a.json:
print(json.dumps(res, indent=1, ensure_ascii=True))
else:
say(res['summary'])
for n in res['notes']:
say(f' note: {n}')
groups = {}
for r in res['entries']:
if r['in_scope'] and r['severity'] != 'ok':
for m in r['problems'] + r['warnings']:
groups.setdefault((m in r['problems'], m), []).append(str(r.get('id')))
for (blk, m), ids in sorted(groups.items(), key=lambda kv: (not kv[0][0], -len(kv[1]))):
say(f" {'BLOCKER' if blk else 'warning'} x{len(ids)}: {m} [{', '.join(ids[:8])}{' ...' if len(ids) > 8 else ''}]")
say(f'wrote {out} and {tab}' + (f' (previous audit kept as {", ".join(backups)})' if backups else ''))
return 1 if res['counts']['blocker'] else 0
if __name__ == '__main__':
sys.exit(main())
scripts/credits.py
#!/usr/bin/env python
"""Credits for every recording audible in the film, built from the mix report and the licence manifests.
python -I <skill>/scripts/credits.py [--root .] [--report out/qa/mix_report.json] [--manifests audio/assets/manifests]
[--design audio/sounddesign.json] [--audit audio/assets/manifests/audit.json]
[--out CREDITS.md] [--desc out/CREDITS_description.txt] [--title "Film name"]
Reads report.files_used (written by mix.py: every recording that is audible), matches each file to its manifest entry
and writes:
CREDITS.md music + every sound with title, author, link and licence (grouped by licence); a CC BY entry uses
the manifest's "attribution" line (the form the author asked for) when there is one
CREDITS_description.txt plain text to paste into the video description (CC BY credits are REQUIRED there; CC0 /
Mixkit / Pixabay items are listed as a courtesy)
The processing sentence (trimmed, equalised, pitch-shifted ...) and the music edit sentence come from the design when
--design is given; the source names (Freesound.org, Mixkit ...) come from the manifests. With --audit, recordings the
licence audit did not confirm are listed as problems. Both texts say that every work was modified (CC BY 4.0
s.3(a)(1)(B)). Exit code 1 when anything needs fixing before publishing, 2 when the report or manifests are missing.
"""
import argparse, json, os, sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import licences as L # noqa: E402
ORDER = ['by4', 'by3', 'by25', 'by2', 'cc0', 'pd', 'mixkit-sfx', 'mixkit-music', 'pixabay']
SHORT = {'cc0': 'CC0', 'pd': 'public domain', 'mixkit-sfx': 'Mixkit free licence', 'mixkit-music': 'Mixkit free licence',
'pixabay': 'Pixabay licence'}
def say(s):
print(str(s).encode('ascii', 'replace').decode())
def and_join(xs):
xs = list(xs)
return ' and '.join([', '.join(xs[:-1]), xs[-1]]) if len(xs) > 1 else ''.join(xs)
def processing(design_path, root):
"""(sentence for the effects, sentence for the music edit, music stretched?) derived from the design"""
if not design_path:
return ('trimmed, layered, level-adjusted, equalised and panned for the film',
'Edited for the film: cut to fit the picture and levels adjusted.', False)
import mix # read-only use of the design loader
mix.ROOT = root
d = mix.load_design(design_path)
opts = list((d.get('samples') or {}).values()) + list((d.get('slices') or {}).values())
rules, place = d.get('rules') or [], d.get('place') or []
did = ['trimmed']
if d.get('slices'):
did.append('cut into single hits')
did.append('layered' if len(rules) + len(place) > len(opts) * 0.5 or len(rules) > 1 else 'placed')
did.append('level-adjusted')
if any(s.get('eq') for s in opts) or any((b or {}).get('eq') for b in (d.get('buses') or {}).values()):
did.append('equalised')
if any(r.get('pan') not in (None, 0, 0.0) for r in rules) or any(p.get('pan') for p in place):
did.append('panned')
if any(s.get('pitch') or s.get('glide') for s in opts) or any(r.get('pitch') or r.get('pitch_ramp') for r in rules):
did.append('pitch-shifted by resampling')
fx = and_join(did)
mu = d.get('music')
if not mu:
return fx + ' for the film', None, False
segs = mu.get('segments') or []
stretched = mu.get('stretch', 'rubberband') != 'none' and abs(mu.get('target_bpm', 120) / mu['src_bpm'] - 1) > 1e-6
parts = ['cut and rearranged at bar lines to fit the picture' if len(segs) > 1 else 'trimmed to the picture']
if stretched:
parts.append(f"time-stretched from {mu['src_bpm']:g} to {mu.get('target_bpm', 120):g} BPM")
parts.append('levels adjusted')
if any(s.get('tail_fade') for s in segs):
parts.append('a fade at the end')
return fx + ' for the film', 'Edited for the film: ' + ', '.join(parts) + '.', stretched
def credits(root='.', report='out/qa/mix_report.json', manifests='audio/assets/manifests', design=None, audit=None,
title=None):
"""build the CREDITS.md and description texts; returns {'summary', 'markdown', 'description', 'rows', 'problems'}"""
R = lambda p: p if os.path.isabs(p) else os.path.join(root, p)
if not os.path.exists(R(report)):
raise FileNotFoundError(f'mix report not found: {R(report)} (run mix.py first, or pass --report)')
if not os.path.isdir(R(manifests)):
raise FileNotFoundError(f'manifest folder not found: {R(manifests)} (pass --manifests)')
rep = json.load(open(R(report), encoding='utf-8'))
used = [L.rel(f, root) for f in rep.get('files_used', [])]
entries, notes = L.load_manifests(R(manifests))
by_file = {}
for e in entries:
by_file.setdefault(L.rel(e['file'], root).lower(), e)
confirmed = None
if audit and os.path.exists(R(audit)):
confirmed = {L.rel((r.get('file_check') or {}).get('path') or '', root).lower(): r
for r in json.load(open(R(audit), encoding='utf-8'))}
rows, problems = [], []
for f in used:
e = by_file.get(f.lower())
if not e:
problems.append(f'no manifest entry for {f}')
continue
lic = L.classify(e.get('license'))
if lic['status'] != 'allowed':
problems.append(f'{f}: licence "{e.get("license")}" is {lic["status"]} ({lic["reason"]})')
if lic['attribution'] and not (e.get('author') and e.get('title') and e.get('page_url')):
problems.append(f'{f}: CC BY needs author, title and page_url for the credit')
if confirmed is not None:
a = confirmed.get(f.lower())
if not a:
problems.append(f'{f}: the licence audit has no record of this file (run audit.py, record licence_seen)')
elif not a.get('licence_confirmed'):
problems.append(f'{f}: audit.json does not confirm this licence (licence_seen / licence_confirmed missing)')
rows.append({'file': f, 'id': e.get('id'), 'title': e.get('title'), 'author': e.get('author'),
'url': e.get('page_url'), 'lic': lic, 'attribution': e.get('attribution'),
'category': e.get('category'), 'source': L.source_name(e)})
music = [r for r in rows if r['category'] == 'music' or '/music/' in '/' + r['file'].lower()]
sfx = [r for r in rows if r not in music]
rank = lambda k: ORDER.index(k) if k in ORDER else len(ORDER)
sfx.sort(key=lambda r: (rank(r['lic']['key']), (r['author'] or '').lower(), r['title'] or ''))
fx_did, mu_did, stretched = processing(design and R(design), root)
def line(r):
"""the author's requested credit (manifest "attribution") or a built one, completed with the source link and
the licence link when the requested form leaves them out (CC BY asks for both)"""
name, url = r['lic']['name'] or r['lic']['key'], r['lic']['url']
if not (r['lic']['attribution'] and r['attribution']):
return f'"{r["title"]}" by {r["author"]} ({r["url"]}), licensed under {name}' + (f' ({url})' if url else '')
base = r['attribution'].strip().rstrip('.')
if r['url'] and 'http' not in base and 'www.' not in base:
base += f' ({r["url"]})'
if url and 'creativecommons.org' not in base:
base += f' ({url})'
return base
srcs = {}
for r in sfx:
srcs[r['source']] = srcs.get(r['source'], 0) + 1
src_txt = and_join(s for s, _ in sorted(srcs.items(), key=lambda kv: -kv[1]))
md = [f'# Credits{": " + title if title else ""}', '',
'All credits live here and in the video description (CC BY credits are required there).', '']
if music:
md += ['## Music', '']
for r in music:
md.append(f'- {line(r)}.')
if mu_did:
md.append(f' {mu_did}')
md.append('')
if sfx:
md += ['## Sound effects and ambience', '',
f'Real recordings from {src_txt or "the sources below"}, {fx_did}. Nothing in the soundtrack is '
'synthesized.', '']
cur = None
for r in sfx:
if r['lic']['key'] != cur:
cur = r['lic']['key']
md += ['', f'### {r["lic"]["name"] or cur}', '']
md.append(f'- {line(r)}')
tools = 'ffmpeg, NumPy/SciPy, soxr, librosa and pyloudnorm' + (', Rubber Band (music tempo)' if stretched else '')
md += ['', '## Tools', '', f'Edited, mixed and mastered with the real-sound skill (mix.py, master.py) using {tools}.', '']
if problems:
md += ['', '## PROBLEMS (fix before publishing)', ''] + [f'- {p}' for p in problems]
d = []
if music: # CC BY 4.0 s.3(a)(1)(B): say that the work was modified
d.append('Music:')
d += [line(r) for r in music] + ([mu_did] if mu_did else []) + ['']
by = [r for r in sfx if r['lic']['attribution']]
if by:
d.append('Sound effects (CC BY):')
d += [line(r) for r in by]
d += [f'All sound effects and ambiences were modified: {fx_did}.', '']
rest = sorted({r['author'] for r in sfx if not r['lic']['attribution'] and r['author']}, key=str.lower)
if rest:
kinds = sorted({SHORT.get(r['lic']['key'], r['lic']['key']) for r in sfx if not r['lic']['attribution']})
d.append(f'More sounds ({", ".join(kinds)}) by: ' + ', '.join(rest))
if sfx and not by:
d.append(f'All sound effects and ambiences were {fx_did}.')
n_by = sum(1 for r in rows if r['lic']['attribution'])
summary = (f'{len(rows)} recordings credited ({len(music)} music, {len(sfx)} sound effects / ambience; {n_by} CC BY '
f'need the credit in the video description)' + (f'; {len(problems)} PROBLEM(S) to fix before publishing'
if problems else '; no problems'))
if any(r['lic']['attribution'] for r in music):
summary += ('. CC BY music: put the credit in the description; if a Content ID claim appears, dispute it with '
'the licence link (references/licences.md)')
return {'summary': summary, 'markdown': '\n'.join(md) + '\n', 'description': '\n'.join(d) + '\n', 'rows': [
{k: (v['key'] if k == 'lic' else v) for k, v in r.items()} for r in rows], 'problems': problems, 'notes': notes}
def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--root', default='.', help='project root: relative paths resolve against it')
ap.add_argument('--report', default='out/qa/mix_report.json')
ap.add_argument('--manifests', default='audio/assets/manifests')
ap.add_argument('--design', help='the design file (derives the processing and music-edit sentences)')
ap.add_argument('--audit', help='audit.json from audit.py: unconfirmed licences become problems')
ap.add_argument('--out', default='CREDITS.md')
ap.add_argument('--desc', default='out/CREDITS_description.txt')
ap.add_argument('--title', help='film name for the heading')
a = ap.parse_args(argv)
root = os.path.abspath(a.root)
try:
res = credits(root, a.report, a.manifests, a.design, a.audit, a.title)
except FileNotFoundError as e:
say(f'credits.py: {e}')
return 2
for p, text in ((a.out, res['markdown']), (a.desc, res['description'])):
p = p if os.path.isabs(p) else os.path.join(root, p)
os.makedirs(os.path.dirname(p) or '.', exist_ok=True)
open(p, 'w', encoding='utf-8').write(text)
say(res['summary'])
say(f'wrote {a.out} and {a.desc}')
for p in res['problems']:
say(f'PROBLEM: {p}')
return 1 if res['problems'] else 0
if __name__ == '__main__':
sys.exit(main())
scripts/licences.py
"""Licence classification and manifest loading shared by audit.py and credits.py (real-sound skill).
import licences as L
L.classify('CC BY 4.0') # {'key': 'by4', 'status': 'allowed', 'name': 'CC BY 4.0', 'url': ..., 'attribution': True}
L.classify('CC BY-NC 4.0') # status 'forbidden', reason 'NonCommercial: ...'
entries, notes = L.load_manifests('audio/assets/manifests')
status is 'allowed' (CC0, public domain, CC BY 2.0-4.0, Mixkit free licences, Pixabay Content License), 'forbidden'
(NonCommercial, NoDerivatives, ShareAlike, Sampling+, personal-use, custom, editorial or disputed licences) or 'unknown'
(anything else: read the item's own page and fix the manifest). The policy and how to verify a licence on each site:
references/licences.md.
"""
import glob
import json
import os
import re
CC = 'https://creativecommons.org/'
FIXED = {
'cc0': ('CC0 1.0 (public domain dedication)', CC + 'publicdomain/zero/1.0/', False),
'pd': ('Public Domain Mark 1.0', CC + 'publicdomain/mark/1.0/', False),
'mixkit-sfx': ('Mixkit Sound Effects Free License', 'https://mixkit.co/license/#sfxFree', False),
'mixkit-music': ('Mixkit Stock Music Free License', 'https://mixkit.co/license/#musicFree', False),
'pixabay': ('Pixabay Content License', 'https://pixabay.com/service/license-summary/', False),
}
FORBID = [ # (pattern on the lower-cased licence text, reason)
(r'disputed', 'disputed: the sources disagree about the licence'),
(r'\bnc\b|non-?commercial', 'NonCommercial: a launch video is commercial use'),
(r'\bnd\b|no-?deriv', 'NoDerivatives: syncing to picture, trimming and editing are derivatives'),
(r'\bsa\b|share-?alike', 'ShareAlike: the whole film would have to be released under CC BY-SA'),
(r'sampling', 'Sampling+ / Sampling licences are retired and restrict use'),
(r'personal', 'personal use only'),
(r'custom|free with credit|social media', 'a custom licence: not a standard licence the audit can trust'),
(r'editorial', 'editorial use only'),
]
RESERVED = r'all rights reserved|\bcopyrighted\b' # forbidden alone; next to a free licence it is ambiguous
def classify(text):
"""licence text (as written in a manifest or seen on a page) -> {key, status, name, url, attribution, reason}"""
s = (text or '').strip().lower()
if not s:
return {'key': 'none', 'status': 'unknown', 'name': None, 'url': None, 'attribution': False,
'reason': 'no licence recorded'}
for pat, why in FORBID:
if re.search(pat, s.replace('_', ' ')):
return {'key': 'forbidden', 'status': 'forbidden', 'name': text, 'url': None, 'attribution': False, 'reason': why}
if 'restricted' in s:
return {'key': 'restricted', 'status': 'unknown', 'name': text, 'url': None, 'attribution': False,
'reason': 'a restricted licence (e.g. Mixkit Restricted): read its terms for this use'}
key = None
if re.search(r'\bcc0\b|cc-0|creative commons 0|publicdomain/zero|\bzero\b|public domain dedication', s):
key = 'cc0'
elif 'public domain' in s or 'publicdomain/mark' in s:
key = 'pd'
elif 'mixkit' in s:
key = 'mixkit-music' if 'music' in s else 'mixkit-sfx'
elif 'pixabay' in s:
key = 'pixabay'
res = None
if key:
name, url, attr = FIXED[key]
res = {'key': key, 'status': 'allowed', 'name': name, 'url': url, 'attribution': attr, 'reason': None}
m = re.search(r'(?:\bby\b|attribution)[^0-9]{0,24}([1-4](?:\.[05])?)', s)
if m and not res:
ver = {'1': '1.0', '2': '2.0', '3': '3.0', '4': '4.0'}.get(m.group(1), m.group(1))
if ver.startswith('1'):
return {'key': 'by1', 'status': 'unknown', 'name': f'CC BY {ver}', 'url': None, 'attribution': True,
'reason': 'CC BY 1.0 is rare: confirm it on the item page'}
res = {'key': 'by' + ver.replace('.0', '').replace('.', ''), 'status': 'allowed', 'name': f'CC BY {ver}',
'url': f'{CC}licenses/by/{ver}/', 'attribution': True, 'reason': None}
reserved = re.search(RESERVED, s)
if reserved and not res:
return {'key': 'forbidden', 'status': 'forbidden', 'name': text, 'url': None, 'attribution': False,
'reason': 'all rights reserved'}
if reserved and 'all rights reserved' in s: # a free licence and "all rights reserved" contradict each other
return {'key': 'unknown', 'status': 'unknown', 'name': text, 'url': None, 'attribution': res['attribution'],
'reason': f'names {res["name"]} and "all rights reserved": read the item page'}
return res or {'key': 'unknown', 'status': 'unknown', 'name': text, 'url': None, 'attribution': False,
'reason': 'not a licence this skill recognises: read the item page and record the exact licence'}
def rel(path, root):
"""path relative to the project root with forward slashes (case-insensitive prefix match, for Windows)"""
p = (path or '').replace('\\', '/')
r = os.path.abspath(root).replace('\\', '/').rstrip('/') + '/'
return p[len(r):] if p.lower().startswith(r.lower()) else p
def load_manifests(folder, skip=('audit.json',)):
"""every manifest entry (dicts with a "file") from folder/*.json, each with "_manifest"; plus notes on skipped files.
A manifest is a JSON list of entries, or an object holding one under "entries" / "items"."""
entries, notes = [], []
for m in sorted(glob.glob(os.path.join(folder, '*.json'))):
if os.path.basename(m) in skip or os.path.basename(m).startswith('audit.'): # audits and their backups
continue
try:
d = json.load(open(m, encoding='utf-8'))
except Exception as e: # noqa: BLE001
notes.append(f'{os.path.basename(m)}: not valid JSON ({e})')
continue
if isinstance(d, dict):
d = d.get('entries') or d.get('items')
if not isinstance(d, list):
notes.append(f'{os.path.basename(m)}: not a list of entries (a decision record?), skipped')
continue
for e in d:
if isinstance(e, dict) and e.get('file'):
entries.append({**e, '_manifest': m.replace('\\', '/')})
return entries, notes
def source_name(entry):
"""a readable source name from the entry's page_url host (Freesound.org, Mixkit, ...) or its "source" field"""
url = entry.get('page_url') or entry.get('download_url') or ''
host = re.sub(r'^www\.', '', re.sub(r'^[a-z]+://', '', url.lower()).split('/')[0])
known = {'freesound.org': 'Freesound.org', 'mixkit.co': 'Mixkit', 'pixabay.com': 'Pixabay',
'archive.org': 'the Internet Archive', 'commons.wikimedia.org': 'Wikimedia Commons', 'github.com': 'GitHub'}
for k, v in known.items():
if host.endswith(k):
return v
return host or (entry.get('source') or 'unknown source').split(' (')[0]
scripts/master.py
#!/usr/bin/env python3
"""
master.py: cinetic's mix bus and mastering chain, plus loudness measurement.
Chain (finish): 28 Hz high-pass -> mono below 120 Hz -> 2:1 bus compressor above -14 dBFS
(5 ms RMS detector, 15 ms attack, 120 ms release) -> soft saturation + air -> loudness passes to
the target LUFS, each followed by a true-peak lookahead limiter (4x oversampled detection, 4 ms
lookahead, 80 ms release, ceiling 0.77 linear = -2.3 dBFS, which leaves room for AAC overshoot)
-> edge fades, always: a 3 ms raised-cosine fade-in from exact zero at sample 0 (a sound already at
level on sample 0 clicks and cuts in mid-action) and a raised-cosine fade-out of at least 10 ms
(60 ms by default) onto the last sample -> last 480 samples zeroed.
score.py imports sidechain_times(), finish() and measure(). Standalone use:
python3 master.py mix.wav public/audio/soundtrack.wav --lufs -14 --ceiling 0.77
python3 master.py music.wav ducked.wav --sidechain kick.wav --sc-depth 0.55 --lufs -14
python3 master.py --measure public/audio/soundtrack.wav --json out/qa/loudness.json
Exit code 0 when the output meets the gate (|LUFS - target| <= --tol and true peak <= --max-tp),
1 when it does not or on error. The JSON report holds integrated LUFS, true peak (dBTP, 4x
oversampled), LRA, the loudest momentary block and its time, limiter gain reduction and tail level.
"""
import argparse
import json
import os
import sys
import numpy as np
from scipy import signal
from scipy.ndimage import maximum_filter1d, minimum_filter1d
SR = 48000
HEAD_FADE = 0.003 # s, raised-cosine fade-in at sample 0, applied after the limiter on every master
END_FADE_MIN = 0.010 # s, the shortest fade-out onto the last sample, whatever end_fade asks for
# ------------------------------------------------------------------------------------------
# measurement
# ------------------------------------------------------------------------------------------
def _stereo(x):
x = np.asarray(x, dtype=np.float64)
return np.stack([x, x], axis=1) if x.ndim == 1 else x
def _kw(x):
"""BS.1770 K-weighting (48 kHz coefficients): high shelf then RLB high-pass."""
b1, a1 = [1.53512485958697, -2.69169618940638, 1.19839281085285], [1.0, -1.69065929318241, 0.73248077421585]
b2, a2 = [1.0, -2.0, 1.0], [1.0, -1.99004745483398, 0.99007225036621]
return signal.lfilter(b2, a2, signal.lfilter(b1, a1, x, axis=0), axis=0)
def blocks(x, win=0.4, hop=0.1):
"""Loudness (LUFS) of sliding blocks; 0.4 s = momentary, 3 s = short-term. Returns (t_mid, L)."""
y = _kw(_stereo(x))
p = np.cumsum(np.concatenate([np.zeros((1, 2)), y ** 2]), axis=0)
w, h = int(win * SR), int(hop * SR)
if len(y) < w:
return np.array([len(y) / SR / 2]), np.array([-70.0])
starts = np.arange(0, len(y) - w + 1, h)
ms = (p[starts + w] - p[starts]) / w
L = -0.691 + 10 * np.log10(np.maximum(ms.sum(axis=1), 1e-12))
return (starts + w / 2) / SR, L
def integrated(x):
import pyloudnorm as pyln
return float(pyln.Meter(SR).integrated_loudness(_stereo(x)))
def lra(x):
"""EBU loudness range: 10th-95th percentile of gated 3 s short-term loudness."""
_, st = blocks(x, 3.0, 0.1)
st = st[st > -70]
if len(st) < 2:
return 0.0
rel = 10 * np.log10(np.mean(10 ** (st / 10))) - 20
st = st[st > rel]
return float(np.percentile(st, 95) - np.percentile(st, 10)) if len(st) > 1 else 0.0
def true_peak_db(x):
up = signal.resample_poly(_stereo(x), 4, 1, axis=0)
return float(20 * np.log10(np.max(np.abs(up)) + 1e-12))
def measure(x):
x = _stereo(x)
tm, m = blocks(x, 0.4, 0.1)
k = int(np.argmax(m))
tail = x[-int(0.05 * SR):]
return {
'lufs': round(integrated(x), 2),
'true_peak_db': round(true_peak_db(x), 2),
'sample_peak_db': round(float(20 * np.log10(np.max(np.abs(x)) + 1e-12)), 2),
'lra': round(lra(x), 2),
'momentary_max': round(float(m[k]), 2),
'momentary_max_t': round(float(tm[k]), 3),
'tail_50ms_db': round(float(20 * np.log10(np.sqrt(np.mean(tail ** 2)) + 1e-12)), 1),
'last_480_zero': bool(np.all(x[-480:] == 0)),
'seconds': round(len(x) / SR, 4),
}
# ------------------------------------------------------------------------------------------
# dynamics
# ------------------------------------------------------------------------------------------
def sidechain_times(hits, n, depth=0.55, tau=0.14, attack=0.003, length=0.6):
"""Duck gain (n, 1) from (t_seconds, weight) pairs: each hit pulls the music down by
depth*weight with a 3 ms attack and recovers with time constant tau."""
g = np.ones(n)
a = int(attack * SR)
ramp = 0.5 - 0.5 * np.cos(np.linspace(0, np.pi, max(a, 1)))
seg_t = np.arange(int(length * SR)) / SR
shape = np.exp(-seg_t / tau)
shape[:a] *= ramp
for t, w in hits:
i = int(round(t * SR))
if i >= n:
continue
j = min(n, i + len(shape))
s0 = max(0, -i)
g[max(i, 0):j] = np.minimum(g[max(i, 0):j], 1 - depth * w * shape[s0:j - i])
return g[:, None]
def sidechain_key(key, depth=0.55, tau=0.14, hop=48):
"""Duck gain (n, 1) that follows a key signal (a kick stem): instant attack, release tau."""
k = np.abs(_stereo(key)).max(axis=1)
nb = int(np.ceil(len(k) / hop))
v = np.pad(k, (0, nb * hop - len(k))).reshape(nb, hop).max(axis=1)
v = v / (np.percentile(v[v > 0], 99.9) if np.any(v > 0) else 1.0)
rel = np.exp(-hop / (tau * SR))
e = np.empty(nb)
cur = 0.0
for i, x in enumerate(v.tolist()):
cur = x if x > cur else cur * rel
e[i] = cur
g = 1 - depth * np.clip(e, 0, 1)
return np.interp(np.arange(len(k)), np.arange(nb) * hop + hop / 2, g)[:, None]
def mono_below(x, fc=120):
"""Sum the stereo image to mono below fc so the low end is centred and translates to phones."""
x = _stereo(x)
mid = x.mean(axis=1, keepdims=True)
side = signal.sosfilt(signal.butter(2, fc, 'high', fs=SR, output='sos'), (x[:, :1] - x[:, 1:]) / 2, axis=0)
return np.concatenate([mid + side, mid - side], axis=1)
def bus_comp(x, thr=-14.0, ratio=2.0, attack=0.015, release=0.12, rms=0.005, hop=48):
"""Glue compressor: true RMS detector (rms seconds), gain computed at a 1 ms control rate.
Returns (y, gain_reduction_db per sample)."""
x = _stereo(x)
det = signal.sosfilt(signal.butter(1, 1 / (2 * np.pi * rms), 'low', fs=SR, output='sos'), np.mean(x ** 2, axis=1))
nb = int(np.ceil(len(x) / hop))
d = np.pad(det, (0, nb * hop - len(det)), mode='edge').reshape(nb, hop).max(axis=1)
over = np.maximum(0, 10 * np.log10(np.maximum(d, 1e-12)) - thr)
target = -over * (1 - 1 / ratio)
att, rel = np.exp(-hop / (attack * SR)), np.exp(-hop / (release * SR))
g = np.empty(nb)
cur = 0.0
for i, v in enumerate(target.tolist()):
cur = v + (cur - v) * (att if v < cur else rel)
g[i] = cur
gr = np.interp(np.arange(len(x)), np.arange(nb) * hop + hop / 2, g)
return x * (10 ** (gr / 20))[:, None], gr
def limiter(x, ceiling=0.77, look=0.004, release=0.08, hop=16):
"""True-peak lookahead limiter. Detection on a 4x oversampled copy catches inter-sample
overs; the gain is computed per 16-sample block (min over neighbouring blocks, so the
interpolated gain never exceeds what any sample needs), attacks over the lookahead and
releases with time constant `release`. Returns (y, gain per sample)."""
x = _stereo(x)
n = len(x)
up = np.abs(signal.resample_poly(x, 4, 1, axis=0)).max(axis=1)
a = np.pad(up, (0, max(0, 4 * n - len(up))))[:4 * n].reshape(n, 4).max(axis=1)
la = max(hop, int(look * SR))
peak = maximum_filter1d(a, size=la, origin=-(la // 2), mode='nearest') # max over [i, i+la)
need = np.minimum(1.0, ceiling / np.maximum(peak, 1e-9))
nb = int(np.ceil(n / hop))
gb = np.pad(need, (0, nb * hop - n), mode='edge').reshape(nb, hop).min(axis=1)
k = max(3, la // hop)
win = np.hanning(k + 2)[1:-1]
win /= win.sum()
gb = np.minimum(gb, np.convolve(np.pad(gb, (len(win) // 2, len(win) - 1 - len(win) // 2), mode='edge'), win, mode='valid'))
rel = np.exp(-hop / (release * SR))
out = np.empty(nb)
cur = 1.0
for i, v in enumerate(gb.tolist()):
cur = v if v < cur else v + (cur - v) * rel
out[i] = cur
out = minimum_filter1d(out, 3, mode='nearest')
g = np.minimum(np.interp(np.arange(n), np.arange(nb) * hop + hop / 2, out), need)
return x * g[:, None], g
def loudness_passes(x, target=-14.0, ceiling=0.77, passes=3, tol=0.05, max_passes=8):
"""Normalize to target LUFS then limit; repeat (limiting lowers loudness) until within tol."""
g = np.ones(len(x))
for p in range(max_passes):
x = x * 10 ** ((target - integrated(x)) / 20)
x, g = limiter(x, ceiling)
if p + 1 >= passes and abs(integrated(x) - target) <= tol:
break
return x, g
def glue(x, drive=1.15, air=0.25):
"""Soft saturation and a gentle high shelf (air)."""
x = np.tanh(x * 0.9 * drive) / np.tanh(drive)
if air:
x = x + signal.sosfilt(signal.butter(1, 2500, 'high', fs=SR, output='sos'), x, axis=0) * air
return x
def edge_fades(x, head=HEAD_FADE, tail=END_FADE_MIN):
"""Raised-cosine fade-in over `head` s from exact zero at sample 0, and fade-out over `tail` s
to zero on the last sample. Run after the limiter, so nothing raises the edges again."""
x = _stereo(x).copy()
n = len(x)
for d, rising in ((head, True), (tail, False)):
k = min(n, int(round(d * SR)))
if k > 1:
w = 0.5 - 0.5 * np.cos(np.pi * np.arange(k) / (k - 1)) # 0 -> 1, raised cosine
if rising:
x[:k] *= w[:, None]
else:
x[-k:] *= w[::-1, None]
return x
def finish(mix, lufs=-14.0, ceiling=0.77, mono_hz=120, comp=True, drive=1.15, air=0.25,
hp_hz=28, fade=0.0, end_fade=0.06, tail_zero=480):
"""Full chain for a summed mix. Returns (master, info) where info has the limiter trace."""
x = _stereo(mix)
if hp_hz:
x = signal.sosfilt(signal.butter(2, hp_hz, 'high', fs=SR, output='sos'), x, axis=0)
if mono_hz:
x = mono_below(x, mono_hz)
gr = np.zeros(len(x))
if comp:
x, gr = bus_comp(x)
if drive:
x = glue(x, drive, air)
if fade and fade > 0:
t = np.arange(len(x)) / SR
x = x * (np.clip((len(x) / SR - 0.03 - t) / fade, 0, 1) ** 2)[:, None]
x, g = loudness_passes(x, lufs, ceiling)
x = edge_fades(x, HEAD_FADE, max(end_fade or 0.0, END_FADE_MIN))
if tail_zero:
x[-tail_zero:] = 0
lim_db = -20 * np.log10(np.maximum(g, 1e-9))
return x, {'limiter_gr_db': lim_db, 'comp_gr_db': -gr}
def gate(report, target, tol, max_tp):
fails = []
if abs(report['lufs'] - target) > tol:
fails.append(f"integrated {report['lufs']} LUFS not within {target}+-{tol}")
if report['true_peak_db'] > max_tp:
fails.append(f"true peak {report['true_peak_db']} dBTP above {max_tp}")
if not report['last_480_zero']:
fails.append('last 480 samples are not digital zero')
return fails
# ------------------------------------------------------------------------------------------
# CLI
# ------------------------------------------------------------------------------------------
def _read(path):
import soundfile as sf
x, sr = sf.read(path, always_2d=True, dtype='float64')
if x.shape[1] == 1:
x = np.repeat(x, 2, axis=1)
x = x[:, :2]
if sr != SR:
from math import gcd
g = gcd(SR, sr)
x = signal.resample_poly(x, SR // g, sr // g, axis=0)
return x
def main(argv=None):
ap = argparse.ArgumentParser(description='Master a WAV to target loudness, or measure one (--measure).')
ap.add_argument('input', help='input WAV (any rate; resampled to 48 kHz)')
ap.add_argument('output', nargs='?', help='output WAV (48 kHz, 24-bit)')
ap.add_argument('--measure', action='store_true', help='only measure the input')
ap.add_argument('--lufs', type=float, default=-14.0, help='target integrated loudness (default -14)')
ap.add_argument('--ceiling', type=float, default=0.77, help='limiter ceiling, linear (default 0.77)')
ap.add_argument('--tol', type=float, default=0.5, help='gate tolerance in LU (default 0.5)')
ap.add_argument('--max-tp', type=float, default=-1.5, help='gate: max true peak dBTP (default -1.5)')
ap.add_argument('--sidechain', metavar='KEY.wav', help='duck the input from this key signal first')
ap.add_argument('--sc-depth', type=float, default=0.55)
ap.add_argument('--sc-tau', type=float, default=0.14)
ap.add_argument('--mono-hz', type=float, default=120, help='mono below this frequency; 0 disables')
ap.add_argument('--no-comp', action='store_true', help='skip the bus compressor')
ap.add_argument('--no-glue', action='store_true', help='skip saturation and air')
ap.add_argument('--fade', type=float, default=0.0, help='quadratic fade-out over the last N seconds')
ap.add_argument('--tail-zero', type=int, default=480, help='samples zeroed at the end (default 480)')
ap.add_argument('--json', metavar='PATH', help="write the report JSON here ('-' for stdout)")
a = ap.parse_args(argv)
try:
x = _read(a.input)
if a.measure:
rep = measure(x)
fails = gate(rep, a.lufs, a.tol, a.max_tp)
else:
if not a.output:
ap.error('output path required unless --measure')
if a.sidechain:
key = _read(a.sidechain)
key = np.pad(key, ((0, max(0, len(x) - len(key))), (0, 0)))[:len(x)]
x = x * sidechain_key(key, a.sc_depth, a.sc_tau)
y, info = finish(x, a.lufs, a.ceiling, a.mono_hz, not a.no_comp,
0 if a.no_glue else 1.15, 0 if a.no_glue else 0.25,
fade=a.fade, tail_zero=a.tail_zero)
import soundfile as sf
os.makedirs(os.path.dirname(os.path.abspath(a.output)), exist_ok=True)
sf.write(a.output, y.astype(np.float32), SR, subtype='PCM_24')
rep = measure(y)
rep['limiter_max_gr_db'] = round(float(info['limiter_gr_db'].max()), 2)
rep['output'] = a.output
fails = gate(rep, a.lufs, a.tol, a.max_tp)
rep['pass'] = not fails
rep['fails'] = fails
except Exception as e: # noqa: BLE001
print('error:', e, file=sys.stderr)
return 1
if a.json == '-':
print(json.dumps(rep, indent=1))
elif a.json:
os.makedirs(os.path.dirname(os.path.abspath(a.json)), exist_ok=True)
json.dump(rep, open(a.json, 'w'), indent=1)
print(f"{'PASS' if not fails else 'FAIL'} {rep['lufs']} LUFS TP {rep['true_peak_db']} dBTP "
f"LRA {rep['lra']} LU loudest {rep['momentary_max']} LUFS-M at {rep['momentary_max_t']}s"
+ ''.join(f'\n - {f}' for f in fails))
return 0 if not fails else 1
if __name__ == '__main__':
sys.exit(main())
scripts/mix.py
#!/usr/bin/env python
"""real-sound mixer: real recordings placed on the picture's cues. Nothing is synthesized here.
python -I <skill>/scripts/mix.py [--root .] [--design audio/sounddesign.json] [--cues out/cues.json]
[--out out/audio/soundtrack.wav] [--stems out/audio/stems] [--no-master]
[--only keyboard] [--dump out/audio-review/dump] [--trace out/qa/adds.json]
Run it from the project root (or pass --root). Every relative path (design, cues, outputs, the "file" of every sample)
is relative to the root. Full reference with examples: references/design-format.md.
The cue file (JSON, from the picture's timeline): {fps, total (frames), events: [{f, kind, id?, weight?, pan?, ...}]}
(or a list whose first element is that object). A design without rules needs no cue file.
The design file (JSON) describes, in film seconds:
duration film length in seconds (default: cue total / fps)
cues optional default cue file path (the --cues flag wins)
include [other design fragments to merge: samples / slices / pools / rules / place / ambience / music]
samples name -> {file, start?, end?, gain_db?, norm_peak_db?, mono?, fade_in?, fade_out?, eq?, pitch?,
glide?: [from, to] semitones (tape-style varispeed across the sample), env?: [[t, dB], ...]}
a trimmed piece of a downloaded recording (file paths are relative to the project root)
slices name -> {file, start?, end?, pre?, len?, min_gap?, max?, pick?, ...sample opts}
cut a recording of many hits (typing, clicks) into single hits at its detected onsets: name#0, name#1...
pools name -> [sample names; "slicegroup#*" expands to all of its slices]
rules [{match: {kind?, id?, from?, to?}, pool, mode: cycle|hash|fixed, gain_db?, weight_db?, jitter_db?,
pitch?: [lo, hi] semitones, pitch_ramp?: [a, b], gain_ramp?: [a, b] (across the matched events),
pan?: "cue"|number, pan_scale?, align: attack|peak|end|start, offset?, bus?}]
every cue event matching a rule gets a sound from the rule's pool (several rules may layer one event)
place [{sample, at, gain_db?, pan?, align?, offset?, pitch?, bus?}] sounds with no cue event
ambience [{name?, file, src_start?, src_end?, at, until, fade_in?, fade_out?, gain_db?, eq?, width?, auto?: [[t, db], ...]}]
(src_start..src_end is the clean window; it is looped with a 1.5 s crossfade if the bed is longer)
music {file, src_bpm, target_bpm?, stretch?: "rubberband"|"none", gain_db?, eq?,
segments: [{src_from, src_to, at, fade_in?, fade_out?, gain_db?}]} src_* in ORIGINAL source seconds
buses {sfx|amb|music: {gain_db?, eq?, auto?: [[t, db], ...]}}
master {lufs?, ceiling?, no_glue?, no_comp?, mono_hz?}
Processing is limited to trimming, fades, gain, pan, EQ, resampling (pitch variation), tempo stretch of the music,
summing and mastering (master.py next to this script). Writes the soundtrack, stems (sfx.wav, music.wav, amb.wav),
out/qa/mix_report.json (every placement, unsounded cue events, bus levels, files_used for the credits) and
out/qa/cues_sounded.json (the cue events that received a sound, for an A/V sync audit).
--only NAME solo render: keep only the include fragments whose path contains NAME (no master; the unmastered mix is
written to --out, default out/audio-review/solo/<NAME>.wav). Variation is keyed on rule names, so a solo
render plays exactly the same sounds as the full mix.
--no-master write the unmastered mix to --out (with master: --out is mastered, the unmastered sum is premaster.wav
next to it).
--dump DIR write every sample / slice of the design as its own WAV + _samples.json (check trims), then exit.
--trace FILE also write every Mix.add() (bus, start sample, length, gain, pan, sample, rule) for QA attribution.
"""
import argparse, hashlib, json, math, os, re, subprocess, sys, zlib
import numpy as np
import soundfile as sf
from scipy import signal
SR = 48000
ROOT = os.getcwd() # the project root (set by --root)
CACHE = os.path.join(ROOT, 'out', 'audio', 'cache')
MASTER = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'master.py')
def P(p):
return p if os.path.isabs(p) else os.path.join(ROOT, p)
def db2a(d):
return 10.0 ** (d / 20.0)
# ------------------------------------------------------------------ decoding (cached)
_dec = {}
def decode(path):
path = P(path)
if path in _dec:
return _dec[path]
os.makedirs(CACHE, exist_ok=True)
st = os.stat(path)
key = hashlib.sha1(f'{path}|{st.st_size}|{st.st_mtime}'.encode()).hexdigest()[:16]
npy = os.path.join(CACHE, key + '.npy')
if os.path.exists(npy):
x = np.load(npy)
else:
p = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-i', path, '-f', 'f32le', '-acodec', 'pcm_f32le',
'-ac', '2', '-ar', str(SR), '-'], capture_output=True)
if p.returncode != 0:
raise RuntimeError(f'decode failed {path}: {p.stderr.decode(errors="replace")[:300]}')
x = np.frombuffer(p.stdout, np.float32).reshape(-1, 2).copy()
tmp = f'{npy}.{os.getpid()}.tmp.npy'
np.save(tmp, x)
os.replace(tmp, npy) # atomic: other mixers may read the cache at the same time
_dec[path] = x
return x
def stretched(path, ratio):
"""Whole-file tempo change with Rubber Band (ffmpeg filter). ratio = target_bpm / src_bpm (speed factor)."""
if abs(ratio - 1) < 1e-6:
return decode(path)
src = P(path)
st = os.stat(src)
key = hashlib.sha1(f'{src}|{st.st_size}|{st.st_mtime}|{ratio:.6f}|rb2'.encode()).hexdigest()[:16]
out = os.path.join(CACHE, f'stretch_{key}.wav')
if not os.path.exists(out):
os.makedirs(CACHE, exist_ok=True)
af = (f'rubberband=tempo={ratio:.6f}:transients=mixed:detector=compound:phase=laminar:window=standard:'
f'smoothing=off:formant=shifted:pitchq=quality:channels=together')
tmp = out.replace('.wav', f'.{os.getpid()}.tmp.wav')
r = subprocess.run(['ffmpeg', '-v', 'error', '-nostdin', '-y', '-i', src, '-af', af, '-ar', str(SR), '-ac', '2',
'-c:a', 'pcm_f32le', tmp], capture_output=True)
if r.returncode != 0:
raise RuntimeError('rubberband failed: ' + r.stderr.decode(errors='replace')[:400])
os.replace(tmp, out)
return decode(out)
# ------------------------------------------------------------------ DSP helpers (processing only)
def rc_fade(n, rising=True):
if n <= 0:
return np.ones(0)
w = 0.5 - 0.5 * np.cos(np.linspace(0, np.pi, n))
return w if rising else w[::-1]
def apply_fades(x, fin, fout):
x = x.copy()
a, b = int(fin * SR), int(fout * SR)
a, b = min(a, len(x)), min(b, len(x))
if a:
x[:a] *= rc_fade(a)[:, None]
if b:
x[len(x) - b:] *= rc_fade(b, False)[:, None]
return x
def biquad(kind, f0, q=0.707, gain_db=0.0):
A = 10 ** (gain_db / 40)
w0 = 2 * np.pi * f0 / SR
al = np.sin(w0) / (2 * q)
c = np.cos(w0)
if kind == 'peak':
b = [1 + al * A, -2 * c, 1 - al * A]; a = [1 + al / A, -2 * c, 1 - al / A]
elif kind == 'lowshelf':
s = 2 * np.sqrt(A) * al
b = [A * ((A + 1) - (A - 1) * c + s), 2 * A * ((A - 1) - (A + 1) * c), A * ((A + 1) - (A - 1) * c - s)]
a = [(A + 1) + (A - 1) * c + s, -2 * ((A - 1) + (A + 1) * c), (A + 1) + (A - 1) * c - s]
elif kind == 'highshelf':
s = 2 * np.sqrt(A) * al
b = [A * ((A + 1) + (A - 1) * c + s), -2 * A * ((A - 1) + (A + 1) * c), A * ((A + 1) + (A - 1) * c - s)]
a = [(A + 1) - (A - 1) * c + s, 2 * ((A - 1) - (A + 1) * c), (A + 1) - (A - 1) * c - s]
else:
raise ValueError(kind)
return np.array(b) / a[0], np.array(a) / a[0]
def eq(x, spec):
if not spec:
return x
y = x.astype(np.float64)
if spec.get('hp'):
sos = signal.butter(spec.get('hp_order', 2), spec['hp'], 'highpass', fs=SR, output='sos')
y = signal.sosfilt(sos, y, axis=0)
if spec.get('lp'):
sos = signal.butter(spec.get('lp_order', 2), spec['lp'], 'lowpass', fs=SR, output='sos')
y = signal.sosfilt(sos, y, axis=0)
for f0, q, g in spec.get('peq', []):
b, a = biquad('peak', f0, q, g); y = signal.lfilter(b, a, y, axis=0)
if spec.get('lowshelf'):
f0, g = spec['lowshelf']; b, a = biquad('lowshelf', f0, 0.707, g); y = signal.lfilter(b, a, y, axis=0)
if spec.get('highshelf'):
f0, g = spec['highshelf']; b, a = biquad('highshelf', f0, 0.707, g); y = signal.lfilter(b, a, y, axis=0)
return y.astype(np.float32)
def pan_gains(p):
p = max(-1.0, min(1.0, float(p)))
th = (p + 1) * np.pi / 4
return np.cos(th) * np.sqrt(2), np.sin(th) * np.sqrt(2)
def resample_pitch(x, semis):
if abs(semis) < 1e-4:
return x
import soxr
r = 2 ** (semis / 12.0)
return soxr.resample(x, SR * r, SR, quality='VHQ').astype(np.float32)
def varispeed(x, s0, s1):
"""tape-style pitch glide from s0 to s1 semitones across the sample (resampling a recording; no synthesis)"""
import soxr
r0, r1 = 2 ** (s0 / 12.0), 2 ** (s1 / 12.0)
up = 4
xo = soxr.resample(x.astype(np.float64), SR, SR * up, quality='VHQ')
rmax = max(r0, r1, 1.0)
if rmax > 1.0: # band-limit before reading faster, so the glide never aliases
sos = signal.butter(8, 0.95 * (SR / 2) / rmax, 'lowpass', fs=SR * up, output='sos')
xo = signal.sosfiltfilt(sos, xo, axis=0)
L = max(2, int(len(x) / ((r0 + r1) / 2)))
r = np.linspace(r0, r1, L)
pos = np.concatenate([[0.0], np.cumsum(r[:-1])]) * up
pos = np.clip(pos, 0, len(xo) - 1.001)
i = pos.astype(np.int64)
f = (pos - i)[:, None]
y = xo[i] * (1 - f) + xo[np.minimum(i + 1, len(xo) - 1)] * f
return y.astype(np.float32)
def env_index(x, mode):
m = np.abs(x).max(axis=1)
if mode == 'start':
return 0
if mode == 'end':
nz = np.where(m > m.max() * 0.003)[0]
return int(nz[-1]) if len(nz) else len(x) - 1
w = int(0.001 * SR) if mode == 'attack' else int(0.005 * SR)
n = len(m) // w
if n == 0:
return 0
e = m[: n * w].reshape(n, w).max(axis=1)
if mode == 'peak':
return int(np.argmax(e) * w + w // 2)
return int(np.argmax(e >= e.max() * 0.1) * w) # attack: first 1 ms within 20 dB of the peak
def auto_curve(points, n, default=0.0):
"""piecewise-linear dB automation over film time -> linear gain per sample"""
if not points:
return None
pts = sorted(points)
t = np.arange(n) / SR
d = np.interp(t, [p[0] for p in pts], [p[1] for p in pts])
return db2a(d).astype(np.float32)
# ------------------------------------------------------------------ samples
class Lib:
def __init__(self, design):
self.d = design
self.samples = {}
self.meta = {}
for name, s in (design.get('samples') or {}).items():
self.samples[name] = self._make(s)
self.meta[name] = {'file': s['file'], 'start': s.get('start', 0), 'end': s.get('end'), 'pitch': s.get('pitch', 0)}
for name, s in (design.get('slices') or {}).items():
for i, (x, t0) in enumerate(self._slice(s)):
self.samples[f'{name}#{i}'] = x
self.meta[f'{name}#{i}'] = {'file': s['file'], 'start': round(t0, 4), 'pitch': s.get('pitch', 0)}
self.pools = {}
for name, items in (design.get('pools') or {}).items():
out = []
for it in items:
if it.endswith('#*'):
g = it[:-2]
out += sorted([k for k in self.samples if k.startswith(g + '#')], key=lambda k: int(k.split('#')[1]))
else:
if it not in self.samples:
raise KeyError(f'pool {name}: unknown sample {it}')
out.append(it)
if not out:
raise ValueError(f'pool {name} is empty')
self.pools[name] = out
self._pitched = {}
def _finish(self, x, s):
if s.get('mono'):
m = x.mean(axis=1, keepdims=True)
x = np.repeat(m, 2, axis=1)
if s.get('reverse'):
raise ValueError('reverse is not used')
x = eq(x, s.get('eq'))
if s.get('pitch'):
x = resample_pitch(x, s['pitch'])
if s.get('glide'): # [from, to] semitones across the sample
x = varispeed(x, *s['glide'])
if s.get('env'): # [[t_seconds_from_sample_start, dB], ...] gain envelope inside the sample
pts = sorted(s['env'])
t = np.arange(len(x)) / SR
x = x * db2a(np.interp(t, [p[0] for p in pts], [p[1] for p in pts]))[:, None].astype(np.float32)
if s.get('norm_peak_db') is not None:
pk = float(np.abs(x).max()) or 1.0
x = x * (db2a(s['norm_peak_db']) / pk)
x = apply_fades(x, s.get('fade_in', 0.002), s.get('fade_out', 0.012))
return (x * db2a(s.get('gain_db', 0.0))).astype(np.float32)
def _make(self, s):
x = decode(s['file'])
a = int(s.get('start', 0) * SR)
b = int(s['end'] * SR) if s.get('end') is not None else len(x)
return self._finish(x[a:b], s)
def _slice(self, s):
import librosa
x = decode(s['file'])
a = int(s.get('start', 0) * SR)
b = int(s['end'] * SR) if s.get('end') is not None else len(x)
seg = x[a:b]
mono = seg.mean(axis=1)
hop = 128
on = librosa.onset.onset_detect(y=mono, sr=SR, hop_length=hop, backtrack=True, units='samples',
delta=s.get('delta', 0.07), wait=int(s.get('min_gap', 0.06) * SR / hop))
on = [int(o) for o in on]
# loudness gate: drop onsets far below the loudest (handling noise, the far keys)
pk = []
for o in on:
w = mono[o: o + int(0.03 * SR)]
pk.append(float(np.abs(w).max()) if len(w) else 0)
if pk:
top = max(pk)
on = [o for o, p in zip(on, pk) if p >= top * db2a(s.get('gate_db', -18))]
pre, ln = int(s.get('pre', 0.004) * SR), int(s.get('len', 0.16) * SR)
out = []
for i, o in enumerate(on):
st = max(0, o - pre)
en = min(len(seg), o + ln)
if s.get('stop_before_next', True) and i + 1 < len(on):
en = min(en, on[i + 1] - pre)
if en - st < int(0.02 * SR):
continue
out.append((self._finish(seg[st:en], {**s, 'fade_out': s.get('fade_out', 0.02)}), (a + st) / SR))
if s.get('pick') is not None:
out = [out[i] for i in s['pick'] if i < len(out)]
if s.get('max'):
out = out[: s['max']]
if not out:
raise ValueError(f'slices {s["file"]}: no onsets found')
return out
def get(self, name, semis=0.0):
if abs(semis) < 1e-3:
return self.samples[name]
k = (name, round(semis, 2))
if k not in self._pitched:
self._pitched[k] = resample_pitch(self.samples[name], k[1])
return self._pitched[k]
# ------------------------------------------------------------------ the mix
def h01(s):
return (zlib.crc32(s.encode()) & 0xffffffff) / 0xffffffff
class Mix:
def __init__(self, dur):
self.n = int(round(dur * SR))
self.bus = {k: np.zeros((self.n, 2), np.float32) for k in ('sfx', 'amb', 'music')}
self.log = []
def add(self, bus, x, start, gain=1.0, pan=0.0, **meta):
self.log.append({'bus': bus, 'start': int(start), 'n': int(len(x)), 'gain': float(gain), 'pan': float(pan), **meta})
if start >= self.n or start + len(x) <= 0:
return
a, b = max(0, start), min(self.n, start + len(x))
seg = x[a - start: b - start] * gain
if pan:
gl, gr = pan_gains(pan)
seg = seg * np.array([gl, gr], np.float32)
self.bus[bus][a:b] += seg
def match(rule, e, fps):
m = rule.get('match', {})
if 'kind' in m and not re.fullmatch(m['kind'], e['kind']):
return False
if 'id' in m and not re.search(m['id'], e.get('id', '')):
return False
t = e['f'] / fps
if 'from' in m and t < m['from'] - 1e-6:
return False
if 'to' in m and t >= m['to'] - 1e-6:
return False
if 'variant' in m and e.get('variant') != m['variant']:
return False
return True
def load_design(path, only=None):
"""the design file, with its "include" fragments merged in (samples / slices / pools / rules / place)"""
if not os.path.exists(P(path)):
sys.exit(f'design file not found: {P(path)}')
d = json.load(open(P(path), encoding='utf-8'))
for inc in d.get('include', []):
if only and only not in inc:
continue
f = load_design(inc)
for k in ('samples', 'slices', 'pools'):
for name, v in (f.get(k) or {}).items():
if name in d.setdefault(k, {}):
raise KeyError(f'{inc}: {k} "{name}" is defined twice')
d[k][name] = v
for k in ('rules', 'place', 'ambience'):
d[k] = (f.get(k) or []) + (d.get(k) or [])
if f.get('music') and not d.get('music'):
d['music'] = f['music']
return d
def missing_files(design):
"""every recording the design references that is not on disk, as 'kind name: path' lines"""
refs = [(f'sample {n}', s.get('file')) for n, s in (design.get('samples') or {}).items()]
refs += [(f'slices {n}', s.get('file')) for n, s in (design.get('slices') or {}).items()]
refs += [(f"ambience {a.get('name') or i}", a.get('file')) for i, a in enumerate(design.get('ambience') or [])]
if design.get('music'):
refs.append(('music', design['music'].get('file')))
return [f'{what}: {f}' for what, f in refs if not f or not os.path.isfile(P(f))]
def check_files(design):
miss = missing_files(design)
if miss:
sys.exit(f'{len(miss)} recording(s) referenced by the design are missing (paths relative to --root {ROOT}):\n '
+ '\n '.join(miss))
def dump(args):
"""write every sample / slice of the design as its own WAV + measurements, to check the trims"""
design = load_design(args.design, args.only)
check_files(design)
lib = Lib(design)
os.makedirs(P(args.dump), exist_ok=True)
rows = []
for name, x in lib.samples.items():
fn = re.sub(r'[^A-Za-z0-9._#-]+', '_', name).replace('#', '-') + '.wav'
sf.write(os.path.join(P(args.dump), fn), x, SR, subtype='PCM_24')
m = np.abs(x).max(axis=1)
pk = float(m.max()) or 1e-9
a = env_index(x, 'attack')
rows.append({'name': name, 'file': fn, 'src': lib.meta.get(name), 'dur': round(len(x) / SR, 3),
'peak_dbfs': round(20 * math.log10(pk), 1),
'rms_dbfs': round(10 * math.log10(float(np.mean(x.astype(np.float64) ** 2)) + 1e-20), 1),
'attack_ms': round(a / SR * 1000, 1), 'peak_ms': round(env_index(x, 'peak') / SR * 1000, 1),
'head_db': round(20 * math.log10(float(m[: int(0.002 * SR)].max()) / pk + 1e-9), 1),
'tail_db': round(20 * math.log10(float(m[-int(0.02 * SR):].max()) / pk + 1e-9), 1)})
for name, items in lib.pools.items():
rows.append({'pool': name, 'size': len(items), 'items': items})
out = os.path.join(P(args.dump), '_samples.json')
json.dump(rows, open(out, 'w'), indent=1)
print(f'dumped {len(lib.samples)} samples, {len(lib.pools)} pools -> {out}')
for r in rows:
if 'pool' in r:
print(f" pool {r['pool']}: {r['size']}")
else:
print(f" {r['name']:<34} {r['dur']:6.3f}s pk {r['peak_dbfs']:6.1f} rms {r['rms_dbfs']:6.1f} "
f"att {r['attack_ms']:6.1f}ms head {r['head_db']:6.1f} tail {r['tail_db']:6.1f}")
def run(args):
if args.dump:
return dump(args)
design = load_design(args.design, args.only)
cue_path = args.cues or design.get('cues') or 'out/cues.json'
if os.path.exists(P(cue_path)):
cues = json.load(open(P(cue_path)))
cue = cues[0] if isinstance(cues, list) else cues
elif design.get('rules'):
sys.exit(f'cue file {cue_path} not found (the design has cue rules); pass --cues')
else:
cue = {'fps': 60, 'events': []}
fps = cue.get('fps', 60)
if 'duration' not in design and 'total' not in cue:
sys.exit('set "duration" (seconds) in the design or "total" (frames) in the cue file')
dur = design.get('duration', cue.get('total', 0) / fps)
check_files(design)
lib = Lib(design)
mix = Mix(dur)
placed_events = []
report = {'placements': [], 'unsounded_events': [], 'buses': {}}
# --- cue-driven effects
events = sorted(cue['events'], key=lambda e: e['f'])
counters, last = {}, {}
sounded = set()
for ri, rule in enumerate(design.get('rules', [])):
pool = lib.pools[rule['pool']]
matched = [e for e in events if match(rule, e, fps)]
rk = rule.get('name', ri) # per-event variation keyed on the rule's name: identical solo and merged
for mi, e in enumerate(matched):
mode = rule.get('mode', 'cycle')
if mode == 'fixed':
name = pool[0]
elif mode == 'hash':
name = pool[int(h01(f'{e.get("id")}|{e["f"]}|{rk}') * len(pool)) % len(pool)]
else:
c = counters.get(ri, int(rule.get('cycle_start', 0)))
name = pool[c % len(pool)]
if len(pool) > 1 and name == last.get(ri):
c += 1; name = pool[c % len(pool)]
counters[ri] = c + 1
last[ri] = name
lo, hi = rule.get('pitch', [0, 0])
semis = lo + (hi - lo) * h01(f'p|{e.get("id")}|{e["f"]}|{rk}') if hi != lo else lo
if rule.get('pitch_ramp'): # climbs across the matched events in time order (growth, stacking)
r0, r1 = rule['pitch_ramp']
semis += r0 + (r1 - r0) * (mi / max(1, len(matched) - 1))
if rule.get('gain_ramp'):
g0, g1 = rule['gain_ramp']
rule_gain_ramp = g0 + (g1 - g0) * (mi / max(1, len(matched) - 1))
else:
rule_gain_ramp = 0.0
g = rule.get('gain_db', 0.0) + rule_gain_ramp
if rule.get('weight_db'):
g += rule['weight_db'] * math.log2(max(e.get('weight', 1.0), 1e-3) / rule.get('weight_ref', 1.0))
if rule.get('jitter_db'):
g += (h01(f'g|{e.get("id")}|{e["f"]}|{rk}') * 2 - 1) * rule['jitter_db']
pan = rule.get('pan', 'cue')
pan = (e.get('pan', 0.0) * rule.get('pan_scale', 1.0)) if pan == 'cue' else float(pan)
x = lib.get(name, semis)
align = rule.get('align', 'attack')
t = e['f'] / fps + rule.get('offset', 0.0)
start = int(round(t * SR)) - env_index(x, align)
mix.add(rule.get('bus', 'sfx'), x, start, db2a(g), pan, sample=name, rule=rule.get('name', ri), kind=e['kind'])
report['placements'].append({'t': round(t, 4), 'event': e.get('id'), 'kind': e['kind'], 'rule': rule.get('name', ri),
'sample': name, 'gain_db': round(g, 2), 'pitch': round(semis, 2),
'pitch_total': round(semis + (lib.meta.get(name, {}).get('pitch') or 0), 2),
'pan': round(pan, 3), 'align': align})
if rule.get('bus', 'sfx') == 'sfx' and not rule.get('no_cue'):
key = (e['f'], e['kind'], e.get('id'))
if key not in sounded:
sounded.add(key)
placed_events.append(e)
for e in events:
if (e['f'], e['kind'], e.get('id')) not in sounded:
report['unsounded_events'].append({'t': round(e['f'] / fps, 3), 'kind': e['kind'], 'id': e.get('id')})
# --- free placements
for p in design.get('place', []):
x = lib.get(p['sample'], p.get('pitch', 0.0))
t = p['at'] + p.get('offset', 0.0)
start = int(round(t * SR)) - env_index(x, p.get('align', 'attack'))
mix.add(p.get('bus', 'sfx'), x, start, db2a(p.get('gain_db', 0.0)), p.get('pan', 0.0), sample=p['sample'], kind='place')
report['placements'].append({'t': round(t, 4), 'event': None, 'kind': 'place', 'sample': p['sample'],
'gain_db': p.get('gain_db', 0.0), 'bus': p.get('bus', 'sfx')})
# --- ambience beds
for a in design.get('ambience', []):
x = decode(a['file'])
s0 = int(a.get('src_start', 0) * SR)
need = int((a['until'] - a['at']) * SR)
src = x[s0: int(a['src_end'] * SR) if a.get('src_end') is not None else len(x)] # the clean window (also the loop)
looped = len(src) < need
if looped: # loop with a 1.5 s equal-power crossfade
xf = int(1.5 * SR)
out = src.copy()
while len(out) < need:
nxt = src.copy()
fo = np.sqrt(rc_fade(xf, False))[:, None]; fi = np.sqrt(rc_fade(xf))[:, None]
out[-xf:] = out[-xf:] * fo + nxt[:xf] * fi
out = np.concatenate([out, nxt[xf:]])
src = out
seg = src[:need]
if a.get('width') is not None:
m, s = (seg[:, 0] + seg[:, 1]) / 2, (seg[:, 0] - seg[:, 1]) / 2 * a['width']
seg = np.stack([m + s, m - s], axis=1)
seg = eq(seg, a.get('eq'))
seg = apply_fades(seg, a.get('fade_in', 1.0), a.get('fade_out', 1.0))
start = int(round(a['at'] * SR))
if a.get('auto'):
curve = auto_curve([[t - a['at'], d] for t, d in a['auto']], len(seg))
seg = seg * curve[:, None]
mix.add('amb', seg.astype(np.float32), start, db2a(a.get('gain_db', 0.0)), name=a.get('name') or a['file'], kind='ambience')
body = seg[int(min(len(seg) - 1, a.get('fade_in', 1.0) * SR)): max(1, len(seg) - int(a.get('fade_out', 1.0) * SR))]
report.setdefault('ambience', []).append({
'name': a.get('name'), 'file': a['file'], 'at': a['at'], 'until': a['until'], 'looped': looped,
'body_rms_dbfs': round(10 * math.log10(float(np.mean(body.astype(np.float64) ** 2)) + 1e-20)
+ a.get('gain_db', 0.0), 1) if len(body) else None})
# --- music
mu = design.get('music')
if mu:
ratio = mu.get('target_bpm', 120) / mu['src_bpm'] if mu.get('stretch', 'rubberband') != 'none' else 1.0
x = stretched(mu['file'], ratio)
k = 1.0 / ratio # source seconds -> stretched seconds
for sg in mu['segments']:
fin, fout = sg.get('fade_in', 0.008), sg.get('fade_out', 0.06)
a = int(round(sg['src_from'] * k * SR)) - int(fin * SR)
b = int(round(sg['src_to'] * k * SR)) + int(fout * SR)
pre = int(fin * SR)
if a < 0:
pre += a; a = 0
seg = apply_fades(x[a:b], fin if sg.get('fade_in_at_start', True) else 0, fout)
if sg.get('tail_fade'): # a longer musical fade over the end of the segment
n = int(sg['tail_fade'] * SR)
seg[-n:] *= rc_fade(n, False)[:, None]
start = int(round(sg['at'] * SR)) - pre
mix.add('music', seg.astype(np.float32), start, db2a(sg.get('gain_db', 0.0)), label=sg.get('label'), kind='music')
report.setdefault('music_segments', []).append({**sg, 'stretched_from': round(a / SR, 4), 'film_start': round(start / SR, 4)})
if mu.get('eq'):
mix.bus['music'] = eq(mix.bus['music'], mu['eq'])
mix.bus['music'] *= db2a(mu.get('gain_db', 0.0))
# --- buses
for name, b in (design.get('buses') or {}).items():
if name.startswith('_'): # a comment key
continue
if name not in mix.bus:
sys.exit(f'buses: unknown bus "{name}" (sfx, amb or music; comment keys start with _)')
y = eq(mix.bus[name], b.get('eq'))
c = auto_curve(b.get('auto'), mix.n)
if c is not None:
y = y * c[:, None]
mix.bus[name] = (y * db2a(b.get('gain_db', 0.0))).astype(np.float32)
# every recording that is audible in the film (for CREDITS.md)
used = {lib.meta[p['sample']]['file'] for p in report['placements'] if p.get('sample') in lib.meta}
used |= {a['file'] for a in design.get('ambience', [])}
if mu:
used.add(mu['file'])
report['files_used'] = sorted(u.replace('\\', '/') for u in used)
total = mix.bus['sfx'] + mix.bus['amb'] + mix.bus['music']
os.makedirs(P(args.stems), exist_ok=True)
for name, y in mix.bus.items():
sf.write(os.path.join(P(args.stems), f'{name}.wav'), y, SR, subtype='FLOAT')
pk = float(np.abs(y).max())
report['buses'][name] = {'peak_dbfs': round(20 * math.log10(max(pk, 1e-9)), 2),
'rms_dbfs': round(10 * math.log10(float(np.mean(y.astype(np.float64) ** 2)) + 1e-20), 2)}
# mastered: the unmastered sum goes next to --out as premaster.wav; --no-master / --only: it IS the output
pre = P(args.out) if args.no_master else os.path.join(os.path.dirname(P(args.out)), 'premaster.wav')
sf.write(pre, total, SR, subtype='FLOAT')
report['premaster_peak_dbfs'] = round(20 * math.log10(max(float(np.abs(total).max()), 1e-9)), 2)
# cues that received an effect, for the A/V audit
cp = {'fps': fps, 'total': cue.get('total', int(dur * fps)), 'events': [
{k: v for k, v in e.items() if k in ('f', 'kind', 'weight', 'id', 'pan', 'dur', 'apexFrac')} for e in placed_events]}
for e in cp['events']:
if e['kind'] in ('riser', 'swell'):
e['ends'] = True
os.makedirs(os.path.dirname(P(args.cues_out)) or '.', exist_ok=True)
json.dump(cp, open(P(args.cues_out), 'w'), indent=0)
os.makedirs(os.path.dirname(P(args.report)), exist_ok=True)
json.dump(report, open(P(args.report), 'w'), indent=1)
if args.trace:
os.makedirs(os.path.dirname(P(args.trace)) or '.', exist_ok=True)
json.dump(mix.log, open(P(args.trace), 'w'), indent=0)
print(f'premaster {pre} peak {report["premaster_peak_dbfs"]} dBFS placements {len(report["placements"])} '
f'unsounded {len(report["unsounded_events"])}')
for k, v in report['buses'].items():
print(f' bus {k}: {v}')
if not args.no_master and not np.any(total):
print('the mix is digital silence (every bus is zero): nothing to master, no soundtrack written'
+ (f' ({P(args.out)} is from an earlier render)' if os.path.exists(P(args.out)) else '')
+ '. Check the trims (--dump) and the gains.')
sys.exit(1)
if not args.no_master:
m = design.get('master', {})
cmd = [sys.executable, '-I', MASTER, pre, P(args.out), '--lufs', str(m.get('lufs', -14)), '--ceiling',
str(m.get('ceiling', 0.77)), '--json', os.path.join(os.path.dirname(P(args.report)), 'master.json')]
if m.get('no_glue', True):
cmd.append('--no-glue')
if m.get('no_comp'):
cmd.append('--no-comp')
if m.get('mono_hz') is not None:
cmd += ['--mono-hz', str(m['mono_hz'])]
r = subprocess.run(cmd, capture_output=True, text=True)
print(r.stdout[-1500:], r.stderr[-1500:])
if r.returncode != 0:
print('master gate FAILED')
sys.exit(1)
if __name__ == '__main__':
ap = argparse.ArgumentParser(description=__doc__.splitlines()[1])
ap.add_argument('--root', default='.', help='project root: every relative path is resolved against it')
ap.add_argument('--design', default='audio/sounddesign.json')
ap.add_argument('--cues', help='cue file (default: the design\'s "cues", else out/cues.json)')
ap.add_argument('--out', help='default out/audio/soundtrack.wav (solo: out/audio-review/solo/<only>.wav)')
ap.add_argument('--stems', help='default out/audio/stems (solo: next to --out)')
ap.add_argument('--no-master', action='store_true')
ap.add_argument('--only', help='solo render of the include fragments whose path contains this text (no master)')
ap.add_argument('--dump', help='write every sample of the design to this folder as WAV + _samples.json, then exit')
ap.add_argument('--report', help='default out/qa/mix_report.json (solo: next to --out)')
ap.add_argument('--cues-out', help='the cue events that received a sound (A/V audit); default out/qa/cues_sounded.json')
ap.add_argument('--trace', help='write every Mix.add() record here (QA: click / masking attribution)')
ap.add_argument('--cache', help='decode cache folder (default <root>/out/audio/cache)')
ap.add_argument('--master-script', help='mastering script (default: master.py next to this file)')
a = ap.parse_args()
ROOT = os.path.abspath(a.root)
CACHE = os.path.abspath(os.path.join(ROOT, a.cache)) if a.cache else os.path.join(ROOT, 'out', 'audio', 'cache')
if a.master_script:
MASTER = os.path.abspath(a.master_script)
if a.only:
a.no_master = True
tag = re.sub(r'[^A-Za-z0-9._-]+', '_', a.only)
a.out = a.out or f'out/audio-review/solo/{tag}.wav'
base = os.path.splitext(a.out)[0]
a.stems = a.stems or base + '_stems'
a.report = a.report or base + '_report.json'
a.cues_out = a.cues_out or base + '_cues.json'
a.out = a.out or 'out/audio/soundtrack.wav'
a.stems = a.stems or 'out/audio/stems'
a.report = a.report or 'out/qa/mix_report.json'
a.cues_out = a.cues_out or 'out/qa/cues_sounded.json'
os.makedirs(os.path.dirname(P(a.out)) or '.', exist_ok=True)
run(a)
scripts/mux.mjs
#!/usr/bin/env node
// Put a soundtrack on the finished picture without re-encoding the video (stream copy; AAC 48 kHz stereo audio).
// node <skill>/scripts/mux.mjs --video out/film.mp4 [--audio out/audio/soundtrack.wav] [--out out/film_sound.mp4]
// [--abr 320k] [--keep-old out/superseded/film_prev.mp4] [--root .]
// Relative paths resolve against --root (default: the current directory). The audio is cut to the video's own duration
// (ffprobe), so a soundtrack that runs long never extends the film; a length mismatch over one frame is reported.
// When --out is the --video file itself, the old file is moved to --keep-old first (default
// out/superseded/<name>_prev<ext>); an existing keep-old file is never overwritten. Exit 0 = written and verified.
import { spawnSync } from 'node:child_process';
import path from 'node:path';
import fs from 'node:fs';
const argv = process.argv.slice(2);
if (argv.includes('-h') || argv.includes('--help')) {
console.log(fs.readFileSync(new URL(import.meta.url), 'utf8').split('\n').slice(1, 8).map((l) => l.replace(/^\/\/ ?/, '')).join('\n'));
process.exit(0);
}
const opt = (k, d) => { const i = argv.indexOf(`--${k}`); return i >= 0 && argv[i + 1] ? argv[i + 1] : d; };
const ROOT = path.resolve(opt('root', '.'));
const R = (p) => (path.isAbsolute(p) ? p : path.join(ROOT, p));
const rel = (p) => path.relative(ROOT, p) || p;
const fail = (m) => { console.error(`mux: ${m}`); process.exit(1); };
function probe(file) {
const r = spawnSync('ffprobe', ['-v', 'error', '-show_entries', 'stream=codec_type,codec_name,r_frame_rate,duration:format=duration',
'-of', 'json', file], { encoding: 'utf8' });
if (r.status !== 0) fail(`ffprobe failed on ${file}: ${(r.stderr || '').slice(0, 300)}`);
const j = JSON.parse(r.stdout || '{}');
const v = (j.streams || []).find((s) => s.codec_type === 'video');
const a = (j.streams || []).find((s) => s.codec_type === 'audio');
const [n, d] = ((v && v.r_frame_rate) || '0/1').split('/').map(Number);
const fmt = Number((j.format || {}).duration || 0);
return { video: v, audio: a, fps: d ? n / d : 0, duration: Number((v && v.duration) || fmt), audioDuration: Number((a && a.duration) || fmt) };
}
const audio = R(opt('audio', 'out/audio/soundtrack.wav'));
let video = R(opt('video', 'out/film.mp4'));
const out = R(opt('out', video));
const abr = opt('abr', '320k');
if (!fs.existsSync(audio)) fail(`no audio ${audio} (pass --audio)`);
if (!fs.existsSync(video)) fail(`no video ${video} (pass --video)`);
const pv = probe(video);
if (!pv.video) fail(`${video} has no video stream`);
const pa = probe(audio);
const frame = pv.fps ? 1 / pv.fps : 1 / 60;
const diff = pa.audioDuration - pv.duration;
if (Math.abs(diff) > frame) {
console.log(`note: the audio is ${diff > 0 ? 'longer' : 'SHORTER'} than the picture by ${Math.abs(diff).toFixed(3)} s ` +
`(${(Math.abs(diff) / frame).toFixed(1)} frames)` + (diff > 0 ? ': it is cut at the picture\'s end' : ': the end will be silent'));
}
if (path.resolve(out) === path.resolve(video)) {
const ext = path.extname(video);
const keep = R(opt('keep-old', path.join('out', 'superseded', path.basename(video, ext) + '_prev' + ext)));
fs.mkdirSync(path.dirname(keep), { recursive: true });
if (fs.existsSync(keep)) fail(`refusing to overwrite ${keep}; pass --keep-old <new path>`);
fs.renameSync(video, keep);
video = keep;
console.log(`previous version kept at ${rel(keep)}`);
}
fs.mkdirSync(path.dirname(out), { recursive: true });
const ext = path.extname(out) || '.mp4';
const tmp = out.slice(0, out.length - path.extname(out).length) + '.muxing' + ext;
const args = ['-v', 'error', '-y', '-i', video, '-i', audio, '-map', '0:v:0', '-map', '1:a:0', '-c:v', 'copy', '-c:a', 'aac',
'-b:a', abr, '-ar', '48000', '-ac', '2', '-t', pv.duration.toFixed(6)];
if (/\.(mp4|m4v|mov)$/i.test(ext)) args.push('-movflags', '+faststart');
const r = spawnSync('ffmpeg', [...args, tmp], { stdio: 'inherit' });
if (r.status !== 0) { try { fs.unlinkSync(tmp); } catch { /* nothing to clean */ } fail('ffmpeg failed'); }
const po = probe(tmp);
if (!po.video || !po.audio) fail(`${tmp} lacks a video or audio stream`);
fs.renameSync(tmp, out);
console.log(`wrote ${rel(out)} (${(fs.statSync(out).size / 1e6).toFixed(1)} MB): video copied (${pv.video.codec_name}, ` +
`${pv.fps.toFixed(3)} fps, ${pv.duration.toFixed(3)} s), audio ${po.audio.codec_name} ${abr} ${po.audioDuration.toFixed(3)} s`);
scripts/qa.py
#!/usr/bin/env python
"""Objective QA of a real-sound mix: what the critic lenses measure, in one pass, tied to the placements that caused it.
python -I <skill>/scripts/qa.py [--root .] [--design audio/sounddesign.json] [--report out/qa/mix_report.json]
[--trace out/qa/adds.json] [--stems out/audio/stems] [--master out/audio/soundtrack.wav]
[--cues out/cues.json] [--only NAME] [--cache out/audio/cache] [--video out/film.mp4]
[--out out/qa/qa.json] [--layers] [--listen] [--realness] [--json]
Run it after `mix.py --trace out/qa/adds.json` (same design, same --only): the trace ties every click, masked hit and
harsh peak to the placement that caused it. With --only, the report / stems default to mix.py's solo outputs
(out/audio-review/solo/<only>_report.json, _stems) and there is no master (levels at stem scale). Without the trace
only clicks, loudness and coverage run. No models needed; --listen / --realness add earshot's ears when installed. Sections (status ok / warn / fail + a plain summary):
clicks > 14 kHz bursts concentrated in 1 ms (the av-audit detector) in the master and each stem, attributed to
the placements whose start / end / body they sit in
edges used samples that start mid-sound or are cut mid-decay (unfaded head / tail level vs their peak, and the
source just outside the trim), a second hit inside a sample, steps in a stem at a placement's start / end
(second-difference ratio), placements cut by the film's ends
audibility every cue event's effect (its layers summed, rebuilt from the trace) vs the bed (music + amb) and vs
everything else, in 1/3-octave bands over its loudest 60 ms and its first 20 ms: read >= 6 dB clear, 3-6 ok,
0-3 barely, < 0 masked (margins capped at +-60 dB); an effect peaking under -70 dBFS is SILENT (a trim
that missed the sound: fail); not judged when the render has no bed (a solo); --layers also lists single
layers that add nothing measurable
harshness each event's 2-6 kHz peak (5 ms RMS, master scale) vs the music's loudest 2-6 kHz 5 ms level
loudness integrated LUFS vs the design's target, true peak, LRA, loudest momentary (time, share of the runtime),
short-term curve, per-act medians when the cue file has acts, head and tail of the master
coverage cue events with no sound (unsounded or placed but silent); classes of action sounded only in part
sync impacts by attack / peak (-45..+75 ms), whooshes and peak-aligned swells by their apex (+-45 ms), risers
and end-aligned swells by their audible end vs the hit (-250..+30 ms); measured norms of professional
launch films. A deviation that exactly one layer's rule offset explains is listed apart as deliberate
density events per second, the most reused samples, "machine-gun" rules (one sample, no pitch or level change)
listen --listen: earshot timeline + sheet + critique of the master; --realness: earshot realness of every source
Thresholds and where they come from: references/process.md. Writes --out (JSON) and prints a digest. Exit code 1 when
a section fails (a click at a placement edge in the master, a silent effect, stems of pure digital silence, the
loudness gate), 2 on missing or mismatched inputs (one line says which flag to pass), else 0.
"""
import argparse, json, math, os, re, sys
import numpy as np
import soundfile as sf
from scipy import signal
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import master as M # noqa: E402
import mix # noqa: E402 (read-only: the design loader, the sample library, envelope helpers)
SR = mix.SR
LONG = ('whoosh', 'riser', 'swell', 'suck')
FC = 10 ** (np.arange(20, 42) / 10) # 1/3-octave centres 100 Hz .. 12.6 kHz
BLO, BHI = FC * 2 ** (-1 / 6), FC * 2 ** (1 / 6)
SILENT_DB = -70.0 # an event whose peak (master scale) is below this makes no sound: a trim that missed the recording
CAP_DB = 60.0 # margins against a (near-)silent bed are reported as at most +60 dB
class InputError(Exception):
pass
def db(v):
return 20 * math.log10(max(float(v), 1e-12))
def pdb(p):
return 10 * math.log10(max(float(p), 1e-20))
def fmt_t(t):
m = int(t // 60)
return f'{m}:{t - 60 * m:05.2f}'
def say(s):
print(str(s).encode('ascii', 'replace').decode())
def lufs(x):
"""integrated LUFS, or None for audio shorter than one 400 ms block or with no gated signal (silence)"""
if x is None or len(x) < int(0.4 * SR):
return None
v = M.integrated(x)
return v if math.isfinite(v) else None
def clean(o):
"""JSON-safe: numpy scalars to Python, NaN / inf to None"""
if isinstance(o, dict):
return {k: clean(v) for k, v in o.items()}
if isinstance(o, (list, tuple)):
return [clean(v) for v in o]
if isinstance(o, np.integer):
return int(o)
if isinstance(o, (float, np.floating)):
return float(o) if math.isfinite(o) else None
return o
# ------------------------------------------------------------------ context
class Ctx:
def __init__(self, a):
self.root = os.path.abspath(a.root)
mix.ROOT = self.root
mix.CACHE = os.path.join(self.root, a.cache) if a.cache else os.path.join(self.root, 'out', 'audio', 'cache')
R = self.R = lambda p: os.path.normpath(p if os.path.isabs(p) else os.path.join(self.root, p))
def need(path, what, flag):
if not os.path.exists(R(path)):
raise InputError(f'{what} not found: {R(path)} ({flag})')
return R(path)
need(a.design, 'design file', 'pass --design')
self.design = mix.load_design(a.design, a.only)
self.report = json.load(open(need(a.report, 'mix report', 'run mix.py first or pass --report'), encoding='utf-8'))
self.trace = json.load(open(R(a.trace))) if a.trace and os.path.exists(R(a.trace)) else None
self.stems = {}
for k in ('sfx', 'amb', 'music'):
p = need(os.path.join(a.stems, k + '.wav'), f'stem {k}.wav', 'pass the --stems folder mix.py wrote')
self.stems[k] = sf.read(p, dtype='float32', always_2d=True)[0]
self.N = len(self.stems['sfx'])
if {len(v) for v in self.stems.values()} != {self.N}:
raise InputError(f'the stems in {R(a.stems)} have different lengths: re-render them with one mix.py run')
if a.master and getattr(a, 'master_given', False):
need(a.master, 'master', 'pass --master none to skip it')
self.master = (sf.read(R(a.master), dtype='float64', always_2d=True)[0]
if a.master and os.path.exists(R(a.master)) else None)
if self.master is not None and abs(len(self.master) - self.N) > 1:
raise InputError(f'the master {R(a.master)} ({len(self.master) / SR:.3f} s) does not match the stems '
f'({self.N / SR:.3f} s): pass the --master rendered with these stems, or --master none')
cp = a.cues or self.design.get('cues') or 'out/cues.json'
c = json.load(open(R(cp))) if os.path.exists(R(cp)) else {'fps': 60, 'events': []}
self.cues = c[0] if isinstance(c, list) else c
self.fps = self.cues.get('fps', 60)
m = self.design.get('master') or {}
self.target = m.get('lufs', -14)
pre = sum(self.stems[k].astype(np.float64) for k in self.stems)
self.pre_lufs = lufs(pre)
self.master_lufs = lufs(self.master)
both = self.master_lufs is not None and self.pre_lufs is not None
self.G = self.master_lufs - self.pre_lufs if both else 0.0 # no master (or silence): stem scale
self.scale = ('master' if both else 'stem') if self.master is not None else 'stem'
self.no_bed = not (np.any(self.stems['music']) or np.any(self.stems['amb']))
self.lib = None
self.only = a.only
def bus_curve(design, bus, n):
b = (design.get('buses') or {}).get(bus) or {}
c = mix.auto_curve(b.get('auto'), n)
return (np.ones(n) if c is None else c.astype(np.float64)) * mix.db2a(b.get('gain_db', 0.0))
def build_layers(ctx):
"""every sfx-bus placement rebuilt as it sits in the stem (sample, pitch, gain, pan, bus ride), grouped into events"""
recs = [a for a in ctx.trace if 'sample' in a]
pl = ctx.report.get('placements', [])
if len(recs) != len(pl):
return None, f'trace ({len(recs)} placements) and report ({len(pl)}) disagree: re-run mix.py with --trace'
ctx.lib = ctx.lib or mix.Lib(ctx.design)
places = iter(ctx.design.get('place') or [])
rules = {str(r.get('name', i)): r for i, r in enumerate(ctx.design.get('rules') or [])}
curve = bus_curve(ctx.design, 'sfx', ctx.N)
out, bad = [], 0
for a, p in zip(recs, pl):
if p['kind'] == 'place':
pp = next(places)
pitch, off, align, cue, rule = pp.get('pitch', 0.0), pp.get('offset', 0.0), pp.get('align', 'attack'), pp['at'], 'place'
ok = pp.get('qa_ok')
else:
r = rules.get(str(p.get('rule')), {})
pitch, off, align, rule = p.get('pitch', 0.0), r.get('offset', 0.0), p.get('align', 'attack'), str(p.get('rule'))
cue, ok = p['t'] - off, r.get('qa_ok')
x = ctx.lib.get(a['sample'], pitch)
if len(x) != a['n']:
bad += 1
continue
if a['bus'] != 'sfx':
continue
s0, s1 = max(0, a['start']), min(ctx.N, a['start'] + a['n'])
if s1 <= s0:
continue
seg = x[s0 - a['start']: s1 - a['start']].astype(np.float64) * a['gain']
if a['pan']:
seg = seg * np.array(mix.pan_gains(a['pan']))
seg = seg * curve[s0:s1, None]
out.append({'sample': a['sample'], 'rule': rule, 'kind': p['kind'], 'event': p.get('event'), 'cue': cue,
'offset': off, 'align': align, 'start': a['start'], 'n': a['n'], 'a': s0, 'b': s1, 'sig': seg, 'qa_ok': ok,
'att': a['start'] + mix.env_index(x, 'attack'), 'peak_db': db(np.abs(seg).max())})
ctx.layers = out
groups, by_cue = [], {}
for L in sorted(out, key=lambda L: L['att']):
if L['kind'] != 'place': # layers of one cue event share its exact cue time
k = (L['event'], round(L['cue'], 4))
if k in by_cue:
by_cue[k]['layers'].append(L)
else:
by_cue[k] = {'att0': L['att'], 'layers': [L]}
groups.append(by_cue[k])
continue
for g in groups[::-1][:12]: # free placements: layers whose attacks sit within 140 ms
if g.get('place') and abs(L['att'] - g['att0']) < 0.14 * SR:
g['layers'].append(L)
break
else:
groups.append({'place': True, 'att0': L['att'], 'layers': [L]})
for g in groups:
ls = g['layers']
g['a'], g['b'] = min(L['a'] for L in ls), max(L['b'] for L in ls)
g['sig'] = np.zeros((g['b'] - g['a'], 2))
for L in ls:
g['sig'][L['a'] - g['a']: L['b'] - g['a']] += L['sig']
g['cue'] = min(L['cue'] for L in ls)
g['loud'] = max(ls, key=lambda L: L['peak_db'])
g['kind'] = g['loud']['kind']
g['label'] = (ls[0]['event'] or 'place:' + ls[0]['sample'])
g['long'] = any(L['kind'] in LONG or L['align'] == 'peak' for L in ls)
g['builds'] = any(L['kind'] in ('riser', 'swell') for L in ls)
g['qa_ok'] = next((L['qa_ok'] for L in ls if L['qa_ok']), None)
g['peak_db'] = db(np.abs(g['sig']).max()) + ctx.G
g['silent'] = g['peak_db'] < SILENT_DB
note = f'{bad} placement(s) could not be rebuilt (sample length differs from the trace)' if bad else None
return groups, note
# ------------------------------------------------------------------ clicks
HP14 = signal.butter(4, 14000, 'high', fs=SR, output='sos')
def detect_clicks(x):
"""> 14 kHz samples > 9x the RMS of their 20 ms and > 6x their 4 ms neighbourhood; conc >= 0.8 of the burst's
> 14 kHz energy inside 1 ms = a click (a cut), else a designed attack"""
out = []
for ch in range(x.shape[1]):
hp = signal.sosfilt(HP14, x[:, ch].astype(np.float64))
e, e2 = np.abs(hp), hp ** 2
loc = np.sqrt(signal.fftconvolve(e2, np.ones(960) / 960, mode='same').clip(0)) + 1e-5
near = np.sqrt(signal.fftconvolve(e2, np.ones(192) / 192, mode='same').clip(0)) + 1e-5
last = -10 ** 9
for i in np.nonzero((e / loc > 9) & (e / near > 6) & (e > 0.01))[0]:
if i - last > SR * 0.01:
conc = float(e2[max(0, i - 24):i + 24].sum() / (e2[max(0, i - 240):i + 240].sum() + 1e-15))
out.append({'t': round(i / SR, 4), 'i': int(i), 'ch': ch, 'level_db': round(db(e[i]), 1),
'conc': round(conc, 2), 'kind': 'click' if conc >= 0.8 else 'attack'})
last = i
out.sort(key=lambda c: (c['t'], c['ch']))
return [c for k, c in enumerate(out) if k == 0 or c['t'] - out[k - 1]['t'] > 0.001]
def owners(trace, i, bus=None, win=960):
"""trace records around sample i, edge-near (within 2 ms of a start / end) first"""
res = []
for a in trace or []:
if bus and a['bus'] != bus:
continue
s, e = a['start'], a['start'] + a['n']
if s - win <= i <= e + win:
d = min(abs(i - s), abs(i - e))
where = 'start' if abs(i - s) <= 96 else 'end' if abs(i - e) <= 96 else 'body'
what = a.get('sample') or a.get('name') or a.get('label') or f"{a.get('kind', a['bus'])} at {s / SR:.3f} s"
res.append((where == 'body', d, {'bus': a['bus'], 'what': what,
'rule': a.get('rule', a.get('kind')), 'where': where,
'from_start_ms': round((i - s) / SR * 1000, 1),
'from_end_ms': round((i - e) / SR * 1000, 1)}))
return [r for _, _, r in sorted(res, key=lambda r: (r[0], r[1]))[:3]]
def hf_owner(ctx, i, ch):
"""the rebuilt sfx layer carrying the > 14 kHz burst at sample i (its level in the stem, where in the layer)"""
best = None
for L in getattr(ctx, 'layers', None) or []:
if not L['a'] - 480 <= i < L['b'] + 480:
continue
j = i - L['a']
lo, hi = max(0, j - 480), min(len(L['sig']), j + 480)
if hi - lo < 64:
continue
hp = signal.sosfilt(HP14, L['sig'][lo:hi, ch])
c = j - lo
v = float(np.abs(hp[max(0, c - 24):c + 24]).max()) if 0 <= c < len(hp) else 0.0
if best is None or v > best[0]:
off = i - L['start']
where = 'start' if abs(off) <= 96 else 'end' if abs(i - L['start'] - L['n']) <= 96 else 'body'
best = (v, {'bus': 'sfx', 'what': L['sample'], 'rule': L['rule'], 'where': where,
'from_start_ms': round(off / SR * 1000, 1), 'level_db': round(db(v), 1)})
return best
def check_clicks(ctx):
sets = [('master', ctx.master)] if ctx.master is not None else []
sets += [(k, ctx.stems[k]) for k in ('sfx', 'amb', 'music')]
items, counts = [], {}
for name, x in sets:
found = detect_clicks(x)
counts[name] = {'clicks': sum(c['kind'] == 'click' for c in found), 'attacks': sum(c['kind'] == 'attack' for c in found)}
for c in found:
if c['kind'] != 'click':
continue
own = hf_owner(ctx, c['i'], c['ch']) if name in ('master', 'sfx') else None
scale = 10 ** (-ctx.G / 20) if name == 'master' else 1.0 # master level back to stem scale
if own and own[0] >= 0.3 * 10 ** (c['level_db'] / 20) * scale:
ow = [own[1]]
else:
ow = owners(ctx.trace, c['i'], None if name == 'master' else name)
items.append({'in': name, **{k: v for k, v in c.items() if k != 'i'}, 'owners': ow,
'at_edge': bool(ow) and ow[0]['where'] in ('start', 'end')})
mc = [c for c in items if c['in'] == 'master']
edge = [c for c in items if c['at_edge']]
sc = len(items) - len(mc)
status = 'fail' if any(c['in'] == 'master' for c in edge) else 'warn' if items else 'ok'
summ = (f'{len(mc)} click(s) in the master, {sc} in the stems; {len(edge)} at a placement edge (a cut: fix the trim / '
'fade)' + (', the rest inside a recording (its own crisp transient or a defect in the source)' if items else '')
+ ('' if items else ' (designed attacks: ' + ', '.join(f"{k} {v['attacks']}" for k, v in counts.items()) + ')'))
return {'status': status, 'summary': summ, 'counts': counts, 'items': items}
# ------------------------------------------------------------------ edges
def rms(z):
return float(np.sqrt((np.asarray(z, np.float64) ** 2).mean())) if len(z) else 0.0
def sample_edges(x, raw_pre, raw_post, rawpk):
"""unfaded head / tail 1 ms RMS vs the sample's peak, the source 20 ms outside the trim vs the trim's peak, and a
second hit (a 5 ms envelope jump > 12 dB to within 18 dB of the peak) in the last 40 %"""
m = np.abs(x).max(axis=1)
pk = float(m.max()) or 1e-9
w = 240
n = len(m) // w
e = m[:n * w].reshape(n, w).max(axis=1) if n else np.zeros(1)
late = None
for k in range(max(1, int(n * 0.6)), n):
if e[k] > pk * 0.12 and e[k] > 4 * e[k - 1]:
late = round(k * w / SR, 3)
break
return {'head_db': round(db(rms(x[:48]) / pk), 1), 'tail_db': round(db(rms(x[-48:]) / pk), 1),
'pre20_db': round(db(rms(raw_pre) / rawpk), 1), 'post20_db': round(db(rms(raw_post) / rawpk), 1),
'attack_ms': round(mix.env_index(x, 'attack') / SR * 1000, 1), 'late_onset': late}
def judge_edges(r, fi, fo, trimmed_start):
"""(defects, notes): a defect is audible as a click or a chop; a note is a craft suggestion"""
bad, notes = [], []
if trimmed_start and r['pre20_db'] > -20:
msg = f"the trim cuts into a sound already playing (source 20 ms before at {r['pre20_db']} dB re peak)"
if fi < 0.005 and (r['attack_ms'] > 10 or r['pre20_db'] > -10): # exposed: no attack right after to cover it
bad.append(msg + f', fade_in only {fi * 1000:.0f} ms')
else:
notes.append(msg + ', softened by the fade-in')
elif r['head_db'] > -30:
notes.append(f"the attack starts in the first ms (head {r['head_db']} dB): the {fi * 1000:.0f} ms fade-in dulls it; "
'start the trim 3-5 ms earlier')
if r['tail_db'] > -35 and r['post20_db'] > -35:
msg = f"the sound continues after the trim (tail {r['tail_db']} dB, source after {r['post20_db']} dB)"
if fo < 0.03:
bad.append(msg + f': chopped with a {fo * 1000:.0f} ms fade')
else:
notes.append(msg + ': decay faded early')
if r['late_onset'] is not None:
notes.append(f"a second hit inside the sample at {r['late_onset']} s (fine for a rattle, wrong for a single hit)")
return bad, notes
def check_splices(ctx):
"""music segment boundaries: a segment end that chops a note still sounding into quieter material, holes and overlaps
between segments, segments that begin inside a sustained sound"""
mu = ctx.design.get('music')
if not mu or not mu.get('segments'):
return []
ratio = mu.get('target_bpm', 120) / mu['src_bpm'] if mu.get('stretch', 'rubberband') != 'none' else 1.0
x = mix.stretched(mu['file'], ratio)
k = 1.0 / ratio
w = int(0.05 * SR)
def lv(i0):
return db(rms(x[max(0, i0):max(0, i0 + w)]))
segs = sorted(mu['segments'], key=lambda sg: sg['at'])
dur = ctx.N / SR
out = []
for n, sg in enumerate(segs):
s0, e0 = int(round(sg['src_from'] * k * SR)), int(round(sg['src_to'] * k * SR))
end_t = sg['at'] + (sg['src_to'] - sg['src_from']) * k
nxt = segs[n + 1] if n + 1 < len(segs) else None
out_l, cut_l = lv(e0 - w), lv(e0)
in_l = lv(int(round(nxt['src_from'] * k * SR))) if nxt else -120.0
item = {'segment': sg.get('label') or n, 'film_end': round(end_t, 3), 'out_db': round(out_l, 1),
'cut_off_db': round(cut_l, 1), 'next_in_db': round(in_l, 1) if nxt else None, 'issues': [], 'notes': []}
fo = sg.get('fade_out', 0.06)
if not sg.get('tail_fade') and fo < 0.1 and end_t < dur - 0.05 and cut_l > out_l - 6 and cut_l > in_l + 6:
msg = (f'chops the music at {end_t:.3f} s: the source keeps sounding at {cut_l:.0f} dBFS and the next '
f'material enters at {in_l:.0f} (fade_out {fo * 1000:.0f} ms)')
if sg.get('qa_ok'):
item['notes'].append(f"acknowledged ({sg['qa_ok']}): {msg}")
else:
item['issues'].append(msg)
if nxt:
gap = nxt['at'] - end_t
if gap > 0.02:
item['notes'].append(f'{gap * 1000:.0f} ms of no music before {nxt["at"]:.3f} s (deliberate silence?)')
elif gap < -0.1:
item['notes'].append(f'overlaps the next segment by {-gap * 1000:.0f} ms (two passages at once)')
if s0 > w and lv(s0 - w) > lv(s0) - 3:
item['notes'].append(f"begins inside a sustained sound (the source before {sg['src_from']} s is as loud): "
'no fresh attack on the splice')
if item['issues'] or item['notes']:
out.append(item)
return out
def check_edges(ctx):
items = []
used = {p.get('sample') for p in ctx.report.get('placements', [])}
lib = ctx.lib = ctx.lib or mix.Lib(ctx.design)
for name, s in (ctx.design.get('samples') or {}).items():
if name not in used:
continue
raw = mix.decode(s['file'])
a = int(s.get('start', 0) * SR)
b = int(s['end'] * SR) if s.get('end') is not None else len(raw)
fi, fo = s.get('fade_in', 0.002), s.get('fade_out', 0.012)
trim_pk = float(np.abs(raw[a:b]).max()) if b - a > 0 else 0.0
if db(trim_pk) < -60: # nothing to judge: the trim missed the sound
why = ('empty (start >= end or past the file end)' if b - a <= 0 else 'digital silence' if trim_pk == 0
else f'near-silent (peak {db(trim_pk):.0f} dBFS)')
items.append({'what': 'sample', 'name': name, 'file': os.path.basename(s['file']), 'start': s.get('start', 0),
'end': s.get('end'), 'trim_peak_dbfs': round(db(trim_pk), 1), 'fade_in': fi, 'fade_out': fo,
'issues': [f'the trim is {why}: it misses the sound (file length {len(raw) / SR:.3f} s)'],
'notes': []})
continue
x = lib._finish(raw[a:b], {**s, 'fade_in': 0, 'fade_out': 0})
r = sample_edges(x, raw[max(0, a - 960):a], raw[b:b + 960], trim_pk)
bad, notes = judge_edges(r, fi, fo, a > 0)
if bad and s.get('qa_ok'):
notes, bad = notes + [f'acknowledged ({s["qa_ok"]}): ' + b_ for b_ in bad], []
if bad or notes:
items.append({'what': 'sample', 'name': name, 'file': os.path.basename(s['file']), 'start': s.get('start', 0),
'end': s.get('end'), **r, 'fade_in': fi, 'fade_out': fo, 'issues': bad, 'notes': notes})
for name, s in (ctx.design.get('slices') or {}).items():
if not any(u and u.startswith(name + '#') for u in used):
continue
try:
sl = lib._slice({**s, 'fade_in': 0, 'fade_out': 0})
except Exception as e: # noqa: BLE001
items.append({'what': 'slices', 'name': name, 'issues': [f'cannot slice: {e}'], 'notes': []})
continue
fi, fo = s.get('fade_in', 0.002), s.get('fade_out', 0.02)
bad, notes = set(), set()
for x, _ in sl:
r = sample_edges(x, x[:0], x[:0], float(np.abs(x).max()) or 1e-9)
r['post20_db'] = r['tail_db'] # a slice ends where the next hit begins: its tail level is what is cut
b_, n_ = judge_edges(r, fi, fo, False)
bad |= {m.split(' (')[0] for m in b_}
notes |= {m.split(' (')[0] for m in n_}
if bad or notes:
items.append({'what': 'slices', 'name': name, 'n': len(sl), 'issues': sorted(bad), 'notes': sorted(notes)})
cut = []
for a in ctx.trace or []:
if a['bus'] != 'music' and (a['start'] < 0 or a['start'] + a['n'] > ctx.N):
cut.append({'bus': a['bus'], 'what': a.get('sample') or a.get('name') or a.get('label'),
'start': round(a['start'] / SR, 3), 'end': round((a['start'] + a['n']) / SR, 3)})
splices = check_splices(ctx)
n_bad = sum(1 for it in items + splices if it['issues'])
n_note = sum(1 for it in items + splices if it['notes'] and not it['issues'])
summ = (f'{n_bad} defect(s) (abrupt entries, chopped decays, chopped music), {n_note} with craft notes only (used '
f'samples, slices, music splices); {len(cut)} placement(s) run past the film ends')
return {'status': 'warn' if n_bad else 'ok', 'summary': summ,
'items': sorted(items, key=lambda it: (not it['issues'], it['name'])), 'splices': splices,
'cut_by_film_ends': cut}
# ------------------------------------------------------------------ audibility / harshness / sync
def bands(seg):
w = signal.windows.tukey(len(seg), 0.25)[:, None]
S = (np.abs(np.fft.rfft(seg * w, axis=0)) ** 2).sum(axis=1)
f = np.fft.rfftfreq(len(seg), 1 / SR)
c = np.concatenate([[0], np.cumsum(S)])
ia, ib = np.searchsorted(f, BLO), np.searchsorted(f, BHI)
return c[ib] - c[ia] + 1e-30
def window_at(arr, a0, s, w):
o = arr[max(0, s - a0): max(0, s - a0) + w]
return np.pad(o, ((0, w - len(o)), (0, 0))) if len(o) < w else o
def read_metrics(ctx, eff, a, b, att, long):
"""m60 vs bed, m60r / m20 vs everything else (best band within 30 dB of the effect's strongest)"""
bed = ctx.stems['music'][a:b].astype(np.float64) + ctx.stems['amb'][a:b]
rest = ctx.stems['sfx'][a:b] - eff + bed
e2 = (eff ** 2).sum(axis=1)
c = np.concatenate([[0], np.cumsum(e2)])
def best(w, s0, s1):
s0 = max(a, s0)
s1 = max(s0, min(b - w, s1))
cand = np.arange(s0, s1 + 1, 48)
en = [c[min(len(c) - 1, s - a + w)] - c[max(0, s - a)] for s in cand]
return int(cand[int(np.argmax(en))])
w60, w20 = int(0.06 * SR), int(0.02 * SR)
s60 = best(w60, att - 240, (b - w60) if long else att + int(0.3 * SR))
s20 = s60 if long else best(w20, att - 144, att + int(0.08 * SR))
E, B, Rr = (bands(window_at(z, a, s60, w60)) for z in (eff, bed, rest))
E20, R20 = bands(window_at(eff, a, s20, w20)), bands(window_at(rest, a, s20, w20))
def mb(Ep, Bp):
ok = Ep >= Ep.max() * 1e-3
v = np.where(ok, np.clip(10 * np.log10(Ep / Bp), -CAP_DB, CAP_DB), -99)
k = int(np.argmax(v))
return round(float(v[k]), 1), int(round(FC[k]))
m60, f60 = mb(E, B)
m60r, _ = mb(E, Rr)
m20, f20 = mb(E20, R20)
read = max(m60r, m20)
verdict = 'clear' if read >= 6 else 'ok' if read >= 3 else 'barely' if read >= 0 else 'masked'
return {'vs_bed_db': m60, 'vs_bed_hz': f60, 'vs_all_db': m60r, 'first20_db': m20, 'first20_hz': f20,
'read_db': read, 'verdict': verdict, 'covered_by': 'music/amb' if m60 < 3 else 'other effects'}
def check_audibility(ctx, groups, layers_too=False):
rows = []
for g in groups:
att = min(L['att'] for L in g['layers'])
r = {'t': round(att / SR, 3), 'label': g['label'], 'kind': g['kind'], 'n_layers': len(g['layers']),
'rules': sorted({L['rule'] for L in g['layers']}), 'peak_db': round(max(g['peak_db'], -120), 1)}
if g['silent']:
r.update({'read_db': None, 'verdict': 'silent',
'samples': sorted({L['sample'] for L in g['layers']})})
elif ctx.no_bed:
r['verdict'] = 'not judged'
else:
r.update(read_metrics(ctx, g['sig'], g['a'], g['b'], att, g['long']))
if g['qa_ok']:
r['qa_ok'] = g['qa_ok']
rows.append(r)
cnt = {}
for r in rows:
cnt[r['verdict']] = cnt.get(r['verdict'], 0) + 1
masked = [r for r in rows if r['verdict'] == 'masked' and not r.get('qa_ok')]
silent = [r for r in rows if r['verdict'] == 'silent' and not r.get('qa_ok')]
sil = (f'; {len(silent)} SILENT (peak under {SILENT_DB:.0f} dBFS: the trim missed the sound) at '
+ ', '.join(f"{r['t']:.2f} {r['label']} ({', '.join(r['samples'])})" for r in silent[:6]) if silent else '')
if ctx.no_bed:
return {'status': 'fail' if silent else 'skipped', 'counts': cnt, 'events': rows, 'items': silent,
'summary': f'{len(rows)} events, not judged against a bed: this render has no music or ambience (a '
'solo render); judge audibility on the full mix' + sil}
judged = sorted((r for r in rows if r['read_db'] is not None), key=lambda r: r['read_db'])
res = {'status': 'fail' if silent else 'warn' if masked else 'ok', 'counts': cnt,
'summary': f'{len(rows)} events: ' + ', '.join(f'{k} {cnt.get(k, 0)}' for k in ('clear', 'ok', 'barely', 'masked'))
+ sil + ('' if not masked else '; masked at ' + ', '.join(f"{r['t']:.2f} {r['label']} (by {r['covered_by']})"
for r in masked[:6])),
'items': (silent + judged)[:40], 'events': rows}
if layers_too:
dead = []
for g in groups:
if len(g['layers']) < 2 or g['silent']:
continue
for L in g['layers']:
if L['peak_db'] + ctx.G < SILENT_DB:
dead.append({'t': round(L['att'] / SR, 3), 'rule': L['rule'], 'sample': L['sample'], 'read_db': None,
'silent': True})
continue
r = read_metrics(ctx, L['sig'], L['a'], L['b'], L['att'], g['long'])
if r['read_db'] < 0:
dead.append({'t': round(L['att'] / SR, 3), 'rule': L['rule'], 'sample': L['sample'], 'read_db': r['read_db']})
res['dead_layers'] = dead
res['summary'] += f'; {len(dead)} layer(s) in multi-layer events add nothing measurable (read < 0 dB)'
return res
def centroid(x):
S = np.abs(np.fft.rfft(x * np.hanning(len(x)))) ** 2
f = np.fft.rfftfreq(len(x), 1 / SR)
return float((f * S).sum() / (S.sum() + 1e-30))
def check_harshness(ctx, groups):
sos = signal.butter(4, [2000, 6000], 'bandpass', fs=SR, output='sos')
sol = signal.butter(4, [80, 400], 'bandpass', fs=SR, output='sos')
mu = signal.sosfilt(sos, ctx.stems['music'].astype(np.float64).mean(axis=1))
e5 = np.convolve(mu ** 2, np.ones(240) / 240, 'same')
has_music = e5.max() > 1e-10 # a solo render without the music: presence is not judged
ref = pdb(e5.max()) + ctx.G if has_music else None
rows = []
for g in groups:
att = min(L['att'] for L in g['layers'])
x = g['sig'].mean(axis=1)
pres = signal.sosfilt(sos, np.r_[np.zeros(480), x])[480:]
lo = signal.sosfilt(sol, np.r_[np.zeros(480), x])[480:]
i0 = max(0, att - g['a'] - 480)
i1 = len(x) if g['long'] else min(len(x), att - g['a'] + int(0.08 * SR))
if i1 - i0 < 240:
continue
pk = pdb(np.convolve(pres[i0:i1] ** 2, np.ones(240) / 240, 'valid').max()) + ctx.G
lk = pdb(np.convolve(lo[i0:i1] ** 2, np.ones(240) / 240, 'valid').max()) + ctx.G
m0, m1 = max(0, att - SR // 4), min(len(mu), att + SR // 4)
r = {'t': round(att / SR, 3), 'label': g['label'], 'rules': sorted({L['rule'] for L in g['layers']}),
'pres_db': round(pk, 1), 'pres_minus_low_db': round(pk - lk, 1), 'centroid_hz': int(centroid(x[i0:i1])),
'music_pres_db': round(pdb((mu[m0:m1] ** 2).mean()) + ctx.G, 1),
'over_music_max_db': round(pk - ref, 1) if has_music else None}
r['issues'] = []
if has_music and r['over_music_max_db'] > 6:
r['issues'].append(f"2-6 kHz peak {r['pres_db']} dBFS, {r['over_music_max_db']:+.1f} dB over the music's loudest")
r['bright'] = r['centroid_hz'] > 6500
if g['qa_ok'] and r['issues']:
r['qa_ok'], r['issues'] = g['qa_ok'], []
rows.append(r)
rows.sort(key=lambda r: -r['pres_db'])
hot = [r for r in rows if r['issues']]
bright = [r for r in rows if r['bright']]
head = (f"{len(hot)} event(s) more than 6 dB over the music's loudest 2-6 kHz peak ({ref:.1f} dBFS)" if has_music else
'no music in this render: presence vs the music not judged')
return {'status': 'warn' if hot else 'ok', 'music_max_db': round(ref, 1) if has_music else None,
'summary': (head + ('' if not hot else ': ' + ', '.join(f"{r['t']:.2f} {r['label']}" for r in hot[:6]))
+ f"; note: {len(bright)} bright event(s) (centroid > 6.5 kHz, above launch-film effects)"),
'items': hot[:20] + [r for r in rows if not r['issues']][:max(0, 10 - len(hot))],
'bright': [{'t': r['t'], 'label': r['label'], 'centroid_hz': r['centroid_hz']} for r in bright]}
def sync_issue(test, r, s):
"""the sync verdict for a measured event shifted by s ms (a rule offset): None when it sits on its cue"""
if test == 'end': # a riser stops on (or 30-250 ms before) its hit
e = r['end_ms'] - s
return f"its audible end is {e:+d} ms from its hit (want -250..+30)" if not -300 <= e <= 60 else None
if test == 'apex':
e = r['apex_ms'] - s
return f"apex {e:+d} ms from the cue (want +-45)" if abs(e) > 45 else None
a, p = r['attack_ms'] - s, r['peak_ms'] - s
return (f"attack {a:+d} ms (sound before picture)" if a < -45 else f"peak {p:+d} ms (lands late)" if p > 75 else None)
def check_sync(ctx, groups):
"""impacts by attack / peak, whooshes and peak-aligned swells by their apex, risers (and swells placed by their
end or attack) by their audible end vs the hit; a deviation that one layer's rule offset explains is deliberate"""
rows, deliberate, n = [], [], 0
for g in groups:
x = np.abs(g['sig']).max(axis=1)
k = len(x) // 240
if not k:
continue
align = g['loud']['align']
test = (('apex' if align == 'peak' else 'end') if g['builds'] else None if align in ('start', 'end') else
'apex' if g['long'] else 'impact')
if test is None: # placed by its start / end on purpose (not a build): no hit to judge
continue
n += 1
pk = (int(np.argmax(x[:k * 240].reshape(k, 240).max(axis=1))) * 240 + 120 + g['a']) / SR
p = np.convolve((g['sig'] ** 2).sum(axis=1), np.ones(2400) / 2400, 'same')
apex = (int(np.argmax(p)) + g['a']) / SR
att = g['loud']['att'] / SR
r = {'t_cue': round(g['cue'], 3), 'label': g['label'], 'kind': g['kind'], 'test': test, 'align': align,
'attack_ms': round((att - g['cue']) * 1000), 'peak_ms': round((pk - g['cue']) * 1000),
'apex_ms': round((apex - g['cue']) * 1000), 'long': g['long']}
if test == 'end':
nz = np.nonzero(x > x.max() * 0.1)[0] # the audible body: above 10 % of the peak (-20 dB)
r['end_ms'] = round(((nz[-1] if len(nz) else 0) + g['a']) / SR * 1000 - g['cue'] * 1000)
r['issue'] = sync_issue(test, r, 0)
if not r['issue']:
continue
offs = sorted({(round(L['offset'] * 1000), L['rule']) for L in g['layers'] if abs(L['offset']) >= 0.03})
why = next(((o, rule) for o, rule in offs if not sync_issue(test, r, o)), None)
if why:
r.update({'offset_s': why[0] / 1000, 'offset_rule': why[1],
'explained': f"{re.sub(r' [(].*', '', r['issue'])} = rule {why[1]}'s offset {why[0] / 1000:+.3f} s"})
deliberate.append(r)
elif g['qa_ok']:
r['qa_ok'] = g['qa_ok']
deliberate.append(r)
else:
if offs:
r['offsets_tried_s'] = [o / 1000 for o, _ in offs]
rows.append(r)
return {'status': 'warn' if rows else 'ok', 'checked': n, 'items': rows, 'deliberate': deliberate,
'summary': f'{n} events checked, {len(rows)} off their cue'
+ (': ' + ', '.join(f"{r['t_cue']:.2f} {r['label']} {r['issue']}" for r in rows[:5]) if rows else '')
+ f'; {len(deliberate)} more sit off the cue by exactly a rule offset or carry a qa_ok note (check those on '
'the frames)'}
def check_density(ctx, groups):
per_s = {}
for g in groups:
s = int(min(L['att'] for L in g['layers']) / SR)
per_s[s] = per_s.get(s, 0) + 1
pl = [p for p in ctx.report.get('placements', []) if p.get('kind') != 'place']
use = {}
for p in ctx.report.get('placements', []):
use[p['sample']] = use.get(p['sample'], 0) + 1
by = {}
for p in pl:
by.setdefault(str(p['rule']), []).append(p)
rules, mg = [], []
for k, v in by.items():
if len(v) < 4:
continue
t = np.array(sorted(p['t'] for p in v))
ioi = np.diff(t)
g = np.array([p['gain_db'] for p in v])
pt = np.array([p.get('pitch', 0) for p in v])
r = {'rule': k, 'n': len(v), 'distinct_samples': len({p['sample'] for p in v}), 'ioi_s': round(float(ioi.mean()), 3),
'ioi_cv': round(float(ioi.std() / max(ioi.mean(), 1e-9)), 2), 'gain_sd_db': round(float(g.std()), 2),
'pitch_sd_st': round(float(pt.std()), 2)}
rules.append(r)
if r['distinct_samples'] == 1 and r['pitch_sd_st'] < 0.1 and r['gain_sd_db'] < 0.5:
mg.append(r)
n = ctx.N / SR
busiest = sorted(per_s.items(), key=lambda kv: -kv[1])[:5]
return {'status': 'warn' if mg else 'ok', 'events_per_s': [per_s.get(s, 0) for s in range(int(math.ceil(n)))],
'busiest_s': busiest, 'most_used_samples': sorted(use.items(), key=lambda kv: -kv[1])[:8], 'rules': rules,
'machine_gun': mg,
'summary': (f'{len(groups)} effect events in {n:.0f} s ({len(groups) / n * 10:.1f} per 10 s); busiest seconds '
+ ', '.join(f'{s}s:{c}' for s, c in busiest) + f'; {len(mg)} machine-gun rule(s)'
+ (': ' + ', '.join(r['rule'] for r in mg) if mg else ''))}
# ------------------------------------------------------------------ loudness / coverage
def check_loudness(ctx):
x = ctx.master
if x is None:
return {'status': 'skipped', 'summary': 'no master WAV (pass --master)'}
if len(x) < int(0.4 * SR):
return {'status': 'skipped', 'summary': f'the master is {len(x) / SR * 1000:.0f} ms long: loudness needs >= 400 ms'}
if not np.any(x):
return {'status': 'fail', 'lufs': None, 'summary': 'the master is digital silence (no gated loudness)'}
I, tp, lr = M.integrated(x), M.true_peak_db(x), M.lra(x)
tm, mom = M.blocks(x, 0.4, 0.1)
ts, st = M.blocks(x, 3.0, 1.0)
k = int(np.argmax(mom))
dur = len(x) / SR
head = db(np.abs(x[:240]).max())
tail_rms = db(np.sqrt((x[-2400:] ** 2).mean()))
tail_pk = db(np.abs(x[-480:]).max())
r = {'lufs': round(I, 2), 'target': ctx.target, 'true_peak_dbtp': round(tp, 2), 'lra_lu': round(lr, 1),
'premaster_lufs': round(ctx.pre_lufs, 2) if ctx.pre_lufs is not None else None, 'loudest_momentary': round(float(mom[k]), 1),
'loudest_at': round(float(tm[k]), 2), 'loudest_frac': round(float(tm[k]) / dur, 2),
'short_term_per_s': [round(float(v), 1) for v in st], 'head_5ms_peak_dbfs': round(head, 1),
'tail_50ms_rms_dbfs': round(tail_rms, 1), 'tail_10ms_peak_dbfs': round(tail_pk, 1)}
acts = ctx.cues.get('acts') or {}
if acts:
r['acts'] = {}
for name, v in acts.items():
t0, t1 = v['from'] / ctx.fps, (v['from'] + v['dur']) / ctx.fps
sel = (tm >= t0) & (tm < t1)
if sel.any():
r['acts'][name] = {'from': round(t0, 2), 'momentary_median': round(float(np.median(mom[sel])), 1),
'momentary_max': round(float(mom[sel].max()), 1)}
fails, warns = [], []
if abs(I - ctx.target) > 0.5:
fails.append(f'integrated {I:.1f} LUFS, target {ctx.target}')
if tp > -1.0:
fails.append(f'true peak {tp:.1f} dBTP (> -1.0: AAC will clip)')
if tail_rms > -45 or tail_pk > -60:
fails.append(f'the end is not silent (last 50 ms {tail_rms:.0f} dBFS RMS, last 10 ms peak {tail_pk:.0f})')
if head > -40:
warns.append(f'first 5 ms peak {head:.0f} dBFS: starts mid-sound or clicks on sample 0')
if lr < 4:
warns.append(f'LRA {lr:.1f} LU: flat (launch films with an arc sit at 5-9 LU)')
elif lr > 10:
warns.append(f'LRA {lr:.1f} LU: wider than launch films (5-9 LU); quiet passages may vanish on phones')
if r['loudest_frac'] > 0.85:
warns.append(f'the loudest moment is at {r["loudest_frac"]:.0%} of the runtime: peak at the drop, resolve at the end')
r['status'] = 'fail' if fails else 'warn' if warns else 'ok'
r['summary'] = (f'{I:.1f} LUFS (target {ctx.target}), true peak {tp:.1f} dBTP, LRA {lr:.1f} LU, loudest momentary '
f'{mom[k]:.1f} LUFS at {fmt_t(tm[k])} ({r["loudest_frac"]:.0%} of the runtime)'
+ ('; ' + '; '.join(fails + warns) if fails or warns else ''))
return r
def stem_id(i):
return re.sub(r'(:\d+)+$', '', str(i or ''))
def check_coverage(ctx, groups=None):
"""cue events with no sound: unsounded in the report, or placed but silent (every layer below SILENT_DB)"""
ev = ctx.cues.get('events') or []
uns = list(ctx.report.get('unsounded_events', []))
quiet = [{'t': round(g['cue'], 3), 'kind': g['kind'], 'id': g['layers'][0]['event']}
for g in groups or [] if g['silent'] and not g.get('place') and not g['qa_ok']]
uns += quiet
tot, miss = {}, {}
for e in ev:
k = (e['kind'], stem_id(e.get('id')))
tot[k] = tot.get(k, 0) + 1
for e in uns:
k = (e['kind'], stem_id(e.get('id')))
miss.setdefault(k, []).append(e['t'])
partial = [{'kind': k[0], 'id': k[1], 'unsounded': len(v), 'of': tot.get(k, len(v)), 'at': v[:12]}
for k, v in miss.items() if len(v) < tot.get(k, len(v))]
silent = [{'kind': k[0], 'id': k[1], 'n': len(v), 'at': v[:12]} for k, v in miss.items() if len(v) >= tot.get(k, len(v))]
return {'status': 'warn' if partial or quiet else 'ok', 'partial': partial, 'unsounded_classes': silent,
'placed_but_silent': quiet,
'summary': (('solo render (events of the other fragments are expected to have no sound): ' if ctx.only else '')
+ f'{len(uns)} of {len(ev)} cue events have no sound'
+ (f' ({len(quiet)} of them placed but silent: ' + ', '.join(f"{q['t']:.2f} {q['id']}" for q in quiet[:5])
+ ')' if quiet else '') + f'; {len(partial)} class(es) of action sounded '
f'only in part' + (': ' + ', '.join(f"{p['kind']} {p['id']} ({p['unsounded']}/{p['of']} silent)"
for p in partial[:5]) if partial else '')
+ (f'; classes never sounded: ' + ', '.join(f"{s['kind']} {s['id']} x{s['n']}" for s in silent[:8])
if silent else ''))}
# ------------------------------------------------------------------ earshot
def listen(ctx, a):
try:
from earshot import api
except Exception as e: # noqa: BLE001
return {'status': 'skipped', 'summary': f'earshot is not importable here ({e}): pip install -e <earshot repo>'}
out_dir = os.path.dirname(ctx.R(a.out))
os.makedirs(out_dir, exist_ok=True)
res, notes = {}, []
video = ctx.R(a.video) if a.video else None
if a.listen and ctx.master is not None:
mp = ctx.R(a.master)
cues_used = ctx.R(a.cues_sounded) if a.cues_sounded and os.path.exists(ctx.R(a.cues_sounded)) else None
tl = {}
steps = (('timeline', lambda: tl.setdefault('r', api.timeline(mp, video=video, cues=cues_used))),
('sheet', lambda: api.sheet(mp, os.path.join(out_dir, 'qa_sheet.png'), video=video, cues=cues_used,
tl=tl.get('r'))),
('critique', lambda: api.critique(mp, video=video, use_ast=False, use_realness=False)))
for name, call in steps: # the critique is the fast scorecard: realness of the sources is --realness's job
try:
r = call()
res[name] = {'summary': r.get('summary'), **({'png': r.get('png') or r.get('path')} if name == 'sheet' else {})}
if name == 'timeline' and r.get('text'):
p = os.path.join(out_dir, 'qa_timeline.txt')
open(p, 'w', encoding='utf-8').write(r['text'])
res[name]['text_file'] = p
if name == 'critique':
res[name].update({'verdict': r.get('verdict'), 'score': r.get('score'), 'actions': r.get('actions'),
'off_range': [i for i in r.get('metrics') or [] if isinstance(i, dict)
and i.get('status') in ('low', 'high')]})
except Exception as e: # noqa: BLE001
notes.append(f'{name}: {type(e).__name__}: {str(e)[:160]}')
if a.realness:
rows = []
for f in ctx.report.get('files_used', []):
try:
r = api.realness(ctx.R(f))
rows.append({'file': f, 'p_real': r.get('p_real'), 'verdict': r.get('verdict')})
except Exception as e: # noqa: BLE001
notes.append(f'realness {os.path.basename(f)}: {str(e)[:160]}')
rows.sort(key=lambda r: (r['p_real'] is None, r['p_real'] if r['p_real'] is not None else 1))
res['realness'] = {'files': rows, 'not_real': [r for r in rows if (r['p_real'] < 0.5 if r['p_real'] is not None
else 'synth' in str(r['verdict']).lower())]}
short = lambda k, v: (v['summary'][:360].rsplit(' ', 1)[0] + f" ... (all of it: {v['text_file']})"
if k == 'timeline' and len(v['summary']) > 400 and v.get('text_file') else v['summary'])
parts = [f"{k}: {short(k, v)}" for k, v in res.items() if k != 'realness' and v.get('summary')]
if 'realness' in res:
rr = res['realness']
parts.append(f"realness: {len(rr['not_real'])} of {len(rr['files'])} sources judged not real" if rr['files'] else
'realness: not available (' + (notes[0] if notes else 'no sources') + ')')
if notes:
parts.append(f'{len(notes)} earshot call(s) failed (see notes)')
bad = bool(res.get('realness', {}).get('not_real'))
return {'status': 'warn' if bad or notes else 'ok', 'summary': '; '.join(parts) or 'nothing ran', **res, 'notes': notes}
# ------------------------------------------------------------------ run
def run_qa(a):
ctx = Ctx(a)
groups, note = build_layers(ctx) if ctx.trace is not None else (None, None)
live = [g for g in groups if not g['silent']] if groups else groups
sec = {'clicks': check_clicks(ctx), 'loudness': check_loudness(ctx), 'coverage': check_coverage(ctx, groups)}
if ctx.trace is None:
msg = 'needs the trace: run mix.py with --trace out/qa/adds.json'
for k in ('edges', 'audibility', 'harshness', 'sync', 'density'):
sec[k] = {'status': 'skipped', 'summary': msg}
else:
sec['edges'] = check_edges(ctx)
if groups is None:
for k in ('audibility', 'harshness', 'sync', 'density'):
sec[k] = {'status': 'skipped', 'summary': note}
else:
sec['audibility'] = check_audibility(ctx, groups, a.layers)
sec['harshness'] = check_harshness(ctx, live)
sec['sync'] = check_sync(ctx, live)
sec['density'] = check_density(ctx, live)
if a.listen or a.realness:
sec['listen'] = listen(ctx, a)
if not any(np.any(v) for v in ctx.stems.values()):
sec['mix'] = {'status': 'fail', 'summary': 'every stem is digital silence: nothing was mixed (check the trims with '
'mix.py --dump, the gains and the cue matches)'}
order = {'fail': 0, 'warn': 1, 'ok': 2, 'skipped': 3}
worst = min((s['status'] for s in sec.values()), key=lambda s: order.get(s, 9))
bad = [k for k, s in sec.items() if s['status'] in ('fail', 'warn')]
skipped = [k for k, s in sec.items() if s['status'] == 'skipped']
summary = (f"QA {worst.upper()}: " + (', '.join(f"{k} {sec[k]['status']}" for k in bad) if bad else
'every check that ran passed' if skipped else 'every check passed')
+ (f" (not run: {', '.join(skipped)})" if skipped else '')
+ (f". Master gain over the stems {ctx.G:+.1f} dB (levels below are at master scale)." if ctx.scale == 'master'
else '. Master or premaster silent / too short: levels below are at stem scale.' if ctx.master is not None
else '. No master: levels below are at stem scale.'))
return {'summary': summary, 'status': worst, 'note': note, 'sections': sec,
'inputs': {'design': a.design, 'report': a.report, 'trace': a.trace, 'stems': a.stems, 'master': a.master,
'only': a.only}}
def digest(res):
lines = [res['summary']]
for k, s in res['sections'].items():
lines.append(f"[{s['status'].upper():7s}] {k}: {s['summary']}")
if k == 'clicks':
for c in s.get('items', [])[:6]:
o = c['owners'][0] if c['owners'] else {}
lines.append(f" {c['in']} {c['t']:.3f} ({fmt_t(c['t'])}) ch{c['ch']} conc {c['conc']}: "
f"{o.get('bus', '?')}:{o.get('what', '?')} [{o.get('rule', '')}] at its {o.get('where', '?')}")
elif k == 'edges':
for it in [i for i in s.get('items', []) + s.get('splices', []) if i['issues']][:8]:
lines.append(f" {it.get('what', 'music')} {it.get('name', it.get('segment'))}: {'; '.join(it['issues'])}")
elif k == 'audibility':
for r in s.get('items', [])[:6]:
if r['verdict'] == 'silent':
lines.append(f" {r['t']:.3f} ({fmt_t(r['t'])}) {r['label']}: SILENT, peak {r['peak_db']} dBFS "
f"[{', '.join(r['samples'])[:80]}]")
elif r['verdict'] in ('masked', 'barely'):
lines.append(f" {r['t']:.3f} ({fmt_t(r['t'])}) {r['label']}: read {r['read_db']} dB ({r['verdict']}), "
f"vs bed {r['vs_bed_db']} dB @ {r['vs_bed_hz']} Hz [{', '.join(r['rules'])[:80]}]")
elif k == 'harshness':
for r in [r for r in s.get('items', []) if r['issues']][:5]:
lines.append(f" {r['t']:.3f} ({fmt_t(r['t'])}) {r['label']}: {'; '.join(r['issues'])} [{', '.join(r['rules'])[:70]}]")
elif k == 'sync':
for r in s.get('items', [])[:6]:
lines.append(f" cue {r['t_cue']:.3f} ({fmt_t(r['t_cue'])}) {r['label']}: {r['issue']}")
for r in s.get('deliberate', [])[:6]:
lines.append(f" cue {r['t_cue']:.3f} ({fmt_t(r['t_cue'])}) {r['label']}: deliberate, "
+ (r.get('explained') or f"{r['issue']} (qa_ok: {r.get('qa_ok')})"))
return '\n'.join(lines)
def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--root', default='.', help='project root: relative paths resolve against it')
ap.add_argument('--design', default='audio/sounddesign.json')
ap.add_argument('--report', help='default out/qa/mix_report.json (--only: out/audio-review/solo/<only>_report.json)')
ap.add_argument('--trace', default='out/qa/adds.json', help='mix.py --trace output (attribution, layers)')
ap.add_argument('--stems', help='default out/audio/stems (--only: out/audio-review/solo/<only>_stems)')
ap.add_argument('--master', help='default out/audio/soundtrack.wav (--only: none); "none" skips it')
ap.add_argument('--cues', help='cue file (default: the design\'s "cues", else out/cues.json)')
ap.add_argument('--cues-sounded', default='out/qa/cues_sounded.json', help='for earshot timeline (--listen)')
ap.add_argument('--only', help='the same --only the mix was rendered with')
ap.add_argument('--cache', help='decode cache (default <root>/out/audio/cache, as mix.py)')
ap.add_argument('--video', help='the film, for earshot (--listen)')
ap.add_argument('--out', default='out/qa/qa.json')
ap.add_argument('--layers', action='store_true', help='also measure every layer of multi-layer events (dead layers)')
ap.add_argument('--listen', action='store_true', help='earshot timeline + sheet + critique of the master')
ap.add_argument('--realness', action='store_true', help='earshot realness of every source recording used')
ap.add_argument('--json', action='store_true', help='print the full result as JSON')
a = ap.parse_args(argv)
base = 'out/audio-review/solo/' + re.sub(r'[^A-Za-z0-9._-]+', '_', a.only) if a.only else None
a.report = a.report or (base + '_report.json' if base else 'out/qa/mix_report.json')
a.stems = a.stems or (base + '_stems' if base else 'out/audio/stems')
a.master_given = bool(a.master) and a.master.lower() != 'none'
a.master = None if (a.master or '').lower() == 'none' else a.master or (None if base else 'out/audio/soundtrack.wav')
out = os.path.normpath(a.out if os.path.isabs(a.out) else os.path.join(os.path.abspath(a.root), a.out))
os.makedirs(os.path.dirname(out), exist_ok=True)
try:
res = run_qa(a)
except InputError as e:
say(f'qa.py: {e}')
return 2
full = clean(res)
json.dump(full, open(out, 'w'), indent=1)
say(json.dumps(full, indent=1) if a.json else digest(res) + f'\nwrote {out}')
return 1 if res['status'] == 'fail' else 0
if __name__ == '__main__':
sys.exit(main())
scripts/variant.py
#!/usr/bin/env python
"""Apply design operations: build a variant to test (critics) or apply the agreed fix (the fixer).
python -I <skill>/scripts/variant.py --ops ops.json --out out/audio-review/critic_tonal/v1 [--render] [--qa]
[--cues out/cues.json] [--cache DIR]
python -I <skill>/scripts/variant.py --ops ops.json --apply [--backup out/audio-review/backup_round1]
[--root .] [--design audio/sounddesign.json]
A variant is a copy of the top-level design and its fragments in --out (design/<fragment>.json, include paths
rewritten), with the operations applied; --render mixes it with every output inside --out (soundtrack.wav, stems/,
report.json, cues.json, adds.json) and --qa runs qa.py on it (qa.json). --apply edits the real files instead, after
copying them to --backup (never overwritten). Operations (a JSON list, applied in order; "f" = fragment file name
without .json, "sd" = a dotted path in the top-level design):
{"f": "keyboard", "sample": "kb.thock.1", "set": {"norm_peak_db": -10}} also "slice": name
{"f": "keyboard", "sample": "kb.thock.1", "eq": {"highshelf": [5000, -6]}} merge into its eq
{"f": "motion", "rule": "mo.leap.air", "set": {"offset": 0.3}} {"..., "rule": name, "delete": true}
{"f": "motion", "rule_add": {...}} {"f": "garden", "place": "gd.crowd_a", "set": {...}} {"f": ..., "place_add": {...}}
{"f": "ambience", "amb": "am.room", "set": {"gain_db": -3}} {"f": "music", "segment": 2, "set": {...}}
{"f": "toy", "pool": "ty.p.row", "value": ["ty.row.a", "ty.row.b"]}
{"sd": "buses.music.auto", "value": [[0, 0], [12, -3]]} {"sd": "master.lufs", "value": -15}
Prints every change as old -> new (or unchanged); an operation that matches nothing is an error (nothing is
written). Only files whose content changes are written, and only the changed values are rewritten in place (hand
alignment, key order, comments and number spelling elsewhere stay as they were), so the fix reads as a small diff; the
files written are listed.
"""
import argparse, copy, difflib, json, os, re, shutil, subprocess, sys
HERE = os.path.dirname(os.path.abspath(__file__))
WS = re.compile(r'[ \t\r\n]*')
DEC = json.JSONDecoder()
# ------------------------------------------------------------------ format-preserving JSON edits
def scan(t, i, path, sp, kids):
"""parse the JSON value at t[i:], recording sp[path] = (item_start, value_start, value_end) for every value and
kids[path] = child keys / indices in order; returns the end index"""
i = WS.match(t, i).end()
item = sp.get(path, (i,))[0]
if t[i] in '{[':
close, kids[path] = '}' if t[i] == '{' else ']', []
j = WS.match(t, i + 1).end()
n = 0
while t[j] != close:
if close == '}':
key, k = json.decoder.scanstring(t, j + 1)
k = WS.match(t, k).end() + 1 # past the colon
sub = path + (key,)
else:
k, sub = j, path + (n,)
sp[sub] = (j,)
kids[path].append(sub[-1])
j = WS.match(t, scan(t, k, sub, sp, kids)).end()
if t[j] == ',':
j = WS.match(t, j + 1).end()
n += 1
sp[path] = (item, i, j + 1)
return j + 1
_, j = DEC.raw_decode(t, i)
sp[path] = (item, i, j)
return j
def dump1(v):
return json.dumps(v, ensure_ascii=False)
def edits_for(t, old, new, path, sp, kids, out):
"""text edits (start, end, replacement) that turn the value at path from old into new, touching only what changed"""
if old == new and type(old) is type(new):
return
if isinstance(old, dict) and isinstance(new, dict) and path in kids:
ks = kids[path]
m = 0
while m < len(ks):
if ks[m] in new:
edits_for(t, old[ks[m]], new[ks[m]], path + (ks[m],), sp, kids, out)
m += 1
continue
m1 = m # a run of deleted entries m..m1, removed with one separator
while m1 + 1 < len(ks) and ks[m1 + 1] not in new:
m1 += 1
at = lambda n, w: sp[path + (ks[n],)][w]
out.append((at(m, 0), at(m1 + 1, 0), '') if m1 + 1 < len(ks) else
(at(m - 1, 2), at(m1, 2), '') if m else (at(m, 0), at(m1, 2), ''))
m = m1 + 1
add = [k for k in new if k not in old]
if add:
insert(t, path, sp, kids, [f'{dump1(k)}: {dump1(new[k])}' for k in add], out)
return
if isinstance(old, list) and isinstance(new, list) and path in kids:
sm = difflib.SequenceMatcher(None, list(map(ident, old)), list(map(ident, new)), autojunk=False)
for op, a0, a1, b0, b1 in sm.get_opcodes():
if op == 'equal' or (op == 'replace' and a1 - a0 == b1 - b0): # the same items (or one for one): patch inside
for d in range(a1 - a0):
edits_for(t, old[a0 + d], new[b0 + d], path + (a0 + d,), sp, kids, out)
elif op == 'insert':
insert(t, path, sp, kids, [dump1(x) for x in new[b0:b1]], out, after=a0 - 1)
else: # delete / uneven replace: the old items' whole stretch (with one separator) -> the new items
s0 = sp[path + (a0,)][0]
e0 = sp[path + (a1 - 1,)][2]
txt = sep(t, path, sp, kids).join(dump1(x) for x in new[b0:b1])
if not txt:
if a1 < len(old):
e0 = sp[path + (a1,)][0]
elif a0:
s0 = sp[path + (a0 - 1,)][2]
out.append((s0, e0, txt))
return
_, s0, e0 = sp[path]
out.append((s0, e0, dump1(new)))
def ident(x):
"""what makes a list item the same item after an edit: a dict's name / sample / label, else its whole value"""
k = [(key, dump1(x[key])) for key in ('name', 'sample', 'label') if isinstance(x, dict) and key in x]
return json.dumps(k) if k else dump1(x)
def sep(t, path, sp, kids):
"""the separator used between the container's items (',' + its line break and indent), from its last item"""
ks = kids[path]
if len(ks) < 2:
return ', '
a, b = sp[path + (ks[-2],)][2], sp[path + (ks[-1],)][0]
return t[a:b] if t[a:b].strip() == ',' else ', '
def insert(t, path, sp, kids, items, out, after=None):
ks = kids[path]
if not ks:
_, s0, e0 = sp[path]
out.append((s0 + 1, s0 + 1, ', '.join(items)))
return
s = sep(t, path, sp, kids)
if after is None:
after = len(ks) - 1
if after < 0:
out.append((sp[path + (ks[0],)][0],) * 2 + (s.join(items) + s,))
else:
e = sp[path + (ks[after],)][2]
out.append((e, e, s + s.join(items)))
def patch_text(text, new):
"""text of a JSON file with only the values that differ from `new` rewritten; None when that is not possible"""
try:
old = json.loads(text)
sp, kids, out = {}, {}, []
scan(text, 0, (), sp, kids)
edits_for(text, old, new, (), sp, kids, out)
out.sort()
if any(out[n][1] > out[n + 1][0] for n in range(len(out) - 1)):
return None
for s0, e0, rep in reversed(out):
text = text[:s0] + rep + text[e0:]
return text if json.loads(text) == new else None
except (ValueError, KeyError, IndexError):
return None
def write_json(path, new, base_text=None):
"""write `new` to path keeping base_text's formatting where possible; returns 'patched' or 'reformatted'"""
txt = patch_text(base_text, new) if base_text is not None else None
how = 'patched' if txt is not None else 'reformatted'
if txt is None:
txt = json.dumps(new, indent=1, ensure_ascii=False) + '\n'
with open(path, 'w', encoding='utf-8', newline='') as fh:
fh.write(txt)
return how
def find(items, key, name):
for i, it in enumerate(items):
if str(it.get(key)) == str(name):
return i
raise KeyError(f'no item with {key} = {name!r}')
def change(what, old, new):
if old == new and type(old) is type(new):
return f'{what}: unchanged ({json.dumps(old)[:80]})'
return f'{what}: {json.dumps(old)[:80]} -> {json.dumps(new)[:80]}'
def apply_ops(top, frags, ops):
"""apply operations to in-memory copies; returns a list of change lines"""
log = []
for op in ops:
if 'sd' in op:
node, keys = top, op['sd'].split('.')
for k in keys[:-1]:
node = node.setdefault(k, {})
log.append(change(f"sounddesign.{op['sd']}", node.get(keys[-1]), op['value']))
node[keys[-1]] = op['value']
continue
f = op['f']
if f not in frags:
raise KeyError(f'no fragment "{f}" (have {", ".join(sorted(frags))})')
d = frags[f]
if 'sample' in op or 'slice' in op:
kind = 'samples' if 'sample' in op else 'slices'
name = op.get('sample') or op.get('slice')
s = d.get(kind, {}).get(name)
if s is None:
raise KeyError(f'{f}: no {kind[:-1]} "{name}"')
for k, v in (op.get('set') or {}).items():
log.append(change(f'{f}.{name}.{k}', s.get(k), v))
s[k] = v
if 'eq' in op:
old = copy.deepcopy(s.get('eq'))
s.setdefault('eq', {}).update(op['eq'])
log.append(change(f'{f}.{name}.eq', old, s['eq']))
elif 'rule' in op or 'place' in op or 'amb' in op:
kind, key, name = (('rules', 'name', op['rule']) if 'rule' in op else ('place', 'sample', op['place'])
if 'place' in op else ('ambience', 'name', op['amb']))
items = d.get(kind, [])
i = find(items, key, name)
if op.get('delete'):
log.append(f'{f}.{kind}[{name}]: deleted')
items.pop(i)
continue
for k, v in (op.get('set') or {}).items():
log.append(change(f'{f}.{kind}[{name}].{k}', items[i].get(k), v))
items[i][k] = v
elif 'rule_add' in op or 'place_add' in op:
kind = 'rules' if 'rule_add' in op else 'place'
d.setdefault(kind, []).append(op.get('rule_add') or op.get('place_add'))
log.append(f'{f}.{kind}: added {json.dumps(d[kind][-1])[:100]}')
elif 'segment' in op:
sg = d['music']['segments'][int(op['segment'])]
for k, v in (op.get('set') or {}).items():
log.append(change(f"{f}.music.segments[{op['segment']}].{k}", sg.get(k), v))
sg[k] = v
elif 'pool' in op:
log.append(change(f"{f}.pools.{op['pool']}", d.get('pools', {}).get(op['pool']), op['value']))
d.setdefault('pools', {})[op['pool']] = op['value']
else:
raise ValueError(f'unknown operation {op}')
return log
def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--root', default='.')
ap.add_argument('--design', default='audio/sounddesign.json')
ap.add_argument('--ops', required=True, help='JSON list of operations (or [] for an unchanged copy)')
ap.add_argument('--out', help='variant folder (critics)')
ap.add_argument('--apply', action='store_true', help='edit the real design files (the fixer)')
ap.add_argument('--backup', default='out/audio-review/backup_before_fix', help='with --apply: copy the originals here first')
ap.add_argument('--render', action='store_true', help='mix the variant (outputs inside --out)')
ap.add_argument('--qa', action='store_true', help='also run qa.py on the rendered variant')
ap.add_argument('--cache', help='decode cache for --render / --qa (default: <root>/out/audio/cache, as mix.py)')
ap.add_argument('--cues', help='cue file for the variant (written into its design; default: the design\'s "cues", '
'else out/cues.json)')
a = ap.parse_args(argv)
root = os.path.abspath(a.root)
R = lambda p: os.path.normpath(p if os.path.isabs(p) else os.path.join(root, p))
if bool(a.out) == bool(a.apply):
sys.exit('give exactly one of --out (a variant) or --apply (edit the design)')
if not os.path.exists(R(a.design)):
sys.exit(f'design file not found: {R(a.design)} (pass --design)')
text = {p: open(R(p), encoding='utf-8', newline='').read() for p in [a.design] + json.load(
open(R(a.design), encoding='utf-8')).get('include', [])}
top = json.loads(text[a.design])
paths = {os.path.splitext(os.path.basename(p))[0]: p for p in top.get('include', [])}
frags = {k: json.loads(text[p]) for k, p in paths.items()}
ops = json.load(open(R(a.ops), encoding='utf-8')) if os.path.exists(R(a.ops)) else json.loads(a.ops)
new_top, new_frags = copy.deepcopy(top), copy.deepcopy(frags)
try:
log = apply_ops(new_top, new_frags, ops)
except (KeyError, ValueError, IndexError) as e:
sys.exit(f'nothing written: {e}')
if a.apply:
bk = R(a.backup)
if os.path.exists(bk):
sys.exit(f'backup {bk} exists: pass a new --backup')
os.makedirs(os.path.join(bk, 'design'))
shutil.copy(R(a.design), os.path.join(bk, os.path.basename(a.design)))
for k, p in paths.items():
shutil.copy(R(p), os.path.join(bk, 'design', os.path.basename(p)))
todo = [(a.design, new_top, top)] + [(p, new_frags[k], frags[k]) for k, p in paths.items()]
written = [(p, write_json(R(p), new, text[p])) for p, new, old in todo if new != old]
dest = f'the design (originals in {bk})'
else:
out = R(a.out)
os.makedirs(os.path.join(out, 'design'), exist_ok=True)
new_top['include'] = [os.path.join(out, 'design', os.path.basename(p)).replace('\\', '/') for p in top['include']]
if a.cues or top.get('cues'):
new_top['cues'] = R(a.cues or top['cues']).replace('\\', '/')
dpath = os.path.join(out, 'sounddesign.json').replace('\\', '/')
written = [(os.path.join(out, 'design', os.path.basename(p)), write_json(
os.path.join(out, 'design', os.path.basename(p)), new_frags[k], text[p])) for k, p in paths.items()]
written.append((dpath, write_json(dpath, new_top, text[a.design])))
dest = dpath
n = sum(1 for line in log if ': unchanged (' not in line)
print(f'{n} change(s), {len(log) - n} already as asked -> {dest}' if n < len(log) else f'{n} change(s) -> {dest}')
for line in log:
print(' ' + line.encode('ascii', 'replace').decode())
for p, how in written:
print(f' wrote {p} ({how}' + (', only the changed values rewritten)' if how == 'patched' else
': could not patch in place, rewritten with indent 1)'))
if a.out and (a.render or a.qa):
o = lambda n: os.path.join(R(a.out), n)
cmd = [sys.executable, '-I', os.path.join(HERE, 'mix.py'), '--root', root, '--design', dest, '--out', o('soundtrack.wav'),
'--stems', o('stems'), '--report', o('report.json'), '--cues-out', o('cues.json'), '--trace', o('adds.json')]
cache = ['--cache', R(a.cache)] if a.cache else []
r = subprocess.run(cmd + cache, capture_output=True, text=True)
print(r.stdout[-1200:], r.stderr[-800:])
if r.returncode:
return r.returncode
if a.qa:
r = subprocess.run([sys.executable, '-I', os.path.join(HERE, 'qa.py'), '--root', root, '--design', dest,
'--report', o('report.json'), '--trace', o('adds.json'), '--stems', o('stems'),
'--master', o('soundtrack.wav'), '--out', o('qa.json')] + cache, capture_output=True, text=True)
print(r.stdout, r.stderr[-800:])
return 0
if __name__ == '__main__':
sys.exit(main())
SKILL.md
---
name: real-sound
description: Build premium, realistic soundtracks for code-made videos and animations from real licensed recordings - launch films, product and feature videos, UI walkthroughs, logo stings, explainers, trailers and animated shorts made with Remotion, HyperFrames, three.js, GSAP or any renderer. Use this whenever a video or animation needs sound design, SFX, foley, ambience, music, a soundtrack, a score, UI click / whoosh / riser / impact / logo-sting sounds, or a music edit to picture; whenever a synthesized or "random" sounding soundtrack must be replaced with real recordings; or when the user asks to source, license, credit, mix, master, critique or mux audio for a film. It covers cue lists from the picture, sourcing from Freesound / Mixkit / composer libraries with licence checks, a music edit at bar lines, a JSON sound design rendered by a deterministic mixer, measured QA (clicks, masking, harshness, sync, loudness) plus earshot listening, a multi-agent critique, mastering, credits and the mux.
---
# real-sound
You are the sound supervisor of a film you cannot hear. The soundtrack is built ONLY from real, professionally
recorded or produced audio (one licensed music track, real foley and SFX, real ambiences), placed on the picture's
cue frames by a deterministic mixer, measured instead of listened to, critiqued through five lenses, fixed by one
agent, mastered, credited and muxed. Code selects, trims, places, levels, pans, EQs, pitches by resampling,
time-stretches the music and masters; code never generates a sound.
The bar is a professional launch film: every sound caused by something on screen, one material palette, the music's
drop on the hero reveal, restraint everywhere else, a resolved logo, a tail into silence. Rules:
`references/sound-design.md` (and the earshot repo's `docs/launch-video-sound.md` for the bundled reference profile:
7 professional launch films).
`<skill>` below is this folder. **Python:** `{{PY}}` below (and in every prompt template) is ONE interpreter that has
numpy, scipy, soundfile and the earshot package: the earshot repo's venv
(`<earshot repo>/.venv/Scripts/python.exe` on Windows, `<earshot repo>/.venv/bin/python` elsewhere), or
any Python where `<python> -c "import earshot"` works. Bare `python` on Windows is often the Microsoft Store stub and
fails. Substitute the full path for `{{PY}}` when you run a command or hand a prompt to a sub-agent (the other
placeholders are listed in `references/process.md`). Run the scripts with `-I` (isolated: downloaded files can never
shadow a module); every script except `scripts/licences.py` (the licence policy module that audit.py and credits.py
import) prints `--help`.
## The process
| step | who | output | gate |
|---|---|---|---|
| 0. cue list | lead | `out/cues.json` | sync frames computed from the picture code |
| 1. brief | lead | `audio/SOUND_BRIEF.md` | every cue has a sound need (or "nothing") |
| 2. sourcing (per category, parallel) | sourcing agents | `audio/assets/<cat>/`, manifests | each file measured + earshot `realness` "real or pro-produced" on the trim used (an "uncertain" only with its evidence and a trim-level re-check written in the manifest) |
| 3. licence audit | auditor | `audit.json` | `audit.py` exit 0 for what the film uses |
| 4. music edit | music supervisor | `audio/design/music.json` | grid verified, drop on the reveal, splices clean |
| 5. design fragments (per category, parallel) | designers | `audio/design/<cat>.json` | solo render + qa clean |
| 6. integrate | lead | soundtrack, stems, report, trace | master gate (+-0.5 LU, <= -1.5 dBTP) |
| 7. QA + listening | lead | `out/qa/qa.json`, earshot timeline / sheet / critique | no qa fail |
| 8. critique (5 lenses, parallel, read-only) | critics | ranked issues + tested operations | |
| 9. fix | ONE fixer | the design | qa: no regression |
| 10. master, credits, mux | lead | `CREDITS.md`, description text, the film | credits exit 0, lag 0 |
The detail of every step, the roles and what each critic measures: `references/process.md`. Every sub-agent prompt
is a ready-to-fill brief in `templates/prompts/`.
### 0. Cue list from the picture
Every sound sits on a frame the picture code computes: a contact frame, a pop-in's 50 % frame, a move's velocity
peak, a riser's hit, the frame the wordmark becomes readable. Cinetic films already have `out/cues.json`
(`export-cues.ts`); for anything else write the same format (`templates/cues.json`) from the timeline constants.
Stable ids (`act:thing:n`) let rules match classes of action with a regex.
### 1. Brief
Copy `templates/SOUND_BRIEF.md` to `audio/SOUND_BRIEF.md`: concept (palette, the music's arc, places), the picture map
(every cue: what is on screen, the sound need), ambience zones, allowed licences, quality bar, file layout. It is the
contract every sub-agent gets verbatim. Pick the brand's personality row (`references/sound-design.md` §3).
### 2. Sourcing, per category (`templates/prompts/sourcing.md`)
Typical launch-film categories: `ui` (clicks, taps, toggles, pops), `keyboard`, `motion` (whooshes, risers),
`impacts` (reveal hits, the logo), `ambience`, `music`; a character film adds `foley`, its worlds, its objects.
Freesound (CC0 / CC BY; HQ preview `cdn.freesound.org/previews/<id//1000>/<id>_<user>-hq.ogg`), Mixkit (WAV
`assets.mixkit.co/active_storage/sfx/<id>/<id>.wav`), composer libraries for music; licences read on each item's own
page (`references/licences.md`); polite sequential downloads; every file measured:
```
{{PY}} -I <skill>/scripts/analyze.py audio/assets/ui/* --png out/audio-review/ui --out out/audio-review/ui.json
{{PY}} -m earshot realness audio/assets/ui/ # gate: "real or pro-produced" (re-check trims: --start --end)
{{PY}} -m earshot rank "a single crisp trackpad click" audio/assets/ui/ --top 5
{{PY}} -m earshot match <file> "a soft cork pop" --alts "a gunshot" "a balloon burst"
{{PY}} -m earshot find <long take> "a single mechanical key press" --top 8 # trims; skip any marked head cut / tail cut
```
### 3. Licence audit (`templates/prompts/licence_audit.md`)
The auditor records what each page says (`licence_seen`, `licence_confirmed`) in
`audio/assets/manifests/audit.json`; the script checks the rest (classification, ids vs URLs, files decode, durations,
credit lines):
```
{{PY}} -I <skill>/scripts/audit.py --root . --report out/qa/mix_report.json [--fetch]
```
Allowed: CC0, public domain, CC BY 2.0-4.0, Mixkit free licences, Pixabay. Forbidden: NC, ND, SA, Sampling+, personal,
custom, disputed. Unverified = blocked.
### 4. Music (`templates/prompts/music_supervisor.md`)
One licensed track whose structure fits the film. Write the arc per film bar (`templates/arc.json`), then:
```
{{PY}} -I <skill>/scripts/arrange.py audio/assets/music/<track> --film-bpm 120 --arc audio/arc.json --out out/audio-review/music/arrange.json
```
It fits the tempo, moves the grid onto the measured attacks (beat trackers lag 20-50 ms), proposes the downbeat
phase, anchors the drop film bar on the strongest drop candidate and drafts segments at bar lines (a ready-to-paste
`music` block). The downbeat is the weak spot: when it prints LOW confidence (on one of the two tracks it was tested on
it was half a bar off), decide between the two printed grids from the first note, the section entrances and the chord
changes in the bar table, and re-run with `--phase N`. Then edit by judgement: harmony continues across each splice,
the drop's downbeat on the hero frame, the final chord on the logo, `tail_fade` into the end.
### 5. Design fragments, per category (`templates/prompts/designer.md`)
`audio/design/<category>.json` with samples (trims), slices (cut a take of many hits into single hits), pools, rules
on cue events, free placements, ambience beds, the music block. Reference with every key and real examples:
`references/design-format.md`; a complete launch-video example: `templates/design_fragment.json` (its file names are
placeholders such as `mk-0000_mouse-click.wav`: it renders once each one is replaced by a real recording from your
manifests; mix.py stops and lists the missing files until then).
Verify alone:
```
{{PY}} -I <skill>/scripts/mix.py --only ui --dump out/audio-review/ui/dump # every trim as WAV + numbers
{{PY}} -I <skill>/scripts/mix.py --only ui --trace out/audio-review/ui/adds.json --out out/audio-review/ui/solo.wav
{{PY}} -I <skill>/scripts/qa.py --only ui --report out/audio-review/ui/solo_report.json --stems out/audio-review/ui/solo_stems --trace out/audio-review/ui/adds.json --out out/audio-review/ui/qa.json
{{PY}} -m earshot timeline out/audio-review/ui/solo.wav --cues out/cues.json
```
### 6-7. Integrate, QA, listen
```
{{PY}} -I <skill>/scripts/mix.py --root . --design audio/sounddesign.json --cues out/cues.json --trace out/qa/adds.json
{{PY}} -I <skill>/scripts/qa.py --layers # clicks, edges, audibility, harshness, loudness, coverage, sync, density
{{PY}} -I <skill>/scripts/qa.py --listen --realness --video out/film.mp4 # + earshot timeline, sheet, critique, realness
{{PY}} -m earshot critique out/audio/soundtrack.wav --video out/film.mp4 # scorecard vs professional launch films
{{PY}} -m earshot sheet out/audio/soundtrack.wav -o out/qa/sheet.png --video out/film.mp4 --cues out/qa/cues_sounded.json
```
`qa.py` ties every finding to the rule and sample that caused it (the trace). It is usable without the models;
`--listen` adds earshot's transcript (`out/qa/qa_timeline.txt`), annotated sheet and scorecard.
### 8-9. Critique and fix
Five read-only critics in parallel, each with its prompt: `critic_sync.md`, `critic_edges.md`, `critic_masking.md`,
`critic_tonal.md`, `critic_director.md`. Each tests its fixes as a variant in its own folder:
```
{{PY}} -I <skill>/scripts/variant.py --ops out/audio-review/critic_tonal/ops_v1.json --out out/audio-review/critic_tonal/v1 --render --qa
```
and returns ranked issues with exact operations and before -> after numbers. Then ONE fixer (`fixer.md`) merges the
operations, resolves conflicts, checks the merged variant against the previous qa.json, and applies it
(`variant.py --apply`, originals backed up). Acknowledge confirmed deliberate exceptions with `"qa_ok": "<reason>"`.
At least one round; a second when levels or the music edit changed.
### 10. Master, credits, mux
`mix.py` masters with `master.py` (-14 LUFS, true-peak limiter, ceiling 0.77 linear; gate +-0.5 LU and <= -1.5 dBTP).
```
{{PY}} -I <skill>/scripts/audit.py --root . --report out/qa/mix_report.json # exit 0
{{PY}} -I <skill>/scripts/credits.py --root . --design audio/sounddesign.json --audit audio/assets/manifests/audit.json
node <skill>/scripts/mux.mjs --video out/film.mp4 --audio out/audio/soundtrack.wav --out out/film_sound.mp4
{{PY}} -m earshot timeline out/film_sound.mp4 --cues out/qa/cues_sounded.json # the finished file, heard
```
`CREDITS.md` + `out/CREDITS_description.txt` (paste into the video description: CC BY credits are required there).
`mux.mjs` stream-copies the picture and cuts the AAC audio to the picture's duration.
## Launch videos: the sound map
| picture | sound | sits on |
|---|---|---|
| cursor moves, hovers, scrolls, progress, streaming text | nothing (the music carries it) | |
| click / tap | ONE real click voice for the film, press + release ~85 ms later, same level every time | the press frame |
| typing | one real keyboard, sliced; <= ~12 keys/s, one per word above ~25 chars/s; music dips 6-10 dB | each sounded character |
| a card / chip / badge appears | a soft real pop or tick, climbing across a run | 50 % of its travel |
| toggle, lock-in, a counter's final value | a real latch / click / crisp tick | the contact / lock frame |
| camera-scale move, whip, zoom (~2 per film) | a real swish, apex on the velocity peak | velocity peak |
| hard cuts | nothing | |
| tension into the reveal | the music's build, or a real riser ending 30-250 ms before the hit | the hit |
| hero reveal | the music's DROP (edit the track so its downbeat lands here) + optionally one real impact | the frame |
| logo / lockup | music stops on a bar line or filters down, 0.2-0.7 s near-silence, ONE soft note in key, tail to silence | the frame the wordmark reads |
Levels: -14 LUFS, true peak <= -1 dBTP after AAC, LRA 5-9 LU, the loudest moment on the drop (30-60 % in), the logo
8-20 dB under it, about 5 designed effects per 10 s, effects dry and centred. Measured norms and the cheap tells:
`references/sound-design.md`.
## Tools
| need | tool |
|---|---|
| measure recordings (format, floor, bandwidth, onsets, spikes, tempo / key) + spectrogram PNGs | `scripts/analyze.py` |
| hear recordings: real vs synthesized, what it sounds like, best match, best take inside a long file | `{{PY}} -m earshot realness / describe / match / rank / find / compare / ab` |
| levels / brightness / onsets of one file or a stretch of it (`--start --end`) | `{{PY}} -m earshot measure` (Python: `from earshot import api; api.measure(...)`, likewise `api.find`, `api.realness` ...) |
| licence + file audit | `scripts/audit.py` (policy in `scripts/licences.py`) |
| music grid, downbeats, bar-line edit draft | `scripts/arrange.py` |
| render the design (+ solo `--only`, `--dump` trims, `--trace` for QA) | `scripts/mix.py` |
| master and loudness measurement | `scripts/master.py` (`--measure`) |
| objective QA tied to placements | `scripts/qa.py` (`--layers`, `--listen`, `--realness`) |
| critic variants, the fixer's apply | `scripts/variant.py` |
| credits for CREDITS.md and the video description | `scripts/credits.py` |
| put the soundtrack on the picture | `scripts/mux.mjs` |
| listen to a whole mix / film; annotated picture; launch-film scorecard | `{{PY}} -m earshot timeline / sheet / critique` |
## Files
```
audio/SOUND_BRIEF.md the contract (templates/SOUND_BRIEF.md)
audio/assets/<category>/ downloaded recordings (<source>-<id>_<slug>.<ext>; never commit or publish them)
audio/assets/manifests/<cat>.json one entry per recording (templates/manifest_entry.json)
audio/assets/manifests/audit.json licence audit (templates/audit_entry.json)
audio/sounddesign.json top level: includes, buses, master (templates/sounddesign.json)
audio/design/<category>.json one fragment per designer (templates/design_fragment.json)
out/cues.json the picture's cue events (templates/cues.json)
out/audio/soundtrack.wav, stems/ the master and its stems
out/qa/ mix_report.json, adds.json, cues_sounded.json, master.json, qa.json
out/audio-review/ per-category and per-critic work, variants, dumps (the audit trail)
CREDITS.md, out/CREDITS_description.txt
```
## References
- `references/process.md`: roles, steps, what each critic measures, thresholds and their sources, variants, rounds.
- `references/design-format.md`: every design key, defaults, processing order, outputs, real examples.
- `references/sound-design.md`: launch-video and film sound rules, levels, sync, variation, tuning, selection, cheap tells.
- `references/licences.md`: allowed / forbidden, verification per site, attribution, Content ID.
- `references/troubleshooting.md`: errors and fixes.
- Cinetic films: `docs/INTEGRATION.md` in the earshot repo (how the cue file, timeline and review loop connect).
templates/arc.json
{
"_doc": "Input for arrange.py --arc: the target loudness of each FILM bar in dB relative to the track's loud bars (0 = its biggest sections, -6 = a groove, -10 = a breakdown, -14 = a soft intro). One number per film bar (bar = beats_per_bar beats at film_bpm). Write it from the picture map in SOUND_BRIEF.md: soft under the cold open and typing, the groove for the feature beats, a breakdown before the turn, the build, the big section from the drop (the hero reveal), a resolve under the logo. This example: a 30 s launch film at 120 BPM = 15 bars, drop on film bar 9 (16.0 s), logo from bar 13.",
"film_bpm": 120,
"beats_per_bar": 4,
"drop_film_bar": 9,
"arc": [-14, -12, -7, -6, -6, -6, -10, -6, 0, 0, 0, -2, -8, -11, -14]
}
templates/audit_entry.json
{
"_doc": "One record of audio/assets/manifests/audit.json. The licence auditor (prompts/licence_audit.md) writes ONLY the three agent fields below after reading the licence on the item's own page; audit.py computes everything else on every run and keeps the agent fields (matched by id + file). Never set licence_confirmed without having read the page in this session.",
"id": "fs-000000",
"licence_seen": "Freesound page https://freesound.org/people/uploader_name/sounds/000000/: licence 'Attribution 4.0' -> https://creativecommons.org/licenses/by/4.0/ (read 2026-10-14); page author 'uploader_name', title 'Trackpad click close.wav'; preview id/user match download_url",
"licence_confirmed": true,
"agent_notes": ["the author asks for credit as \"<title> by uploader_name (freesound.org)\": the manifest attribution follows it"],
"_computed_by_audit_py": {
"manifest": "audio/assets/manifests/ui.json", "need": "ui_click", "rank": 1, "in_scope": true, "severity": "ok",
"licence_in_manifest": "CC BY 4.0", "licence": "by4", "licence_status": "allowed", "attribution_required": true,
"attribution_ok": true, "attribution_in_manifest": "\"Trackpad click close.wav\" by uploader_name (...), licensed under CC BY 4.0 (...)",
"file_ok": true,
"file_check": {"path": "audio/assets/ui/fs-000000_trackpad-click.ogg", "exists": true, "codec": "vorbis", "sample_rate": 48000,
"channels": 1, "kbps": 192, "duration_s": 0.84, "decodes_clean": true, "bytes_match_download_url": true},
"problems": [], "warnings": [], "audit_tool": "real-sound audit.py v1"
}
}
templates/cues.json
{
"_doc": "The picture's cue file: the contract between picture and sound (Cinetic's scripts/export-cues.ts writes this exact format from the picture code; write it from the timeline for any other engine). f = absolute frame the sound belongs on (fractional is fine); kind = what happens (tick, tock, key, click, pop, snap, land, hit, drop, whoosh, riser, swell, suck, bell, or your own word); id = a stable label rules match with a regex ('act:thing:n'); weight 0..2 (1 = normal), pan -1..1 (the object's screen x), variant (key: word/space/return/back; land: light; click: confirm), dur in frames for whoosh / riser / swell. Optional: acts {name: {from, dur}} and cue {name: frame} (qa.py reports loudness per act). mix.py also accepts a list whose first element is this object. Sync points belong where the eye reads them: an impact on its contact frame, a pop-in at 50 % of its travel, a whoosh on the move's velocity peak, a riser on the frame of the hit it leads into, a logo sound on the frame the wordmark becomes readable.",
"fps": 60,
"bpm": 120,
"total": 720,
"acts": {"ui": {"from": 0, "dur": 240}, "cards": {"from": 240, "dur": 240}, "reveal": {"from": 480, "dur": 240}},
"cue": {"click": 60, "typeStart": 120, "transition": 240, "reveal": 480, "lockup": 600},
"events": [
{"f": 60, "kind": "click", "weight": 1, "pan": 0.2, "id": "demo:cursor:click"},
{"f": 120, "kind": "key", "weight": 0.6, "id": "demo:type:key:0", "variant": "word"},
{"f": 125, "kind": "key", "weight": 0.5, "id": "demo:type:key:1"},
{"f": 130, "kind": "key", "weight": 0.5, "id": "demo:type:key:2"},
{"f": 135, "kind": "key", "weight": 0.5, "id": "demo:type:key:3"},
{"f": 141, "kind": "key", "weight": 0.5, "id": "demo:type:space", "variant": "space"},
{"f": 146, "kind": "key", "weight": 0.6, "id": "demo:type:key:4", "variant": "word"},
{"f": 151, "kind": "key", "weight": 0.5, "id": "demo:type:key:5"},
{"f": 156, "kind": "key", "weight": 0.5, "id": "demo:type:key:6"},
{"f": 162, "kind": "key", "weight": 0.5, "id": "demo:type:key:7"},
{"f": 168, "kind": "key", "weight": 0.5, "id": "demo:type:key:8"},
{"f": 210, "kind": "click", "weight": 1, "pan": 0.35, "id": "demo:cursor:submit", "variant": "confirm"},
{"f": 240, "kind": "whoosh", "weight": 1, "pan": -0.6, "dur": 24, "apexFrac": 0.5, "id": "demo:transition"},
{"f": 330, "kind": "pop", "weight": 0.8, "pan": -0.4, "id": "demo:card:0"},
{"f": 345, "kind": "pop", "weight": 0.8, "pan": 0.0, "id": "demo:card:1"},
{"f": 360, "kind": "pop", "weight": 0.8, "pan": 0.4, "id": "demo:card:2"},
{"f": 480, "kind": "riser", "weight": 1, "dur": 120, "id": "demo:build"},
{"f": 480, "kind": "hit", "weight": 1.5, "id": "demo:reveal"},
{"f": 600, "kind": "bell", "weight": 0.7, "id": "demo:lockup"}
]
}
templates/design_fragment.json
{
"_doc": "A complete design showing every feature mix.py understands, for a 12 s launch-video beat (cursor click, typing, whoosh into a transition, cards popping in, riser -> hero reveal hit, logo sting). It works as a single-file design (mix.py --design this.json) or as one category fragment listed in sounddesign.json 'include' (then 'duration', 'buses' and 'master' are read only from the top-level file). Keys starting with _ are comments. Every 'file' is a real licensed recording listed in a manifest (audio/assets/manifests/*.json). Prefix every name with the fragment's category so fragments never collide. Reference: references/design-format.md.",
"duration": 12.0,
"cues": "out/cues.json",
"samples": {
"ui.press": {"_doc": "the press half of a real mouse click: trim 3-5 ms before the attack, end before the release", "file": "audio/assets/ui/mk-0000_mouse-click.wav", "start": 0.096, "end": 0.150, "mono": true, "eq": {"hp": 400}, "norm_peak_db": -12, "fade_in": 0.002, "fade_out": 0.008},
"ui.release": {"file": "audio/assets/ui/mk-0000_mouse-click.wav", "start": 0.180, "end": 0.230, "mono": true, "eq": {"hp": 400}, "norm_peak_db": -15, "fade_in": 0.002, "fade_out": 0.008},
"kb.space": {"file": "audio/assets/keyboard/fs-000001_mech-spacebar.ogg", "start": 0.48, "end": 0.78, "mono": true, "eq": {"hp": 90, "highshelf": [5000, -3]}, "norm_peak_db": -10, "fade_out": 0.08},
"mo.whoosh": {"_doc": "a real swish (stick / cloth / bamboo past the mic); placed by its peak on the move's velocity peak", "file": "audio/assets/motion/fs-000002_stick-swish.ogg", "start": 0.0, "end": 0.55, "eq": {"hp": 150, "lp": 12000}, "norm_peak_db": -6, "fade_in": 0.01, "fade_out": 0.12},
"mo.whoosh.tail": {"file": "audio/assets/motion/fs-000002_stick-swish.ogg", "start": 0.05, "end": 0.55, "pitch": -3, "eq": {"hp": 120, "lp": 6000}, "norm_peak_db": -8, "fade_in": 0.02, "fade_out": 0.2},
"fx.pop": {"file": "audio/assets/ui/fs-000003_cork-pop.ogg", "start": 0.02, "end": 0.20, "mono": true, "eq": {"hp": 250, "peq": [[3200, 1.2, -2]]}, "norm_peak_db": -10, "fade_out": 0.04},
"fx.riser": {"_doc": "a real cymbal roll / orchestral swell; aligned by its END so it stops on the reveal", "file": "audio/assets/motion/fs-000004_cymbal-roll.ogg", "start": 0.3, "end": 2.3, "eq": {"hp": 200}, "norm_peak_db": -9, "fade_in": 0.6, "fade_out": 0.03, "env": [[0, -12], [1.5, -2], [2.0, 0]]},
"fx.hit": {"file": "audio/assets/impacts/mk-0005_deep-drum-impact.wav", "start": 0.0, "end": 1.6, "eq": {"hp": 30, "peq": [[300, 1.0, -3]]}, "norm_peak_db": -4, "fade_out": 0.4},
"fx.hit.top": {"file": "audio/assets/impacts/fs-000006_wood-slam.ogg", "start": 0.01, "end": 0.35, "mono": true, "eq": {"hp": 500}, "norm_peak_db": -10, "fade_out": 0.08},
"logo.chime": {"_doc": "the sting: one soft real bell / celesta note, in the music's key (resample with pitch to tune, within +-2 st)", "file": "audio/assets/magic/fs-000007_celesta-E5.ogg", "start": 0.0, "end": 2.4, "pitch": 0.0, "norm_peak_db": -9, "fade_out": 0.9},
"logo.air": {"file": "audio/assets/motion/fs-000002_stick-swish.ogg", "start": 0.0, "end": 0.5, "glide": [2, -1], "eq": {"hp": 300, "lp": 9000}, "norm_peak_db": -14, "fade_in": 0.05, "fade_out": 0.25}
},
"slices": {
"kb.type": {"_doc": "one recording of a mechanical keyboard typing, cut at its onsets into single keys: kb.type#0, kb.type#1 ... (check them with --dump)", "file": "audio/assets/keyboard/fs-000008_mech-typing.ogg", "start": 0.5, "end": 9.0, "pre": 0.004, "len": 0.14, "min_gap": 0.07, "gate_db": -15, "mono": true, "eq": {"hp": 110, "highshelf": [5000, -4]}, "norm_peak_db": -12, "fade_out": 0.03, "max": 16}
},
"pools": {
"ui.press": ["ui.press"],
"ui.release": ["ui.release"],
"kb.type": ["kb.type#*"],
"kb.space": ["kb.space"],
"mo.whoosh": ["mo.whoosh"],
"mo.whoosh.tail": ["mo.whoosh.tail"],
"fx.pop": ["fx.pop"],
"fx.riser": ["fx.riser"],
"fx.hit": ["fx.hit"],
"fx.hit.top": ["fx.hit.top"],
"logo.chime": ["logo.chime"]
},
"rules": [
{"name": "ui.click press", "match": {"kind": "click"}, "pool": "ui.press", "mode": "fixed", "gain_db": 0, "pan": 0, "align": "attack", "_doc": "one voice at one level for every click in the film"},
{"name": "ui.click release", "match": {"kind": "click"}, "pool": "ui.release", "mode": "fixed", "gain_db": -2, "pan": 0, "align": "attack", "offset": 0.085},
{"name": "kb.type", "match": {"kind": "key", "id": "^demo:type:key"}, "pool": "kb.type", "mode": "cycle", "gain_db": -2, "jitter_db": 1.5, "pitch": [-0.3, 0.3], "pan": 0, "align": "attack"},
{"name": "kb.space", "match": {"kind": "key", "id": "^demo:type:space"}, "pool": "kb.space", "mode": "fixed", "gain_db": -1, "pan": 0, "align": "attack"},
{"name": "mo.whoosh", "match": {"kind": "whoosh"}, "pool": "mo.whoosh", "mode": "fixed", "align": "peak", "gain_db": -6, "pan": "cue", "pan_scale": 0.8},
{"name": "mo.whoosh tail", "match": {"kind": "whoosh"}, "pool": "mo.whoosh.tail", "mode": "fixed", "align": "peak", "offset": 0.06, "gain_db": -12, "pan": "cue", "pan_scale": 0.5},
{"name": "fx.pops", "match": {"kind": "pop", "from": 5.0, "to": 7.0}, "pool": "fx.pop", "mode": "cycle", "gain_db": -4, "jitter_db": 1.0, "pitch": [-0.2, 0.2], "pitch_ramp": [0, 2], "gain_ramp": [-1, 1], "pan": "cue", "pan_scale": 0.3, "align": "attack"},
{"name": "fx.riser", "match": {"kind": "riser"}, "pool": "fx.riser", "mode": "fixed", "align": "end", "offset": -0.03, "gain_db": -6, "pan": 0, "_doc": "ends 30 ms before the reveal: a beat of air before the hit"},
{"name": "fx.hit", "match": {"kind": "hit", "id": "^demo:reveal$"}, "pool": "fx.hit", "mode": "fixed", "gain_db": -2, "pan": 0, "align": "attack"},
{"name": "fx.hit top", "match": {"kind": "hit", "id": "^demo:reveal$"}, "pool": "fx.hit.top", "mode": "fixed", "gain_db": -8, "pan": 0, "align": "attack", "offset": 0.004},
{"name": "logo.chime", "match": {"kind": "bell", "id": "^demo:lockup$"}, "pool": "logo.chime", "mode": "fixed", "gain_db": -3, "pan": 0, "align": "attack", "_doc": "on the frame the wordmark becomes readable (its settle), never later"}
],
"place": [
{"sample": "logo.air", "at": 9.98, "align": "peak", "gain_db": -6, "pan": 0.0, "_doc": "a free placement (no cue event): a soft air lift that peaks just before the chime"}
],
"ambience": [
{"name": "am.room", "_doc": "a quiet real room tone under the UI part; src_start..src_end is a clean window (no events), looped with a 1.5 s crossfade if needed", "file": "audio/assets/ambience/fs-000009_quiet-office-roomtone.ogg", "src_start": 5.0, "src_end": 45.0, "at": 0.0, "until": 8.2, "fade_in": 0.3, "fade_out": 0.4, "gain_db": -6, "width": 0.8, "eq": {"hp": 80, "lp": 12000}, "auto": [[0, 0], [7.5, 0], [8.0, -8]]}
],
"music": {
"_doc": "one licensed track, edited at bar lines (arrange.py drafts the segments). src_* are ORIGINAL source seconds; 'at' is film time. 'at' can be nudged a few ms so the measured downbeat transient lands on the picture's downbeat.",
"file": "audio/assets/music/sb-0000_bright-launch-track.mp3",
"src_bpm": 120,
"target_bpm": 120,
"stretch": "none",
"gain_db": -4,
"eq": {"hp": 30},
"segments": [
{"label": "intro bars 1-4 (soft, no drums) under the UI", "src_from": 0.5, "src_to": 8.5, "at": 0.0, "fade_in": 0.15, "fade_out": 0.03},
{"label": "drop bars 17-18 on the hero reveal, ring out under the logo", "src_from": 32.5, "src_to": 36.5, "at": 8.0, "fade_in": 0.008, "fade_out": 0.06, "tail_fade": 1.2}
]
},
"_buses_doc": "buses and master are read from the top-level design only. auto = [[film t, dB], ...] piecewise-linear fader rides: here the music dips 6 dB under the typing and comes back for the whoosh",
"buses": {
"music": {"gain_db": 0.0, "auto": [[0, 0], [1.9, 0], [2.0, -6], [3.6, -6], [3.8, 0], [12, 0]]},
"amb": {"gain_db": 0.0},
"sfx": {"gain_db": 0.0, "eq": {"hp": 40}}
},
"master": {"_doc": "read from the top-level design only. -14 LUFS for web/social; ceiling 0.77 linear (-2.3 dBFS): measure about -1 dBTP after AAC", "lufs": -14, "ceiling": 0.77, "no_glue": true}
}
templates/manifest_entry.json
[
{
"_doc": "One object per downloaded recording, in audio/assets/manifests/<category>.json (a JSON list). Values here are placeholders: copy the shape, never the data. Every field is read from the item's own page; never guess a licence. 'attribution' is required for CC BY (paste the form the author asks for on their page, else: \"<title>\" by <author> (<page_url>), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)); null for CC0 / Mixkit / Pixabay. 'analysis' is what analyze.py measured; 'best_segment' the clean part in seconds.",
"id": "fs-000000",
"category": "ui",
"file": "audio/assets/ui/fs-000000_trackpad-click.ogg",
"title": "Trackpad click close.wav",
"author": "uploader_name",
"source": "freesound",
"page_url": "https://freesound.org/people/uploader_name/sounds/000000/",
"download_url": "https://cdn.freesound.org/previews/0/000000_0000000-hq.ogg",
"license": "CC BY 4.0",
"license_url": "https://creativecommons.org/licenses/by/4.0/",
"attribution": "\"Trackpad click close.wav\" by uploader_name (https://freesound.org/people/uploader_name/sounds/000000/), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
"need": "ui_click",
"rank": 1,
"best_segment": [0.10, 0.32],
"notes": "What it is and how it was recorded (from the page), what you measured, what to use and avoid: e.g. 'close-miked laptop trackpad, press at 0.12 + release at 0.21 (85 ms later, 3 dB quieter), floor -78 dBFS, no room; use as the film's one click voice'.",
"analysis": {"duration": 0.84, "codec": "vorbis", "kbps": 192, "src_rate": 48000, "channels": 1, "peak_dbfs": -9.6,
"lufs": -31.2, "noise_floor_dbfs": -78.0, "clipped_samples": 0, "attack_at": 0.118, "centroid_hz": 3300,
"rolloff95_hz": 11800, "onsets": [0.118, 0.205]},
"earshot": {"realness": 0.97, "match": {"query": "a single crisp trackpad click", "z": 3.4, "verdict": "strong"}}
},
{
"id": "mk-0000",
"category": "impacts",
"file": "audio/assets/impacts/mk-0000_deep-drum-impact.wav",
"title": "Deep drum impact",
"author": "Mixkit",
"source": "mixkit",
"page_url": "https://mixkit.co/free-sound-effects/impact/",
"download_url": "https://assets.mixkit.co/active_storage/sfx/0000/0000.wav",
"license": "Mixkit Sound Effects Free License",
"license_url": "https://mixkit.co/license/#sfxFree",
"attribution": null,
"need": "hero_reveal_hit",
"rank": 2,
"best_segment": [0.0, 1.6],
"notes": "Mixkit has no per-item page for sound effects: record the category page that lists the item and the item's title there.",
"analysis": {"duration": 2.1, "codec": "pcm_s16le", "src_rate": 44100, "channels": 2, "peak_dbfs": -1.2}
}
]
templates/prompts/critic_director.md
# Critic lens: director (does the soundtrack tell the film's story?)
You are one of five read-only critics of {{FILM}}'s soundtrack, and the one who judges it as a whole. Your lens:
**the arc and the story**. Would a viewer feel the hook, the turn, the reveal and the resolution with their eyes
closed? Is the loudest moment the hero reveal, is the logo resolved rather than loud, does the ending subtract into
silence? Is anything cheap, generic, empty or "random"?
## Context
- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Brief: `audio/SOUND_BRIEF.md` (concept, picture map,
reference bar: {{BAR_REFERENCE}}). Render `{{RENDER}}`, film `{{VIDEO}}`, qa `{{QA_JSON}}`. Round {{ROUND}};
changed: {{CHANGED}}; fixed: {{FIXED}}.
- You write ONLY in `out/audio-review/critic_director/`.
## Measure
1. `{{PY}} -m earshot timeline {{VIDEO}} --cues {{CUES}}` (the listening transcript: events, sections, loudness, sync)
and `{{PY}} -m earshot sheet {{RENDER}}/soundtrack.wav -o out/audio-review/critic_director/sheet.png --video {{VIDEO}} --cues {{CUES}}`
(an annotated picture: frames, sections, tagged events, spectrogram, loudness, cue sync; open it).
2. `{{PY}} -m earshot critique {{RENDER}}/soundtrack.wav --video {{VIDEO}}`: the scorecard against the profile of
professional launch films (each metric, its range, what to do).
3. qa.py `loudness`: integrated, LRA, loudest momentary and its position, per-act medians, head and tail; the
short-term curve against the picture map's beats (write a small plot in your folder if it helps: you can read PNGs).
4. Walk the picture map beat by beat: for each, what the transcript hears there, the sfx and music levels, whether the
sound makes the picture feel better or is just there.
5. Name the weakest 3 seconds and why.
## Test your fixes
`{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_director/ops_v1.json --out out/audio-review/critic_director/v1 --render --qa`
(bus rides, the music edit, which events are sounded, the ending). Report the arc numbers before -> after.
## Rules
- At most 12 issues, ranked by impact. Taste is allowed, but every claim needs a measurement behind it.
- Do not re-report fixed issues; do not propose synthesized sounds; do not propose on-screen text.
## Return (JSON only)
`{"lens": "director", "weakest_3s": {"from", "to", "why"}, "issues": [{"id", "priority", "t", "problem",
"evidence", "ops": [...], "verified"}], "overall": "...", "score": 1-10}`
templates/prompts/critic_edges.md
# Critic lens: edges (clicks, cuts, truncations, splices)
You are one of five read-only critics of {{FILM}}'s soundtrack. Your lens: **anything cut, clicked or chopped**. A
click at a trim, a sample that starts inside a sound, a decay cut short, a music splice that chops a note, a bed that
starts or stops abruptly: these are the defects that make a real-recordings soundtrack sound amateur.
## Context
- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Render `{{RENDER}}` (report `{{REPORT}}`, trace
`{{TRACE}}`, stems, master), qa `{{QA_JSON}}`. Round {{ROUND}}; changed: {{CHANGED}}; already fixed: {{FIXED}}.
- You write ONLY in `out/audio-review/critic_edges/`.
## Measure
1. qa.py `clicks`: every > 14 kHz burst concentrated in 1 ms, in the master and each stem, with the layer that carries
it and where (start / end = a cut: P0 or P1; body = inside the recording: decide whether it is the sound's own crisp
transient or a defect in the source).
2. qa.py `edges`: used samples / slices with abrupt entries or chopped decays (defects) and craft notes (attacks
starting in the first ms, decays faded early, second hits); music splices (chopped notes, holes, overlaps,
segments starting inside a sustained sound); placements past the film ends.
3. For each doubtful sample: `mix.py --only <fragment> --dump out/audio-review/critic_edges/dump`, read
`_samples.json` (head / tail dB, attack ms) and the source around the trim (`analyze.py <file> --png ...`).
4. The master's head (first 5 ms) and tail (last 50 ms) in qa.py loudness.
## Test your fixes
`{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_edges/ops_v1.json --out out/audio-review/critic_edges/v1 --render --qa`
and report the click / edge counts before and after (a fix must not create a new click elsewhere).
## Typical fixes
`fade_in` 3-5 ms (never more than the attack allows), `start` 3-5 ms before the attack, `end` after the decay or before
the next hit, `fade_out` 20-80 ms, `stop_before_next`, a different take, a splice on another bar, `tail_fade`.
## Return (JSON only)
`{"lens": "edges", "issues": [{"id", "priority", "t", "what", "problem", "evidence", "ops": [...], "verified"}],
"overall": "...", "score": 1-10}`
templates/prompts/critic_masking.md
# Critic lens: masking and density (can every sound be heard; is the effect layer dense or repetitive?)
You are one of five read-only critics of {{FILM}}'s soundtrack. Your lens: **audibility and density**. An effect the
music covers is a wasted placement and a story beat lost; a layer that adds nothing is clutter; one sample fired like
a machine gun is cheap; a wall of effects under the full groove is noise.
## Context
- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Render `{{RENDER}}`, qa `{{QA_JSON}}` (run qa.py with
`--layers` if it was not). Brief: `audio/SOUND_BRIEF.md`. Round {{ROUND}}; changed: {{CHANGED}}; fixed: {{FIXED}}.
- You write ONLY in `out/audio-review/critic_masking/`.
## Measure
1. qa.py `audibility`: per event `read_db` (>= 6 clear, 3-6 ok, 0-3 barely, < 0 masked), the band where it reads
best, `covered_by` (music/amb or other effects); `dead_layers` (layers of multi-layer events that add nothing).
2. qa.py `density`: events per second (about 5 designed effects per 10 s is the norm for launch films; dense where the
music is sparse, thin after the drop), the most reused samples, per-rule regularity (inter-onset CV, gain / pitch
spread, distinct samples), machine-gun rules.
3. Key beats first: the hook (first 2 s), every story beat in the picture map, the reveal, the logo.
4. `{{PY}} -m earshot timeline {{RENDER}}/soundtrack.wav --cues {{CUES}}`: does the transcript hear an event on each
key beat, with tags that read as the intended sound?
## Test your fixes
`{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_masking/ops_v1.json --out out/audio-review/critic_masking/v1 --render --qa`
Report read_db before -> after for every touched event, and check nothing else got masked (a louder effect can hide
its neighbour; a bus duck can drop the music's own downbeat).
## Typical fixes
Rule `gain_db`; a short music duck (`buses.music.auto`) under one key moment; EQ that carves the effect's best band (a
`peq` boost on the effect or a cut on the bed, never both large); a brighter or tighter take; removing dead layers;
more takes in a pool; `jitter_db` / `pitch` for variation; dropping effects the music already carries.
## Return (JSON only)
`{"lens": "masking", "issues": [{"id", "priority", "t", "event", "problem", "evidence", "ops": [...], "verified"}],
"overall": "...", "score": 1-10}`
templates/prompts/critic_sync.md
# Critic lens: sync (does every sound land on its cause?)
You are one of five read-only critics of {{FILM}}'s soundtrack. You cannot hear; you measure. Your lens: **timing**.
Every sound must sit on the frame where the eye reads its cause: impacts on contact, pop-ins at 50 % of their travel,
whooshes peaking on the velocity peak, risers ending on (or 30-250 ms before) their hit, the logo sound on the frame the
wordmark becomes readable.
## Context
- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Brief: `audio/SOUND_BRIEF.md` (picture map).
- Render under review: `{{RENDER}}` (soundtrack, stems, report `{{REPORT}}`, trace `{{TRACE}}`), film `{{VIDEO}}`,
cues `{{CUES}}`, qa `{{QA_JSON}}`. Round {{ROUND}}. Changed since last round (check hardest): {{CHANGED}}.
Already fixed, do not re-report: {{FIXED}}.
- You write ONLY in `out/audio-review/critic_sync/`. Never edit the design.
## Measure
1. `qa.py` sync section (`{{QA_JSON}}`): per event attack / peak / apex / end vs its cue; deliberate offsets listed apart.
2. The picture: `{{PY}} -m earshot timeline {{VIDEO}} --cues {{CUES}}` (cut / cue sync, events with offsets) and frames
around doubtful events (`ffmpeg -ss <t> -i {{VIDEO}} -frames:v 1 out/audio-review/critic_sync/f_<t>.png`, a few frames
apart; open them) to see where the contact / peak really is. A cue can be wrong too: say so.
3. The sounds: `mix.py --only <fragment> --dump out/audio-review/critic_sync/dump` and the dumped samples' envelopes:
a slow-rising sample aligned by `attack` peaks late; a whoosh aligned by `attack` instead of `peak` lands early.
4. Rule offsets (`offset` >= 30 ms) are deliberate: check each on the frames anyway.
## Test your fixes
Write the operations to `out/audio-review/critic_sync/ops_v1.json` and render the variant (format in
`templates/prompts/fixer.md`):
```
{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_sync/ops_v1.json --out out/audio-review/critic_sync/v1 --render --qa
```
Report the sync numbers before and after.
## Rules
- Move the picture's cue only when the cue is provably wrong (frames); otherwise move the sound (`align`, `offset`, trim).
- A sound may lead its cause by at most ~45 ms; impacts may land up to ~75 ms late; whoosh apex within +-45 ms.
- Rank by impact per minute of fixing. P0 = audible on a key beat (hook, reveal, logo); P1 = noticeable on a normal
listen; P2 = a refinement.
## Return (JSON only)
`{"lens": "sync", "issues": [{"id", "priority", "t", "event", "problem", "evidence": "<numbers + command>",
"ops": [<operations>], "verified": "<before -> after numbers from the variant>"}], "overall": "...", "score": 1-10}`
templates/prompts/critic_tonal.md
# Critic lens: tonal (pleasant on phones and laptops: harshness, brightness, low end, width, key, limiter)
You are one of five read-only critics of {{FILM}}'s soundtrack. Your lens: **tone and comfort**. Most viewers hear the
film on a phone or a laptop: harsh 2-6 kHz spikes, ice-pick highs, a muddy or wide low end, out-of-key bells and a
limiter squashing the drop are what make a mix tiring or cheap.
## Context
- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Render `{{RENDER}}`, qa `{{QA_JSON}}`, the music's key
{{KEY}}. Round {{ROUND}}; changed: {{CHANGED}}; fixed: {{FIXED}}.
- You write ONLY in `out/audio-review/critic_tonal/`.
## Measure
1. qa.py `harshness`: events whose 2-6 kHz 5 ms peak is more than 6 dB over the music's loudest 2-6 kHz peak; the
`bright` list (centroid > 6.5 kHz: above launch-film effects; fine if it is the film's palette).
2. Band levels over time (master vs stems): write a short script in your folder (scipy band-pass 2-6k, 6-10k, 10-16k,
>16 kHz, 20 ms frames) or use `analyze.py --png` spectrograms of the stems; look for effects that poke out of the
music's spectral envelope.
3. Low end: the impact's partials under 200 Hz vs the music's at the same moment (beating, mud); master.py folds
below 120 Hz to mono: check nothing important lives in the side signal.
4. Width: side/mid per section (effects should be near mono, only whooshes, beds and music wide).
5. Key: tuned samples (bells, celesta, motif notes) vs {{KEY}}: `{{PY}} -m earshot describe <dumped sample>` gives f0
/ note; `analyze.py --music` gives the track's key.
6. Limiter: `master.json` (gain reduction, loudest momentary); if the drop is squashed, test a lower `master.lufs` or
a bus ride on the loudest peaks.
## Test your fixes
`{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/critic_tonal/ops_v1.json --out out/audio-review/critic_tonal/v1 --render --qa`
Report the harshness numbers, band levels and audibility (a softer top must not mask the event) before -> after.
## Typical fixes
`highshelf` / `peq` cuts on the offending sample, `norm_peak_db` down a few dB, `lp` on a bright bed, `mono` on point
sources, `pitch` to the key (within +-2 st) or another note's recording, `master.lufs` -1 for a dynamic track.
## Return (JSON only)
`{"lens": "tonal", "issues": [{"id", "priority", "t", "what", "problem", "evidence", "ops": [...], "verified"}],
"overall": "...", "score": 1-10}`
templates/prompts/designer.md
# Brief: design the {{CATEGORY}} fragment
You are the sound designer for **{{CATEGORY}}** in {{FILM}}. You turn the ranked recordings in
`audio/assets/manifests/{{CATEGORY}}.json` into `audio/design/{{CATEGORY}}.json`: trimmed samples, pools and rules on
the picture's cue events. You cannot hear: you verify every trim, level and sync point by measurement and by earshot.
## Context
- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Brief: `audio/SOUND_BRIEF.md`.
- Format: `{{SKILL}}/references/design-format.md`; craft: `{{SKILL}}/references/sound-design.md`; an example of every
feature: `{{SKILL}}/templates/design_fragment.json`.
- Your cue events (kind / id regex / times): {{EVENTS}}
- Prefix every name with `{{PREFIX}}.` (samples, slices, pools, rule names).
- You own ONLY `audio/design/{{CATEGORY}}.json` and `out/audio-review/{{CATEGORY}}/`. Never edit another fragment, the
top-level design, the manifests or the assets.
## Steps
1. **Trims.** For each need, take the manifest's best take and its notes; get exact trims from earshot and check them:
```
{{PY}} -m earshot find <file> "{{SINGLE_EVENT_DESCRIPTION}}" --top 8 # start / end ready to use
{{PY}} -I {{SKILL}}/scripts/mix.py --only {{CATEGORY}} --dump out/audio-review/{{CATEGORY}}/dump
```
Read `_samples.json`: attack 3-5 ms after the start, head and tail far below the peak (head < -30 dB, tail < -35 dB
re peak unless faded), no second hit, a sensible duration. Use `slices` for recordings of many hits.
2. **Shape.** `mono` for point sources, `eq` (high-pass the rumble, tame 2-6 kHz harshness with a `highshelf` or `peq`),
`pitch` for scale (+2..+6 st makes it smaller), `norm_peak_db` so a pool is even, fades long enough to never click.
3. **Rules.** One rule per layer per event class: `align: attack` for impacts / clicks / keys / pops, `peak` for
whooshes, `end` for risers. Pools of 4+ takes with `mode: cycle` for repeated actions, `jitter_db` 1-2, small
`pitch` ranges; one fixed voice for UI clicks. Name every rule (variation is keyed on the name). Write `_doc` for
every non-obvious number.
4. **Verify alone**:
```
{{PY}} -I {{SKILL}}/scripts/mix.py --only {{CATEGORY}} --trace out/audio-review/{{CATEGORY}}/adds.json --out out/audio-review/{{CATEGORY}}/solo.wav
{{PY}} -I {{SKILL}}/scripts/qa.py --only {{CATEGORY}} --report out/audio-review/{{CATEGORY}}/solo_report.json --trace out/audio-review/{{CATEGORY}}/adds.json --stems out/audio-review/{{CATEGORY}}/solo_stems --out out/audio-review/{{CATEGORY}}/qa.json
{{PY}} -m earshot timeline out/audio-review/{{CATEGORY}}/solo.wav --cues out/cues.json
```
(`--only` writes the report and stems next to `--out`: `solo_report.json`, `solo_stems/`.) In qa.json: no clicks at
an edge, no edge defects, sync within limits, no machine-gun rule, every event of your classes sounded. In the
timeline: an event on every cue, tags that read as the intended sound.
5. **Levels in context.** When the lead has an integrated mix, read its qa.py audibility rows for your events
(read_db >= 3, ideally >= 6) and adjust rule gains; never louder than the cause deserves.
## Rules
- Only real recordings from the manifest (licence confirmed in audit.json). Processing: trim, fade, gain, pan, EQ,
resampling, gain envelope. Never synthesize.
- Every claim in your report comes from a command you ran.
## Return
The fragment's contents in a table (need -> sample(s) -> rule -> events), the qa.json summary of your solo render, the
earshot timeline lines for your key moments, what you are unsure about, commands run.
templates/prompts/fixer.md
# Brief: the fixer (one agent applies the critics' changes)
You are the only agent allowed to edit the design during this fix round of {{FILM}}. Five read-only critics (sync,
edges, masking, tonal, director) returned ranked issues with exact operations and the numbers of the variants they
tested. Your job: merge them into one consistent change, apply it, re-render, prove nothing regressed.
## Context
- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Design `audio/sounddesign.json` + `audio/design/*.json`.
- Critic reports: {{CRITIC_REPORTS}} (JSON). Previous qa: `{{QA_JSON}}`. Round {{ROUND}}.
## Operations (the format every critic used; `scripts/variant.py` applies them)
```
{"f": "<fragment file name without .json>", "sample": "<name>", "set": {...}} also "slice": name
{"f": "...", "sample": "<name>", "eq": {...}} merged into the sample's eq
{"f": "...", "rule": "<rule name>", "set": {...}} | {"f": "...", "rule": "<name>", "delete": true} | {"f": "...", "rule_add": {...}}
{"f": "...", "place": "<sample>", "set": {...}} | {"f": "...", "place_add": {...}}
{"f": "ambience", "amb": "<name>", "set": {...}} {"f": "music", "segment": <index>, "set": {...}}
{"f": "...", "pool": "<name>", "value": [...]} {"sd": "buses.music.auto", "value": [[t, dB], ...]}
```
## Steps
1. **Collect** every P0 / P1 with its operations; drop issues whose `verified` numbers do not show an improvement.
2. **Resolve conflicts.** Two critics touching the same sample, rule or ride: find the change that satisfies both (a
hit that must be louder AND less harsh -> more low body, a `highshelf` cut on top); masking vs density (louder vs
fewer): prefer removing a dead layer over raising a level; any bus ride change against the director's arc wins only
with numbers. Write each decision down.
3. **Apply** all operations at once:
```
{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/fix_round{{ROUND}}.json --out out/audio-review/fix_round{{ROUND}} --render --qa
```
compare its qa.json with `{{QA_JSON}}` section by section: every targeted issue improved, no new fail, no new click,
no new masked event, loudness still on target. Iterate on the variant until it holds, then:
```
{{PY}} -I {{SKILL}}/scripts/variant.py --ops out/audio-review/fix_round{{ROUND}}.json --apply --backup out/audio-review/backup_round{{ROUND}}
{{PY}} -I {{SKILL}}/scripts/mix.py --trace out/qa/adds.json
{{PY}} -I {{SKILL}}/scripts/qa.py --layers
```
`--apply` rewrites only the values that changed (it lists the files it wrote), so `diff -r` against the backup is
the reviewable record of the fix; a line `unchanged (...)` means that operation was already in the design.
4. **Acknowledge** confirmed deliberate exceptions with `"qa_ok": "<reason>"` on the sample / rule / segment (one
sentence each) so later rounds do not re-litigate them.
5. **Record** in each touched fragment's `_doc` / the top level's `_buses_doc` why the numbers are what they are.
## Rules
- You alone write the design this round; critics and designers do not.
- Never synthesize; never change `mix.py` or `master.py`; never loosen a qa threshold to pass.
- Every number in your report comes from a command you ran.
## Return
The applied operations (merged list), each conflict and how you resolved it, the qa.json summary before -> after per
section, issues you rejected and why, open questions for the next round, commands run.
templates/prompts/licence_audit.md
# Brief: licence audit of every recording
You are the licence auditor for {{FILM}}. You decide nothing about sound; you make sure every recording that can reach
the film is allowed, verified on its own page, credited correctly and intact. Read
`{{SKILL}}/references/licences.md` first.
## Context
- Project root: `{{ROOT}}`. Python: `{{PY}}`. Skill: `{{SKILL}}`.
- Manifests: `audio/assets/manifests/*.json`. Audit file: `audio/assets/manifests/audit.json` (create it as `[]` if
missing). Mix report (when a mix exists): `out/qa/mix_report.json`.
- You own ONLY the agent fields of `audit.json` (`licence_seen`, `licence_confirmed`, `agent_notes`) and may correct a
manifest's `license`, `license_url`, `author`, `title`, `page_url`, `attribution` when the page says otherwise
(say so in your report).
## Steps
1. Run the audit to get the list and the file checks:
```
{{PY}} -I {{SKILL}}/scripts/audit.py --root . [--report out/qa/mix_report.json]
```
Without a report every entry is in scope; with it, the recordings audible in the film are.
2. For every in-scope entry, open its `page_url` (Freesound item page; for Mixkit the licence text
https://mixkit.co/license/ plus the category page listing the item; for music the composer's track page) and read:
the licence as written, the author, the title, the requested credit form, any restriction (no raw redistribution,
no stock use). Check the download URL belongs to the same item (Freesound: the preview's id and user id match the page).
3. Write into `audit.json` for that entry (match `id` and `file_check.path`; see `templates/audit_entry.json`):
`licence_seen`: "<site> page <url>: licence '<exact text>' -> <licence url> (read <date>); page author '<x>', title '<y>'",
`licence_confirmed`: true only if the licence is allowed AND you read it on the page in this session,
`agent_notes`: requested credit forms, restrictions, doubts.
If the page shows a different or forbidden licence, set `licence_confirmed: false`, fix the manifest, and list it.
4. Re-run audit.py (with `--fetch` once the film's recordings are final: it compares the local bytes with
`download_url`, sequentially and politely) until it exits 0 for the in-scope set. Then:
```
{{PY}} -I {{SKILL}}/scripts/credits.py --root . --audit audio/assets/manifests/audit.json --design audio/sounddesign.json
```
## Rules
- Never confirm a licence you did not read on the item's own page in this session. Aggregators are not sources.
- Disputed or unclear = rejected. CC BY-SA / NC / ND / Sampling+ = rejected.
- Be polite to the sites: one page at a time, a pause between requests.
## Return
The audit summary line, every blocker / warning with what you did, every manifest correction, items rejected (and a
replacement suggestion from the same manifest), the credit lines that follow an author's requested form, and the
commands you ran.
templates/prompts/music_supervisor.md
# Brief: music supervisor (choose the track, verify its grid, edit it to the picture)
You are the music supervisor for {{FILM}}. One licensed track carries the film's emotion; your job is to pick it and
cut it to the picture at bar lines so that its own events (entrance, drop, stop, final chord) land on the picture's
beats. You cannot hear: you measure the grid, the bars, the sections and every splice.
## Context
- Project root `{{ROOT}}`, Python `{{PY}}`, skill `{{SKILL}}`. Brief: `audio/SOUND_BRIEF.md` (picture map, mood, arc).
- Picture grid: {{BPM}} BPM, {{BEATS_PER_BAR}}/4, bar = {{BAR}} s; film bars: {{FILM_BARS}}; key moments: drop / hero
reveal at {{DROP_T}} s (film bar {{DROP_BAR}}), breakdown {{BREAKDOWN}}, logo at {{LOGO_T}} s, end {{DURATION}} s.
- Candidates: `audio/assets/music/` with manifests `audio/assets/manifests/music*.json` (licences audited).
- You own: `audio/assets/manifests/music_choice.json` (the decision record), `audio/design/music.json`,
`out/audio-review/music/`.
## Steps
1. **Measure every candidate**:
```
{{PY}} -I {{SKILL}}/scripts/analyze.py audio/assets/music/* --music --png out/audio-review/music --out out/audio-review/music.json
{{PY}} -I {{SKILL}}/scripts/arrange.py <track> --film-bpm {{BPM}} --arc out/audio-review/music/arc.json --out out/audio-review/music/<id>_arrange.json
{{PY}} -m earshot describe <track> # mood / instrumentation in words, sections
{{PY}} -m earshot timeline <track> # events, sections, beat grid, loudness as a transcript
```
Write the arc file first (`templates/arc.json`): one target dB per film bar from the picture map.
2. **Verify the grid by hand**, not by trust: the tempo (arrange.py fits a line through every beat; residual and
inlier fraction say how steady), the downbeat phase (confidence; check against the low-band hits and the chord
changes in the bar table and the spectrogram), and the transient offset (arrange.py moves the grid onto the measured
attacks; a cut on a lagging grid chops the downbeat). Write the verified grid as `downbeats_s` in the decision record.
3. **Choose** with reasons: structure fits the film's arc (a soft opening, a drop that can land on the reveal, an
ending that resolves), mood fits the brand, tempo needs no or little stretch (within ~6 %; `stretch: none` within
0.2 %), clean master, licence confirmed. Record the runner-up.
4. **Edit**: take arrange.py's segment draft, then fix it by musical judgement: splices from bar to bar where the
harmony continues (V -> I, or the same chord), the drop's downbeat on the hero frame, the final chord on the logo
or the cut to black, a `tail_fade` into the end. Nudge `at` by a few ms so the measured downbeat transient sits on
the picture's downbeat. Label every segment with its bars, its chords and why.
5. **Verify** the edit alone and check every splice:
```
{{PY}} -I {{SKILL}}/scripts/mix.py --only music --trace out/audio-review/music/adds.json --out out/audio-review/music/solo.wav
{{PY}} -m earshot timeline out/audio-review/music/solo.wav --cues out/cues.json
```
and read the qa.py edges section of the integrated mix (chopped splices, holes, segments starting inside a sustained
note). Set `music.gain_db` so the big sections sit around -14 LUFS short-term before the master.
## Rules
- One track; real, professionally produced; licence confirmed in audit.json; never a rip.
- Edit only at bar lines; never stretch more than ~6 %; never loop a bar so often it becomes audible.
- Every number in the decision record comes from a command you ran.
## Return
The winner and runner-up with measured reasons, the verified grid, the segment list with splice reasons, the solo
render's timeline highlights (entrance, drop, final chord times), open risks (Content ID for CC BY music), commands run.
templates/prompts/sourcing.md
# Brief: source real recordings for {{CATEGORY}}
You are the sourcing agent for the **{{CATEGORY}}** sounds of {{FILM}}. You cannot hear, so you measure: every choice
you make is backed by numbers from the commands below. Read `{{ROOT}}/audio/SOUND_BRIEF.md` first (the contract) and
`{{SKILL}}/references/licences.md` + `{{SKILL}}/references/sound-design.md`.
## Context
- Project root: `{{ROOT}}`. Python: `{{PY}}` (run scripts with `-I`). Skill: `{{SKILL}}`.
- Your needs (from the picture map), each with its cue times and what it must sound like:
{{NEEDS}}
<!-- e.g. "a_ui_click: the film's one click voice, 2 clicks at 1.0 and 3.5 s; crisp, short (earshot decay < 30 ms), dry,
a clear press + release" / "b_whoosh: 2 camera-scale moves at 4.0 and 12.5 s; real air, 0.2-0.6 s, no tonal swoosh" -->
- You own ONLY: `audio/assets/{{CATEGORY}}/`, `audio/assets/manifests/{{CATEGORY}}.json`, `out/audio-review/{{CATEGORY}}/`.
## Steps
1. **Search** Freesound (CC0 / CC BY only: use its licence filter, then read each item's page anyway), Mixkit (lossless
WAV), and for music the composers' own sites. Prefer recordings that contain many takes of the same object
(variation for free), close-miked, dry, lossless or HQ, 44.1 kHz+. 15-40 candidates for the category, 3-6 per need.
2. **Read each item's own page**: title, author, licence (exact text), the author's requested credit form, anything the
description says about the recording (mic, room, what happens when). Reject NC / ND / SA / Sampling+ / custom /
unclear on sight. Never fabricate or guess a licence.
3. **Download politely**: sequentially, a pause between requests, the public HQ file
(`https://cdn.freesound.org/previews/<id//1000>/<id>_<userid>-hq.ogg`, Mixkit
`https://assets.mixkit.co/active_storage/sfx/<id>/<id>.wav`). Save as `audio/assets/{{CATEGORY}}/<source>-<id>_<slug>.<ext>`.
Audio files only; never run or unpack anything downloaded. Pixabay blocks scripts: list such items for the user.
4. **Measure every file**:
```
{{PY}} -I {{SKILL}}/scripts/analyze.py audio/assets/{{CATEGORY}}/* --png out/audio-review/{{CATEGORY}} --out out/audio-review/{{CATEGORY}}.json
{{PY}} -m earshot realness audio/assets/{{CATEGORY}}/
```
Open the spectrogram PNGs of the best candidates (the Read tool shows images). Reject: clipped, floor above -55
dBFS between events, a cliff above 16 kHz (low bitrate), room echo on foley, voices / bumps / dogs in the part you
need, realness verdict "likely code-synthesized". An "uncertain" file may stay only when you re-run realness on
the trim the designer will use (`--start --end`) and write its evidence and the trim's verdict in the manifest
`notes` (designed library sounds - whooshes, chimes - often sit there).
5. **Listen through earshot** for each need:
```
{{PY}} -m earshot rank "{{NEED_DESCRIPTION}}" audio/assets/{{CATEGORY}}/ --top 6
{{PY}} -m earshot match <file> "{{NEED_DESCRIPTION}}" --alts "<what it must not be>" "<another wrong reading>"
{{PY}} -m earshot find <long file> "{{SINGLE_EVENT_DESCRIPTION}}" --top 8 # the best takes inside it as trims (skip any marked head cut / tail cut)
{{PY}} -m earshot describe <file>
```
Quote earshot's own verdict words, not your own z cut-offs: strong (z >= 3), plausible (2.5-3), weak (2-2.5),
no match (< 2). `p_vs_alts` only says which of your wordings wins. Use find's trims as they are, but never one
marked 'head cut' / 'tail cut'; for two candidate takes, `{{PY}} -m earshot ab <a> <b> --query "{{NEED_DESCRIPTION}}"
--a-start .. --a-end .. --b-start .. --b-end ..`. When approved references exist,
`{{PY}} -m earshot compare <file> --refs <folder>`.
6. **Write the manifest** (`templates/manifest_entry.json` shape): every field, `attribution` for CC BY (the author's
form, else the default line), `need`, `rank` (1 = best for that need), `best_segment`, `analysis` (from analyze.py),
`earshot` (realness, match z), and `notes` that tell the designer exactly what to use: the times of the best takes,
what to avoid (a bump at 9.6 s), how loud it is, suggested trims / EQ / pitch.
## Rules
- Real recordings only; no generated or synthesized audio, no YouTube rips, nothing behind a login.
- Every number in your notes comes from a command you ran.
- Do not write design JSON; do not touch other categories.
## Return
A short report: per need the ranked candidates (id, why, best segment, measured key numbers, realness, match z), the
rejected ones with the reason, any item the user must download by hand (Pixabay), and the commands you ran.
templates/SOUND_BRIEF.md
# SOUND BRIEF: {{FILM}} soundtrack from professional recordings
<!-- Copy to audio/SOUND_BRIEF.md and fill every {{FIELD}}. Every sub-agent (sourcing, music, design, critics, fixer)
receives this file verbatim: it is the contract. Delete the example rows. -->
The picture is final: `{{VIDEO}}`, {{DURATION}} s, {{FPS}} fps, cut on a {{BPM}} BPM grid (1 beat = {{BEAT}} s,
1 bar = {{BAR}} s). Cue events: `{{CUES}}` (computed from the picture code; never re-time them by hand).
The soundtrack uses ONLY real, professionally recorded or professionally produced audio from licensed libraries:
- one licensed music track, edited to the picture at bar lines;
- real foley and SFX recordings placed on the picture's cues;
- real field-recorded ambiences where the picture has a place.
Code may only select, trim, place, level, pan, EQ, crossfade, time-stretch the music (small ratios), pitch by
resampling and master these recordings. Code never generates a tone, a noise or a "sound" (no oscillators, no
synthesized noise, no synthesized reverb tails). The bar: {{BAR_REFERENCE}} (e.g. "a premium product launch film:
crisp, tactile, confident, every sound caused by something on screen"), not a game jam.
## Sound concept
<!-- 4-8 bullets: the palette (materials), the music's character and arc, the places. Example for a launch film: -->
- **One tactile UI palette.** {{PRODUCT}} is precise and calm: every interface sound comes from one family of real
close-miked sources (a laptop trackpad click, soft felt-tip taps, one mechanical keyboard), dry and centred.
- **One music track carries the film.** {{MOOD}}: a soft intro under the cold open (no drums), a groove for the feature
beats, a breakdown at {{BREAKDOWN_T}} s before the turn, the DROP on the hero reveal at {{DROP_T}} s, a resolved ending
under the logo ({{LOGO_T}} s), tail to {{DURATION}} s. Instrumental only, {{TEMPO_RANGE}} BPM.
- **Restraint.** About 5 designed effects per 10 s, clustered in the cold open and the logo build; the music carries
the cuts.
- **Places.** {{ZONES_SUMMARY}} (e.g. "a quiet office room tone under the UI part only; no ambience in the abstract
sections").
## Picture map (film seconds; cue ids from `{{CUES}}`)
| t (s) | what happens on screen | cue (kind / id) | sound need |
|---|---|---|---|
| 0.0 | the cursor glides onto the empty search field | (none) | nothing: the music enters soft |
| 1.0 | click into the field | click / demo:cursor:click | the film's one click voice (press + release) |
| 2.0-3.5 | the query types itself (12 characters) | key / demo:type:key:n, demo:type:space | real keyboard, one key per sounded character, space deeper; music dips 6 dB |
| 4.0 | whip pan into the results | whoosh / demo:transition | one real swish, apex on the velocity peak |
| 5.5-6.0 | three result cards pop in | pop / demo:card:n | soft real pops climbing 2 st across the three |
| 7.0-8.0 | everything pulls back; the product mark flies in | riser / demo:build | the music's build (or a real cymbal roll ending 30 ms before 8.0) |
| **8.0** | **hero reveal: the whole product UI lands** | hit / demo:reveal | **the music's DROP on the frame**; a low real impact under it |
| 10.0 | the wordmark settles | bell / demo:lockup | music stops on the bar line; one soft celesta note in key; tail to the end |
Ambience zones: {{ZONES}} (e.g. "office room tone 0-8.2 s; none after the reveal").
## Licences: what is allowed
Allowed (record each in the manifest, read on the item's own page): CC0 1.0 and public domain; CC BY 3.0 / 4.0
(credits in CREDITS.md and the video description, never required on screen); Mixkit Sound Effects / Stock Music Free
License (read the current text at https://mixkit.co/license/); Pixabay Content License (manual download only).
NOT allowed: NonCommercial (BY-NC), NoDerivatives (BY-ND: syncing to picture IS a derivative), ShareAlike (BY-SA),
Sampling+, "personal use only", custom or unclear licences, disputed licences, YouTube rips, anything behind a login
or paywall you would have to bypass, AI-generated or code-synthesized audio. Never fabricate a licence.
Details: `<skill>/references/licences.md`.
## Quality bar
- SFX / foley: real recordings, close and clean. Prefer lossless WAV/FLAC (Mixkit WAV
`assets.mixkit.co/active_storage/sfx/<id>/<id>.wav`) or Freesound's HQ preview
(`cdn.freesound.org/previews/<id//1000>/<id>_<userid>-hq.ogg`). No clipping, floor under -55 dBFS between events,
95 % rolloff above 12 kHz, no untreated room on foley, no voices / handling noise / background events.
- earshot: `realness` verdict "real or pro-produced" for every file used; `match` z >= 2.5 for the need it fills.
- Ambiences: clean (no people / traffic / planes / dogs unless the story wants them), no loud single events, >= 40 s
usable, real stereo.
- Music: professionally produced and mastered, full bandwidth, instrumental, steady tempo, 4/4, {{TEMPO_RANGE}} BPM
(stretched to {{BPM}} only if needed). Avoid: vocals, corporate ukulele + whistling, EDM brostep, lo-fi hiss,
bombastic trailer music, moods against the brand, famous overused stock tracks.
- Mix: -14 LUFS integrated, true peak <= -1 dBTP after AAC, LRA 5-9 LU, the loudest moment on the drop, the logo
8-20 dB under it, a tail into silence.
## Files, manifests, tools
- Downloads: `audio/assets/<category>/` (audio files only; never a zip with programs; never run anything downloaded).
Name: `<source>-<id>_<short-slug>.<ext>`, e.g. `fs-450282_mech-typing.ogg`, `mk-1386_keyboard-typing.wav`.
- Manifests: `audio/assets/manifests/<category>.json`, a list of entries shaped like
`<skill>/templates/manifest_entry.json` (id, category, file, title, author, source, page_url, download_url, license,
license_url, attribution, need, rank, best_segment, notes, analysis).
- Audit: `audio/assets/manifests/audit.json` (agent fields per `<skill>/templates/audit_entry.json`) + `audit.py`.
- Design: `audio/sounddesign.json` (lead) + `audio/design/<category>.json` (one designer each).
- Review output: `out/audio-review/<category or critic>/`. QA: `out/qa/`.
- Analyse: `python -I <skill>/scripts/analyze.py <files> [--music] [--png out/audio-review/<category>] [--out ....json]`;
listen: `python -m earshot realness|describe|match|rank|find ...`.
- Be polite to the sites: sequential downloads, a pause between requests, at most ~40 files per category.
templates/sounddesign.json
{
"_doc": "The film's soundtrack (real-sound skill): professional recordings only, credited in CREDITS.md. Built by <skill>/scripts/mix.py from this file and one fragment per category in audio/design/ (each designer owns one fragment; names are prefixed with the category). Keys starting with _ are comments. Reference: <skill>/references/design-format.md.",
"duration": 30.0,
"cues": "out/cues.json",
"include": [
"audio/design/music.json",
"audio/design/ambience.json",
"audio/design/ui.json",
"audio/design/keyboard.json",
"audio/design/motion.json",
"audio/design/impacts.json",
"audio/design/logo.json"
],
"_buses_doc": "Fader rides that shape the arc (film seconds, dB). Write a reason for every ride here: e.g. the music dips 6 dB under the typing (2.0-3.6), comes back for the transition, lifts 1.5 dB into the drop, and resolves 3 dB down under the logo so the sting reads. Keep steps on a musical splice or under a transient.",
"buses": {
"music": {"gain_db": 0.0, "auto": [[0, 0], [1.9, 0], [2.0, -6], [3.6, -6], [3.8, 0], [14.9, 0], [15.0, 1.5], [24.0, 1.5], [26.0, -3], [30.0, -3]]},
"amb": {"gain_db": 0.0},
"sfx": {"gain_db": 0.0}
},
"_master_doc": "-14 LUFS integrated for web / social (YouTube, X, LinkedIn normalise to about -14). Ceiling 0.77 linear (-2.3 dBFS) leaves room for the AAC encode (measure about -1 dBTP after the mux). Lower the target to -15 / -16 only for a reason you write here (e.g. a dynamic orchestral track whose tutti the limiter would squash at -14).",
"master": {"lufs": -14, "ceiling": 0.77, "no_glue": true}
}