agents/openai.yaml
interface:
display_name: "Tesseract: Edit Video"
short_description: "Edit footage into finished videos with Tesseract"
icon_small: "./assets/icon.png"
icon_large: "./assets/icon.png"
default_prompt: "Use $tesseract-video to edit my footage and preview the finished video."
references/ad-hooks.md
# Build a visual hook for an ad
For a new paid-social, YouTube, or other video ad, deliberately design its opening few seconds. Derive the hook from the actual product, footage, audience, promise, and evidence. Make it visually compelling from the first frame, with a clear reason for the intended viewer to keep watching. Sound and transitions should strengthen that idea. A hook may be a powerful shot, an overlay, or a complete motion scene; it need not be an extra intro card.
Apply this when creating an ad or improving its opening. A narrow revision elsewhere does not require rebuilding the hook. Preserve an opening the user has explicitly chosen unless changing it is part of the request.
## Choose the idea before the treatment
Inspect the source material and brief. Identify the viewer's relevant problem/desire, the strongest truthful benefit or surprising moment, and where the body of the ad delivers on it. Use the most compelling source moment even if it occurs late in the footage, provided reordering does not distort what happened or what the speaker meant.
For a substantial new ad, consider a few distinct concepts—usually two or three is enough—then select and build the strongest within the authorized edit. Compare concepts by immediate clarity, relevance, visual interest, available evidence, fit with the brand/music, and the transition into the body. Different fonts or whooshes on the same opening are treatment variations, not different hook ideas. No separate approval is needed for each ordinary creative choice; render multiple variants when requested or when a close choice makes a short comparison useful.
| Hook approach | What the viewer sees | Use it when |
|---|---|---|
| Result first | The best real outcome or most impressive working moment, then its explanation. | The supplied result is immediately legible and credible. |
| Recognizable friction | A specific frustrating task, awkward moment, or bottleneck in action. | The audience can recognize its relevance without a long setup. |
| Visible contrast | An A/B, a matched comparison, or a transformation that makes the difference obvious. | Both sides are supported by the material; labels and timing make the comparison fair. |
| Unexpected perspective | A tight detail, unusual crop, scale shift, or reveal that resolves into the product/use case. | The surprise helps the viewer understand the offering instead of becoming unrelated spectacle. |
| A visual question | An incomplete action, interrupted process, or concealed detail whose answer begins to emerge quickly. | The ad actually supplies the answer; suspense does not postpone all meaning until later. |
| A distinctive human moment | A revealing expression, direct statement, or action already present in the footage, framed around its meaning. | The person and moment carry more interest than an effects sequence. |
| An idea made visible | Editable type, objects, paths, or supplied assets collide, sort, multiply, connect, or transform to explain the benefit. | Motion graphics communicate something the available footage cannot. |
| Evidence up close | An actual product interaction, useful detail, or supported demonstration becomes the opening focal point. | The evidence itself is compelling enough to carry attention. |
These are prompts, not an exhaustive library or a mandated style. Combine approaches only when the opening still has one clear idea. Avoid generic “stop scrolling” copy, unsupported superlatives, fabricated metrics/testimonials, unrelated shock footage, and mystery that never connects to the product. A stronger hook can come from better shot selection or a better opening line without adding more effects.
## Give the opening a small narrative
Choose its duration from the placement and material; roughly the first 2–5 seconds is a useful working range, not a universal rule. Start with meaningful visual information rather than black, a slow fade, or a logo-only wait. In the first few seconds establish the relevant tension/result, reveal enough to orient the viewer, and hand off naturally into the demonstration or explanation. Integrate brand/product recognition into the action when possible.
Keep one dominant focal point. Use concise text only where it adds context, with enough stable reading time at the actual display size. A caption, hook headline, UI screen, and animated label should not all compete. Fit the supplied brand and ad style: a direct creator-led opening and an elaborate launch-film opening require different treatments. A quiet but surprising image can work; maximum speed and volume are not the objective.
Match the placement:
- **Scrolling feeds, Reels, and Shorts:** make the first frame meaningful and the proposition visually understandable without audio; sound-on playback should add impact. Preserve UI safe areas and readable phone-scale text. TikTok recommends introducing the proposition within the first three seconds and developing the hook in the first six; treat these as planning guidance, not a requirement to spend six seconds on an intro. [TikTok creative guidance](https://ads.tiktok.com/resources/help/article/creative-best-practices?lang=en)
- **YouTube skippable in-stream:** give a clear reason to keep watching and establish product/brand relevance before the skip opportunity. Google documents skipping after five seconds; this is not the timing model for every YouTube placement. [Google ad formats](https://support.google.com/google-ads/answer/2375464?hl=en)
- **Very short ads:** let the hook and main benefit form the same concise action; do not spend most of the runtime on setup. Across formats, Google's creative guidance emphasizes entering the story quickly, early branding, and audio/text that reinforce rather than compete with the message. [Google ABCDs](https://support.google.com/google-ads/answer/14783551?hl=en)
Use the specified placement/delivery brief. If unspecified, make a sensible assumption from the request and state it in the edit plan; do not block creative work with a platform questionnaire.
## Time the hook to the actual music
When music is supplied or already generated, follow [waveform editing](waveform-editing.md) **before authoring the hook timing**. Inspect and audition the beginning of the music segment actually used by the edit. Zoom into its useful opening accents, pickup, first downbeat, build, dropout, or change in texture. The candidate detector helps locate events; confirm them by listening.
Record a compact cue map: `music source time → project time → visual event → SFX event`. Specify the actual reveal/arrival/impact, not just “sync to beat.” Include the selected music in-point and its project start so source, project, and layer clocks stay distinct.
For each chosen cue:
1. Set the perceptual visual event on that cue: a cut, fully exposed result, shutter reopening, key word settling, or meaningful motion arrival. Animate the lead-in before it and leave a useful hold afterward.
2. Compute SFX placement from its own perceptual event. A riser may start earlier so it resolves on the cue; a downlifter may begin on the cue and trail afterward; a shutter or ping aligns its audible transient. File starts and largest sample peaks are not automatically the right anchors.
3. Quantize picture events to the actual project frame rate, keeping the error within the nearest representable frame and aligning the audible event as closely as the runtime permits. Check the rendered result; matching JSON timestamps alone does not prove perceptual synchronization.
For example, if listening confirms an opening accent at **0.8 s in project time**, show a meaningful close-up from frame zero, begin the reveal shortly beforehand, expose the result at 0.8 s, and let the next shot explain it. This timing is illustrative; derive real values from the track. If the music starts softly or the first big hit comes late, create immediate visual interest and use an earlier subtle cue or motivated SFX. Do not make viewers wait through an empty intro for the drop. Preserve explicitly chosen music/in-points; do not silently replace or retime them. Without music, derive rhythm from speech, action, and selected sound accents; do not invent a soundtrack requirement.
## Use diverse sound and transition treatments
Choose by the hook's idea and physical motion. Possible treatments include a match cut, object wipe, shutter close/reopen, brief light sweep or leak, freeze-and-release, mask reveal, whip movement, split-screen change, directional blur, or a hard cut into an unexpected scale. Keep useful subject/product detail visible and resolve into readable content. A light leak or shutter should earn its place in this particular ad, not become the plugin's default opening.
Sounds can include bass downlifters, impacts with audible upper harmonics, pings, pops, clicks, camera shutters, paper/physical textures, or a short pause in the bed. Choose compatible cues and leave space for the music and voice. A subtle transient may beat a pile of risers, hits, and whooshes; a strong existing musical hit may need no additional accent. Check that the hook still communicates when muted and that low-frequency cues translate beyond headphones.
Use supplied/local audio or prepare and import procedural accents through [sound design](sound-design.md). The procedural set includes `downlifter`, `soft-impact`, `clear-ping`, `soft-pop`, and `dry-click`; it does not include a literal `camera-shutter` preset. Use an available local shutter asset or an intentional stylized accent rather than inventing a helper command. Follow [motion design](motion-design.md) and the actual schema to build transitions with editable shapes, masks, transforms, and supported effects. There is no assumed one-click `lightLeak` or `cameraShutter` field. Use native constructs or permitted local assets, with the timing kept editable.
## Review the opening as its own deliverable
Render the opening plus its transition into the body early, before polishing the entire ad. Inspect a filmstrip containing the first frame, setup, key reveal, readable hold, and the next scene. Around synchronized events, inspect neighboring frames and the rendered audio waveform; play the result at speed with sound. If timing is off, correct the responsible layer/cue and re-render the affected interval.
Then review it muted and at phone scale. Ask: Is there a clear focal point immediately? Why would this audience care? What benefit or question does the opening establish? Does the body deliver on it? Is the product/brand connection clear enough? Are text and motion readable? Do music and SFX strengthen the same event without masking words? Reject openings that are impressive but confusing, misleading, disconnected, or slow to reveal relevance.
Record the selected concept, source evidence, actual cue times, and what was inspected in the edit notes. Describe the intended mechanism rather than claiming guaranteed retention, conversion, or virality. If campaign results are later supplied, assess early retention alongside the campaign's downstream objective; attention alone does not establish a successful ad. No publishing, ad spend, or campaign changes are part of this hook-authoring workflow.
references/audio-and-timing.md
# Audio and timing
The optional [sound helper](../scripts/tesseract_sound.py) uses Python 3.9+ and local FFmpeg/ffprobe.
It creates and inspects local files; it does not install a renderer or mutate a
project. If a tool is missing, report the unavailable check and use the host's
media tools where they can establish the same result.
Read [waveform editing](waveform-editing.md) before timing an edit to music or trimming dialogue. Use its local `waveform` command for timestamped overview/zoom PNGs, inspect those views, and verify music events and word boundaries by listening. Recheck changed speech joins on the rendered edit clock before final mixing.
## Listen to the source before mixing
Import reports metadata, not what a shot depicts or what its audio means. Probe
for streams with local FFmpeg tools:
```sh
ffprobe -v error -show_streams -show_format -of json /absolute/path/footage.mov
```
Then audition the dialogue, important production sounds, and music. Preserve speech intelligibility and meaningful foley. A clip with an audio stream is not necessarily usable sound. Add appropriate local sound design as described in [sound design](sound-design.md); do not invent narration or replace deliberately chosen music.
When a transcript is supplied, align cuts to its actual source timestamps. If there is no local transcription tool and speech cannot be understood reliably, ask for the transcript or flag the limitation. Never invent spoken words or caption timing.
For uneven, noisy, boxy, or reverberant dialogue, read [speech cleanup](speech-cleanup.md) before processing. It covers source selection, EQ, dynamics, de-essing, room-sound limits, and matched-loudness review of local derivatives.
## Select or make a local accent
```bash
python3 "$SKILL/scripts/tesseract_sound.py" list
python3 "$SKILL/scripts/tesseract_sound.py" make --preset air-whoosh --duration-ms 600 --seed 12 --output "$WORK/whoosh-v1.wav"
```
Generate a fresh WAV with `make`; no preset WAV files or rendering binaries are bundled. The helper contains soft-tap, dry-click, soft-pop, clear-ping, air-whoosh, short-riser, downlifter, and soft-impact. Each is a small procedural accent, not a music track or recorded foley. Listen in context before choosing it; the default file peak is −12 dBFS and does not establish its perceived mix level.
The helper creates and inspects local files. Import a selected cue, music track,
or cleaned dialogue derivative as audio before
adding its editable Audio layer. See [media import](media-import.md) for commands.
Import packages the bytes; placement and mixing are separate document edits.
Use the imported asset ID in an Audio layer, with `captionsEnabled: false` for music/effects. Keep the cue's source interval inside the actual file and its active interval inside the composition. A cue peaking 300 ms into the file should start 300 ms before the visual event it punctuates. Trim/fade a tail near the edit's end rather than unintentionally extending the project.
## Local mix
New Video layers need `volume: 1.0` to enable embedded source audio at unity gain; omitted/null volume disables it. Preserve existing gains on revisions. FX `Audio` layers can place music, a supplied sound effect, or dialogue independently. They have no visual transform. A minimal source carries `assetId`; the source range and windowed `playback` are explicit (Audio has no `activeRange`).
```json
{
"id": 90,
"name": "Supplied music bed",
"type": "Audio",
"playback": {"type": "windowed", "inputRange": {"start": 0, "duration": 8000},
"mapping": {"type": "linear", "input": {"start": 0, "duration": 8000},
"output": {"start": 0, "duration": 8000}}, "inputOffsetMs": 1500},
"sourceRange": {"start": 1500, "duration": 8000},
"sourceIntrinsicDuration": 30000,
"source": {"assetId": "local-music-id"},
"volume": 0.2,
"captionsEnabled": false
}
```
Use the schema to check additional source metadata or per-channel features. Do not infer all audio effects of a DAW from the presence of an Audio layer.
`volume` is linear: unity is `1.0`; `0.5` is approximately −6 dB. Convert dB with `10 ** (dB / 20)`. A fixed value such as `0.2` is a starting point, not a calibrated mix target. Loud source material needs different gain.
## Envelopes and ducking
For an FX Audio layer, target `propertyType: "volume"` using `layerTimeJsCode`. This is called `AudioVolume` in Rust, but the JSON wire name is `volume`, not `audioVolume`. Explicit ramps prevent clicks and allow a music bed to duck under dialogue. For example, fade in, hold quietly during speech, then lift:
```js
var t = input.time.seconds;
function lerp(a,b,p){return a+(b-a)*Math.max(0,Math.min(1,p));}
if(t < 0.2) return lerp(0,0.12,t/0.2);
if(t < 3.8) return 0.12;
if(t < 4.2) return lerp(0.12,0.30,(t-3.8)/0.4);
if(t < 7.5) return 0.30;
return lerp(0.30,0,(t-7.5)/0.5);
```
A `volume` animator supplies the **final linear layer gain**, overriding the static `volume`; it is not automatically multiplied by that static value. Include the chosen bed level in every returned value.
Use `setFxPropertyAnimator` for the script, or supported volume keyframe actions
from the installed action schema. Apply them as an action batch. For a
bed at −14 dB that ducks by 6 dB, return the linear equivalent of −20 dB during
speech. Set times from the actual dialogue, not the illustrative script above.
If a root Audio layer starts at project time 5s, its local 950 ms duck occurs at
project time 5.95s; account for ancestor playback when nested.
For another duration, change the envelope accordingly. The times are owner-layer-local. Moving the Audio layer should move the envelope with it.
## Native controls versus local audio preparation
This package supports native Audio layers, source/active timing, linear gain, and time-varying volume. Use envelopes for predictable ducking, fades, and planned dropouts. This is **timed ducking**, not an audio-reactive sidechain compressor.
Do not invent native `compressor`, `sidechain`, `EQ`, `pan`, or `truePeakLimiter` fields. When the source needs EQ/compression, prepare a new **local audio derivative** with installed FFmpeg, keep the original source and processing recipe, and import it as audio and mix it through Tesseract. Do not flatten the editable project or quietly change the original media.
For example, a gentle music presence dip can be auditioned with:
```bash
ffmpeg -nostdin -n -i /absolute/path/music.wav -af "equalizer=f=2000:t=q:w=0.8:g=-3" -ar 48000 -c:a pcm_s24le /absolute/path/music-presence-v1.wav
```
Real signal-driven ducking, when needed, can be prepared with FFmpeg `sidechaincompress` using **aligned full-length** music and dialogue stems, both starting at the same edit zero. Its first input is the music to process; the second is the dialogue detector. Preserve the separate dialogue track when assembling the Tesseract mix. Adjust threshold/ratio from the source, check attack/release and actual reduction, and do not apply both the preprocessed duck and a second unplanned native duck. Misaligned, trimmed, or short detector stems need alignment/padding first. A source with music and voice already mixed together is not a clean dialogue detector.
Use `ffmpeg -h filter=<name>` to verify the locally installed filter/options before processing. Filter behavior and available controls are described in the [official FFmpeg filter reference](https://ffmpeg.org/ffmpeg-filters.html#sidechaincompress).
## Editing sound
- An audio prelap or tail uses independent placement and source selection. Do not detach dialogue without retaining lip sync where the speaker is visible.
- Avoid duplicate playback from both a Video layer and a copied Audio layer. If moving sound into independent layers, deliberately mute the corresponding original audio.
- Use whooshes/impacts only for a movement or transition that benefits from them. A sound on every text entrance quickly becomes distracting.
- Preserve the selected music track's identity. Do not replace, regenerate, or globally retime supplied music unless the user asks.
- Speed-changing a group also changes descendant audio clocks. Check source duration and pitch/time-stretch behavior in the actual render instead of assuming a silent visual retime.
## Review
Listen to the final muxed video, not just separate audio files. Check the head and tail, dialogue-to-music balance, abrupt gain edges, and source synchronization. Measure loudness/peaks locally with FFmpeg if needed; for example `ffmpeg -i edit.mp4 -af ebur128=peak=true -f null -`. Treat readings as measurements, not proof that speech is intelligible. Keep reasonable peak headroom and follow an explicit delivery spec when one is provided.
For a dialogue-only diagnostic, check out the working document and commit a
separate `.tsrct` copy with music/SFX gains and any gain animators muted. Export
that copy's audio with `export-audio` and inspect it with the waveform helper.
Preserve the working mix and source files; do not claim unperformed listening checks.
## Measure and check the final delivery
```bash
python3 "$SKILL/scripts/tesseract_sound.py" measure /absolute/path/edit-v1.mp4 --report "$WORK/checks/loudness.json" --overwrite
python3 "$SKILL/scripts/tesseract_sound.py" mobile-preview /absolute/path/edit-v1.mp4 --output "$WORK/checks/mobile.wav" --overwrite
```
The helpers never replace an existing file unless you pass `--overwrite`, which works only inside `WORK`; use it for routine checks such as these, which each pass refreshes.
`measure` reports the first audio stream's integrated LUFS, true peak in dBTP, and loudness range; it does not normalize or modify the input. The working range is −16 to −14 LUFS and true peak at most −1 dBTP, subject to the actual delivery spec. Short isolated effects or silence may not have meaningful integrated measurements. If the deliverable has several audio programs, measure the intended stream explicitly with FFmpeg.
For mastering, first adjust the native mix. If final normalization is needed, preserve the editable mix, use a measured two-pass `loudnorm` process on a derived deliverable, and record the recipe. Check the local [loudnorm options](https://ffmpeg.org/ffmpeg-filters.html#loudnorm): use the measured input loudness, range, peak, threshold, and target offset from pass one in pass two, then remeasure the encoded output. Do not normalize each individual cue to the full-program LUFS target or erase an intentional dropout to satisfy a meter.
`mobile-preview` creates a separate mono, 150 Hz–7 kHz listening copy. It is useful for noticing disappearing bass-dependent cues, phase cancellation, or poor voice balance, but it is not the final mix and does not replace listening on an actual phone.
references/capabilities.md
# Capability map for Tesseract
This skill uses the installed Tesseract CLI and its version-matched schemas.
Instructions describe intent; exact fields come from the installed action schema
and document schema (editable JSON).
Print them with `project schema` and `project schema --document`.
A successful save is not proof
that every requested combination renders correctly; inspect the actual result.
| Area | Tesseract mechanism | Read |
|---|---|---|
| Footage editing | Imported Video layers; independent source/active ranges, source audio, transform and framing | [native authoring](native-authoring.md), [FX authoring](fx-authoring.md#local-footage) |
| Compositing | Native text, shapes, groups, media, masks, adjustments, and effects supported by the document schema | [FX authoring](fx-authoring.md) |
| Motion graphics | Editable keyframe actions, procedural scripts, and coordinated groups | [motion](motion.md), [motion design](motion-design.md) |
| Type | Local TTF/OTF/TTC import, editable text, paragraph layout and schema-supported text animators | [text](fx-authoring.md#text-with-an-available-font) |
| Captions | Editable phrase text layers with actual words/timing from supplied or verified material | [bottom captions](fx-authoring.md#bottom-captions) |
| Color grading | Primary, wheels, curves, Creative LUT (`.cube`), log Input Transform, hue/saturation/luminance selections, opt-in ACES color management (ACEScct grading, ACES 2.0 SDR/HDR output, built-in camera input transforms); measured from rendered frames | [color grading](color-grading.md) |
| Custom effects | Bounded premultiplied-alpha WGSL with declared parameters and layer texture inputs | [custom shader contract](custom-shader.md) |
| Audio | Embedded video sound; imported Audio layers, source timing, linear gain and envelopes | [audio and timing](audio-and-timing.md) |
| Local sound tools | Procedural WAV accents, source/edit waveform PNGs, loudness measurements and mono listening copies | [sound design](sound-design.md), [waveform editing](waveform-editing.md) |
| Inspection/output | PNG preview, labeled project/solo filmstrips, H.264/AAC MP4, ProRes 4444 MOV (transparent with silent solo), HEVC HDR (PQ/HLG) MP4 and PQ PNGs | [local operation](local-operation.md), [filmstrip review](filmstrip-review.md), [HDR](hdr.md) |
| 3D scenes | A Scene3D layer with cameras, lights, meshes, imported `.glb`/`.blend` models, materials and environment; keyframes and scripts on scene item properties; model rig, clip and cloth preparation inside `project apply` | [Scene3D basics](scene3d-basics.md), [recipes](scene3d-recipes.md), [Blender map](scene3d-blender.md), [Scene3D properties](scene3d-properties.md) |
## Current boundaries
- A document owns one FX composition. Use groups for scenes and Video layers for
cuts, not `baseVideoTrack`, `fxCompositions`, or caption/audio tracks.
- Import packages local MP4/MOV/M4V footage, TTF/OTF/TTC font files, and
3D `.cube` LUTs.
A 3D-enabled CLI also packages models, IES photometry, cloth point caches
and HDR environment images; see [Scene3D basics](scene3d-basics.md).
Installed system fonts are not discovered
automatically. Audio and images use [media import](media-import.md).
Import helper-generated files before placing layers.
- Export writes H.264 MP4 or ProRes 4444 MOV at 720p, 1080p, or 4K and
any positive integer frame rate (default: 1080p/30/MP4), for the whole
timeline or a segment. MP4 bitrate scales automatically. Inspect the encoded
output before delivery.
Export a timeline segment with `--start-ms` and/or `--duration-ms`.
`export-audio` writes only the mix, as a WAV, without rendering video.
- Canvas choices are listed in [local operation](local-operation.md). A bottom
shape or the document's `backgroundColor` sets the background.
- Native masks/mattes, effects, and supported 3D layer transforms are available
only as allowed by the installed schema and required local resources. Verify
person segmentation's model/resource availability and a real matte before
planning around it. Do not approximate a person with a hand-drawn silhouette.
- JS animation is not HTML/React rendering.
3D transform properties tilt ordinary 2D layers; meshes, imported models,
rigging and cloth are Scene3D items (see [Scene3D basics](scene3d-basics.md)).
For path-traced CLI previews and exports, follow [renderer selection](scene3d-basics.md#choose-the-renderer). Browser and iOS playback use raster.
- No generative video, avatar service, transcription, dedicated dereverberation
model, cloud rendering, publishing, or editable Premiere/AE export is bundled.
- Native gain envelopes are timed ducking, not signal-driven compression. Do not
invent DSP fields. Local derivatives use available FFmpeg and are imported as audio; see [speech cleanup](speech-cleanup.md).
- CLI setup follows [installation](installation.md).
- Optional audio helpers need
Python 3.9+ and local FFmpeg/ffprobe. Font import requires a local TTF/OTF/TTC file;
subsequent rendering uses the document's packaged bytes.
## Look up only what the edit needs
Save the two schemas locally and search for the specific layer, property, effect,
or action definition. Preserve existing fields and animation relationships.
Use action batches where the action schema covers the edit, and direct document
JSON edits for document-level edits and media placement.
The action schema includes only supported standalone operations; use
checkout/commit for the document JSON. Use the installed CLI’s schemas and available tools.
If the requested feature is unavailable, explain the specific
limit and continue supported work without switching engines silently.
references/cli-version.txt
0.6.0
references/color-grading.md
# Color grading
Tesseract grades footage with native, editable color operations. Measure first,
change one thing at a time, and check the result with numbers and frames.
Exact fields and limits come from the installed action schema; search it for the
effect or action you need. Every grading value is an ordinary effect field, so the result
stays editable in Tesseract.
## What is available
| Need | Mechanism | Where |
|---|---|---|
| Scene-referred ACES color management (filmic highlights, light-like compositing, one grade for SDR and HDR) | document `colorManagement` plus layer `renderDomain`; see [ACES](#aces-scene-referred-color-management) | whole document |
| Normalize camera log footage | `source.inputTransform` on a Video or Image layer (a technical 3D LUT; in ACES a LUT-less `cameraInput`) | before every effect of that layer |
| Exposure, white balance, contrast, saturation | `primaryGrade` effect | layer effect stack |
| Global/shadows/midtones/highlights color balance (not Lift/Gamma/Gain) | `tonalColor` effect | layer effect stack |
| Tone and channel curves | `colorCurves` effect | layer effect stack |
| Hue-selective curves | `hueCurves` (five static linear curves) | layer effect stack |
| Cross-channel mixing / monochrome | `channelMixer` (RGB rows and constants) | layer effect stack |
| Subtractive range correction | `selectiveColor` (`relative` or `absolute`; Approximate Adobe mapping) | layer effect stack |
| A creative look from a `.cube` file | `lookTransform` effect (Creative LUT) with `mix` 0–1 | layer effect stack |
| Adjust only some colors or areas | `compositingOptions` on any of the grading effects above: `qualifier` (hue/saturation/luminance selection), `maskReferences`, `effectOpacity` | that effect |
| Measure the picture | `preview` / `filmstrip` PNGs, measured with local tools | read-only |
| Spatial denoise | `noiseReduction` (`luminance`, `chrominance`); no temporal mode | layer effect stack |
| Finishing | `halation` (warm highlight bloom), existing `vignette`, `grain`, `sharpen`, `levels`, `vibrance`, `hueSaturation`, `tintTritone` effects | layer effect stack |
Grade one shot by adding effects to its Video layer. Grade everything below a
point by adding them to an Adjustment layer. Effects run in stack order; keep the
usual order: Input Transform (fixed, first), then `primaryGrade`, `tonalColor`,
`colorCurves`, and finally `lookTransform`. This effect order is a recommendation,
not an automatic reorder. The Input Transform belongs to the media source, not
the effect stack: use it to normalize camera footage, and use `lookTransform`
for a creative look after correction. Do not apply the same camera conversion
again as a Creative LUT. Neither HDR output nor `hdrGraphicsWhiteNits` replaces
camera normalization or exposure correction.
In a default (`legacyDisplay`) document, grading controls operate on
display-referred SDR Rec.709. HDR (PQ or HLG) export, preview, and filmstrip are
available; see [HDR](hdr.md). These do not turn the grading controls into a
scene-linear HDR grading system. For scene-referred grading with filmic
highlights and a managed SDR/HDR output, opt the document into
[ACES](#aces-scene-referred-color-management).
## Static hue curves, channel mixer and selective color
All three use `semanticVersion: 1`, preserve alpha and bypass exact identity without a render pass. Use `addFxLayerEffect`/`updateFxLayerEffect` with complete payloads. They do not expose animatable scalar parameters in v1. In an ACES scene group they run unchanged on ACEScct, not scene-linear RGB; HDR otherwise uses the existing SDR-domain split.
- Hue curves: `{"type":"hueCurves","semanticVersion":1,"hueVsSaturation":[{"x":0,"y":0.4},{"x":0.5,"y":0.6},{"x":1,"y":0.4}]}`. y=0.5 is neutral. All five curves sample original qualifier H, HSV S and Rec.709 L; hue wraps, saturation gains multiply and clamp, then luminance offsets. Reconstruction scales HSV(H,S,1) to requested luminance before RGB clipping, which may shift hue/saturation in highlights.
- Channel mixer: for an R/G swap, use red row `{red:0,green:1,blue:0,constant:0}`, green row `{red:1,green:0,blue:0,constant:0}`, blue row `{red:0,green:0,blue:1,constant:0}` and `monochrome:false`. Weights are -2..2, constants -1..1. `monochrome:true` uses the red row for all outputs.
- Selective color: initialize all nine ranges with `{cyan:0,magenta:0,yellow:0,black:0}`, choose `method:"relative"`, then try `reds.cyan=0.2`. Inks subtract RGB, with original-input range weights; relative positive ink scales by value and negative ink by headroom. This is an approximation; do not assume an exact match to After Effects or Photoshop.
## What Primary Grade controls do
Controls work on display-referred code values (0 black, 1 white), in this order:
| Control | Range | Effect |
|---|---|---|
| `temperature` | −1..1 | Red ×2^(0.5·T), blue ×2^(−0.5·T). +1 makes red one full stop stronger than blue (warmer). |
| `tint` | −1..1 | Green ×2^(0.5·t), red and blue ×2^(−0.25·t). Positive is greener, negative more magenta. |
| `exposure` | −5..5 | Stops: multiplies RGB by 2^stops. +1 doubles code values. |
| `contrast` | −2..2 | Stops of slope around code value 0.5. +1 doubles the distance from mid-gray. |
| `blacks` / `shadows` | −1..1 | Adds up to ±0.25 code value to pixels with luma below 0.25 / 0.5. |
| `highlights` / `whites` | −1..1 | Adds up to ±0.25 code value to pixels with luma above 0.5 / 0.75. |
| `saturation` | −1..1 | Scales color about Rec.709 luma by 1+s. −1 is grayscale. |
| `vibrance` | −1..1 | Like saturation, but strongest on muted colors. |
In SDR rendering, the grade's output is contained to 0..1. In HDR rendering,
the SDR-domain grade is applied while excess highlight light carries through:
darkening can reduce highlights, but brightening cannot raise them beyond the
SDR-domain ceiling. Use exposure for overall brightness and tonal controls for
one tonal range. This is the stable released behaviour.
**SDR-domain effect colors: 0..1** — Tint/TintTritone shadow, midtone and
highlight RGB, Hue/Saturation Colorize scalar controls, and similar engine color
shaders remain in `SdrDisplay` in HDR. Their validation/output clamps are
unchanged: widening cannot create light above SDR white. Content-painting
Fill/solid paint and layer-style colors retain the extended paint contract.
Grades are not the extended authored-paint contract: shapes, text, style colors, gradients and
backgrounds accept encoded-sRGB RGB 0..64 with alpha 0..1; HDR continues the
transfer function and SDR ratio-fits headroom. Native Adobe export clips paint
RGB per channel to 1 with diagnostics. Custom-shader controls are scalar and
remain subject to their declarations/WGSL; a `colorR/G/B` control can declare
0..64, but its name does not trigger automatic transfer conversion.
## Workflow
Shot-to-shot consistency is part of grading, even when the user only asks for
a look. Consistency means a coherent scene, not identical pixel values. Keep
intentional differences such as shade, backlight, practical lighting and changes
of location or time of day. Keep the work within the requested scope; a local
correction does not require regrading unrelated scenes.
1. **Inspect and group.** Open representative source frames from every shot.
A file or Video layer may contain many shots; do not treat a baked edit as
one shot. Identify cut ranges, then group related shots by environment,
lighting conditions and subject, including shots separated in the edit.
The same location can have different lighting; do not group by filename or
proximity alone. Briefly note the relevant references, differences to retain
and corrections needed; no particular table or report format is required.
For a genuine single shot, skip cross-shot matching.
2. **Normalize and measure.** Choose color management and, for log footage, set
its Input Transform before judging correction. Measure brightness, clipping
and useful neutral/skin regions; retain source frames for before/after checks.
A neutral-colored object under colored light need not appear neutral.
3. **Set a scene reference.** Pick a representative, usable shot for each group,
not an unusually dark, clipped or atypically lit frame. Balance exposure and
white balance with `primaryGrade`, then establish the intended key look with
wheels, curves or a Creative LUT. Lower a LUT's `mix` rather than stacking
corrections to undo it. Record the reference shot and sample time briefly.
4. **Match the group.** Compare each shot with the scene reference, then turn
observed mismatches into shot-local corrections before the look. Use
[shot-local control](#baked-edits-and-shot-local-control) even when the input
is a single imported video. A shared creative look does not perform shot
balancing. Leave a shot unchanged when comparison supports that decision,
not merely because it shares a layer or preset with the reference. Compare
corresponding subjects under comparable light using [Matching shots](#matching-shots);
preserve motivated lighting differences instead of equalizing every shot.
5. **Refine and protect.** For skin, sky or a product color, add a qualifier
when needed and inspect an exaggerated correction to verify its selection.
Compare source/output shadow detail and noise at full resolution. If the
grade crushes useful detail or exposes noise, revise that shot's correction
or look strength before reaching for denoise; do not trade useful detail for
a match. Disclose any remaining problem the tools cannot resolve.
6. **Review together and in sequence.** Open a side-by-side filmstrip of every
group's graded shots, including nonadjacent ones, then review playback across
cuts for unintended exposure, white-balance or saturation jumps. Inspect
multiple moments when lighting changes within a shot. Correct outliers,
re-render and repeat on the final export. Numbers support this review; they
do not replace looking at frames or justify removing intentional contrast.
Ask before a strong stylistic change the user did not request. Keep corrections
subtle unless asked otherwise. Apply consistency checks to each delivered look,
not just one variant.
## Baked edits and shot-local control
A baked edit has cuts inside one media asset. "Keep the existing edit" means
preserve the visible frames, cut times, framing and audio; it does not require
keeping one Video layer. A static grade on that layer affects every shot.
Choose native controls to suit the edit. Correct an existing shot layer, use
[separate playback windows](native-authoring.md#shot-local-corrections-without-recutting)
for a plain baked Video, or use supported time-keyed corrections. At cuts,
avoid interpolating corrections across unrelated shots. A shared creative look
can sit on an Adjustment layer, with shot-local balance applied before it.
A single shared grade is sufficient when visual comparison shows the shots
already work together; do not split layers merely to follow this workflow.
When local windows are needed, inspect around candidate cuts to confirm their
boundaries; a sparse filmstrip or scene-change score can miss short inserts or
mistake movement for a cut. Preserve the source clock, framing and audio, and
compare the reference and corrected shot together afterward. Do not render
replacement media or rebuild the edit just to obtain local color control.
## Measuring the picture
Render the saved document with `preview` (one full-resolution frame) or
`filmstrip` (several frames; set the tile size to the project's
aspect ratio, for example 135×240 for portrait; a solo strip isolates one layer
over its active window, with transparent pixels around it), then measure the
PNG with tools available on the host, for example Python. Look at the frames
as well; numbers do not replace seeing the picture.
```sh
tsrct preview --project project.tsrct --time 2.5 --output /absolute/path/edit/.tesseract-work/checks/grade-2500.png
```
SDR preview and filmstrip PNGs are 8-bit display-referred SDR
Rec.709 code values. For HDR PNGs, PNGs whose transfer is unknown, HDR export
or `hdrGraphicsWhiteNits`, read [HDR](hdr.md). Divide SDR values by 255
for normalized values (0 black, 1 white). Pixels with alpha 0 are empty canvas:
exclude them, or footage that does not fill the frame reads as black.
| Measurement | How | Typical check |
|---|---|---|
| Brightness | Luma `Y = 0.2126 R + 0.7152 G + 0.0722 B`; mean and 1st/50th/99th percentiles | Compare source/output and comparable scene regions; no universal mean target |
| Clipping | Share of pixels with any channel at 255 (or all channels at 0 for crushed shadows); luma alone misses single-channel clipping | Above about 0.01 is usually visible |
| Color cast | `cb = (B − Y) / 1.8556`, `cr = (R − Y) / 1.5748` averaged over a region | Near 0 for something neutral; about ±0.02 is noticeable |
| Saturation | Mean `sqrt(cb² + cr²)` (chroma). `primaryGrade.saturation` scales this chroma linearly; HSV saturation does not change by the same ratio | Compare shots when matching |
| Hue for a qualifier | HSV hue (degrees) and saturation of the region's mean color | The same definitions the qualifier uses |
### Source-relative neutral and skin checks
Identify a genuinely neutral region and a skin region on the **original source
frame**; record their coordinates, source/edit timestamps and any mapping for
crop or scale. Sample the same regions on the output at the same source moment,
away from cuts and fades. Use full-resolution frames, not resized filmstrip tiles.
Decode each pixel using all four signalled stream properties (ffprobe
`color_range`, `color_space`, `color_transfer`, `color_primaries`), in this order:
1. Normalize code values according to `color_range` (limited or full), including
the chroma offset and scaling.
2. Apply the `color_space` matrix coefficients to convert Y′CbCr to nonlinear
R′G′B′ (for example BT.709 vs BT.2020 non-constant luminance).
3. Invert `color_transfer` to get linear RGB.
4. Use `color_primaries` for the RGB→XYZ conversion, then convert to CIE Lab
with D65 white.
A color-managed decoder that honours all four properties is also acceptable. Matrix
coefficients can be signalled independently of the primaries, and getting the
order or the matrix wrong skews Lab and ΔE. For SDR use a 100-nit
reference white; for HDR decode PQ to absolute nits (formula in
[HDR](hdr.md#hdr-preview-and-measurement)) or HLG with a documented reference
display/system gamma, not an SDR transfer. Use the same documented
Lab reference white for both sides. Report component-wise **median per-pixel
Lab**, not mean Lab or Lab of mean/median encoded RGB. Report neutral a*/b*
(target |a*| and |b*| ≲ 5), skin hue
`atan2(b*, a*)` in degrees (typically ≈ 40–70°, undefined near zero chroma),
and ΔE76 between source/output median Lab vectors. Keep the region, statistic,
transfer assumptions and ΔE definition identical across checks.
For these source-preservation checks, compare against the original source,
**not an earlier preview of your own grade**. This is separate from comparing
graded shots against the selected scene reference for consistency. Preview checks
are provisional: repeat on frames decoded from the final export. Different
statistics or baselines can pass a self-check while independent checks fail. If a task specifies a different frozen statistic (for example mean
per-pixel ΔE), report that separately; do not substitute this median recipe.
Source-HDR → SDR ΔE includes delivery tone mapping, not just grading changes;
disclose that limitation rather than claiming a failed preservation target passed.
### Neutralizing a cast
For an initial code-value gain estimate only (not the Lab acceptance check),
average a region that should be neutral (white or gray wall, paper, gray card),
measured where the `primaryGrade` will apply: before any later look or LUT
(temporarily disable later effects in a working copy if needed). With that mean
color `R, G, B` (all strictly positive), the white-balance gains above give the
exact neutralizing values:
```text
temperature = log2(B) − log2(R)
tint = (4/3) · ((log2(R) + log2(B)) / 2 − log2(G))
exposure = log2(target / Y') (optional; Y' is the luma after the gains)
```
Both `temperature` and `tint` are limited to −1..1; a stronger cast needs a
second correction or a different approach. Add the values to the
`primaryGrade` (keep its other fields), render again, and confirm `cb` and
`cr` near 0.
In an ACES document the gains act on scene light, while `preview` PNGs show
output-transformed display values. The formulas then give an estimate, not
exact values: apply them, re-render, re-measure the same region, and refine
with the remaining difference. The same applies to the shot-matching formulas
below. Because the gain is constant in scene light, a neutral patch that
measures neutral stays neutral across the patch's brighter and darker areas.
### Matching shots
Choose the scene reference and the lighting differences to preserve using the
[workflow](#workflow) before calculating corrections. Compare corresponding
regions under comparable illumination (the same wall or person's skin, for
example), not merely any two regions labeled "skin" or "sky". Do not force a
shaded subject to match a sunlit one. If no comparable region exists, use visual
scene context and conservative adjustments rather than forcing a numeric match.
For regions that should match, measure representative times in both shots. For
source `s = (Rs, Gs, Bs)` and reference target `t = (Rt, Gt, Bt)`, use these
source-to-target corrections, not the neutralizing formulas above. These are
balance estimates before the look, not a way to infer a complete creative grade
from a final styled reference:
```text
temperature = log2(Bs/Rs) − log2(Bt/Rt)
tint = (4/3) · (((log2(Rs) + log2(Bs))/2 − log2(Gs))
− ((log2(Rt) + log2(Bt))/2 − log2(Gt)))
exposure = log2(Yt / Ys')
```
`Ys'` is source luma after these white-balance gains; `Yt` is target luma.
The gains match channel ratios; exposure then matches brightness. Identical
source and target give zero correction; a neutral target reduces to the
neutralizing formulas. A colored target preserves the reference's color
rather than driving the source toward gray.
Use strictly positive, unclipped straight RGB code values measured at the same
SDR grading stage, before later looks/LUTs. Zero or negative channels make the
logarithms undefined; very dark samples are unstable. These formulas describe
white-balance/exposure gains only, with other grading controls neutral. If
measuring an already white-balanced/exposed source, the values are adjustments
to those existing controls, not replacements; nonneutral contrast, tonal or
saturation controls require separate measurement/verification. Final
`temperature` and `tint` must each stay in −1..1 and `exposure` in −5..5;
out-of-range corrections or output clipping prevent an exact match. Re-render
and verify instead of assuming clamped values match.
Compare whole-frame percentiles only when both content and illumination are
comparable; otherwise their distributions should differ. For unwanted residual
tonal differences, `colorCurves` can help, but do not mechanically align channel
percentiles: that can erase motivated contrast or alter colors. Re-render and
compare the final look side by side and across cuts, checking shadow detail,
highlights and noise against the source. Matching cannot recover clipped detail;
a sky shot and an interior need not share a brightness distribution.
## Creative LUTs
Import a 3D `.cube` file as a LUT, then reference its asset ID:
```sh
tsrct project import-asset --project project.tsrct --file /absolute/path/look.cube --asset-id film-look --kind lut
```
```json
[{"type":"addFxLayerEffect","compositionId":"main","layerId":4,"effectId":20,
"effect":{"type":"lookTransform","semanticVersion":1,"mix":0.7,
"transform":{"type":"lut3d","assetId":"film-look",
"inputEncoding":"jerboa:sdr-rec709-display:v1",
"outputEncoding":"jerboa:sdr-rec709-display:v1",
"interpolation":"trilinear"}}}]
```
`"transform": null` is a Creative LUT with no LUT yet; it changes nothing.
Replace the look with `updateFxLayerEffect`; keep `mix` unless asked to change it.
Only use LUT files the user supplied or is licensed to use.
## Log footage (Input Transform)
Footage import reports `cameraEncoding`. `"camera:apple-log:v1"` means
the file declares Apple Log; `null` means unknown, not "not log" (exported
copies often drop the declaration). Confirm a flat, low-contrast look in a frame
before treating unknown footage as log.
Registered camera labels (`CAMERA_INPUT_ENCODINGS`):
| `inputEncoding` | Curve / gamut expected by the vendor LUT |
|---|---|
| `camera:apple-log:v1` | Apple Log / Rec.2020 |
| `camera:sony-slog3-sgamut3-cine:v1` | Sony S-Log3 / S-Gamut3.Cine |
| `camera:sony-slog3-sgamut3:v1` | Sony S-Log3 / S-Gamut3 |
| `camera:panasonic-vlog:v1` | Panasonic V-Log / V-Gamut |
| `camera:canon-log3-cinema-gamut:v1` | Canon Log 3 / Cinema Gamut |
| `camera:fujifilm-flog2:v1` | Fujifilm F-Log2 / F-Gamut |
| `camera:nikon-nlog:v1` | Nikon N-Log / Rec.2020 |
| `camera:dji-dlog-m:v1` | DJI D-Log M / D-Gamut |
| `camera:arri-logc3:v1` | ARRI LogC3 (EI 800) / Wide Gamut 3 |
| `camera:arri-logc4:v1` | ARRI LogC4 / Wide Gamut 4 |
| `camera:blackmagic-film-gen5:v1` | Blackmagic Film Gen 5 / Wide Gamut Gen 5 |
| `camera:red-log3g10:v1` | RED Log3G10 / REDWideGamutRGB |
These labels do not detect or convert the footage automatically.
In an ACES document, do not use a LUT: declare the camera with `cameraInput`
(see [Camera log footage in ACES](#camera-log-footage-in-aces)). The rest of this
section is for legacy documents.
No camera conversion LUTs are bundled. Use the camera maker's
official conversion `.cube` (for Apple Log, Apple's Apple Log to Rec.709 LUT):
one the user supplies, or the maker's free download when you have network
access. Never author or approximate a conversion LUT yourself. Import it as a
LUT, then set it on the layer:
```json
[{"type":"updateLayerField","compositionId":"main","layerId":4,"propertyName":"inputTransform",
"value":{"semanticVersion":1,"transform":{"type":"lut3d","assetId":"apple-log-709",
"inputEncoding":"camera:apple-log:v1",
"outputEncoding":"jerboa:sdr-rec709-display:v1",
"interpolation":"trilinear"}}}]
```
Use `"value": null` to remove it. If no conversion LUT is available, say so
instead of approximating log conversion with contrast and saturation.
## ACES (scene-referred color management)
By default a document is `legacyDisplay`: everything below in this guide that
says "display-referred SDR" describes that mode. A document can instead opt in
to ACES. Then footage is treated as scene light, composited in linear ACEScg,
graded on ACEScct, and shown through the ACES 2.0 output transform. That output
is SDR BT.1886 by default, and HDR 1000-nit (P3-D65 limited) in a PQ or HLG
export.
**Choose ACES** when footage is camera log or HDR, when highlights should roll
off filmically instead of clipping, when glows, dissolves and light leaks should
add like light, or when one grade must serve both SDR and HDR deliveries. **Keep
legacy** for graphics-led edits that need exact sRGB brand colors, or when the
user did not ask for a cinematic or managed look. Switching changes the picture
of every existing grade, so decide before grading.
### Turning it on
`colorManagement` is a document setting. Set it with an action batch:
```json
[{"type":"setColorManagement","colorManagement":{"workingSpace":"acescct","outputTransform":"aces2.0","compositing":"footageOnly"}}]
```
Use `{"workingSpace":"legacyDisplay"}` to return to legacy. Editing the root
`colorManagement` field in the document JSON also
works. ACES requires both `outputTransform` and `compositing`; omit them in
legacy mode. Use only fields and values supported by the installed schema.
| `compositing` | What renders in ACES |
|---|---|
| `footageOnly` (start here) | Video, Image and PAG-image footage, plus Groups marked `renderDomain: "scene"`. Text, shapes and graphics stay display-referred, so white text stays 255. |
| `perClip` | Only layers and Groups marked `renderDomain: "scene"`. |
| `wholeProgram` | Everything, including graphics. sRGB white 1.0 renders at about 184/255 in SDR, and `renderDomain: "display"` is ignored. |
`renderDomain` overrides one Video, Image or Group: `"scene"` puts it, and for a
Group its whole subtree, in ACES; `"display"` keeps that layer legacy (for
example a graphic on top of footage). Set it with an action batch:
```json
[{"type":"updateLayerField","compositionId":"main","layerId":4,"propertyName":"renderDomain","value":"scene"}]
```
Use `"value": null` to inherit the preset. Put footage plus the glows, blurs,
dissolves and light layers that should mix as light in one `"scene"` Group.
A `"display"` override inside a scene Group cannot take the child out of ACES;
place display graphics outside that Group. Legacy mode ignores these overrides
but retains them when switching color-management modes.
### What changes inside ACES
- **Light effects** (compositing, blend modes inside the Group, blurs, glow,
halation, the `exposure` effect, `vignette`, luma mattes) work on linear
scene light. Both `primaryGrade.exposure` and the separate `exposure` effect
adjust scene light in stops: +1 doubles it and -1 halves it, including near
black and above white. `primaryGrade` temperature/tint are the same
documented gains applied to scene light, so a correction shifts shadows,
midtones and highlights equally. Primary Grade applies exposure and white
balance before its other controls.
- **Other grade controls** (`primaryGrade` contrast, tone and saturation;
`tonalColor`, `colorCurves`, qualifiers, keys and `lookTransform`) run on
ACEScct log values. Values tuned for legacy can look stronger. Curve and
channel-mixer weights multiply log values, changing contrast rather than
applying uniform scene exposure. Start from small values and judge by
measured frames.
- **Authored colors** (solids, text, gradients) are sRGB values brought into
ACES, so sRGB white is not SDR white under `wholeProgram`.
- **Above-white light** (RGB above 1, glows) rolls off
smoothly through the output transform instead of clipping.
### Practical ACES notes
- **SDR footage tops out at scene 1.0.** Rec.709 clips enter ACES with highlights
at linear 1.0, so they look dimmer and flatter than in legacy, and neon or sky
cannot bloom or roll off by itself. Add a few tenths of a stop with the
`exposure` effect per shot, and/or lift highlights with a curve, before glow
or halation.
- **Offsets are exposure-like; gains change contrast.** Adding a constant to
an ACEScct channel shifts it by constant stops only while both values remain
in the unclipped log branch: 1/17.52 ACEScct per stop. This equivalence does
not hold in the linear toe near black. Multiplying ACEScct values (gain or
diagonal channel-mixer weights) changes log contrast, producing input-dependent
stop shifts; cross-channel weights also mix colors. For uniform stop
adjustments, use `primaryGrade.exposure`, or the separate `exposure` effect
with `offset: 0` and `gammaCorrection: 1`. Both multiply scene-linear RGB by
2^exposure without clipping to white, including signed values. Alpha is
preserved; the finite half-float storage limit still applies. The separate
effect's offset and gamma operate on straight, not premultiplied, scene RGB.
- **Luminance-preserving desaturation darkens saturated colors.** Lowering
`luminanceVsSaturation` or `hueVsSaturation` on bright reds keeps their Rec.709
luminance, which is low for red, so they get much darker.
- **Grain** runs on ACEScct values inside ACES, like the grades; tune its
amount by eye there.
- **HDR.** `primaryGrade`, `tonalColor`, `colorCurves`, the three color tools
and looks render in HDR export too: inside ACES on ACEScct values; in legacy
documents in the SDR display domain with highlights carried through.
### Looks (LMT)
A Creative LUT inside ACES must map ACEScct to ACEScct (an ACES LMT):
```json
[{"type":"addFxLayerEffect","compositionId":"main","layerId":4,"effectId":25,
"effect":{"type":"lookTransform","semanticVersion":1,"mix":0.8,
"transform":{"type":"lut3d","assetId":"aces-look",
"inputEncoding":"aces:acescct:v1",
"outputEncoding":"aces:acescct:v1",
"interpolation":"trilinear"}}}]
```
`addFxLayerEffect` accepts this only in an ACES document. An ordinary
`jerboa:sdr-rec709-display:v1` look is refused inside ACES; put it on a layer or
Group with `renderDomain: "display"`. Only use LUTs that are actually built for
ACEScct.
### Camera log footage in ACES
Do not import a vendor LUT. Declare the camera, and ACES applies the camera's
published log curve and gamut from the ACES reference config:
```json
[{"type":"updateLayerField","compositionId":"main","layerId":4,"propertyName":"inputTransform",
"value":{"semanticVersion":1,"transform":{"type":"cameraInput","inputEncoding":"camera:sony-slog3-sgamut3-cine:v1"}}}]
```
`cameraInput` supports Apple Log, ARRI LogC3 (EI 800) and LogC4, Blackmagic
Film Gen 5, Canon Log 3, Panasonic V-Log, RED Log3G10, and Sony S-Log3
(S-Gamut3 and S-Gamut3.Cine). Fujifilm F-Log2, Nikon N-Log and DJI D-Log M have
no ACES input transform: keep their vendor `lut3d` Input Transform and set the
layer to `renderDomain: "display"`. A `lut3d` Input Transform with a supported
camera encoding also works in ACES (the LUT is ignored there), but `cameraInput`
is the clearer form. `cameraInput` is refused in legacy documents.
### Checking ACES pictures
Measure the exported frames as usual. Reference points in SDR for colors that
pass through ACES: sRGB grey 0.5 renders at about 106/255, sRGB white 1.0 at
about 184/255, and RGB 4.0 at about 251/255 (rolled off, not clipped). Graphics under `footageOnly` keep their legacy values. In HDR
export, ACES content peaks near 1000 nits. Typed errors name the fix: a display
look or unsupported camera inside ACES, an ACES look outside ACES, or HDR/log
media whose display rendition cannot enter ACES.
## Secondary selections
Add `compositingOptions.qualifier` to a `primaryGrade`, `tonalColor`,
`colorCurves`, or `lookTransform` effect to apply it only to matching pixels.
Hue is in degrees (0–360); saturation and luminance are 0–1 with a soft edge;
`refine` cleans the selection (`denoise` 0–10, `blurRadius` 0–100, `choke`
−50–50); `invert` selects everything else. `maskReferences` further limits the
correction to the layer's masks, and `effectOpacity` blends it. Use
`setFxLayerEffectCompositingOptions` to change them. For example, to halve the
saturation of a blue sky only:
```json
[{"type":"addFxLayerEffect","compositionId":"main","layerId":4,"effectId":21,
"effect":{"type":"primaryGrade","semanticVersion":1,"temperature":0,"tint":0,"exposure":0,
"contrast":0,"highlights":0,"shadows":0,"whites":0,"blacks":0,"saturation":-0.5,"vibrance":0},
"compositingOptions":{"effectOpacity":1,"maskReferences":[],
"qualifier":{"semanticVersion":1,"hue":{"center":215,"width":40,"softness":10},
"saturation":{"low":0.3,"high":1,"softness":0.05},"luminance":{"low":0,"high":1,"softness":0},
"refine":{"denoise":0,"blurRadius":0,"choke":0},"invert":false}}}]
```
Measure the target's HSV hue and saturation first and center the range on them.
Dedicated secondary matte preview is not available in the current engine.
Inspect the selection with this visual diagnostic instead; do not
promise a matte-only readback.
Mask references identify masks on the same layer; use mask mode `none` for an
effect-only mask. Unknown mask IDs are rejected by authoring.
Check the selection before judging the correction: temporarily exaggerate the
correction (for example `saturation: -1` or a large exposure change) in a
working copy, render `preview`, and compare it with an unchanged frame. The
affected area is the selection. Widen or soften the range when the subject is
patchy; narrow it when the background leaks in. Then set the intended values.
### Keeping skin natural under a look
Apply the look, then protect skin with a secondary: start with HSV hue about
15–60° plus saturation and luminance ranges measured on the actual skin.
These HSV selection angles are not the Lab hue angles above. Use an inverted
skin qualifier on the look to move the background while excluding skin, or add
a qualified skin-restore correction after the look. Tune softness and inspect
an exaggerated correction for leakage; qualifiers do not track subjects.
Verify source-relative skin ΔE76 ≤ about 10–15 with the median Lab recipe above,
plausible skin hue, and a measurable background change at several timestamps.
A plausible hue alone does not prove preserved skin brightness or color.
If delivery tone mapping prevents the preservation target, report it explicitly.
## Highlight finishing
Add `halation` to a shot's Video layer or a finishing Adjustment layer after
the grade for a warm highlight halo:
```json
[{"type":"addFxLayerEffect","compositionId":"main","layerId":4,"effectId":26,
"effect":{"type":"halation","threshold":0.8,"radius":24,"intensity":0.5,"tintRed":1,"tintGreen":0.25,"tintBlue":0.1}}]
```
The six controls shown are animatable. Optional `saturation` defaults to 1;
0 makes the halo neutral. `threshold` is relative to SDR white (1), `radius`
is in composition pixels, and `intensity: 0` bypasses the effect. In ACES,
halation works on scene light and retains above-white highlights. Start subtly
and inspect bright edges and skin. Native After Effects/Premiere conversion
omits halation with a diagnostic.
## Noise reduction
`noiseReduction` is spatial only. Use **one instance**; do not stack instances
to chase a noise score: repeated passes smear detail before removing more noise.
Compare the same low-texture regions and edge measurements against the original
source, using identical decoding, scale and statistics on the final export.
Check edge retention (for example ≥ 85% of source edge magnitude) and visually
inspect face texture, hair and clothing; a lower spatial residual alone is not
proof of successful noise removal. If one instance cannot meet both noise and
detail targets, report the limit. Temporal noise reduction is unavailable; do
not invent a temporal field.
## Final-export picture checks
Decode and inspect several timestamps of the exported file, safely inside each
shot and away from cuts/fades, with matching original-source frames. Sources
may fade in from black; encoder timestamp rounding near a fade boundary can
select a neighboring black frame. Confirm actual frame timestamps and move
sampling into stable picture before diagnosing missing footage. Never deliver
if a frame that should show picture is black; investigate rather than dismissing
it as an HDR viewing artifact. Verify footage as well as overlays.
## Limitations
- In legacy documents grades remain SDR-domain in HDR export: highlights carry
through but cannot be brightened above the grade's SDR ceiling. Use ACES for
scene-referred grading with a managed SDR/HDR output.
- No tracking; masks and qualifiers do not follow subjects automatically.
- No built-in automatic shot-matching command: the agent must compare frames
and author native shot-local corrections where needed. This is not a limit
on correcting individual shots inside a baked edit.
- No Dolby Vision or MaxCLL metadata export.
- Noise reduction is spatial only; temporal NR is absent.
- HDR images are unsupported on web.
- Camera detection reads metadata and recognizes Apple Log only; declare other
cameras explicitly. In legacy documents a camera encoding label declares what
the supplied vendor LUT expects, not a built-in conversion: use official vendor
LUTs only, never approximate one. In ACES, `cameraInput` is the built-in
conversion for the supported cameras.
- LUTs must be 3D `.cube` files; measure representative frames and review visually.
references/conversion.md
# Convert files from other editing tools
The separate `tsrct-conv` CLI converts between `.tsrct` and other editing tools’
project formats, such as Adobe Premiere Pro and After Effects. Check the
[Tesseract Converter repository](https://github.com/mirage-hq/Tesseract-Converter)
for current supported formats and conversion directions, installation, usage,
releases, and known limitations. The converter is installed and versioned
separately from the Tesseract plugin and `tsrct` rendering CLI.
The converter also turns a Blender `.blend` into the model glb that Scene3D
renders (`tsrct-conv model scene.blend --output scene.glb`). A 3D-enabled
`tsrct` accepts the `.blend` directly in `project import-asset --kind model3d`,
so use the converter only when you need the glb itself. The Tesseract: 3D
Scenes skill places the imported model.
Conversions may omit or approximate features, so the result can look or behave
differently from the source. Use conversion diagnostics and the repository’s
support documentation to understand those differences. A later converter release
may improve the result without a schema or renderer change.
For compatibility questions or unexpected output, see
[versions and compatibility](versions.md). Keep the original source when converting.
references/critic-instructions.md
# Critic instructions
You review a deliverable in the role and with the focus your prompt gives,
judging it against the brief from the evidence your prompt lists.
## Review the evidence
- Read the brief first, and `words.md` if your prompt lists it.
- Open every listed image with your image-viewing tool before scoring, since a
file name is not evidence. If you cannot view images, reply
`Cannot view images` with no score.
- Open the images a few at a time, in parallel when you can, each batch grouped
by subject, such as the overview strip with its denser strips, or one design's
views. Note in one line what each image shows before the next batch, and score
only after every image is open.
- Open only the files your prompt lists and the frames you render, and write
only to the folder your prompt gives.
- Do not load the Tesseract skills or their references, and do not run any
command that changes the project or the deliverable.
- Quick measurements are fine when an issue or score depends on one, such as a
contrast ratio, a luma or color sample across a cut, sharpness, or a safe-area
overlay; cite the number as the issue's evidence.
- Score only what the evidence shows, not the effort behind it.
## Render more frames
Render only for a specific question the supplied views leave open, and only when
your prompt gives the `tsrct` executable and the document. Add the flags your
prompt gives to each command that accepts them, so your frames match the
supplied views. Put every time you need in one filmstrip call, at the strip tile
size your prompt gives:
`<tsrct> filmstrip --project <document> --timestamps-ms <ms,...> --tile-width <w> --tile-height <h> --output <folder>/<name>.png`.
For a frame that needs full-size detail, use
`<tsrct> preview --project <document> --time <seconds> --output <folder>/<name>.png`,
and crop it with any available image tool if you need a closer look, keeping
each crop within about 1500 px on its long side and 1 megapixel, or the host's
image limit if lower, so it is viewed unscaled. Keep times inside the piece: a
time exactly at its end shows an inactive end state, not the final frame. Open
the results together. Render one more batch only for a new, specific question
they raise; otherwise list the question under Not evaluated.
## Score
- **10:** nothing meaningful left to improve for this brief.
- **9:** a professional in this role would publish it as is; the notes are polish.
- **8:** ready for the audience, with a few noticeable flaws worth fixing.
- **7:** competent, but with problems the audience would notice or that this role
would send back.
- **5:** works technically but misses the brief's purpose or has serious craft
problems.
- **3 or below:** broken, misleading, or unusable.
## Reply
Reply in this form:
```text
Score: <0–10, one decimal place>
Verdict: <one sentence>
Issues, most important first (at most five): <time or location> — <problem> — <evidence> — <suggested fix>
Keep: <what works and should survive a revision>
Needs the user: <issues that only a change to a fixed choice or new input from the user would fix>
Viewed: <the ID of each evidence file you opened, one by one, not as a range, and the paths of frames you rendered>
Not evaluated: <what the evidence could not show>
```
For a set of designs or variants, review the whole set in one pass and reply with
one block per design, named by its file (Score, Verdict, up to three Issues, Keep,
Needs the user), then one line on consistency across the set and the Viewed and
Not evaluated lines. When the designs serve different channels, judge each
against its own channel.
references/critic-review.md
# Critic review
Before you present a finished deliverable, have two independent critics score it,
and revise between rounds until both rate it at least 8.5/10, for at most three
review rounds. This applies to new work and to revisions; each new request from
the user gets its own rounds. A preview shared only to agree on a direction does
not need a panel; the result you deliver does. The panel runs once, on the piece
you deliver: when you build part of a larger piece with another Tesseract skill,
such as a 3D scene for a motion graphic, that part is reviewed within the larger
piece. Skip the panel only when the user explicitly asks you to; then say that the
result was not reviewed and give no score.
The panel adds to the checks the skill already requires. It does not replace
technical checks such as export validity, alpha, loudness, or font rendering, or
your own playback and listening. Finish those first so the critics judge the work
rather than a broken render.
## Choose the panel
- **Art director:** judges craft against the brief and the supplied references:
the idea, composition, hierarchy, typography, color, motion and pacing,
consistency, and finish.
- **Specialist:** an expert in what the deliverable must achieve for its audience
and channel. Choose the role from the brief, for example:
| Deliverable | Specialist and focus |
|---|---|
| YouTube video | YouTube strategist: hook, retention, pacing, payoff, call to action |
| Paid social ad or short-form video | Performance creative strategist: the opening seconds, sound-off clarity, offer, call to action |
| Product launch or brand film | Product marketing lead: message, proof, positioning |
| Explainer or tutorial | Instructional designer: clarity, sequence, what the viewer can do afterward |
| Interview, documentary, or event edit | Story editor: structure, continuity, emotional arc |
| Titles, lower thirds, or overlays on footage | Broadcast designer: legibility over picture, safe areas, timing with the cut |
| Data animation or diagram | Information designer: accuracy, clarity, honest scales |
| 3D product shot or scene | Product visualization lead: form, materials, lighting, camera |
| Poster or print-style graphic | Campaign designer for its venue: legibility at distance, message at a glance |
| Social graphic | Social media strategist for that platform: stopping power in the feed, crop safety, copy |
| Slide layout | Presentation designer: one idea per slide, readability in the room |
| Web or app mockup | Product designer: task flow, hierarchy, accessibility, responsive layout |
| Custom browser editor | Creative-tool UX designer: workflow, feedback, error states, saving |
When the channel is unclear, choose the specialist whose success criteria the
brief states. Keep the same two roles for every round.
## Prepare the evidence
The critics' folders and your review notes go in the working folder.
That is the project's `.tesseract-work/`.
For each user request, before its first round, create or empty
`critic-input/<deliverable>/` there, copy
[critic-instructions.md](critic-instructions.md) into it unchanged (the critics'
rules, score anchors, and reply form, with no pass mark), and write the brief
there as `brief.md`; keep review notes and scores out of that folder. Reuse the
brief unchanged in every round unless the user changes the request; rebuild the
rest of the evidence from the current version each round, first removing earlier
views from that folder. Give both critics the same evidence, each file as an
absolute path, with names that do not reveal a version or round:
- **The brief:** the user's request in their own words; the audience, channel,
format, and duration or size; the choices the user fixed (such as music, copy,
colors, or footage), which are not to be changed; content the user has not
supplied, shown as placeholders; and the supplied brand assets and references
(logos, palettes, guidelines, fonts, reference images or videos), each marked
as required or for inspiration. Give a reference video as a few stills or a
strip, and name fonts rather than passing font files. List each reference
file, such as a logo, still, or guideline PDF, in the evidence list too,
labeled as a required or inspiration reference. For a social format,
name the platform's crop and safe areas. For a revision, give the original
brief when known and the user's change request, and list the parts the user
did not ask to change as fixed.
- **The deliverable:** the exported video, the rendered PNGs, or the editor's
source. Critics see a video through its views, so they need not get the video
file itself.
- **Views at review resolution:**
- Moving work: an overview filmstrip of the whole piece, about every 1–2 s for
short pieces and every 5–10 s for pieces over two minutes, plus denser strips
around the opening, the ending, each key reveal, and the cuts most likely to
show a join or carry the story. Add a full-size frame of each distinct scene
or look, typically one to three, for finish and defects anywhere in the
frame. For small text or fine detail in a known region, add 100% crops
instead of more full frames: cut them from full-size `preview` PNGs with any
available image tool, following the region if it moves, with one region's
crops at several times stacked or side by side in one sheet within about
1500 px on its long side and 1 megapixel, or the host's image limit if lower,
so it is viewed unscaled. Keep each strip to about 8–16 samples and the
whole set to about 20 images; for a long piece, widen the overview spacing
to stay within that, and favor the opening, the ending, and the riskiest
cuts.
- An overlay or graphic on longer footage: instead of the whole-piece
overview, its active window plus a second or two on each side, showing the
entrance, the hold over the busiest background, and the exit, with a
full-size frame of the hold.
- Static designs: each PNG at full size and downscaled to its intended viewing
size, such as phone width for a social graphic; for designs larger than about
2000 px on a side, 100% crops of text-dense regions made with any available
image tool, each within about 1500 px on its long side and 1 megapixel, or
the host's image limit if lower.
- Work with sound: the loudness measurements and waveform images (for an
overlay, of its window).
- An editor: screenshots of a loaded project, an edit, playback, a save, and an
error message you can trigger, such as opening an invalid file. If you cannot
capture the running editor, say so; the critics then review the code and
layout only.
- **Words and structure, for speech-led or narrated work,** written each round to
`words.md` in the input folder: what is said against the current edit times,
from a supplied transcript, the project's caption text, or words you verified
by listening, with unverified parts marked; which source clip and range plays
when; and the usable footage left out of the edit. Give facts only, not your
intent, your judgment, or how the audio sounds. Without a reliable transcript,
say so; the critics then list spoken content under Not evaluated.
- **Sources:** when the work uses the CLI, the `.tsrct` document, the resolved
`tsrct` executable, the supplied strips' tile size, matched to the
project's aspect, and the render flags the supplied views used, so a critic
can render more frames to its own folder (critic-instructions.md has the
commands). A critic that cannot run commands reviews the views you supplied.
Critics review stills and measurements; they cannot judge playback feel or sound
by ear, so your own playback and listening checks still apply.
Reuse views you already opened in your own checks of this version where they
fit, copied to a neutral name if theirs marks a version, round, fix, or
before/after.
Use the strip's stdout JSON to confirm `outputPath`, `frameCount`, and
dimensions. Inspect `resolvedSampling` when it is non-null; it is normally
null for explicit `--timestamps-ms` requests. Those samples follow the
requested times, while the visible labels are rounded to tenths of a second.
Open the strip to catch a blank or stale render and assess the actual image.
Open every view no JSON describes, such as a preview frame, crop, design view,
or screenshot, before listing it.
## Run the critics independently
Start each critic as a separate agent with fresh context (a subagent, task, or
worker, depending on the host), and run both at the same time when the host
allows. Neither critic sees the other's review, anything from earlier rounds
(scores, feedback, or what you changed), your own assessment or intent, or the
pass mark, and neither is told about earlier rounds. Its prompt holds only its
role and focus (with any scope note from the skill), these must-do lines, and the
evidence list:
```text
You are the <role>. Focus: <focus, with any scope note>.
First read in full: <input>/critic-instructions.md, <input>/brief.md, <input>/words.md.
Open every image listed below with your image-viewing tool before scoring, or reply `Cannot view images` with no score.
Open only the files listed here and frames you render; write only to <frames>.
Do not load the Tesseract skills or their references, or change the project or deliverable.
Reply in the form in critic-instructions.md, listing Viewed items by ID.
Evidence:
E1 <absolute path> — <what it shows>
Renderer: <tsrct> for <each document>; strip tiles <w>×<h>; flags <render flags, or none>
```
`<input>` is the critic-input folder and `<frames>` the critic's own folder in the
working folder, such as `critic-frames/<deliverable>/art-director/`, both absolute.
Name `words.md` only when you wrote it.
Omit the Renderer line when the work does
not use the CLI, and its strip tiles when you supplied no strips.
Label each
view by what it shows and its times, such as "100% crops of the title at 0:03,
0:05, and 0:07", never by version, round, or judgment. Replies give Score,
Verdict, Issues, Keep, Needs the user, Viewed, and Not evaluated, with one block
per design for a set.
If a critic's Viewed line leaves out the ID of an image you supplied of the
deliverable (its PNGs, strips, frames, crops, or screenshots) or of a required
reference, ask it once to open those images and update its reply. If the host
cannot start separate agents, or a critic still cannot view images, say so,
review the evidence yourself in that role from the brief and the evidence alone,
opening every view and following critic-instructions.md, and label that score as
self-assessed rather than independent. Use this only when the host cannot run
an independent critic, never to save time. A self-assessed score can end the
loop, but report it as self-assessed and never describe the work as
independently reviewed.
## Iterate
1. Assemble the evidence, run both critics, and report the round.
2. The panel score is the lower of the two critics' scores. For a set, each
design has its own panel score, and the set's panel score is the lowest of
them. If the panel score is at least 8.5, the panel passes; deliver that
version.
3. Otherwise, save a checkpoint of the version the critics reviewed, one per
design for a set (for an editor, a copy of its source), and name it in the
round note.
For a `.tsrct`, use `tsrct project checkout --project <document> --output Versions/<name>-<request>-round-<n>.json`,
where `<name>` is the document's base name and `<request>` a short label for
the user's request.
Then:
- If three rounds have run, or none of this round's issues can be fixed
within the user's scope, stop and deliver as in step 4. List the remaining
notes as suggestions, and ask about anything only the user can supply.
- Otherwise, revise. Open the view each note cites, fix the issues with the
most impact first, especially those both critics raise, and keep what they
list under Keep. Resolve conflicting notes in favor of the brief. Stay
within the user's scope: do not change choices the user fixed, invent
footage, claims, or assets, or add work the user did not ask for. When the
user names a choice loosely, such as "coral", you may tune the exact value;
say what you picked. Put anything only the user can supply, such as new
footage or approval to change the scope, in the round report as a question,
and keep fixing the rest.
- Re-render, repeat the skill's own checks, and start the next round with
fresh critics, the same roles, and the same brief. Never start a round on
an unchanged deliverable.
4. To deliver after a round that did not pass, choose the reviewed version with
the highest panel score; on a tie, the later one. If that is not the latest
version, restore the chosen checkpoint, re-render it, repeat the skill's
checks, and tell the user which later changes were dropped and which
checkpoint holds them. Restoring a reviewed checkpoint is not a change; its
scores still apply.
If anything else changes after the last scored round, for example to pass a
technical check, start a new round if one remains; the changed version's earlier
score no longer counts, so step 4 cannot choose it. If no round remains, tell the
user what changed since the reviewed version, and that the scores apply to the
reviewed version.
Report each score exactly as returned. Never edit a critic's score or reply,
coach a critic toward a higher score, or rerun a round or replace a critic to get
a better result. If a critic fails to return a review, restart it once with the
same evidence and say so; if it fails again, use the self-assessed fallback for
that role.
## Keep the user informed
Report the loop in short messages as it runs:
- **Before the first round:** the review plan: the two roles and why that
specialist fits, the pass mark of 8.5/10 from both critics, and the limit of
three rounds.
- **After each round:** each critic's score (for a set, per design) and main
feedback in a few concrete points with times or locations, then what you will
change next and any note you will not act on and why, or that the panel
passed.
- **At delivery:** the delivered version's scores from both critics and its
panel score (for a set, per design), naming its round; how the scores moved
across rounds; what you changed in response; feedback you did not address and
why; and what the critics could not evaluate. If the panel did not reach 8.5,
say so plainly and name what holds it back.
For example:
```text
Review plan: an art director and a YouTube strategist, each a separate agent,
will score the export; the strategist because the video must hold viewers on
your channel. It passes at 8.5/10 from both, with up to three rounds.
Round 1: art director 7.5/10, YouTube strategist 6.5/10.
Art director: the title at 0:03 collides with the speaker's face.
YouTube strategist: the first five seconds never show the promised result; the
call to action comes after most viewers have left.
Next: open on the result and move the title clear of the face. Not changing:
the call to action stays at the end, as you asked.
Round 2: art director 8.8/10, YouTube strategist 8.6/10. The panel passes.
Art director: the lower third at 2:10 sits slightly low on a phone.
YouTube strategist: the middle section, 1:20 to 2:40, could be tighter.
Final: art director 8.8/10, YouTube strategist 8.6/10, panel score 8.6/10 after
two rounds, up from 6.5/10. Not addressed: the call to action, as you asked; the
lower third and the middle section, left as suggestions. Not evaluated: the
sound by ear and the feel at full speed.
```
Scores are the critics' judgment of the evidence they saw. Present them as that,
alongside the checks you actually ran, not as proof of quality.
Keep a short note of each round's replies, with the version they reviewed, in
`review-notes/` in the working folder, where critics are never pointed. Empty each
critic's frame folder before its round, and delete the critic-input and frame
folders after delivery.
Keep the round checkpoints with the project so the user can return to a reviewed
version, and retire them later under the skill's storage guidance; checkpoints of
a detailed scene can be large.
references/custom-shader.md
# Custom shader effects
Author a new `customShader` when no supported native layer effect expresses the
requested visual operation. It is the general effect primitive underlying many
engine-provided effects.
## Model
```text
CustomShader
├── name stable upsert label
├── description agent/human-facing intent
├── wgsl fragment shader module
├── params[] scalar uniform inputs
└── textureInputs[] additional textures bound from layers
```
The owning layer's rendered output is the implicit primary texture.
## Minimal shape
```json
{
"id": 40,
"effect": {
"type": "customShader",
"name": "exampleTint",
"description": "Multiplies the layer RGB by an animatable amount.",
"wgsl": "...complete WGSL module...",
"params": [
{
"name": "amount",
"description": "Tint multiplier; 0 is unchanged.",
"min": 0,
"max": 1,
"default": 0.5
}
]
}
}
```
`name` is the upsert key, not the animation address. Animate a parameter using
the enclosing effect instance's stable `id` and the parameter `name`.
## Scalar uniform inputs
Each `params[]` entry declares one scalar `f32` input:
```text
name shader-local field and animator parameter name
description explain units and visual meaning
min/max authoring/interpolation range
default value used when no animator drives the parameter
```
Parameters are packed into the WGSL `Params` uniform in declaration order.
Keep the JSON order and WGSL struct field order identical. The runtime pads the
uniform payload to the required alignment.
Animate an ordinary effect parameter through the installed action schema's
`setFxPropertyAnimator` effect-property target, using the owning layer ID,
effect ID, and parameter name. Use `layerTimeJsCode` for a script; do not copy an
internal `PropertyTarget` or `animator.code` object into an action. Read
[motion](motion.md) and the installed action schema for the exact wire shape.
A shader that only needs a clock can declare the reserved `animationTime`
parameter instead, subject to support in the installed renderer.
## Parameter naming drives editor controls
Editors classify parameters by **name** and pick an After Effects-style
control. Prefer the explicit control suffix — a `.suffix` on the parameter
name declares the control outright, with no heuristics involved:
```text
<base>.colorR/.colorG/.colorB color picker — declare each channel 0..64 for extended-sRGB paint (alpha remains 0..1)
(read as normalized RGB); optional
<base>.colorA adds alpha
<base>.x + <base>.y [+ .z] 2- or 3-value row of paired inputs
<base>.angle / .direction / angle dial in degrees — all six render the
.rotation / .hue / .phase / same control; the word records the
.evolution parameter's meaning
<base>.toggle on/off switch writing 0 / 1
<base>.slider slider paired with a numeric input
<base>.number numeric input alone
anything else slider + numeric input when the declared
range is sweepable, else numeric input alone
```
The suffix never reaches the user: labels strip it (`wet.toggle` reads as
"Wet"; `tint.colorR` as "Tint R" inside the color group; `spin.rotation`
as "Spin"). Choosing among the angle words documents intent for the next
reader of the document — it does not change the control today. WGSL is unaffected —
struct field names are yours and packing is positional; only the JSON `name`
carries the suffix. Grouped suffixes need their sibling set complete
(`.colorR/.colorG/.colorB`; `.x` + `.y`); an incomplete group falls back to
numeric inputs.
The declared range drives the angle dial's write semantics. Declare exactly
`0`/`360` for a modular direction (the shader consumes it through
`sin`/`cos`, so a full turn returns to the same result): the dial and its
numeric input wrap — typing 450 lands on 90. Any other declared range —
including deliberately narrow ones like -45..45 — means the parameter is
consumed linearly (a signed twirl-style amount where -120 and 240 render
differently): both inputs honor the declared `min`/`max` and clamp, so
negative, extended, and narrow values stay exactly as authored. Omitting
`min`/`max` on an angle-suffixed parameter leaves the engine defaults of
exactly 0/1, which editors read as "no range declared" and treat as the
full-turn wrapping encoding — but declare `0`/`360` explicitly so the
intent survives review. Editors only infer a dial for un-suffixed names when
`max` ≤ 360 — multi-turn spans like a 0..720 shutter duration stay numeric
inputs.
Color channels are edited through an 8-bit hex swatch, so a swatch edit
quantizes each channel to 1/255 steps; the per-channel numeric rows keep
full float precision.
The declared range is what earns a plain scalar its slider: bounds must be
finite, `min` below `max`, and both within ±1000. The editor owns that
threshold — `SLIDER_BOUND_LIMIT` in its param classifier — because
`EffectParam` carries no control hint for the engine to be authoritative with;
this document tracks the editor, not the reverse. Past that the bound reads as a
safety cap on a coordinate or technical space (a ±5000-pixel offset, a 0..100000
iteration count) rather than an endpoint anyone sweeps toward, so those get the
numeric input alone. The pair exists because each control reaches what the other
cannot: the slider sweeps the whole range in one control width, the input hits
the exact values the slider's pixel resolution skips.
`.slider` and `.number` force that decision either way — a slider on a range
wider than the guard allows, or a plain input for a parameter meant to be typed
exactly. Unlike the other suffixes they name a widget rather than a meaning,
because a fraction, a level in pixels, and a sample count all want the same
control. Use them only to override; a well-declared range needs neither.
## Reserved parameter names
One parameter name is reserved by the engine: `animationTime`. A parameter
declared with that exact name is not authored, animated, or edited — the engine
overwrites its uniform slot every frame with the owning layer's animation clock
as an `f32` in seconds. It is the same value a `jsScript` effect-parameter
animator on that layer reads as `input.time.seconds`: the local
composition/group clock, rebased to the nearest enclosing Group's in-point and
clamped to that group's duration, with a Group's own effects using the group's
own clock. With no enclosing Group it is the composition playhead. Declare it and
the shader animates with no animator anywhere in the document.
The reservation is total:
- No editor control appears for it. It is filtered out of the effect's parameter
schema, so agents do not see it either.
- Writes are rejected: effect-parameter update actions refuse the name, and an
`effectProperty` animator cannot bind to it.
- `min`, `max`, and `default` are documentation only. The engine overwrites the
packed slot every frame whatever they say — declare a truthful range anyway so
the intended time domain survives review.
- The parameter must stay declared in `params[]`. Uniform packing is positional,
so dropping it or reordering around it shifts every later parameter's slot.
```json
{
"type": "customShader",
"name": "examplePulse",
"description": "Pulses the layer brightness on the layer's own clock.",
"wgsl": "...complete WGSL module...",
"params": [
{
"name": "animationTime",
"description": "Engine-driven layer animation clock, in seconds.",
"min": 0,
"max": 3600,
"default": 0
},
{
"name": "rate",
"description": "Pulses per second.",
"min": 0.1,
"max": 10,
"default": 2
}
]
}
```
In WGSL the slot is an ordinary `f32` uniform in the positional `Params` struct —
nothing about reading it differs from any other parameter:
```wgsl
struct Params {
animationTime: f32,
rate: f32,
_pad0: f32,
_pad1: f32,
};
```
### Shaping time in WGSL
Shape time inside the shader, from `animationTime` plus scalar knob params —
not with an animator. Each pattern is one expression; expose the knob as a
normal parameter so it gets a control:
| Want | WGSL (`let t = params.animationTime;`) | Knob param |
|---|---|---|
| Speed / rate | `t * params.rate` | `rate` (cycles/sec) |
| Offset / delay | `max(t - params.delay, 0.0)` | `delay` (sec) |
| Loop | `fract(t / params.period)` | `period` (sec) |
| Ping-pong | `abs(fract(t / params.period) * 2.0 - 1.0)` | `period` (sec) |
| Hold / stop-motion | `floor(t / params.holdTime)` | `holdTime` (sec/step) |
Use a `jsScript` animator on a normal parameter only for what a pure function
of the clock cannot express: values keyframed by hand, or values driven by
another declared property dependency; there is no automatic audio-analysis input.
## Complete single-texture WGSL template
The owning layer is the only texture when `textureInputs` is empty. This module
is complete and uses the required vertex/fragment entry points, binding layout,
uniform padding, and premultiplied-alpha output contract:
```wgsl
struct VertexInput {
@location(0) position: vec2<f32>,
@location(1) tex_coords: vec2<f32>,
@location(2) color: vec4<f32>,
};
struct VertexOutput {
@builtin(position) clip_position: vec4<f32>,
@location(0) tex_coords: vec2<f32>,
};
@vertex
fn vs_main(input: VertexInput) -> VertexOutput {
var out: VertexOutput;
out.clip_position = vec4<f32>(input.position, 0.0, 1.0);
out.tex_coords = input.tex_coords;
return out;
}
struct Params {
amount: f32,
_pad0: f32,
_pad1: f32,
_pad2: f32,
};
@group(0) @binding(0) var t_texture: texture_2d<f32>;
@group(0) @binding(1) var s_sampler: sampler;
@group(0) @binding(2) var<uniform> params: Params;
fn author_frag(uv: vec2<f32>) -> vec4<f32> {
let content = textureSample(t_texture, s_sampler, uv);
let grayscale = vec3<f32>(dot(content.rgb, vec3<f32>(0.299, 0.587, 0.114)));
return vec4<f32>(mix(content.rgb, grayscale, params.amount), content.a);
}
@fragment
fn fs_main(input: VertexOutput) -> @location(0) vec4<f32> {
let raw = author_frag(input.tex_coords);
// The renderer uses premultiplied-alpha blending. Transparent pixels must
// carry zero RGB, and every RGB channel must stay at or below alpha.
return vec4<f32>(
clamp(raw.rgb, vec3<f32>(0.0), vec3<f32>(raw.a)),
raw.a,
);
}
```
Each scalar parameter adds one `f32` field in `params[]` order. Pad the WGSL
`Params` field count to a multiple of four `f32` values (`_pad0`, `_pad1`, …).
## Additional texture inputs
`textureInputs[]` declares named texture slots supplied by other layers:
```json
"textureInputs": [
{
"name": "map",
"description": "Grayscale displacement map.",
"sourceLayerId": 12
}
]
```
- `sourceLayerId` references a layer in the same composition.
- Slots resolve in declared order.
- Every required slot must be bound for the shader to run.
- At most eight additional texture inputs are supported.
- A successfully bound source layer is consumed as shader input and is not
separately painted in the normal layer stack.
- Unknown references, cycles, unbound slots, or excess inputs make the shader
fall back to a no-op/pass-through behavior.
### Multi-texture binding layout
For `N` additional inputs, `K = N + 1` total textures:
```text
binding 0 owning layer content texture
binding 1..N textureInputs[] in declaration order
binding K shared sampler
binding K + 1 Params uniform
```
For example, one additional displacement map uses content at binding 0, the map
at binding 1, the sampler at binding 2, and `Params` at binding 3:
```wgsl
@group(0) @binding(0) var t_content: texture_2d<f32>;
@group(0) @binding(1) var t_map: texture_2d<f32>;
@group(0) @binding(2) var s_sampler: sampler;
@group(0) @binding(3) var<uniform> params: Params;
```
The `textureInputs[]` names describe logical slots; WGSL variable names are
chosen by the shader author. Declaration order, not the logical name, determines
the binding number.
## Authoring checklist
1. Inspect the installed document schema's layer-effect definitions for an
existing effect before authoring a `customShader`.
2. Give the shader a stable, descriptive `name` and explain its intent.
3. Describe every parameter, including units and useful range, and declare
its control with an explicit name suffix (`.colorR/.colorG/.colorB`,
`.x`/`.y`, `.angle`, `.toggle`) so it gets an intuitive control. Declare a
real `min`/`max` on every plain scalar — that is what earns it a slider.
4. Keep parameter declaration order aligned with the WGSL `Params` struct.
5. Describe every texture input and bind it to a valid source layer.
6. Preserve premultiplied alpha: output RGB must not exceed output alpha.
7. Keep sampling and loops bounded for render-time performance.
8. Animate knobs through `effectId + paramName`, not by rewriting WGSL per frame.
When the shader needs a clock, declare the reserved `animationTime` parameter
rather than an animator.
9. Check the installed document schema, save through a direct document JSON
edit or a supported action batch, then render a `preview` so the renderer
validates the WGSL and bindings. Inspect the actual effect; a pass-through
fallback is not a successful shader.
## Prefer self-describing shaders
Descriptions travel with the document and become context for future agents.
Write what an input means, not merely its type:
```text
Bad: "map texture"
Good: "Grayscale displacement map; lighter pixels shift samples farther right."
```
references/editorial-decisions.md
# Choose the treatment from the job
When music is already selected, establish its phrase/beat map before planning visual timing. When tightening speech, inspect waveforms before choosing trims and verify the rendered joins afterward. Follow [waveform editing](waveform-editing.md) for the actual local views and source/edit timing checks.
## Three modes, one video
| Mode | Use it when | What would make it wrong |
|---|---|---|
| Footage alone | The action, expression, product behavior, or spoken thought carries the point. | Decorating a strong moment distracts from the evidence. |
| Footage with an overlay | The viewer needs a name, emphasis, measurement, comparison, pointer, or context while seeing the real action. | The graphic repeats speech, hides the subject, competes with captions, or becomes a full-screen card by accident. |
| Full motion-graphics scene | The idea is abstract, a relationship needs explaining, there is no useful footage, or a chapter/brand moment needs a deliberate visual reset. | Interrupting emotional or product evidence with a generic title card, or turning the edit into consecutive slides. |
Choose the amount from the material, not a fixed footage-to-graphics ratio. A dialogue edit may need almost no graphics. A technical explainer can need several full scenes. A motion-only brief may use none of the footage. Preserve those distinctions.
## Source selection comes before embellishment
Inspect enough of each clip to identify usable in/out points, action, camera movement, eyelines, and audio. Keep a compact source log with filename, source time range, and why the moment matters. A contact sheet alone cannot establish speech meaning or whether an action completes cleanly.
For speech: preserve complete thoughts and natural breaths. Remove redundancy with care; an aggressive silence cut can sound fragmented. Use an L-cut or J-cut only when it improves continuity and keep source audio mapped explicitly. Use B-roll to support what is being said, not just to hide every edit.
For action: cut on motivated movement or changes in information. Leave enough pre-action and post-action handles to read the motion. A speed ramp is purposeful emphasis; do not accelerate a shot merely to fit an arbitrary runtime.
For products: show actual behavior clearly before calling it a feature. Exact UI text can come from supplied screens/assets. Avoid invented product screens, metrics, quotes, or claims.
For ads, use [ad hooks](ad-hooks.md) to select the opening idea and align it to the music/action before expanding the full edit plan. Choose the source evidence and viewer takeaway first, then the motion and sound treatment.
## Talking-head eyeline continuity
When cutting between shots of the same speaker, align the speaker's eyes across the cut for a natural transition. Compare their eye height and horizontal screen position in the **final composition**, after crops, scaling, and aspect-ratio changes. Use the midpoint between visible eyes as a practical reference rather than the top of the head, which changes with head tilt and framing.
Inspect the outgoing and incoming frames plus a short interval on either side; a blink or passing nod should not determine the framing of an entire shot. Apply modest per-shot position/crop and uniform scale adjustments to keep the eyeline at a consistent screen height and avoid an accidental sideways jump. Preserve useful headroom, looking room, facial proportions, and source resolution. For an intentional punch-in, allow the face size to change while keeping the eyes near the same screen position. A close-up need not match the wide shot's head size.
Keep natural head movement; do not introduce jitter by chasing the eyes frame by frame. Screen position and gaze direction are separate: reframing cannot correct a change in where the speaker is looking. Respect deliberate camera-angle/framing changes and interview looking direction; do not mirror or alter the speaker's eyes to manufacture continuity. If matching requires an awkward crop or excessive enlargement, choose a better source boundary or shot while preserving complete words and audio sync.
Follow [filmstrip review](filmstrip-review.md) to compare the last outgoing and first incoming frames at the same display size, then play the join at normal speed. Recheck adjacent cuts after changing a shot's framing, including captions and overlays that may now collide with the face. Record any intentional framing change that remains.
## A compact plan
For a new edit a table like this is sufficient; keep it proportional to complexity:
| Edit time | Source moment | Viewer takeaway | Treatment | Sound |
|---|---|---|---|---|
| 0–3s | Close-up where the mechanism engages | See the result immediately | Footage; no title obstruction | Keep the actual click |
| 3–7s | Wider view / explanation | Understand how it works | Small tracked-in-space callout if supported, otherwise a stable pointer | Dialogue forward |
| 7–10s | No footage explains the relationship | Understand the two-part process | Full native diagram with a readable hold | Continue the music bed |
| 10–15s | Return to the result | Connect explanation to proof | Footage; restrained end label | Resolve naturally |
These numbers are illustrative, not a template or required pacing.
For each beat, consider the sound roles in [sound design](sound-design.md): primary voice, music, a small visual accent, a structural cue, and physical texture. Mark only those that earn a place, plus intentional silence. Plan the timing of a sound’s perceptual peak against the action, then audition it.
## Rhythm and continuity
- Identify phrases, downbeats, impacts, and quiet intervals in supplied music. Let those inform structure; avoid cutting on every beat.
- Establish one main transition language tied to the content: a match cut, shared shape, directional movement, or sound bridge. Cut directly where a transition adds nothing.
- Reuse a few design decisions—type hierarchy, grid, palette, motion character—so scenes feel related. Repetition of a design system is useful; repetition of the same scene arrangement is usually not.
- Build toward a specific payoff. Keep some scenes quieter so the important moment has contrast.
- A requested duration is a delivery constraint. Achieve it through selection and timing, not a globally sped-up finished video or unreadably short text.
For a revision, change only the necessary source ranges/layers/animators, retain the existing language, and inspect neighboring beats for continuity.
references/filmstrip-review.md
# Mid-edit review with Tesseract filmstrips
Tesseract can render sampled project frames into a labeled PNG grid without encoding a full MP4. Use this during editing to catch problems before building more work on top of them.
## The review loop
After the rough assembly and after a meaningful visual change (a new graphic, text layout, crop, timing, or transition):
1. Save the current revision of the document. The strip must come from that revision, not a stale document or the source footage.
2. In one chained shell command, render a filmstrip covering the changed beat and its neighboring cuts, plus any [detail views](#choose-the-view-size) the change needs. For a rough edit, first make a sparse overview covering the sequence.
3. **Open the PNGs with the host's image-view tool and inspect the actual pixels.** Open all of a check's images together, in one message. A successful command or an image path alone does not count as review. In Codex, use the available local image viewer, such as `view_image`.
4. Identify the timestamp and observable problem: text collision, premature exit, blank frame, wrong crop, inconsistent scale, missing asset, or distracting repetition. Correct the native project, save, and render a fresh strip of the affected range. Compare before/after side by side with the same sample times.
5. Keep a short note identifying the inspected project revision, strip path, times, issues fixed, and any remaining concern. This can live in the edit notes; no separate formal report is required.
Do not run a new strip after every scalar tweak. Review coherent changes while they are still cheap to correct. Finish by playing a short preview or the final edit with audio; a filmstrip cannot establish sound quality, smooth motion, or lip sync.
## Choose useful samples
- **Assembly overview:** roughly every 1–2 seconds, plus important scene boundaries. Split long edits into readable strips rather than making one giant sheet.
- **A changed beat:** entrance, mid-motion, readable hold, and exit, with context from the preceding/following shot. About 100–250 ms spacing is useful for a short reveal; use explicit times when the important states are known.
- **A suspect cut:** sample the frame just before and just after the cut, as well as the cut time. At 30 fps, a frame is about 33.3 ms; for a cut at 2.0 s, 1967, 2000, and 2033 ms are useful requests. Inspect the resolved labels and source cadence; a rounded millisecond request is not a promise of exact source-frame indexing.
- **Talking-head cuts:** compare outgoing/incoming frames and a few surrounding samples at the same tile size for eye height, horizontal position, and headroom. Inspect the final crop, including punch-ins; apply [eyeline continuity](editorial-decisions.md#talking-head-eyeline-continuity), then re-render and play the join to check for an unnatural face jump.
- **Color grading:** compare the reference with related shots, including nonadjacent ones, as described in the [grading workflow](color-grading.md#workflow). Preserve intentional lighting differences; use full-resolution previews for shadow detail/noise and playback for jumps across cuts.
Uniform sampling can miss a one-frame flash. It also cannot prove a complete absence of overlap between samples. Use denser targeted samples or playback when the behavior is uncertain.
## Choose the view size
- **Timing, motion, cuts, and composition:** default-size cells at the project aspect: 240×135, or 135×240 for portrait. Do not enlarge every tile to see detail.
- **Text, edges, materials, lighting, noise, or grade detail:** a full-size `preview` frame where it matters. Thumbnail text is not enough evidence of final-size legibility. When a reduced view shows enough, one large cell will do, such as a filmstrip of one timestamp in a 960×540 tile, one per row (540×960 for portrait).
- **Small text or matte detail in a known region** (a title, lower third, matte edge, or an artifact already seen): crop that region at 100% from a full-size `preview` with any available local image tool, and inspect the crop instead of the whole frame. Keep each crop, or sheet of one region's crops at several times stacked or side by side, within about 1500 px on its long side and 1 megapixel, or the host's image limit if lower, so the viewer shows it unscaled. For moving text, crop each time where the text actually is.
Crops do not replace full frames for a first look at a new shot, scene, or look, for hunting defects whose location is unknown, or for the final check. Where the cells show enough, prefer one filmstrip of explicit timestamps to a chain of previews; each `preview` reloads the document.
## Sampling a strip
A readable landscape overview samples 0–8 s every 1 s, four per row.
```sh
tsrct filmstrip --project /absolute/path/edit/project.tsrct --start-ms 0 --duration-ms 8000 --interval-ms 1000 --items-per-row 4 --output /absolute/path/edit/Previews/Filmstrip.png
```
A closer review around a cut samples 1700, 1900, 1967, 2000, 2033, 2100, 2300,
and 2600 ms, four per row.
```sh
tsrct filmstrip --project /absolute/path/edit/project.tsrct --timestamps-ms 1700,1900,1967,2000,2033,2100,2300,2600 --items-per-row 4 --output /absolute/path/edit/.tesseract-work/checks/cut.png
```
Keep ranges inside the actual document duration. Range sampling is half-open;
explicit timestamps preserve input order. Do not sample exactly at the edit end
and mistake an inactive end state for a bad final frame. About 8–16 samples are
usually easier to inspect than a giant sheet. Set the number per row so the rows
are full, as with 4 for the 8 samples above.
Strips at the default cell size stay small; full-size `preview` frames are much
larger, so save them only where pixel detail matters. Overwrite routine strips on
the next pass. For a before/after comparison, write the new strip to a second
name and keep both until the comparison is noted (see
[storage](local-operation.md#storage)).
Cells default to 240×135 with five columns and ordinal/time labels. Use
135×240 tiles for portrait; match the project aspect for
square/4:5 work; for detail, see [choose the view size](#choose-the-view-size).
Set them with `--tile-width` and `--tile-height`. The command's JSON output
reports `outputPath`, `frameCount`, and dimensions. `resolvedSampling` and
`resolvedFraming` are populated only when the CLI resolves a target-range or
content-bounds request; they are `null` for ordinary range or explicit-time
sampling and default framing. For `--timestamps-ms`, use the requested times
and strip order; the visible time labels are rounded to tenths of a second, so
nearby requested times can share a label.
Use resolved times when the JSON provides them. For explicit timestamps,
compare the requested times and visible labels for cut-level diagnosis.
## Fonts must match the final render
Import the intended local font file before authoring
text. Preview, filmstrip, and export use that document's imported fonts; there
is no separate runtime-font mapping. Inspect the glyphs, line breaks,
and overflow. If import or font resolution fails, report the exact error and
follow [local operation](local-operation.md); do not silently substitute a font.
## Targeted inspection
A solo strip renders one layer or group by itself. Use an actual
composition/group ID from inspection, such as `main:3`; this example takes up
to 8 frames cropped to the content bounds, in 320×180 tiles four per row.
```sh
tsrct filmstrip --project /absolute/path/edit/project.tsrct --fx-solo main:3 --max-frames 8 --content-bounds --tile-width 320 --tile-height 180 --items-per-row 4 --output /absolute/path/edit/.tesseract-work/checks/graphic.png
```
This samples the selected layer/group's full active window. Skip the
content-bounds crop to preserve its canvas placement. With motion blur enabled,
sample at the export frame rate so each sample uses that frame's shutter.
Solo cannot be combined with `--timestamps-ms`, `preview` has no solo option,
and only one sampling mode applies at a time. Pass the frame rate as
`--frame-rate`. The CLI has no crop or target-offset flags.
A solo strip is for diagnosis: also inspect the graphic in the complete
composition before calling it finished. For a closer view, crop a full-size
`preview`.
references/fx-authoring.md
# FX authoring
Inspect the document for existing content. Exact fields come from the installed
action schema, or the document schema for editable JSON.
Use `project inspect`, `project schema`, and `project schema --document`.
The footage example edits document JSON; the other examples are action batches.
## Document color management
Set `colorManagement` on the document root, not on a composition or layer.
Omitting it uses `legacyDisplay`. For ACES setup, compositing presets, layer
`renderDomain` overrides, and highlight finishing with `halation`, read
[color grading](color-grading.md#aces-scene-referred-color-management).
## Structure, placement, and resources
- The document owns one composition starting at time zero.
Its root `duration` is seconds.
Layer windows use milliseconds: Video/Audio/Group use
`playback.inputRange` on the immediate parent clock; other kinds use
`activeRange`. Keep document and ancestor windows long enough.
- Layer IDs are integers unique within a composition. Choose unused IDs after
inspection; preserve IDs when editing. Names describe visible roles.
- Sibling index zero is frontmost. Create actions append behind siblings when
`insertIndex` is omitted; use `insertIndex: 0` for a foreground insertion.
- `position` is in parent coordinates; `anchorPoint` is in layer-local coordinates.
Scale and opacity use percentages (100 is unchanged/opaque). Authored paint RGB is finite encoded extended-sRGB in 0–64; alpha remains 0–1 (see [paint colors above 1](#paint-colors-above-1)). Native Adobe exports clip RGB to 1 with diagnostics; see `native-authoring.md`.
Read the project's canvas dimensions rather than treating source pixels as
composition coordinates. Parent transforms also affect children.
- `sourceRange` selects media time independently of `playback.inputRange`.
Video/Audio/Group require windowed `playback` with an authored `mapping` and
`inputOffsetMs`; do not also supply `activeRange`. Moving a visibility window
and changing its content clock are different operations.
- Keyframe `layerTime` on a windowed Video/Audio/Group is on its content clock
(the mapped `playback` output, so `inputOffsetMs` included), clamped to the
window, not "milliseconds since the layer appeared". With
`inputOffsetMs` 1700, the first visible frame is at `layerTime` 1700. To animate
from the moment a clip appears regardless of its source offset, wrap it in a
Group and animate the Group.
- Import local footage before referencing its asset ID.
An arbitrary path or asset ID in an action does not import its bytes.
Import audio/images through [media import](media-import.md); package fonts as below.
## Local footage
Package supplied footage first; import saves the video bytes inside the
project without modifying the source file.
```sh
tsrct project import-video --project project.tsrct \
--file /path/to/clip.mp4 --asset-id footage-1
```
Use a new ID containing letters, numbers, hyphens, or underscores. Existing IDs
are rejected to avoid replacing another layer's media. The JSON response reports
`assetId`, `durationMs`, `width`, `height` (display orientation), and `hdrTransfer`
(`"hlg"`, `"pq"`, or `null` for SDR). MP4, MOV, and
M4V containers are accepted; actual decoding uses the engine's supported codecs.
No external metadata tool is required.
Import adds the resource; the following editable document JSON places it on the
timeline. Append the layer below to the composition's layers and set the
document duration to three seconds.
Run `project checkout` first and
`project commit --project project.tsrct --file .tesseract-work/editable.json` afterwards.
Preserve other layers; append footage behind overlays because index zero is frontmost.
For a three-second clip, for example (adapt duration, IDs, canvas, and transform
to the returned metadata and the user's requested framing):
```json
[
{"type":"Video","id":1,"name":"Main footage",
"playback":{"type":"windowed","inputRange":{"start":0,"duration":3000},
"mapping":{"type":"linear","input":{"start":0,"duration":3000},"output":{"start":0,"duration":3000}},"inputOffsetMs":0},
"sourceRange":{"start":0,"duration":3000},
"sourceIntrinsicDuration":3000,"volume":1.0,
"transform":{"anchorPoint":[0,0],"position":[0,0],"scale":[100,100],"rotation":0,"opacity":100},
"source":{"assetId":"footage-1","fit":"cover",
"sourceRect":{"x":0,"y":0,"width":1920,"height":1080}}}
]
```
Import each distinct file once; multiple layers can reference the same asset.
Use the actual clip duration for `sourceIntrinsicDuration`, and select the wanted
section with `sourceRange`. Preserve footage audio unless the user asks to mute
or replace it. Set `volume: 1.0` on new Video layers to enable embedded audio
at unity gain; an omitted/null volume leaves it disabled. Preserve existing
gain and animators on revisions, and verify the exported audio.
`source.sourceRect` is the layer's media **frame** in layer-local pixels, not a
crop of the source. Without it the frame is the source's natural size at the
layer origin, so a 2560×1440 or vertical clip does not fill a 1920×1080 canvas.
Set it to the canvas rectangle (as above, using the project's real canvas size)
and choose `fit`: `cover` fills and clips, `contain` letterboxes. Select time
with `sourceRange` and the playback mapping, never with `sourceRect`.
For the unit-rate linear mapping above, the visible source moment is parent
time plus `inputOffsetMs` and must lie inside `sourceRange`: to start a clip at
source 5 s, set `inputOffsetMs: 5000` and `sourceRange.start: 5000` together. Keyframes on the Video use that same
content clock (see below), so animate a wrapping Group for "time since the clip
appeared".
### `timeRemap` and physical media time
All playback times are **milliseconds**, not frame indices. A windowed playback's
`inputRange` is its visible half-open parent-clock window. `inputOffsetMs` is added
to the incoming parent-clock time **before** evaluating `mapping` (including a
`timeRemap` curve). In a unit-rate zero-origin linear map this also shifts the
source time by that amount; for other rates or nonlinear curves, evaluate the
mapping rather than treating the offset as an output-time addition.
For example, this maps parent time 0–4000 ms identically to source time:
```json
{"type":"timeRemap","property":{"keyframes":[
{"id":"a","time":0,"value":0,"easing":{"type":"linear"}},
{"id":"b","time":4000,"value":4000,"easing":{"type":"linear"}}
],"before":"inactive","after":"inactive"}}
```
Use this as `playback.mapping`, with `inputOffsetMs: 0`. Key `time` is the
mapping's input clock in ms; `value` is its output/source time in ms (fractional
values are allowed). `before`/`after` select extrapolation outside the keys;
`inactive` produces no mapped sample there. The Video's `sourceRange` further
restricts the mapped source window. A decreasing curve plays backward, and a
held value freezes that source moment. Wrapping Groups compose their own clock
mappings before the Video's mapping.
Media sampling retains the precise physical output sample through those maps,
quantizing only at the decoder's microsecond interface. Property/visibility
clocks still use their existing integer-ms rules. Source selection remains the
last presentation timestamp at or before the request, with the existing native
499 µs guard; it is not nearest-frame selection. Reverse/discontinuous mappings
retain their boundary side: right limits floor to µs, left limits use
`ceil(µs) - 1`, then the same decoder guard applies. Consequently a left-limit
request less than 499 µs before a PTS can still select that PTS.
This removes former whole-ms media truncation and can change fractional-boundary
selection, including VFR and reverse samples. `333` means literally 333 ms, not
"source frame 10" for every codec or frame rate. At 30 fps it is close enough to
frame 10 for the native guard; for VFR or arbitrary rates use the actual source
presentation times. Do not add a blanket 5 ms bias to remap keys. Exact behavior
of externally supplied/browser decoders remains the host decoder's contract.
Inspect a preview to verify crop, orientation, and composition placement before
adding overlays or exporting. If decoding fails, report the error and source format.
## Paint colors above 1
Rect/Solid `fillColor` and `strokeColor` and `ShapePaint.type="solid"`
fill/stroke RGB can use finite values from 0 to 64; alpha stays from 0 to 1.
Text, gradients, backgrounds and content-painting effect/style colors also accept
extended RGB. In SDR export, RGB is divided by its largest channel when that
channel exceeds 1, preserving channel ratios (no extra brightness); alpha is
unchanged. Existing colors at or below 1 are unchanged. HDR export renders values
above 1 brighter than graphics white; see [HDR](hdr.md#authored-rgb-above-1).
**SDR-domain effect colors: 0..1** — Tint/TintTritone and similar engine color
shaders retain their validation/clamps in HDR; Hue/Saturation Colorize remains
scalar SDR controls. They cannot produce light above SDR white.
## Rectangle overlay
In the default `main` composition, this adds a blue card. Adapt the inspected
composition ID, unused layer IDs, and geometry to the existing document.
```json
[
{"type":"createFxRectLayer","compositionId":"main","layerId":1,"name":"Card","insertIndex":0,
"activeRange":{"start":0,"duration":3000},
"transform":{"anchorPoint":[0,0],"position":[80,80],"scale":[100,100],"rotation":0,"opacity":100},
"rect":{"size":[400,160],"fillColor":[0.05,0.25,0.8,1]}}
]
```
## Group related parts
After the previous example, add an accent in front and group the two siblings.
The identity group transform preserves their placement. Animate the group to
move the callout together; animate a child for a detail within it.
```json
[
{"type":"createFxRectLayer","compositionId":"main","layerId":2,"name":"Accent","insertIndex":0,
"activeRange":{"start":0,"duration":3000},
"transform":{"anchorPoint":[0,0],"position":[80,80],"scale":[100,100],"rotation":0,"opacity":100},
"rect":{"size":[8,160],"fillColor":[1,0.7,0.1,1]}},
{"type":"groupFxCompositionLayers","compositionId":"main","layerIds":[2,1],"groupLayerId":3,"name":"Callout group",
"transform":{"anchorPoint":[0,0],"position":[0,0],"scale":[100,100],"rotation":0,"opacity":100}}
]
```
Grouping requires contiguous siblings with the same parent. Use the grouping
operation rather than manually reparenting children and rewriting their timing.
## Text with an available font
Choose typography for the brief before acquiring files. Honor supplied brand
fonts; otherwise choose the character, width, weight, and hierarchy that suit
the actual headline and supporting copy. The example's Inter Bold is a wire-format
example, not a recommended design default. Availability in an OS folder, cache,
or test fixture is not a reason to choose a face.
Use supplied font files or obtain the selected TTF/OTF/TTC from an official
source such as Google Fonts when downloads are allowed. Keep a recognizable
filename and the source/license with the working files. If the requested font
cannot be obtained, explain the gap before substituting it.
Import the selected files into the document before authoring text.
```sh
tsrct project import-font --project project.tsrct --file /path/to/Inter-Bold.ttf
```
Use the returned `fontFamily` and `fontStyle`, not names guessed from the filename.
For a collection, choose the intended face. The filename is a
readable asset label; parsed metadata identifies the face, and the asset ID
identifies the file's bytes. Import only the faces needed for the chosen design.
Collection faces are listed under `faces`. Repeat import is a no-op. Preview
and export use the packaged bytes after the document moves to another machine.
Before building text-led animation, preview the actual words at their intended
size and placement. Check character, hierarchy, line breaks, and glyph coverage;
if the choice is uncertain, compare a small number of candidates in a scratch
project before importing the selection into the deliverable. A successful import
proves the file is usable, not that its typography fits the brief.
This adds box text to group 3 above using the imported Inter Bold face.
```json
[
{"type":"createFxTextLayer","compositionId":"main","parentLayerId":3,"insertIndex":0,"layerId":4,"name":"Callout label",
"activeRange":{"start":0,"duration":3000},
"transform":{"anchorPoint":[0,0],"position":[112,112],"scale":[100,100],"rotation":0,"opacity":100},
"sourceText":{"text":"A useful detail","fontFamily":"Inter","fontStyle":"Bold","fontSize":32,
"fillColor":[1,1,1,1],"justification":"left","boxText":true,"boxPosition":[0,0],"boxSize":[336,96]}}
]
```
Box text wraps but does not automatically shrink or clip overflow. Point text's
local origin is its baseline start, not its visual center. Inspect a rendered
frame for font metrics, line breaks, and overflow before animating the layout.
## Text on a background
For a label on a pill, tag, chip, badge, or button, let a Group draw the
background around point text instead of sizing a separate rectangle by guess.
A Group's fill, padding, and corner radii paint a rectangle that hugs its
rendered children, so the background fits the words at any length or font and
the text sits centered in it. Give each label its own Group. This adds one tag;
`createFxGroupLayer` takes the Group's window as `activeRange` and stores it as
its `playback`:
```json
[
{"type":"createFxGroupLayer","compositionId":"main","layerId":5,"name":"Tag","insertIndex":0,
"activeRange":{"start":0,"duration":3000},
"transform":{"anchorPoint":[0,0],"position":[0,0],"scale":[100,100],"rotation":0,"opacity":100}},
{"type":"createFxTextLayer","compositionId":"main","parentLayerId":5,"insertIndex":0,"layerId":6,"name":"Tag label",
"activeRange":{"start":0,"duration":3000},
"transform":{"anchorPoint":[0,0],"position":[150,1400],"scale":[100,100],"rotation":0,"opacity":100},
"sourceText":{"text":"Limited offer","fontFamily":"Inter","fontStyle":"Bold","fontSize":44,
"fillColor":[0.1,0.1,0.1,1],"justification":"left"}},
{"type":"updateGroupLayer","compositionId":"main","layerId":5,
"values":{"paddingTop":20,"paddingBottom":20,"paddingLeft":32,"paddingRight":32,
"cornerRadiusTopLeft":43,"cornerRadiusTopRight":43,"cornerRadiusBottomRight":43,"cornerRadiusBottomLeft":43,
"fillEnabled":true,"fillColor":[0.98,0.97,0.96,1]}}
]
```
Place and animate the tag by its Group. The background follows the label when
its text changes. Read the FX `layers` documentation (Group bounds, padding,
and background) for gradient fills and nested Groups.
A Group hugs its children's line height, not their letters: the room above
capitals and below the baseline counts, so a small label over a large number
comes out with more space below than above. Measure the letters instead of
judging from thumbnails: the `fx.text-fit` and `fx.text-overlap` evaluate
checks read each text's glyph outlines against its backdrop (its background
Group, or a filled shape under it) over every frame. `overflow` means letters
cross the backdrop's edge, `offCenter` gives the four margins when opposite
sides differ, and `overlap` means two texts' letters intersect, including in
the middle of a pop or slide. Fix every overflow and overlap; shift padding by
the reported margins, or judge an offCenter against the design.
```sh
tsrct evaluate --project /absolute/path/edit/project.tsrct --checks fx.text-fit,fx.text-overlap
```
Add `--report fx.text-layout --time-ms <t>` for every text's letter box,
backdrop and margins at one frame.
## Bottom captions
Keep spoken captions editable as FX text layers, with one short phrase per
`activeRange`. Obtain the actual words and timings from a supplied transcript,
reviewed audio, or an available transcription tool. The CLI does not transcribe.
If none is available, ask for the spoken text rather than inventing dialogue.
Map source timestamps through the footage's authored `playback.mapping` and
`inputOffsetMs`, respecting its visibility window and `sourceRange` selection.
Import the selected legible font file first. On a 1080×1920 portrait canvas, a useful
starting point is centered box text at `[90, 1550]`, box size `[900, 220]`,
font size `64`, and white fill. Adapt to the inspected canvas and shot. Keep
phrases to one or two lines and use a dark rounded rectangle behind them when
footage contrast needs it. Put both layers above the footage with matching
active ranges, and keep the text ahead of its background in sibling order.
Review phrase boundaries and the longest caption in the composed filmstrip.
Verify the text stays inside the frame, clears the face and bottom edge, and
matches the audible words. Preserve the video's embedded audio gain.
references/hdr-blend-modes.md
# Blend modes in HDR
HDR white=1 is graphics white; alpha remains coverage. SDR behavior is unchanged.
| Modes | HDR behavior |
| --- | --- |
| Normal, Add, Multiply, Lighten, Darken, Subtract, Divide | Native linear BT.2020, no upper RGB clamp |
| Screen, Overlay, SoftLight, HardLight, ColorDodge/Burn, classic dodge/burn, LinearBurn/Light, VividLight, PinLight, HardMix | Blend the bounded SDR display component, carry the channelwise highlight envelope |
| Difference, classic Difference, Exclusion | Same display split/envelope; do **not** difference HDR excess |
| Hue, Saturation, Color | SDR display math; carry destination excess |
| Luminosity | SDR display math; carry source excess |
| DarkerColor, LighterColor | Select using SDR RGB sum; carry selected operand's excess |
Display split: linear Rec.709 `P=clamp(L,0,1)`, `E=L-P`; blend sRGB-encoded
`P`, decode, restore excess, convert to linear BT.2020. Envelope means
`max(Es,Ed,0)+min(Es,Ed,0)` per channel, not additive highlight doubling.
White ties choose source for DarkerColor, destination for LighterColor.
Example: opaque linear source=2/destination=4 yields Screen=4, Difference=3,
Color=4, Luminosity=2; native Add=6 and Multiply=8. HDR blend alpha is
source-over coverage. Bounded contrast output does not cap carried HDR light.
These are defined HDR semantics, not an Adobe HDR fidelity claim.
The packaged table above defines the HDR blend contract. Renderer implementation
and bounded regression evidence are maintained separately from this distribution.
Subtract and Divide are native linear arithmetic because converters use them
structurally, including Premiere channel-Levels division by a linear gray
coverage carrier. Subtract floors at zero; Divide retains the SDR nonpositive
source guard (returns white), but does not cap positive HDR quotients.
For display-domain modes, only the blend function runs in display encoding;
coverage/alpha compositing runs in linear premultiplied BT.2020.
references/hdr.md
# HDR output and images
Export, preview and filmstrip are SDR unless you add `--hdr`. Use this reference
for `--hdr` on any of them,
keeping footage that reports `hdrTransfer` `hlg` or
`pq` HDR, revising a project delivered in HDR, `hdrGraphicsWhiteNits`, authored
RGB above 1 as HDR light, and HDR still images.
## HDR export
An HDR export encodes HEVC Main 10 MP4 with a PQ or HLG transfer. By default it
matches the majority transfer of source videos, or HLG when none declare one. Use
PQ for graphics/image-only documents or any content above HLG's approximately
766 cd/m² peak. PQ can represent up to 10,000 cd/m²; HLG is the existing
convention (0.75 = 203 cd/m², peak approximately 766). The transfer changes
encoding, not source luminance.
Select it with `export --hdr [auto|pq|hlg]`; bare `--hdr` means `auto`. `--hdr`
cannot be combined with `--format prores` or `--fx-solo`.
Inspect HDR with a display or tool that reads its color metadata, not an 8-bit
SDR image viewer.
```sh
tsrct export --project project.tsrct --hdr pq --output finished-hdr.mp4
```
In an HDR export, white graphics (text, shapes, images; color 1.0) default to
about 609 cd/m², three times the 203 cd/m² diffuse white of HDR video, so text
reads clearly white on phones. Set the document's `hdrGraphicsWhiteNits`
(100–1000) in the document JSON to change it; 203 puts graphics on the
video's diffuse white (broadcast convention). Above about 766, HLG clips
highlights; choose PQ to preserve them. HDR video and HDR images retain their
source luminance independently of this setting; it does not set the absolute
luminance of ungraded HDR media. SDR output is unaffected.
### Choosing graphics white
Keep the default unless the delivery specification or the intended appearance
calls for a change; it is a product default, not a universal HDR mastering target.
- Follow the recipient's delivery specification first. When it calls for
BT.2408 reference white, use 203 cd/m². This is the nominal Graphics White in
[ITU-R BT.2408, section 2.1](https://www.itu.int/dms_pub/itu-r/opb/rep/R-REP-BT.2408-8-2024-PDF-E.pdf),
not a requirement shared by every broadcaster or streaming platform.
- If graphics look glaring or detached, especially large white areas over dark
video, try a lower value and review representative shots on an HDR-aware display.
Preserve the intended caption legibility rather than matching luminance blindly.
- To measure, render an HDR frame and decode it as in
[HDR preview and measurement](#hdr-preview-and-measurement).
Sample the opaque interior of white graphics, avoiding antialiased edges, and
compare it with a representative diffuse-white object in the video. Avoid
specular highlights and do not treat skin as a white reference. Lighting and
creative grading change scene whites, so there is no universal graphics-to-video
brightness ratio. Re-render and review after changing the value.
For neutral authored RGB, absolute luminance is `linear(value) ×
hdrGraphicsWhiteNits`, where `linear` is the sRGB EOTF continued above 1:
`((value + 0.055) / 1.055)^2.4` for values above 0.04045.
| Encoded RGB value | At graphics white 609 (cd/m²) | At graphics white 203 (cd/m²) |
| --- | ---: | ---: |
| 1.0 | 609 | 203 |
| 1.1 | 757 | 252 |
| 1.35 | 1211 | 404 |
| 2.0 | 3017 | 1006 |
These are intended neutral-paint luminances before output clipping, not a
per-channel luminance formula for colored paint. HLG clips at approximately
766 cd/m², so the default 609 leaves little real headroom. For extended-color
headroom, lower `hdrGraphicsWhiteNits` to around 203 and export with `--hdr pq`.
## HDR preview and measurement
`preview --hdr` and `filmstrip --hdr` write a 16-bit straight-alpha BT.2020 PQ
PNG tagged with `cICP` `[9, 16, 0, 1]` (always PQ, never auto or HLG). Without
the flag they stay 8-bit SDR.
Do not judge HDR luminance from an SDR viewer or an
8-bit decode, and do not measure PQ codes as SDR luma.
```sh
tsrct preview --project project.tsrct --time 1 --hdr --output .tesseract-work/checks/hdr.png
tsrct filmstrip --project project.tsrct --timestamps-ms 0,1000 --hdr --output .tesseract-work/checks/hdr-strip.png
```
Read RGB samples as `E = sample / 65535`; decode to cd/m² using
`10000 * (max(E^(1/m2) - c1, 0) / (c2 - c3 * E^(1/m2)))^(1/m1)` with
`m1=2610/16384`, `m2=2523/32`, `c1=3424/4096`, `c2=2413/128`, `c3=2392/128`.
For physical luminance, use linear BT.2020 weights 0.2627/0.6780/0.0593 after
decoding. Alpha is straight, linear coverage; skip empty pixels. Graphics white
(linear 1.0) is the document's `hdrGraphicsWhiteNits`, or 609 cd/m² by default
(gain × 203 cd/m²); HDR video luminance is independent of this setting.
`filmstrip --hdr` draws its labels at the document's graphics white. Cells are
downsampled in premultiplied linear light before PQ encoding; this is not an
8-bit contact sheet. HDR filmstrips have a 200 MB float-canvas limit
(16 bytes/pixel).
The public CLI exports video, not PNG stills: use preview/filmstrip for PNG
measurements.
If another host supplies export PNGs, inspect their bit depth and
transfer metadata before choosing SDR or PQ measurement.
## Authored RGB above 1
Paint RGB up to 64 (see [FX authoring](fx-authoring.md#paint-colors-above-1))
continues the sRGB transfer function in HDR export: 1 is graphics white and 2 is
about 4.9538 times its linear brightness (luminances in the table above). SDR
export ratio-fits the same color by its largest channel instead, with no extra
brightness. SDR-domain effect colors such as Tint stay 0..1 and cannot produce
light above SDR white. For blend modes over HDR light, see
[HDR blend modes](hdr-blend-modes.md).
## HDR images
Native image import decodes HDR only from these encodings, with the metadata
they need and a native decoder on that platform; a file extension alone is not
an HDR guarantee:
- 16-bit BT.2020 PQ/HLG PNG with full-range RGB `cICP`: supported on native platforms.
- EXR: single non-deep RGB(A) layer, matching data/display windows, Rec.709 or
BT.2020 D65 primaries. Linear associated alpha is preserved; `whiteLuminance`
defines cd/m² at 1.0, otherwise 203 cd/m². ACES/custom primaries, deep/multilayer
images, and nonfinite samples are rejected. Decode limit: 200 MB of float pixels.
- ISO 21496-1 gain-map JPEG and HEIC, plus tagged PQ/HLG HEIC/AVIF: ImageIO on
macOS 15+/iOS 18+. ImageIO must return a recognized BT.2100 PQ/HLG profile.
JPEG gain-map reconstruction is required, not the SDR base.
Native HLG's 1000-nit OOTF is undone to match video/PNG's scene-linear convention
(0.75 = 203 cd/m², peak ≈766); PQ and gain maps retain absolute luminance.
- XMP-only Ultra HDR JPEG is rejected when ImageIO does not reconstruct its map.
The existing JPEG decoder has no gain-map reconstruction; adding a separate
Ultra HDR decoder is not implemented. Prefer ISO 21496-1 sources. AVIF gain-map
decoding is delegated to ImageIO but has no committed acceptance fixture yet.
- HEIC/AVIF/gain-map HDR is not implemented on Linux/Windows: the existing
image crate lacks those HDR decoders; no libheif/libultrahdr integration is
bundled. Untagged/SDR HEIC/AVIF are not accepted by this HDR path.
- Web HDR still decoding is unsupported. Do not use an SDR browser thumbnail as
evidence of HDR luminance. Native HDR images retain absolute source nits after
compensating the document's graphics-white gain, just like video/PNG.
Use `preview --hdr` or `filmstrip --hdr` to inspect preserved HDR samples.
references/install-cli.md
# Installing the CLI
## Choose the bundle
The CLI supports macOS, 64-bit Windows, and Linux x86_64 (glibc 2.35+). Pick the bundle before installing:
- macOS: `arm64` for Apple Silicon and `x86_64` for Intel. Check `uname -s` and
`uname -m`. `sysctl -in sysctl.proc_translated` returning 1 means Apple
Silicon under Rosetta; use the `arm64` bundle.
- Windows: `windows-x86_64` for 64-bit Windows 10 or later. Check
`$env:PROCESSOR_ARCHITEW6432` when set (a 32-bit PowerShell on 64-bit Windows),
otherwise `$env:PROCESSOR_ARCHITECTURE`; either must be `AMD64`. Nothing
needs to be preinstalled; the bundle carries its runtime libraries.
- Linux: `linux-x86_64`; check `uname -m` and `getconf GNU_LIBC_VERSION`.
Installation requires Python 3, bash, coreutils, and unzip. Runtime needs zlib,
ALSA, libstdc++, the Vulkan loader, and a compatible Vulkan driver (Mesa lavapipe is a software
option). These host libraries are not bundled.
On any other host, report that this host is unsupported and stop installation.
## Public CLI downloads
Use the exact version from `cli-version.txt` beside this guide.
Download these two files from
`https://github.com/mirage-hq/tesseract/releases/download/v<VERSION>/`:
- `tesseract-<VERSION>-<PLATFORM>-<ARCH>.zip`
- `tesseract-<VERSION>-<PLATFORM>-<ARCH>.zip.sha256`
Use `darwin-arm64`, `darwin-x86_64`, `windows-x86_64`, or `linux-x86_64` for
`<PLATFORM>-<ARCH>`, following the host checks in [Choose the bundle](#choose-the-bundle).
Use an available downloader (for example, `curl -fL` on macOS or
`Invoke-WebRequest` on Windows) or a browser. GitHub login and GitHub CLI are
not required. Save both files in a fresh download directory.
If network permissions block the download, use the agent's normal permission
flow. If the release or platform asset is unavailable, report the exact URL and
error and stop installation; do not request private repository access or guess
another version.
The release page is
`https://github.com/mirage-hq/tesseract/releases/tag/v<VERSION>`.
Downloaded archives are not installed until the checks below pass.
## Verify and install
1. macOS: from the download directory run `shasum -a 256 -c <archive>.zip.sha256`,
extract with `ditto -x -k <archive>.zip <destination>`, then run
`bash <extracted-CLI-folder>/install.sh`.
Linux: run `sha256sum -c <archive>.zip.sha256`, extract with
`unzip <archive>.zip -d <destination>`, then run `bash <extracted-CLI-folder>/install.sh`.
The default command is `~/.local/share/Tesseract/bin/tsrct`
(`$XDG_DATA_HOME/Tesseract/bin/tsrct` when set).
Windows: compare `(Get-FileHash <archive>.zip -Algorithm SHA256).Hash` with the
`.sha256` file, extract with `Expand-Archive <archive>.zip <destination>`, then run
`powershell -ExecutionPolicy Bypass -File <extracted-CLI-folder>\install.ps1`.
2. Run the installed command with `--version` using its quoted absolute path
(`"$HOME/Library/Application Support/Tesseract/bin/tsrct"` on macOS,
`"$env:LOCALAPPDATA\Tesseract\bin\tsrct.cmd"` on Windows, or
`"${XDG_DATA_HOME:-$HOME/.local/share}/Tesseract/bin/tsrct"` on Linux), check it
against `cli-version.txt`, and use that path for subsequent commands.
The installers check platform, architecture, minimum OS version, and bundle
checksums before switching the installed command. Installing the bundle does not
require sudo, administrator rights, Rust, Homebrew, or Node. Installing missing
Linux system libraries may require administrator access. Mac releases are
Developer ID signed and Apple notarized; first launch may require internet
access for notarization checks, and macOS may still block launch
depending on device policy. The Windows bundles are unsigned, so SmartScreen may
warn. If blocked, let the user review the warning and follow their
organization's approval process. Do not bypass Gatekeeper or
SmartScreen or delete quarantine attributes. Report the exact error if a download,
installation, or launch fails.
## Linux requirements
Rendering requires a compatible Vulkan driver. On servers and other headless
systems, Mesa lavapipe can provide software rendering. The Linux bundle does
not include an H.264 encoder. To export H.264, supply a separately installed
FFmpeg with libx264 and AAC using the existing external encoder backend:
```sh
tsrct export --project project.tsrct --output output.mp4 \
--encoder-backend external-ffmpeg-command --ffmpeg-path /absolute/path/to/ffmpeg
```
Use the installed CLI's absolute path as described above. The supplied FFmpeg
is not included in the bundle. CPU encoding still requires a working renderer.
## Runtime failures
Use the CLI's full error to diagnose the environment before choosing a fix.
`No suitable GPU adapter found` can indicate driver, device-access, or capability
problems; it does not prove a driver is missing. A Vulkan loader alone is not a
working renderer. Compatible Mesa lavapipe can provide CPU rendering without a GPU.
In the ChatGPT desktop app, GPU discovery failures may come from the chat's
execution permissions. Retry through the normal approval flow. If that fails,
suggest checking the permissions menu below the composer and trying “Approve for
me” or “Full access,” then rerun the command. If it still fails, provide the exact
command for the user to try directly in Terminal.
`No usable H.264 encoder` is a separate encoding failure. Installing an `ffmpeg`
executable does not add encoders to the bundled libraries; select the external
backend explicitly as described above. Use its launch or encoding error to
diagnose an invalid executable, missing encoder, or other runtime failure.
Resolve environment issues using the host's available tools and permissions,
then verify with a small preview or export. Preserve the project and rendering
fidelity. If blocked, report the specific failure and what access or dependency
is needed to continue.
references/installation.md
# CLI installation
The skills and plugin contain instructions, not the CLI. First look for the installed
command (`~/Library/Application Support/Tesseract/bin/tsrct` on macOS,
`%LOCALAPPDATA%\Tesseract\bin\tsrct.cmd` on Windows, or
`${XDG_DATA_HOME:-$HOME/.local/share}/Tesseract/bin/tsrct` on Linux), then `tsrct` on PATH,
and run `--version`.
`cli-version.txt` beside this guide pins the required CLI version for normal
installations. Check the executable's reported version against it. If the user
explicitly supplies a CLI path or build for the task, use that executable even
when its version differs; do not replace it with the pinned version or another
build. Verify that it supports the requested features and explain any limits.
If the CLI is missing, reports another version, or fails to launch or is
blocked, follow [installing the CLI](install-cli.md). If launch is blocked, let
the user review the warning and follow their organization's approval process. Do
not bypass Gatekeeper or SmartScreen or delete quarantine attributes. Report the
exact error.
If preview, filmstrip, or export reports a GPU, Vulkan, or encoder error, read
[runtime failures](install-cli.md#runtime-failures) before changing the
environment. The Linux bundle has no H.264 encoder: for H.264 MP4 export, add
`--encoder-backend external-ffmpeg-command --ffmpeg-path <absolute path to an FFmpeg with libx264 and AAC>`
([Linux requirements](install-cli.md#linux-requirements)).
references/local-operation.md
# Running Tesseract locally
Resolve this skill from the directory containing its `SKILL.md`. In commands,
`SKILL` means that absolute directory, whether installed alone or in a
plugin, and `WORK` means the project's absolute `.tesseract-work` folder (see
[project lifecycle](#project-lifecycle)). Quote paths containing spaces.
Tesseract uses `tsrct` to edit and render portable `.tsrct` projects. Follow
[CLI installation](installation.md) to locate the executable,
check its version, or install the matching CLI release. Use the resolved
executable path in the commands below; it does not have to be on PATH.
See [telemetry and attribution](telemetry.md) for optional CLI attribution and user opt-out.
## Scope
For creating or restructuring layers, read [FX authoring](fx-authoring.md).
For adding or revising animation, read [Motion](motion.md).
Use editable FX compositions, text, shapes, media layers, effects, keyframes,
and procedural scripts. Use existing local media. This CLI has no asset
generation, transcription, dubbing, or cloud job tools.
## Project lifecycle
Choose the intended project before writing. Reuse an existing project only when
the conversation, supplied paths, or project notes establish continuation; a
matching name alone is insufficient. Inspect that destination before modifying it.
Ask only when intent to modify existing work remains ambiguous.
For new work, claim a fresh project root with an atomic operation that fails if
the path already exists, such as `mkdir "$project_root"` without `-p`. Check that
it succeeded before writing any project files or creating subfolders. If the path
exists, leave it untouched and choose another name; do not merge new work into it.
`mkdir -p` is appropriate for subfolders only after the project root has been
claimed or established as the intended existing project. A successful `mkdir -p`
or an earlier existence check is not proof that this task created the directory.
Use one persistent project root for authoring, review, revisions, and handoff.
Keep retained working files in subfolders within that root, not in a separate
workspace left behind at delivery. Hand back the same root. A separate client
package is an additional export when requested; relocating the project means
moving the retained workspace together and checking its references.
For example (names are flexible; create only useful folders):
```text
Project/
Project.tsrct current editable document with packaged media and fonts
Project.mp4 matching current video
Previews/ useful filmstrips, poster frames, or comparisons
Versions/ meaningful earlier drafts, when needed
.tesseract-work/ retained notes, scripts, JSON, and diagnostics
```
Keep a readable filmstrip of the delivered revision visible, even if it began as
an internal check. Retain other artifacts according to their value for reviewing,
reusing, or continuing the work. Schema dumps and temporary debugging renders
belong in the working subfolder (or a version-matched schema cache). The `.tsrct`
document is self-contained; loose copies of embedded assets are not sidecars.
Preserve original user media and retain external sources when independently useful.
Save meaningful checkpoints before substantial revisions and keep enough notes
to resume: intent, decisions, current revision, actual checks, and unresolved issues.
Associate exports and previews with their source revision; avoid saving every
small tweak. Tidy disposable intermediates without deleting useful history.
These are workspace conventions, not file-format requirements or mandatory files.
### Storage
A `.tsrct` carries every packaged image, font, clip, and model, so each full copy
of the document stores all of them again. Review stills, exports, and JSON dumps
accumulate the same way. Keep useful history without repeating it:
- Checkpoint a revision as checked-out JSON, for example
`tsrct project checkout --project project.tsrct --output Versions/02-layout.json`.
To return to it, commit that file to the current document; the packaged media it
references must still be in the document. Keep a full `.tsrct` copy only for a
milestone the user approved or a state the JSON cannot restore.
- `checkout` and `inspect --pretty` write indented copies of the whole document,
which can be large for a detailed scene. Reuse their names in `.tesseract-work/`
instead of keeping one per revision, and keep checkpoints only for meaningful
revisions.
- Write routine checks to stable names in `.tesseract-work/checks/` and overwrite
them on the next pass. Name a still for its revision only when it records a
decision or a before/after comparison you will show. Once a round's findings are
in the notes, delete that round's checks.
- Keep the current export and the exports the user reviewed. Delete superseded
iteration exports; their checkpoints can render them again.
- Delete frame dumps, lossless or decoded intermediates, and analysis caches once
their results are recorded.
- In long sessions, check the size of `.tesseract-work/` and `Versions/` with
`du -sh` and retire superseded material before it reaches a few gigabytes.
Never delete the user's original media, the current document and export, or a
milestone the user approved.
## Before editing
Run `tsrct --version` and `tsrct project --help` once per session.
If `import-asset` is missing, follow [installation](installation.md).
The examples below run from the chosen project root; adapt filenames to the task.
Create an empty document for new work, or inspect the intended existing document:
```sh
tsrct project create --project project.tsrct
```
Creation makes one empty composition (`main`) on a 1080×1920 canvas, lasting
three seconds. Inspect its ID before adding content. Change canvas, duration,
or composition name by checking out and committing the editable document JSON.
The document `duration` is in seconds; layer ranges and action times are milliseconds.
Preview, filmstrip and export are SDR by default, even for footage that reports
`hdrTransfer` `hlg` or `pq`. Read [HDR](hdr.md) before using `--hdr` on any of
them (16-bit PQ PNGs cannot be judged by eye), when such footage should stay HDR
in the delivery, when revising a project delivered in HDR, when setting
`hdrGraphicsWhiteNits` or RGB above 1, or when importing EXR, HEIC/HEIF, AVIF,
16-bit PNG or gain-map JPEG images.
Native preview renders at the document's authored dimensions, including custom
canvas sizes. Dimensions must be positive, at most 7680 pixels per side and
33,177,600 pixels total, subject to device limits. Set `backgroundColor` for the
canvas background or use a bottom FX shape layer.
Import supplied footage with `project import-video` before adding its video
layer through JSON checkout/commit. Read [local footage](fx-authoring.md#local-footage).
For images, music, sound effects, and processed audio, follow [media import](media-import.md).
Before adding text, choose and obtain the intended font files, then package them
with `project import-font --file`; see [text](fx-authoring.md#text-with-an-available-font).
Handle setup and editing yourself; the user should only describe the result.
For spoken captions, read [bottom captions](fx-authoring.md#bottom-captions).
Inspect before constructing edits:
```sh
mkdir -p .tesseract-work
tsrct project inspect --project project.tsrct --pretty > .tesseract-work/project.json
tsrct project schema > .tesseract-work/project-action.schema.json
tsrct project schema --document > .tesseract-work/document.schema.json
```
The document contains one composition. Use groups for related parts; do not
create extra compositions or base-video, caption, or audio tracks.
Use IDs from `.tesseract-work/project.json`. Search the saved schemas for the
relevant definition; never read them whole (the action schema is over 1 MB).
Do not invent fields.
## Edit loop
For shape, text, grouping, and animation edits, write a JSON array of supported
composition-local actions in `.tesseract-work/edits.json`, applied as one atomic batch.
The action schema excludes operations this standalone document rejects:
project metadata, canvas changes, composition lifecycle, and asset-introducing actions.
For those document edits and media layers, use the editable JSON schema:
```sh
tsrct project checkout --project project.tsrct --output .tesseract-work/editable.json
# Edit dimensions, duration, composition, or layers using .tesseract-work/document.schema.json.
tsrct project commit --project project.tsrct --file .tesseract-work/editable.json
```
Preserve existing animation graphs and unknown fields when editing JSON. Commit
validates the document and packaged asset references before atomically saving.
Re-checkout after action batches so a stale JSON file cannot undo those changes.
Use the working examples in the references, adapting IDs, timing, and geometry
to the inspected project.
Apply, then render a strip across the changed beat's active range (here 0–3 s).
When text or layout changed, also render a full preview at a time the changed
element is visible:
```sh
mkdir -p Previews .tesseract-work/checks
tsrct project apply --project project.tsrct --actions .tesseract-work/edits.json
tsrct filmstrip --project project.tsrct --start-ms 0 --duration-ms 3000 \
--interval-ms 250 --items-per-row 4 --tile-width 135 --tile-height 240 \
--output Previews/Filmstrip.png
# Only if text or layout changed:
tsrct preview --project project.tsrct --time 1 --output .tesseract-work/checks/layout.png
```
`project apply` and `project commit` print a JSON result whose `warnings` array
lists authoring mistakes that render without an error but not as intended, such
as a hidden track-matte layer. `project inspect`, `project checkout`, `preview`,
`filmstrip`, and `export` print the same warnings to stderr as `warning:` lines.
Each names the layers and the fix; correct them before reviewing frames.
Open and inspect the generated images together. Correct visible problems, save,
and render the affected range again.
## Finish
Before the first export, run `tsrct export --help`; if `--format` is missing,
follow [installation](installation.md).
```sh
tsrct export \
--project project.tsrct \
--output finished.mp4
```
For transparent motion graphics on macOS, export ProRes 4444:
```sh
tsrct export --project project.tsrct --format prores --output finished.mov
tsrct export --project project.tsrct --format prores --fx-solo main:3 --output overlay.mov
```
Full-project export includes the mix and the document background. For alpha,
use solo export of the overlay layer or a group containing the graphic. Solo uses the selected
layer/group's active window, preserves canvas placement, and omits audio. Inspect
its alpha channel and composite over contrasting backgrounds before delivery.
Export defaults to 1080p, 30 fps, and H.264 MP4. Select `--resolution 720p|1080p|4k`,
any positive integer with `--fps`, and `--format mp4|prores`. Resolution preserves the
project aspect ratio: landscape 4K is 3840×2160, portrait 4K is 2160×3840, and square
4K is 2160×2160. Named sizes always set the short side (720, 1080, or 2160 pixels),
even when this upscales a smaller canvas. The proportional long side rounds to the
nearest even pixel, with exact ties rounded upward. Keep the authoring canvas unchanged.
These standalone sizes are not the legacy/iOS canvas multipliers. MP4 bitrate scales automatically
with output size and frame rate (10 Mbps at 1080p30, up to 60 Mbps); ProRes uses
its codec's quality settings. Fractional/source-matched FPS is not exposed.
Inspect the encoded dimensions, frame rate, audio, and visuals before delivery.
```sh
tsrct export --project project.tsrct --resolution 4k --fps 60 --output finished.mp4
```
Export is SDR unless you add `--hdr [auto|pq|hlg]`. Read [HDR](hdr.md) before an
HDR export, when revising a project delivered in HDR, or when setting
`hdrGraphicsWhiteNits`.
To export only a timeline segment, use milliseconds (for example, 2–5 seconds):
```sh
tsrct export --project project.tsrct --start-ms 2000 --duration-ms 3000 --output segment.mp4
```
Either flag may be omitted: start defaults to zero, and duration defaults to the
remaining timeline. With `--fx-solo`, start is relative to the selected layer's
active window; solo exports still omit audio. An out-of-bounds range fails.
To export only the audio mix as a WAV, use `export-audio`. It takes the same
range flags as `export`:
```sh
tsrct export-audio --project project.tsrct --start-ms 2000 --duration-ms 3000 --output .tesseract-work/checks/mix.wav
```
A long export can take many minutes.
Path-traced Scene3D exports are the usual case.
If the host can run a command in the background, start it that way with `--json` and
stderr (NDJSON progress) sent to a log such as `.tesseract-work/export.log`;
otherwise run it in the foreground with a timeout longer than the render. Wait in
blocking steps of about 4 minutes or less, reading only the log's last line
(`completed`, `total`, `etaSeconds`) between steps. Run one render at a time.
Never split an export or lower its settings to fit a wait. Open or probe the
output only after the command exits, and do not edit the document while it renders.
The `.tsrct` document is the portable editable source. Checked-out JSON becomes
the saved source only through `project commit`; do not edit the ZIP manually.
## Safety and failure behavior
- Never edit `project.tsrct` as text.
- Preserve the input with `--output .tesseract-work/revised.tsrct` when experimenting.
To keep the change, re-apply it to the project document or move the trial over
it; delete a discarded trial.
- `project apply` is atomic: a rejected action batch does not publish partial
project changes.
- Preserve the `.tsrct` document and its packaged resources.
- This release exposes the supported subset of the engine action schema. Use only actions
required for the user's request.
- For `missing_fonts`, obtain and import the requested files, then retry.
Follow [font selection and import](fx-authoring.md#text-with-an-available-font);
names in the error identify the requested faces.
If import fails, report the actual error. Do not silently replace a requested face.
- FX text style and layout changes go through `project apply`.
- If preview, filmstrip, or export reports a GPU, Vulkan, or encoder error, read
[runtime failures](install-cli.md#runtime-failures) before changing the environment.
references/media-import.md
# Import images and audio
Import copies bytes into the portable document. It does not create timeline layers.
Use a unique flat asset ID; existing IDs cannot be replaced accidentally.
```sh
tsrct project import-asset --project project.tsrct --file /absolute/path/logo.png --asset-id logo --kind image
tsrct project import-asset --project project.tsrct --file /absolute/path/music.wav --asset-id music --kind audio
```
Accepted image suffixes: PNG, JPG/JPEG, WebP, EXR, HEIC/HEIF, AVIF.
Audio: WAV, MP3, M4A, AAC, FLAC, OGG.
Before importing an EXR, HEIC/HEIF, AVIF, 16-bit PNG or gain-map (Ultra HDR)
JPEG, or when an image fails to decode, read [HDR images](hdr.md#hdr-images) for
the supported encodings, platforms and rejected forms.
The underlying decoder must support the file's encoding.
Import checks the file and suffix and packages it, but does not decode or
return dimensions/duration.
Probe metadata with local media tools, then verify decoding in a preview/export.
Footage, local font files, and 3D `.cube` color LUTs are imported too; for
LUTs, see [color grading](color-grading.md).
Use `project import-video` for footage, `project import-font --file` for font
files, and `--kind lut` for LUTs.
A 3D-enabled CLI also imports models, IES photometry, cloth point caches and
HDR environment images; [Scene3D basics](scene3d-basics.md) lists the kinds.
After import, add an Image or Audio layer to the document JSON using the
installed document schema and the imported asset ID.
Check out the document first and commit it afterwards.
Preserve existing layers and use unused numeric layer IDs. For a 200×100 image:
```json
{"type":"Image","id":1,"name":"Logo","activeRange":{"start":0,"duration":3000},"transform":{"anchorPoint":[100,50],"position":[320,180],"scale":[100,100],"rotation":0,"opacity":100},"source":{"assetId":"logo","fit":"contain"}}
```
Use actual image dimensions for the anchor and choose placement for the canvas.
For Audio layer fields, source timing, gain, and envelopes, read
[audio and timing](audio-and-timing.md). Generated cues and FFmpeg-processed
derivatives follow the same import path. Keep originals and processing recipes;
mute the original dialogue when replacing it to avoid doubled playback.
references/motion-design.md
# Motion design: full scenes and overlays
## Design the motion before the syntax
State one visual idea in concrete terms: a number grows into a comparison; a line reveals the flow through a process; a label follows the subject's direction; typography is revealed by the product's silhouette. If the only idea is “things fade in,” reconsider the design.
Decide the focal element, its relationship to the footage, and how the composition changes over time. A full scene should have composition, progression, and a payoff, not just a centered heading above three rounded boxes.
For motion that opens a paid-social or YouTube ad, read [ad hooks](ad-hooks.md) for concept selection, placement context, music/SFX synchronization, and the early opening review.
## Overlays
- Start with the footage's actual safe area. Account for the face, hands, product, subtitles, and platform cropping. Check placement across the whole interval, not one still.
- Prefer a small label or shape that points to meaningful evidence. A large opaque panel is appropriate only if the brief calls for that treatment.
- Animate in, hold long enough to read, animate out. Entrance and exit should relate to the cut or the subject's movement.
- Use contrast from placement first, then a subtle scrim/stroke/shadow as needed. Do not blanket every shot in a heavy gradient.
- For graphics behind a person, use a person matte when its required local model/resource is available, and confirm its polarity. Verify support on an actual frame first; the skill does not install segmentation models. A person-white matte with **lumaInverted** reveals graphics outside the person; **luma** reveals them inside the person. Some engine recipe prose describes the latter without stating the intended visual relationship—pick and test the polarity for the actual design.
- For a tracked pointer, do not describe it as tracked unless there are actual time-varying positions aligned to observed footage. Static arrows are fine when appropriate.
## Full-frame scenes
Build a hierarchy: focal content, supporting context, environment. Use spatial and temporal relationships to explain meaning. Possibilities include a native vector schematic, dimensional arrangements of 2D layers, split-screen comparisons, mask-driven type reveals, data with accurate values, or footage integrated into a composited scene.
Reserve motion for changes that guide attention. Give the viewer a stable interval to understand the result. Avoid defaulting every scene to parallax, glow, particles, bouncing labels, and a camera push at once.
## Native mechanism map
| Design need | Tesseract mechanism | Look up |
|---|---|---|
| Type hierarchy / editable text | `Text.sourceText` and imported font files | [FX authoring](fx-authoring.md#text-with-an-available-font) and the document schema |
| Per-character reveals | Text Animator and selectors; drive fields with dynamics | [FX authoring](fx-authoring.md#text-with-an-available-font) and [motion](motion.md) |
| Coordinated movement | `Group` with child layers and stable parent IDs | [FX authoring](fx-authoring.md) |
| Drawn paths / linework | `Shape` paths, fills, strokes; animate supported path fields | the document schema's Shape definitions |
| A moving reveal | Track matte or path mask using a real layer reference; leave a track-matte layer visible (it is not drawn itself, and hiding it drops the matte) | the document schema's mask and track-matte definitions |
| Treatment across several layers | Group effects or an Adjustment layer above the affected siblings | the document schema's effect definitions |
| Soft lighting, blur, grade | Ordered typed layer effects | the document schema's effect definitions |
| 3D-looking layer arrangement | Supported 3D transform properties and depth ordering | the document schema and [motion](motion.md) |
| Custom pixel treatment | Bounded premultiplied-alpha WGSL custom shader | [custom shader contract](custom-shader.md) |
This is not a Blender runtime.
Transforming 2D layers in 3D belongs here; general meshes, cameras, lights, imported models and physics are Scene3D items, authored with the Tesseract: 3D Scenes skill and [Scene3D basics](scene3d-basics.md).
## Timing and motion character
Tesseract FX uses editable keyframe actions and `AnimationGraph` expressions, not CSS or AE keyframe-array JSON. Read [motion](motion.md) for the installed action workflow. Use `layerTimeJsCode` and the owning layer's clock. An expression returns its value; it is not a general browser script.
For a simple reveal, 200–450ms is a useful starting range, adjusted to the scale of movement and edit. Give secondary details a small deliberate offset. Choose easing based on mass and purpose; a restrained ease-out usually reads more deliberately than an automatic spring. Overshoot can fit playful material but should be intentional.
Use standard motion blur only where it improves fast transforms; enable both composition and layer switches. Long tails of blur can erase text or mask a timing error. Render the actual movement before committing to the treatment.
## Brand application
Use supplied fonts, palette, logo geometry, and references. Keep exact logo art separate from generated scenery. This plugin's Mirage icon is not a default asset for client work. With no supplied style, choose a restrained, legible direction appropriate to the footage and describe it briefly.
## Reusable examples
[FX authoring](fx-authoring.md) contains editable examples for footage, a card, grouped labels, and captions. [Motion](motion.md) shows keyframes and procedural animation. Adapt the composition and motion to the brief rather than copying the entire example look.
references/motion.md
# Motion
Use editable keyframes for specified poses, entrances, holds, and exits. Use
scripts for periodic motion or relationships that would otherwise require many
keys. Keep the structure editable: move a group for coordinated motion and its
children for independent details.
## Clocks and keyframes
`layerTime` is integer milliseconds from the owning layer's local start, not
project time. Ancestor group playback affects that clock. Keys outside the
visible interval may remain authored; the active window controls visibility.
This fades in group 3 from the FX authoring examples, holds it, then fades out:
```json
[
{"type":"setFxPropertyKeyframes","compositionId":"main",
"property":{"layerId":3,"propertyType":"opacity"},
"keyframes":[
{"id":"callout-opacity-in","layerTime":0,"value":{"type":"float","value":0},"easing":{"type":"linear"}},
{"id":"callout-opacity-visible","layerTime":300,"value":{"type":"float","value":100},"easing":{"type":"cubicBezier","x1":0.2,"y1":0,"x2":0.2,"y2":1}},
{"id":"callout-opacity-hold","layerTime":2500,"value":{"type":"float","value":100},"easing":{"type":"linear"}},
{"id":"callout-opacity-out","layerTime":2900,"value":{"type":"float","value":0},"easing":{"type":"linear"}}
]}
]
```
Easing belongs to the destination key. Before/after the track the first/last
value is held. Keep key times distinct and IDs stable and composition-unique.
`setFxPropertyKeyframes` upserts by ID: omitted keys remain. Use
`removeFxPropertyKeyframes` to remove specific existing keys. Inspect the track
before revising it; do not accidentally append another animation over old keys.
Use `setFxPositionKeyframes` for paired spatial position paths with tangents.
Let the action API maintain the stored animation representation.
## Procedural motion
Check the installed schema before constructing a `setFxPropertyAnimator` action.
A `jsScript` body uses `layerTimeJsCode`, explicitly returns the target's value,
and reads `input.time.seconds` or `input.time.milliseconds`. For a scalar rotation
in degrees, a bounded oscillation body is:
```js
return Math.sin(input.time.seconds * Math.PI * 2) * 3;
```
Animation must evaluate correctly at arbitrary timestamps: do not accumulate
state frame by frame or use wall-clock time. Declare property dependencies when
following another property; dependency values are canonically ordered, not
necessarily in authored array order. Preserve existing relationships when editing.
## Design and review
Choose motion to support the requested emphasis. Keep text still long enough to
read; avoid moving every part independently. Establish the final layout first,
then add motion. Use easing for a deliberate arrival and linear interpolation
for constant-rate movement. Check overshoot against the available space.
Sample the entrance, settled hold, and exit, including just before and after a
transition. For the example, sample 0, 150, 300, 1400, 2500, 2700, and 2900 ms.
These are project timestamps: offset them when the layer or its group starts
later. The exact end of a half-open active window is already invisible.
Review both the whole project and a solo strip of `main:3`. A content-bounds crop
helps inspect details but cannot prove correct canvas placement. Filmstrips
show sampled poses; inspect playback of the exported video for pacing, flicker,
and audio that sparse stills can miss.
references/native-authoring.md
# Authoring native Tesseract projects
## Local workflow
Use [local operation](local-operation.md) to create or inspect the document,
edit its JSON directly, and apply composition-local action batches.
Read [FX authoring](fx-authoring.md) before changing project structure, and
[motion](motion.md) before changing animation. These contain working examples;
look up the installed fields in the document and action schemas.
## Essential invariants
1. `layers[0]` is topmost. Preserve sibling order when inserting footage behind graphics.
2. IDs are identities, not array positions. Layer IDs are unique throughout the composition's nested tree. Effect, mask, style, and text-animator IDs used by dynamics must remain stable.
3. New visual media uses `Video` or `Image`, not the `Media` tag. Import images before placing them; see [media import](media-import.md).
4. FX transform position is in parent pixels; anchor is layer-local. Scale and transform opacity are percentages, with `100` as identity. Authored paint RGB uses finite encoded extended-sRGB `0..64`; coverage alpha remains `0..1`. SDR ratio-fits RGB; for HDR light, see [HDR](hdr.md). AEP/Premiere export clips extended RGB per channel to `1` with diagnostics, losing HDR brightness and ratios; existing alpha/export limitations remain. Intrinsic emoji and baked PAG bytes are not amplified. Audio gain is linear. Check each field's schema rather than applying one universal percentage conversion.
5. The document owns one composition beginning at zero.
Document `duration` is seconds.
Layer `activeRange`/`playback` and `sourceRange` are milliseconds. Nested layer ranges are relative to the immediate parent. Moving a clip must not change its selected source moment.
6. `layerTimeJsCode` reads `input.time.seconds` from the owning layer's start. It must explicitly return a finite scalar/vector/color matching the target. Use stable IDs and declare dependencies. There is no DOM, CSS, or browser timeline.
7. Group related layers with supported grouping actions. Group effects see precomposed children; a normal layer effect sees only that layer. An Adjustment affects siblings below it. Inspect the schema before constructing a nontrivial feature.
8. Never reconstruct an unfamiliar document from a minimal template to make one correction: that drops existing layers, animation, and resources. Preserve unknown fields, and re-read the document JSON after action batches so a stale copy cannot undo them.
## Footage editing
Import each distinct local video once. Select cuts
using Video layers: a clip from source 2.2–5.7s placed at edit 0–3.5s has
`sourceRange: {start: 2200, duration: 3500}` and a windowed
`playback` with `inputRange: {start: 0, duration: 3500}` and
`inputOffsetMs: 2200` at composition root (Video/Audio/Group use `playback`,
never `activeRange`; see the footage example in fx-authoring). Set
`sourceIntrinsicDuration` to the actual full source duration returned by import.
Set `volume: 1.0` on new Video layers to enable embedded source audio; an
omitted/null volume disables it. Preserve existing gain on revisions unless
deliberately changing the mix. Do not add a second
audible copy of the same sound. Use the Video layer's transform for a footage
push-in, or a Group transform when footage and graphics should move together.
For speed changes, maintain both source and edit duration deliberately. Inspect
the installed playback schema and render the result rather than inferring speed
behavior from different duration values. See [audio and timing](audio-and-timing.md).
### Shot-local corrections without recutting
An imported Video can contain a complete multi-shot edit. Separate native
windows let you correct one shot without replacing media or changing its cut
times. When independent windows are needed on a plain Video layer with no
existing animation, apply a `splitFxLayer` action rather than
manually recalculating its source clock.
Check the installed action schema first. For example, if layer 1 in composition
`main` spans project time 0–10 seconds and ID 2 is unused:
```json
[{"type":"splitFxLayer","compositionId":"main","layerId":1,
"newLayerId":2,"timeMs":2000}]
```
This creates adjacent windows 0–2s and 2–10s. The original keeps ID 1; the right
part gets ID 2. `timeMs` is project-absolute milliseconds and must be strictly
inside the layer window. Use actual confirmed cut times and unused IDs, not
these example values. Re-inspect after each split: later cuts target the layer
whose window contains them, and copied effect IDs may change.
Video splitting partitions the playback window while retaining the source
asset, source selection and mapping. Do not reset `inputOffsetMs`, restart the
right-hand clip at source zero, or shorten `sourceIntrinsicDuration` to the
shot's length. A source selection wider than a resulting playback window is
expected. Preserve transforms, source framing and audio gain; do not add a
second audible copy of the original full-length layer.
Before grading, preview before, at and after the split and within both windows
against the unsplit project. Confirm identical source frames, no gap/overlap,
unchanged duration/framing and continuous audio. Then add the shot-specific
correction and review the final export too.
Do not blindly apply this plain-layer recipe to an animated, nested or retimed
edit. In particular, composition-level property animators stay on the original
layer when splitting; the right part does not automatically inherit them.
Inspect dependencies and the installed capabilities, preserve the existing
behavior, or use supported time-keyed color controls instead. If preservation
cannot be verified, report the limitation rather than silently losing animation.
## Assets and fonts
Resources are packaged in the document. A filename or invented asset
ID in JSON does not import bytes. Use the asset ID that import returns.
Choose and obtain fonts using the [text workflow](fx-authoring.md#text-with-an-available-font).
Import a local TTF/OTF/TTC before creating text. Use the returned `fontFamily`
and `fontStyle`. Import embeds font bytes in the document. Import audio and
images as in [media import](media-import.md).
Footage uses `project import-video`, fonts `project import-font --project
project.tsrct --file /path/to/font.ttf`, and audio and images `project import-asset`.
If rendering reports missing fonts, import the
required files and retry. Do not silently replace the requested font.
Preview, filmstrip, and export use the same packaged font bytes. Check rendered
glyphs and spacing; a fallback font is not proof the intended face was used.
references/review-and-delivery.md
# Review and delivery
## Review a render, not an intention
There are three different checks:
1. **Structural:** the document saves without errors; assets resolve; output is produced.
2. **Technical:** source frames and intervals are correct; alpha and audio are present as intended; duration, resolution, and codec match the brief; fonts render correctly.
3. **Creative:** the composition communicates the intended idea; hierarchy, timing, and style match the brief and supplied references.
Passing the first two does not justify calling the result excellent. Avoid inflated ratings or unearned claims of professional quality. Be specific about what was reviewed. Scores come from the [critic review](critic-review.md); report them exactly as the critics returned them.
## What to inspect
- Composition: clear hierarchy, readable text, consistent type and palette, and
enough time to understand the content.
- Placement: intended spatial relationships, framing, and no accidental overlap
or clipping throughout the relevant interval.
- Appearance: fonts, images, materials, lighting, and effects match the intended
result at delivery size.
- Motion, when present: entrances, holds, exits, and transitions work at playback
speed. Sample feature-critical times and inspect full-size frames wherever
text or detail matters. A filmstrip alone does not establish smooth motion.
- Editability: native elements remain editable, stable IDs and timing are
preserved, and the saved document can be reopened with its packaged resources.
Inspect the saved output from the current revision. Apply the specialist checks
selected by the skill for the task, and report any checks that could not run.
## Focused technical checks
For transparent overlays, export ProRes 4444 MOV as described in [local operation](local-operation.md).
Extract a representative alpha frame with FFmpeg `-vf alphaextract` and inspect it;
verify both uncovered transparent pixels and the intended opaque content. Composite
over light and dark backgrounds to check edges and shadows. MP4 does not carry alpha.
For fonts, render representative text with the imported files and check it against
[the chosen typography](fx-authoring.md#text-with-an-available-font). Check both
visual fit and glyph/layout correctness; successful export alone proves neither.
## Handoff
After the checks above, hand off the version the [critic review](critic-review.md)
settled on, running the review first if you have not; if anything changed after
its last scored round, say so as described there.
Use the same project root established in [project lifecycle](local-operation.md#project-lifecycle).
Check that the current document, export, and previews correspond and packaged
resources resolve. Review the supporting artifacts: make useful previews visible,
keep retained authoring files within the working subfolder, and tidy disposable
scraps. Retire superseded full document copies, routine checks, and iteration
exports as described in [storage](local-operation.md#storage); keep the JSON
checkpoints of meaningful revisions with the project. Do not turn handoff into a
second workspace containing selected copies.
Show the rendered preview or playable video appropriate to the task and link
the editable document, useful previews, and project folder. Briefly describe the result or changes, the critics' final scores,
and any actual limitations.
Surface actionable review findings rather than listing internal files or raw
measurements. Do not describe generated clips as real footage or mocked workflows
as working integrations.
references/scene3d-basics.md
# Scene3D basics
Scene3D is an editable 3D scene rendered as one ordinary layer in Tesseract's
composition. Its children are 3D items, not ordinary 2D layers. Cameras select the
view; lights and materials describe illumination and surfaces. Keep 2D titles,
footage and effects outside the scene when that makes the composition clearer.
These capabilities require a Tesseract CLI built with 3D support.
## Coordinates, clocks and identities
X points right, Y down and Z into the screen. Positions and sizes are pixels,
rotations are degrees and scale is a percentage. Imported models keep their own
up axis: a Blender or glTF model already stands upright, so do not add a
corrective 90° or 180° rotation. Material color inputs (`baseColor`,
`emissiveColor`, …) are scene-linear BT.709 RGB: convert a hex or sRGB picker
value with the sRGB transfer function first (#C0392B is about
[0.527, 0.041, 0.024]). Light, fog, world and 2D colors are extended sRGB.
Every Scene3D has one required `pixelsPerMeter`, the scene's only physical
calibration: imported GLB metres, physical-lens apertures, cloth, fog distances
(metres), volume density (per metre) and material lengths (metres) all use it.
It does not rescale authored pixel geometry. With `pixelsPerMeter` 100, one
Blender unit (one metre) is 100 pixels: a 1.8 m figure is 180 pixels tall.
Lights are physical: point, spot
and area lights author emitted `power` in watts; parallel lights author
`irradiance` in W/m². Exposure is in stops: the scene's owned `view` (with its
own item `id`) plus the active camera's `exposure`. An added light or mesh that
omits `castsShadows` casts no shadow; set it to `true` where shadows matter.
Look up each property's range in the installed `project schema`; not every
numeric field animates.
The document duration is seconds. Layer windows and keyframe `layerTime` use
milliseconds. Scene3D has windowed playback; its content clock follows the
mapping and input offset, including ancestor group playback. Use content-clock
key times, not a guessed project-time offset. Preview timestamps are project time.
The owning `layerId` is an ordinary composition-layer identity. Child `itemId`,
material, environment and nested-control IDs are composition-wide item identities.
Choose unused integer IDs after inspection and preserve them during revisions.
A create or add that names an ID already in use fails, and the apply changes
nothing.
Scene contents are flat `scene: {items, data, materials}` tables. Objects carry a
required nullable `parentId`; Mesh/Text/Model/Curve objects reference a `dataId`.
Geometry and model assets belong to Data, while stable material slots belong to
objects. Publish dependent Data and objects atomically with `addFxScene3DFragment`.
`styleProperty` addresses use the owning Scene3D `layerId`, the particular control's
`itemId`, and an exact `propertyName`.
Text Data `bevelSize` rounds glyph edges inward by default, as Cinema 4D does:
the silhouette keeps the outline, the front and back caps shrink, and strokes
thinner than twice the size rise to a ridge. `bevelSegments` sets the profile's
smoothness (1 is a flat chamfer). Set `"bevelOutside": true` to keep the caps
and grow the walls outward instead; if any glyph's walls would touch, the whole
text is clamped to the largest size that clears them. The size is limited to
half `depth`.
## Create a scene and animate a mesh
Start a new document and save the following batch as `scene.json`. It assumes
composition `main` and an otherwise empty document with at least three seconds.
The camera looks along +Z at the box; its orthographic view is 640 pixels wide (`scale` is the span along the camera's sensor-fit axis,
which defaults to horizontal here). Every camera requires a tagged `lens`:
Angle owns its projection and artistic aperture controls; Physical owns focal
length, sensor dimensions and f-number. Animate controls on the camera's item ID.
The layer renders through its active camera, which `setFxScene3DActiveCamera`
selects. Without one (`activeCamera` is `null`), it frames the canvas and ignores
its authored cameras, so camera edits change nothing.
Replacing the lens retains compatible channels; incompatible animation losses
require explicit `pruneAnimation` consent, otherwise the edit rejects atomically.
```json
[
{"type":"createFxScene3DLayer","compositionId":"main","layerId":1,"name":"Product scene","pixelsPerMeter":100,"view":{"id":9,"exposure":0},"activeRange":{"start":0,"duration":3000},"transform":{"anchorPoint":[0,0],"position":[0,0],"scale":[100,100],"rotation":0,"opacity":100},"world":{"id":2},"renderSettings":{"id":97,"filmGate":null,"overscan":null,"lightGroups":[],"lightCollections":[],"lightLinks":[],"outputs":[],"previewOutput":null}},
{"type":"addFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Camera3D","id":3,"parentId":null,"name":"Front camera","transform":{"position":[540,960,-1200],"relationships":{"constraints":[{"type":"aim","id":80,"target":{"id":800,"target":{"type":"world"},"offset":[540,960,0],"offsetSpace":"local"},"upSpace":"local","forwardAxis":"positiveZ"}]}},"lens":{"type":"angle","projection":{"type":"orthographic","scale":640}}}},
{"type":"setFxScene3DActiveCamera","compositionId":"main","layerId":1,"cameraItemId":3},
{"type":"addFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Light3D","id":4,"parentId":null,"name":"Key light","emitter":{"type":"parallel","irradiance":3}}},
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"mesh","id":100,"name":"Box geometry","geometry":{"type":"box","size":[180,180,180]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]}],"items":[{"type":"Mesh3D","id":5,"parentId":null,"name":"Box","dataId":100,"packing":"whole","transform":{"position":[540,960,0]},"materialSlots":[{"id":101,"name":"Blue","binding":{"id":6,"inputs":{"baseColor":{"type":"constant","value":[0.010022825574869039,0.21404114048223255,0.7874122893956174]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":101},{"region":"side","slotId":101}]}]},
{"type":"setFxPropertyValue","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":5,"propertyName":"rotationX"},"value":{"type":"float","value":-20}},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":5,"propertyName":"rotationY"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"box-start","layerTime":0,"value":{"type":"float","value":-25},"easing":{"type":"linear"}},{"id":"box-end","layerTime":2500,"value":{"type":"float","value":65},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]}
]
```
```sh
mkdir -p .tesseract-work
tsrct project create --project scene.tsrct
tsrct project apply --project scene.tsrct --actions scene.json
tsrct project inspect --project scene.tsrct --pretty > .tesseract-work/project.json
tsrct preview --project scene.tsrct --time 1 --output scene.png
tsrct filmstrip --project scene.tsrct --timestamps-ms 0,1000,2500 --output scene-filmstrip.png
```
## Model constraints and node selectors
Model constraints are authored with `addFxScene3DModelConstraint`, using the
owning Scene3D `layerId` and Model3D object's `itemId`. Each constraint needs its
own unused `id`; target bindings also need distinct IDs. `lookAt` selects `node`
and an `aimAxis`, while `twoBoneIk` selects `tip` and optionally `mid` and `root`.
Both take a `target` binding (for example a world target with an `offset` and
`offsetSpace: "world"`). IK optionally takes a `pole` binding and
`keepTipRotation`; `influence` blends the result from 0 to 1.
These numeric selectors are **zero-based glTF node indices in the model asset**,
not the IDs in the object's `nodes` overrides, nor positions in a skin's `joints`
array. Use the import report's `model.nodes[].index` alongside its node names and
parents. A non-skin node is not invalid: ordinary node hierarchies work too.
IK requires `mid` to be a proper ancestor of `tip`, and `root` a proper ancestor
of `mid`; an omitted selector uses the next node's immediate parent. For the
in-repo limb rig, mesh node 0 has no chain, whereas shoulder 1 → elbow 2 → wrist 3
supports `tip: 3, mid: 2, root: 1`. An override with `id: 7000` for wrist 3 does
not make `tip: 7000` valid.
Run `tsrct project check --project scene.tsrct` after authoring constraints.
It resolves packaged GLBs and reports invalid selectors with the constraint ID,
field, JSON pointer, and valid node range or ancestry requirement. It does not
require a skin joint and does not reinterpret selectors as override IDs. Check
is read-only; it diagnoses invalid addresses even in an already-saved project.
## Rules the validator enforces
Every batch is atomic: a rejected batch changes nothing, and `project apply`
exits non-zero. Chain the following render with `&&` (or check the exit
status) so that a rejection is not followed by a preview of the unchanged
document. Most rejections in practice come from these rules:
- Enum values are exact and case-sensitive. A rotation is tagged:
`{"type":"euler","order":"XYZ","orientation":[0,0,0],"rotationX":-20,"rotationY":30,"rotationZ":0}`
(`orientation` is a static base, the three channels animate) or
`{"type":"quaternion","value":[x,y,z,w]}` with a unit quaternion. Its length
must be within 2e-6 of 1, so float32 values (for example from a glTF file) and
values with at least six decimal places are accepted. The document stores
such a value as its unit form, which is the same rotation. Normalize values
with fewer decimal places in double precision first. A rejection reports the
measured length.
- Mesh, Text and Model objects take `{position, rotation, scale, anchorPoint}`;
cameras and lights take only `{position, rotation}` and optional
`relationships`.
- `setFxPropertyValue` and `setFxAnimator` have no top-level `layerId`; the
layer, item and property belong inside `target`.
- Mesh Data `regions` start with `{"key":"primary","name":"Primary"}` then
`{"key":"side","name":"Side"}` (every region needs a `name`); custom regions
follow. Text Data regions are exactly `front`, `back`, `side`. The object's
`regionSlots` map every declared region exactly once; `"packing":"whole"`
needs every region bound to the same slot, otherwise use `"regions"`.
- Actions apply in order, so an object can reference only objects that an
earlier action or the same fragment adds (an aim, follow or path target, a
scatter target). Within one fragment, list a modifier's operand (a
shrinkwrap, boolean or data-transfer target) before the mesh that uses it.
- Look up a definition instead of guessing its fields. Search the saved schema;
never read it whole. After
`tsrct project schema > .tesseract-work/project-action.schema.json`, print
`json.load(open(".tesseract-work/project-action.schema.json"))["$defs"]["JBSetFxScene3DTiltShiftAction"]`
(or any `JB…` name that an error or another definition references) with
Python. Its top-level `discriminator.mapping` lists every action type.
[Scene3D properties](scene3d-properties.md) lists every `propertyName` each kind
of scene item accepts, including the ones that exist only while a control such
as depth of field or fog is present. Names outside that list are rejected.
Search it rather than reading it whole; the section headings in the output show
the kind each hit belongs to:
```sh
grep -n -e '^## ' -e '`<propertyName>`' <skill-root>/references/scene3d-properties.md
```
A repeated name is a separate mode-specific row, possibly with another value
type or unit; if it repeats within one section (modifiers, geometry), read that
section before choosing. Texture, indexed and font-axis names
(`baseColorTextureOffset`, `curveVertex_3`, `opsz`) have no rows of their own:
they are a slot prefix plus a suffix row (`Offset`) or a pattern, all explained
in the file's last section.
`subsurfaceColor` replaces `baseColor` in the subsurface share rather than
tinting it (OpenPBR). Leave it unset to scatter with the base color. The
packaged CLI's action schema does not expose `setFxScene3DMaterialInput`;
inspect its material-edit actions before changing an existing input.
`setFxPropertyValue` changes the static base without detaching animation. Existing
animation can still determine the sampled value. For style targets use
`setFxAnimator` with an editable keyframe animator, as above. Preserve key IDs when
revising motion; do not inadvertently replace a useful animation stack.
## Physical tilt-shift setup
Use the single `setFxScene3DTiltShift` action on a Camera3D, not a 2D blur.
Give the camera a 200mm Physical lens with a 36×24mm sensor, then apply it:
```json
[
{"type":"updateFxScene3DCameraControls","compositionId":"main","layerId":1,"itemId":3,"changes":{"lens":{"type":"physical","focalLengthMm":200,"sensorMm":[36,24],"fNumber":8}}},
{"type":"setFxScene3DTiltShift","compositionId":"main","layerId":1,"itemId":3,"focusPlaneTiltDeg":8,"tiltDirectionDeg":90,"focus":{"type":"distance","distanceM":30},"aperture":{"type":"inFocusDepth","inFocusDepthM":4},"circleOfConfusionMm":0.03}
]
```
This writes existing camera controls: focus-plane Euler tilt [8,0], focus
30 metres, f-number about 2.94991, and circular bokeh. The axial near/far depths
are about 28.13275–32.13275m, not 28–32m. The 8° is the **focus plane**, not
mechanical lens tilt. The sharp region is a tilted wedge, not a horizontal band.
`shiftMm` defaults to [0,0]; positive axes frame right/down. A fixed-aperture
request uses `{"type":"fNumber","fNumber":2.8}` instead. Default CoC criterion
is sensor diagonal / 1500 when omitted.
The action never guesses physical dimensions. Switch an Angle camera to a
Physical lens first with `updateFxScene3DCameraControls`, as above; the
action's own `physicalLens` conversion needs a precomputed consent plan that
the CLI cannot produce. To focus on an object instead of a distance, use
`"focus":{"type":"target","itemId":5,"fallbackDistanceM":30}`; the target
must be an existing object of the same scene. The apply report lists the
action as applied; read the resulting `focusPlaneTilt`, `focusDistance`,
`fNumber` and `lensShift` from `project inspect` and judge the sharp region in
a preview.
Zooms and moving shots are supported directly: key `focalLengthMm` (and any
camera or target motion) **before** applying the action, and do not remove the
zoom to apply it. An `inFocusDepth` request holds that depth over the whole
shot: when the zoom or focus distance changes it, the action writes `fNumber`
keyframes solved from the evaluated focal length and focus distance at each
time (report `shot.fNumberKeys`). Never pick one focal length (for example the
median) and solve the aperture yourself. An `fNumber` request stays constant, so
the depth then changes with the zoom, as with a real lens. Do not combine
`physicalLens` with an animated `focalLengthMm`; it is rejected. With
`inFocusDepth`, author the zoom and the camera/target motion as ordinary
keyframes: scripts and animation-layer (property source or model clip) motion
of the camera or target are rejected.
After editing the zoom or motion, rerun the action. An unchanged rerun is a
no-op; replacing different `fNumber` keys needs a consent plan that the CLI
cannot produce, so first clear the camera's `fNumber` animator with
`clearFxAnimator`, then rerun.
The action resets blades/rotation/ratio to a circular pupil and rejects other
affected camera animation, including disabled tracks. Follow up with ordinary
`focusPlaneTilt`, `focusDistance` or `lensShift` keyframes when needed.
No recipe or live depth-width lock is stored. Export retains its existing
aperture integration and Playback retains its approximation.
## Structural edits
`moveFxScene3DLayer` retains item IDs and animation. `toIndex` is an insertion
slot **after removal**; `parentItemId` names a destination Group3D, or omit it for
the scene root. Reparenting normally retains local values and therefore changes
world placement. Request `preserveWorldTransform` only when its supported static
transform can represent the result; unsupported animation or shear rejects.
`duplicateFxScene3DLayer` takes `itemId` and `newItemId`. It copies geometry
independently, with fresh identities for owned children, slots, controls and
animation. Inspect references after copying:
light-link remapping has a known limitation. Shared material definitions are not
necessarily made unique merely by duplicating geometry.
`realizeFxScene3DArray` turns a Group3D's array into ordinary editable copy
groups at content time `time` (default 0); copy 0 keeps the original IDs, for
example `{"type":"realizeFxScene3DArray","compositionId":"main","layerId":1,"itemId":8}`.
It needs `pruneAnimation` only when it would drop animation (animated counts or
seeds, or animation of an inline material the array's per-copy paint replaces);
other animated array controls reject. It cannot read Text3D or Model3D surfaces
yet, so a scatter onto one, or paint on Model3D copies, rejects.
`removeFxScene3DLayer` removes the item/subtree with everything its objects own
and prunes their animation and dependencies. It needs `pruneAnimation` only when
it would drop animation (of the removed items or what they own, or
animation-layer uses and masks naming them) or a reference a surviving item holds
to them, such as a camera's focus; removing static content needs no flag.
Current removal is not uniformly reference-preserving: some references reject,
others are cleaned up or can remain dangling. Inspect dependent items before
removal; retain a backup. Whole-item `updateFxScene3DLayer` can also
prune tracks for omitted controls. Prefer the focused actions listed by
`project schema`; when replacing a whole item, preserve the complete payload and
all unrelated IDs.
For the scene above, save this as `duplicate-move.json`. The original and copy are translated
left and right so both boxes fit; moving changes stack order, not position.
```json
[
{"type":"duplicateFxScene3DLayer","compositionId":"main","layerId":1,"itemId":5,"newItemId":7},
{"type":"moveFxScene3DLayer","compositionId":"main","layerId":1,"itemId":7,"toIndex":0},
{"type":"setFxPropertyValue","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":5,"propertyName":"position"},"value":{"type":"vector3","value":[400,960,0]}},
{"type":"setFxPropertyValue","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":7,"propertyName":"position"},"value":{"type":"vector3","value":[680,960,0]}}
]
```
```sh
tsrct project apply --project scene.tsrct --actions duplicate-move.json
tsrct project inspect --project scene.tsrct --pretty > .tesseract-work/project.json
tsrct preview --project scene.tsrct --time 1 --output duplicate.png
```
Then save this as `remove-copy.json` and remove only the copy. Its copied
rotation keys need explicit `pruneAnimation` consent; without it, the action
rejects atomically rather than losing those tracks:
```json
[
{"type":"removeFxScene3DLayer","compositionId":"main","layerId":1,"itemId":7,"pruneAnimation":true}
]
```
```sh
tsrct project apply --project scene.tsrct --actions remove-copy.json
tsrct preview --project scene.tsrct --time 1 --output removed.png
```
## Document edits and resources
Use `project checkout --project scene.tsrct --output editable.json`, edit the
canonical document without discarding unrelated fields, then
`project commit --project scene.tsrct --file editable.json`. Document-level edits
are useful for canvas/duration and blocks without scoped actions. Start from the
checked-out Scene3D JSON rather than a schema; commit validates it. Reload and inspect after commit;
valid JSON alone does not prove the intended picture. Saved JSON omits fields
that equal their default (for example `bevelSegments` 3), so a value you
authored can be absent from readback without being lost.
Inline mesh geometry and baked animation data are stored as text in the document,
so they dominate the size of a detailed scene; each full copy of one, including a
checkout or inspection dump, can take hundreds of megabytes. Keep checkpoints,
checkouts and inspection dumps as described in [storage](local-operation.md#storage).
Import assets before referencing their IDs:
`tsrct project import-asset --project scene.tsrct --file bottle.blend --asset-id bottle --kind model3d`.
You choose `--asset-id`; items then reference that ID. Images must be packaged
as images (a Radiance `.hdr` is accepted for environments); Model3D uses
`--kind model3d` with a binary glTF `.glb` or a Blender `.blend` (see
[imported models](#imported-models)). IES photometry uses `--kind ies` and a
prepared cloth point cache `--kind point-cache`. Arbitrary paths,
network IDs and wrong-kind assets do not substitute for imports. Fonts use the
ordinary `import-font` workflow. See `project import-asset --help` for the other
asset kinds. Layer-source textures must name
an existing layer in the same composition, not a file path or a 3D item ID.
## Imported models
A `.blend` is converted natively without Blender. The import result lists the
model's nodes, materials, clips, `sourceRevision` and bounds, plus
`conversion.diagnostics` for everything that did not carry over. Read the
diagnostics and tell the user about each loss. Typical losses: unapplied
modifiers (Bevel, Subdivision, Cloth; apply them in Blender to keep the
result), Principled features outside glTF's metallic-roughness model
(transmission, coat), area lights, light and shader node trees, drivers and
constraints. A missing glass or coat can be rebuilt as an authored material on
the affected region (`modelRegionSlots`). An imported model is one Model3D
object: it cannot be turned into an editable mesh or take a cloth simulation.
Importing packages the asset only. Add a Model3D object (and its Data) to show
it. Cameras and lights inside the file are listed as model nodes; they are
not scene cameras or lights. Recreate them as Camera3D and Light3D items from
the reported transforms when the user wants the file's framing or lighting, and
say which values (sensor fit, shift, area lights) you had to assume.
To stand a model (or any item) on a floor item, measure it at one time and
write ordinary values. Write `scale` and `position` as whole vectors, starting
from the stored values that `project inspect` shows:
`{"type":"setFxPropertyValue","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":20,"propertyName":"position"},"value":{"type":"vector3","value":[0,74.05,0]}}`.
1. For a real height, read the item's `worldBounds` in `scene3d.items`
(`tsrct evaluate --project scene.tsrct --time-ms <t>`). Multiply each
`scale` component by `metres × pixelsPerMeter / (max[1] − min[1])` (the
height must be above 0) and apply it before you measure the gap.
2. Check the gap to the floor:
`tsrct evaluate --project scene.tsrct --time-ms <t> --checks scene3d.grounding --check-arg scene3d.grounding:pair=<itemId>=<floorId>`.
Add the `measured` gap of a `floating` or `penetrating` issue to the
item's `position` Y (+Y is down). `notOverSupport` means the item is not
above the floor.
3. Run the check again. The item is grounded only when there is no issue
and the summary's `notEvaluated` is empty; a pair listed there (a wrong
ID, or an item that is not drawn at that time) was not measured.
The gap is in world pixels, so under a parent convert it to the parent's
space. `setFxPropertyValue` writes only the base value under an animator. If
`project inspect` shows an animator on `position` or `scale` (constant,
keyframes, a disabled track with a parked value, or a script), change that
animator instead, then measure the height and the gap again. A Mesh3D floor is measured by its box corners (`approximate`), so use
a flat floor such as a box or plane.
A Model3D's stored controls do nothing until its ordered `poseStack` lists
them. A bound `clip` plays only with `{"type":"clip","source":{"type":"primary"}}`
in `poseStack`; a cross-fade `blend` needs `{"type":"clip","source":{"type":"blend"}}`;
a morph (shape key) control needs `{"type":"morph","control":<id>}` and a node
(bone) override `{"type":"localTransform","control":<id>}`. Without these
entries the model renders its rest pose with no error. Check animation with a
filmstrip whose frames must differ.
## Choose the renderer
`preview`, `filmstrip` and `export` render Scene3D layers with `raster` unless
you pass `--renderer path-tracer`. The path tracer traces light paths, so soft
shadows, bounced light, glass and metal reflections and refraction, and media
come from the physics rather than from raster's helpers. It is much slower, and
far slower again without a hardware GPU. The first denoised render downloads
the denoiser's weights. The renderer is a flag of the command, not part of the
document, and it applies to every Scene3D layer that the command renders.
Browser and iOS playback use raster even when a CLI export used the path tracer.
```sh
tsrct preview --project scene.tsrct --time 2 --output frame.png --renderer path-tracer --samples 16
tsrct export --project scene.tsrct --output scene.mp4 --renderer path-tracer --json
```
Choose the path tracer for explicit path-traced requests. Prefer it for realism,
photorealistic product imagery, bounced lighting, glass refraction and metal
reflections. Use raster for stylized graphics and fast layout or motion checks.
Use `--samples 16` with denoising for previews and drafts.
For final review and delivery, try the default 128 with denoising (omit
`--samples`). Before the full export, check the final-settings previews and
any short export (below) at full size for noise, and consecutive frames for
shimmer (flicker in areas that should be still). If either is visible, raise
`--samples`; halving noise takes four times the samples. Render time grows
about in proportion to the samples, so tell the user the new expected time.
Keep other settings at their defaults unless a specific scene requirement
justifies a change. `tsrct preview --help` lists all path-tracer controls and
their defaults, including bounce limits and `--no-denoise`.
Verify representative times with the intended final renderer and settings;
a raster/path-tracer comparison is not required. Preview uses the document's
authored size and a 30 fps shutter cadence. For another export size or frame
rate, check a short export with the final resolution, fps and renderer flags.
Preview reports `renderSeconds`; filmstrip and `export --json` report
`renderSecondsPerFrame`. These measure the render call, including scene building
and first-use shader compilation, but not readback, encoding or file writing.
Filmstrip timing is per rendered thumbnail, not a full-size export estimate.
Export progress also reports `elapsedSeconds` and `etaSeconds`; the ETA updates
when frames complete, not while a frame is still rendering. Use a short export
to estimate a long export's cost and tell the user the expected time. Cold
compilation and different scene content affect timing; an estimate is not an
upper bound.
A path-traced frame fails rather than silently switching to raster. Report
any failure diagnostic; do not change the requested renderer without the
user's approval. `export --json` puts the failure's `code` on its error event:
- `scene3d_device_limit`: the scene's data exceeds one GPU buffer; the message
names the buffer and both sizes. Reducing copies or subdivision may help.
- `scene3d_path_tracer_frame`: the path tracer could not finish the frame (its
posed geometry could not be read back, a GPU round or the denoiser failed).
A retry may succeed.
The path tracer renders like `raster`, except:
- subsurface scattering is a random walk: light crosses thin objects, and base
color maps stay sharp where raster blurs them;
- glass shadows take the glass's base color and Fresnel;
- there are no caustics: point, spot and sun lights without a size shadow
through rough glass and refracting solids by the light that passes straight
through, and a prism does not throw a rainbow onto a wall; area lights and
lights with a radius or disk reach through glass only along refracted paths;
- a smooth slab shifts what is behind it along the refracted direction and,
when dispersive, shows color fringes; a rough slab does not shift it;
- a blended caster shadows with its alpha, so a zero-opacity caster casts none;
- material occlusion maps, reflection probes, the indirect-light grid and
soft-shadow sample counts are ignored (the tracer computes these effects);
- a light-group output includes the indirect light of its lights.
Rendered alpha is coverage: glass over a transparent background is opaque
where it covers, so footage composited behind a transparent render is not
refracted. Place the footage inside the scene (for example as a layer-source
texture) when refraction of it matters.
In the delivery, say which renders used `--renderer path-tracer` and with
which flags, and name any of these differences that affect the scene.
## Review and current limits
Inspect the actual preview, not merely command success. In the example verify a
centered blue box, visible lit faces and changing Y rotation across the filmstrip.
Check canvas placement, clipping, shadows, texture availability and temporal
continuity. Sparse filmstrips do not prove playback smoothness or platform parity.
`project inspect` prints the stored document JSON. Evaluated positions come from
`tsrct evaluate` (below). Resource preparation (model replacement, rigging, clip
bake, cloth bake) is requested inside `project apply`. Scene3D action and request
fields follow the examples here and `project schema`; report missing operations
rather than guessing command names.
### Run placement checks
`tsrct evaluate` reports, per sample, where every drawn Mesh3D, Text3D and
Model3D is (`scene3d.items`: world matrix and bounds, frame bounds, status) and
checks framing, text overlap, grounding and on-screen motion. Run the checks
that match the scene before delivery, correct actionable findings, then run
them again:
```sh
# Check framing and screen motion over an animated shot.
tsrct evaluate --project scene.tsrct \
--checks scene3d.framing,scene3d.motion \
--start-ms 0 --duration-ms 4000 --fps 30
# Check a still title for frame clipping and overlap with other Text3D items.
tsrct evaluate --project scene.tsrct --time-ms 0 \
--checks scene3d.framing,scene3d.text-overlap
# Check that item 12 rests on support item 4 at a given time.
tsrct evaluate --project scene.tsrct --time-ms 0 \
--checks scene3d.grounding \
--check-arg 'scene3d.grounding:pair=12=4'
```
Replace the example path and IDs with the project's file, item and support
IDs. Grounding needs one `pair=ITEM_ID=SUPPORT_ID` argument for each expected
contact. Checks emit issues and a summary as JSON Lines. When `--checks` is
present, no measurement report is included by default; add
`--report scene3d.items` if you also need per-item positions and bounds. A
`scene3d.items` report alone measures placement but does not run checks.
These checks are approximate. Text overlap compares Text3D bounds in the same
Scene3D occurrence; it does not check ordinary 2D titles or occlusion. Grounding
uses world bounds and the item's footprint, so confirm the result in the
rendered frame. Framing can flag intentional full-bleed backgrounds and objects
passing the camera in a fly-through. Motion can flag intentionally fast
entrances, spins or near-camera objects; inspect the sampled filmstrip and, when
appropriate, check the settled interval separately. Explain any findings left
unfixed. `scene3d.mesh` and `scene3d.topology` measure a Mesh3D's drawn
triangles and editable polygons. `tsrct evaluate --list` describes every report
and check with its parameters.
`project apply --dry-run` reports a batch, including what each mesh edit moves,
without writing anything.
references/scene3d-blender.md
# Scene3D for Blender users
Use this map when the user describes the work in Blender terms. It names the
closest Tesseract control, or says plainly that there is none. Exact fields
come from `tsrct project schema`; [Scene3D basics](scene3d-basics.md) explains
the rules every batch must follow and [recipes](scene3d-recipes.md) has
copy-ready batches. When a Blender feature has no equivalent, tell the user
which approximation you used, or ask, rather than silently substituting.
## Scene, units and axes
| Blender | Tesseract |
| --- | --- |
| Unit scale, 1 unit = 1 m | Positions and sizes are pixels; `pixelsPerMeter` on the Scene3D layer converts. With 100, a 1.8 m figure is 180 px. Imported models, lens apertures, fog and material lengths use metres through it. |
| Z up, −Y forward | X right, Y down, Z into the screen. Imported `.blend`/`.glb` models already stand upright. |
| Scene camera | `setFxScene3DActiveCamera`; with none, the layer frames the canvas and ignores cameras. |
| Objects / Empties / parenting | Flat `scene.items` with `parentId`; an Empty is a `Group3D`. |
| Collections, View Layers | No equivalent. Use `Group3D` for hierarchy and `renderSettings.lightCollections` for light-linking selections. |
| Object data, linked duplicates | Objects reference shared Data by `dataId`; several objects may share one Data. |
| Frame rate, frame range | Document duration in seconds; layer windows and keyframes in milliseconds; export `--fps`. |
| Drivers, Python expressions | No equivalent; keyframe the controls instead. |
## Modeling
| Blender | Tesseract |
| --- | --- |
| Cube, UV sphere, cylinder, cone, torus, plane | Mesh Data `geometry` `box`, `sphere`, `cylinder`, `cone`, `torus`, `plane`. Segment counts and an icosphere are not primitive options; author an `editable` mesh for faceted shapes. |
| Edit-mode mesh | `editable` geometry (vertices, faces with a region per face); `editFxScene3DMesh` edits it; `makeFxScene3DMeshEditable` converts a primitive or curve-built mesh. |
| Sculpt Mode (Grab, Smooth, Inflate, Draw) | An `editFxScene3DMesh` `sculpt` operation on an editable mesh: an ordered list of brush dabs with explicit centers, plus symmetry and a vertex mask. No other brushes and no dynamic topology; subdivide first for detail. Dry-run dabs with `tsrct mesh3d preview`. |
| Shade Smooth / Flat | Mesh `normalMode`. |
| Curve with bevel, Screw, Loft | Mesh geometry `sweep` (along a `Curve3D` Data), `lathe`, `loft`, `extrude`. A `Curve3D` object itself does not render. |
| Text object (extrude, bevel) | `Text3D` with front, back and side regions; per-character animators and selectors. |
| Array modifier | `Group3D.array` `linear`, `grid`, `radial`, `curve`, `scatter`; offsets are pixels (no relative offset); count and seed animate. `realizeFxScene3DArray` makes copies editable. |
| Bevel, Solidify, Mirror, Boolean, Decimate, Remesh, Displace, Simple Deform (Bend/Twist), Lattice, Shrinkwrap, UV Project, Data Transfer, Vertex Weight Edit/Mix/Proximity | Mesh `modifiers` `bevel`, `solidify`, `mirror`, `boolean`, `decimate`, `remesh`, `displace`, `bend`, `twist`, `lattice`, `shrinkwrap`, `uvProject`, `dataTransfer`, `vertexWeightEdit`, `vertexWeightMix`, `vertexWeightProximity`, in stack order. |
| Subdivision Surface | `smoothSubdivide` is Loop subdivision, not Catmull–Clark; `subdivide` splits without smoothing. Quads come out slightly different from Blender. |
| Geometry Nodes scatter (Distribute Points / Instance on Points) | `Group3D.array` `scatter` on a mesh, text or model target with a seed. Other node trees have no equivalent. |
| Shape keys | Editable-mesh shape keys, or a model's morph controls (listed in `poseStack`). |
| Cloth | `cloth` modifier on an editable Mesh3D, baked with the `prepareScene3dCloth` apply request. A `.blend` cloth modifier is not imported. |
## Materials and lighting
| Blender | Tesseract |
| --- | --- |
| Principled BSDF | One material per slot with inputs `baseColor`, `metallic`, `roughness`, `transmission`, `ior`, `subsurface`…, `clearcoat`…, `sheen`…, `emissiveColor`; colors are scene-linear (convert hex values). The material section of [Scene3D properties](scene3d-properties.md) lists them all. |
| Image Texture + Mapping node | `setFxScene3DMaterialTexture` (image, procedural, layer or UDIM source) with a UV transform; `editFxScene3DMaterialGraph` for connections. Arbitrary shader node trees have no equivalent. |
| World color / HDRI | `world.background` and `world.light` (an `.hdr` image with `rotation`, or a color). |
| Film > Transparent | Leave `world.background` unset; preview PNGs keep alpha; export ProRes 4444 with `--fx-solo <composition>:<layer>` for a transparent video. |
| Point / Spot / Area light, Power in W | `Light3D` `emitter` `point`, `spot`, `area` with `power` in watts. IES profiles attach to `point` emitters only. |
| Sun, Strength in W/m² | `parallel` emitter with `irradiance` in W/m². |
| Light linking, shadow linking | `setFxScene3DLightLinks` (illuminated objects, shadow casters and receivers). |
| Exposure (Color Management) | The layer `view.exposure` plus the active camera's `exposure`, in stops. |
| Volume Scatter, Mist | `Volume3D` objects (with world lighting set to `integration: "sampled"`) and world fog distances in metres. |
| Eevee Reflection Cubemap, Irradiance Volume | `ReflectionProbe3D`, `IrradianceVolume3D` (raster only; the path tracer computes these effects). |
| Bloom / Glare | A Glow effect on the Scene3D layer in the composition. |
## Cameras and rendering
| Blender | Tesseract |
| --- | --- |
| Focal length, sensor, f-stop, Depth of Field, focus object | Camera `lens` `physical` (`focalLengthMm`, `sensorMm`, `fNumber`), `focusDistance` or a focus target. `angle` lenses cover perspective, orthographic, fisheye and equirectangular projections. |
| Shift X/Y | `lensShift`. |
| Tilt-shift look | `setFxScene3DTiltShift` (a real tilted focus plane). |
| Track To / Damped Track | `setFxScene3DAim`. |
| Child Of, Copy Location/Rotation | An `attachment` constraint in the object's `relationships` (`setFxScene3DAttachmentBinding` rebinds an existing one), `setFxScene3DFollowTarget`, space switches (`prepareScene3dSpaceSwitch` apply request). |
| Follow Path | `setFxScene3DFollowPath` with a `Curve3D` and animated `progress`. |
| Cycles | `--renderer path-tracer`, `--samples`, bounce flags, `--no-denoise`; a command flag, not saved in the document. |
| Eevee | The default `raster` renderer; browser and iOS playback always use it. |
| Motion blur, shutter | The composition's `motionBlur` (`enabled`, `shutterAngle`, `shutterPhase`, `samplesPerFrame` 2–64) and the Scene3D layer's `motionBlur` switch, edited through checkout and commit. A 180° Blender shutter (0.5 frames) is `shutterAngle` 180. Pass the export frame rate (`filmstrip --frame-rate`) when sampling blurred frames. |
| Render passes, AOVs, Cryptomatte, multilayer EXR | Besides the beauty image, only a light group's direct light (`directLightGroupRgb`) can be selected for preview and export, one product per render, and it needs the ACES scene color domain. Depth, normal and Cryptomatte passes and EXR output are not available. |
## Characters and animation
| Blender | Tesseract |
| --- | --- |
| Armature, skinning, Actions | Import a `.blend` or `.glb` (`import-asset --kind model3d`); bind clips on the Model3D and list them in `poseStack`. Skins keep four influences per vertex. |
| NLA strips, action blending | Model `blend` clip with `blendWeight`, or ordered animation layers and masks (`addFxScene3DAnimationLayer`, `addFxScene3DAnimationMask`). |
| Pose-mode bone keys | Model `nodes` overrides listed as `localTransform` entries in `poseStack`. |
| IK, bone Track To | Model `constraints` `twoBoneIk`, `lookAt`, `transform`. |
| Root motion | Clip `rootMotion` (`source`, `inPlace`, `accumulate`). |
| Bake Action | `prepareScene3dModelBake` apply request. |
| Graph Editor curves | Keyframe `easing` per key; see `project schema` for the easing kinds. |
## What a `.blend` import keeps
Meshes, armatures and skins, shape keys, Actions (sampled), metallic-roughness
materials with their textures, and node transforms are converted. Unapplied
modifiers, transmission and coat, area lights, light and shader node trees,
drivers and constraints are not; `conversion.diagnostics` lists each loss.
Cameras and lights appear only as model nodes. See
[imported models](scene3d-basics.md#imported-models) for the follow-up work.
references/scene3d-properties.md
# Scene3D property reference
Generated by `cargo scene3d-properties-regen` from the canonical typed registry.
Verify with `cargo scene3d-properties-check`. Scene3D is default-off and its wire format is not stable.
These are `propertyName` controls, not all structural JSON fields. A name is accepted only when
its descriptor applies to the concrete target and its current selected mode. Repeated names
represent distinct mode-specific rows; they do not make a control universal. Optional controls
retain their documented unset meaning until a permitted write materializes them.
Geometry, materials, modifiers, camera lens modes, emitters, environment sources, scene constraints,
model joint/clip operators, text records and array kinds use the same admission descriptors in
editing and sampled evaluation. Parent, target, topology and selector changes remain structural.
For a particular item, use the canonical property visitor and typed getter; `NotApplicable`,
`Unset` and an authored value are distinct. Coupled constraints also require final-record validation.
## POSE_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `rotation` | Pose | Quaternion | UnitQuaternion | Finite | ShortestArc | Animatable | Quaternion | Some(Transform) | MaterializeOnWrite | Render |
| `position` | Pose | Vector3 | Pixels | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
| `orientation` | Pose | Vector3 | Degrees | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
| `rotationX` | Pose | Float | Degrees | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
| `rotationY` | Pose | Float | Degrees | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
| `rotationZ` | Pose | Float | Degrees | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
## TRANSFORM_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `anchorPoint` | Transform | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | Some(Transform) | RejectWhileUnset | Render |
| `scale` | Transform | Vector3 | Percent | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | RejectWhileUnset | Render |
## ENVIRONMENT_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `intensity` | EnvironmentSpatial | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `sunThreshold` | EnvironmentSpatial | Float | Ratio | Positive | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `fogColor` | EnvironmentSpatial | Color | ExtendedSrgb | ExtendedColor | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `fogStartDistance` | EnvironmentSpatial | Float | Meters | Nonnegative | Continuous | Animatable | Numeric | Some(FogDistances) | RejectWhileUnset | Render |
| `fogEndDistance` | EnvironmentSpatial | Float | Meters | Positive | Continuous | Animatable | Numeric | Some(FogDistances) | RejectWhileUnset | Render |
| `color` | EnvironmentSpatial | Color | ExtendedSrgb | ExtendedColor | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `rotation` | EnvironmentSpatial | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `rotationX` | EnvironmentSpatial | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `rotationZ` | EnvironmentSpatial | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `sunAzimuth` | EnvironmentSpatial | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `sunElevation` | EnvironmentSpatial | Float | Degrees | SkyElevation | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `turbidity` | EnvironmentSpatial | Float | Ratio | SkyTurbidity | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `backgroundColor` | EnvironmentSpatial | Color | ExtendedSrgb | ExtendedColor | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `backgroundRotation` | EnvironmentSpatial | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `backgroundRotationX` | EnvironmentSpatial | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `backgroundRotationZ` | EnvironmentSpatial | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `backgroundSunAzimuth` | EnvironmentSpatial | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `backgroundSunElevation` | EnvironmentSpatial | Float | Degrees | SkyElevation | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `backgroundTurbidity` | EnvironmentSpatial | Float | Ratio | SkyTurbidity | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## SPATIAL_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `size` | Item | Vector3 | Pixels | Positive | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `blendDistance` | Item | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `density` | Item | Float | InverseMeters | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `scatteringColor` | Item | Color | ExtendedSrgb | UnitColor | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `anisotropy` | Item | Float | Ratio | OpenSignedUnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `spacing` | Item | Float | Pixels | Positive | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `intensity` | Item | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `color` | EnvironmentSpatial | Color | Ratio | ExtendedColor | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `exposure` | EnvironmentSpatial | Float | ExposureStops | ExposureGain | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## ANIMATION_LAYER_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `influence` | AnimationLayer | Float | Ratio | UnitInterval | Continuous | Animatable | BaseOnly | None | RejectWhileUnset | Render |
| `weight` | AnimationLayer | Float | Ratio | UnitInterval | Continuous | Animatable | BaseOnly | None | RejectWhileUnset | Render |
| `timeSeconds` | AnimationLayer | Float | Seconds | Finite | Continuous | Animatable | BaseOnly | None | MaterializeOnWrite | Render |
## CAMERA_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `exposure` | Camera | Float | ExposureStops | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `angleOfView` | Camera | Float | Degrees | PerspectiveAngle | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `focalLengthMm` | Camera | Float | Millimeters | Positive | Continuous | Animatable | Numeric | Some(CameraOptics) | MaterializeOnWrite | Render |
| `fNumber` | Camera | Float | Ratio | Positive | Continuous | Animatable | Numeric | Some(CameraOptics) | MaterializeOnWrite | Render |
| `horizontalAngle` | Camera | Float | Degrees | FullAngle | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `verticalAngle` | Camera | Float | Degrees | HalfAngle | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `fieldOfView` | Camera | Float | Degrees | FullAngle | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `orthographicScale` | Camera | Float | Pixels | Positive | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `perspectiveBlend` | Camera | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | Some(Clipping) | MaterializeOnWrite | Render |
| `near` | Camera | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | Some(Clipping) | MaterializeOnWrite | Render |
| `far` | Camera | Float | Pixels | Positive | Continuous | Animatable | Numeric | Some(Clipping) | MaterializeOnWrite | Render |
| `lensShift` | Camera | Vector2 | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `interocularDistance` | Camera | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `convergenceDistance` | Camera | Float | Pixels | Positive | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `radial` | Camera | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | Some(Distortion) | MaterializeOnWrite | Render |
| `tangential` | Camera | Vector2 | Ratio | Finite | Continuous | Animatable | Numeric | Some(Distortion) | MaterializeOnWrite | Render |
| `calibrationRadius` | Camera | Float | Ratio | Positive | Continuous | Animatable | Numeric | Some(Distortion) | MaterializeOnWrite | Render |
| `focusDistance` | Camera | Float | Pixels | Positive | Continuous | Animatable | Numeric | Some(Focus) | MaterializeOnWrite | Render |
| `aperturePx` | Camera | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | Some(CameraOptics) | MaterializeOnWrite | Render |
| `blurLevel` | Camera | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | Some(CameraOptics) | MaterializeOnWrite | Render |
| `apertureRotation` | Camera | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `anamorphicRatio` | Camera | Float | Ratio | Positive | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `focusPlaneTilt` | Camera | Vector2 | Degrees | FocusTilt | Continuous | Animatable | Numeric | Some(Focus) | MaterializeOnWrite | Render |
| `focusTargetOffset` | Camera | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | Some(Focus) | MaterializeOnWrite | Render |
| `focusDistanceOffset` | Camera | Float | Pixels | Finite | Continuous | Animatable | Numeric | Some(Focus) | MaterializeOnWrite | Render |
## LIGHT_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `color` | Light | Color | ExtendedSrgb | ExtendedColor | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `power` | Light | Float | Watts | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `irradiance` | Light | Float | WattsPerSquareMeter | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `temperature` | Light | Float | Kelvin | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `exposure` | Light | Float | ExposureStops | ExposureGain | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `diffuseFactor` | Light | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `specularFactor` | Light | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `volumeFactor` | Light | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `angularDiameter` | Light | Float | Degrees | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `sourceRadius` | Light | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `coneAngle` | Light | Float | Degrees | PerspectiveAngle | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `coneFeather` | Light | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `goboDensity` | Light | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `goboOffset` | Light | Vector2 | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `goboRotation` | Light | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `goboScale` | Light | Vector2 | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `barnDoorLeft` | Light | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `barnDoorRight` | Light | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `barnDoorTop` | Light | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `barnDoorBottom` | Light | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `barnDoorRoll` | Light | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `areaSize` | Light | Vector2 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `spread` | Light | Float | Degrees | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
## GEOMETRY_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `size` | Geometry | Vector3 | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `size` | Geometry | Vector2 | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `radius` | Geometry | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `height` | Geometry | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `tubeRadius` | Geometry | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `depth` | Geometry | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `sectionScale` | Geometry | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `taper` | Geometry | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `twist` | Geometry | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `revealStart` | Geometry | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `revealEnd` | Geometry | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `angle` | Geometry | Float | Degrees | Revolution | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
## MODIFIER_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `rotation` | Modifier | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `angle` | Modifier | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `angle` | Modifier | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `direction` | Modifier | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `pivot` | Modifier | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `lower` | Modifier | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `upper` | Modifier | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `width` | Modifier | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `profile` | Modifier | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `thickness` | Modifier | Float | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `offset` | Modifier | Float | Ratio | SignedUnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `center` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `mergeDistance` | Modifier | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `position` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `orientation` | Modifier | Vector3 | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `scale` | Modifier | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `mixFactor` | Modifier | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `maxDistance` | Modifier | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `ratio` | Modifier | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `voxelSize` | Modifier | Float | Pixels | Positive | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `offset` | Modifier | Float | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `projectLimit` | Modifier | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `scale` | Modifier | Float | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `midlevel` | Modifier | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `defaultWeight` | Modifier | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `addThreshold` | Modifier | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `removeThreshold` | Modifier | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `maskConstant` | Modifier | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `defaultWeightA` | Modifier | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `defaultWeightB` | Modifier | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `maskConstant` | Modifier | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `minDist` | Modifier | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `maxDist` | Modifier | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `maskConstant` | Modifier | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
## UV_OPTIONAL_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `center` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `size` | Modifier | Float | Pixels | Positive | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
## CLOTH_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `damping` | Modifier | Float | PerSecond | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `gravity` | Modifier | Vector3 | MetersPerSecondSquared | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `windDirection` | Modifier | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `windSpeed` | Modifier | Float | MetersPerSecond | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `windTurbulence` | Modifier | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `pressure` | Modifier | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `shapeStiffness` | Modifier | Float | PerSecondSquared | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `shapeDamping` | Modifier | Float | PerSecond | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
## MATERIAL_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `baseColor` | Material | SceneLinearColor | LinearBt709 | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `roughness` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `metallic` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `emissiveColor` | Material | SceneLinearColor | LinearBt709 | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `coverage` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `clearcoat` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `clearcoatRoughness` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `sheenColor` | Material | SceneLinearColor | LinearBt709 | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `sheenRoughness` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `normal` | Material | SurfaceNormal | NormalDirection | FiniteF32 | Continuous | Animatable | SurfaceNormal | None | RejectWhileUnset | Render |
| `clearcoatNormal` | Material | SurfaceNormal | NormalDirection | FiniteF32 | Continuous | Animatable | SurfaceNormal | None | RejectWhileUnset | Render |
| `clearcoatIor` | Material | Float | Ratio | AtLeastOne | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `clearcoatTint` | Material | SceneLinearColor | LinearBt709 | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `sheen` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `diffuseTransmissionColor` | Material | SceneLinearColor | LinearBt709 | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `transmission` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `thickness` | Material | Float | Meters | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `attenuationDistance` | Material | Float | Meters | Positive | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `dispersion` | Material | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `subsurface` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `diffuseTransmission` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `iridescence` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `iridescenceIor` | Material | Float | Ratio | AtLeastOne | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `iridescenceThicknessMin` | Material | Float | Nanometers | Nonnegative | Continuous | Animatable | Numeric | Some(IridescenceThickness) | RejectWhileUnset | Render |
| `iridescenceThicknessMax` | Material | Float | Nanometers | Nonnegative | Continuous | Animatable | Numeric | Some(IridescenceThickness) | RejectWhileUnset | Render |
| `attenuationColor` | Material | SceneLinearColor | LinearBt709 | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `subsurfaceColor` | Material | SceneLinearColor | LinearBt709 | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `subsurfaceRadius` | Material | Vector3 | Meters | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `ior` | Material | Float | Ratio | AtLeastOne | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `specular` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `specularColor` | Material | SceneLinearColor | LinearBt709 | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `clearcoatNormalScale` | Material | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `anisotropy` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `anisotropyRotation` | Material | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `emissiveStrength` | Material | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `alphaCutoff` | Material | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `normalScale` | Material | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `occlusionStrength` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `occlusion` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `height` | Material | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `transmissionThickness` | SheetMaterial | Float | Meters | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `heightStrength` | Material | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `heightDistance` | Material | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `heightMidlevel` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `iridescenceThicknessMix` | Material | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## TEXTURE_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `Offset` | Material | Vector2 | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `Rotation` | Material | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `Scale` | Material | Vector2 | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `MappingPosition` | Material | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `MappingRotation` | Material | Quaternion | UnitQuaternion | Finite | ShortestArc | Animatable | Quaternion | Some(Transform) | RejectWhileUnset | Render |
| `MappingOrientation` | Material | Vector3 | Degrees | Finite | Continuous | Animatable | Numeric | Some(Transform) | RejectWhileUnset | Render |
| `MappingRotationX` | Material | Float | Degrees | Finite | Continuous | Animatable | Numeric | Some(Transform) | RejectWhileUnset | Render |
| `MappingRotationY` | Material | Float | Degrees | Finite | Continuous | Animatable | Numeric | Some(Transform) | RejectWhileUnset | Render |
| `MappingRotationZ` | Material | Float | Degrees | Finite | Continuous | Animatable | Numeric | Some(Transform) | RejectWhileUnset | Render |
| `MappingAnchorPoint` | Material | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `MappingScale` | Material | Vector3 | Percent | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `MappingSharpness` | Material | Float | Ratio | Positive | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## GRAPH_CONSTANT_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `value` | Material | Float | Ratio | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `value` | Material | Vector2 | Ratio | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `value` | Material | Vector3 | Ratio | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `value` | Material | SceneLinearColor | LinearBt709 | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `value` | Material | SurfaceNormal | NormalDirection | FiniteF32 | Continuous | Animatable | SurfaceNormal | None | RejectWhileUnset | Render |
| `valueX` | Material | Float | Ratio | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `valueY` | Material | Float | Ratio | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `valueZ` | Material | Float | Ratio | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `valueX` | Material | Float | LinearBt709 | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `valueY` | Material | Float | LinearBt709 | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `valueZ` | Material | Float | LinearBt709 | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## MODEL_CLIP_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `clipTime` | ModelClip | Float | Seconds | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `clipAnimation` | ModelClip | ModelSourceKey | Index | Finite | Held | Animatable | Discrete | None | RejectWhileUnset | Render |
| `blendClipTime` | ModelClip | Float | Seconds | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `blendClipAnimation` | ModelClip | ModelSourceKey | Index | Finite | Held | Animatable | Discrete | None | RejectWhileUnset | Render |
| `blendWeight` | ModelClip | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## INSTANCE_MOTION_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `instanceMotionRotation` | ModelInstanceMotion | Quaternion | UnitQuaternion | Finite | ShortestArc | Animatable | Quaternion | Some(Transform) | MaterializeOnWrite | Render |
| `instanceMotionPosition` | ModelInstanceMotion | Vector3 | Pixels | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
| `instanceMotionOrientation` | ModelInstanceMotion | Vector3 | Degrees | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
| `instanceMotionRotationX` | ModelInstanceMotion | Float | Degrees | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
| `instanceMotionRotationY` | ModelInstanceMotion | Float | Degrees | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
| `instanceMotionRotationZ` | ModelInstanceMotion | Float | Degrees | Finite | Continuous | Animatable | LocalTransform | Some(Transform) | MaterializeOnWrite | Render |
## MODEL_CONSTRAINT_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `influence` | ModelConstraint | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `up` | ModelConstraint | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `positionOffset` | ModelConstraint | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `rotationOffset` | ModelConstraint | Vector3 | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `weight` | ModelConstraint | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `roll` | ModelConstraint | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `offset` | TargetBinding | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## SCENE_CONSTRAINT_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `influence` | SceneConstraint | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `progress` | SceneConstraint | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `offsetFactor` | SceneConstraint | Float | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `up` | SceneConstraint | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `roll` | SceneConstraint | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `positionOffset` | SceneConstraint | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `rotationOffset` | SceneConstraint | Vector3 | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `rotationX` | SceneConstraint | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `rotationY` | SceneConstraint | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `rotationZ` | SceneConstraint | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `orientation` | SceneConstraint | Vector3 | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `scale` | SceneConstraint | Vector3 | Percent | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `weight` | SceneConstraint | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `offset` | TargetBinding | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## ARRAY_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `count` | Array | Float | Count | SteppedU32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `counts` | Array | Vector3 | Count | SteppedU32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `seed` | Array | Float | Index | SteppedU32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `copyMaterialsSeed` | Array | Float | Index | SteppedU32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `offset` | Array | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `spacing` | Array | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `radius` | Array | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `startAngle` | Array | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `sweep` | Array | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `step` | Array | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `firstMargin` | Array | Float | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `lastMargin` | Array | Float | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `up` | Array | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `roll` | Array | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `twist` | Array | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `minDistance` | Array | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `randomRotation` | Array | Vector3 | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `scaleMin` | Array | Float | Percent | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `scaleMax` | Array | Float | Percent | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## TEXT_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `depth` | Text | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `fontSize` | Text | Float | Pixels | Positive | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `tracking` | Text | Float | ThousandthEm | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `leading` | Text | Float | Pixels | Positive | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `baselineShift` | Text | Float | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `bevelSize` | Text | Float | Pixels | Nonnegative | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `boxSize` | Text | Vector2 | Pixels | Positive | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `boxPosition` | Text | Vector2 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `boxFirstBaseline` | Text | Float | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## TEXT_ANIMATOR_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `position` | Text | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `anchorPoint` | Text | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `scale` | Text | Vector3 | Percent | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `rotationX` | Text | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `rotationY` | Text | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `rotationZ` | Text | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `opacity` | Text | Float | Percent | Percent | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `baseColor` | Text | Color | ExtendedSrgb | ExtendedColor | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
## TEXT_SELECTOR_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `start` | Text | Float | SelectorUnits | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `end` | Text | Float | SelectorUnits | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `offset` | Text | Float | SelectorUnits | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `amount` | Text | Float | Ratio | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `easeHigh` | Text | Float | Ratio | SignedUnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `easeLow` | Text | Float | Ratio | SignedUnitInterval | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `randomSeed` | Text | Float | Ratio | FiniteF32 | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## TEXT_PATH_CONTROLS3D
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `firstMargin` | Text | Float | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `lastMargin` | Text | Float | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `roll` | Text | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `up` | Text | Vector3 | Ratio | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
## Identity-owned controls
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `exposure` | View | Float | ExposureStops | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `isHidden` | Item | Bool | Ratio | Finite | Held | Animatable | Discrete | None | RejectWhileUnset | Render |
| `offset` | TargetBinding | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `groupingAlignment` | Text | Vector3 | Percent | GroupingAlignment | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `strength` | ClothPin | Float | Ratio | UnitInterval | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `weight` | ShapeKey | Float | Ratio | MorphWeight | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
## Parameterized addresses (representative canonical records)
| Property | Owner family | Value | Units | Admission | Interpolation | Animation | Blend | Coupled group | Unset write | Impact |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `curveVertex_0` | Geometry | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `curveInTangent_0` | Geometry | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `curveOutTangent_0` | Geometry | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `curveTilt_0` | Geometry | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `curveRadius_0` | Geometry | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `curveVertex_1` | Geometry | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `curveInTangent_1` | Geometry | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `curveOutTangent_1` | Geometry | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `curveTilt_1` | Geometry | Float | Degrees | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `curveRadius_1` | Geometry | Float | Ratio | Nonnegative | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `point_0_0_0` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `point_1_0_0` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `point_0_1_0` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `point_1_1_0` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `point_0_0_1` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `point_1_0_1` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `point_0_1_1` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `point_1_1_1` | Modifier | Vector3 | Pixels | Finite | Continuous | Animatable | Numeric | None | MaterializeOnWrite | Render |
| `morphWeight0` | ModelMorph | Float | Ratio | MorphWeight | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
| `wght` | Text | Float | FontAxis | Finite | Continuous | Animatable | Numeric | None | RejectWhileUnset | Render |
Curve controls use the owner's prefix (`path` or `curve`) and canonical decimal vertex index: `<prefix>Vertex_<i>`, `<prefix>InTangent_<i>`, `<prefix>OutTangent_<i>`, `<prefix>Tilt_<i>`, `<prefix>Radius_<i>`. Indices must exist; optional tilt and radius arrays can materialize on write.
Lattice `point_<i>_<j>_<k>` indices are bounded by the authored resolution. Model `morphWeight<i>` indices must exist in the morph record. Font-axis properties are existing four-byte printable OpenType tags, never invented axes. Texture rows apply once per set material texture slot, using the following prefixes:
- `baseColorTexture`
- `roughnessTexture`
- `metallicTexture`
- `normalTexture`
- `occlusionTexture`
- `emissiveTexture`
- `specularTexture`
- `specularColorTexture`
- `clearcoatTexture`
- `clearcoatRoughnessTexture`
- `clearcoatNormalTexture`
- `sheenColorTexture`
- `sheenRoughnessTexture`
- `anisotropyTexture`
- `transmissionTexture`
- `thicknessTexture`
- `diffuseTransmissionTexture`
- `iridescenceTexture`
- `iridescenceThicknessTexture`
- `subsurfaceTexture`
- `subsurfaceColorTexture`
- `clearcoatTintTexture`
- `sheenTexture`
- `subsurfaceRadiusTexture`
- `diffuseTransmissionColorTexture`
- `heightTexture`
- `coverageTexture`
- `clearcoatIorTexture`
- `iridescenceIorTexture`
- `iridescenceThicknessMinTexture`
- `iridescenceThicknessMaxTexture`
- `clearcoatNormalScaleTexture`
- `anisotropyRotationTexture`
- `emissiveStrengthTexture`
- `normalScaleTexture`
- `occlusionStrengthTexture`
- `heightStrengthTexture`
- `heightDistanceTexture`
- `heightMidlevelTexture`
references/scene3d-recipes.md
# Scene3D recipes
Copy-ready starting points for the 3D-enabled CLI; see [basics](scene3d-basics.md)
for clocks, coordinates and renderer tradeoffs, and [properties](scene3d-properties.md)
for valid animation addresses. Use `tsrct project schema` for your installed build.
## Start each recipe in a fresh scene
Save this batch as `base.json`; it creates a camera and above-front sun, but no geometry.
The examples reserve their IDs for an empty `main` composition with a three-second duration.
Do not concatenate every recipe into one document: their IDs intentionally overlap.
```json
[
{"type":"createFxScene3DLayer","compositionId":"main","layerId":1,"name":"Recipe","pixelsPerMeter":100,"view":{"id":9,"exposure":0},"activeRange":{"start":0,"duration":3000},"transform":{"anchorPoint":[0,0],"position":[0,0],"scale":[100,100],"rotation":0,"opacity":100},"world":{"id":2},"renderSettings":{"id":97,"filmGate":null,"overscan":null,"lightGroups":[],"lightCollections":[],"lightLinks":[],"outputs":[],"previewOutput":null}},
{"type":"addFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Camera3D","id":3,"parentId":null,"name":"Camera","transform":{"position":[300,-220,-1000],"relationships":{"constraints":[{"type":"aim","id":80,"target":{"id":81,"target":{"type":"world"},"offset":[0,0,0],"offsetSpace":"local"},"up":[0,1,0],"upSpace":"world","forwardAxis":"positiveZ","influence":1}]}},"lens":{"type":"angle","projection":{"type":"orthographic","scale":640}}}},
{"type":"setFxScene3DActiveCamera","compositionId":"main","layerId":1,"cameraItemId":3},
{"type":"addFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Light3D","id":4,"parentId":null,"name":"Sun","transform":{"rotation":{"type":"euler","order":"XYZ","orientation":[0,0,0],"rotationX":-45,"rotationY":0,"rotationZ":0}},"emitter":{"type":"parallel","irradiance":3}}}
]
```
```sh
tsrct project create --project scene.tsrct
tsrct project apply --project scene.tsrct --actions base.json
```
Unless stated otherwise, save a recipe's JSON as `recipe.json` and apply it after
`base.json`; recipes marked **box first** also need the primitive batch below.
Model follow-ups each start from the model-placement recipe, not from each other.
For `setFxPropertyValue` and `setFxAnimator`, put `layerId` **inside `target`**,
not beside `compositionId`; their style targets also require `itemId` and `propertyName`.
For a landscape canvas, checkout the document, change `dimensions` to
`{"width":640,"height":480}`, then commit it; the examples were checked at that size.
```sh
tsrct project apply --project scene.tsrct --actions recipe.json &&
tsrct filmstrip --project scene.tsrct --timestamps-ms 0,1250,2500 \
--frame-rate 30 --output check.png
```
Check the actual picture, not just successful apply output. `filmstrip --fx-solo main:1`
samples the layer's active window; it cannot be combined with `--timestamps-ms`.
## 1. Mesh rotations, poses and an above-front sun
Place two boxes using equivalent Euler and quaternion rotations, with an explicitly directed sun.
```json
[
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"mesh","id":100,"name":"Geometry","geometry":{"type":"box","size":[140,180,90]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]}],"items":[{"type":"Mesh3D","id":5,"parentId":null,"name":"Box","dataId":100,"packing":"whole","transform":{"position":[-120,0,0],"scale":[100,100,100],"rotation":{"type":"euler","order":"XYZ","orientation":[0,0,0],"rotationX":0,"rotationY":25,"rotationZ":0}},"materialSlots":[{"id":150,"name":"Finish","binding":{"id":151,"inputs":{"baseColor":{"type":"constant","value":[0.2,0.5,0.8]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":150},{"region":"side","slotId":150}]},{"type":"Mesh3D","id":6,"parentId":null,"name":"Box","dataId":100,"packing":"whole","transform":{"position":[120,0,0],"scale":[100,100,100],"rotation":{"type":"quaternion","value":[0,0.21643961393810288,0,0.9762960071199334]}},"materialSlots":[{"id":160,"name":"Finish","binding":{"id":161,"inputs":{"baseColor":{"type":"constant","value":[0.2,0.5,0.8]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":160},{"region":"side","slotId":160}]}]},
{"type":"updateFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Light3D","id":4,"parentId":null,"name":"Above-front sun","transform":{"position":[0,0,0],"rotation":{"type":"euler","order":"XYZ","orientation":[0,0,0],"rotationX":-45,"rotationY":0,"rotationZ":0}},"emitter":{"type":"parallel","irradiance":3}}}
]
```
Check matching box orientations and illuminated top/front faces, not an upside-down camera.
Euler `order` is uppercase (`XYZ`); quaternion values are unit-length `[x,y,z,w]` (within 2e-6; six decimal places suffice).
Mesh transforms accept `position`, `rotation`, `scale` (percent) and `anchorPoint`;
camera/light poses accept position and rotation, **not scale**. The starter camera
shows an aim pose: `up:[0,1,0]` gives an upright view even though Y points down;
`[0,-1,0]` turns it upside down. The hint names the camera's positive local Y basis,
not a literal world-space “sky” direction.
A parallel, spot or area light shines along **local +Z** at zero rotation
(area: its emitting side); the parallel light's position does not affect direction.
With Y down, `rotationX:-45` sends rays down-forward (`+Y,+Z`), so light arrives
from above-front (`-Y,-Z`) and illuminates top faces. `rotationX:+45` sends rays
up-forward and leaves those tops dark. For targeted spots/areas, aim positive Z
at the illuminated point, as in the fog recipe.
## 2. Primitive regions and material slots
Create a box whose two canonical regions share one editable finish.
```json
[
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"mesh","id":100,"name":"Geometry","geometry":{"type":"box","size":[180,180,180]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]}],"items":[{"type":"Mesh3D","id":5,"parentId":null,"name":"Box","dataId":100,"packing":"whole","transform":{"position":[0,0,0]},"materialSlots":[{"id":150,"name":"Finish","binding":{"id":151,"inputs":{"baseColor":{"type":"constant","value":[0.2,0.5,0.8]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":150},{"region":"side","slotId":150}]}]}
]
```
Save this as `box.json` for recipes marked **box first**; check a blue box with lit faces.
Mesh Data regions begin with `primary`, then `side`; assign one `regionSlots` row
per declared region to an existing object-owned material slot. A slot's `binding`
is the material, not a top-level `material` field.
In one table, put referenced objects **before** their users: parents before children,
Curve3D before a follower, and a shrinkwrap target before its decal. Put shared Data
in the same fragment's `data` table; avoid forward references between objects.
## 3. Extruded text with front/back/side finishes
Give an extruded title separate face, back and side materials.
```json
[
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"text","id":200,"name":"Type geometry","text":"UP","fontFamily":"Inter","fontStyle":"Regular","fontSize":120,"depth":30,"regions":[{"key":"front","name":"Front"},{"key":"back","name":"Back"},{"key":"side","name":"Side"}]}],"items":[{"type":"Text3D","id":20,"parentId":null,"name":"Extruded title","dataId":200,"transform":{"position":[0,40,0]},"materialSlots":[{"id":210,"name":"Finish","binding":{"id":211,"inputs":{"baseColor":{"type":"constant","value":[0.9,0.5,0.05]},"roughness":{"type":"constant","value":0.4}}}},{"id":212,"name":"Finish","binding":{"id":213,"inputs":{"baseColor":{"type":"constant","value":[0.1,0.2,0.7]},"roughness":{"type":"constant","value":0.4}}}},{"id":214,"name":"Finish","binding":{"id":215,"inputs":{"baseColor":{"type":"constant","value":[0.05,0.08,0.1]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"front","slotId":210},{"region":"back","slotId":212},{"region":"side","slotId":214}]}]}
]
```
First package a real font: `tsrct project import-font --project scene.tsrct --file /path/to/Inter-Regular.ttf`.
Check upright gold lettering with dark extrusion; turn it around to inspect the blue back.
Text Data regions are exactly `front`, `back`, `side`, each with one region-slot row;
do not copy Mesh Data's `primary` key into Text Data.
## 4. Import a model and play a clip
Import a `.blend` or `.glb` before placing its Model3D object, retaining native materials
except for an optional torso override.
```sh
tsrct project import-asset --project scene.tsrct --file character_rig.blend \
--asset-id character --kind model3d > character-import.json
```
`--asset-id` is required. Use the report's node/material/animation/morph identities,
not guesses based on visible labels. These examples use a rig with `Idle`, `Wave`,
`Face/Smile` and a `RobotRig/.../head` bone; another model needs its own bindings.
The slot source below identifies node 1, primitive 0 of this imported revision.
```json
[
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"model","id":200,"name":"Character","assetId":"character"}],"items":[{"type":"Model3D","id":20,"parentId":null,"name":"Character","dataId":200,"transform":{"position":[0,150,0],"scale":[100,100,100]},"materialSlots":[{"id":310,"name":"Finish","binding":{"id":311,"inputs":{"baseColor":{"type":"constant","value":[0.7,0.15,0.03]},"roughness":{"type":"constant","value":0.4}}}}],"modelRegionSlots":[{"source":{"sourceRevision":"ae98e808db968c076c1a0ed826a804f5f157eb6bd8f9a4f30285e28a023ee846","node":1,"primitive":0},"slotId":310}],"clip":{"animation":"animation:path:Wave","loop":true,"time":0},"poseStack":[{"type":"clip","source":{"type":"primary"}}]}]},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":20,"propertyName":"clipTime"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"clipTime-start","layerTime":0,"value":{"type":"float","value":0},"easing":{"type":"linear"}},{"id":"clipTime-end","layerTime":2500,"value":{"type":"float","value":2.5},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]}
]
```
Save as `model.json`; refresh its revision from the import report before applying:
```sh
python3 - <<'PY'
import json
p = 'model.json'
actions = json.load(open(p))
source = actions[0]['items'][0]['modelRegionSlots'][0]['source']
source['sourceRevision'] = json.load(open('character-import.json'))['model']['sourceRevision']
with open(p, 'w') as f:
json.dump(actions, f)
PY
tsrct project apply --project scene.tsrct --actions model.json &&
tsrct filmstrip --project scene.tsrct --timestamps-ms 0,1250,2500 --output model.png
```
Check a waving arm and orange torso; unmapped primitives keep native materials.
For no overrides, set both `materialSlots` and `modelRegionSlots` to `[]`.
`clipTime` values are seconds, keyframe `layerTime` values are milliseconds.
The clip binding alone is inert: `poseStack` must contain `clip` / `primary`.
A clip named Idle may be static; a successful time write does not prove visible motion.
The following three batches replace the complete model item; retain unrelated controls
when adapting them. Refresh their `sourceRevision` from the same report too, or remove
both material arrays if you do not want the override. Each starts after `model.json`.
## 5. Cross-fade to a second clip
Blend the waving clip toward Idle while advancing both clip clocks.
```json
[
{"type":"updateFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Model3D","id":20,"parentId":null,"name":"Character","dataId":200,"transform":{"position":[0,150,0],"scale":[100,100,100]},"materialSlots":[{"id":310,"name":"Finish","binding":{"id":311,"inputs":{"baseColor":{"type":"constant","value":[0.7,0.15,0.03]},"roughness":{"type":"constant","value":0.4}}}}],"modelRegionSlots":[{"source":{"sourceRevision":"ae98e808db968c076c1a0ed826a804f5f157eb6bd8f9a4f30285e28a023ee846","node":1,"primitive":0},"slotId":310}],"clip":{"animation":"animation:path:Wave","loop":true,"time":0},"poseStack":[{"type":"clip","source":{"type":"primary"}},{"type":"clip","source":{"type":"blend"}}],"blend":{"clip":{"animation":"animation:path:Idle","loop":true,"time":0},"weight":0}}},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":20,"propertyName":"blendClipTime"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"blendClipTime-start","layerTime":0,"value":{"type":"float","value":0},"easing":{"type":"linear"}},{"id":"blendClipTime-end","layerTime":2500,"value":{"type":"float","value":2.5},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":20,"propertyName":"blendWeight"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"blendWeight-start","layerTime":0,"value":{"type":"float","value":0},"easing":{"type":"linear"}},{"id":"blendWeight-end","layerTime":2500,"value":{"type":"float","value":1},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]}
]
```
Check the raised arm settling toward the idle pose as `blendWeight` approaches 1.
Both `primary` and `blend` entries must be in `poseStack`; merely adding `blend`
or animating its weight does not activate it.
## 6. Animate a model morph / shape key
Add a Smile morph after the sampled clip pose.
```json
[
{"type":"updateFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Model3D","id":20,"parentId":null,"name":"Character","dataId":200,"transform":{"position":[0,150,0],"scale":[100,100,100]},"materialSlots":[{"id":310,"name":"Finish","binding":{"id":311,"inputs":{"baseColor":{"type":"constant","value":[0.7,0.15,0.03]},"roughness":{"type":"constant","value":0.4}}}}],"modelRegionSlots":[{"source":{"sourceRevision":"ae98e808db968c076c1a0ed826a804f5f157eb6bd8f9a4f30285e28a023ee846","node":1,"primitive":0},"slotId":310}],"clip":{"animation":"animation:path:Wave","loop":true,"time":0},"poseStack":[{"type":"clip","source":{"type":"primary"}},{"type":"morph","control":30}],"morphs":[{"id":30,"node":"node:path/RobotRig/Face","mode":"replace","weights":[{"target":"morph:mesh:path:Face/Smile","weight":0}]}]}},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":30,"propertyName":"morphWeight0"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"morphWeight0-start","layerTime":0,"value":{"type":"float","value":0},"easing":{"type":"linear"}},{"id":"morphWeight0-end","layerTime":2500,"value":{"type":"float","value":1},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]}
]
```
Check the mouth changing from neutral to smiling across the filmstrip.
Use the imported morph key (`Face/Smile` here, not an older asset's `Cube.002/Smile`);
the `morph` pose-stack entry is required and its control points to the morph record ID.
## 7. Animate a bone-local transform
Apply an additive head turn after the native clip.
```json
[
{"type":"updateFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Model3D","id":20,"parentId":null,"name":"Character","dataId":200,"transform":{"position":[0,150,0],"scale":[100,100,100]},"materialSlots":[{"id":310,"name":"Finish","binding":{"id":311,"inputs":{"baseColor":{"type":"constant","value":[0.7,0.15,0.03]},"roughness":{"type":"constant","value":0.4}}}}],"modelRegionSlots":[{"source":{"sourceRevision":"ae98e808db968c076c1a0ed826a804f5f157eb6bd8f9a4f30285e28a023ee846","node":1,"primitive":0},"slotId":310}],"clip":{"animation":"animation:path:Wave","loop":true,"time":0},"poseStack":[{"type":"clip","source":{"type":"primary"}},{"type":"localTransform","control":30}],"nodes":[{"id":30,"source":"node:path/RobotRig/root/hips/spine/chest/head","mode":"additive","transform":{"position":[0,0,0],"scale":[100,100,100],"rotation":{"type":"euler","order":"XYZ","orientation":[0,0,0],"rotationX":0,"rotationY":0,"rotationZ":0}}}]}},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":30,"propertyName":"rotationY"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"rotationY-start","layerTime":0,"value":{"type":"float","value":0},"easing":{"type":"linear"}},{"id":"rotationY-end","layerTime":2500,"value":{"type":"float","value":45},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]}
]
```
Check the head turning independently of the torso and arm motion.
The node uses its imported source key, and `localTransform` must list its control ID
in `poseStack`; order matters when a later clip would otherwise overwrite the pose.
## 8. HDRI background and illumination
Use one imported HDR environment for visible background and sampled world lighting (**box first**).
```json
[
{"type":"setFxScene3DWorld","compositionId":"main","layerId":1,"world":{"id":2,"background":{"type":"image","assetId":"studio","rotation":90},"light":{"source":{"type":"image","assetId":"studio","rotation":90},"intensity":1,"integration":"sampled"}}}
]
```
First run `tsrct project import-asset --project scene.tsrct --file studio.hdr --asset-id studio --kind image`.
Check both the environment background and its effect on the box; background rotation
and light-source rotation are independent, so set both when rotating the entire world.
## 9. Transparent overlay and ProRes 4444
Keep world illumination but remove the world background for compositing (**box first**).
```json
[
{"type":"setFxScene3DWorld","compositionId":"main","layerId":1,"world":{"id":2,"light":{"source":{"type":"color","color":[1,1,1,1]},"intensity":0.2,"integration":"sampled"}}}
]
```
Check transparency outside the box in the PNG's alpha channel, not just a black viewer backdrop;
remove any separate opaque composition background too. For delivery, use ProRes 4444 MOV:
```sh
tsrct export --project scene.tsrct --output overlay.mov --format prores --fx-solo main:1 --fps 30
```
This is an export command, not an export verification result; check the delivered alpha
in your editor. A whole-project export composites onto the black canvas;
`--fx-solo` keeps the layer's alpha. MP4 is not the transparent-overlay format.
## 10. Fog with a spotlight beam
Scatter a downward spotlight through a bounded volume (**box first**).
```json
[
{"type":"setFxScene3DWorld","compositionId":"main","layerId":1,"world":{"id":2,"background":{"type":"color","color":[0.01,0.01,0.02,1]},"light":{"source":{"type":"color","color":[1,1,1,1]},"intensity":0.02,"integration":"sampled"}}},
{"type":"updateFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Light3D","id":4,"parentId":null,"name":"Beam","transform":{"position":[-180,-350,-100],"relationships":{"constraints":[{"type":"aim","id":82,"target":{"id":83,"target":{"type":"world"},"offset":[0,80,0],"offsetSpace":"local"},"up":[0,1,0],"upSpace":"world","forwardAxis":"positiveZ","influence":1}]}},"emitter":{"type":"spot","power":300,"coneAngle":30,"coneFeather":0.2,"sourceRadius":2}}},
{"type":"addFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Volume3D","id":30,"parentId":null,"name":"Fog","transform":{"position":[0,-100,0]},"size":[900,800,900],"density":0.08,"scatteringColor":[0.8,0.9,1,1],"anisotropy":0.2}}
]
```
Check a visible beam and the box interrupting it, using the path tracer:
```sh
tsrct preview --project scene.tsrct --time 1 --output fog.png --renderer path-tracer --samples 16
```
With world lighting present, Volume3D requires `light.integration:"sampled"`;
a plain ambient environment is not a substitute. Density is per metre at the scene's
`pixelsPerMeter`; avoid hiding a scale mistake with an enormous density.
## 11. Light linking and a light-group preview
Restrict the sun to the box and preview its direct contribution (**box first**).
```json
[
{"type":"setColorManagement","colorManagement":{"workingSpace":"acescct","outputTransform":"aces2.0","compositing":"wholeProgram"}},
{"type":"setFxScene3DRenderSettings","compositionId":"main","layerId":1,"settings":{"id":97,"filmGate":null,"overscan":null,"lightGroups":[{"id":40,"name":"Key","exposure":0}],"lightCollections":[],"lightLinks":[{"lightId":4,"illuminatedObjects":{"type":"objects","items":[5]},"shadowCasters":{"type":"all"},"shadowReceivers":{"type":"all"}}],"outputs":[{"type":"directLightGroupRgb","id":41,"group":40}],"previewOutput":41}},
{"type":"updateFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Light3D","id":4,"parentId":null,"name":"Grouped sun","transform":{"rotation":{"type":"euler","order":"XYZ","orientation":[0,0,0],"rotationX":-45,"rotationY":0,"rotationZ":0}},"emitter":{"type":"parallel","irradiance":3},"lightGroup":40}}
]
```
Check the box in the selected direct-light pass; add another object outside the selection
to check exclusion. All three link roles are explicit; illumination selection does not
implicitly select shadow casters or receivers.
A selected light-group preview requires a scene color domain: this uses ACES with
`wholeProgram` compositing, which changes color management for the whole
document, including 2D layers and grading; tell the user before switching it. Only `directLightGroupRgb` is selectable as a light-group
preview here; declaring `lightGroupRgb` or `indirectLightGroupRgb` does not make them
selectable. Set `previewOutput:null` to restore the ordinary beauty view.
## 12. IES point light
Shape a point light with an imported photometric distribution (**box first**).
```json
[
{"type":"updateFxScene3DLayer","compositionId":"main","layerId":1,"item":{"type":"Light3D","id":4,"parentId":null,"name":"IES point","transform":{"position":[0,-200,-300],"relationships":{"constraints":[{"type":"aim","id":82,"target":{"id":83,"target":{"type":"world"},"offset":[0,0,0],"offsetSpace":"local"},"up":[0,1,0],"upSpace":"world","forwardAxis":"positiveZ","influence":1}]}},"emitter":{"type":"point","power":100,"sourceRadius":0,"iesProfile":"photometry"}}}
]
```
First run `tsrct project import-asset --project scene.tsrct --file lamp.ies --asset-id photometry --kind ies`.
Check the profile's directional falloff on a receiving surface, not just whether the light exists.
IES is supported on **point** emitters only; a spot/area emitter cannot own `iesProfile`.
The accepted import is LM-63-2002 Type C with `TILT=NONE`.
## 13. Scene3D shutter motion blur
Spin a thin blade during an explicit shutter interval.
```json
[
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"mesh","id":100,"name":"Geometry","geometry":{"type":"box","size":[260,20,30]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]}],"items":[{"type":"Mesh3D","id":5,"parentId":null,"name":"Box","dataId":100,"packing":"whole","transform":{"position":[0,0,0]},"materialSlots":[{"id":150,"name":"Finish","binding":{"id":151,"inputs":{"baseColor":{"type":"constant","value":[0.2,0.5,0.8]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":150},{"region":"side","slotId":150}]}]},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":5,"propertyName":"rotationZ"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"rotationZ-start","layerTime":0,"value":{"type":"float","value":0},"easing":{"type":"linear"}},{"id":"rotationZ-end","layerTime":2500,"value":{"type":"float","value":7200},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]},
{"type":"setFxCompositionMotionBlur","compositionId":"main","settings":{"enabled":true,"shutterAngle":180,"shutterPhase":-90,"samplesPerFrame":8,"adaptiveSampleLimit":16}},
{"type":"setFxLayerMotionBlur","compositionId":"main","layerId":1,"enabled":true}
]
```
The shutter lives at `composition.motionBlur`, not in the Scene3D render settings;
the ordinary Scene3D layer also needs `motionBlur:true`. `samplesPerFrame` controls
temporal integration, whereas CLI `--samples` controls path-tracer camera samples.
```sh
tsrct filmstrip --project scene.tsrct --timestamps-ms 500,1250,2000 --frame-rate 30 \
--renderer path-tracer --samples 16 --output shutter.png
tsrct preview --project scene.tsrct --time 1.25 --renderer path-tracer --samples 16 --output shutter-frame.png
```
Check swept blade edges rather than a single sharp pose. Preview uses 30fps shutter
cadence. Filmstrip sample spacing is **not** a frame rate: without `--frame-rate 30`,
this path-traced filmstrip fails with “Scene3D shutter exposure requires valid explicit
output frame timing.” Use the intended delivery cadence, matching export `--fps`.
## 14. Animate an array count
Grow a row from one to five copies using a Group3D array.
```json
[
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"mesh","id":100,"name":"Geometry","geometry":{"type":"box","size":[60,60,60]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]}],"items":[{"type":"Group3D","id":10,"parentId":null,"name":"Copies","transform":{},"array":{"type":"linear","id":11,"count":1,"offset":[90,0,0]}},{"type":"Mesh3D","id":5,"parentId":10,"name":"Box","dataId":100,"packing":"whole","transform":{"position":[-200,0,0]},"materialSlots":[{"id":150,"name":"Finish","binding":{"id":151,"inputs":{"baseColor":{"type":"constant","value":[0.2,0.5,0.8]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":150},{"region":"side","slotId":150}]}]},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":11,"propertyName":"count"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"count-start","layerTime":0,"value":{"type":"float","value":1},"easing":{"type":"linear"}},{"id":"count-end","layerTime":2500,"value":{"type":"float","value":5},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]}
]
```
Check one, three and five boxes across the filmstrip; count is stepped to an integer.
The array belongs to the group and the source mesh is its child, with the parent
listed first; animate the array's ID 11, not the group's ID 10.
## 15. Bevel then subdivide
Round a box by evaluating an ordered modifier stack.
```json
[
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"mesh","id":100,"name":"Geometry","geometry":{"type":"box","size":[180,180,180]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]}],"items":[{"type":"Mesh3D","id":5,"parentId":null,"name":"Box","dataId":100,"packing":"whole","transform":{"position":[0,0,0]},"materialSlots":[{"id":150,"name":"Finish","binding":{"id":151,"inputs":{"baseColor":{"type":"constant","value":[0.2,0.5,0.8]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":150},{"region":"side","slotId":150}],"modifiers":[{"type":"bevel","id":30,"width":15,"segments":3,"profile":0.5,"limitAngle":30},{"type":"smoothSubdivide","id":31,"levels":2,"creaseAngle":180}]}]}
]
```
Check rounded corners and narrow edge highlights; reversing modifier order is not equivalent.
The modifier records live on Mesh3D, not on its shared geometry Data.
## 16. Shrinkwrap a decal
Subdivide an orange patch and project it onto a sphere.
```json
[
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"mesh","id":100,"name":"Geometry","geometry":{"type":"sphere","radius":120},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]},{"type":"mesh","id":101,"name":"Geometry","geometry":{"type":"plane","size":[140,140]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]}],"items":[{"type":"Mesh3D","id":5,"parentId":null,"name":"Target sphere","dataId":100,"packing":"whole","transform":{"position":[0,0,0]},"materialSlots":[{"id":150,"name":"Finish","binding":{"id":151,"inputs":{"baseColor":{"type":"constant","value":[0.2,0.5,0.8]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":150},{"region":"side","slotId":150}]},{"type":"Mesh3D","id":6,"parentId":null,"name":"Decal","dataId":101,"packing":"whole","transform":{"position":[0,0,-130]},"materialSlots":[{"id":160,"name":"Finish","binding":{"id":161,"inputs":{"baseColor":{"type":"constant","value":[1,0.25,0.04]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":160},{"region":"side","slotId":160}],"modifiers":[{"type":"subdivide","id":30,"levels":4},{"type":"shrinkwrap","id":31,"target":5,"wrapMethod":"nearestSurfacePoint","wrapMode":"aboveSurface","offset":1.3,"positiveDirection":true,"cullFace":"off"}]}]}
]
```
Check the orange patch curving around the blue sphere without z-fighting.
List target 5 before decal 6, subdivide before shrinkwrap, and use a small positive
surface offset; a coarse un-subdivided patch cannot follow the curvature.
## 17. Curve3D, FollowPath and aim
Move a box along a shared curve while aiming it at a fixed point.
```json
[
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"curve","id":200,"name":"Route","curve":{"vertices":[[-180,-80,-100],[180,80,-100]],"inTangents":[[0,0,0],[0,0,0]],"outTangents":[[0,0,0],[0,0,0]],"closed":false}},{"type":"mesh","id":100,"name":"Geometry","geometry":{"type":"box","size":[70,50,110]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]}],"items":[{"type":"Curve3D","id":20,"parentId":null,"name":"Route","dataId":200,"transform":{}},{"type":"Mesh3D","id":5,"parentId":null,"name":"Box","dataId":100,"packing":"whole","transform":{"position":[0,0,0],"relationships":{"constraints":[{"type":"followPath","id":30,"curve":{"id":31,"target":{"type":"item","item":20},"offset":[0,0,0],"offsetSpace":"local"},"metric":"localArcLength","positionMode":"replace","boundary":"clamp","forwardAxis":"positiveZ","frame":{"type":"worldUp"},"up":[0,1,0],"influence":1,"orient":false,"progress":0},{"type":"aim","id":32,"target":{"id":33,"target":{"type":"world"},"offset":[0,0,200],"offsetSpace":"local"},"up":[0,1,0],"upSpace":"world","forwardAxis":"positiveZ","influence":1}]}},"materialSlots":[{"id":150,"name":"Finish","binding":{"id":151,"inputs":{"baseColor":{"type":"constant","value":[0.2,0.5,0.8]},"roughness":{"type":"constant","value":0.4}}}}],"regionSlots":[{"region":"primary","slotId":150},{"region":"side","slotId":150}]}]},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":30,"propertyName":"progress"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"progress-start","layerTime":0,"value":{"type":"float","value":0},"easing":{"type":"linear"}},{"id":"progress-end","layerTime":2500,"value":{"type":"float","value":1},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]}
]
```
Check travel from upper-left to lower-right with a changing aim toward `[0,0,200]`.
Curve3D itself is nonrendering; its `dataId` owns the spline. FollowPath runs before
aim in this stack, and progress animation addresses constraint ID 30.
## 18. Glow on the rendered Scene3D layer
Add a screen-space glow to the Scene3D result (**box first**).
```json
[
{"type":"addFxLayerEffect","compositionId":"main","layerId":1,"effectId":50,"effect":{"type":"glow","glowThreshold":35,"glowRadius":30,"glowIntensity":1.5}}
]
```
Check a halo around bright box edges; lower the threshold if the source is too dim.
The effect belongs to ordinary composition layer 1, not to a Mesh3D child;
this is post-processing, not a light that illuminates other 3D objects.
## 19. Exposure
Set a static view exposure and animate the camera exposure (**box first**).
```json
[
{"type":"setFxPropertyValue","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":9,"propertyName":"exposure"},"value":{"type":"float","value":0.5}},
{"type":"setFxAnimator","compositionId":"main","target":{"type":"styleProperty","layerId":1,"itemId":3,"propertyName":"exposure"},"animator":{"type":"keyframes","enabled":true,"keyframes":[{"id":"exposure-start","layerTime":0,"value":{"type":"float","value":0},"easing":{"type":"linear"}},{"id":"exposure-end","layerTime":2500,"value":{"type":"float","value":1},"easing":{"type":"linear"}}]},"dependencies":[],"layerRefBindings":[]}
]
```
Check increasing brightness while the box stays in place; camera and view exposure add in stops.
The view (ID 9 here) is the scene's own exposure; the camera's travels with that camera.
## 20. Clear product glass with a refraction target
Make a clear solid glass sphere against a checker backdrop, with a ground to anchor it.
Start after `base.json` on the 640×480 canvas described above; save as `glass.json`.
```json
[
{"type":"setFxScene3DWorld","compositionId":"main","layerId":1,"world":{"id":2,"background":{"type":"color","color":[0.08,0.08,0.08,1]},"light":{"source":{"type":"color","color":[1,1,1,1]},"intensity":0.35,"integration":"sampled"}}},
{"type":"addFxScene3DFragment","compositionId":"main","layerId":1,"data":[{"type":"mesh","id":105,"name":"Glass geometry","geometry":{"type":"sphere","radius":120},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]},{"type":"mesh","id":106,"name":"Backdrop geometry","geometry":{"type":"plane","size":[1000,700]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]},{"type":"mesh","id":107,"name":"Ground geometry","geometry":{"type":"box","size":[1000,20,900]},"regions":[{"key":"primary","name":"Primary"},{"key":"side","name":"Side"}]}],"items":[{"type":"Mesh3D","id":5,"parentId":null,"name":"Glass","dataId":105,"packing":"whole","transform":{"position":[0,0,0]},"materialSlots":[{"id":160,"name":"Clear glass","binding":{"id":161,"transmissionMedium":{"type":"volume"},"volumeMode":"solid","inputs":{"baseColor":{"type":"constant","value":[1,1,1]},"metallic":{"type":"constant","value":0},"roughness":{"type":"constant","value":0.02},"transmission":{"type":"constant","value":1},"ior":{"type":"constant","value":1.5},"thickness":{"type":"constant","value":0},"attenuationColor":{"type":"constant","value":[1,1,1]},"attenuationDistance":{"type":"constant","value":1}}}}],"regionSlots":[{"region":"primary","slotId":160},{"region":"side","slotId":160}]},{"type":"Mesh3D","id":6,"parentId":null,"name":"Checker backdrop","dataId":106,"packing":"whole","transform":{"position":[0,0,240]},"materialSlots":[{"id":162,"name":"Checker","binding":{"id":163,"unlit":true,"doubleSided":true,"inputs":{"baseColor":{"type":"texture","factor":[1,1,1],"sample":{"texture":{"source":{"type":"procedural","texture":{"size":[512,512],"generator":{"type":"checker","cellSize":[0.125,0.125],"offset":[0,0]}}}},"interpretation":{"type":"color"},"selection":{"type":"rgb"}}},"roughness":{"type":"constant","value":1}}}}],"regionSlots":[{"region":"primary","slotId":162},{"region":"side","slotId":162}]},{"type":"Mesh3D","id":7,"parentId":null,"name":"Ground","dataId":107,"packing":"whole","transform":{"position":[0,130,0]},"materialSlots":[{"id":164,"name":"Ground","binding":{"id":165,"inputs":{"baseColor":{"type":"constant","value":[0.3,0.3,0.3]},"roughness":{"type":"constant","value":0.3}}}}],"regionSlots":[{"region":"primary","slotId":164},{"region":"side","slotId":164}]}]},
{"type":"makeFxScene3DMeshEditable","compositionId":"main","layerId":1,"itemId":5,"newDataId":200}
]
```
```sh
tsrct project apply --project scene.tsrct --actions glass.json &&
tsrct preview --project scene.tsrct --time 1 --output glass.png --renderer path-tracer --samples 16
```
Check a crisp, strongly warped checker inside the sphere, reflective rim and ground
contact. This is a solid glass test object, not a hollow bottle or a glass of water.
The checker is inside Scene3D so rays can refract it; its unlit, double-sided finish
keeps the contrast readable. Keep the base sun: it supplies sharp highlights, while
the sampled white world fills reflections and the ground.
- `transmission:1` removes the diffuse share; `metallic:0` keeps dielectric transmission.
White `baseColor` and `attenuationColor` keep the glass neutral rather than gray or tinted.
- `roughness:0.02` keeps transmitted detail sharp; `ior:1.5` gives visible bending
and reflective edges. Leave alpha coverage opaque: transmission is not opacity.
- Make Editable supplies a closed polygon boundary for `volumeMode:"solid"`;
its actual entry/exit geometry determines the optical path. `thickness:0` here
is not a zero-thickness wall. At `pixelsPerMeter:100`, the radius is 1.2 metres.
`attenuationDistance:1` is metres; white attenuation adds no absorption.
- Sheet/slab transport is a different model: `thickness` is camera-path thickness,
while `transmissionThickness` is Sheet-only light-path thickness, both in metres.
Omit `transmissionThickness` on this Volume material; a nonzero value rejects.
The tested primitive/slab alternatives did not produce the solid's strong checker bending.
Avoid these observed failure modes:
- `roughness:0.4` blurred away the checker and looked frosted gray; `transmission:0.5`
left a bright milky diffuse body even with low roughness.
- Gray `baseColor:[0.2,0.2,0.2]` darkened the transmitted pattern; neutral glass did
not need a gray tint. Colored attenuation also visibly tinted the solid.
- `ior:1` nearly erased the rim and transmitted the pattern almost unchanged;
a plain dark backdrop likewise gave much weaker evidence of refraction than the checker.
## 21. Stop on rejected edits
Do not render an old document after a rejected batch: `project apply` exits non-zero.
```sh
#!/bin/sh
set -eu
tsrct project apply --project scene.tsrct --actions recipe.json
tsrct preview --project scene.tsrct --time 1 --output check.png
```
Check the apply report before trusting the image; with `set -e` (or `&&`), a failed
apply prevents the preview from running. Inspect stored values with `project inspect`;
it shows the saved document, not evaluated poses, so judge results from renders.
references/sound-design.md
# Sound design for social video and ads
Sound helps establish rhythm, direct attention, and make the edit feel intentional. Design it with the pictures, not as decoration added after every animation. Keep speech understandable on small speakers at low volume.
## Permission and source selection
When making a new edit or when asked to improve its sound, add appropriate local sounds without asking for each cue. Use the user's supplied audio, meaningful production sound from the footage, an available local sound library cleared for the project, or this plugin's procedural accents. Preserve an explicitly chosen track and obey requests for silence or a clean dialogue-only edit. A narrow visual revision does not authorize restyling the whole mix.
The sound helper offers eight procedural accents. List them with `python3 <skill-root>/scripts/tesseract_sound.py list` and use `make` to generate a WAV with the desired duration, seed, or level locally. These are synthesized design accents, not recordings of the actual product. Do not imply that a synthetic click is factual evidence of how something sounds. For documentary/product claims, prefer authentic source sound.
Import selected supplied or generated files as audio, then use separate native Audio layers and keep source/gain/timing traceable. Follow [audio and timing](audio-and-timing.md) for placement and mixing. Do not bake all sound into a replacement video or change the source footage. The local-only plugin does not contact a music, voice, or sound-generation service. If a bespoke soundtrack is essential and no suitable local track exists, identify that missing input; do not invent narration or silently replace the user's music.
For an ad opening, follow [ad hooks](ad-hooks.md) to map the chosen visual event to the music and a motivated sound accent. Align perceptual arrivals/transients, preserve speech, and use the existing music hit alone when it already provides the impact.
## Five roles to consider
These are five possible roles, not five compulsory tracks. Use only the layers that improve the piece; absence and silence are useful decisions.
| Role | What it contributes | How to mix and place it |
|---|---|---|
| Primary dialogue / voiceover | Meaning, character, narrative clarity | First priority. Keep the main voice centered; use clip gain and, when needed, compression for consistent intelligibility. Preserve breaths and natural emphasis. Heavy compression is a choice for uneven material, not an automatic preset. |
| Background music | Emotional direction, pacing, continuity | Establish a useful bed level, then duck it around speech. A gentle presence-region dip around 1–3 kHz can help if the music masks consonants; listen rather than hollowing out the track. Use lifts or dropouts to support a reveal or punchline. |
| Micro-SFX / UI accents | Points attention at a particular event | Pops, taps, clicks, paper sounds, and pings should match meaningful motion or interaction. Short upper-mid/high-frequency detail often reads on a phone; avoid piercing transients or repeated notification-like pings. |
| Structural transitions | Marks a change in scene, energy, or argument | Choose a whoosh, riser, downlifter, impact, or deliberate pause when the transition earns emphasis. Align the arrival/peak with the reveal, not automatically the sound file's beginning. Low impacts need audible harmonics so their meaning survives small speakers. |
| Diegetic foley / ambience | Gives footage physical context and continuity | Prefer source product clicks, handling, room tone, and environmental texture. For background texture, roughly 18–24 dB below dialogue is a starting relationship, not an absolute fader setting. Featured product sounds can come forward in speech gaps. |
For a fast ad, reassess attention every 2–4 seconds; that does not mean inserting a whoosh every 2–4 seconds. Avoid stacking a pop, hit, whoosh, and bass drop on the same trivial text entry. One deliberate cue often reads more clearly.
For speech that needs repair or tonal/level improvement, use [speech cleanup](speech-cleanup.md). Establish a natural, intelligible voice before building the mix around it; avoid amplifying room sound with excessive compression.
## Stack the mix around the voice
1. Establish the timing anchor. For talking-led work, tighten and verify the dialogue/source edit before mixing around it. When music was supplied or generated first, map its beats, phrases, and major changes before planning picture timing. Follow [waveform editing](waveform-editing.md) in either case; listen to confirm the waveform candidates. Speech remains intelligible and complete even in a music-led edit.
2. Place the music to establish the arc. Start a music duck around 3–6 dB below its unducked level, adjusting by ear to the actual tracks. Begin the reduction just before speech; use a short smooth attack and a slower release so the bed does not pump between syllables. As a starting point, try about 30–80 ms down and 150–350 ms back up.
3. Add micro-SFX only to actions the viewer should notice. Shape their tails and reduce them during important words.
4. Add structural cues to the few major changes. Check the sequence with those cues muted; they should improve the cut rather than conceal a weak one.
5. Restore subtle ambience/foley where it helps the footage feel connected. Retain useful production sound rather than burying it under added effects.
The relative duck is a multiplier: −3 dB ≈ 0.708 and −6 dB ≈ 0.501. Apply it to the selected bed gain. A fader value of 0.2 does not mean the bed is a fixed number of dB below the voice; the input waveforms determine that relationship.
## A brief drop in density
An 80–200 ms pause in music and background texture before a reveal can create contrast. Treat it as an option to audition, not a universal retention technique. Preserve any word or authentic sound carrying the meaning. If actual silence is intended, account for all audible layers and lingering tails; muting music alone may not create silence. Short 5–20 ms ramps can avoid clicks while still feeling abrupt. Let the reveal's cue/music return make the payoff legible.
## Frequency and mobile translation
- Protect the speech's presence, broadly in the mids. Start with level and arrangement before EQ; do not remove the entire 500 Hz–4 kHz region from every background track.
- Reserve deep low end for the music's bass/kick or a selected impact, and avoid overlapping several bass-heavy effects. Do not depend on sub-bass for information a phone listener must hear.
- Micro sounds can use upper-mid/high detail, often around 2–8 kHz, but brightness is not permission to boost everything above 5 kHz. Check sibilance and fatigue.
- Center the narrative voice and keep essential cues audible in mono. Check headphones, low playback volume, and a phone when available. The helper's mono/band-limited preview is a diagnostic aid, not an exact device simulation.
## Loudness and delivery
Use approximately −15 LUFS integrated, with an acceptable starting range of −16 to −14 LUFS, when there is no explicit delivery spec. This is a working social-video target, not a claim that every feed has the same normalization policy. Preserve intended contrast rather than forcing every short cue or quiet passage to that loudness.
Keep the **final encoded** true peak at or below **−1 dBTP**. True peak is measured in dBTP, not the sample-peak dBFS reading. Leaving a little extra margin before AAC encoding, such as −1.5 dBTP, can help, but remeasure the actual deliverable. Follow a supplied platform/client spec over this default.
Listen to the completed muxed video. Check speech clarity, cue density, abrupt edges, unintended silence, bass clutter, and the head/tail. Measure the final audio and keep the report with the project. A good meter reading is not proof of a good mix.
For exact local operations, native envelopes, and the distinction between timed ducking and signal-driven compression, read [audio and timing](audio-and-timing.md).
references/speech-cleanup.md
# Clean up talking audio
Read this when dialogue needs clearer tone, steadier levels, or less room noise/reverb. Apply only the processing the recording needs. Keep the speaker's identity, natural breathing, and complete words. A clean source may need only gain adjustment.
## Diagnose before choosing a treatment
Audition representative loud and quiet phrases, sibilant words, pauses, and phrase endings where reflections are audible. Check the source waveform and, when useful, a spectrum/spectrogram. A waveform helps with timing and levels; it cannot distinguish all noise, reflections, or intelligibility problems.
Separate **room tone** (background ambience/noise), **reverb** (the voice's decaying reflections), and **echo** (audible repeats). Also check for clipping, mic handling, plosives, hum, and changing mic distance. Lowering gain does not repair clipping already in the recording.
Probe and listen to available audio streams/channels first. A clean lavalier or boom track is preferable to repairing a distant camera mic. Do not blindly sum different mics: delayed copies can create comb filtering and more apparent room sound. Center a selected mono voice deliberately while preserving meaningful stereo production sound elsewhere.
## Choose a light treatment
These are starting points to audition, not a fixed chain or a requirement to use every processor.
| Problem | Useful first treatment | What to listen for |
|---|---|---|
| Uneven speaking levels | Ride phrase/clip gain before relying on heavy compression; preserve intentional emphasis. | The voice should stay present without making breaths and room tails unnaturally loud. Peak normalization alone does not level speech. |
| Rumble or handling noise | Try a gentle high-pass around 60–90 Hz, lowering the cutoff for a voice with useful bass. Address isolated plosives locally when possible. | Stop before the voice loses body. A high-pass is not a reverb remover. |
| Mud or boxiness | Find the actual resonance; audition a broad 1–3 dB reduction somewhere around 200–500 Hz. | Avoid automatically scooping all low mids or making the voice thin. Room coloration can occur outside this range. |
| Harshness or poor intelligibility | Reduce the offending band modestly before adding brightness. Keep room for speech by reducing competing music. | A presence boost around 2–5 kHz can also amplify harshness, hiss, and reflections; clearer is not always brighter. |
| Steady hiss, fan noise, or hum | Use restrained noise reduction; try roughly 3–6 dB first. A narrow hum notch is appropriate only at a confirmed hum frequency/harmonic. | Listen for watery/bubbly artifacts and lost consonants. Learn a noise profile only from an actual speech-free segment; a changing background is not stationary noise. |
| Excess dynamics after gain rides | Try a soft-knee compressor around 2:1–3:1, attack 10–30 ms, release 100–250 ms. Set threshold from this recording; a few dB of reduction on louder phrases is a useful first audition. | Too-fast attack dulls consonants; a fast release or excessive makeup gain can bring up room sound and cause pumping. Adjust by phrase behavior, not a preset label. |
| Sharp “s” and “sh” sounds | Apply selective de-essing in the actual sibilant region, often roughly 4–10 kHz. Recheck after EQ/compression. | Preserve articulation. Excess de-essing creates a lisp; a permanent large treble cut dulls the whole voice. |
| Audible noise between phrases | Prefer natural room tone, gentle gain rides, or mild expansion when needed. | Hard gates can chop breaths and word tails and make the room switch on/off. Silence detection is not word-boundary detection. |
A useful order to audition is source/channel selection → repair or light noise reduction → corrective EQ → leveling/compression → selective de-essing → mix gain. Change the order or omit stages according to the recording. Avoid stacking several enhancement passes that each remove speech detail. Use [waveform editing](waveform-editing.md) when shortening pauses; processing does not make an unsafe trim safe.
## Room reverb and echo
EQ may reduce room boxiness, but cannot separate all reflections from the direct voice. Gating reduces tails in gaps and does not remove reverb underneath words. Heavy compression often makes the problem more noticeable. Retain some room character if removing it damages the voice.
This plugin has **no bundled dedicated dereverberation or speech-isolation model**. FFmpeg's `afftdn` is a noise reducer; `aecho` adds echo. Neither is a general speech dereverberator. If a suitable local enhancement tool/model is already available and permitted, audition it conservatively on a copy. Do not silently download a model or upload dialogue to an external service. Echo cancellation that requires a clean reference signal is not a general solution when that reference is absent.
Compare a moderate treatment before increasing strength. Check phrase endings, quiet consonants, natural breaths, and voice texture. Back off if it sounds metallic, watery, phasey, or syllables disappear. For a badly reverberant recording, report the remaining limitation honestly; improved intelligibility is a useful result even when the room cannot be fully removed. Missing words and a clean studio recording cannot be guaranteed from damaged source audio.
## Apply locally and preserve timing
Tesseract's native controls are source/active ranges, gain, and gain envelopes. EQ, compression, denoising, and de-essing use a **new local audio derivative**, not invented Tesseract DSP fields. Inspect installed filter options with `ffmpeg -h filter=<name>` before using them. In particular, `acompressor` threshold/makeup use linear amplitude and attack/release use milliseconds; FFmpeg `deesser`'s `f` parameter is normalized, not a frequency in Hz.
This optional example auditions gentle rumble/boxiness reduction and compression on an **already aligned dialogue-only WAV**. Choose its settings from the actual recording, or omit unnecessary filters:
```bash
ffmpeg -nostdin -n -i /absolute/path/edit/dialogue-aligned.wav -map 0:a:0 -vn -af "highpass=f=70:p=2,equalizer=f=300:t=q:w=1:g=-2,acompressor=threshold=0.125:ratio=2:attack=15:release=150:makeup=1" -ar 48000 -c:a pcm_s24le /absolute/path/edit/dialogue-clean-v1.wav
```
`threshold=0.125` is approximately −18 dBFS and may be inappropriate for another input level. This example performs no de-reverb, loudness normalization, or true-peak limiting. For steady noise, the locally available `afftdn` can be auditioned separately with modest `nr` and a noise floor derived from the recording; do not call its defaults a universal cleanup preset. Its noise-only output mode can help reveal whether removed material includes words.
Keep the original file and record the selected stream/channel, processing recipe, source range, and mapping to project time. Favor processing continuous dialogue takes before trimming into many short clips; independent per-clip processing can create changing noise floors and processor transients. If processing an excerpt, include handles and record its offset.
Verify duration, head/tail, channel layout, processing latency, and lip sync before replacing audio in the project. A decoded WAV may start at zero even when its source video's audio starts later. Preserve that offset explicitly. Never time-stretch or trim the source to make a derivative appear aligned. Import the processed derivative as audio using a new asset ID. Then replace only the intended dialogue or mute its original playback when using an independent Audio layer, avoiding doubled speech. Keep music and SFX separate.
## Decide whether it is actually better
Compare original and processed versions over the **same phrases at matched perceived loudness**. Use integrated/short-term measurements to establish a comparison level, then listen; peak matching alone can bias the comparison. Audition quiet phrases, loud words, “s” sounds, and room tails with music muted, then in the full mix. Change one major treatment at a time when diagnosing artifacts.
Keep the milder version if it preserves more speech detail. Check headphones, mono, and low-volume playback; verify waveform joins and lip sync after processing. Follow [audio and timing](audio-and-timing.md) for final encoded loudness/true-peak checks. Do not normalize each voice clip independently to the complete-program target.
Record the actual benefit and any remaining room sound or artifacts. If audio cannot be auditioned in the host, processing and meter results remain unverified for listening quality; do not describe them as successful cleanup.
references/telemetry.md
# Usage telemetry
The CLI may send basic usage telemetry for some commands. Telemetry is enabled
by default in releases.
- `tsrct telemetry disable`: turn telemetry off.
- `tsrct telemetry enable`: turn telemetry on.
- `tsrct telemetry status`: check the current setting.
The setting persists across runs. Respect the CLI's opt-out setting; never enable
telemetry on the user's behalf. Do not reset settings or request additional
network permissions just to send telemetry.
See the [privacy policy](https://mirage.app/legal/privacy-policy).
## Agent attribution
The CLI emits telemetry itself. You can optionally pass these attribution variables
with CLI calls made through the skill:
| Variable | Value |
| --- | --- |
| `TESSERACT_CALLER` | `agent-skill` |
| `TESSERACT_SKILL` | `tesseract-motion` or `tesseract-video` |
| `TESSERACT_HOST` | `chatgpt`, `codex`, `claude`, or `cursor`, when known |
| `TESSERACT_DISTRIBUTION` | `openai-plugin` or `standalone-skill`, when known |
Missing/unrecognized values become `unknown`. Do not infer the host from
`.codex-plugin`; several agents support it. Set attribution in each shell call,
not in persistent user/system variables or profiles.
Shell example (use the actual host and skill):
```sh
TESSERACT_CALLER=agent-skill TESSERACT_HOST=codex \
TESSERACT_SKILL=tesseract-motion TESSERACT_DISTRIBUTION=unknown \
"/absolute/path/to/tsrct" project create --project Project.tsrct
```
PowerShell:
```powershell
$env:TESSERACT_CALLER = 'agent-skill'
$env:TESSERACT_HOST = 'codex'
$env:TESSERACT_SKILL = 'tesseract-motion'
$env:TESSERACT_DISTRIBUTION = 'unknown'
& 'C:\absolute\path\to\tsrct.cmd' project create --project Project.tsrct
```
Attribution is self-reported, not proof of skill loading. The CLI emits events;
do not send analytics HTTP requests from the skill.
## Collected data
Only `project.create`, `export`, `preview`, and `filmstrip` emit start/completion
events. Other commands and installation are not tracked.
Collected information includes the command, CLI version, operating system,
architecture, optional attribution, and random IDs for events, command runs,
and the installation. Completion events include success/failure and duration.
The installation ID is saved locally and reused across runs.
No prompts, contents, filenames, paths, raw arguments, or error text are sent.
## Saved settings
Preferences and the installation ID are stored in `settings.json` under:
- macOS: `~/Library/Application Support/Tesseract/telemetry`
- Windows: `%LOCALAPPDATA%\Tesseract\telemetry`
- Linux: `$XDG_CONFIG_HOME/tesseract/telemetry`, or
`~/.config/tesseract/telemetry` if XDG_CONFIG_HOME is absent or not absolute.
Telemetry failures do not prevent editing or rendering. If the CLI cannot read
or save its settings, it skips telemetry; settings commands report an error.
The service may also disable telemetry for a CLI version or installation.
If status reports it as retired, leave it disabled.
references/versions.md
# Versions and compatibility
Use this reference when opening shared or older files, converting projects,
updating tools, or investigating a load failure or unexpected render. Versions
are diagnostic evidence, not a requirement that every component have the same number.
## Where to look
| Component | Evidence |
| --- | --- |
| Skill/plugin release | The plugin manifest identifies the instructions release. The separate `references/cli-version.txt` pin identifies the CLI to download using [installing the CLI](install-cli.md). These versions can differ. |
| Native CLI and renderer | Resolved `tsrct --version` reports the CLI release and source revision; `tsrct --help` reports its FX schema revision in newer builds. Use that release's build information for the bundled rendering engine. |
| Browser runtime | The loaded module's `version()` and `engineVersion()` report its release and engine build. Check the actual deployed runtime; updating a skill or CLI does not update an existing editor. Keep its JavaScript, types, and WASM from the same release. |
| Converter | Check the installed converter’s version and the [Tesseract Converter repository and releases](https://github.com/mirage-hq/Tesseract-Converter) for how to identify its build and associated schema or engine revisions. Its releases are independent of the plugin and rendering runtime. |
| Saved `.tsrct` | Read the archive's `metadata.json` without modifying it: `formatVersion`, optional `fxSchemaVersion`, and `generator.name`, `version`, `engineVersion`, and optional `gitRevision`. `tsrct project inspect` returns editable project JSON, not this metadata. |
These fields answer different questions: `formatVersion` identifies the `.tsrct`
file format, `fxSchemaVersion` identifies the version of the editable FX data
schema used when saving, and `generator` identifies the tool that last saved it.
They are separate version numbers. Saving again can update the schema and tool
metadata; it does not tell you which tool originally created the project.
## Interpret differences
An FX schema difference alone does not prove incompatibility: an added optional
field may not affect this project. Matching schema revisions do not guarantee
identical rendering or conversion either; renderer behavior and converter mappings
can change independently. Use the actual error or visual difference, installed
schemas, conversion diagnostics, known limitations, and relevant release notes
to decide which component needs attention.
Older files can lack `fxSchemaVersion`; treat it as unknown. Older native writers
could stamp `0.1.0` even when using a later CLI such as `0.3.0`; do not infer the
original release from that stamp. A save by a fixed writer records its own version
and schema revision; it cannot recover the original tool version.
New readers accept legacy files without `fxSchemaVersion`, but older readers may
reject newer metadata fields, including `fxSchemaVersion` itself. An unknown-field
error is a concrete reader compatibility issue even when the FX content is usable.
Do not remove metadata or relabel versions to bypass it.
Choose a targeted remedy supported by the evidence: a compatible skill/runtime
release, a converter release with the relevant fix, or a documented workaround
for unsupported content. Respect the skill's CLI pin; do not replace it with an
arbitrary latest binary. Investigate the versions relevant to the issue; a
version difference alone does not call for an update or a full compatibility test.
references/waveform-editing.md
# Edit with the audio waveform
Use the waveform as an editing view, alongside listening and the filmstrip. When music is supplied or generated first, inspect it **before deciding picture and motion timing**. When trimming talking footage, inspect its source audio **before cutting** and check the rendered joins afterward. Generating the image is only preparation: open it with the host's image-view tool and record what it shows.
## Generate an overview, then zoom
The local [sound helper](../scripts/tesseract_sound.py) accepts audio files or the audio inside a video. It writes a labeled PNG and a JSON sidecar containing 10 ms per-channel peak/RMS measurements and candidate markers. It does not alter the source, transcribe speech, or edit a project.
```bash
# Inspect the selected music before planning the visual beats.
python3 "$SKILL/scripts/tesseract_sound.py" waveform /absolute/path/music.wav --mode music --output "$WORK/checks/music-overview.png"
# Inspect a range of talking footage; times remain relative to the SOURCE file.
python3 "$SKILL/scripts/tesseract_sound.py" waveform /absolute/path/talking.mov --mode speech --start-ms 12000 --duration-ms 8000 --output "$WORK/checks/speech-source.png"
# Zoom around proposed source cuts, keeping the original timestamps on the axis.
python3 "$SKILL/scripts/tesseract_sound.py" waveform /absolute/path/talking.mov --mode speech --start-ms 14500 --duration-ms 1800 --markers-ms 15080,15720 --output "$WORK/checks/speech-cut-check.png"
# Inspect the RENDERED edit around the new join, on the EDIT file's clock.
python3 "$SKILL/scripts/tesseract_sound.py" waveform /absolute/path/edit/dialogue-v2.wav --mode speech --clock edit --start-ms 2000 --duration-ms 2000 --markers-ms 3100 --output "$WORK/checks/join-v2.png"
```
Choose ranges and markers from the actual material; the times above are syntax examples. Default is the whole input, up to ten minutes per view. Split longer material into labeled ranges; even shorter overviews need zooms for word boundaries. Use `--audio-stream N` to select a zero-based audio-stream ordinal after probing files with multiple audio tracks. Prefer an isolated dialogue track over a camera scratch mix with background music. The helper never replaces an existing view or sidecar unless you pass `--overwrite`; use it to refresh a routine check in `$WORK/checks/`, and give a view you will compare against later its own name.
The display separates channels, uses a square-root amplitude scale to expose quieter details, and does not normalize gain. Gold lines are transient/energy-rise candidates, pink lines are approximate energy changes, green bars are low-energy intervals, and white lines are the requested review markers. The plot caps each candidate category at 100 markers for legibility; stdout and the sidecar list every candidate. Their numeric timestamps are authoritative; use the image to see their context. Threshold comparisons use the original decoded sample levels, not display height. The sidecar's 10 ms envelope is for scripts (stdout omits it): query a time range with a short `python3` command rather than opening the file.
A `--clock edit` label means the input is an exported edit or stem. It does **not** convert source time to project time. A short preview file begins at zero on its own clock; add its project-range offset explicitly. The helper preserves delayed audio relative to the container start and fills timestamp gaps for inspection. A requested window can end at audio EOF; check the reported actual window. No audio stream is a missing input, not a silent waveform.
## Music first: give the edit a musical structure
1. Open the overview and listen to the selected track. Identify the first usable downbeat, recurring accents, phrase lengths, builds, breaks/dropouts, drops, arrangement changes, and the ending. The waveform helps locate them; a loud transient might be a snare or sound effect, and a new section can begin without getting louder.
2. Use the helper's accent and energy-change candidates as places to investigate. Zoom around important arrivals and audition them. Confirm a beat grid over several bars if useful; check for half/double-time errors, swing, pickups, and tempo changes. Do not assume every peak is a beat or infer a definitive BPM/section map from these heuristics. A dense mastered track may need listening to establish most of the map.
3. Save a small `audio-map.json` or table with music-source time, edit time, confirmed event, and intended visual action. Plan the opening, scene changes, product reveal, motion arrival, and ending against this map. Choose a few meaningful accents; cuts on every beat quickly become mechanical. Motion can start before a beat so its arrival lands on it.
4. Keep the chosen music and its recognizable phrases intact unless the brief calls for a music edit. When shortening it, cut at compatible phrase boundaries, audition the join, and preserve release/reverb tails. Do not globally time-stretch footage or speech to force alignment.
5. Recheck the picture against the music after timing changes. Pair the audio map with filmstrip samples, then play the assembled video with sound; a still image cannot prove synchronization or rhythm.
For music played at normal speed, `edit_time = layer_edit_start + music_source_time - selected_music_source_start`. For a constant source playback rate `r`, divide that source-time difference by `r`. Variable speed requires the actual time mapping; do not use the simple formula across a speed ramp. Preserve separate source, project, and layer clocks when authoring native timing.
## Talking footage: tighten pauses without damaging words
1. Audition the source and read any supplied transcript. Identify complete thoughts, meaningful emphasis, and pauses that can be shortened. Make a speech waveform view before changing source ranges. Never fabricate a transcript or assume quiet pixels mean no speech.
2. Inspect candidate pauses. By default, a green interval means **all channels** stay below a −45 dBFS sample-peak threshold for at least 200 ms, measured in 10 ms bins. Adjust `--quiet-db` and `--min-quiet-ms` for the recording; these are candidate settings, not universal speech thresholds. Quiet consonants, trailing syllables, breaths, and room tone can sit below a threshold. Constant noise or music can hide real pauses.
3. Zoom to roughly 0.5–2 seconds around each intended boundary and listen to the surrounding words. Choose trims in verified gaps, retaining enough lead-in and tail for consonants and natural breathing. Approximately 50–120 ms of handles is a useful first audition, not guaranteed protection. Keep longer pauses that carry meaning. Shorten an awkward pause rather than automatically deleting all silence.
4. Update video and its linked source audio together. Preserve source in/out points and edit positions. At video-frame boundaries, favor retaining a little more material over clipping a word. Use a deliberate J/L cut or room-tone bridge when it improves continuity, with source handles intact. Tiny fades may prevent clicks; fades and crossfades cannot restore missing phonemes and must not blur adjacent words.
5. Render the changed dialogue and inspect its waveform around every changed join, marked on the edit clock. Listen to the previous word, the join, and the next word at normal speed, with music muted first; inspect video frames for jump cuts and lip sync. Then listen in the full mix. Check word completeness, breathing, pacing, clicks, doubled syllables, and any new silence. Correct and re-render the affected range before continuing.
For a dialogue-only check, export a temporary copy of the document with music/SFX gains and gain animators muted, and pass the export directly to the waveform helper.
`tsrct export-audio` exports only the copy's audio, without rendering video.
Preserve the working mix and original source. For very short preview exports, retain the offset from preview time to full project time in the review notes.
## Record the actual checks
Record the revision, source file/stream, inspected ranges, confirmed music events
or speech boundaries, source-to-edit mapping, and what changed after review.
Waveforms support visual and auditory judgment; they do not establish intact words or a good mix on their own. If the host cannot audition audio, state that speech cuts and musical timing remain unverified instead of claiming they passed.
All analysis is local with Python's standard library and local FFmpeg/ffprobe. PNG labeling uses FFmpeg's `drawtext` filter and a locally available diagnostic font; project typography still uses the document's imported fonts. No new service, model, upload, or paid generation is involved.
FFmpeg builds without libfreetype, such as Homebrew's default `ffmpeg`, have no `drawtext`. The helper then draws the same plot, gridlines, and colored markers without text, reports `"image_labels": false`, and explains in `image_labels_note` how to read the unlabeled image; read times from the JSON. For labeled images, use an FFmpeg build with `drawtext`, such as Homebrew's `ffmpeg-full`.
scripts/tesseract_media.py
"""Local file and FFmpeg helpers used by the sound and waveform tools."""
import contextlib
import errno
import hashlib
import json
import os
from pathlib import Path
import shutil
import stat
import subprocess
import sys
import tempfile
if os.name == "nt":
import msvcrt
else:
import fcntl
WORK = ".tesseract-work"
PUBLISH_LOCK = ".tesseract-publish.lock"
# Errors that mean the filesystem has no advisory locks (some network shares).
NO_LOCKS = {errno.ENOLCK, errno.ENOTSUP, errno.EOPNOTSUPP, errno.EINVAL}
def emit(value):
# Compact: agents read this stdout. Keep every field.
print(json.dumps(value, ensure_ascii=False, separators=(",", ":")))
def sha256(path):
h = hashlib.sha256()
with open(path, "rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def existing(value):
p = Path(value).expanduser().resolve()
if not p.is_file():
raise ValueError(f"Local file does not exist: {p}")
return p
def external(name):
p = shutil.which(name)
if p:
return p
for parent in ("/opt/homebrew/bin", "/usr/local/bin"):
p = Path(parent) / name
if p.is_file():
return str(p)
raise ValueError(f"{name} is needed for media inspection. It was not found locally.")
def run(command, timeout=600):
result = subprocess.run([str(x) for x in command], capture_output=True, text=True, timeout=timeout)
if result.returncode:
raise RuntimeError(f"{Path(str(command[0])).name} failed ({result.returncode}):\n{result.stderr[-6000:]}\n{result.stdout[-2000:]}")
if result.stderr:
print(result.stderr[-4000:], file=sys.stderr)
return result.stdout
def output_path(value, overwrite=False, source=None):
p = Path(value).expanduser().resolve()
# samefile also catches case variants, symlinks and hard links of the input.
if source is not None and p.exists() and p.samefile(source):
raise ValueError(f"Output would replace the input: {p}.")
if p.exists():
if not overwrite:
raise ValueError(f"Output already exists: {p}. Choose a new versioned filename, "
"or pass --overwrite to refresh a routine check in .tesseract-work/.")
if work_root(p) is None:
raise ValueError(f"--overwrite only replaces files in a .tesseract-work folder: {p}.")
if not p.is_file():
raise ValueError(f"Output exists and is not a file: {p}.")
p.parent.mkdir(parents=True, exist_ok=True)
return p
def work_root(path):
"""The innermost .tesseract-work folder that contains path, or None."""
parents = path.parts[:-1]
for end in range(len(parents), 0, -1):
if parents[end - 1] == WORK:
return Path(*parents[:end])
return None
def _lock(fd):
if os.name == "nt":
os.lseek(fd, 0, os.SEEK_SET)
while True:
try:
# LK_LOCK retries for about 10 s, then raises EDEADLOCK; keep waiting.
msvcrt.locking(fd, msvcrt.LK_LOCK, 1)
return
except OSError as error:
if error.errno != errno.EDEADLOCK:
raise
try:
fcntl.flock(fd, fcntl.LOCK_EX)
except OSError as error:
if error.errno not in NO_LOCKS:
raise
# Some network shares support only POSIX record locks.
fcntl.lockf(fd, fcntl.LOCK_EX)
def _unlock(fd):
if os.name == "nt":
os.lseek(fd, 0, os.SEEK_SET)
msvcrt.locking(fd, msvcrt.LK_UNLCK, 1)
@contextlib.contextmanager
def publish_lock(outputs, required):
"""Hold the publish lock of every .tesseract-work folder among outputs.
Concurrent publishes into the same folder then run one whole set at a time, so two
refreshes of a PNG and its JSON sidecar never leave a mixed pair. The lock is released
when the process exits, so a crash leaves no stale lock. Where the filesystem has no
advisory locks, a replacing (required) publish is refused; an exclusive-create publish
still refuses any output a concurrent run created."""
roots = sorted({root for root in map(work_root, outputs) if root is not None}, key=str)
with contextlib.ExitStack() as stack:
for root in roots:
path = root / PUBLISH_LOCK
fd = os.open(path, os.O_RDWR | os.O_CREAT, 0o666)
stack.callback(os.close, fd)
try:
_lock(fd)
except OSError as error:
if error.errno not in NO_LOCKS:
raise
if required:
raise ValueError(f"--overwrite needs a file lock, which this filesystem does not "
f"support: {path}. Write a new versioned filename instead.") from error
continue
stack.callback(_unlock, fd)
yield
def regular_file(path):
try:
return stat.S_ISREG(os.lstat(path).st_mode)
except FileNotFoundError:
return False
def publish(staged, overwrite=False):
"""Move (temporary, output) pairs into place; stage each temporary beside its output.
Replace only a regular file that already exists in a .tesseract-work folder, and only
with overwrite; refuse any other existing output, even one a concurrent run created. The
whole set is published under the publish lock of its .tesseract-work folders, so
concurrent refreshes land one complete set after another: the last one wins whole. On
any failure or interrupt, undo the outputs already placed, restoring replaced ones, so a
failed refresh keeps the previous result."""
with publish_lock([output for _, output in staged], overwrite):
_publish(staged, overwrite)
def _publish(staged, overwrite):
placed = []
try:
for temporary, output in staged:
if overwrite and work_root(output) is not None and regular_file(output):
backup = temporary.with_name(temporary.name + ".previous")
try:
os.link(output, backup)
except OSError:
# No hard links here (exFAT, FAT32, some shares): copy, so the output stays
# in place until the replace below.
shutil.copy2(output, backup)
placed.append((output, backup))
os.replace(temporary, output)
continue
try:
os.link(temporary, output)
except FileExistsError:
raise
except OSError:
# No hard links here. Reserve the name first so a concurrent output is still refused.
os.close(os.open(output, os.O_WRONLY | os.O_CREAT | os.O_EXCL))
placed.append((output, None))
os.replace(temporary, output)
else:
placed.append((output, None))
except BaseException:
for output, backup in reversed(placed):
try:
if backup:
os.replace(backup, output)
else:
output.unlink()
except OSError as error:
print(f"Could not restore {output}: {error}", file=sys.stderr)
raise
def staged_write(output, write, overwrite=False):
with tempfile.TemporaryDirectory(prefix=".tesseract-", dir=output.parent) as tmp:
temporary = Path(tmp) / output.name
write(temporary)
publish([(temporary, output)], overwrite)
return output
def atomic_run(output, command, overwrite=False, source=None):
def render(temporary):
run(command(temporary))
if not temporary.is_file() or not temporary.stat().st_size:
raise RuntimeError("Media command returned without producing a nonempty output.")
return staged_write(output_path(output, overwrite, source), render, overwrite)
def write_json(path, data, overwrite=False, source=None):
def dump(temporary):
with temporary.open("x") as f:
json.dump(data, f, indent=2, ensure_ascii=False)
f.write("\n")
return staged_write(output_path(path, overwrite, source), dump, overwrite)
scripts/tesseract_sound.py
#!/usr/bin/env python3
"""Local sound accents and mix inspection; no account, download, or paid service."""
import argparse
from array import array
import json
import math
import random
import re
import subprocess
import sys
import wave
import tesseract_media as local
import tesseract_waveform
RATE = 48000
PRESETS = {
"soft-tap": (100, "micro", "Muted tactile accent for a button or product contact."),
"dry-click": (65, "micro", "Short bright tick for a precise graphic event."),
"soft-pop": (180, "micro", "Rounded accent for a small reveal; avoid every text entry."),
"clear-ping": (480, "micro", "Light two-partial tone for a meaningful confirmation."),
"air-whoosh": (450, "transition", "Filtered air sweep; align its peak to the movement."),
"short-riser": (1100, "transition", "Brief building texture into a reveal, with a clean end."),
"downlifter": (800, "transition", "Descending texture after a scene change."),
"soft-impact": (400, "transition", "Low hit with audible upper harmonics for small speakers."),
}
def synthesize(name, duration_ms, peak_db=-12.0, seed=0):
if name not in PRESETS or not 40 <= duration_ms <= 4000:
raise ValueError("Use a listed preset and a duration from 40 to 4000 ms.")
if not math.isfinite(peak_db) or not -60 <= peak_db <= -3:
raise ValueError("Cue peak must be between -60 and -3 dBFS; leave mix headroom.")
rng = random.Random(seed)
count = round(RATE * duration_ms / 1000)
samples, phase, smooth = [], 0.0, 0.0
duration = count / RATE
for i in range(count):
t = i / RATE
u = i / (count - 1)
noise = rng.uniform(-1.0, 1.0)
smooth += 0.12 * (noise - smooth)
bright = noise - smooth
if name == "soft-tap":
value = (0.8 * smooth + 0.2 * math.sin(2 * math.pi * 420 * t)) * math.exp(-8 * u)
elif name == "dry-click":
value = bright * math.exp(-15 * u)
elif name == "soft-pop":
phase += 2 * math.pi * (210 + 850 * math.exp(-20 * u)) / RATE
value = (math.sin(phase) + 0.13 * bright) * math.exp(-7 * u)
elif name == "clear-ping":
value = (math.sin(2 * math.pi * 1100 * t) + 0.25 * math.sin(2 * math.pi * 1650 * t)) * math.exp(-6 * u)
elif name == "soft-impact":
phase += 2 * math.pi * (65 + 170 * math.exp(-12 * u)) / RATE
value = (math.sin(phase) + 0.3 * math.sin(2 * phase) + 0.25 * bright * math.exp(-35 * u)) * math.exp(-7 * u)
else:
amount = u if name == "short-riser" else 1-u if name == "downlifter" else math.sin(math.pi * u)
phase += 2 * math.pi * (500 + 1900 * amount * amount) / RATE
value = (0.7 * smooth + 0.12 * bright + 0.06 * math.sin(phase)) * amount ** 1.7
# Brief ramps remove hard waveform discontinuities without dulling the cue.
fade = min(1.0, t / 0.002, (duration - 1/RATE - t) / 0.008)
samples.append(value * max(0.0, fade))
peak = max(abs(x) for x in samples)
scale = 10 ** (peak_db / 20) / peak
return array("h", (round(x * scale * 32767) for x in samples))
def save_cue(name, path, duration_ms=None, peak_db=-12.0, seed=0, overwrite=False):
duration_ms = PRESETS[name][0] if duration_ms is None else duration_ms
samples = synthesize(name, duration_ms, peak_db, seed)
output = local.output_path(path, overwrite)
if output.suffix.lower() != ".wav":
raise ValueError("Local procedural cues are PCM WAV files; use .wav.")
if sys.byteorder != "little":
samples.byteswap()
def write(temporary):
with temporary.open("xb") as f:
with wave.open(f, "wb") as w:
w.setnchannels(1)
w.setsampwidth(2)
w.setframerate(RATE)
w.writeframes(samples.tobytes())
local.staged_write(output, write, overwrite)
return {"path": str(output), "preset": name, "duration_ms": duration_ms,
"sample_peak_dbfs": peak_db, "seed": seed,
"origin": "Locally synthesized from oscillators and seeded noise; no recorded samples."}
def measure(path):
src = local.existing(path)
result = subprocess.run([local.external("ffmpeg"), "-hide_banner", "-nostdin", "-i", str(src),
"-map", "0:a:0", "-af", "loudnorm=I=-15:TP=-1:LRA=7:print_format=json",
"-f", "null", "-"], capture_output=True, text=True, timeout=600)
if result.returncode:
raise RuntimeError(result.stderr[-2500:])
candidates = re.findall(r'\{\s*"input_i".*?\}', result.stderr, re.S)
if not candidates:
raise RuntimeError("FFmpeg did not return loudness measurements.")
raw = json.loads(candidates[-1])
def finite(key):
v = float(raw[key])
return v if math.isfinite(v) else None
il, tp = finite("input_i"), finite("input_tp")
return {"path": str(src), "audio_stream": "first audio stream", "integrated_lufs": il,
"true_peak_dbtp": tp, "loudness_range_lu": finite("input_lra"),
"working_target": {"integrated_lufs": [-16, -14], "true_peak_max_dbtp": -1},
"within_working_target": il is not None and tp is not None and -16 <= il <= -14 and tp <= -1,
"note": "Measurement only. Targets are a social-edit starting point, not a platform guarantee. Listen to the mix; silence/very short cues may not have meaningful integrated loudness."}
def main():
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest="command", required=True)
sub.add_parser("list")
replace = "Replace an existing output, such as a routine check from an earlier pass"
p = sub.add_parser("make")
p.add_argument("--preset", choices=PRESETS, required=True)
p.add_argument("--output", required=True)
p.add_argument("--duration-ms", type=int)
p.add_argument("--peak-db", type=float, default=-12)
p.add_argument("--seed", type=int, default=0)
p.add_argument("--overwrite", action=argparse.BooleanOptionalAction, default=False, help=replace)
p = sub.add_parser("measure")
p.add_argument("input")
p.add_argument("--report")
p.add_argument("--overwrite", action=argparse.BooleanOptionalAction, default=False, help=replace)
p = sub.add_parser("waveform", help="Labeled waveform PNG + JSON candidates for music or speech review")
p.add_argument("input")
p.add_argument("--output", required=True)
p.add_argument("--report")
p.add_argument("--mode", choices=("music", "speech"), required=True)
p.add_argument("--start-ms", type=int, default=0)
p.add_argument("--duration-ms", type=int)
p.add_argument("--quiet-db", type=float, default=-45)
p.add_argument("--min-quiet-ms", type=int, default=200)
p.add_argument("--markers-ms", help="Comma-separated times on the input file clock; mark proposed cuts or accents")
p.add_argument("--clock", choices=("source", "edit"), default="source")
p.add_argument("--audio-stream", type=int, default=0, help="Zero-based audio stream ordinal")
p.add_argument("--overwrite", action=argparse.BooleanOptionalAction, default=False, help=replace)
p = sub.add_parser("mobile-preview")
p.add_argument("input")
p.add_argument("--output", required=True)
p.add_argument("--overwrite", action=argparse.BooleanOptionalAction, default=False, help=replace)
args = parser.parse_args()
if args.command == "list":
local.emit([{"preset": k, "duration_ms": v[0], "role": v[1], "use": v[2]} for k,v in PRESETS.items()])
elif args.command == "make":
local.emit(save_cue(args.preset, args.output, args.duration_ms, args.peak_db, args.seed, args.overwrite))
elif args.command == "measure":
src = local.existing(args.input)
# Check the report path before the measurement pass, which can take minutes.
if args.report and local.output_path(args.report, args.overwrite, src).suffix.lower() != ".json":
raise ValueError("Use .json for the loudness report.")
report = measure(src)
if args.report:
local.write_json(args.report, report, args.overwrite, src)
local.emit(report)
elif args.command == "waveform":
markers = [float(x) for x in args.markers_ms.split(",")] if args.markers_ms else []
local.emit(tesseract_waveform.waveform(args.input, args.output, args.report, args.mode,
args.start_ms, args.duration_ms, args.quiet_db, args.min_quiet_ms, markers,
args.clock, args.audio_stream, args.overwrite))
else:
src = local.existing(args.input)
if local.output_path(args.output, args.overwrite, src).suffix.lower() != ".wav":
raise ValueError("Use .wav for the diagnostic listening copy.")
p = local.atomic_run(args.output, lambda tmp: [local.external("ffmpeg"), "-hide_banner", "-nostdin", "-n",
"-i", src, "-map", "0:a:0", "-vn", "-af", "aformat=channel_layouts=mono,highpass=f=150,lowpass=f=7000",
"-ar", "48000", "-c:a", "pcm_s16le", tmp], args.overwrite, src)
local.emit({"output": str(p), "purpose": "Mono, band-limited diagnostic copy. Not the deliverable or an exact phone simulation."})
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except (ValueError, RuntimeError, OSError, subprocess.TimeoutExpired) as error:
print(f"tesseract-sound: {error}", file=sys.stderr)
raise SystemExit(1)
scripts/tesseract_waveform.py
#!/usr/bin/env python3
"""Timestamped local waveform views and review candidates; never automatic edits."""
from array import array
import json
import math
from pathlib import Path
import statistics
import sys
import tempfile
import tesseract_media as local
RATE = 48000
BIN_MS = 10
MAX_WINDOW_MS = 600000
# The plot needs these; text labels also need drawtext, which builds without
# libfreetype (such as Homebrew's default ffmpeg) do not include.
PLOT_FILTERS = ('showwavespic', 'format', 'pad', 'drawbox')
NO_LABELS_NOTE = ('Text labels were omitted: this FFmpeg has no drawtext filter (it needs libfreetype). '
'The waveform, markers and quiet bands are complete. Faint gridlines divide the window into sixths '
'from window_start_ms to window_end_ms, channels are stacked from channel 1 at the top, and colors '
'follow the waveform reference; read exact times from this JSON. For labeled images, use an FFmpeg '
'with drawtext, such as Homebrew ffmpeg-full.')
def db(value):
return round(20 * math.log10(value), 2) if value > 0 else None
def envelope(raw, channels, start_ms):
"""Keep channels separate: anti-phase stereo must not look like silence."""
points = []
frames = 0
with Path(raw).open('rb') as f:
while True:
block = f.read(RATE * BIN_MS // 1000 * channels * 4)
if not block:
break
if len(block) % (channels * 4):
raise ValueError('Incomplete decoded audio frame.')
samples = array('f', block)
if sys.byteorder != 'little':
samples.byteswap()
if not all(math.isfinite(s) for s in samples):
raise ValueError('Audio contains non-finite samples.')
count = len(samples) // channels
peaks, rms = [], []
for c in range(channels):
values = samples[c::channels]
peaks.append(max(abs(s) for s in values))
rms.append(math.sqrt(sum(s*s for s in values) / count))
points.append({'start_ms': round(start_ms + frames * 1000 / RATE, 3),
'end_ms': round(start_ms + (frames + count) * 1000 / RATE, 3),
'channel_peak_dbfs': [db(v) for v in peaks],
'channel_rms_dbfs': [db(v) for v in rms]})
frames += count
if not points:
raise ValueError('No audio in the requested window.')
return points, frames
def peak(point):
return max((v for v in point['channel_peak_dbfs'] if v is not None), default=-160)
def rms(point):
return max((v for v in point['channel_rms_dbfs'] if v is not None), default=-160)
def candidates(points, mode, quiet_db, min_quiet_ms):
quiet, begin = [], None
for i, p in enumerate(points):
low = peak(p) < quiet_db
if low and begin is None:
begin = p['start_ms']
if begin is not None and (not low or i == len(points)-1):
end = p['end_ms'] if low else p['start_ms']
if end - begin >= min_quiet_ms:
quiet.append({'start_ms': begin, 'end_ms': end})
begin = None
result = {'quiet_intervals': quiet}
if mode != 'music':
return result
# Abrupt short-window energy rises, not beat/downbeat or tempo estimation.
levels = [rms(p) for p in points]
accents = []
for i in range(1, len(points)):
previous = statistics.median(levels[max(0, i-10):i])
rise = levels[i] - max(previous, -80)
if levels[i] > -45 and rise >= 6 and levels[i] - levels[i-1] >= 3:
if not accents or points[i]['start_ms'] - accents[-1]['time_ms'] >= 120:
accents.append({'time_ms': points[i]['start_ms'], 'rise_db': round(rise, 2)})
# Compare one-second neighborhoods; do not label these as semantic sections.
changes = []
for i in range(100, len(points)-100, 25):
before = statistics.mean(max(v, -80) for v in levels[i-100:i])
after = statistics.mean(max(v, -80) for v in levels[i:i+100])
delta = after - before
if abs(delta) >= 6:
changes.append({'time_ms': points[i]['start_ms'], 'energy_change_db': round(delta, 2)})
selected = []
for candidate in sorted(changes, key=lambda v: -abs(v['energy_change_db'])):
if all(abs(candidate['time_ms'] - v['time_ms']) >= 1500 for v in selected):
selected.append(candidate)
changes = sorted(selected, key=lambda v: v['time_ms'])
result.update({'accent_candidates': accents, 'energy_change_candidates': changes})
return result
def ffmpeg_filters(ffmpeg):
"""Names of the filters compiled into this FFmpeg build."""
listing = local.run([ffmpeg, '-hide_banner', '-filters'], 30)
# Filter rows read `<flags> <name> <inputs>-><outputs> <description>`.
return {fields[1] for fields in map(str.split, listing.splitlines())
if len(fields) > 2 and '->' in fields[2]}
def text_labels(ffmpeg):
"""Whether this FFmpeg can label the plot; name any filter the plot itself lacks."""
available = ffmpeg_filters(ffmpeg)
missing = [name for name in PLOT_FILTERS if name not in available]
if missing:
raise ValueError(f"{ffmpeg} lacks the FFmpeg filter(s) {', '.join(missing)} needed to draw "
'waveform views. Install a standard FFmpeg build.')
return 'drawtext' in available
def draw_filter(report):
width, height, left, top = 1320, 360, 60, 86
start, end = report['window_start_ms'], report['window_end_ms']
def x(t):
return left + min(width-1, max(0, round(width * (t-start) / (end-start))))
diagnostic_font = Path('/System/Library/Fonts/Supplemental/Arial.ttf')
font_option = f"fontfile='{diagnostic_font}':" if diagnostic_font.is_file() else ''
def label(text, px, py, size=18, color='0xe5e7eb'):
if not report['image_labels']:
return None
# Only internally generated labels/numbers are passed into filter syntax.
return f"drawtext={font_option}text='{text}':expansion=none:fontsize={size}:fontcolor={color}:x={px}:y={py}"
filters = [f'showwavespic=s={width}x{height}:split_channels=1:colors=0x7dd3fc|0xc4b5fd|0x86efac|0xfda4af|0xfde68a|0x67e8f9|0xd8b4fe|0xfdba74:scale=sqrt:filter=peak',
'format=rgb24', f'pad=1440:560:{left}:{top}:color=0x111827']
for i in range(7):
px = left + round((width-1)*i/6)
t = start + (end-start)*i/6
filters.append(f'drawbox=x={px}:y={top}:w=1:h={height}:color=white@0.15:t=fill')
filters.append(label(f'{t/1000:.3f}s', f'{px}-text_w/2', top+height+12, 17))
marks = report['candidates']
# Overview markers are capped for readability; JSON keeps every candidate.
for q in marks['quiet_intervals'][:100]:
filters.append(f"drawbox=x={x(q['start_ms'])}:y={top+height-8}:w={max(1,x(q['end_ms'])-x(q['start_ms']))}:h=8:color=0x86efac:t=fill")
for q in marks.get('accent_candidates', [])[:100]:
filters.append(f"drawbox=x={x(q['time_ms'])}:y={top}:w=1:h={height}:color=0xfbbf24@0.55:t=fill")
for q in marks.get('energy_change_candidates', [])[:100]:
filters.append(f"drawbox=x={x(q['time_ms'])}:y={top}:w=2:h={height}:color=0xf472b6@0.85:t=fill")
for t in report['review_markers_ms']:
filters.append(f'drawbox=x={x(t)}:y={top}:w=2:h={height}:color=white:t=fill')
for c in range(report['channels']):
filters.append(label(f'CH {c+1}', left+8, top+c*height//report['channels']+6, 13))
filters.append(label(f"{report['mode'].upper()} WAVEFORM - {report['clock'].upper()} TIME", 60, 22, 24))
filters.append(label('Seconds from media start / separate channels / square-root amplitude / no gain normalization', 60, 57, 16, '0x9ca3af'))
if report['mode'] == 'music':
legend = 'Gold = accent candidates Pink = energy shifts Green = quiet intervals White = review markers'
else:
legend = 'Green = quiet candidates across ALL channels White = review markers Quiet does not mean safe to cut'
filters.append(label(legend, 60, 492, 17))
filters.append(label('Open and inspect this image. Listen before confirming a beat or edit. Zoom around speech cuts.', 60, 525, 16, '0x9ca3af'))
return ','.join(f for f in filters if f)
def waveform(path, output, report_path=None, mode='music', start_ms=0, duration_ms=None,
quiet_db=-45, min_quiet_ms=200, markers_ms=(), clock='source', audio_stream=0, overwrite=False):
if mode not in ('music', 'speech') or clock not in ('source', 'edit'):
raise ValueError('Use music/speech mode and source/edit clock.')
if start_ms < 0 or not math.isfinite(quiet_db) or not -90 <= quiet_db <= -15:
raise ValueError('Start must be nonnegative; quiet threshold must be -90 to -15 dBFS.')
if min_quiet_ms < BIN_MS or min_quiet_ms > MAX_WINDOW_MS:
raise ValueError('Quiet duration must be 10 to 600000 ms.')
src = local.existing(path)
out = local.output_path(output, overwrite, src)
if out.suffix.lower() != '.png':
raise ValueError('Use .png for the waveform view.')
dest = local.output_path(report_path or out.with_suffix('.json'), overwrite, src)
if dest == out or dest.suffix.lower() != '.json':
raise ValueError('Use a distinct .json report path.')
meta = json.loads(local.run([local.external('ffprobe'), '-v', 'error', '-show_streams', '-show_format', '-of', 'json', src], 30))
streams = [s for s in meta.get('streams', []) if s.get('codec_type') == 'audio']
if audio_stream < 0 or audio_stream >= len(streams):
raise ValueError('Requested audio stream does not exist; probe the file first.')
stream = streams[audio_stream]
channels = int(stream['channels'])
if not 1 <= channels <= 8:
raise ValueError('Waveform views support 1 to 8 channels per audio stream.')
fmt = meta.get('format', {})
origin = float(fmt.get('start_time') or 0)
media_duration = float(fmt['duration']) * 1000 if fmt.get('duration') else None
if duration_ms is None:
if media_duration is None:
raise ValueError('Duration unknown; provide --duration-ms.')
duration_ms = math.ceil(media_duration) - start_ms
if not 0 < duration_ms <= MAX_WINDOW_MS:
raise ValueError('Choose a window of 1 to 600000 ms; split long media into labeled ranges.')
if media_duration is not None and start_ms >= media_duration:
raise ValueError('Window starts after the end of the media.')
markers = sorted(set(markers_ms))
if len(markers) > 48 or any(not math.isfinite(t) or not start_ms <= t < start_ms+duration_ms for t in markers):
raise ValueError('Use up to 48 review markers within the requested window.')
ffmpeg = local.external('ffmpeg')
labels = text_labels(ffmpeg)
with tempfile.TemporaryDirectory(prefix='.tesseract-waveform-', dir=out.parent) as tmp:
raw = Path(tmp)/'segment.f32'
# Preserve timestamps relative to the container origin, including delayed audio.
# Resampling fills timestamp gaps; no mono summing or level normalization occurs.
af = (f'asetpts=PTS-({origin:.9f})/TB,aresample={RATE}:async=1:first_pts=0,'
f'atrim=start_sample={start_ms*RATE//1000}:end_sample={(start_ms+duration_ms)*RATE//1000},asetpts=PTS-STARTPTS')
local.run([ffmpeg, '-v', 'error', '-nostdin', '-n', '-copyts', '-i', src,
'-map', f'0:a:{audio_stream}', '-vn', '-af', af, '-t', f'{duration_ms/1000:.6f}',
'-c:a', 'pcm_f32le', '-f', 'f32le', raw])
points, frames = envelope(raw, channels, start_ms)
end = round(start_ms+frames*1000/RATE, 3)
if any(t >= end for t in markers):
raise ValueError('A review marker is beyond the decoded audio window.')
report = {'input': str(src), 'input_sha256': local.sha256(src), 'mode': mode, 'clock': clock,
'audio_stream_ordinal': audio_stream, 'container_origin_seconds': origin,
'channels': channels, 'analysis_sample_rate': RATE, 'envelope_bin_ms': BIN_MS,
'window_start_ms': start_ms, 'window_end_ms': end,
'requested_duration_ms': duration_ms, 'decoded_duration_ms': round(frames*1000/RATE, 3),
'quiet_threshold_dbfs': quiet_db, 'min_quiet_ms': min_quiet_ms,
'review_markers_ms': markers, 'image': str(out), 'image_labels': labels, 'report': str(dest),
'candidates': candidates(points, mode, quiet_db, min_quiet_ms), 'envelope': points,
'interpretation': 'All timestamps are milliseconds from the input media start on the labeled clock. Quiet intervals require every channel below the peak threshold. Accent and energy candidates are heuristics, not beat/downbeat, word-boundary, or section detection. Inspect the PNG, zoom and listen; never auto-delete candidates. No source or project was changed.'}
if not labels:
report['image_labels_note'] = NO_LABELS_NOTE
png = Path(tmp)/'waveform.png'
local.run([ffmpeg, '-v', 'error', '-nostdin', '-n', '-f', 'f32le', '-ar', str(RATE),
'-ac', str(channels), '-i', raw, '-filter_complex', draw_filter(report),
'-frames:v', '1', '-update', '1', png])
# Stage the sidecar beside its own destination, which --report may put on another filesystem.
with tempfile.TemporaryDirectory(prefix='.tesseract-waveform-', dir=dest.parent) as staged:
js = Path(staged)/'waveform.json'
js.write_text(json.dumps(report, indent=2)+'\n')
# No incomplete-looking deliverable after processing failure; a failed refresh
# keeps the previous pair.
local.publish([(png, out), (js, dest)], overwrite)
# Return small tool output; detailed envelope is in the JSON sidecar.
return {k:v for k,v in report.items() if k != 'envelope'}
SKILL.md
---
name: tesseract-video
description: Edit existing footage into finished videos locally with Tesseract. Make cuts, preserve dialogue, add purposeful sound design, mix audio, and decide where native motion-graphics scenes or overlays improve the story. Use for video editing and revising Tesseract projects, not avatar or video generation.
---
# Edit video with Tesseract
For converting files to or from other editing tools, such as Premiere Pro or After Effects, read [conversion](references/conversion.md). When opening shared or older `.tsrct` files, updating tools, or investigating load/render differences, read [versions and compatibility](references/versions.md).
The CLI may send basic usage telemetry for some commands. You can provide optional attribution via `TESSERACT_SKILL=tesseract-video` and the [other attribution variables](references/telemetry.md). Respect the CLI's telemetry opt-out setting; never enable telemetry on the user's behalf.
Make a deliberate edit from the user's material. Tesseract is the local editing and rendering engine; the agent supplies editorial judgment. The output is both a playable video and its editable project.
## Start with the material
Resolve this skill's root from the directory containing this `SKILL.md`. Read [CLI installation](references/installation.md) and check the required Tesseract version once per session. Use the resolved executable path throughout the task. Scripts and references are relative to the installed skill, not a guessed home directory or development checkout.
1. Read [local operation](references/local-operation.md). Probe the supplied media, sample frames at meaningful points, and listen to its audio using the host's available media tools. Metadata does not tell you what a shot depicts. If audio cannot be auditioned, disclose that limit rather than claiming a complete review.
2. Establish the deliverable, audience, aspect ratio, duration, must-keep moments, and supplied brand assets from the request. Ask only for missing information that materially changes the edit. For a small correction, preserve the existing edit's style and scope.
3. Read [editorial decisions](references/editorial-decisions.md) before planning a new edit. For each beat decide **footage**, **footage with graphics**, or **full graphics** and why. This is not a quota: do not add motion graphics just to demonstrate the engine.
For a new ad or a revision to its opening, read [ad hooks](references/ad-hooks.md). Build a visual hook in the opening seconds from the actual footage, proposition, and evidence. Consider distinct concepts, select the strongest, and map its reveal/motion and purposeful SFX to confirmed opening music cues when music is provided. Render and inspect the hook plus its handoff into the ad before polishing the rest. Preserve the scope of unrelated revisions.
Read [waveform editing](references/waveform-editing.md) when music sets the pacing or when trimming talking footage. Generate and **open** the music waveform before planning picture timing; confirm beats, phrases, and major changes by listening and map visual arrivals to them. For speech, use overview and zoomed source waveforms to choose pause trims, protect quiet word boundaries, and retain natural breathing. Waveform markers are candidates, never automatic cut decisions.
For a substantial new video, share a concise edit plan with the source moments, their narrative purpose, graphic treatment, and audio intent. Routine authorized edits can proceed. Do not require a new approval for every local preview or revision.
## Author the edit
- Import local footage before placing its layers. Preserve the original and stable IDs. Save edits to the document JSON, or apply supported action batches, before rendering.
New edit: `tsrct project create` creates a portable `.tsrct` document. Existing `.tsrct`: inspect and check out a copy of its editable JSON. Save JSON through `project commit`, or apply action batches with `project apply`.
- Read [native authoring](references/native-authoring.md) before changing project structure. Source time, edit time, and layer time are different clocks. In particular, moving a clip must not silently change which source moment it plays.
- Keep footage cuts, full scenes, and editable overlays as native layers in the document's single composition. Use groups for scenes. In document JSON, Video/Audio/Group use windowed `playback` for placement; other layer kinds use `activeRange`. `sourceRange` selects the source interval. Do not create base-video, caption, or audio tracks.
- For talking-head cuts, follow [eyeline continuity](references/editorial-decisions.md#talking-head-eyeline-continuity): align the speaker’s eye height and screen position across cuts with appropriate framing, including punch-ins. Check the rendered outgoing/incoming frames and playback while preserving natural head movement and intentional camera changes.
- For motion work, read [motion design](references/motion-design.md), then only the relevant authoring reference. The examples demonstrate real schema; their visual style is not a required template.
- For every new edit or sound pass, read [sound design](references/sound-design.md). Consider dialogue, music, micro-SFX, structural cues, and foley/ambience; use the roles the piece needs. Add appropriate sounds from supplied/local assets or the procedural helper without per-cue approval. Import the selected files, then keep them as editable Audio layers.
Audio files import with `project import-asset --kind audio`.
Use [audio and timing](references/audio-and-timing.md) for placement, ducking, local preparation, and measurement. Preserve useful source audio and chosen music. Do not produce invented speech or captions.
- When footage needs color correction, a look, log conversion, or shot matching, or the user asks about color, read [color grading](references/color-grading.md). Follow its [scene-aware workflow](references/color-grading.md#workflow), including [shot-local control](references/color-grading.md#baked-edits-and-shot-local-control) when needed for a baked edit. Shot consistency is expected without a separate request, but intentional lighting differences should remain. Follow its [source-relative neutral and skin checks](references/color-grading.md#source-relative-neutral-and-skin-checks): use median Lab regions from the original source and final output at matching timestamps, not your earlier graded preview. Keep corrections native and editable; [protect skin under looks](references/color-grading.md#keeping-skin-natural-under-a-look) and check secondary selections against an exaggerated preview (the CLI has no matte-preview switch). Use only one spatial `noiseReduction` instance and check edge retention; see [noise reduction](references/color-grading.md#noise-reduction). In a default document, HDR output does not make grades HDR-domain; for filmic highlights, light-like compositing, camera log without vendor LUTs, or one grade for SDR and HDR, use that reference's [ACES](references/color-grading.md#aces-scene-referred-color-management) section. See its Limitations before promising tracking or automatic matching.
- When the work involves `--hdr`, an HDR delivery (including keeping `hdrTransfer` `hlg` or `pq` footage HDR), `hdrGraphicsWhiteNits`, RGB above 1 or HDR images, read [HDR](references/hdr.md).
- When talking audio needs cleanup, read [speech cleanup](references/speech-cleanup.md). Diagnose room noise versus reverb/echo, use restrained EQ/dynamics or local noise reduction as appropriate, and compare at matched loudness. Preserve voice detail and timing; the plugin has no bundled dereverberation model.
- Keep typography, graphics, footage, and audio native and editable. Import supplied images as assets, not as a flattened replacement for text, vector graphics, or entire scenes.
Images import with `project import-asset --kind image`. Use existing footage; this plugin provides no generative-video or avatar service.
- Respect the user's brand references.
Mirage branding identifies this plugin, not every video created with it.
## Look up capabilities only as needed
Look up the document and action schemas as [local operation](references/local-operation.md) describes.
Read [capabilities](references/capabilities.md) for the feature map and limits,
[FX authoring](references/fx-authoring.md) for native layers and footage, and
[motion](references/motion.md) for supported keyframe and script actions. Search
for the specific definition needed rather than dumping the whole schema. The
action schema includes supported standalone operations and their field restrictions; use
[local operation](references/local-operation.md) to choose actions versus editing
the document JSON. The installed schema and actual render are authoritative.
## Review and handoff
Use [filmstrip review](references/filmstrip-review.md) to inspect and correct visual
changes. For dialogue cuts and music timing, follow [waveform editing](references/waveform-editing.md).
Before delivery, export and run the checks in [review and delivery](references/review-and-delivery.md), including playback with audio. Inspect several exported timestamps away from cuts/fades and compare them with the source, checking footage as well as overlays; near a fade, confirm the actual frame timestamps before diagnosing missing footage. Block delivery on unexplained black picture; investigate it, never dismiss it as an HDR viewing artifact.
Check the opening and ending, reading time, overlay placement, and source-frame
continuity. For talking-head cuts, verify eyeline continuity at playback speed.
For graded footage, follow the [grading review](references/color-grading.md#workflow).
Probe the encoded output with local `ffprobe` to check streams, duration,
resolution, and codec. Listen for clear speech, motivated cues, smooth gain
changes, retained source sound, no doubled playback or clipped transients, and
correct lip sync. An audio stream alone does not prove audible sound. Use the
[audio delivery checks](references/audio-and-timing.md#measure-and-check-the-final-delivery)
for encoded loudness, true peak, and mono listening; after speech cleanup, compare
the same phrases at matched loudness and listen in the full mix.
Then read and follow the [critic review](references/critic-review.md): two independent critics, an art director and a specialist for the video's channel and audience, such as a YouTube strategist for a YouTube video, score the export, and you revise between rounds until both rate it at least 8.5/10, for at most three review rounds. Tell the user the review plan, each round's scores and feedback, and the final score.
Hand off as described in [review and delivery](references/review-and-delivery.md#handoff); report actual
checks, the critics' final scores, and any limitations.
Use absolute local file links and show the video inline
when supported. Cloud sync and publishing are not implemented.
If the host cannot execute local commands or access the local GPU/filesystem, explain that this package needs a local execution environment. Do not substitute a hosted service or silently switch rendering engines.