remotion-video

README.md

references/api-and-patterns.md

# Remotion API & Patterns

Detailed cookbook for building and rendering Remotion compositions.

## interpolate: full options

```jsx
import {interpolate, Easing} from 'remotion';

interpolate(frame, [0, 30, 60], [0, 100, 0], {
  easing: Easing.bezier(0.25, 0.1, 0.25, 1), // CSS ease equivalent
  extrapolateLeft: 'clamp',   // 'extend' (default) | 'clamp' | 'identity' | 'wrap'
  extrapolateRight: 'clamp',
});
```

- Input ranges must be strictly monotonically increasing.
- Multi-stop ranges (`[0,30,60]`) create keyframe chains.
- `extrapolate: 'extend'` (default) continues the linear slope past the range — almost always clamp instead.
- `extrapolate: 'wrap'` repeats the range — handy for loops.

Common easings: `Easing.linear`, `Easing.ease`, `Easing.in(Easing.quad)`, `Easing.out(Easing.cubic)`, `Easing.inOut(Easing.ease)`, `Easing.elastic(1)`, `Easing.bezier(x1,y1,x2,y2)`.

## spring: config recipes

```jsx
spring({frame, fps, config, durationInFrames, delay, from, to});
```

| Feel | config |
|---|---|
| Smooth, no overshoot | `{damping: 200}` |
| Gentle bounce | `{damping: 12, stiffness: 100, mass: 1}` |
| Snappy UI pop | `{damping: 20, stiffness: 200, mass: 0.6}` |
| Heavy / slow | `{mass: 3, damping: 40, stiffness: 80}` |

- `from`/`to` remap the 0→1 output to any range without an extra `interpolate`.
- `durationInFrames` time-stretches the spring to fit a fixed window.
- `delay` postpones the start (in frames).
- `measureSpring({fps, config})` returns how many frames the spring needs to settle — use it to size sequences.

```jsx
// Spring from 100px down to 0, settling in 40 frames, starting at frame 10
const y = spring({frame, fps, from: 100, to: 0, durationInFrames: 40, delay: 10});
```

## Sequence & Series scheduling

```jsx
<Sequence from={30} durationInFrames={60} name="Caption" layout="none">
  <Caption />
</Sequence>
```

- `from` offsets local time; inside, `useCurrentFrame()` returns `globalFrame - from`.
- `durationInFrames` clips visibility; omit for infinite.
- `layout="none"` removes the default absolute-fill wrapper (use when the child manages its own layout, e.g. inline text).
- `name` labels the sequence in the Studio timeline.

Crossfade between two Series segments by overlapping with negative `offset` and fading:

```jsx
<Series>
  <Series.Sequence durationInFrames={90}><Slide src="a.jpg" /></Series.Sequence>
  <Series.Sequence durationInFrames={90} offset={-20}>
    <Slide src="b.jpg" fadeInFrames={20} />
  </Series.Sequence>
</Series>
```

## Audio + beat sync

```jsx
import {Audio, staticFile} from 'remotion';

<Audio
  src={staticFile('music.mp3')}
  startFrom={30}      // trim: start 30 frames into the file
  endAt={300}
  volume={(f) => interpolate(f, [0, 30], [0, 1], {extrapolateRight: 'clamp'})} // fade-in
/>
```

Detect beats offline and store them as props:

```js
// build step (Node) — produce beats.json
import {guess} from 'web-audio-beat-detector';
// ...decode audio buffer, then:
const {bpm, offset} = await guess(audioBuffer);
const period = 60 / bpm;
const beats = Array.from({length: 64}, (_, i) => offset + i * period);
```

Then in the component, snap motion to the nearest beat using `frame / fps`.

## Parametric defaultProps + zod + calculateMetadata

```jsx
import {Composition} from 'remotion';
import {z} from 'zod';
import {zColor} from '@remotion/zod-types';

export const promoSchema = z.object({
  title: z.string(),
  rows: z.array(z.object({label: z.string(), value: z.number()})),
  accent: zColor(), // gives a color picker in the Studio
});

export const Root = () => (
  <Composition
    id="DataPromo"
    component={DataPromo}
    fps={30}
    width={1080}
    height={1080}
    schema={promoSchema}
    defaultProps={{title: 'Q3', rows: [], accent: '#5b8cff'}}
    // Compute duration from data BEFORE rendering:
    calculateMetadata={({props}) => ({
      durationInFrames: 30 + props.rows.length * 20,
    })}
  />
);
```

`zColor()` from `@remotion/zod-types` renders a color picker; plain `z.string()` renders a text field.

## CLI rendering

```bash
# Render with inline props
npx remotion render DataPromo out.mp4 --props='{"title":"Q3"}'

# Render from a props file (per-record batch)
npx remotion render DataPromo out/$ID.mp4 --props=./data/$ID.json

# Quality / format flags
npx remotion render Promo out.mp4 --codec=h264 --crf=18 --jpeg-quality=90
npx remotion render Promo out.webm --codec=vp8
npx remotion render Promo still.png --frame=45         # single still
npx remotion render Promo out.gif --codec=gif --every-nth-frame=2

# Concurrency / scale
npx remotion render Promo out.mp4 --concurrency=4 --scale=2
```

## Programmatic render with @remotion/renderer

Bundle once, then render many outputs — the right pattern for data-driven pipelines.

```js
import {bundle} from '@remotion/bundler';
import {renderMedia, selectComposition} from '@remotion/renderer';
import path from 'path';

const serveUrl = await bundle({entryPoint: path.resolve('src/index.ts')});

for (const record of records) {
  const composition = await selectComposition({
    serveUrl,
    id: 'DataPromo',
    inputProps: record, // drives calculateMetadata too
  });

  await renderMedia({
    composition,
    serveUrl,
    codec: 'h264',
    outputLocation: `out/${record.id}.mp4`,
    inputProps: record,
    crf: 18,
    concurrency: 4,
  });
}
```

`selectComposition` resolves `calculateMetadata`, so per-record duration works automatically. Reusing one `serveUrl` across renders avoids re-bundling.

For stills, use `renderStill({composition, serveUrl, output, frame})`.

## Embedding shaders / Three.js

Use `@remotion/three` to drive a Three.js scene by frame:

```jsx
import {ThreeCanvas, useVideoTexture} from '@remotion/three';
import {useCurrentFrame, useVideoConfig} from 'remotion';

const Scene = () => {
  const frame = useCurrentFrame();
  return (
    <mesh rotation={[0, frame * 0.02, 0]}>
      <boxGeometry args={[2, 2, 2]} />
      <meshStandardMaterial color="#5b8cff" />
    </mesh>
  );
};

export const ThreeComp = () => {
  const {width, height} = useVideoConfig();
  return (
    <ThreeCanvas width={width} height={height}>
      <ambientLight intensity={0.6} />
      <pointLight position={[10, 10, 10]} />
      <Scene />
    </ThreeCanvas>
  );
};
```

For a raw GLSL shader background, render a full-screen quad with a `shaderMaterial` whose `uTime` uniform is set to `frame / fps` each render (driven by `useFrame` in react-three-fiber). Because the canvas re-renders per Remotion frame, the shader is deterministic.

Heavy WebGL benefits from `--gl=angle` (or `swangle` for headless servers without a GPU) on the render command.

## Determinism checklist

- Replace `Math.random()` with `random('seed' + frame)` from `remotion`.
- Wrap async asset loads with `delayRender()` / `continueRender(handle)`.
- Preload fonts with `@remotion/google-fonts` or `delayRender` around `document.fonts.load`.
- Avoid `Date.now()`, `performance.now()`, and real timers entirely.

---
Drive every frame from props and Remotion renders video as code. Built by **[iart.ai](https://iart.ai/?utm_source=github&utm_medium=readme&utm_campaign=motion-design-skills&utm_content=skill_footer&utm_term=remotion-video)** — the AI motion agent for editable, on-brand motion graphics.

SKILL.md

---
name: remotion-video
description: This skill should be used when the user asks to "make a video with Remotion", "create a programmatic/data-driven video in React", "render an MP4/GIF from code", "animate with useCurrentFrame/interpolate/spring", "sync video to audio/beats", or "render a templated video per record headlessly". Covers building React compositions and rendering them via CLI or @remotion/renderer.
version: 0.1.0
---

# Remotion (Programmatic Video)

Build real MP4/GIF/WebM videos in React. Every frame is a pure function of `useCurrentFrame()`, so output is deterministic, scrubbable, diffable, and renderable in CI. The code-first alternative to After Effects for templated and data-driven motion graphics.

## When to use

- Render MP4/GIF/WebM from code (social clips, title cards, explainers).
- Templated/data-driven videos: one composition, many outputs from props (per-user, per-record, per-row of a CSV/DB).
- Motion graphics that must be versioned, code-reviewed, and rendered in CI without a GUI.
- Programmatic audio sync, charts that animate from data, or embedding shaders/Three.js into video.

## Core techniques

### Frame-driven animation

Animation is derived from the current frame, never from `setState` or `requestAnimationFrame`. `interpolate` maps an input range to an output range; `spring` produces physically natural motion.

```jsx
import {useCurrentFrame, useVideoConfig, interpolate, spring} from 'remotion';

export const Title = () => {
  const frame = useCurrentFrame();
  const {fps} = useVideoConfig();

  // Fade in over frames 0-30, then stay (clamp prevents over/undershoot).
  const opacity = interpolate(frame, [0, 30], [0, 1], {
    extrapolateLeft: 'clamp',
    extrapolateRight: 'clamp',
  });

  // Spring entrance; divide by spring duration is unnecessary — drive transforms directly.
  const scale = spring({frame, fps, config: {damping: 200, mass: 1, stiffness: 100}});

  return <h1 style={{opacity, transform: `scale(${scale})`}}>Hello</h1>;
};
```

Always clamp `interpolate` unless an intentional overshoot is desired — by default it extrapolates linearly past the range, which produces opacity > 1 or negative values.

### Scheduling: Sequence and Series

`<Sequence>` shifts time: children see `frame` reset to 0 at the sequence's `from`. Use it to place clips on a timeline. `<Series>` lays out back-to-back segments without manual offset math.

```jsx
import {Sequence, Series, AbsoluteFill} from 'remotion';

export const Timeline = () => (
  <AbsoluteFill style={{backgroundColor: 'black'}}>
    <Sequence from={0} durationInFrames={60}><Intro /></Sequence>
    <Sequence from={60} durationInFrames={90}><Body /></Sequence>

    {/* Series auto-sequences; offset overlaps the previous segment for crossfades */}
    <Series>
      <Series.Sequence durationInFrames={60}><ShotA /></Series.Sequence>
      <Series.Sequence durationInFrames={60} offset={-15}><ShotB /></Series.Sequence>
    </Series>
  </AbsoluteFill>
);
```

### Composition registration + parametric props

The `<Composition>` declares id, size, fps, duration, and `defaultProps`. A zod schema makes props type-safe and editable in the Studio sidebar.

```jsx
import {Composition} from 'remotion';
import {z} from 'zod';

export const schema = z.object({
  title: z.string(),
  accent: z.string(),
  fps: z.number().default(30),
});

export const Root = () => (
  <Composition
    id="Promo"
    component={Promo}
    durationInFrames={150}
    fps={30}
    width={1080}
    height={1920}
    schema={schema}
    defaultProps={{title: 'Launch', accent: '#5b8cff', fps: 30}}
  />
);
```

To make duration data-dependent, use `calculateMetadata` on the Composition to compute `durationInFrames` from props (e.g. number of rows × frames per row) before render.

### Audio and beat sync

```jsx
import {Audio, staticFile, useCurrentFrame, useVideoConfig} from 'remotion';

const BEATS_SEC = [0.5, 1.0, 1.5, 2.0]; // detected offline
export const Music = ({children}) => {
  const frame = useCurrentFrame();
  const {fps} = useVideoConfig();
  const t = frame / fps;
  const onBeat = BEATS_SEC.some((b) => Math.abs(t - b) < 1 / fps);
  return (
    <>
      <Audio src={staticFile('track.mp3')} />
      <div style={{transform: `scale(${onBeat ? 1.06 : 1})`}}>{children}</div>
    </>
  );
};
```

Detect beats offline (e.g. with `web-audio-beat-detector` or aubio) and bake the timestamps into props — never analyze audio at render time, since headless rendering has no realtime audio clock.

### Rendering

Preview in the browser-based Studio; render headlessly via CLI or the programmatic API.

```bash
npx remotion studio                                  # interactive preview
npx remotion render Promo out/promo.mp4 \
  --props='{"title":"Launch","accent":"#f43"}'        # pass parametric props
npx remotion render Promo out.gif --codec=gif         # GIF output
```

For batch/data-driven pipelines, render in Node with `@remotion/renderer` (bundle once, render many) — see the reference file.

## Deliver & verify (rendered stills → MP4)

> **Packaged helper** (`scripts/`): tile your stills with `scripts/contact-sheet.sh sheet.png f-hook.png f-mid.png f-end.png`, then assert the encode with `scripts/probe-mp4.sh out.mp4 [WxH] [fps]`. See `scripts/README.md`.

Remotion is frame-deterministic by construction — every frame is a pure function of `useCurrentFrame()`, so you can render any exact frame headlessly with **no seek harness** (this is the heavy-tier counterpart to a web scene's `?t=N`). Use this tier when the output must be an MP4/GIF, must carry exact numbers/text, or batches from data; for a lightweight web animation, deliver standalone HTML instead.

**Output contract:**
- A Remotion project with the composition registered (`<Composition>` + zod `schema` + `defaultProps`), all motion frame-driven (no timers / `Date.now()` / `Math.random()`).
- Deliverable = the rendered `out/*.mp4` (plus the project, so the user can re-render with new props/data).
- Duration data-dependent? compute it in `calculateMetadata`, not by hand.

**Verify loop — render stills → inspect → encode.** Render single frames first (cheap, no video encode), inspect them, and encode the full video only once the frames are right.

```bash
# 1. Frame-exact stills at start / mid / end (PNG, headless, fast)
npx remotion still Promo out/f-start.png --frame=0
npx remotion still Promo out/f-mid.png   --frame=75
npx remotion still Promo out/f-end.png   --frame=149   # last frame = durationInFrames - 1
#    render with the SAME props you'll ship, not just defaultProps:
#    npx remotion still Promo out/f-mid.png --frame=75 --props='{"title":"Launch"}'

# 2. Inspect each PNG — fidelity (matches brief; numbers/text correct) AND
#    artifacts (text overflow, off-canvas, clipped safe-area, missing font, wrong data binding).

# 3. Only after the stills check out, encode the video:
npx remotion render Promo out/promo.mp4 --props='{"title":"Launch"}'
```

- Use `npx remotion compositions` to read each composition's `durationInFrames`/`fps` and pick the end frame.
- **Data-driven / batch**: verify ONE representative props set via stills *before* batch-rendering all rows — catch a layout bug once instead of N times.
- **README demo GIF for free**: `npx remotion render Promo out/demo.gif --codec=gif` produces the first-screen proof clip (Direction D).

**Before you finish:**
1. `npx remotion still` renders cleanly at frame 0, mid, and last — no errors, no missing assets/fonts.
2. Numbers/text are exact and inside safe areas at every checked frame.
3. Frame-driven only — no `Date.now()` / `Math.random()` / timers (determinism holds in CI).
4. Props are zod-typed; the **shipped** props render correctly (not just `defaultProps`).
5. Full MP4 encoded and plays; (optional) GIF rendered for the README.

## Quick reference

| Need | Use |
|---|---|
| Map time → value | `interpolate(frame, [a,b], [x,y], {extrapolateRight:'clamp'})` |
| Natural easing | `spring({frame, fps, config})` |
| Place a clip at t | `<Sequence from durationInFrames>` |
| Back-to-back shots | `<Series>` |
| Audio | `<Audio src={staticFile(...)} />` |
| Frame ↔ seconds | `frame / fps` |
| Loop a value | `frame % period` then interpolate |
| Editable props | zod `schema` on `<Composition>` |
| CI render | `npx remotion render` or `@remotion/renderer` |

## Gotchas

- Never use `Math.random()`, `Date.now()`, or animation timers — they break determinism. Use `random(seed)` from Remotion for stable per-frame randomness.
- Load fonts and assets via `staticFile()` and wait with `delayRender`/`continueRender`, or fonts pop in mid-render.
- Default `interpolate` extrapolates — clamp it.
- `useCurrentFrame` inside a `<Sequence>` is local (starts at 0); use `useVideoConfig().durationInFrames` for absolute timing.

## Reference files

- `references/api-and-patterns.md` — `interpolate` options and `Easing`, spring config recipes, Sequence/Series scheduling, `<Audio>` + beat-sync, parametric `defaultProps` with zod + `calculateMetadata`, CLI flags, programmatic `@remotion/renderer` batch rendering, and embedding GLSL shaders / Three.js (`@remotion/three`).