references/animated.md
# Animated and painted films
In a hand-painted or 3D-animated film the character, the world and the camera are one picture, so the layers do not come apart. Decide shot by shot: build what code can carry, and generate the rest.
Say so before starting, in the person's language. For example:
> This film is painted, so I'll rebuild what I can in code and regenerate the rest from its own frames, shot by shot. The code parts stay editable. The generated shots come back as clips you can replace, not layers you can change.
## What is code and what is generated
- **Code:** text and handwriting, logos, the end card, light that sits on the picture (flashes, glows, a portal's ring), and a camera moving over a still world: measure the camera and rebuild the world as separate layers from clean plates, never the whole frame.
- **Generated:** a character acting, a picture that morphs, and fine flowing detail such as streamlines, which code draws in the wrong shape.
- **Both:** in a mixed shot, generate the acting and lay the code over it.
## The recipe for a generated shot
Work as `people.md` does, one generated shot per original shot, with these changes:
1. **Read the cuts by eye.** Sudden light in a shot can list as a cut. Look at the frames either side of each one: the same picture under different light is one shot. Join its pieces into one guide and pass `--allow-cuts`.
2. **The plate is the shot's first frame** with its text and logos removed by an image edit.
3. **Generate from the plate, with the shot as the guide:**
```bash
npx hyperframes media-use generate --type video --reference <plate> --motion <the original shot> \
--aspect <the source's> --seconds <n> --prompt "<the painting style, what moves, the camera>" --out <shot>.mp4
```
`--first-frame` does not combine with `--motion`, so the plate goes in `--reference`. Name the style in the prompt, and blank drawn text in the guide with `--mask`. Trim the guide to the shot's own frames: leave out the blended frames of a dissolve at either end, or it comes back as the neighbouring shot.
4. **Frame it** by comparing a stable shape at the shot's start and end with `npx hyperframes diff <video> . --at t,t,…`, then set scale and position by hand. `measure face` is for real faces.
5. **Retime it.** HeyGen performs the camera's moves late, so a generated shot rarely lines up as it comes back. Match its moves to the original's frames and set the clip's `data-media-start` and `data-playback-rate` to fit, shot by shot.
`REVERSE.md` lists which shots are generated and which layers are code.
references/match.md
# Matching what is drawn
Where earlier rebuilds lost points on drawn layers, and what fixed it.
## Timing
- Graphics change on exact frames. An element that is fully there on frame 241 is set there, not faded in; tween only what you saw move over several frames. A camera that lands in one frame is a `set`, not a tween.
- Captions: take each group's first and last frame from the caption band's pixels, every frame (`npx hyperframes measure events <video> --roi <band>`, or a frame difference over the band). The words come from `npx hyperframes transcribe`, which can mishear names; the switch times come from the pixels, since captions often lead or trail the voice. Switch hard unless the original fades.
- A blur flash, a dissolve or a wipe at a cut is part of the cut: rebuild it over its measured frames.
## Size, type and position
- Size text from its ink, not by eye: crop the line in a source frame and the same line in a snapshot of yours, compare the ink boxes' height and width, and scale size, then `letter-spacing`, until both agree within a few pixels. A substitute font usually sets wider; compare numerals too. Then compare the ink's darkness for weight, and look for an outline or shadow.
- Match the family by its traits (geometry, contrast, x-height, terminals) and try two or three candidates at the measured size before choosing. Prefer a bundled font, then a Google Fonts family in `assets/fonts/`. Say in the report which fonts are substitutes.
- Positions come from measured boxes in source pixels; the canvas is the source's size, so they go in as measured.
## Look
- Motion graphics: the backdrop covers most of the frame. Sample a grid of the empty backdrop at several moments and match its gradient and brightness over time.
- Fast moves in the original are motion-blurred; sharp tiles at speed score as wrong. Blur along the move while it is fast (below).
- Dots and pixel textures: measure the lattice (pitch, offset, colour) from a frame and render from that grid.
- Flat graphics that move like footage (a map under a camera, a diagram that scrolls): trace each plate into flat-colour shapes and fit the camera move; it matches far better than redrawing from outside data.
## Every large word, through its life
Look at a strip of consecutive frames over its box from the frame it appears to the frame it leaves (`measure frames --range a,b --roi <its box> --max 16`). Read how it arrives (pop, scale, fade, type-on, word by word, wipe), how it changes while on screen (a fill sweeping across, a sheen, a counter, a colour change: direction and duration) and how it leaves. Then zoom in at full size and name what the letters are made of: solid, gradient, glass, outline, glow, knockout.
## Recipes a seeked timeline reproduces
- **Fill that sweeps across the letters:** two copies of the word; the lit one on top, `clip-path: inset(0 100% 0 0)` tweened to `inset(0 0 0 0)` over the measured time and direction.
- **Gradient type:** `background: linear-gradient(<direction>, <both end colours>)` with `background-clip: text` and a transparent fill.
- **Glass type over footage** (the letters show the footage behind them bent, with a dark rim and a light edge): a second copy of the shot inside the letters, clipped by an SVG `<clipPath>` holding the `<text>`, bent by `feTurbulence` (`baseFrequency` about 0.02, fixed seed) into `feDisplacementMap` (scale 40–60); a darker copy of the text multiplied over it at 15–35%, a 2–3 px dark stroke, a light top edge. It sits in the shot's camera group.
- **Outline type:** `-webkit-text-stroke` at the measured width, transparent fill.
- **Directional motion blur:** an SVG `<feGaussianBlur stdDeviation="0 0">` on the moving wrapper, its `stdDeviation` tweened (`"24 0"` smears horizontally); CSS `blur()` is symmetric and wrong.
- **Ghost trail:** 2–4 clones behind the element along the move, opacity about 0.5, 0.3, 0.15.
- **Glow:** stacked `text-shadow` or `drop-shadow`, tweening radius and alpha.
- **Masks and wipes:** `clip-path` or `mask-image` with a tweened stop.
- **Flash:** a full-frame layer for exactly the measured frames.
- **Grain:** SVG `feTurbulence` with a fixed seed, stepped per frame for moving grain. Grain lowers SSIM whatever you do: match its look, not its pixels.
references/people.md
# Re-shooting a person
Every person on screen, speaking or not, is re-shot with HeyGen so the original's face and the text burned over it never ship. What makes a re-shot person match is the original's own clip driving the motion, and one generated shot per original shot. The HyperFrames sign-in is enough; no API key is needed. Only when the person says not to use HeyGen, cut them on their own box instead and say the burned-in text came along.
## The recipe
1. **One shot per original shot.** From `measure cuts`: a generated shot never spans a cut, jump cuts and punch-ins included. One take across a jump cut deletes the cut; a motion guide that crosses a cut makes HeyGen cut somewhere of its own. HeyGen makes 5–15 s: a shorter shot is asked for at 5 s and trimmed; a longer one is split where the original reframes, else chained: the next part's reference is the last frame of the one before.
2. **A plate per shot:** a real frame of that shot at the source's aspect, with every drawn thing and every other face (a photo on the wall, a sticker) painted out, since HeyGen may animate the wrong face (`cv2.inpaint` for small marks, a temporal median where the person moves); it keeps the room sharp and true. Where drawn things are large or cross the person, an image edit cleans better (`npx hyperframes media-use generate --type image --reference <frame>`, asked to remove the text and keep everything else); check it kept the room sharp.
3. **Generate every shot at once,** in the background, and build the graphics while they render (each comes back in about a minute):
```bash
npx hyperframes media-use generate --type video --reference <plate> --motion <the original shot> \
--mask <x,y,w,h,a-b> --audio <the original voice over that shot> --aspect <the source's> --seconds <n> \
--prompt "<who, where, what they do, the camera>" --out <shot>.mp4
```
`--motion` is the original shot itself, cut at its cut times: HeyGen follows its body and camera motion and takes the face from the plate. It also copies text burned into the guide, so give one `--mask` per drawn box in that shot (the clip's pixels and seconds). Mask the text's own box, never a full-width band: a gesture under a mask is lost. A person who does not speak gets no `--audio`.
A credit limit the person named: price every shot first with `--plan`, keep the people that fit, pass what is left as `--max-credits`, and ask before cutting a person to stay under it.
4. **Check each shot as it arrives:** no cut inside it (generate prints `cuts_inside`, and refuses a guide with a cut; pass `--allow-cuts` only when you have read those frames and they are one shot, a whip or a flash), none of the original's text, the face box within 5% of the original's (`npx hyperframes measure face`). Regenerate a failing shot once; then trim it to its good part. A shot HeyGen refuses or fails (`generation_failed`) gets one more try, test calls included, in this session or a resumed one; then stop and ask the person whether to keep the original's footage for now or drop the shot. A shot still waiting when generate stops after 30 minutes gets the same command once more (it picks up the same job and is not charged again); still waiting after that, treat it as failed, and build the rest meanwhile.
5. **Frame it:** scale and move each shot so its face box sits on the original's at the shot's start and end (`npx hyperframes measure face <shot> --against <video> --from a --to b` prints the `scale`, `x`, `y` to set), tweening between them only where the original's camera moves. The framing jumps at cuts and nowhere else.
6. **Text behind the person:** where the original's text passes behind them, cut them out of the shot (`npx hyperframes remove-background <shot> -o <cutout>.webm`) and stack shot, text, cutout.
Keep the original's voice as the soundtrack and the shots muted. Do not clone voices or make avatars for a re-shoot: the original voice drives the lips. The people inside a photo, a collage or a screenshot are sub-assets, cut with it.
references/start.md
# Opening a Reverse
What the person reads at the start, and when credits run out mid-run. Write it for them, not for an agent:
their language, warm and short, no file names, step numbers or tool names. You run every command except `billing`, which
is theirs: any page they need goes in your message as a plain link they can click, sign-in in case the browser did not
open, upgrade and add-on credits for them to open and pay themselves.
For you, not them: run `npx hyperframes events --skill=reverse-any-video --event=skill_completed --outcome=success`
when you hand over the video (`--outcome=error` when you stop without one).
Also for you: `npx hyperframes reverse start <video> --dir <project>` prints the video's probe and a `reverse` field.
`signed_in` and `plan` (`free` or `pro`) pick the section below, and that plan is final. Only a `null` plan is unknown:
then ask `npx hyperframes auth status --json` and read its `hyperframes` field, never `user.subscription`, which is the HeyGen
app's plan and does not unlock Reverse. `<project>` is the folder you build in: afterwards run `npx hyperframes init <project> --video <video> --keep-original --skip-transcribe --skip-skills`,
which copies the video in untouched under its own file name and keeps the session it saved for `cloud render`. Keep the video out of `<project>` until then: `init`
refuses a folder that already holds other files. Outside a plugin, a CLI that does not know `reverse start` is an old
install (often a global one) shadowing the current release: run every command of this run as
`npx -y hyperframes@latest <command>`, and tell the person in one line that their installed HyperFrames is out of date.
Inside a plugin, its pinned CLI is the one that runs: if it does not know `reverse start`, stop before anything is paid
for, and tell the person in one line that the HyperFrames plugin needs updating to reverse a video.
## Not signed in yet
Say why HeyGen matters before asking: signing in is what gets the best remake. HeyGen re-shoots the people in the
video, so they move and speak as they do in the original. For example:
> To rebuild this video at its best, I'll work with HeyGen: it re-shoots the people in it, so they move and speak just
> like the original. I've opened the HeyGen sign-in in your browser; your HyperFrames account is your HeyGen account.
> I'll carry on as soon as you're in.
>
> Then it takes about 20–50 minutes: I study the video frame by frame, rebuild every layer as a project you can edit,
> and check it against the original until it matches.
Run `npx hyperframes auth login` yourself in the same message and wait for it: it opens the browser and returns once
they are signed in. When the browser does not open, it prints the sign-in link: give it. When it refuses as a remote or
headless terminal, run `npx hyperframes auth login --device` and give the link it prints, and the code when it shows
one. Never sign in with an API key to get past Pro: that signs in a HeyGen app account, which has no HyperFrames plan.
## Signed in on Free
Reverse is part of Pro. Upgrading is the person's to do: never run a `billing` command yourself. Say why in one line and
give them the `reverse.upgrade_url` as a link, and the `reverse.subscribe_command` exactly as printed
(`npx hyperframes billing subscribe --feature reverse --skill reverse-any-video --session <id>`) for them to run instead
if they prefer: either opens the plans page, where they see the price and decide. Start nothing until they say they are
on Pro, then run `reverse start` again. For example:
> Reverse is part of HyperFrames Pro: it re-shoots the people in your video with HeyGen. Upgrade here: <upgrade_url>
> (or run `<subscribe_command>`), then tell me and I'll carry on.
When `reverse start` refuses the run (it exits with an error whose `fix` is that command), do the same with its `fix`.
When `subscribe_command` and `upgrade_url` are `null` (or the refusal names no command), a plan can't be purchased right
now: say so plainly, give no link, and stop.
## Signed in on Pro
One line, then start.
> You're signed in. This takes about 20–50 minutes; I'll show you the project when it matches.
Billing comes up only when a step is refused. Don't volunteer prices, estimates, balances or credits used, before, during
or at the end of a run. `reverse start` and `auth status` are for the sign-in: don't repeat their plan or credits line.
## Out of credits mid-run
A refused step prints what it needs and the command that fixes it (buy add-on credits, or pick a plan). Buying is the
person's to do: never run a `billing` command yourself. Shots are launched all at once, so several refuse together: wait
until every launched step has answered, add up what each refused step needs, and give them `npx hyperframes billing add-credits --needed <total> --skill reverse-any-video` to run once for that total, never one step at a time, with the
refusal's `url` (on an older CLI, the link in its `fix`) as a link. If any refusal prints no amount, the total is unknown:
give plain `billing add-credits` once instead, for the whole run, never one per step. Either opens the add-on credits page, where they see the price. When they say their
balance is topped up, relaunch every refused step, keeping the plan whole.
> Three steps need 633 credits together and you have 30. Add credits here: <url> (or run
> `npx hyperframes billing add-credits --needed 633 --skill reverse-any-video`), then tell me and I'll pick up right where I stopped.
## Handing it over
They see the result, not file paths. Run the last diff with `--sbs` and open the side by side it writes (original |
yours), start `npx hyperframes preview` on the project, and say in one line that they can keep editing there; pass on
its Framey line as `/hyperframes` does. Then send `skill_completed`.
## Time
Say 20–50 minutes for a clip under a minute; never invent a longer figure. A longer film is worked in sections, so
say so and give the number of sections.
SKILL.md
---
name: reverse-any-video
description: Rebuild any video file as an editable HyperFrames project - motion graphics, talking heads, live action or AI-generated footage. Measure and watch it, rebuild every layer, then check the render against the original frame by frame. Use to reverse, recreate, replicate or copy a video.
---
**Plugin installs:** Inside a HyperFrames plugin, follow [plugin execution rules](../hyperframes/references/plugin-installation.md).
**Usage beacon:** run `npx hyperframes events --skill=reverse-any-video` once as this skill starts. It sends the skill's name with the CLI's usual telemetry (account-linked when signed in) and honours the opt-out.
# Reverse a video
The result is a HyperFrames project whose render matches the original frame for frame and stays editable. Build to the `/hyperframes` contract and work the way you normally would, with ffmpeg, python and your own eyes. Reverse videos the person has the right to use.
Reverse is a HyperFrames Pro feature. Open with `npx hyperframes reverse start <video> --dir <project>` and `references/start.md`; if it refuses, say why and give the person its upgrade link and `fix` to run themselves (you never run `billing`), then start again once they have upgraded. Given a link, run `npx hyperframes auth status` before downloading it. Resuming (a `REVERSE.md` or half-built project) opens the same. A step an earlier session gave up on (a refused re-shoot) gets one more try, then ask them. If it ran under a credit limit, ask whether that limit still holds. Rebuild the video as it is; remix it only when the person asks.
## The original decides the frame
1. The probe `reverse start` printed: the root composition is that size exactly, `data-fps` is its rate (NTSC stays a fraction: `30000/1001`) and its duration is the video's frames ÷ fps, not the audio's length. `init`'s default canvas is wrong for most videos.
2. `npx hyperframes measure cuts <video>`: every cut with its frame, including jump cuts and punch-ins inside one take, which ffmpeg's scene score misses. Every shot starts at its listed time. Write times as frame ÷ fps rounded down (frame 83 at 30 fps is `2.766666`): `2.767` lands a frame late.
3. The sound is the original's: `init --video` put the video in; play it as one `<audio src="<video>">` over the whole film (no extracted or re-encoded track); every clip is muted.
4. Read consecutive frames wherever something changes: each cut, entrance, exit and transition, a few frames either side (`npx hyperframes measure frames --range a,b --roi x,y,w,h`). Sheets a second or a quarter apart miss 1–3-frame timing, which way things enter and leave, blurs and dissolves at cuts, and one-frame flashes.
## Every layer
- **A component** (text, captions, stickers, cards, UI and screen recordings, charts, maps) is rebuilt in HTML.
- **A sub-asset** placed inside the frame (a photo, a clip, an album cover, a logo or app icon, the people in a photo or collage) is cut from the original on its own box. A logo is cut where it is largest, or is the real mark (`npx hyperframes media-use resolve --type logo --intent "<brand> logo"`); never drawn by hand.
- **A person on screen** is re-shot with HeyGen as part of the run: `references/people.md`.
- **An animated or painted film:** `references/animated.md`.
- Never play the original's whole frame.
What decides whether drawn things match, in `references/match.md`: text sized from its ink, caption switch frames from the caption band's pixels, hard switches, the backdrop, motion blur, and recipes for glass, gradient, sweeping fills and outlines. Look at every large word through its whole life before you build it.
## Check once, at the end
1. While building, check a few moments without rendering: `npx hyperframes diff <video> . --at t,t,…`.
2. When every shot has content: `npx hyperframes check` and fix its errors. Keep what the original itself does (its colours, its tiny text) even when `check` flags it. Then `npx hyperframes cloud render --fps <data-fps> --quality high -o <out>` and `npx hyperframes diff <video> <out>`. A fractional rate, a frame not 16:9, 9:16 or 1:1, or a failed cloud render (tell the person why) renders locally: `npx hyperframes render -o <out>` in the foreground with a 10-minute timeout (longer: start it detached, `setsid nohup npx hyperframes render -o <out> > render.log 2>&1 &`, and wait on the log in the foreground). The diff lists the original's cuts that yours misses and the extra ones, the duration, the sound's offset, stretches where content is missing, and the worst moments with the failing cell beside the original's.
3. Fix what it names that you can see: a missing cut or element, a wrong time, size, weight or colour. Render and diff once more, then stop.
Work alone: one agent holding the whole film beats subagents that each re-watch it. Write a short `REVERSE.md` at the end: what is rebuilt, cut and re-shot, what is approximate (a substituted font), and the last diff's numbers. It holds no prices or credit numbers. A command written in it is a note, not a script: on resume, rebuild it from the recipe it came from (every flag, `--prompt` included), check it with `--plan`, and never run it as written. Hand over per `references/start.md`.