Skip to content

0021 — In-memory render: lift the render local-only gate

Status: IMPLEMENTED (2026-06-28). §6 decisions resolved per the recommendations: (1) ffmpeg materialises so the gate lifts for every project + backend; (2) materialise only when the fs is not OS-backed (no overhead for local renders); (3) explicit fs param on the seam, mirroring the generator; (4) one MR.) Date: 2026-06-28 Supersedes the constraint in: spec 0015 D1 + 0017 §4 (render is local-only). Enabled by: the afmpeg in-memory wasm backend (handover; keryx !71/!72).

Key enabler: afmpeg renders over any afero.Fs — it ships with the OS fs and an in-memory fs, and go-tool-base provides a billy→afero wrapper for the in-memory git-based worktrees the studio's remote projects use. So the afmpeg backend needs no materialisation for any project: it reads inputs from, and writes the mp4 into, whatever fs it is handed. Only the legacy shell-out ffmpeg backend (the binary can read only real OS paths) needs the materialise↔readback bridge.

1. Goal

A studio project that lives in memory (an in-memory remote / RAM worktree, workFS set — R-GIT-5) currently cannot be rendered: the studio returns "render is local-only — switch to a local checkout" (handlers_render.go isLocalProject). Now that afmpeg renders a reel entirely in memory, lift that gate so any project renders — without forcing a local checkout.

2. Why it's local-only today (the real reason)

Removing the gate alone breaks in-memory renders, because the pipeline reaches past props.FS to the OS filesystem in several places:

  • build.RenderInto reads the storyboard via props.FS (✓ afero) but renders cards to an os.MkdirTemp dir and resolves the cover with os.Stat.
  • internal/render/cards.Render(card, mediaPath, outPath, …) writes the PNG to an OS path.
  • internal/render/afmpeg stageIn does os.ReadFile; Render writes the mp4 with os.WriteFile.
  • internal/render/ffmpeg passes MediaPath/OutputPath straight to the ffmpeg binary, which can only read real OS files.

An in-memory project's files live in workFS (an afero.MemMapFs), so none of the above can find them. The generator already solved this shape — Generate(ctx, cfg, fs, req) takes the worktree fs explicitly (pkg/studio/generator.go). Render must do the same.

3. Design — thread the worktree fs through render

  1. RenderInto takes an fs afero.Fs (the worktree fs), mirroring the generator. All workspace reads/writes (storyboard, card PNGs, staging, output, cover Stat) go through fs; the temp card dir becomes a path in fs (a MemMapFs for in-memory, the OS fs for local). The studio passes workFS (or the OS fs when local); the CLI passes its OS-backed fs.
  2. cards.Render writes to fs (accept an afero.Fs / write bytes) instead of an OS path.
  3. The Renderer seam gains the fs: Render(ctx, fs, Timeline) and Probe(ctx, fs, path). The Timeline's paths become fs paths.
  4. afmpeg reads inputs from fs (its stageIn reads fs, not os) and writes the mp4 into fs — already in-memory native, so this is a small change.
  5. ffmpeg can't read a non-OS fs, so it materialises: copy the fs inputs to a temp OS dir, run the binary, copy the mp4 back into fs (the "materialise↔readback bridge" from the spike). Transparent to callers.
  6. Drop the gate. handlers_render.go no longer rejects workFS != nil; render proceeds for every project. Available() still gates on a resolvable backend.

4. Reuse / blast radius

Change Files
fs param on render core pkg/cmd/reel/build/main.go (RenderInto, renderCards, cover Stat)
fs on the seam pkg/provider/provider.go (Renderer), mocks/…/Renderer.go
afmpeg over fs internal/render/afmpeg (stageIn/Render/Probe read+write fs)
ffmpeg materialise internal/render/ffmpeg (copy fs→temp, run, copy back)
cards over fs internal/render/cards
studio wiring + gate pkg/studio/renderer.go (pass workFS), handlers_render.go (drop gate)
CLI wiring pkg/cmd/reel/build passes an OS-backed fs

5. Testing / DoD

  • afmpeg in-memory: render a whole reel where the workspace (storyboard, cards' source media, VO/music) lives in a MemMapFs and the output lands back in it — no OS files touched. Gated on KERYX_FFMPEG_WASI.
  • ffmpeg materialise: the bridge copies in/out correctly (a fake exec asserts it sees real temp paths; the mp4 is read back into fs).
  • cards over fs: a card renders to a MemMapFs.
  • studio: the render handler no longer 409s an in-memory project; a godog/handler test renders an in-memory project end-to-end (afmpeg faked).
  • Docs: update providers.md + the render concept (render is no longer local-only; afmpeg = in-memory, ffmpeg = materialised), and retire the 0015 D1 / 0017 §4 caveats.

6. Decisions to confirm before building

  1. ffmpeg on an in-memory project. Materialise transparently (rec — then the gate lifts for every project + backend) vs keep ffmpeg local-only and require afmpeg for in-memory (smaller, but the gate only half-lifts). Confirm materialise.
  2. Materialise always, or only for non-OS fs? Always-materialise is uniform but adds a redundant copy to the common local-ffmpeg path; OS-fs-skip avoids it but needs an fs-type check. Rec: skip when the fs is OS-backed (detect once), materialise otherwise — no overhead for today's local renders.
  3. Seam shape. Render(ctx, fs, Timeline) + Probe(ctx, fs, path) (rec — explicit, mirrors the generator) vs the Timeline carrying the fs. Confirm the explicit param.
  4. Scope/phasing. Land the fs-threading + afmpeg-in-memory + ffmpeg-materialise + gate removal together (rec — so the gate can truly go) vs afmpeg-only first with ffmpeg materialise as a fast-follow. Confirm one MR or two.