Brimkern video pipeline — BRIK (int8)

AnimateDiff-Lightning motion modules on an SD 1.5 base, converted to the BRIK format so text→video runs inside the browser on the visitor's GPU (WebGPU). No inference server, no upload: the prompt and the frames never leave the machine.

A Lightning checkpoint fixes its own step count; only the motion module changes between them (same UNet, same CLIP). The app loads one of them: full pipeline 1.54 GB (UNet + one motion module

• CLIP + RIFE + the 4.7 MB TAESD decoder, fetched from its own repository).

Frame interpolation. AnimateDiff produces 16 unique frames (1.3 s at 12 fps). RIFE then inserts 3 frames between each pair (the last pair wraps to the first, since the clip loops): 64 frames, a 4-second clip at 16 fps. It smooths and slows the motion; it does not invent new motion. The port runs on the engine's WGSL kernels and matches the PyTorch reference to 5·10⁻⁵ (max abs error on an interpolated frame).

ByteDance describes the 1-step checkpoint as provided "for research purposes". We made it the default because waiting time decides whether anyone uses the feature: on our checks (a beach at sunset, a cat walking in a garden, 16 frames, fixed seed) it renders the same scene and motion as the 2-step one, slightly softer.

Try it

👉 https://brimkern.com/chat → Browse / load a model → the video card.

This is a pipeline, not a single-file model: several files load together, so the ?model=repo deeplink does not apply. Every file is streamed by HTTP Range, cached, and reusable offline.

⚠️ Honest cost, announced before the click: 1.54 GB to download, then about 10 s of GPU work per clip on an M4 Mac (measured, see below), longer on smaller GPUs. Clip length is adjustable from 16 to 24 generated frames (~4 to ~6 s once interpolated) (8 frames was removed: the motion modules are trained on 16, and below that the frames fall apart). A progress fraction (UNet blocks per step, then frames decoded) drives a real progress bar and a remaining-time estimate — a diffusion wait with no feedback is indistinguishable from a hang.

Measured

The video path shares the SD 1.5 UNet with the image pipeline, so it inherits its measured kernel work: the tiled int8 3×3 convolution (×1.84 on that kernel, ×1.67 end-to-end on a 256 px image). Replayable benches in scripts/e2e/.

End to end, one 16-frame 256 px clip — Mac M4 (24 GB), Chrome, production build, GPU duty 1, seed 42, same prompt, settings alternated, two runs each, two series on 2026-10-08 (the 1-step module ran in both and gave 10.0 s every time):

Frame interpolation on top (same machine and settings, alternated): 16 frames without RIFE 8.5 / 8.5 s, with RIFE ×4 (64 frames, 4 s) 10.5 / 10.0 s.

These include two engine changes of the same week, each measured against a switch that turns it off: convolutions as an implicit GEMM (8-frame clip 28 → 23 s) and a clip shown as soon as the GPU is done (the WebM encoding used to record a ~10 s loop in real time first: −9 to −10 s per clip). Other GPUs will differ; the benches are replayable.

Format

A .brik is a self-describing container: topology and quantization tiers travel inside the file, and every shard is one contiguous HTTP range — which is what makes a 1.53 GB pipeline loadable in a tab. Specification: BRIK_FORMAT.md.

License

Derived weights; the sources keep their terms:

• AnimateDiff-Lightning (ByteDance/AnimateDiff-Lightning) — creativeml-openrail-m.
• epiCRealism (emilianJR/epiCRealism, SD 1.5 fine-tune) — creativeml-openrail-m.
• RIFE 4.26 (hzwer/Practical-RIFE, weights from hzwer/RIFE) — MIT, © 2021 hzwer; converted to f16 safetensors without other change.
• TAESD decoder (madebyollin/taesd) — MIT, not mirrored here.

The Brimkern engine itself is MIT. Converting weights does not change their license.