Skip to main content

@remotion/video-mattingv4.0.523

Separate a video into an opaque base layer and a foreground layer with alpha, locally in the browser using WebGPU.

Installation​

npx remotion add @remotion/video-matting @huggingface/transformers

Example​

separate-video.ts
import {separateVideoLayers} from '@remotion/video-matting';

const file = new File([], 'input.mp4');

const {base, foreground} = await separateVideoLayers({
  src: file,
});

const baseBlob = await base.getBlob();
const foregroundBlob = await foreground.getBlob();

await Promise.all([base.dispose(), foreground.dispose()]);

Call dispose() after retrieving the blobs to release package-owned temporary storage.

The model is downloaded on the first call and cached by the browser. Call loadVideoMattingModel() first to display download progress.

Model hosting​

Model files are downloaded from immutable paths on remotion.media. We find that downloads are faster from a R2 mirror than from Hugging Face directly. The models are byte-identical to the ones you can find from Hugging Face.

APIs​

canUseVideoMatting()
Check whether a model is supported
getAvailableModels()
List models and their download sizes
isVideoMattingModelCached()
Check whether a model is downloaded
loadVideoMattingModel()
Download and initialize a model
separateVideoLayers()
Create base and foreground WebM layers
disposeVideoMattingModel()
Release model memory

Requirements​

WebGPU requires HTTPS in production or localhost during development. Use canUseVideoMatting() to check whether the selected model is supported.

If the page uses a strict Content Security Policy, add https://remotion.media to its connect-src directive, for example connect-src 'self' https://remotion.media.

The browser must also be able to decode the input video and encode VP9 with alpha through WebCodecs. separateVideoLayers() checks these media capabilities for the selected input.

License​

The package is MIT licensed. Downloaded model weights retain the licenses listed under getAvailableModels().