Skip to main content

useOffthreadVideoTexture()

available from v4.0.83

Allows you to use a video in React Three Fiber that is synchronized with Remotion's useCurrentFrame(), similar to useVideoTexture(), but uses <OffthreadVideo> instead.

This hook only works during rendering. In the Player and the Remotion Studio, use useVideoTexture() instead. See below for an example.

This hook was designed to combat limitations of the default <Video> element that is used with useVideoTexture() hook. See: <OffthreadVideo> vs <Video>

The return type of it is a THREE.Texture | null which you can assign as a map to for example meshBasicMaterial. We recommend to only render the material when the texture is not null to prevent bugs.

Example​

Simple usage (only works during rendering)
tsx
import { ThreeCanvas, useOffthreadVideoTexture } from "@remotion/three";
import { staticFile, useVideoConfig } from "remotion";
 
const videoSrc = staticFile("/vid.mp4");
 
const My3DVideo = () => {
  const { width, height } = useVideoConfig();
 
  const videoTexture = useOffthreadVideoTexture({ src: videoSrc });
 
  return (
    <ThreeCanvas width={width} height={height}>
      <mesh>
        {videoTexture ? <meshBasicMaterial map={videoTexture} /> : null}
      </mesh>
    </ThreeCanvas>
  );
};
Simple usage (only works during rendering)
tsx
import { ThreeCanvas, useOffthreadVideoTexture } from "@remotion/three";
import { staticFile, useVideoConfig } from "remotion";
 
const videoSrc = staticFile("/vid.mp4");
 
const My3DVideo = () => {
  const { width, height } = useVideoConfig();
 
  const videoTexture = useOffthreadVideoTexture({ src: videoSrc });
 
  return (
    <ThreeCanvas width={width} height={height}>
      <mesh>
        {videoTexture ? <meshBasicMaterial map={videoTexture} /> : null}
      </mesh>
    </ThreeCanvas>
  );
};
Use useVideoTexture() only during rendering
tsx
import {
  ThreeCanvas,
  useOffthreadVideoTexture,
  useVideoTexture,
} from "@remotion/three";
import { useRef } from "react";
import {
  getRemotionEnvironment,
  staticFile,
  useVideoConfig,
  Video,
} from "remotion";
 
const videoSrc = staticFile("/vid.mp4");
 
const useVideoOrOffthreadVideoTexture = (
  videoSrc: string,
  videoRef: React.RefObject<HTMLVideoElement>,
) => {
  if (getRemotionEnvironment().isRendering) {
    // eslint-disable-next-line react-hooks/rules-of-hooks
    return useOffthreadVideoTexture({ src: videoSrc });
  }
 
  // eslint-disable-next-line react-hooks/rules-of-hooks
  return useVideoTexture(videoRef);
};
 
const My3DVideo = () => {
  const { width, height } = useVideoConfig();
 
  const videoRef = useRef<HTMLVideoElement | null>(null);
  const videoTexture = useVideoOrOffthreadVideoTexture(videoSrc, videoRef);
 
  return (
    <>
      {getRemotionEnvironment().isRendering ? null : (
        <Video
          ref={videoRef}
          src={videoSrc}
          style={{ position: "absolute", opacity: 0 }}
        />
      )}
      <ThreeCanvas width={width} height={height}>
        <mesh>
          {videoTexture ? <meshBasicMaterial map={videoTexture} /> : null}
        </mesh>
      </ThreeCanvas>
    </>
  );
};
Use useVideoTexture() only during rendering
tsx
import {
  ThreeCanvas,
  useOffthreadVideoTexture,
  useVideoTexture,
} from "@remotion/three";
import { useRef } from "react";
import {
  getRemotionEnvironment,
  staticFile,
  useVideoConfig,
  Video,
} from "remotion";
 
const videoSrc = staticFile("/vid.mp4");
 
const useVideoOrOffthreadVideoTexture = (
  videoSrc: string,
  videoRef: React.RefObject<HTMLVideoElement>,
) => {
  if (getRemotionEnvironment().isRendering) {
    // eslint-disable-next-line react-hooks/rules-of-hooks
    return useOffthreadVideoTexture({ src: videoSrc });
  }
 
  // eslint-disable-next-line react-hooks/rules-of-hooks
  return useVideoTexture(videoRef);
};
 
const My3DVideo = () => {
  const { width, height } = useVideoConfig();
 
  const videoRef = useRef<HTMLVideoElement | null>(null);
  const videoTexture = useVideoOrOffthreadVideoTexture(videoSrc, videoRef);
 
  return (
    <>
      {getRemotionEnvironment().isRendering ? null : (
        <Video
          ref={videoRef}
          src={videoSrc}
          style={{ position: "absolute", opacity: 0 }}
        />
      )}
      <ThreeCanvas width={width} height={height}>
        <mesh>
          {videoTexture ? <meshBasicMaterial map={videoTexture} /> : null}
        </mesh>
      </ThreeCanvas>
    </>
  );
};

API​

Takes an object with the following properties:

src​

The video source. Can be a URL or a staticFile().

playbackRate​

The playback rate of the video. Defaults to 1.

transparent​

optional, boolean

If set to true, frames will be extracted as PNG, enabling transparency but also slowing down your render.

If set to false (default), frames will be extracted as bitmap (BMP), which is faster.

toneMappedv4.0.117​

Since Remotion 4.0.117, Remotion will adjust the colors of videos in different color spaces (such as HDR) when converting to RGB, to counteract color shifts.
Since the browser is painting in sRGB, this is necessary to ensure that the colors are displayed correctly.
This behavior is by default true and can be disabled by setting toneMapped to false.
Disabling tone mapping will speed up rendering, but may result in less vibrant colors.

Prior to Remotion 4.0.117, tone mapping was always disabled, and the toneMapped prop was not available.

Looping a texture​

Like <OffthreadVideo>, looping a video is not supported out of the box but can be achieved with the <Loop> component.

See also​