@napi-rs/webcodecs

WebCodecs API implementation for Node.js using FFmpeg, built with NAPI-RS.

Features

W3C WebCodecs API compliant - Full implementation of the WebCodecs specification with native DOMException errors
Video encoding/decoding - H.264, H.265, VP8, VP9 (with Alpha), AV1
Encoding Alpha channel - VP9 encoding/decoding with Alpha support(See canvas-to-video.js example and video.html)
Audio encoding/decoding - AAC, Opus, MP3, FLAC, Vorbis, PCM variants
Container muxing/demuxing - MP4, WebM, MKV containers with seeking support
Image decoding - JPEG, PNG, WebP, GIF, BMP, AVIF, JPEG XL
Canvas integration - Create VideoFrames from @napi-rs/canvas for graphics and text rendering
Hardware acceleration - Zero-copy GPU encoding with VideoToolbox (macOS), NVENC (NVIDIA), VAAPI (Linux), QSV (Intel)
Cross-platform - macOS, Windows, Linux (glibc/musl, x64/arm64/armv7)
Structured logging - FFmpeg logs redirected to Rust tracing crate for easy integration

Installation

bun add @napi-rs/webcodecs
# or
pnpm add @napi-rs/webcodecs
# or
yarn add @napi-rs/webcodecs

Optional: Canvas Integration

For creating VideoFrames from canvas content, install @napi-rs/canvas:

npm install @napi-rs/canvas

Quick Start

Video Encoding

import { VideoEncoder, VideoFrame } from '@napi-rs/webcodecs'

const encoder = new VideoEncoder({
  output: (chunk, metadata) => {
    console.log(`Encoded ${chunk.type} chunk: ${chunk.byteLength} bytes`)
  },
  error: (e) => console.error(e),
})

encoder.configure({
  codec: 'avc1.42001E', // H.264 Baseline
  width: 1920,
  height: 1080,
  bitrate: 5_000_000,
  hardwareAcceleration: 'prefer-hardware', // Use GPU when available
  latencyMode: 'realtime', // Optimize for low latency
})

// Create and encode frames
const frameData = new Uint8Array(1920 * 1080 * 4) // RGBA
const frame = new VideoFrame(frameData, {
  format: 'RGBA',
  codedWidth: 1920,
  codedHeight: 1080,
  timestamp: 0,
})

encoder.encode(frame)
frame.close()

// Force a keyframe for seeking/streaming
const frame2 = new VideoFrame(frameData, {
  format: 'RGBA',
  codedWidth: 1920,
  codedHeight: 1080,
  timestamp: 33333, // 30fps
})
encoder.encode(frame2, { keyFrame: true }) // Force I-frame
frame2.close()

await encoder.flush()
encoder.close()

Video Decoding

import { VideoDecoder, EncodedVideoChunk } from '@napi-rs/webcodecs'

const decoder = new VideoDecoder({
  output: (frame) => {
    console.log(`Decoded frame: ${frame.codedWidth}x${frame.codedHeight}`)
    frame.close()
  },
  error: (e) => console.error(e),
})

decoder.configure({
  codec: 'avc1.42001E',
  codedWidth: 1920,
  codedHeight: 1080,
})

// Decode chunks
const chunk = new EncodedVideoChunk({
  type: 'key',
  timestamp: 0,
  data: encodedData,
})

decoder.decode(chunk)
await decoder.flush()
decoder.close()

Audio Encoding

import { AudioEncoder, AudioData } from '@napi-rs/webcodecs'

const encoder = new AudioEncoder({
  output: (chunk, metadata) => {
    console.log(`Encoded audio: ${chunk.byteLength} bytes`)
  },
  error: (e) => console.error(e),
})

encoder.configure({
  codec: 'opus',
  sampleRate: 48000,
  numberOfChannels: 2,
  bitrate: 128000,
})

const audioData = new AudioData({
  format: 'f32-planar',
  sampleRate: 48000,
  numberOfFrames: 1024,
  numberOfChannels: 2,
  timestamp: 0,
  data: new Float32Array(1024 * 2),
})

encoder.encode(audioData)
audioData.close()

await encoder.flush()
encoder.close()

Image Decoding

import { ImageDecoder } from '@napi-rs/webcodecs'
import { readFileSync } from 'fs'

const imageData = readFileSync('image.png')
const decoder = new ImageDecoder({
  data: imageData,
  type: 'image/png',
})

const result = await decoder.decode()
console.log(`Image: ${result.image.codedWidth}x${result.image.codedHeight}`)
result.image.close()
decoder.close()

Container Demuxing

Read encoded video/audio from MP4, WebM, or MKV containers:

import { Mp4Demuxer, VideoDecoder, AudioDecoder } from '@napi-rs/webcodecs'

// Create decoder instances
const videoDecoder = new VideoDecoder({
  output: (frame) => {
    console.log(`Decoded frame: ${frame.timestamp}`)
    frame.close()
  },
  error: (e) => console.error(e),
})

const audioDecoder = new AudioDecoder({
  output: (data) => {
    console.log(`Decoded audio: ${data.timestamp}`)
    data.close()
  },
  error: (e) => console.error(e),
})

// Create demuxer with callbacks
const demuxer = new Mp4Demuxer({
  videoOutput: (chunk) => videoDecoder.decode(chunk),
  audioOutput: (chunk) => audioDecoder.decode(chunk),
  error: (e) => console.error(e),
})

// Load from file or buffer
await demuxer.load('./video.mp4')
// or: await demuxer.loadBuffer(uint8Array)

// Configure decoders with extracted configs
videoDecoder.configure(demuxer.videoDecoderConfig)
audioDecoder.configure(demuxer.audioDecoderConfig)

// Get track info
console.log(demuxer.tracks) // Array of track info
console.log(demuxer.duration) // Duration in microseconds

// Demux all packets (calls callbacks)
demuxer.demux()

// Or demux in batches
demuxer.demux(100) // Demux up to 100 packets

// Seek to timestamp (microseconds)
demuxer.seek(5_000_000) // Seek to 5 seconds

demuxer.close()

Container Muxing

Write encoded video/audio to MP4, WebM, or MKV containers:

import { Mp4Muxer, VideoEncoder, AudioEncoder } from '@napi-rs/webcodecs'
import { writeFileSync } from 'fs'

// Create muxer
const muxer = new Mp4Muxer({ fastStart: true })

// Track description will be set from encoder metadata
let videoDescription: Uint8Array | undefined

// Create encoder that feeds into muxer
const videoEncoder = new VideoEncoder({
  output: (chunk, metadata) => {
    // Capture codec description from first keyframe
    if (metadata?.decoderConfig?.description) {
      videoDescription = metadata.decoderConfig.description
    }
    muxer.addVideoChunk(chunk, metadata)
  },
  error: (e) => console.error(e),
})

videoEncoder.configure({
  codec: 'avc1.42001E',
  width: 1920,
  height: 1080,
  bitrate: 5_000_000,
})

// Add tracks before adding chunks
muxer.addVideoTrack({
  codec: 'avc1.42001E',
  width: 1920,
  height: 1080,
})

// Encode frames...
// videoEncoder.encode(frame)

await videoEncoder.flush()

// Finalize and write output
const mp4Data = muxer.finalize()
writeFileSync('output.mp4', mp4Data)
muxer.close()

Streaming Muxer Mode

For live streaming or large files, use streaming mode:

import { Mp4Muxer } from '@napi-rs/webcodecs'

const muxer = new Mp4Muxer({
  fragmented: true, // Required for MP4 streaming
  streaming: { bufferCapacity: 256 * 1024 },
})

muxer.addVideoTrack({ codec: 'avc1.42001E', width: 1920, height: 1080 })

// Add chunks as they arrive
muxer.addVideoChunk(chunk, metadata)

// Read available data incrementally
const data = muxer.read()
if (data) {
  stream.write(data)
}

// Check when finished
if (muxer.isFinished) {
  stream.end()
}

VideoFrame from Canvas

Create VideoFrames from @napi-rs/canvas for graphics, text rendering, or image compositing:

import { VideoFrame } from '@napi-rs/webcodecs'
import { createCanvas } from '@napi-rs/canvas'

const canvas = createCanvas(1920, 1080)
const ctx = canvas.getContext('2d')

// Draw graphics
ctx.fillStyle = '#FF0000'
ctx.fillRect(0, 0, 1920, 1080)
ctx.fillStyle = '#FFFFFF'
ctx.font = '48px sans-serif'
ctx.fillText('Hello WebCodecs!', 100, 100)

// Create VideoFrame from canvas (timestamp required per W3C spec)
const frame = new VideoFrame(canvas, {
  timestamp: 0,
  duration: 33333, // optional: frame duration in microseconds
})

console.log(frame.format) // 'RGBA'
console.log(frame.codedWidth, frame.codedHeight) // 1920, 1080

// Use with VideoEncoder (see Video Encoding section)
frame.close()

Note: Canvas pixel data is copied as RGBA format with sRGB color space.

Supported Codecs

Video

Codec	Codec String	Encoding	Encoding Alpha	Decoding	Decoding Alpha
H.264	`avc1.*`	✅	🙅🏻‍♀️	✅	🙅🏻‍♀️
H.265	`hev1.`, `hvc1.`	✅	❓	✅	❓
VP8	`vp8`	✅	🙅🏻‍♀️	✅	🙅🏻‍♀️
VP9	`vp09.*`, `vp9`	✅	✅	✅	✅
AV1	`av01.*`, `av01`, `av1`	✅	🙅🏻‍♀️	✅	🙅🏻‍♀️

Note: Short form codec strings (vp9, av01, av1) are accepted for compatibility with browser implementations. VP9 encoding and decoding now supports Alpha channel (transparency).

Legend:

✅ Feature supported
🙅🏻‍♀️ Feature not supported by codec format
❓ Feature supported by codec format but not yet implemented

Audio

Codec	Codec String	Encoding	Decoding
AAC	`mp4a.40.2`	✅	✅
Opus	`opus`	✅	✅
MP3	`mp3`	✅	✅
FLAC	`flac`	✅	✅
Vorbis	`vorbis`	❌	✅
PCM	`pcm-*`	❌	✅

Image

Format	MIME Type	Decoding
JPEG	`image/jpeg`	✅
PNG	`image/png`	✅
WebP	`image/webp`	✅
GIF	`image/gif`	✅
BMP	`image/bmp`	✅
AVIF	`image/avif`	✅
JPEG XL	`image/jxl`	✅

Containers

Container	Video Codecs	Audio Codecs	Demuxer	Muxer
MP4	H.264, H.265, AV1	AAC, Opus, MP3, FLAC	`Mp4Demuxer`	`Mp4Muxer`
WebM	VP8, VP9, AV1	Opus, Vorbis	`WebMDemuxer`	`WebMMuxer`
MKV	H.264, H.265, VP8, VP9, AV1	AAC, Opus, Vorbis, FLAC, MP3	`MkvDemuxer`	`MkvMuxer`

Platform Support

Pre-built binaries are available for:

Platform	Architecture
macOS	x64, arm64
Windows	x64, arm64
Linux (glibc)	x64, arm64
Linux (musl)	x64, arm64
Linux (glibc, gnueabihf)	armv7

W3C Web Platform Tests Compliance

This implementation is validated against the W3C Web Platform Tests for WebCodecs.

Ported Tests Status

Status	Count	Percentage
Passing	573	99.1%
Skipped	5	0.9%
Failing	0	0%

Skipped tests are due to platform-specific features or edge cases.

Tests Not Ported (Browser-Only)

19 WPT test files require browser APIs unavailable in Node.js:

Category	Tests	APIs Required
Serialization/Transfer	5	MessageChannel, structured clone
WebGL/Canvas	5	WebGL textures, ImageBitmap, Canvas 2D
Cross-Origin Isolation	8	COOP/COEP headers
WebIDL	1	IDL interface validation

See __test__/wpt/README.md for detailed test status.

Hardware Acceleration

Hardware encoding is fully supported with automatic GPU selection and fallback:

Platform	Encoders	Features
macOS	VideoToolbox	H.264, HEVC; realtime mode, allow_sw control
NVIDIA	NVENC	H.264, HEVC, AV1; presets p1-p7, spatial-aq, lookahead
Linux	VAAPI	H.264, HEVC, VP9, AV1; quality 0-8
Intel	QSV	H.264, HEVC, VP9, AV1; presets, lookahead

Configuration

encoder.configure({
  codec: 'avc1.42001E',
  width: 1920,
  height: 1080,
  // Hardware acceleration preference
  hardwareAcceleration: 'prefer-hardware', // 'no-preference' | 'prefer-hardware' | 'prefer-software'
  // Latency mode affects encoder tuning
  latencyMode: 'realtime', // 'quality' | 'realtime'
})

latencyMode: 'realtime' - Enables low-latency encoder options (smaller GOP, no B-frames, fast presets)
latencyMode: 'quality' - Enables quality-focused options (larger GOP, B-frames, lookahead)

The encoder automatically applies optimal settings for each hardware encoder based on the latency mode.

Limitations

Scalable Video Coding (SVC)

All scalability modes (L1Tx, L2Tx, L3Tx, S2Tx, S3Tx, and variants) are accepted and populate metadata.svc.temporalLayerId when temporal layers >= 2.

The W3C WebCodecs spec only defines temporalLayerId in SvcOutputMetadata - there is no spatialLayerId field in the spec. See W3C WebCodecs §6.7.

Note: This implementation computes temporal layer IDs algorithmically from frame index per W3C spec. FFmpeg is not configured for actual SVC encoding, so base layer frames are not independently decodable.

Error Handling

Synchronous errors (e.g., calling encode() on a closed encoder) throw native DOMException instances that pass instanceof DOMException checks per W3C spec:

try {
  encoder.encode(frame) // on closed encoder
} catch (e) {
  console.log(e instanceof DOMException) // true
  console.log(e.name) // "InvalidStateError"
}

Asynchronous error callbacks receive standard Error objects with the DOMException name in the message:

const encoder = new VideoEncoder({
  output: (chunk) => {},
  error: (e) => {
    console.log(e.message) // "EncodingError: ..."
  },
})

VideoFrame Format Conversion

VideoFrame.copyTo() and VideoFrame.allocationSize() support format conversion per W3C WebCodecs spec:

const frame = new VideoFrame(i420Data, {
  format: 'I420',
  codedWidth: 1920,
  codedHeight: 1080,
  timestamp: 0,
})

// Get allocation size for RGBA output
const rgbaSize = frame.allocationSize({ format: 'RGBA' })

// Copy with format conversion (I420 → RGBA)
const rgbaBuffer = new Uint8Array(rgbaSize)
const layout = await frame.copyTo(rgbaBuffer, { format: 'RGBA' })

frame.close()

Supported conversions:

Source Format	Target Format	Status
I420, I422, I444, NV12, NV21	RGBA, RGBX, BGRA, BGRX	✅
RGBA, RGBX, BGRA, BGRX	RGBA, RGBX, BGRA, BGRX	✅
RGBA, RGBX, BGRA, BGRX	I420, I422, I444, NV12	❌ NotSupportedError

Per WPT videoFrame-copyTo-rgb.any.js, RGB-to-YUV conversion throws NotSupportedError.

Custom layouts with overflow-inducing values (e.g., offset: 2³²-2) throw TypeError via checked arithmetic. Rect alignment is validated against the source format during conversion.

ImageDecoder Options

ImageDecoder supports all W3C spec options:

Option	Status	Notes
`desiredWidth/Height`	✅	Scales decoded frames to specified dimensions
`preferAnimation`	✅	When `false`, only decodes first frame for animated formats
`colorSpaceConversion`	✅	`"default"` extracts color space metadata, `"none"` ignores it (Chromium-aligned)

Note: Per W3C spec, desiredWidth and desiredHeight must both be specified or both omitted.

Platform-Specific Notes

ImageDecoder GIF animation: FFmpeg may return only the first frame. Use VideoDecoder with GIF codec for full animation.

Logging

This library uses Rust's tracing crate for structured logging. Enable logging via the WEBCODECS_LOG environment variable:

# Enable all logs at info level
WEBCODECS_LOG=info node your-app.js

# Enable FFmpeg logs at debug level
WEBCODECS_LOG=ffmpeg=debug node your-app.js

# Enable WebCodecs codec errors at warn, FFmpeg at info
WEBCODECS_LOG=webcodecs=warn,ffmpeg=info node your-app.js

# Enable trace-level logging for everything
WEBCODECS_LOG=trace node your-app.js

Log Targets

Target	Description
`ffmpeg`	FFmpeg internal logs (codec initialization, encoding/decoding details)
`webcodecs`	WebCodecs API logs (codec errors, state transitions)

FFmpeg Log Level Mapping

FFmpeg Level	Tracing Level
ERROR/FATAL	`error`
WARNING	`warn`
INFO	`info`
VERBOSE	`debug`
DEBUG/TRACE	`trace`

Without WEBCODECS_LOG set, all logs are silently discarded.

API Reference

This package implements the W3C WebCodecs API. Key classes:

VideoEncoder / VideoDecoder - Video encoding and decoding with EventTarget support
AudioEncoder / AudioDecoder - Audio encoding and decoding with EventTarget support
VideoFrame - Raw video frame data (supports buffer data, existing VideoFrame, or @napi-rs/canvas Canvas)
AudioData - Raw audio sample data
EncodedVideoChunk / EncodedAudioChunk - Encoded media data
ImageDecoder - Static image decoding
VideoColorSpace - Color space information
Mp4Demuxer / WebMDemuxer / MkvDemuxer - Container demuxing with seeking
Mp4Muxer / WebMMuxer / MkvMuxer - Container muxing with streaming support

All encoders and decoders implement the EventTarget interface with addEventListener(), removeEventListener(), and dispatchEvent().

For full API documentation, see the W3C WebCodecs specification.

Development

Requirements

Rust (latest stable)
Node.js 18+
pnpm

Build

pnpm install
pnpm build

Test

pnpm test

Lint

pnpm lint
cargo clippy

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 144 Commits
.cargo		.cargo
.github		.github
.husky		.husky
__test__		__test__
benchmark		benchmark
example		example
src		src
tools/build-ffmpeg		tools/build-ffmpeg
.gitignore		.gitignore
.oxfmtrc.json		.oxfmtrc.json
.oxlintrc.json		.oxlintrc.json
.prettierignore		.prettierignore
CLAUDE.md		CLAUDE.md
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md
build.rs		build.rs
dts-header.d.ts		dts-header.d.ts
index.d.ts		index.d.ts
index.js		index.js
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
polyfill.d.ts		polyfill.d.ts
polyfill.js		polyfill.js
rustfmt.toml		rustfmt.toml
standard.d.ts		standard.d.ts
tsconfig.json		tsconfig.json
tsconfig.polyfill.json		tsconfig.polyfill.json

Uh oh!

License

Brooooooklyn/webcodecs-node

Folders and files

Latest commit

History

Repository files navigation

@napi-rs/webcodecs

Features

Installation

Optional: Canvas Integration

Quick Start

Video Encoding

Video Decoding

Audio Encoding

Image Decoding

Container Demuxing

Container Muxing

Streaming Muxer Mode

VideoFrame from Canvas

Supported Codecs

Video

Audio

Image

Containers

Platform Support

W3C Web Platform Tests Compliance

Ported Tests Status

Tests Not Ported (Browser-Only)

Hardware Acceleration

Configuration

Limitations

Scalable Video Coding (SVC)

Error Handling

VideoFrame Format Conversion

ImageDecoder Options

Platform-Specific Notes

Logging

Log Targets

FFmpeg Log Level Mapping

API Reference

Development

Requirements

Build

Test

Lint

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 5

Sponsor this project

Uh oh!

Packages 0

Uh oh!

Contributors 3

Uh oh!

Languages

Packages