scikit-video

scikit-video - Scientific Video Processing

scikit-video simplifies the complex world of video codecs and containers by providing a consistent NumPy-based interface. It allows for the calculation of motion vectors, video quality assessment (VQA), and seamless integration with the rest of the scientific Python stack.

When to Use

• Reading and writing video files in various formats (MP4, AVI, MKV) via FFmpeg.
• Extracting specific frames or segments from long videos without loading them entirely into memory.
• Calculating motion estimation (Block Matching, Optical Flow).
• Measuring video quality (PSNR, SSIM, VIF, NIQE).
• Generating video datasets for machine learning.
• Visualizing temporal changes in pixel data (e.g., scientific recordings).
• Handling raw YUV data streams.

Reference Documentation

Official docs: http://www.scikit-video.org/
GitHub: https://github.com/scikit-video/scikit-video
Search patterns: skvideo.io.vread, skvideo.io.FFmpegReader, skvideo.motion, skvideo.measure

Core Principles

Video as 4D Arrays

A video is represented as a NumPy array with shape (T, H, W, C):

• T: Time (number of frames)
• H: Height
• W: Width
• C: Channels (usually 3 for RGB)

FFmpeg Backend

Scikit-video does not contain its own codecs; it is a bridge to FFmpeg. You must have FFmpeg installed on your system for skvideo.io to function.

Generators for Large Data

For long videos, scikit-video provides generator-based readers (vreader) to process frames one by one, preventing RAM exhaustion.

Quick Reference

Installation

pip install scikit-video
# Note: Ensure ffmpeg is in your system PATH

Standard Imports

import skvideo.io
import skvideo.motion
import skvideo.measure
import numpy as np

Basic Pattern - Read and Inspect

import skvideo.io

# 1. Read the whole video into a NumPy array
# Shape: (frames, height, width, 3)
video_data = skvideo.io.vread("experiment.mp4")

# 2. Get basic info
n_frames, height, width, channels = video_data.shape
print(f"FPS: {n_frames / 10}, Resolution: {width}x{height}")

# 3. Access a specific frame
frame_10 = video_data[10]