# FFmpeg Integration Specification

## Overview
This document specifies the implementation details for integrating FFmpeg into the Maya Image Plane Node plugin for video frame decoding.

## Requirements
- Decode MP4 video files using FFmpeg libraries
- Extract video frames as raw pixel data (RGB or RGBA format)
- Provide frame-by-frame access for synchronization with Maya's timeline
- Handle various video codecs commonly used in MP4 containers
- Error handling for corrupted or unsupported video files

## Implementation Details

### Library Linking
- Link against FFmpeg libraries: libavcodec, libavformat, libavutil, libswscale
- Use pre-built FFmpeg binaries or build from source as needed
- Ensure compatibility with Maya 2023's compiler toolchain (Visual Studio 2017)

### Core Components

#### VideoDecoder Class
Responsible for initializing FFmpeg, opening video files, and decoding frames.

Key methods:
- `bool open(const std::string& filename)` - Opens video file and initializes FFmpeg context
- `void close()` - Releases FFmpeg resources
- `bool readFrame(AVFrame* frame)` - Decodes next video frame into provided AVFrame
- `int getWidth() const` - Returns video width
- `int getHeight() const` - Returns video height
- `double getFrameRate() const` - Returns video frame rate
- `int64_t getFrameCount() const` - Returns total number of frames
- `int64_t getCurrentFrame() const` - Returns current frame position

#### Pixel Format Conversion
- Convert decoded frames to RGB or RGBA format using libswscale
- Store converted frames in CPU memory for further processing
- Provide access to raw pixel data for viewport rendering

### Threading Considerations
- FFmpeg decoding should occur on a separate thread to avoid blocking Maya's main thread
- Use thread-safe queue for frame delivery to main thread
- Implement proper synchronization mechanisms (mutexes, condition variables)

### Error Handling
- Check return values from all FFmpeg functions
- Provide meaningful error messages for common issues:
  - File not found
  - Unsupported codec
  - Corrupted file
  - Memory allocation failures
- Graceful degradation when FFmpeg is unavailable

### Integration with Maya Node
- VideoDecoder instance managed by the MPxNode implementation
- Frame requests triggered by Maya's time changes
- Caching mechanism to avoid re-decoding frames

## Configuration
- FFmpeg library paths configurable via CMake
- Option to use system-installed FFmpeg or bundled binaries
- Build-time option to enable/disable FFmpeg integration (for testing)

## Testing
- Unit tests for VideoDecoder class
- Integration tests with sample MP4 files
- Performance testing for frame decoding speed
- Memory leak detection

## Dependencies
- FFmpeg 4.0 or later (tested with 4.4)
- libavcodec, libavformat, libavutil, libswscale
## Overview
This document specifies the implementation details for integrating FFmpeg into the Maya Image Plane Node plugin for video frame decoding.

## Requirements
- Decode MP4 video files using FFmpeg libraries
- Extract video frames as raw pixel data (RGB or RGBA format)
- Provide frame-by-frame access for synchronization with Maya's timeline
- Handle various video codecs commonly used in MP4 containers
- Error handling for corrupted or unsupported video files

## Implementation Details

### Library Linking
- Link against FFmpeg libraries: libavcodec, libavformat, libavutil, libswscale
- Use pre-built FFmpeg binaries or build from source as needed
- Ensure compatibility with Maya 2023's compiler toolchain (Visual Studio 2017)

### Core Components

#### VideoDecoder Class
Responsible for initializing FFmpeg, opening video files, and decoding frames.

Key methods:
- `bool open(const std::string& filename)` - Opens video file and initializes FFmpeg context
- `void close()` - Releases FFmpeg resources
- `bool readFrame(AVFrame* frame)` - Decodes next video frame into provided AVFrame
- `int getWidth() const` - Returns video width
- `int getHeight() const` - Returns video height
- `double getFrameRate() const` - Returns video frame rate
- `int64_t getFrameCount() const` - Returns total number of frames
- `int64_t getCurrentFrame() const` - Returns current frame position

#### Pixel Format Conversion
- Convert decoded frames to RGB or RGBA format using libswscale
- Store converted frames in CPU memory for further processing
- Provide access to raw pixel data for viewport rendering

### Threading Considerations
- FFmpeg decoding should occur on a separate thread to avoid blocking Maya's main thread
- Use thread-safe queue for frame delivery to main thread
- Implement proper synchronization mechanisms (mutexes, condition variables)

### Error Handling
- Check return values from all FFmpeg functions
- Provide meaningful error messages for common issues:
  - File not found
  - Unsupported codec
  - Corrupted file
  - Memory allocation failures
- Graceful degradation when FFmpeg is unavailable

### Integration with Maya Node
- VideoDecoder instance managed by the MPxNode implementation
- Frame requests triggered by Maya's time changes
- Caching mechanism to avoid re-decoding frames

## Configuration
- FFmpeg library paths configurable via CMake
- Option to use system-installed FFmpeg or bundled binaries
- Build-time option to enable/disable FFmpeg integration (for testing)

## Testing
- Unit tests for VideoDecoder class
- Integration tests with sample MP4 files
- Performance testing for frame decoding speed
- Memory leak detection

## Dependencies
- FFmpeg 4.0 or later (tested with 4.4)
- libavcodec, libavformat, libavutil, libswscale