143 lines
5.5 KiB
Markdown
143 lines
5.5 KiB
Markdown
# FFmpeg Integration Specification
|
|
|
|
## Overview
|
|
This document specifies the implementation details for integrating FFmpeg into the Maya Image Plane Node plugin for video frame decoding.
|
|
|
|
## Requirements
|
|
- Decode MP4 video files using FFmpeg libraries
|
|
- Extract video frames as raw pixel data (RGB or RGBA format)
|
|
- Provide frame-by-frame access for synchronization with Maya's timeline
|
|
- Handle various video codecs commonly used in MP4 containers
|
|
- Error handling for corrupted or unsupported video files
|
|
|
|
## Implementation Details
|
|
|
|
### Library Linking
|
|
- Link against FFmpeg libraries: libavcodec, libavformat, libavutil, libswscale
|
|
- Use pre-built FFmpeg binaries or build from source as needed
|
|
- Ensure compatibility with Maya 2023's compiler toolchain (Visual Studio 2017)
|
|
|
|
### Core Components
|
|
|
|
#### VideoDecoder Class
|
|
Responsible for initializing FFmpeg, opening video files, and decoding frames.
|
|
|
|
Key methods:
|
|
- `bool open(const std::string& filename)` - Opens video file and initializes FFmpeg context
|
|
- `void close()` - Releases FFmpeg resources
|
|
- `bool readFrame(AVFrame* frame)` - Decodes next video frame into provided AVFrame
|
|
- `int getWidth() const` - Returns video width
|
|
- `int getHeight() const` - Returns video height
|
|
- `double getFrameRate() const` - Returns video frame rate
|
|
- `int64_t getFrameCount() const` - Returns total number of frames
|
|
- `int64_t getCurrentFrame() const` - Returns current frame position
|
|
|
|
#### Pixel Format Conversion
|
|
- Convert decoded frames to RGB or RGBA format using libswscale
|
|
- Store converted frames in CPU memory for further processing
|
|
- Provide access to raw pixel data for viewport rendering
|
|
|
|
### Threading Considerations
|
|
- FFmpeg decoding should occur on a separate thread to avoid blocking Maya's main thread
|
|
- Use thread-safe queue for frame delivery to main thread
|
|
- Implement proper synchronization mechanisms (mutexes, condition variables)
|
|
|
|
### Error Handling
|
|
- Check return values from all FFmpeg functions
|
|
- Provide meaningful error messages for common issues:
|
|
- File not found
|
|
- Unsupported codec
|
|
- Corrupted file
|
|
- Memory allocation failures
|
|
- Graceful degradation when FFmpeg is unavailable
|
|
|
|
### Integration with Maya Node
|
|
- VideoDecoder instance managed by the MPxNode implementation
|
|
- Frame requests triggered by Maya's time changes
|
|
- Caching mechanism to avoid re-decoding frames
|
|
|
|
## Configuration
|
|
- FFmpeg library paths configurable via CMake
|
|
- Option to use system-installed FFmpeg or bundled binaries
|
|
- Build-time option to enable/disable FFmpeg integration (for testing)
|
|
|
|
## Testing
|
|
- Unit tests for VideoDecoder class
|
|
- Integration tests with sample MP4 files
|
|
- Performance testing for frame decoding speed
|
|
- Memory leak detection
|
|
|
|
## Dependencies
|
|
- FFmpeg 4.0 or later (tested with 4.4)
|
|
- libavcodec, libavformat, libavutil, libswscale
|
|
## Overview
|
|
This document specifies the implementation details for integrating FFmpeg into the Maya Image Plane Node plugin for video frame decoding.
|
|
|
|
## Requirements
|
|
- Decode MP4 video files using FFmpeg libraries
|
|
- Extract video frames as raw pixel data (RGB or RGBA format)
|
|
- Provide frame-by-frame access for synchronization with Maya's timeline
|
|
- Handle various video codecs commonly used in MP4 containers
|
|
- Error handling for corrupted or unsupported video files
|
|
|
|
## Implementation Details
|
|
|
|
### Library Linking
|
|
- Link against FFmpeg libraries: libavcodec, libavformat, libavutil, libswscale
|
|
- Use pre-built FFmpeg binaries or build from source as needed
|
|
- Ensure compatibility with Maya 2023's compiler toolchain (Visual Studio 2017)
|
|
|
|
### Core Components
|
|
|
|
#### VideoDecoder Class
|
|
Responsible for initializing FFmpeg, opening video files, and decoding frames.
|
|
|
|
Key methods:
|
|
- `bool open(const std::string& filename)` - Opens video file and initializes FFmpeg context
|
|
- `void close()` - Releases FFmpeg resources
|
|
- `bool readFrame(AVFrame* frame)` - Decodes next video frame into provided AVFrame
|
|
- `int getWidth() const` - Returns video width
|
|
- `int getHeight() const` - Returns video height
|
|
- `double getFrameRate() const` - Returns video frame rate
|
|
- `int64_t getFrameCount() const` - Returns total number of frames
|
|
- `int64_t getCurrentFrame() const` - Returns current frame position
|
|
|
|
#### Pixel Format Conversion
|
|
- Convert decoded frames to RGB or RGBA format using libswscale
|
|
- Store converted frames in CPU memory for further processing
|
|
- Provide access to raw pixel data for viewport rendering
|
|
|
|
### Threading Considerations
|
|
- FFmpeg decoding should occur on a separate thread to avoid blocking Maya's main thread
|
|
- Use thread-safe queue for frame delivery to main thread
|
|
- Implement proper synchronization mechanisms (mutexes, condition variables)
|
|
|
|
### Error Handling
|
|
- Check return values from all FFmpeg functions
|
|
- Provide meaningful error messages for common issues:
|
|
- File not found
|
|
- Unsupported codec
|
|
- Corrupted file
|
|
- Memory allocation failures
|
|
- Graceful degradation when FFmpeg is unavailable
|
|
|
|
### Integration with Maya Node
|
|
- VideoDecoder instance managed by the MPxNode implementation
|
|
- Frame requests triggered by Maya's time changes
|
|
- Caching mechanism to avoid re-decoding frames
|
|
|
|
## Configuration
|
|
- FFmpeg library paths configurable via CMake
|
|
- Option to use system-installed FFmpeg or bundled binaries
|
|
- Build-time option to enable/disable FFmpeg integration (for testing)
|
|
|
|
## Testing
|
|
- Unit tests for VideoDecoder class
|
|
- Integration tests with sample MP4 files
|
|
- Performance testing for frame decoding speed
|
|
- Memory leak detection
|
|
|
|
## Dependencies
|
|
- FFmpeg 4.0 or later (tested with 4.4)
|
|
- libavcodec, libavformat, libavutil, libswscale
|