# Post-Effects Specification ## Overview This document specifies the implementation details for post-effects processing on video frames, including crop, resize, and flip operations. ## Requirements - Implement crop functionality to select a region of interest from the video frame - Implement resize functionality to scale the video frame to different dimensions - Implement flip functionality to mirror the video frame horizontally and/or vertically - Allow combining multiple post-effects in any order - Maintain high performance for real-time playback - Support various pixel formats (RGB, RGBA, etc.) - Provide intuitive user controls for each effect ## Implementation Details ### Effect Definitions #### Crop - Parameters: x, y, width, height (in pixels) - Defines rectangular region to extract from source frame - Coordinates relative to top-left corner of source frame - Width and height must be within source frame bounds - If crop region extends beyond source frame, clamp to valid region #### Resize - Parameters: width, height (in pixels) - Target dimensions for output frame - Maintain aspect ratio option (to be determined) - Use appropriate filtering algorithm (bilinear recommended for balance of quality/performance) #### Flip - Parameters: flipX (bool), flipY (bool) - flipX: Mirror frame horizontally (left-right) - flipY: Mirror frame vertically (top-bottom) - Can be applied independently or together ### Processing Order The recommended order of operations is: 1. Crop (select region of interest) 2. Resize (scale to desired dimensions) 3. Flip (apply mirroring) This order ensures that flipping operates on the final composed image. ### Implementation Components #### FrameProcessor Class Responsible for applying post-effects to decoded video frames. Key methods: - `bool processFrame(const AVFrame* srcFrame, AVFrame* dstFrame)` - Apply all enabled effects - `void setCropParameters(int x, int y, int width, int height)` - Configure crop - `void setResizeParameters(int width, int height)` - Configure resize - `void setFlipParameters(bool flipX, bool flipY)` - Configure flip - `void enableCrop(bool enabled)` - Enable/disable crop effect - `void enableResize(bool enabled)` - Enable/disable resize effect - `void enableFlip(bool enabled)` - Enable/disable flip effect - `bool isCropEnabled() const` - Check if crop is enabled - `bool isResizeEnabled() const` - Check if resize is enabled - `bool isFlipEnabled() const` - Check if flip is enabled #### Internal Processing Steps 1. Validate input frame 2. Apply crop if enabled: - Calculate source rectangle - Extract region from source frame - Handle format conversion if needed 3. Apply resize if enabled: - Use libswscale for scaling with appropriate filters - Allocate temporary frame if needed 4. Apply flip if enabled: - Perform in-place mirroring of frame data - Handle different pixel formats correctly 5. Output processed frame ### Integration Points - FFmpeg Decoder: Receives raw frames from decoder - Maya Node: Applies post-effects before outputting frame data - Viewport 2.0: Receives post-processed frames for display - Caching: Option to cache frames before or after post-effects (to be determined) ### Threading Considerations - Post-processing should occur on the same thread as frame retrieval from decoder - Can be done either in background decoding thread or main thread - If done in background thread, ensure thread-safe delivery to main thread - Minimize processing time to avoid blocking pipeline ### Error Handling - Validate effect parameters (non-negative dimensions, etc.) - Handle memory allocation failures gracefully - Provide fallback to unprocessed frame if processing fails - Log errors to Maya's script editor using MGlobal::displayError ### Performance Considerations - Use hardware-accelerated operations where possible (though limited in CPU-based processing) - Minimize memory allocations and copies - Use efficient scaling algorithms (bilinear as good compromise) - Consider processing only when parameters change - Reuse buffers when possible to avoid reallocations ### Pixel Format Support - Primary support for RGB24 and RGBA32 formats - Convert other formats to supported ones if needed - Maintain format consistency through processing chain - Handle format conversion with libswscale when necessary ### Maya API Specifics - Store effect parameters as node attributes - Use MFnNumericAttribute for numeric parameters (x, y, width, height, flip bools) - Use MFnBooleanAttribute for enable/disable toggles - Implement attribute validation in node compute method - Provide default values that result in no-op when effects disabled ## Dependencies - FFmpeg libraries (libswscale for scaling and format conversion) - MPxNode implementation - Frame data structures ## Overview This document specifies the implementation details for post-effects processing on video frames, including crop, resize, and flip operations. ## Requirements - Implement crop functionality to select a region of interest from the video frame - Implement resize functionality to scale the video frame to different dimensions - Implement flip functionality to mirror the video frame horizontally and/or vertically - Allow combining multiple post-effects in any order - Maintain high performance for real-time playback - Support various pixel formats (RGB, RGBA, etc.) - Provide intuitive user controls for each effect ## Implementation Details ### Effect Definitions #### Crop - Parameters: x, y, width, height (in pixels) - Defines rectangular region to extract from source frame - Coordinates relative to top-left corner of source frame - Width and height must be within source frame bounds - If crop region extends beyond source frame, clamp to valid region #### Resize - Parameters: width, height (in pixels) - Target dimensions for output frame - Maintain aspect ratio option (to be determined) - Use appropriate filtering algorithm (bilinear recommended for balance of quality/performance) #### Flip - Parameters: flipX (bool), flipY (bool) - flipX: Mirror frame horizontally (left-right) - flipY: Mirror frame vertically (top-bottom) - Can be applied independently or together ### Processing Order The recommended order of operations is: 1. Crop (select region of interest) 2. Resize (scale to desired dimensions) 3. Flip (apply mirroring) This order ensures that flipping operates on the final composed image. ### Implementation Components #### FrameProcessor Class Responsible for applying post-effects to decoded video frames. Key methods: - `bool processFrame(const AVFrame* srcFrame, AVFrame* dstFrame)` - Apply all enabled effects - `void setCropParameters(int x, int y, int width, int height)` - Configure crop - `void setResizeParameters(int width, int height)` - Configure resize - `void setFlipParameters(bool flipX, bool flipY)` - Configure flip - `void enableCrop(bool enabled)` - Enable/disable crop effect - `void enableResize(bool enabled)` - Enable/disable resize effect - `void enableFlip(bool enabled)` - Enable/disable flip effect - `bool isCropEnabled() const` - Check if crop is enabled - `bool isResizeEnabled() const` - Check if resize is enabled - `bool isFlipEnabled() const` - Check if flip is enabled #### Internal Processing Steps 1. Validate input frame 2. Apply crop if enabled: - Calculate source rectangle - Extract region from source frame - Handle format conversion if needed 3. Apply resize if enabled: - Use libswscale for scaling with appropriate filters - Allocate temporary frame if needed 4. Apply flip if enabled: - Perform in-place mirroring of frame data - Handle different pixel formats correctly 5. Output processed frame ### Integration Points - FFmpeg Decoder: Receives raw frames from decoder - Maya Node: Applies post-effects before outputting frame data - Viewport 2.0: Receives post-processed frames for display - Caching: Option to cache frames before or after post-effects (to be determined) ### Threading Considerations - Post-processing should occur on the same thread as frame retrieval from decoder - Can be done either in background decoding thread or main thread - If done in background thread, ensure thread-safe delivery to main thread - Minimize processing time to avoid blocking pipeline ### Error Handling - Validate effect parameters (non-negative dimensions, etc.) - Handle memory allocation failures gracefully - Provide fallback to unprocessed frame if processing fails - Log errors to Maya's script editor using MGlobal::displayError ### Performance Considerations - Use hardware-accelerated operations where possible (though limited in CPU-based processing) - Minimize memory allocations and copies - Use efficient scaling algorithms (bilinear as good compromise) - Consider processing only when parameters change - Reuse buffers when possible to avoid reallocations ### Pixel Format Support - Primary support for RGB24 and RGBA32 formats - Convert other formats to supported ones if needed - Maintain format consistency through processing chain - Handle format conversion with libswscale when necessary ### Maya API Specifics - Store effect parameters as node attributes - Use MFnNumericAttribute for numeric parameters (x, y, width, height, flip bools) - Use MFnBooleanAttribute for enable/disable toggles - Implement attribute validation in node compute method - Provide default values that result in no-op when effects disabled ## Dependencies - FFmpeg libraries (libswscale for scaling and format conversion) - MPxNode implementation - Frame data structures