233 lines
9.4 KiB
Markdown
233 lines
9.4 KiB
Markdown
# Post-Effects Specification
|
|
|
|
## Overview
|
|
This document specifies the implementation details for post-effects processing on video frames, including crop, resize, and flip operations.
|
|
|
|
## Requirements
|
|
- Implement crop functionality to select a region of interest from the video frame
|
|
- Implement resize functionality to scale the video frame to different dimensions
|
|
- Implement flip functionality to mirror the video frame horizontally and/or vertically
|
|
- Allow combining multiple post-effects in any order
|
|
- Maintain high performance for real-time playback
|
|
- Support various pixel formats (RGB, RGBA, etc.)
|
|
- Provide intuitive user controls for each effect
|
|
|
|
## Implementation Details
|
|
|
|
### Effect Definitions
|
|
|
|
#### Crop
|
|
- Parameters: x, y, width, height (in pixels)
|
|
- Defines rectangular region to extract from source frame
|
|
- Coordinates relative to top-left corner of source frame
|
|
- Width and height must be within source frame bounds
|
|
- If crop region extends beyond source frame, clamp to valid region
|
|
|
|
#### Resize
|
|
- Parameters: width, height (in pixels)
|
|
- Target dimensions for output frame
|
|
- Maintain aspect ratio option (to be determined)
|
|
- Use appropriate filtering algorithm (bilinear recommended for balance of quality/performance)
|
|
|
|
#### Flip
|
|
- Parameters: flipX (bool), flipY (bool)
|
|
- flipX: Mirror frame horizontally (left-right)
|
|
- flipY: Mirror frame vertically (top-bottom)
|
|
- Can be applied independently or together
|
|
|
|
### Processing Order
|
|
The recommended order of operations is:
|
|
1. Crop (select region of interest)
|
|
2. Resize (scale to desired dimensions)
|
|
3. Flip (apply mirroring)
|
|
This order ensures that flipping operates on the final composed image.
|
|
|
|
### Implementation Components
|
|
|
|
#### FrameProcessor Class
|
|
Responsible for applying post-effects to decoded video frames.
|
|
|
|
Key methods:
|
|
- `bool processFrame(const AVFrame* srcFrame, AVFrame* dstFrame)` - Apply all enabled effects
|
|
- `void setCropParameters(int x, int y, int width, int height)` - Configure crop
|
|
- `void setResizeParameters(int width, int height)` - Configure resize
|
|
- `void setFlipParameters(bool flipX, bool flipY)` - Configure flip
|
|
- `void enableCrop(bool enabled)` - Enable/disable crop effect
|
|
- `void enableResize(bool enabled)` - Enable/disable resize effect
|
|
- `void enableFlip(bool enabled)` - Enable/disable flip effect
|
|
- `bool isCropEnabled() const` - Check if crop is enabled
|
|
- `bool isResizeEnabled() const` - Check if resize is enabled
|
|
- `bool isFlipEnabled() const` - Check if flip is enabled
|
|
|
|
#### Internal Processing Steps
|
|
1. Validate input frame
|
|
2. Apply crop if enabled:
|
|
- Calculate source rectangle
|
|
- Extract region from source frame
|
|
- Handle format conversion if needed
|
|
3. Apply resize if enabled:
|
|
- Use libswscale for scaling with appropriate filters
|
|
- Allocate temporary frame if needed
|
|
4. Apply flip if enabled:
|
|
- Perform in-place mirroring of frame data
|
|
- Handle different pixel formats correctly
|
|
5. Output processed frame
|
|
|
|
### Integration Points
|
|
- FFmpeg Decoder: Receives raw frames from decoder
|
|
- Maya Node: Applies post-effects before outputting frame data
|
|
- Viewport 2.0: Receives post-processed frames for display
|
|
- Caching: Option to cache frames before or after post-effects (to be determined)
|
|
|
|
### Threading Considerations
|
|
- Post-processing should occur on the same thread as frame retrieval from decoder
|
|
- Can be done either in background decoding thread or main thread
|
|
- If done in background thread, ensure thread-safe delivery to main thread
|
|
- Minimize processing time to avoid blocking pipeline
|
|
|
|
### Error Handling
|
|
- Validate effect parameters (non-negative dimensions, etc.)
|
|
- Handle memory allocation failures gracefully
|
|
- Provide fallback to unprocessed frame if processing fails
|
|
- Log errors to Maya's script editor using MGlobal::displayError
|
|
|
|
### Performance Considerations
|
|
- Use hardware-accelerated operations where possible (though limited in CPU-based processing)
|
|
- Minimize memory allocations and copies
|
|
- Use efficient scaling algorithms (bilinear as good compromise)
|
|
- Consider processing only when parameters change
|
|
- Reuse buffers when possible to avoid reallocations
|
|
|
|
### Pixel Format Support
|
|
- Primary support for RGB24 and RGBA32 formats
|
|
- Convert other formats to supported ones if needed
|
|
- Maintain format consistency through processing chain
|
|
- Handle format conversion with libswscale when necessary
|
|
|
|
### Maya API Specifics
|
|
- Store effect parameters as node attributes
|
|
- Use MFnNumericAttribute for numeric parameters (x, y, width, height, flip bools)
|
|
- Use MFnBooleanAttribute for enable/disable toggles
|
|
- Implement attribute validation in node compute method
|
|
- Provide default values that result in no-op when effects disabled
|
|
|
|
## Dependencies
|
|
- FFmpeg libraries (libswscale for scaling and format conversion)
|
|
- MPxNode implementation
|
|
- Frame data structures
|
|
## Overview
|
|
This document specifies the implementation details for post-effects processing on video frames, including crop, resize, and flip operations.
|
|
|
|
## Requirements
|
|
- Implement crop functionality to select a region of interest from the video frame
|
|
- Implement resize functionality to scale the video frame to different dimensions
|
|
- Implement flip functionality to mirror the video frame horizontally and/or vertically
|
|
- Allow combining multiple post-effects in any order
|
|
- Maintain high performance for real-time playback
|
|
- Support various pixel formats (RGB, RGBA, etc.)
|
|
- Provide intuitive user controls for each effect
|
|
|
|
## Implementation Details
|
|
|
|
### Effect Definitions
|
|
|
|
#### Crop
|
|
- Parameters: x, y, width, height (in pixels)
|
|
- Defines rectangular region to extract from source frame
|
|
- Coordinates relative to top-left corner of source frame
|
|
- Width and height must be within source frame bounds
|
|
- If crop region extends beyond source frame, clamp to valid region
|
|
|
|
#### Resize
|
|
- Parameters: width, height (in pixels)
|
|
- Target dimensions for output frame
|
|
- Maintain aspect ratio option (to be determined)
|
|
- Use appropriate filtering algorithm (bilinear recommended for balance of quality/performance)
|
|
|
|
#### Flip
|
|
- Parameters: flipX (bool), flipY (bool)
|
|
- flipX: Mirror frame horizontally (left-right)
|
|
- flipY: Mirror frame vertically (top-bottom)
|
|
- Can be applied independently or together
|
|
|
|
### Processing Order
|
|
The recommended order of operations is:
|
|
1. Crop (select region of interest)
|
|
2. Resize (scale to desired dimensions)
|
|
3. Flip (apply mirroring)
|
|
This order ensures that flipping operates on the final composed image.
|
|
|
|
### Implementation Components
|
|
|
|
#### FrameProcessor Class
|
|
Responsible for applying post-effects to decoded video frames.
|
|
|
|
Key methods:
|
|
- `bool processFrame(const AVFrame* srcFrame, AVFrame* dstFrame)` - Apply all enabled effects
|
|
- `void setCropParameters(int x, int y, int width, int height)` - Configure crop
|
|
- `void setResizeParameters(int width, int height)` - Configure resize
|
|
- `void setFlipParameters(bool flipX, bool flipY)` - Configure flip
|
|
- `void enableCrop(bool enabled)` - Enable/disable crop effect
|
|
- `void enableResize(bool enabled)` - Enable/disable resize effect
|
|
- `void enableFlip(bool enabled)` - Enable/disable flip effect
|
|
- `bool isCropEnabled() const` - Check if crop is enabled
|
|
- `bool isResizeEnabled() const` - Check if resize is enabled
|
|
- `bool isFlipEnabled() const` - Check if flip is enabled
|
|
|
|
#### Internal Processing Steps
|
|
1. Validate input frame
|
|
2. Apply crop if enabled:
|
|
- Calculate source rectangle
|
|
- Extract region from source frame
|
|
- Handle format conversion if needed
|
|
3. Apply resize if enabled:
|
|
- Use libswscale for scaling with appropriate filters
|
|
- Allocate temporary frame if needed
|
|
4. Apply flip if enabled:
|
|
- Perform in-place mirroring of frame data
|
|
- Handle different pixel formats correctly
|
|
5. Output processed frame
|
|
|
|
### Integration Points
|
|
- FFmpeg Decoder: Receives raw frames from decoder
|
|
- Maya Node: Applies post-effects before outputting frame data
|
|
- Viewport 2.0: Receives post-processed frames for display
|
|
- Caching: Option to cache frames before or after post-effects (to be determined)
|
|
|
|
### Threading Considerations
|
|
- Post-processing should occur on the same thread as frame retrieval from decoder
|
|
- Can be done either in background decoding thread or main thread
|
|
- If done in background thread, ensure thread-safe delivery to main thread
|
|
- Minimize processing time to avoid blocking pipeline
|
|
|
|
### Error Handling
|
|
- Validate effect parameters (non-negative dimensions, etc.)
|
|
- Handle memory allocation failures gracefully
|
|
- Provide fallback to unprocessed frame if processing fails
|
|
- Log errors to Maya's script editor using MGlobal::displayError
|
|
|
|
### Performance Considerations
|
|
- Use hardware-accelerated operations where possible (though limited in CPU-based processing)
|
|
- Minimize memory allocations and copies
|
|
- Use efficient scaling algorithms (bilinear as good compromise)
|
|
- Consider processing only when parameters change
|
|
- Reuse buffers when possible to avoid reallocations
|
|
|
|
### Pixel Format Support
|
|
- Primary support for RGB24 and RGBA32 formats
|
|
- Convert other formats to supported ones if needed
|
|
- Maintain format consistency through processing chain
|
|
- Handle format conversion with libswscale when necessary
|
|
|
|
### Maya API Specifics
|
|
- Store effect parameters as node attributes
|
|
- Use MFnNumericAttribute for numeric parameters (x, y, width, height, flip bools)
|
|
- Use MFnBooleanAttribute for enable/disable toggles
|
|
- Implement attribute validation in node compute method
|
|
- Provide default values that result in no-op when effects disabled
|
|
|
|
## Dependencies
|
|
- FFmpeg libraries (libswscale for scaling and format conversion)
|
|
- MPxNode implementation
|
|
- Frame data structures
|