213 lines
9.0 KiB
Markdown
213 lines
9.0 KiB
Markdown
# Viewport 2.0 Integration Specification
|
|
|
|
## Overview
|
|
This document specifies the implementation details for integrating video frame display into Maya's Viewport 2.0 using MHWRender::MPxDrawOverride.
|
|
|
|
## Requirements
|
|
- Display video frames in Viewport 2.0 as an image plane
|
|
- Support for RGB and RGBA frame formats
|
|
- Proper handling of frame rate and playback speed
|
|
- Integration with Maya's viewport rendering pipeline
|
|
- Support for post-effects (crop, resize, flip) applied to displayed frames
|
|
- Efficient texture updates to minimize performance impact
|
|
- Compatibility with Maya 2023's Viewport 2.0 API
|
|
|
|
## Implementation Details
|
|
|
|
### Core Components
|
|
|
|
#### VideoFrameDrawOverride Class
|
|
Derived from MHWRender::MPxDrawOverride, responsible for rendering video frames in the viewport.
|
|
|
|
Key methods:
|
|
- `bool isBounded(const MDagPath& objPath) const` - Returns whether the object has a bounding box
|
|
- `MBoundingBox boundingBox(const MDagPath& objPath) const` - Returns bounding box of the object
|
|
- `uint32_t drawFlags() const` - Returns draw flags for the override
|
|
- `void prepareForDraw(...)` - Prepares resources needed for drawing
|
|
- `void addUIDrawables(...)` - Adds UI drawables (if needed)
|
|
- `void hasUIDrawables() const` - Returns whether UI drawables are present
|
|
- `void draw(...)` - Main drawing method where video frame is rendered
|
|
|
|
#### Texture Management
|
|
- OpenGL texture object to hold current video frame
|
|
- Texture updates when new frame is available
|
|
- Proper texture format matching (GL_RGB, GL_RGBA, etc.)
|
|
- Efficient texture sub-data updates when possible
|
|
- Texture cleanup when node is destroyed
|
|
|
|
#### Frame Data Handling
|
|
- Receive frame data from Maya node via output attributes
|
|
- Convert frame data to appropriate OpenGL format if needed
|
|
- Update texture with new frame data
|
|
- Maintain frame timestamp for synchronization
|
|
|
|
### Integration Points
|
|
- Maya Node Integration: Connect to MPxNode's output attributes to receive frame data
|
|
- Frame Rate Synchronization: Use frame timestamp to synchronize with Maya's timeline
|
|
- Post-effects: Apply transformations to frame data before texture upload
|
|
- Caching: Interface with frame caching mechanism to get frames efficiently
|
|
|
|
### Rendering Approach
|
|
1. Receive new frame data from MPxNode (via attribute change notification)
|
|
2. Convert frame data to OpenGL-compatible format if necessary
|
|
3. Update OpenGL texture with new frame data
|
|
4. In viewport draw call:
|
|
- Set up appropriate shader program (simple texture shader)
|
|
- Bind texture
|
|
- Draw quad covering the image plane area
|
|
- Apply any viewport-specific transformations
|
|
|
|
### Shader Program
|
|
- Simple vertex and fragment shaders for texture rendering
|
|
- Vertex shader: Pass through texture coordinates
|
|
- Fragment shader: Sample texture and output color
|
|
- Optional: Support for color correction or other viewport-specific effects
|
|
|
|
### Texture Updates
|
|
- Use GL_PIXEL_UNPACK_BUFFER for efficient updates if supported
|
|
- Fallback to glTexSubImage2D for broader compatibility
|
|
- Handle different pixel formats (RGB, RGBA, BGRA, etc.)
|
|
- Flip Y-axis if necessary (OpenGL vs image coordinate systems)
|
|
|
|
### Bounding Box and Selection
|
|
- Return appropriate bounding box based on frame dimensions and aspect ratio
|
|
- Support for viewport selection of the image plane
|
|
- Handle transform nodes (position, rotation, scale) if applicable
|
|
|
|
### Threading Considerations
|
|
- Viewport drawing occurs on main thread
|
|
- Texture updates must happen on main thread (OpenGL context)
|
|
- Use thread-safe queue to transfer frame data from decoding thread to main thread
|
|
- Synchronize access to shared frame data
|
|
|
|
### Error Handling
|
|
- Check for OpenGL errors after texture operations
|
|
- Handle texture creation failures gracefully
|
|
- Provide fallback rendering (e.g., colored quad) when frame data unavailable
|
|
- Log errors to Maya's script editor using MGlobal::displayError
|
|
|
|
### Performance Considerations
|
|
- Minimize texture format conversions
|
|
- Update only changed portions of texture when possible
|
|
- Use appropriate texture filtering (GL_LINEAR for smooth playback)
|
|
- Avoid expensive operations during viewport drawing
|
|
- Consider using persistent mapped buffers for frequent updates
|
|
|
|
### Maya API Specifics
|
|
- Use MHWRender::MPxDrawOverride as base class
|
|
- Register draw override with MHWRender::MRenderer
|
|
- Use MImage class for image format conversions if needed
|
|
- Leverage MHWRender::MShaderManager for shader programs
|
|
- Follow Viewport 2.0 API guidelines for draw overrides
|
|
|
|
## Dependencies
|
|
- Maya Node implementation (for frame data)
|
|
- Frame caching mechanism
|
|
- OpenGL 3.2+ (required for Viewport 2.0)
|
|
- Maya API 2023 (MHWRender module)
|
|
## Overview
|
|
This document specifies the implementation details for integrating video frame display into Maya's Viewport 2.0 using MHWRender::MPxDrawOverride.
|
|
|
|
## Requirements
|
|
- Display video frames in Viewport 2.0 as an image plane
|
|
- Support for RGB and RGBA frame formats
|
|
- Proper handling of frame rate and playback speed
|
|
- Integration with Maya's viewport rendering pipeline
|
|
- Support for post-effects (crop, resize, flip) applied to displayed frames
|
|
- Efficient texture updates to minimize performance impact
|
|
- Compatibility with Maya 2023's Viewport 2.0 API
|
|
|
|
## Implementation Details
|
|
|
|
### Core Components
|
|
|
|
#### VideoFrameDrawOverride Class
|
|
Derived from MHWRender::MPxDrawOverride, responsible for rendering video frames in the viewport.
|
|
|
|
Key methods:
|
|
- `bool isBounded(const MDagPath& objPath) const` - Returns whether the object has a bounding box
|
|
- `MBoundingBox boundingBox(const MDagPath& objPath) const` - Returns bounding box of the object
|
|
- `uint32_t drawFlags() const` - Returns draw flags for the override
|
|
- `void prepareForDraw(...)` - Prepares resources needed for drawing
|
|
- `void addUIDrawables(...)` - Adds UI drawables (if needed)
|
|
- `void hasUIDrawables() const` - Returns whether UI drawables are present
|
|
- `void draw(...)` - Main drawing method where video frame is rendered
|
|
|
|
#### Texture Management
|
|
- OpenGL texture object to hold current video frame
|
|
- Texture updates when new frame is available
|
|
- Proper texture format matching (GL_RGB, GL_RGBA, etc.)
|
|
- Efficient texture sub-data updates when possible
|
|
- Texture cleanup when node is destroyed
|
|
|
|
#### Frame Data Handling
|
|
- Receive frame data from Maya node via output attributes
|
|
- Convert frame data to appropriate OpenGL format if needed
|
|
- Update texture with new frame data
|
|
- Maintain frame timestamp for synchronization
|
|
|
|
### Integration Points
|
|
- Maya Node Integration: Connect to MPxNode's output attributes to receive frame data
|
|
- Frame Rate Synchronization: Use frame timestamp to synchronize with Maya's timeline
|
|
- Post-effects: Apply transformations to frame data before texture upload
|
|
- Caching: Interface with frame caching mechanism to get frames efficiently
|
|
|
|
### Rendering Approach
|
|
1. Receive new frame data from MPxNode (via attribute change notification)
|
|
2. Convert frame data to OpenGL-compatible format if necessary
|
|
3. Update OpenGL texture with new frame data
|
|
4. In viewport draw call:
|
|
- Set up appropriate shader program (simple texture shader)
|
|
- Bind texture
|
|
- Draw quad covering the image plane area
|
|
- Apply any viewport-specific transformations
|
|
|
|
### Shader Program
|
|
- Simple vertex and fragment shaders for texture rendering
|
|
- Vertex shader: Pass through texture coordinates
|
|
- Fragment shader: Sample texture and output color
|
|
- Optional: Support for color correction or other viewport-specific effects
|
|
|
|
### Texture Updates
|
|
- Use GL_PIXEL_UNPACK_BUFFER for efficient updates if supported
|
|
- Fallback to glTexSubImage2D for broader compatibility
|
|
- Handle different pixel formats (RGB, RGBA, BGRA, etc.)
|
|
- Flip Y-axis if necessary (OpenGL vs image coordinate systems)
|
|
|
|
### Bounding Box and Selection
|
|
- Return appropriate bounding box based on frame dimensions and aspect ratio
|
|
- Support for viewport selection of the image plane
|
|
- Handle transform nodes (position, rotation, scale) if applicable
|
|
|
|
### Threading Considerations
|
|
- Viewport drawing occurs on main thread
|
|
- Texture updates must happen on main thread (OpenGL context)
|
|
- Use thread-safe queue to transfer frame data from decoding thread to main thread
|
|
- Synchronize access to shared frame data
|
|
|
|
### Error Handling
|
|
- Check for OpenGL errors after texture operations
|
|
- Handle texture creation failures gracefully
|
|
- Provide fallback rendering (e.g., colored quad) when frame data unavailable
|
|
- Log errors to Maya's script editor using MGlobal::displayError
|
|
|
|
### Performance Considerations
|
|
- Minimize texture format conversions
|
|
- Update only changed portions of texture when possible
|
|
- Use appropriate texture filtering (GL_LINEAR for smooth playback)
|
|
- Avoid expensive operations during viewport drawing
|
|
- Consider using persistent mapped buffers for frequent updates
|
|
|
|
### Maya API Specifics
|
|
- Use MHWRender::MPxDrawOverride as base class
|
|
- Register draw override with MHWRender::MRenderer
|
|
- Use MImage class for image format conversions if needed
|
|
- Leverage MHWRender::MShaderManager for shader programs
|
|
- Follow Viewport 2.0 API guidelines for draw overrides
|
|
|
|
## Dependencies
|
|
- Maya Node implementation (for frame data)
|
|
- Frame caching mechanism
|
|
- OpenGL 3.2+ (required for Viewport 2.0)
|
|
- Maya API 2023 (MHWRender module)
|