MediaPlane/docs/Maya_Node_Implementation_Spec.md

Maya Node Implementation Specification

Overview

This document specifies the implementation details for the Maya MPxNode plugin that handles video file input and frame output for the Image Plane Node.

Requirements

• Create a custom MPxNode that accepts video file path as input
• Output video frame data that can be consumed by viewport rendering
• Support attributes for frame rate control and playback rate adjustment
• Integrate with FFmpeg video decoding component
• Provide frame caching interface
• Follow Maya API best practices for node creation

Implementation Details

Node Definition

• Node Name: imagePlaneVideo
• Node ID: Unique ID obtained from Autodesk
• Classification: filter or image (to be determined based on Maya's categorization)
• Register as a Maya plugin using MFnPlugin

Attributes

Input Attributes

1. videoFile (message or string) - Path to the video file
2. frameRate (double) - Playback rate multiplier (1.0 = normal speed)
3. useMayaFrameRate (bool) - Whether to synchronize with Maya's timeline frame rate
4. currentTime (double) - Connection to Maya's timeline (for frame calculation)
5. loop (bool) - Whether to loop the video playback
6. postEffectCrop (double4) - Crop parameters (x, y, width, height)
7. postEffectResize (double2) - Resize parameters (width, height)
8. postEffectFlip (bool2) - Flip parameters (flipX, flipY)

Output Attributes

1. outFrameData (message or custom data type) - Pointer to video frame data
2. outFrameWidth (int) - Width of current frame
3. outFrameHeight (int) - Height of current frame
4. outFrameFormat (int) - Pixel format of current frame (RGB, RGBA, etc.)
5. outFrameTimestamp (double) - Timestamp of current frame
6. isValid (bool) - Whether the current frame data is valid

Core Components

VideoFrameData Class

Custom data class to hold video frame information:

• Pointer to pixel data (RGB/RGBA buffer)
• Width and height
• Pixel format
• Timestamp
• Reference counting for proper cleanup

Node Computation

In the compute() method:

1. Check if video file attribute has changed
2. If changed, initialize FFmpeg decoder with new file
3. Calculate target frame based on:
   • Maya's current time (if useMayaFrameRate is true)
   • Frame rate multiplier attribute
   • Loop settings
4. Request frame from FFmpeg decoder (or cache)
5. Apply post-effects if specified
6. Update output attributes with frame data
7. Mark node as clean

Integration Points

• FFmpeg Integration: Use VideoDecoder class from FFmpeg integration spec
• Viewport 2.0 Integration: Provide frame data to MPxDrawOverride via output attributes
• Caching: Interface with frame caching mechanism (to be specified separately)
• Post-effects: Apply transformations after decoding but before output

Threading Considerations

• Node computation occurs on Maya's main thread
• FFmpeg decoding should happen on background thread
• Use thread-safe mechanisms to transfer decoded frames to main thread
• Cache access must be thread-safe

Error Handling and Validation

• Validate video file path exists and is readable
• Check FFmpeg initialization success
• Handle end-of-file conditions (loop or stop)
• Provide error states via output attributes
• Log errors to Maya's script editor using MGlobal::displayError

Node Initialization and Cleanup

• initialize(): Define all attributes and set up attribute affects
• constructor(): Initialize member variables
• destructor(): Clean up FFmpeg decoder and frame data
• postConstructor(): Set node to be internally managed if needed

Attribute Affects

Specify which attributes affect which outputs:

• videoFile -> outFrameData, outFrameWidth, outFrameHeight, outFrameFormat, isValid
• frameRate -> outFrameData, outFrameTimestamp
• useMayaFrameRate -> outFrameData, outFrameTimestamp
• currentTime -> outFrameData, outFrameTimestamp
• loop -> outFrameData, outFrameTimestamp
• postEffect* -> outFrameData (if implemented as part of node computation)

Performance Considerations

• Minimize computation in compute() method
• Cache frequently accessed values
• Avoid expensive operations during node evaluation
• Use dirty propagation to minimize unnecessary recomputation

Maya API Specifics

• Use MFnTypedAttribute for message/string attributes
• Use MFnNumericAttribute for double/int/bool attributes
• Use MFnEnumAttribute for format options if needed
• Set appropriate attribute properties (keyable, storable, readable, writable)
• Use MTypeId for unique node identification
• Register node with MFnPlugin::registerNode

Dependencies

• FFmpeg integration component
• Frame caching mechanism
• Maya API 2023 (OpenMaya, OpenMayaFX)

Overview

This document specifies the implementation details for the Maya MPxNode plugin that handles video file input and frame output for the Image Plane Node.

Requirements

• Create a custom MPxNode that accepts video file path as input
• Output video frame data that can be consumed by viewport rendering
• Support attributes for frame rate control and playback rate adjustment
• Integrate with FFmpeg video decoding component
• Provide frame caching interface
• Follow Maya API best practices for node creation

Implementation Details

Node Definition

• Node Name: imagePlaneVideo
• Node ID: Unique ID obtained from Autodesk
• Classification: filter or image (to be determined based on Maya's categorization)
• Register as a Maya plugin using MFnPlugin

Attributes

Input Attributes

1. videoFile (message or string) - Path to the video file
2. frameRate (double) - Playback rate multiplier (1.0 = normal speed)
3. useMayaFrameRate (bool) - Whether to synchronize with Maya's timeline frame rate
4. currentTime (double) - Connection to Maya's timeline (for frame calculation)
5. loop (bool) - Whether to loop the video playback
6. postEffectCrop (double4) - Crop parameters (x, y, width, height)
7. postEffectResize (double2) - Resize parameters (width, height)
8. postEffectFlip (bool2) - Flip parameters (flipX, flipY)

Output Attributes

1. outFrameData (message or custom data type) - Pointer to video frame data
2. outFrameWidth (int) - Width of current frame
3. outFrameHeight (int) - Height of current frame
4. outFrameFormat (int) - Pixel format of current frame (RGB, RGBA, etc.)
5. outFrameTimestamp (double) - Timestamp of current frame
6. isValid (bool) - Whether the current frame data is valid

Core Components

VideoFrameData Class

Custom data class to hold video frame information:

• Pointer to pixel data (RGB/RGBA buffer)
• Width and height
• Pixel format
• Timestamp
• Reference counting for proper cleanup

Node Computation

In the compute() method:

1. Check if video file attribute has changed
2. If changed, initialize FFmpeg decoder with new file
3. Calculate target frame based on:
   • Maya's current time (if useMayaFrameRate is true)
   • Frame rate multiplier attribute
   • Loop settings
4. Request frame from FFmpeg decoder (or cache)
5. Apply post-effects if specified
6. Update output attributes with frame data
7. Mark node as clean

Integration Points

• FFmpeg Integration: Use VideoDecoder class from FFmpeg integration spec
• Viewport 2.0 Integration: Provide frame data to MPxDrawOverride via output attributes
• Caching: Interface with frame caching mechanism (to be specified separately)
• Post-effects: Apply transformations after decoding but before output

Threading Considerations

• Node computation occurs on Maya's main thread
• FFmpeg decoding should happen on background thread
• Use thread-safe mechanisms to transfer decoded frames to main thread
• Cache access must be thread-safe

Error Handling and Validation

• Validate video file path exists and is readable
• Check FFmpeg initialization success
• Handle end-of-file conditions (loop or stop)
• Provide error states via output attributes
• Log errors to Maya's script editor using MGlobal::displayError

Node Initialization and Cleanup

• initialize(): Define all attributes and set up attribute affects
• constructor(): Initialize member variables
• destructor(): Clean up FFmpeg decoder and frame data
• postConstructor(): Set node to be internally managed if needed

Attribute Affects

Specify which attributes affect which outputs:

• videoFile -> outFrameData, outFrameWidth, outFrameHeight, outFrameFormat, isValid
• frameRate -> outFrameData, outFrameTimestamp
• useMayaFrameRate -> outFrameData, outFrameTimestamp
• currentTime -> outFrameData, outFrameTimestamp
• loop -> outFrameData, outFrameTimestamp
• postEffect* -> outFrameData (if implemented as part of node computation)

Performance Considerations

• Minimize computation in compute() method
• Cache frequently accessed values
• Avoid expensive operations during node evaluation
• Use dirty propagation to minimize unnecessary recomputation

Maya API Specifics

• Use MFnTypedAttribute for message/string attributes
• Use MFnNumericAttribute for double/int/bool attributes
• Use MFnEnumAttribute for format options if needed
• Set appropriate attribute properties (keyable, storable, readable, writable)
• Use MTypeId for unique node identification
• Register node with MFnPlugin::registerNode

Dependencies

• FFmpeg integration component
• Frame caching mechanism
• Maya API 2023 (OpenMaya, OpenMayaFX)