MediaPlane/docs/Frame_Rate_Synchronization_Spec.md

Frame Rate Synchronization Specification

Overview

This document specifies the implementation details for synchronizing video playback with Maya's timeline frame rate and providing user-adjustable playback rate control.

Requirements

• Read Maya's current frame rate from the timeline
• Synchronize video frame advancement with Maya's timeline when enabled
• Allow user to override playback speed with a multiplier
• Support looping and non-looping playback modes
• Handle frame rate changes in Maya's timeline dynamically
• Provide smooth playback even when Maya's frame rate differs from video's native frame rate

Implementation Details

Frame Rate Sources

1. Maya's Timeline Frame Rate: Obtained from MTime::uiUnit() or MTime::getFrameRate()
2. Video's Native Frame Rate: Obtained from FFmpeg decoder (getFrameRate() method)
3. User Playback Rate Multiplier: Custom attribute on the MPxNode

Core Logic

Frame Calculation Algorithm

When useMayaFrameRate is enabled:

effectiveFrameRate = mayaFrameRate * userPlaybackRate
targetFrame = (currentTime - startTime) * effectiveFrameRate

When useMayaFrameRate is disabled:

effectiveFrameRate = videoFrameRate * userPlaybackRate
targetFrame = (currentTime - startTime) * effectiveFrameRate

Where:

• currentTime: Maya's current time in seconds
• startTime: Time when video playback started or was reset
• targetFrame: Frame number to display (0-based)
• mayaFrameRate: Frames per second from Maya's timeline
• videoFrameRate: Native frames per second from video file
• userPlaybackRate: User-defined multiplier (1.0 = normal speed)

Implementation Components

Time Management Class

Handles time calculations and frame targeting:

• Store start time when playback begins or resets
• Calculate target frame based on current time and settings
• Handle looping by wrapping target frame within video duration
• Provide methods to get current frame with sub-frame precision for interpolation

Attribute Definitions

In the MPxNode:

• useMayaFrameRate (bool): Toggle between Maya-synced and video-native frame rates
• playbackRate (double): User-adjustable playback rate multiplier (default 1.0)
• startTime (double): Internal attribute to track when playback started
• isPlaying (bool): Internal attribute to track playback state

Integration with Maya Timeline

• Connect to Maya's timeChanged event via node attributes
• Use currentTime input attribute driven by Maya's timeline
• Update node computation when timeline changes
• Handle scrubbing (jumping to different times) correctly

Threading Considerations

• Time calculations occur on Maya's main thread during node computation
• No shared state with decoding thread except for frame requests
• Frame requests to decoder should be thread-safe

Error Handling

• Handle invalid frame rates (zero or negative)
• Clamp playback rate to reasonable range (e.g., 0.01 to 10.0)
• Handle case where video frame rate is unavailable
• Graceful degradation to approximate synchronization

Performance Considerations

• Minimize calculations in node compute method
• Cache Maya's frame rate when possible (update only when changed)
• Avoid expensive trigonometric or transcendental functions in real-time paths

Integration Points

• MPxNode: Provides frame calculation logic and attributes
• FFmpeg Decoder: Receives frame requests based on calculated target frame
• Viewport 2.0: Displays the frame requested by the synchronization logic
• Caching: Requests frames from cache based on target frame

Maya API Specifics

• Use MTime class for time manipulations
• Use MAnimControl to get Maya's current time if not using attribute connection
• Use MTime::uiUnit() to get current UI time unit
• Register time changed callbacks if needed for more responsive updates

Dependencies

• Maya API 2023 (MTime, MAnimControl)
• MPxNode implementation
• FFmpeg decoder interface

Overview

This document specifies the implementation details for synchronizing video playback with Maya's timeline frame rate and providing user-adjustable playback rate control.

Requirements

• Read Maya's current frame rate from the timeline
• Synchronize video frame advancement with Maya's timeline when enabled
• Allow user to override playback speed with a multiplier
• Support looping and non-looping playback modes
• Handle frame rate changes in Maya's timeline dynamically
• Provide smooth playback even when Maya's frame rate differs from video's native frame rate

Implementation Details

Frame Rate Sources

1. Maya's Timeline Frame Rate: Obtained from MTime::uiUnit() or MTime::getFrameRate()
2. Video's Native Frame Rate: Obtained from FFmpeg decoder (getFrameRate() method)
3. User Playback Rate Multiplier: Custom attribute on the MPxNode

Core Logic

Frame Calculation Algorithm

When useMayaFrameRate is enabled:

effectiveFrameRate = mayaFrameRate * userPlaybackRate
targetFrame = (currentTime - startTime) * effectiveFrameRate

When useMayaFrameRate is disabled:

effectiveFrameRate = videoFrameRate * userPlaybackRate
targetFrame = (currentTime - startTime) * effectiveFrameRate

Where:

• currentTime: Maya's current time in seconds
• startTime: Time when video playback started or was reset
• targetFrame: Frame number to display (0-based)
• mayaFrameRate: Frames per second from Maya's timeline
• videoFrameRate: Native frames per second from video file
• userPlaybackRate: User-defined multiplier (1.0 = normal speed)

Implementation Components

Time Management Class

Handles time calculations and frame targeting:

• Store start time when playback begins or resets
• Calculate target frame based on current time and settings
• Handle looping by wrapping target frame within video duration
• Provide methods to get current frame with sub-frame precision for interpolation

Attribute Definitions

In the MPxNode:

• useMayaFrameRate (bool): Toggle between Maya-synced and video-native frame rates
• playbackRate (double): User-adjustable playback rate multiplier (default 1.0)
• startTime (double): Internal attribute to track when playback started
• isPlaying (bool): Internal attribute to track playback state

Integration with Maya Timeline

• Connect to Maya's timeChanged event via node attributes
• Use currentTime input attribute driven by Maya's timeline
• Update node computation when timeline changes
• Handle scrubbing (jumping to different times) correctly

Threading Considerations

• Time calculations occur on Maya's main thread during node computation
• No shared state with decoding thread except for frame requests
• Frame requests to decoder should be thread-safe

Error Handling

• Handle invalid frame rates (zero or negative)
• Clamp playback rate to reasonable range (e.g., 0.01 to 10.0)
• Handle case where video frame rate is unavailable
• Graceful degradation to approximate synchronization

Performance Considerations

• Minimize calculations in node compute method
• Cache Maya's frame rate when possible (update only when changed)
• Avoid expensive trigonometric or transcendental functions in real-time paths

Integration Points

• MPxNode: Provides frame calculation logic and attributes
• FFmpeg Decoder: Receives frame requests based on calculated target frame
• Viewport 2.0: Displays the frame requested by the synchronization logic
• Caching: Requests frames from cache based on target frame

Maya API Specifics

• Use MTime class for time manipulations
• Use MAnimControl to get Maya's current time if not using attribute connection
• Use MTime::uiUnit() to get current UI time unit
• Register time changed callbacks if needed for more responsive updates

Dependencies

• Maya API 2023 (MTime, MAnimControl)
• MPxNode implementation
• FFmpeg decoder interface