# Video Frame Caching Specification ## Overview This document specifies the implementation details for video frame caching to enable smooth playback and random access to video frames. ## Requirements - Cache decoded video frames to avoid re-decoding during playback - Support random access to any frame in the video - Implement intelligent cache replacement policy (LRU or similar) - Handle memory limits to prevent excessive RAM usage - Provide thread-safe access for concurrent reading and writing - Support cache invalidation when video file changes - Enable prefetching of future frames for smooth playback - Support different cache policies (cache all, cache recent, cache keyframes only) ## Implementation Details ### Cache Design #### FrameCache Class Manages storage and retrieval of decoded video frames. Key methods: - `bool getFrame(int64_t frameIndex, AVFrame*& frame)` - Retrieve frame from cache - `bool putFrame(int64_t frameIndex, const AVFrame* frame)` - Store frame in cache - `void clear()` - Remove all frames from cache - `bool contains(int64_t frameIndex) const` - Check if frame is cached - `size_t size() const` - Number of frames currently cached - `size_t memoryUsage() const` - Total memory used by cached frames - `void setMaxMemorySize(size_t maxBytes)` - Set memory limit for cache - `void setMaxFrameCount(size_t maxFrames)` - Set maximum number of frames to cache #### Cache Entry Each cache entry contains: - Frame index (int64_t) - AVFrame pointer (with proper reference counting) - Timestamp of when frame was cached (for LRU) - Access count (for LFU variants) - Dirty flag (if frame has been modified by post-effects) ### Cache Algorithms #### LRU (Least Recently Used) - Remove least recently accessed frame when cache is full - Implement using doubly-linked list + hash map for O(1) operations - Update access time on both get and put operations #### LFU (Least Frequently Used) - Optional - Remove least frequently accessed frame when cache is full - May provide better performance for certain access patterns - More complex to implement than LRU #### FIFO (First In, First Out) - Optional - Remove oldest frame (by insertion time) when cache is full - Simple to implement but may not optimal for playback patterns ### Implementation Components #### Frame Data Management - Proper reference counting for AVFrame objects (av_frame_ref/av_frame_unref) - Deep copy of frame data when needed for post-processing - Handle different pixel formats correctly - Manage allocation and deallocation of frame buffers #### Thread Safety - Mutex protection for cache operations - Read-write lock for better concurrency (multiple readers, single writer) - Atomic operations for reference counting where possible - Lock-free techniques for high-performance scenarios (if needed) #### Cache Policies - **Cache All**: Attempt to cache every decoded frame (memory permitting) - **Cache Recent**: Cache only the most recently accessed frames (sliding window) - **Cache Keyframes Only**: Cache only I-frames for faster seeking - **Predictive Caching**: Prefetch frames ahead of current playback position ### Integration Points - FFmpeg Decoder: Stores decoded frames in cache, retrieves from cache when available - Maya Node: Requests frames from cache based on synchronization logic - Post-effects: Can operate on cached frames or request uncached frames - Viewport 2.0: Displays frames retrieved from cache ### Memory Management - Calculate memory usage based on frame dimensions and pixel format - Implement automatic eviction when memory limit exceeded - Provide statistics on cache hit/miss ratios - Allow configuration of memory limits via attributes or configuration file ### Error Handling - Handle out-of-memory conditions gracefully - Validate frame indices (negative, beyond video duration) - Handle cache corruption (though unlikely in memory-only cache) - Provide fallback to direct decoding when cache operations fail ### Performance Considerations - Minimize lock contention in multithreaded scenarios - Use efficient hash functions for frame index lookup - Consider using memory pools for frame allocations - Optimize for sequential access patterns (common in video playback) - Implement asynchronous prefetching to hide latency ### Maya API Specifics - Expose cache statistics as node attributes for debugging - Provide attributes to control cache behavior (size limit, policy) - Implement cache clearing when video file attribute changes - Use MGlobal::displayInfo for cache statistics reporting ## Dependencies - FFmpeg libraries (for AVFrame management) - MPxNode implementation - Standard C++ library (unordered_map, mutex, etc.) ## Overview This document specifies the implementation details for video frame caching to enable smooth playback and random access to video frames. ## Requirements - Cache decoded video frames to avoid re-decoding during playback - Support random access to any frame in the video - Implement intelligent cache replacement policy (LRU or similar) - Handle memory limits to prevent excessive RAM usage - Provide thread-safe access for concurrent reading and writing - Support cache invalidation when video file changes - Enable prefetching of future frames for smooth playback - Support different cache policies (cache all, cache recent, cache keyframes only) ## Implementation Details ### Cache Design #### FrameCache Class Manages storage and retrieval of decoded video frames. Key methods: - `bool getFrame(int64_t frameIndex, AVFrame*& frame)` - Retrieve frame from cache - `bool putFrame(int64_t frameIndex, const AVFrame* frame)` - Store frame in cache - `void clear()` - Remove all frames from cache - `bool contains(int64_t frameIndex) const` - Check if frame is cached - `size_t size() const` - Number of frames currently cached - `size_t memoryUsage() const` - Total memory used by cached frames - `void setMaxMemorySize(size_t maxBytes)` - Set memory limit for cache - `void setMaxFrameCount(size_t maxFrames)` - Set maximum number of frames to cache #### Cache Entry Each cache entry contains: - Frame index (int64_t) - AVFrame pointer (with proper reference counting) - Timestamp of when frame was cached (for LRU) - Access count (for LFU variants) - Dirty flag (if frame has been modified by post-effects) ### Cache Algorithms #### LRU (Least Recently Used) - Remove least recently accessed frame when cache is full - Implement using doubly-linked list + hash map for O(1) operations - Update access time on both get and put operations #### LFU (Least Frequently Used) - Optional - Remove least frequently accessed frame when cache is full - May provide better performance for certain access patterns - More complex to implement than LRU #### FIFO (First In, First Out) - Optional - Remove oldest frame (by insertion time) when cache is full - Simple to implement but may not optimal for playback patterns ### Implementation Components #### Frame Data Management - Proper reference counting for AVFrame objects (av_frame_ref/av_frame_unref) - Deep copy of frame data when needed for post-processing - Handle different pixel formats correctly - Manage allocation and deallocation of frame buffers #### Thread Safety - Mutex protection for cache operations - Read-write lock for better concurrency (multiple readers, single writer) - Atomic operations for reference counting where possible - Lock-free techniques for high-performance scenarios (if needed) #### Cache Policies - **Cache All**: Attempt to cache every decoded frame (memory permitting) - **Cache Recent**: Cache only the most recently accessed frames (sliding window) - **Cache Keyframes Only**: Cache only I-frames for faster seeking - **Predictive Caching**: Prefetch frames ahead of current playback position ### Integration Points - FFmpeg Decoder: Stores decoded frames in cache, retrieves from cache when available - Maya Node: Requests frames from cache based on synchronization logic - Post-effects: Can operate on cached frames or request uncached frames - Viewport 2.0: Displays frames retrieved from cache ### Memory Management - Calculate memory usage based on frame dimensions and pixel format - Implement automatic eviction when memory limit exceeded - Provide statistics on cache hit/miss ratios - Allow configuration of memory limits via attributes or configuration file ### Error Handling - Handle out-of-memory conditions gracefully - Validate frame indices (negative, beyond video duration) - Handle cache corruption (though unlikely in memory-only cache) - Provide fallback to direct decoding when cache operations fail ### Performance Considerations - Minimize lock contention in multithreaded scenarios - Use efficient hash functions for frame index lookup - Consider using memory pools for frame allocations - Optimize for sequential access patterns (common in video playback) - Implement asynchronous prefetching to hide latency ### Maya API Specifics - Expose cache statistics as node attributes for debugging - Provide attributes to control cache behavior (size limit, policy) - Implement cache clearing when video file attribute changes - Use MGlobal::displayInfo for cache statistics reporting ## Dependencies - FFmpeg libraries (for AVFrame management) - MPxNode implementation - Standard C++ library (unordered_map, mutex, etc.)