221 lines
9.1 KiB
Markdown
221 lines
9.1 KiB
Markdown
# Video Frame Caching Specification
|
|
|
|
## Overview
|
|
This document specifies the implementation details for video frame caching to enable smooth playback and random access to video frames.
|
|
|
|
## Requirements
|
|
- Cache decoded video frames to avoid re-decoding during playback
|
|
- Support random access to any frame in the video
|
|
- Implement intelligent cache replacement policy (LRU or similar)
|
|
- Handle memory limits to prevent excessive RAM usage
|
|
- Provide thread-safe access for concurrent reading and writing
|
|
- Support cache invalidation when video file changes
|
|
- Enable prefetching of future frames for smooth playback
|
|
- Support different cache policies (cache all, cache recent, cache keyframes only)
|
|
|
|
## Implementation Details
|
|
|
|
### Cache Design
|
|
|
|
#### FrameCache Class
|
|
Manages storage and retrieval of decoded video frames.
|
|
|
|
Key methods:
|
|
- `bool getFrame(int64_t frameIndex, AVFrame*& frame)` - Retrieve frame from cache
|
|
- `bool putFrame(int64_t frameIndex, const AVFrame* frame)` - Store frame in cache
|
|
- `void clear()` - Remove all frames from cache
|
|
- `bool contains(int64_t frameIndex) const` - Check if frame is cached
|
|
- `size_t size() const` - Number of frames currently cached
|
|
- `size_t memoryUsage() const` - Total memory used by cached frames
|
|
- `void setMaxMemorySize(size_t maxBytes)` - Set memory limit for cache
|
|
- `void setMaxFrameCount(size_t maxFrames)` - Set maximum number of frames to cache
|
|
|
|
#### Cache Entry
|
|
Each cache entry contains:
|
|
- Frame index (int64_t)
|
|
- AVFrame pointer (with proper reference counting)
|
|
- Timestamp of when frame was cached (for LRU)
|
|
- Access count (for LFU variants)
|
|
- Dirty flag (if frame has been modified by post-effects)
|
|
|
|
### Cache Algorithms
|
|
|
|
#### LRU (Least Recently Used)
|
|
- Remove least recently accessed frame when cache is full
|
|
- Implement using doubly-linked list + hash map for O(1) operations
|
|
- Update access time on both get and put operations
|
|
|
|
#### LFU (Least Frequently Used) - Optional
|
|
- Remove least frequently accessed frame when cache is full
|
|
- May provide better performance for certain access patterns
|
|
- More complex to implement than LRU
|
|
|
|
#### FIFO (First In, First Out) - Optional
|
|
- Remove oldest frame (by insertion time) when cache is full
|
|
- Simple to implement but may not optimal for playback patterns
|
|
|
|
### Implementation Components
|
|
|
|
#### Frame Data Management
|
|
- Proper reference counting for AVFrame objects (av_frame_ref/av_frame_unref)
|
|
- Deep copy of frame data when needed for post-processing
|
|
- Handle different pixel formats correctly
|
|
- Manage allocation and deallocation of frame buffers
|
|
|
|
#### Thread Safety
|
|
- Mutex protection for cache operations
|
|
- Read-write lock for better concurrency (multiple readers, single writer)
|
|
- Atomic operations for reference counting where possible
|
|
- Lock-free techniques for high-performance scenarios (if needed)
|
|
|
|
#### Cache Policies
|
|
- **Cache All**: Attempt to cache every decoded frame (memory permitting)
|
|
- **Cache Recent**: Cache only the most recently accessed frames (sliding window)
|
|
- **Cache Keyframes Only**: Cache only I-frames for faster seeking
|
|
- **Predictive Caching**: Prefetch frames ahead of current playback position
|
|
|
|
### Integration Points
|
|
- FFmpeg Decoder: Stores decoded frames in cache, retrieves from cache when available
|
|
- Maya Node: Requests frames from cache based on synchronization logic
|
|
- Post-effects: Can operate on cached frames or request uncached frames
|
|
- Viewport 2.0: Displays frames retrieved from cache
|
|
|
|
### Memory Management
|
|
- Calculate memory usage based on frame dimensions and pixel format
|
|
- Implement automatic eviction when memory limit exceeded
|
|
- Provide statistics on cache hit/miss ratios
|
|
- Allow configuration of memory limits via attributes or configuration file
|
|
|
|
### Error Handling
|
|
- Handle out-of-memory conditions gracefully
|
|
- Validate frame indices (negative, beyond video duration)
|
|
- Handle cache corruption (though unlikely in memory-only cache)
|
|
- Provide fallback to direct decoding when cache operations fail
|
|
|
|
### Performance Considerations
|
|
- Minimize lock contention in multithreaded scenarios
|
|
- Use efficient hash functions for frame index lookup
|
|
- Consider using memory pools for frame allocations
|
|
- Optimize for sequential access patterns (common in video playback)
|
|
- Implement asynchronous prefetching to hide latency
|
|
|
|
### Maya API Specifics
|
|
- Expose cache statistics as node attributes for debugging
|
|
- Provide attributes to control cache behavior (size limit, policy)
|
|
- Implement cache clearing when video file attribute changes
|
|
- Use MGlobal::displayInfo for cache statistics reporting
|
|
|
|
## Dependencies
|
|
- FFmpeg libraries (for AVFrame management)
|
|
- MPxNode implementation
|
|
- Standard C++ library (unordered_map, mutex, etc.)
|
|
## Overview
|
|
This document specifies the implementation details for video frame caching to enable smooth playback and random access to video frames.
|
|
|
|
## Requirements
|
|
- Cache decoded video frames to avoid re-decoding during playback
|
|
- Support random access to any frame in the video
|
|
- Implement intelligent cache replacement policy (LRU or similar)
|
|
- Handle memory limits to prevent excessive RAM usage
|
|
- Provide thread-safe access for concurrent reading and writing
|
|
- Support cache invalidation when video file changes
|
|
- Enable prefetching of future frames for smooth playback
|
|
- Support different cache policies (cache all, cache recent, cache keyframes only)
|
|
|
|
## Implementation Details
|
|
|
|
### Cache Design
|
|
|
|
#### FrameCache Class
|
|
Manages storage and retrieval of decoded video frames.
|
|
|
|
Key methods:
|
|
- `bool getFrame(int64_t frameIndex, AVFrame*& frame)` - Retrieve frame from cache
|
|
- `bool putFrame(int64_t frameIndex, const AVFrame* frame)` - Store frame in cache
|
|
- `void clear()` - Remove all frames from cache
|
|
- `bool contains(int64_t frameIndex) const` - Check if frame is cached
|
|
- `size_t size() const` - Number of frames currently cached
|
|
- `size_t memoryUsage() const` - Total memory used by cached frames
|
|
- `void setMaxMemorySize(size_t maxBytes)` - Set memory limit for cache
|
|
- `void setMaxFrameCount(size_t maxFrames)` - Set maximum number of frames to cache
|
|
|
|
#### Cache Entry
|
|
Each cache entry contains:
|
|
- Frame index (int64_t)
|
|
- AVFrame pointer (with proper reference counting)
|
|
- Timestamp of when frame was cached (for LRU)
|
|
- Access count (for LFU variants)
|
|
- Dirty flag (if frame has been modified by post-effects)
|
|
|
|
### Cache Algorithms
|
|
|
|
#### LRU (Least Recently Used)
|
|
- Remove least recently accessed frame when cache is full
|
|
- Implement using doubly-linked list + hash map for O(1) operations
|
|
- Update access time on both get and put operations
|
|
|
|
#### LFU (Least Frequently Used) - Optional
|
|
- Remove least frequently accessed frame when cache is full
|
|
- May provide better performance for certain access patterns
|
|
- More complex to implement than LRU
|
|
|
|
#### FIFO (First In, First Out) - Optional
|
|
- Remove oldest frame (by insertion time) when cache is full
|
|
- Simple to implement but may not optimal for playback patterns
|
|
|
|
### Implementation Components
|
|
|
|
#### Frame Data Management
|
|
- Proper reference counting for AVFrame objects (av_frame_ref/av_frame_unref)
|
|
- Deep copy of frame data when needed for post-processing
|
|
- Handle different pixel formats correctly
|
|
- Manage allocation and deallocation of frame buffers
|
|
|
|
#### Thread Safety
|
|
- Mutex protection for cache operations
|
|
- Read-write lock for better concurrency (multiple readers, single writer)
|
|
- Atomic operations for reference counting where possible
|
|
- Lock-free techniques for high-performance scenarios (if needed)
|
|
|
|
#### Cache Policies
|
|
- **Cache All**: Attempt to cache every decoded frame (memory permitting)
|
|
- **Cache Recent**: Cache only the most recently accessed frames (sliding window)
|
|
- **Cache Keyframes Only**: Cache only I-frames for faster seeking
|
|
- **Predictive Caching**: Prefetch frames ahead of current playback position
|
|
|
|
### Integration Points
|
|
- FFmpeg Decoder: Stores decoded frames in cache, retrieves from cache when available
|
|
- Maya Node: Requests frames from cache based on synchronization logic
|
|
- Post-effects: Can operate on cached frames or request uncached frames
|
|
- Viewport 2.0: Displays frames retrieved from cache
|
|
|
|
### Memory Management
|
|
- Calculate memory usage based on frame dimensions and pixel format
|
|
- Implement automatic eviction when memory limit exceeded
|
|
- Provide statistics on cache hit/miss ratios
|
|
- Allow configuration of memory limits via attributes or configuration file
|
|
|
|
### Error Handling
|
|
- Handle out-of-memory conditions gracefully
|
|
- Validate frame indices (negative, beyond video duration)
|
|
- Handle cache corruption (though unlikely in memory-only cache)
|
|
- Provide fallback to direct decoding when cache operations fail
|
|
|
|
### Performance Considerations
|
|
- Minimize lock contention in multithreaded scenarios
|
|
- Use efficient hash functions for frame index lookup
|
|
- Consider using memory pools for frame allocations
|
|
- Optimize for sequential access patterns (common in video playback)
|
|
- Implement asynchronous prefetching to hide latency
|
|
|
|
### Maya API Specifics
|
|
- Expose cache statistics as node attributes for debugging
|
|
- Provide attributes to control cache behavior (size limit, policy)
|
|
- Implement cache clearing when video file attribute changes
|
|
- Use MGlobal::displayInfo for cache statistics reporting
|
|
|
|
## Dependencies
|
|
- FFmpeg libraries (for AVFrame management)
|
|
- MPxNode implementation
|
|
- Standard C++ library (unordered_map, mutex, etc.)
|