10 KiB

Raw Permalink Blame History

Design Document

Overview

This design addresses the file path storage issue in the VFX Project Management System where absolute paths stored in the database become invalid when deploying to different environments, particularly Linux. The solution involves modifying the FileHandler and all file-related components to store relative paths and resolve them dynamically at runtime, ensuring cross-platform compatibility for all file types (submissions, attachments, project thumbnails, user avatars, and generated thumbnails) while maintaining backward compatibility with existing data.

Architecture

The solution follows a centralized approach where all file path operations go through the FileHandler class. The FileHandler will be enhanced with path resolution methods that can handle both relative and absolute paths, providing a smooth migration path.

Key Components:

FileHandler: Enhanced with relative path storage and dynamic resolution
Path Resolution Layer: New methods for converting between relative and absolute paths
Migration Utilities: Tools for converting existing absolute paths to relative paths
Database Migration Tools: Scripts to convert existing absolute paths to relative paths
Project Thumbnail Handler: Updated to use relative paths for project thumbnail storage
User Avatar Handler: Updated to use relative paths for user avatar storage
Avatar File Serving: New endpoint to serve user avatars with proper access control

Components and Interfaces

Enhanced FileHandler Class

class FileHandler:
    def store_relative_path(self, absolute_path: str) -> str:
        """Convert absolute path to relative path for database storage"""
        
    def resolve_absolute_path(self, stored_path: str) -> str:
        """Resolve stored path (relative or absolute) to absolute path"""
        
    def is_relative_path(self, path: str) -> bool:
        """Check if a path is relative to backend directory"""
        
    def migrate_path_to_relative(self, absolute_path: str) -> str:
        """Convert legacy absolute path to new relative format"""

File Serving Endpoints

The file serving endpoints in routers/files.py will be updated to use the new path resolution methods:

# Before: Direct path usage
file_path = submission.file_path

# After: Dynamic path resolution  
file_path = file_handler.resolve_absolute_path(submission.file_path)

Avatar File Serving

A new avatar serving endpoint will be added to routers/files.py:

@router.get("/users/{user_id}/avatar")
async def serve_user_avatar(user_id: int, db: Session, current_user: User):
    """Serve user avatar with access control"""
    # Resolve relative avatar path to absolute path for serving
    absolute_avatar_path = file_handler.resolve_absolute_path(user.avatar_url)

Database Schema

No changes to the database schema are required. The existing file_path columns will continue to store string paths, but the format will change from absolute to relative paths.

Data Models

Path Storage Format

Current (Absolute):

# Submissions/Attachments
/home/user/vfx-system/backend/uploads/submissions/123/v001_render_20241211_143022_a1b2c3d4.jpg

# Project Thumbnails  
/home/user/vfx-system/backend/uploads/project_thumbnails/project_1_20241211_143022_a1b2c3d4.jpg

# User Avatars
/home/user/vfx-system/backend/uploads/avatars/user_5_20241211_143022_a1b2c3d4.jpg

New (Relative):

# Submissions/Attachments
uploads/submissions/123/v001_render_20241211_143022_a1b2c3d4.jpg

# Project Thumbnails
uploads/project_thumbnails/project_1_20241211_143022_a1b2c3d4.jpg

# User Avatars
uploads/avatars/user_5_20241211_143022_a1b2c3d4.jpg

Path Resolution Logic

Storage: When saving files, store paths relative to backend directory in database
Retrieval: When accessing files, resolve relative paths to absolute paths for filesystem operations
Migration: Convert all existing absolute paths in database to relative paths

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system-essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

<function_calls> file-path-linux-fix

Property 1: File storage uses relative paths For any file saved through the FileHandler, the path stored in the database should be relative to the backend directory Validates: Requirements 1.1

Property 2: Path resolution produces valid absolute paths For any stored file path (relative or absolute), the FileHandler resolution should produce a valid absolute path that points to an existing file Validates: Requirements 1.2

Property 3: All thumbnail paths are relative For any thumbnail created by the system (submissions, attachments, projects, generated thumbnails), the thumbnail path stored in the database should be relative to the backend directory Validates: Requirements 1.3, 2.1

Property 4: Migration converts all paths to relative For any absolute path in the database before migration, after migration it should be converted to a relative path format Validates: Requirements 1.4

Property 5: File existence checks work with relative paths For any relative path stored in the database, file existence validation should correctly resolve and check the file Validates: Requirements 1.5

Property 6: All thumbnail URLs are accessible For any file with a thumbnail (submissions, attachments, projects), the thumbnail URL returned by the API should be accessible via HTTP request Validates: Requirements 2.2

Property 7: File serving resolves paths correctly For any file request to the serving endpoints, the system should resolve the stored path and serve the correct file Validates: Requirements 2.3

Property 8: Missing thumbnails are regenerated For any image file with a missing thumbnail, requesting the thumbnail should trigger regeneration from the original file Validates: Requirements 2.4

Property 9: Consistent relative path format For any multiple files stored in the system, all relative paths should follow the same format pattern Validates: Requirements 3.2

Property 10: System portability For any change in backend directory location, file resolution should continue to work without requiring database changes Validates: Requirements 3.4

Property 11: Migration preserves file access For any file accessible before migration, it should remain accessible after migration is complete Validates: Requirements 4.3

Property 12: Database migration completeness For any absolute path in the database before migration, after migration it should be converted to relative format Validates: Requirements 4.1

Property 13: Post-migration path consistency For any file operation performed after migration, the system should only use relative path logic Validates: Requirements 4.2, 4.5

Property 16: Project thumbnail paths are relative For any project thumbnail uploaded to the system, the thumbnail_path stored in the project table should be relative to the backend directory Validates: Requirements 1.1, 1.3

Property 17: User avatar paths are relative For any user avatar uploaded to the system, the avatar_url stored in the user table should be relative to the backend directory Validates: Requirements 2.4, 3.2

Property 18: Avatar serving resolves paths correctly For any avatar request to the serving endpoint, the system should resolve the stored relative path and serve the correct file Validates: Requirements 3.4

Property 19: Avatar migration converts paths to relative For any absolute avatar path in the database before migration, after migration it should be converted to relative format Validates: Requirements 1.4, 4.6

Property 20: Avatar file serving works after migration For any avatar accessible before migration, it should remain accessible via the serving endpoint after migration is complete Validates: Requirements 4.6

Error Handling

File Not Found Scenarios

Missing Original File: Return 404 with clear error message indicating the resolved absolute path
Missing Thumbnail: Attempt regeneration from original file, fallback to 404 if original is missing
Invalid Path Format: Log warning and attempt path resolution with fallback logic

Migration Error Handling

Path Conversion Failures: Log errors but continue migration, maintaining original paths for failed conversions
File System Errors: Validate file accessibility before and after path conversion
Database Transaction Failures: Rollback changes and maintain data integrity

Backward Compatibility Errors

Absolute Path Resolution: Try absolute path first, then relative path resolution
Mixed Path Formats: Handle gracefully by testing both resolution methods
Legacy Data Issues: Provide migration utilities to fix problematic paths

Testing Strategy

Unit Testing

Path conversion functions (absolute to relative, relative to absolute)
File existence validation with different path formats
Error handling for missing files and invalid paths
Migration utility functions

Property-Based Testing

The system will use pytest with hypothesis for property-based testing. Each property-based test will run a minimum of 100 iterations to ensure comprehensive coverage.

Property-based tests will be tagged with comments referencing the design document properties:

Format: # **Feature: file-path-linux-fix, Property {number}: {property_text}**

Integration Testing

End-to-end file upload and serving workflows
API endpoint responses with correct thumbnail URLs
Project thumbnail upload and serving workflows
Cross-platform deployment scenarios
Migration process validation

Test Data Management

Create test files with known absolute paths for migration testing
Generate various file types (images, videos, documents) for comprehensive testing
Test with different backend directory locations for portability validation

10 KiB Raw Permalink Blame History