indigo/LinkDesk
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Init Repo

Browse Source
This commit is contained in:
indigo
2026-02-28 03:22:04 +08:00
commit de59b57ee7
883 changed files with 156857 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/file-path-linux-fix/design.md
+222
View File
@@ -0,0 +1,222 @@
# Design Document
## Overview
This design addresses the file path storage issue in the VFX Project Management System where absolute paths stored in the database become invalid when deploying to different environments, particularly Linux. The solution involves modifying the FileHandler and all file-related components to store relative paths and resolve them dynamically at runtime, ensuring cross-platform compatibility for all file types (submissions, attachments, project thumbnails, user avatars, and generated thumbnails) while maintaining backward compatibility with existing data.
## Architecture
The solution follows a centralized approach where all file path operations go through the FileHandler class. The FileHandler will be enhanced with path resolution methods that can handle both relative and absolute paths, providing a smooth migration path.
### Key Components:
- **FileHandler**: Enhanced with relative path storage and dynamic resolution
- **Path Resolution Layer**: New methods for converting between relative and absolute paths
- **Migration Utilities**: Tools for converting existing absolute paths to relative paths
- **Database Migration Tools**: Scripts to convert existing absolute paths to relative paths
- **Project Thumbnail Handler**: Updated to use relative paths for project thumbnail storage
- **User Avatar Handler**: Updated to use relative paths for user avatar storage
- **Avatar File Serving**: New endpoint to serve user avatars with proper access control
## Components and Interfaces
### Enhanced FileHandler Class
```python
class FileHandler:
def store_relative_path(self, absolute_path: str) -> str:
"""Convert absolute path to relative path for database storage"""
def resolve_absolute_path(self, stored_path: str) -> str:
"""Resolve stored path (relative or absolute) to absolute path"""
def is_relative_path(self, path: str) -> bool:
"""Check if a path is relative to backend directory"""
def migrate_path_to_relative(self, absolute_path: str) -> str:
"""Convert legacy absolute path to new relative format"""
```
### File Serving Endpoints
The file serving endpoints in `routers/files.py` will be updated to use the new path resolution methods:
```python
# Before: Direct path usage
file_path = submission.file_path
# After: Dynamic path resolution
file_path = file_handler.resolve_absolute_path(submission.file_path)
```
### Avatar File Serving
A new avatar serving endpoint will be added to `routers/files.py`:
```python
@router.get("/users/{user_id}/avatar")
async def serve_user_avatar(user_id: int, db: Session, current_user: User):
"""Serve user avatar with access control"""
# Resolve relative avatar path to absolute path for serving
absolute_avatar_path = file_handler.resolve_absolute_path(user.avatar_url)
```
### Database Schema
No changes to the database schema are required. The existing `file_path` columns will continue to store string paths, but the format will change from absolute to relative paths.
## Data Models
### Path Storage Format
**Current (Absolute):**
```
# Submissions/Attachments
/home/user/vfx-system/backend/uploads/submissions/123/v001_render_20241211_143022_a1b2c3d4.jpg
# Project Thumbnails
/home/user/vfx-system/backend/uploads/project_thumbnails/project_1_20241211_143022_a1b2c3d4.jpg
# User Avatars
/home/user/vfx-system/backend/uploads/avatars/user_5_20241211_143022_a1b2c3d4.jpg
```
**New (Relative):**
```
# Submissions/Attachments
uploads/submissions/123/v001_render_20241211_143022_a1b2c3d4.jpg
# Project Thumbnails
uploads/project_thumbnails/project_1_20241211_143022_a1b2c3d4.jpg
# User Avatars
uploads/avatars/user_5_20241211_143022_a1b2c3d4.jpg
```
### Path Resolution Logic
1. **Storage**: When saving files, store paths relative to backend directory in database
2. **Retrieval**: When accessing files, resolve relative paths to absolute paths for filesystem operations
3. **Migration**: Convert all existing absolute paths in database to relative paths
## Correctness Properties
*A property is a characteristic or behavior that should hold true across all valid executions of a system-essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
<function_calls>
<invoke name="prework">
<parameter name="featureName">file-path-linux-fix
Property 1: File storage uses relative paths
*For any* file saved through the FileHandler, the path stored in the database should be relative to the backend directory
**Validates: Requirements 1.1**
Property 2: Path resolution produces valid absolute paths
*For any* stored file path (relative or absolute), the FileHandler resolution should produce a valid absolute path that points to an existing file
**Validates: Requirements 1.2**
Property 3: All thumbnail paths are relative
*For any* thumbnail created by the system (submissions, attachments, projects, generated thumbnails), the thumbnail path stored in the database should be relative to the backend directory
**Validates: Requirements 1.3, 2.1**
Property 4: Migration converts all paths to relative
*For any* absolute path in the database before migration, after migration it should be converted to a relative path format
**Validates: Requirements 1.4**
Property 5: File existence checks work with relative paths
*For any* relative path stored in the database, file existence validation should correctly resolve and check the file
**Validates: Requirements 1.5**
Property 6: All thumbnail URLs are accessible
*For any* file with a thumbnail (submissions, attachments, projects), the thumbnail URL returned by the API should be accessible via HTTP request
**Validates: Requirements 2.2**
Property 7: File serving resolves paths correctly
*For any* file request to the serving endpoints, the system should resolve the stored path and serve the correct file
**Validates: Requirements 2.3**
Property 8: Missing thumbnails are regenerated
*For any* image file with a missing thumbnail, requesting the thumbnail should trigger regeneration from the original file
**Validates: Requirements 2.4**
Property 9: Consistent relative path format
*For any* multiple files stored in the system, all relative paths should follow the same format pattern
**Validates: Requirements 3.2**
Property 10: System portability
*For any* change in backend directory location, file resolution should continue to work without requiring database changes
**Validates: Requirements 3.4**
Property 11: Migration preserves file access
*For any* file accessible before migration, it should remain accessible after migration is complete
**Validates: Requirements 4.3**
Property 12: Database migration completeness
*For any* absolute path in the database before migration, after migration it should be converted to relative format
**Validates: Requirements 4.1**
Property 13: Post-migration path consistency
*For any* file operation performed after migration, the system should only use relative path logic
**Validates: Requirements 4.2, 4.5**
Property 16: Project thumbnail paths are relative
*For any* project thumbnail uploaded to the system, the thumbnail_path stored in the project table should be relative to the backend directory
**Validates: Requirements 1.1, 1.3**
Property 17: User avatar paths are relative
*For any* user avatar uploaded to the system, the avatar_url stored in the user table should be relative to the backend directory
**Validates: Requirements 2.4, 3.2**
Property 18: Avatar serving resolves paths correctly
*For any* avatar request to the serving endpoint, the system should resolve the stored relative path and serve the correct file
**Validates: Requirements 3.4**
Property 19: Avatar migration converts paths to relative
*For any* absolute avatar path in the database before migration, after migration it should be converted to relative format
**Validates: Requirements 1.4, 4.6**
Property 20: Avatar file serving works after migration
*For any* avatar accessible before migration, it should remain accessible via the serving endpoint after migration is complete
**Validates: Requirements 4.6**
## Error Handling
### File Not Found Scenarios
- **Missing Original File**: Return 404 with clear error message indicating the resolved absolute path
- **Missing Thumbnail**: Attempt regeneration from original file, fallback to 404 if original is missing
- **Invalid Path Format**: Log warning and attempt path resolution with fallback logic
### Migration Error Handling
- **Path Conversion Failures**: Log errors but continue migration, maintaining original paths for failed conversions
- **File System Errors**: Validate file accessibility before and after path conversion
- **Database Transaction Failures**: Rollback changes and maintain data integrity
### Backward Compatibility Errors
- **Absolute Path Resolution**: Try absolute path first, then relative path resolution
- **Mixed Path Formats**: Handle gracefully by testing both resolution methods
- **Legacy Data Issues**: Provide migration utilities to fix problematic paths
## Testing Strategy
### Unit Testing
- Path conversion functions (absolute to relative, relative to absolute)
- File existence validation with different path formats
- Error handling for missing files and invalid paths
- Migration utility functions
### Property-Based Testing
The system will use **pytest** with **hypothesis** for property-based testing. Each property-based test will run a minimum of 100 iterations to ensure comprehensive coverage.
Property-based tests will be tagged with comments referencing the design document properties:
- Format: `# **Feature: file-path-linux-fix, Property {number}: {property_text}**`
### Integration Testing
- End-to-end file upload and serving workflows
- API endpoint responses with correct thumbnail URLs
- Project thumbnail upload and serving workflows
- Cross-platform deployment scenarios
- Migration process validation
### Test Data Management
- Create test files with known absolute paths for migration testing
- Generate various file types (images, videos, documents) for comprehensive testing
- Test with different backend directory locations for portability validation
.kiro/specs/file-path-linux-fix/requirements.md
+81
View File
@@ -0,0 +1,81 @@
# Requirements Document
## Introduction
The VFX Project Management System currently stores absolute file paths in the database for submissions, attachments, and thumbnails. When deploying to Linux environments, these absolute paths become invalid, causing thumbnail URLs and file serving to fail. The system needs to store relative file paths and resolve them dynamically at runtime to ensure cross-platform compatibility.
## Glossary
- **File_Handler**: The utility class responsible for file upload, storage, and path management
- **Submission**: User-uploaded work files (videos, images) with version control
- **Attachment**: Supporting files attached to tasks (documents, references)
- **Thumbnail**: Generated preview images for visual files
- **Avatar**: User profile image stored in the system
- **Backend_Directory**: The root directory of the backend application
- **Relative_Path**: File path stored relative to the backend directory
- **Absolute_Path**: Complete file system path resolved at runtime
## Requirements
### Requirement 1
**User Story:** As a system administrator, I want to migrate existing database records with absolute file paths to relative paths, so that thumbnail URLs work correctly on Linux deployments.
#### Acceptance Criteria
1. WHEN the migration script processes the submissions table THEN it SHALL convert all file_path values from absolute to relative paths
2. WHEN the migration script processes the task_attachments table THEN it SHALL convert all file_path values from absolute to relative paths
3. WHEN the migration script processes the projects table THEN it SHALL convert all thumbnail_path values from absolute to relative paths
4. WHEN the migration script processes the users table THEN it SHALL convert all avatar_url values from absolute to relative paths
5. WHEN the migration encounters a path that cannot be converted THEN it SHALL log the error and continue processing
6. WHEN the migration is complete THEN all file paths in the database SHALL be relative to the backend directory
### Requirement 2
**User Story:** As a system administrator, I want file paths to be stored relative to the backend directory, so that the application works correctly when deployed across different environments.
#### Acceptance Criteria
1. WHEN the system saves any file THEN the File_Handler SHALL store the file path relative to the Backend_Directory in the database
2. WHEN the system serves any file THEN the File_Handler SHALL resolve the relative path to an absolute path at runtime
3. WHEN the system creates any thumbnails THEN the thumbnail paths SHALL be stored relative to the Backend_Directory
4. WHEN the system migrates existing data THEN all absolute paths SHALL be converted to relative paths
5. WHEN the system validates file existence THEN it SHALL resolve relative paths to absolute paths for filesystem operations
### Requirement 2
**User Story:** As a developer, I want the FileHandler to only use relative path logic, so that the system is simplified and works consistently across all environments.
#### Acceptance Criteria
1. WHEN the system saves any file THEN it SHALL store only relative paths in the database
2. WHEN the system serves any file THEN it SHALL resolve relative paths to absolute paths for filesystem access
3. WHEN the system creates thumbnails THEN it SHALL store only relative paths for thumbnail locations
4. WHEN the system saves user avatars THEN it SHALL store only relative paths for avatar locations
5. WHEN the FileHandler validates file existence THEN it SHALL resolve relative paths to absolute paths
6. WHEN the system encounters any file path THEN it SHALL assume the path is relative to the backend directory
### Requirement 3
**User Story:** As a user, I want all thumbnail URLs (submissions, attachments, projects, and generated thumbnails) to work correctly on Linux deployments, so that I can preview uploaded files regardless of the deployment environment.
#### Acceptance Criteria
1. WHEN a user uploads any image file THEN the system SHALL generate a thumbnail with a relative path
2. WHEN a user uploads an avatar image THEN the system SHALL store the avatar with a relative path
3. WHEN the API returns any data with thumbnails or avatars THEN the URLs SHALL be accessible via the file serving endpoints
4. WHEN the file serving endpoint receives any thumbnail or avatar request THEN it SHALL resolve the relative path and serve the correct file
5. WHEN any thumbnail or avatar file is missing THEN the system SHALL return a 404 error with appropriate messaging
### Requirement 4
**User Story:** As a system administrator, I want to migrate existing database records to use relative paths only, so that the system is simplified and works consistently across all deployments.
#### Acceptance Criteria
1. WHEN the migration script runs THEN it SHALL convert all absolute paths in the database to relative paths
2. WHEN the migration is complete THEN the system SHALL only use relative path logic for all file operations
3. WHEN the system processes any file path THEN it SHALL assume the path is relative to the Backend_Directory
4. WHEN the migration encounters invalid paths THEN it SHALL log errors but continue processing other records
5. WHEN the migration is complete THEN all file serving endpoints SHALL work with the converted relative paths
6. WHEN the migration processes avatar URLs THEN it SHALL convert them to relative paths and ensure avatar serving works correctly
.kiro/specs/file-path-linux-fix/tasks.md
+164
View File
@@ -0,0 +1,164 @@
# Implementation Plan
- [x] 1. Create database migration script for existing file paths
- Create script to convert absolute paths to relative paths in submissions table
- Create script to convert absolute paths to relative paths in task_attachments table
- Create script to convert absolute paths to relative paths in projects table (thumbnail_path)
- Create script to convert absolute paths to relative paths in users table (avatar_url)
- Add validation and error handling for problematic paths
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6_
- [ ]* 1.1 Write property test for database migration completeness
- **Property 12: Database migration completeness**
- **Validates: Requirements 1.1, 1.2, 1.3, 1.4**
- [ ]* 1.2 Write property test for migration file preservation
- **Property 11: Migration preserves file access**
- **Validates: Requirements 1.5**
- [x] 2. Update FileHandler to use only relative path logic
- Modify save_file method to store only relative paths in database
- Update thumbnail creation to use only relative paths
- Remove any absolute path handling logic
- Add methods for converting absolute paths to relative paths
- _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5_
- [ ]* 2.1 Write property test for relative path storage
- **Property 1: File storage uses relative paths**
- **Validates: Requirements 2.1**
- [ ]* 2.2 Write property test for path resolution
- **Property 2: Path resolution produces valid absolute paths**
- **Validates: Requirements 2.2**
- [ ]* 2.3 Write property test for thumbnail path storage
- **Property 3: All thumbnail paths are relative**
- **Validates: Requirements 2.3**
- [x] 3. Update file serving endpoints to resolve relative paths
- Modify files router to resolve relative paths to absolute paths
- Update attachment serving endpoint to use relative path resolution
- Update submission serving endpoint to use relative path resolution
- Update project thumbnail serving endpoint to use relative path resolution
- _Requirements: 2.2, 3.2, 3.3_
- [ ]* 3.1 Write property test for file serving
- **Property 7: File serving resolves paths correctly**
- **Validates: Requirements 3.3**
- [ ]* 3.2 Write property test for thumbnail URL accessibility
- **Property 6: All thumbnail URLs are accessible**
- **Validates: Requirements 3.2**
- [ ]* 3.3 Write property test for thumbnail regeneration
- **Property 8: Missing thumbnails are regenerated**
- **Validates: Requirements 3.4**
- [x] 4. Update project thumbnail handling
- Modify project thumbnail upload to use only relative paths
- Update project thumbnail serving logic to resolve relative paths
- Ensure project model stores only relative paths
- _Requirements: 2.1, 2.3, 3.1_
- [ ]* 4.1 Write property test for project thumbnail paths
- **Property 16: Project thumbnail paths are relative**
- **Validates: Requirements 2.1, 2.3**
- [x] 4.5 Update user avatar handling to use only relative paths
- Modify user avatar upload to use only relative paths
- Update user avatar storage logic to use FileHandler methods
- Create avatar serving endpoint to resolve relative paths
- Ensure user model stores only relative paths
- _Requirements: 2.4, 3.2, 3.4_
- [ ]* 4.6 Write property test for user avatar paths
- **Property 17: User avatar paths are relative**
- **Validates: Requirements 2.4, 3.2**
- [ ]* 4.7 Write property test for avatar serving
- **Property 18: Avatar serving resolves paths correctly**
- **Validates: Requirements 3.4**
- [ ] 5. Run database migration and validate results
- Execute migration script on development database
- Validate that all file paths are now relative
- Test file serving endpoints with migrated data
- Verify thumbnail URLs work correctly after migration
- _Requirements: 1.5, 4.3, 4.5_
- [ ]* 5.1 Write property test for post-migration consistency
- **Property 13: Post-migration path consistency**
- **Validates: Requirements 4.2, 4.5**
- [ ]* 5.2 Write property test for avatar migration
- **Property 19: Avatar migration converts paths to relative**
- **Validates: Requirements 1.4, 4.6**
- [ ]* 5.3 Write property test for avatar serving after migration
- **Property 20: Avatar file serving works after migration**
- **Validates: Requirements 4.6**
- [ ] 6. Add system portability support
- Ensure file resolution works when backend directory changes
- Test path resolution with different backend locations
- Validate that database changes are not required for portability
- _Requirements: 4.4_
- [ ]* 6.1 Write property test for system portability
- **Property 10: System portability**
- **Validates: Requirements 4.4**
- [ ] 7. Update error handling and logging
- Add clear error messages for file not found scenarios
- Improve logging for path resolution debugging
- Handle edge cases gracefully with appropriate HTTP status codes
- _Requirements: 3.5_
- [ ]* 7.1 Write unit test for 404 error handling
- Test that missing files return proper 404 responses
- **Validates: Requirements 3.5**
- [ ] 8. Checkpoint - Ensure all tests pass
- Ensure all tests pass, ask the user if questions arise.
- [ ] 9. Create production migration script and documentation
- Create standalone migration script for production use
- Add documentation for deployment and migration process
- Include validation steps to verify migration success
- _Requirements: 1.4, 1.5_
- [ ] 10. Final integration testing
- Test complete file upload and serving workflow
- Verify thumbnail generation and serving works correctly
- Test avatar upload and serving workflow
- Test cross-platform compatibility scenarios
- Validate migration process with real data
- _Requirements: 3.2, 3.3, 3.4, 4.3, 4.6_
- [ ] 11. Final Checkpoint - Ensure all tests pass
- Ensure all tests pass, ask the user if questions arise.