indigo/LinkDesk
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Init Repo

Browse Source
This commit is contained in:
indigo
2026-02-28 03:22:04 +08:00
commit de59b57ee7
883 changed files with 156857 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/.env.example
+11
View File
@@ -0,0 +1,11 @@
# Database configuration
DATABASE_URL=sqlite:///./vfx_project_management.db
# JWT configuration
SECRET_KEY=your-secret-key-here
ALGORITHM=HS256
ACCESS_TOKEN_EXPIRE_MINUTES=30
# File upload configuration
UPLOAD_DIR=./uploads
MAX_FILE_SIZE=100000000 # 100MB in bytes
backend/SCHEMA_UPDATE_SUMMARY.md
+99
View File
@@ -0,0 +1,99 @@
# Project Schema Update Summary
## New Fields Added to Project Model
The project database schema has been updated to include three new required fields:
### 1. Code Name (`code_name`)
- **Type**: String (VARCHAR)
- **Constraints**: NOT NULL, UNIQUE, Indexed
- **Purpose**: Unique project identifier/code (e.g., "PROJ_001", "MARVEL_THOR_2024")
- **Validation**: 1-50 characters, must be unique across all projects
### 2. Client Name (`client_name`)
- **Type**: String (VARCHAR)
- **Constraints**: NOT NULL, Indexed
- **Purpose**: Name of the client or studio commissioning the project
- **Validation**: 1-255 characters
### 3. Project Type (`project_type`)
- **Type**: Enum
- **Values**:
- `tv` - TV Series/Shows
- `cinema` - Cinema/Film projects
- `game` - Game development projects
- **Constraints**: NOT NULL
- **Purpose**: Categorize projects by production type for better organization
## Database Migration
A migration script (`migrate_project_fields.py`) was created and executed to:
- Add the new columns to existing `projects` table
- Set default values for existing projects:
- `code_name`: Generated from project name + ID (e.g., "PROJECT_NAME_001")
- `client_name`: Set to "Default Client"
- `project_type`: Set to "tv"
- Create unique index on `code_name` field
## Backend Changes
### Models (`models/project.py`)
- Added `ProjectType` enum with TV, Cinema, Game values
- Updated `Project` model with new fields
- Added proper column constraints and indexing
### Schemas (`schemas/project.py`)
- Updated `ProjectBase`, `ProjectCreate`, `ProjectUpdate` schemas
- Added field validation and descriptions
- Updated response schemas to include new fields
### API Endpoints (`routers/projects.py`)
- Added validation for unique `code_name` constraint
- Enhanced error handling for duplicate code names
- Updated create/update endpoints to handle new fields
## Frontend Changes
### Services (`services/project.ts`)
- Updated `Project`, `ProjectCreate`, `ProjectUpdate` interfaces
- Added new fields with proper TypeScript types
### Stores (`stores/projects.ts`)
- Enhanced icon assignment logic to use `project_type` field
- Updated project filtering and organization
### UI (`views/ProjectsView.vue`)
- Added form fields for code name, client name, and project type
- Enhanced project cards to display new information
- Updated search functionality to include new fields
- Added project type formatting helper
## Benefits
1. **Better Organization**: Projects can now be categorized by type (TV, Cinema, Game)
2. **Unique Identification**: Code names provide consistent project references
3. **Client Tracking**: Clear client/studio association for each project
4. **Enhanced Search**: Users can search by code name, client name, or project type
5. **Visual Indicators**: Project cards show type badges and client information
6. **Industry Standards**: Aligns with common VFX production workflows
## Usage Examples
### Creating a New Project
```json
{
"name": "Marvel Thor: Love and Thunder",
"code_name": "MARVEL_THOR_2024",
"client_name": "Marvel Studios",
"project_type": "cinema",
"description": "VFX work for Thor sequel",
"status": "planning"
}
```
### Project Types
- **TV**: Series, shows, streaming content
- **Cinema**: Feature films, movies
- **Game**: Video game cinematics, in-game VFX
The schema update maintains backward compatibility while adding essential production management features commonly used in the VFX industry.
backend/analyze_database_indexes.py
+169
View File
@@ -0,0 +1,169 @@
#!/usr/bin/env python3
"""
Script to analyze current database indexes and identify optimization opportunities.
"""
import sqlite3
import sys
from pathlib import Path
def analyze_database_indexes():
"""Analyze current database indexes and suggest optimizations."""
# Database path
db_path = Path(__file__).parent / "vfx_project_management.db"
if not db_path.exists():
print(f"Database file not found at {db_path}")
return False
try:
# Connect to database
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("=== Current Database Schema Analysis ===\n")
# Get all tables
cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
tables = [row[0] for row in cursor.fetchall()]
for table in tables:
print(f"Table: {table}")
# Get table info
cursor.execute(f"PRAGMA table_info({table})")
columns = cursor.fetchall()
print(" Columns:")
for col in columns:
col_name, col_type, not_null, default, pk = col[1], col[2], col[3], col[4], col[5]
pk_str = " (PRIMARY KEY)" if pk else ""
print(f" - {col_name}: {col_type}{pk_str}")
# Get existing indexes
cursor.execute(f"PRAGMA index_list({table})")
indexes = cursor.fetchall()
if indexes:
print(" Existing Indexes:")
for idx in indexes:
idx_name, unique, origin = idx[1], idx[2], idx[3]
unique_str = " (UNIQUE)" if unique else ""
print(f" - {idx_name}{unique_str}")
# Get index details
cursor.execute(f"PRAGMA index_info({idx_name})")
idx_info = cursor.fetchall()
if idx_info:
cols = [info[2] for info in idx_info]
print(f" Columns: {', '.join(cols)}")
else:
print(" No indexes found")
print()
# Analyze foreign key relationships for index optimization
print("=== Foreign Key Analysis ===\n")
for table in tables:
cursor.execute(f"PRAGMA foreign_key_list({table})")
fks = cursor.fetchall()
if fks:
print(f"Table: {table}")
for fk in fks:
from_col, to_table, to_col = fk[3], fk[2], fk[4]
print(f" FK: {from_col} -> {to_table}.{to_col}")
print()
return True
except sqlite3.Error as e:
print(f"Database error: {e}")
return False
except Exception as e:
print(f"Unexpected error: {e}")
return False
finally:
if conn:
conn.close()
def suggest_index_optimizations():
"""Suggest additional indexes for performance optimization."""
print("=== Suggested Index Optimizations ===\n")
suggestions = [
{
"table": "tasks",
"index": "idx_tasks_assigned_user",
"columns": ["assigned_user_id"],
"reason": "Optimize queries filtering tasks by assigned user"
},
{
"table": "tasks",
"index": "idx_tasks_status",
"columns": ["status"],
"reason": "Optimize queries filtering tasks by status"
},
{
"table": "tasks",
"index": "idx_tasks_type",
"columns": ["task_type"],
"reason": "Optimize queries filtering tasks by type"
},
{
"table": "submissions",
"index": "idx_submissions_created_at",
"columns": ["created_at"],
"reason": "Optimize queries ordering submissions by creation date"
},
{
"table": "activities",
"index": "idx_activities_entity",
"columns": ["entity_type", "entity_id"],
"reason": "Optimize activity queries by entity"
},
{
"table": "activities",
"index": "idx_activities_created_at",
"columns": ["created_at"],
"reason": "Optimize activity feed queries by date"
},
{
"table": "shots",
"index": "idx_shots_episode",
"columns": ["episode_id"],
"reason": "Optimize queries filtering shots by episode"
},
{
"table": "assets",
"index": "idx_assets_project",
"columns": ["project_id"],
"reason": "Optimize queries filtering assets by project"
},
{
"table": "tasks",
"index": "idx_tasks_composite",
"columns": ["shot_id", "asset_id", "status"],
"reason": "Optimize complex queries filtering by parent and status"
}
]
for suggestion in suggestions:
print(f"Table: {suggestion['table']}")
print(f" Suggested Index: {suggestion['index']}")
print(f" Columns: {', '.join(suggestion['columns'])}")
print(f" Reason: {suggestion['reason']}")
print()
if __name__ == "__main__":
print("Database Index Analysis Tool")
print("=" * 50)
if analyze_database_indexes():
suggest_index_optimizations()
else:
print("Failed to analyze database")
sys.exit(1)
backend/asset_router_optimization_summary.md
+126
View File
@@ -0,0 +1,126 @@
# Asset Router Optimization Summary
## Task Completed: Backend Asset Router Optimization
### Requirements Addressed
✅ **Requirement 2.1**: Replace N+1 query pattern in `list_assets()` endpoint with single JOIN query
- Implemented single query with `outerjoin(Task, ...)` to fetch assets and tasks together
- Eliminated the previous N+1 pattern where each asset required a separate task query
- Added pre-fetching of project data to avoid repeated project queries
✅ **Requirement 2.3**: Modify asset query to include task status aggregation using SQLAlchemy joins
- Implemented task status aggregation in the single query using `add_columns()`
- Added task data grouping and aggregation logic to build `task_status` and `task_details`
- Pre-fetch all task types for all projects to eliminate repeated queries
✅ **Requirement 3.1**: Update `get_asset()` endpoint to fetch task data in single query
- Replaced separate task count query with single optimized query using `selectinload(Asset.tasks)`
- Used `joinedload(Asset.project)` for eager loading of project data
- Count tasks from already loaded relationship to avoid separate COUNT query
✅ **Backward Compatibility**: Ensure backward compatibility with existing response format
- Maintained all existing response fields and structure
- No changes to API endpoints or response schemas
- All existing functionality preserved
### Optimization Techniques Implemented
1. **Single Query Operations**
- `list_assets()`: Uses `outerjoin(Task, ...)` to fetch assets and tasks in one query
- `get_asset()`: Uses `selectinload(Asset.tasks)` for efficient task loading
2. **Eager Loading**
- `joinedload(Asset.project)` for project data
- `selectinload(Asset.tasks).options(selectinload(Task.assigned_user))` for task data
- Eliminates N+1 query problems
3. **Pre-fetching Patterns**
- Pre-fetch all project data and custom task types in single query
- Cache project information to avoid repeated database calls
- Use pre-fetched data for task status sorting
4. **Enhanced Data Tracking**
- Added `task_updated_at` tracking for better task status monitoring
- Improved task details with comprehensive information
5. **Efficient Aggregation**
- Group results by asset and aggregate task data efficiently
- Build task status maps and task details in application layer using pre-fetched data
### Performance Improvements
- **Before**: N+1 queries (1 for assets + 1 per asset for tasks + 1 per project for task types)
- **After**: Single optimized query with joins and pre-fetching
- **Expected**: Significant reduction in database round trips for asset listing operations
### Code Quality
- ✅ Follows same optimization pattern as shot router
- ✅ Comprehensive optimization comments explaining changes
- ✅ Maintains existing function signatures and response formats
- ✅ Proper error handling and access control preserved
- ✅ No syntax errors or import issues
### Testing
- ✅ Code imports successfully without errors
- ✅ Function signatures are correct
- ✅ Optimization patterns are properly implemented
- ✅ Follows established patterns from shot router optimization
## Implementation Details
### list_assets() Optimization
```python
# OPTIMIZATION: Use single query with optimized JOIN to fetch assets and their tasks
assets_with_tasks = (
base_query
.outerjoin(Task, (Task.asset_id == Asset.id) & (Task.deleted_at.is_(None)))
.options(
joinedload(Asset.project), # Eager load project
selectinload(Asset.tasks).options( # Use selectinload for better performance with tasks
selectinload(Task.assigned_user) # Eager load assigned users
)
)
.add_columns(
Task.id.label('task_id'),
Task.task_type,
Task.status.label('task_status'),
Task.assigned_user_id,
Task.updated_at.label('task_updated_at') # Include task update time for better tracking
)
.offset(skip)
.limit(limit)
.all()
)
```
### get_asset() Optimization
```python
# OPTIMIZATION: Use single query with optimized JOINs to fetch asset and all related data
asset_query = (
db.query(Asset)
.options(
joinedload(Asset.project), # Eager load project
selectinload(Asset.tasks).options( # Use selectinload for better performance with tasks
selectinload(Task.assigned_user) # Eager load assigned users if needed
)
)
.filter(Asset.id == asset_id, Asset.deleted_at.is_(None))
)
```
## Conclusion
The asset router optimization has been successfully implemented following the same patterns as the shot router optimization. The implementation:
1. ✅ Eliminates N+1 query patterns
2. ✅ Uses single database operations for data fetching
3. ✅ Maintains full backward compatibility
4. ✅ Follows established optimization patterns
5. ✅ Includes comprehensive error handling and access control
The optimization is ready for production use and should provide significant performance improvements for asset data operations.
backend/check_admin_user.py
+10
View File
@@ -0,0 +1,10 @@
import sqlite3
conn = sqlite3.connect('database.db')
cursor = conn.cursor()
cursor.execute('SELECT email, is_admin, is_approved FROM users WHERE is_admin = 1')
rows = cursor.fetchall()
print("Admin users:")
for row in rows:
print(f" Email: {row[0]}, is_admin: {row[1]}, is_approved: {row[2]}")
conn.close()
backend/check_current_user.py
+26
View File
@@ -0,0 +1,26 @@
import sqlite3
conn = sqlite3.connect('vfx_project_management.db')
cursor = conn.cursor()
# Check all users
cursor.execute('SELECT id, email, role, is_admin FROM users')
users = cursor.fetchall()
print("All users:")
for user in users:
print(f" ID: {user[0]}, Email: {user[1]}, Role: {user[2]}, Is Admin: {user[3]}")
# Check admin user specifically
cursor.execute('SELECT id, email, role, is_admin FROM users WHERE email = "admin@vfx.com"')
result = cursor.fetchone()
if result:
print(f"\nAdmin user: Email: {result[1]}, Role: {result[2]}, Is Admin: {result[3]}")
# Check project membership
cursor.execute('SELECT project_id FROM project_members WHERE user_id = ?', (result[0],))
projects = cursor.fetchall()
print(f"Project memberships: {[p[0] for p in projects]}")
else:
print("Admin user not found")
conn.close()
backend/check_custom_types.py
+11
View File
@@ -0,0 +1,11 @@
import sqlite3
import json
conn = sqlite3.connect('backend/vfx_project_management.db')
cursor = conn.cursor()
cursor.execute('SELECT id, custom_asset_task_types, custom_shot_task_types FROM projects')
for row in cursor.fetchall():
asset_types = json.loads(row[1]) if row[1] else []
shot_types = json.loads(row[2]) if row[2] else []
print(f'Project {row[0]}: Asset={asset_types}, Shot={shot_types}')
conn.close()
backend/check_db_schema.py
+37
View File
@@ -0,0 +1,37 @@
#!/usr/bin/env python3
"""
Check database schema for enum constraints.
"""
import sqlite3
def check_db_schema():
"""Check database schema for enum constraints."""
print("Checking Database Schema")
print("=" * 30)
db_path = "vfx_project_management.db"
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Get the CREATE TABLE statement for projects
cursor.execute("SELECT sql FROM sqlite_master WHERE type='table' AND name='projects'")
create_sql = cursor.fetchone()
if create_sql:
print("Projects table CREATE statement:")
print(create_sql[0])
else:
print("Projects table not found")
except Exception as e:
print(f"Error: {e}")
finally:
if conn:
conn.close()
if __name__ == "__main__":
check_db_schema()
backend/check_indexes.py
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env python3
"""
Script to check current database indexes
"""
import sqlite3
def check_current_indexes():
conn = sqlite3.connect('database.db')
cursor = conn.cursor()
# Get all non-system indexes
cursor.execute("SELECT name FROM sqlite_master WHERE type='index' AND name NOT LIKE 'sqlite_%' ORDER BY name;")
indexes = cursor.fetchall()
print('Current indexes:')
for idx in indexes:
print(f' {idx[0]}')
# Get index details
cursor.execute(f"PRAGMA index_info('{idx[0]}')")
info = cursor.fetchall()
if info:
columns = [col[2] for col in info]
print(f' Columns: {", ".join(columns)}')
# Get table info for tasks table
print('\nTasks table structure:')
cursor.execute("PRAGMA table_info(tasks)")
columns = cursor.fetchall()
for col in columns:
print(f' {col[1]} ({col[2]})')
conn.close()
if __name__ == "__main__":
check_current_indexes()
backend/check_statuses.py
+14
View File
@@ -0,0 +1,14 @@
from database import SessionLocal
from models.project import Project
db = SessionLocal()
p = db.query(Project).filter(Project.id == 1).first()
if p and p.custom_task_statuses:
statuses = p.custom_task_statuses
print(f"Total custom statuses: {len(statuses)}")
print("\nLast 5 statuses:")
for s in statuses[-5:]:
print(f" • {s['name']} ({s['id']}) - {s['color']}")
else:
print("No custom statuses found")
db.close()
backend/check_user_role.py
+14
View File
@@ -0,0 +1,14 @@
import sqlite3
conn = sqlite3.connect('backend/vfx_project_management.db')
cursor = conn.cursor()
cursor.execute('SELECT email, role, is_admin FROM users WHERE email = "admin@vfx.com"')
result = cursor.fetchone()
print(f"Email: {result[0]}, Role: {result[1]}, Is Admin: {result[2]}")
# Check project membership
cursor.execute('SELECT project_id FROM project_members WHERE user_id = (SELECT id FROM users WHERE email = "admin@vfx.com")')
projects = cursor.fetchall()
print(f"Project memberships: {[p[0] for p in projects]}")
conn.close()
backend/create_admin.py
+165
View File
@@ -0,0 +1,165 @@
#!/usr/bin/env python3
"""
Script to create an admin user for testing the VFX Project Management System.
"""
from sqlalchemy.orm import Session
from database import SessionLocal, engine
from models.user import User, UserRole
from utils.auth import get_password_hash
def create_admin_user():
"""Create an admin user for testing."""
# Create database session
db: Session = SessionLocal()
try:
# Check if admin user already exists
existing_admin = db.query(User).filter(
User.email == "admin@vfx.com"
).first()
if existing_admin:
print("Admin user already exists!")
print(f"Email: {existing_admin.email}")
print(f"Role: {existing_admin.role}")
print(f"Approved: {existing_admin.is_approved}")
return existing_admin
# Create admin user
password = "admin123"
if len(password.encode('utf-8')) > 72:
password = password[:72]
admin_user = User(
email="admin@vfx.com",
password_hash=get_password_hash(password),
first_name="Admin",
last_name="User",
role=UserRole.COORDINATOR, # Default functional role
is_admin=True, # Grant admin permission
is_approved=True # Admin is automatically approved
)
db.add(admin_user)
db.commit()
db.refresh(admin_user)
print("✅ Admin user created successfully!")
print(f"Email: {admin_user.email}")
print(f"Password: admin123")
print(f"Role: {admin_user.role}")
print(f"ID: {admin_user.id}")
return admin_user
except Exception as e:
print(f"❌ Error creating admin user: {e}")
db.rollback()
return None
finally:
db.close()
def create_test_users():
"""Create additional test users for different roles."""
db: Session = SessionLocal()
test_users = [
{
"email": "director@vfx.com",
"password": "director123",
"first_name": "John",
"last_name": "Director",
"role": UserRole.DIRECTOR,
"is_approved": True
},
{
"email": "coordinator@vfx.com",
"password": "coord123",
"first_name": "Jane",
"last_name": "Coordinator",
"role": UserRole.COORDINATOR,
"is_approved": True
},
{
"email": "artist@vfx.com",
"password": "artist123",
"first_name": "Bob",
"last_name": "Artist",
"role": UserRole.ARTIST,
"is_approved": True
}
]
try:
created_users = []
for user_data in test_users:
# Check if user already exists
existing_user = db.query(User).filter(
User.email == user_data["email"]
).first()
if existing_user:
print(f"User {user_data['email']} already exists, skipping...")
continue
# Create user
password = user_data["password"]
if len(password.encode('utf-8')) > 72:
password = password[:72]
user = User(
email=user_data["email"],
password_hash=get_password_hash(password),
first_name=user_data["first_name"],
last_name=user_data["last_name"],
role=user_data["role"],
is_approved=user_data["is_approved"]
)
db.add(user)
created_users.append(user_data)
db.commit()
if created_users:
print(f"\n✅ Created {len(created_users)} test users:")
for user_data in created_users:
print(f" - {user_data['email']} (password: {user_data['password']}) - {user_data['role']}")
else:
print("\n📝 All test users already exist")
except Exception as e:
print(f"❌ Error creating test users: {e}")
db.rollback()
finally:
db.close()
if __name__ == "__main__":
print("Creating admin user for VFX Project Management System...")
# Create admin user
admin = create_admin_user()
if admin:
print("\n" + "="*50)
print("ADMIN LOGIN CREDENTIALS")
print("="*50)
print("Email: admin@vfx.com")
print("Password: admin123")
print("="*50)
# Ask if user wants to create additional test users
create_more = input("\nCreate additional test users? (y/n): ").lower().strip()
if create_more in ['y', 'yes']:
create_test_users()
print("\n🚀 You can now login to the system!")
print("📖 API Documentation: http://127.0.0.1:8000/docs")
print("🔍 Health Check: http://127.0.0.1:8000/health")
backend/create_example_data.py
+320
View File
@@ -0,0 +1,320 @@
#!/usr/bin/env python3
"""
Script to create comprehensive example data for the VFX project management system.
This includes episodes, assets, and project members for the Dragon Quest project.
"""
import sqlite3
import json
from datetime import date, datetime
from pathlib import Path
def get_database_path():
"""Get the database path."""
possible_paths = [
"vfx_project_management.db",
"database.db"
]
for path in possible_paths:
if Path(path).exists():
return path
return "vfx_project_management.db"
def create_example_episodes(cursor, project_id):
"""Create example episodes for the project."""
episodes_data = [
{
"name": "The Dragon's Awakening",
"episode_number": 1,
"description": "Opening sequence where the ancient dragon awakens from its thousand-year slumber. Features extensive particle effects, environmental destruction, and creature animation.",
"status": "in_progress"
},
{
"name": "The Quest Begins",
"episode_number": 2,
"description": "Heroes embark on their journey through magical forests and mystical landscapes. Requires complex environment work and magical effect sequences.",
"status": "planning"
},
{
"name": "Battle of the Crystal Caves",
"episode_number": 3,
"description": "Epic battle sequence in underground crystal caves with magical creatures. Heavy focus on lighting effects, crystal simulations, and creature interactions.",
"status": "planning"
},
{
"name": "The Final Confrontation",
"episode_number": 4,
"description": "Climactic battle between heroes and the dragon. Most VFX-intensive episode featuring fire effects, destruction, magical spells, and complex creature animation.",
"status": "planning"
}
]
episode_ids = []
current_time = datetime.now().isoformat()
for episode_data in episodes_data:
# Check if episode already exists
cursor.execute("""
SELECT id FROM episodes
WHERE project_id = ? AND episode_number = ?
""", (project_id, episode_data["episode_number"]))
existing_episode = cursor.fetchone()
if existing_episode:
episode_ids.append(existing_episode[0])
continue
insert_query = """
INSERT INTO episodes (
project_id, name, episode_number, description, status,
created_at, updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?)
"""
cursor.execute(insert_query, (
project_id,
episode_data["name"],
episode_data["episode_number"],
episode_data["description"],
episode_data["status"],
current_time,
current_time
))
episode_ids.append(cursor.lastrowid)
return episode_ids
def create_example_assets(cursor, project_id):
"""Create example assets for the project."""
assets_data = [
{
"name": "Ancient Dragon",
"category": "characters",
"description": "Main antagonist dragon character with detailed scales, wings, and fire-breathing capabilities. Requires complex rigging and animation systems.",
"status": "in_progress"
},
{
"name": "Hero Character - Warrior",
"category": "characters",
"description": "Main protagonist warrior character with armor, weapons, and facial animation capabilities.",
"status": "completed"
},
{
"name": "Hero Character - Mage",
"category": "characters",
"description": "Magical character with spell-casting animations and mystical effects integration.",
"status": "in_progress"
},
{
"name": "Crystal Cave Environment",
"category": "sets",
"description": "Underground cave system with glowing crystals, stalactites, and magical lighting effects.",
"status": "in_progress"
},
{
"name": "Enchanted Forest",
"category": "sets",
"description": "Magical forest environment with animated trees, floating particles, and dynamic lighting.",
"status": "not_started"
},
{
"name": "Dragon's Lair",
"category": "sets",
"description": "Massive cave environment where the dragon resides, featuring treasure piles and ancient architecture.",
"status": "not_started"
},
{
"name": "Excalibur Sword",
"category": "props",
"description": "Legendary sword with magical glow effects and particle systems.",
"status": "approved"
},
{
"name": "Magic Staff",
"category": "props",
"description": "Mage's staff with crystal orb and magical energy effects.",
"status": "in_progress"
},
{
"name": "Dragon Armor Set",
"category": "props",
"description": "Protective armor made from dragon scales with metallic and organic textures.",
"status": "not_started"
},
{
"name": "Dragon Wings",
"category": "props",
"description": "Detailed dragon wing assets for close-up shots and animation reference.",
"status": "in_progress"
},
{
"name": "Flying Carpet",
"category": "vehicles",
"description": "Magical flying carpet for transportation sequences with cloth simulation.",
"status": "completed"
},
{
"name": "War Chariot",
"category": "vehicles",
"description": "Battle chariot for epic combat sequences with destruction capabilities.",
"status": "not_started"
}
]
asset_ids = []
current_time = datetime.now().isoformat()
for asset_data in assets_data:
# Check if asset already exists
cursor.execute("""
SELECT id FROM assets
WHERE project_id = ? AND name = ?
""", (project_id, asset_data["name"]))
existing_asset = cursor.fetchone()
if existing_asset:
asset_ids.append(existing_asset[0])
continue
insert_query = """
INSERT INTO assets (
project_id, name, category, description, status,
created_at, updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?)
"""
cursor.execute(insert_query, (
project_id,
asset_data["name"],
asset_data["category"],
asset_data["description"],
asset_data["status"],
current_time,
current_time
))
asset_ids.append(cursor.lastrowid)
return asset_ids
def create_example_data():
"""Create comprehensive example data for the VFX project."""
print("Creating Example VFX Project Data")
print("=" * 40)
db_path = get_database_path()
print(f"Using database: {db_path}")
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Get the Dragon Quest project ID
cursor.execute("SELECT id FROM projects WHERE code_name = 'DRAGON_QUEST_2024'")
project_result = cursor.fetchone()
if not project_result:
print("❌ Dragon Quest project not found. Please run create_example_project.py first.")
return False
project_id = project_result[0]
print(f"✅ Found Dragon Quest project (ID: {project_id})")
# Create episodes
print("\n📺 Creating example episodes...")
episode_ids = create_example_episodes(cursor, project_id)
print(f"✅ Created/found {len(episode_ids)} episodes")
# Create assets
print("\n🎨 Creating example assets...")
asset_ids = create_example_assets(cursor, project_id)
print(f"✅ Created/found {len(asset_ids)} assets")
# Commit all changes
conn.commit()
# Show summary
print(f"\n📊 Summary:")
print(f" Project: Dragon Quest: The Awakening")
print(f" Episodes: {len(episode_ids)}")
print(f" Assets: {len(asset_ids)}")
# Show episode breakdown
cursor.execute("""
SELECT name, episode_number, status
FROM episodes
WHERE project_id = ?
ORDER BY episode_number
""", (project_id,))
episodes = cursor.fetchall()
print(f"\n📺 Episodes:")
for episode in episodes:
status_emoji = {"not_started": "⏳", "planning": "📋", "in_progress": "🎬", "completed": "✅"}
print(f" {status_emoji.get(episode[2], '❓')} Episode {episode[1]}: {episode[0]} ({episode[2].replace('_', ' ').title()})")
# Show asset breakdown by category
cursor.execute("""
SELECT category, COUNT(*) as count
FROM assets
WHERE project_id = ?
GROUP BY category
ORDER BY count DESC
""", (project_id,))
asset_categories = cursor.fetchall()
print(f"\n🎨 Assets by Category:")
category_emojis = {"characters": "👤", "sets": "🌍", "props": "⚔️", "vehicles": "🚗"}
for category, count in asset_categories:
emoji = category_emojis.get(category, "📦")
print(f" {emoji} {category.title()}: {count}")
# Show asset status breakdown
cursor.execute("""
SELECT status, COUNT(*) as count
FROM assets
WHERE project_id = ?
GROUP BY status
ORDER BY count DESC
""", (project_id,))
asset_statuses = cursor.fetchall()
print(f"\n📈 Asset Status:")
status_emojis = {"not_started": "⏳", "planning": "📋", "in_progress": "🎬", "completed": "✅"}
for status, count in asset_statuses:
emoji = status_emojis.get(status, "❓")
print(f" {emoji} {status.replace('_', ' ').title()}: {count}")
return True
except sqlite3.Error as e:
print(f"❌ Database error: {e}")
if conn:
conn.rollback()
return False
except Exception as e:
print(f"❌ Unexpected error: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("VFX Project Management - Example Data Creator")
print("=" * 50)
success = create_example_data()
if success:
print("\n🎬 Example data created successfully!")
print("The Dragon Quest project now has realistic episodes and assets for testing.")
else:
print("\n❌ Failed to create example data.")
backend/create_example_project.py
+245
View File
@@ -0,0 +1,245 @@
#!/usr/bin/env python3
"""
Script to create an example VFX project in the database with realistic data.
"""
import sqlite3
import json
from datetime import date, datetime
from pathlib import Path
def get_database_path():
"""Get the database path."""
possible_paths = [
"vfx_project_management.db",
"database.db"
]
for path in possible_paths:
if Path(path).exists():
return path
return "vfx_project_management.db"
def create_example_project():
"""Create an example VFX project with realistic data."""
print("Creating Example VFX Project")
print("=" * 40)
db_path = get_database_path()
print(f"Using database: {db_path}")
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Check if projects table exists
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='projects'")
if not cursor.fetchone():
print("❌ Projects table not found. Please run the application first to create the database schema.")
return False
# Check if example project already exists
cursor.execute("SELECT id FROM projects WHERE code_name = 'DRAGON_QUEST_2024'")
existing_project = cursor.fetchone()
if existing_project:
print(f"✅ Example project already exists (ID: {existing_project[0]})")
return True
# Create example project data
project_data = {
"name": "Dragon Quest: The Awakening",
"code_name": "DRAGON_QUEST_2024",
"client_name": "Epic Fantasy Studios",
"project_type": "cinema",
"description": "A high-budget fantasy film featuring dragons, magic, and epic battles. Requires extensive VFX work including creature animation, environmental effects, and magical elements.",
"status": "in_progress",
"start_date": "2024-01-15",
"end_date": "2024-12-20",
"frame_rate": 24.0,
"data_drive_path": "/projects/dragon_quest_2024/data",
"publish_storage_path": "/projects/dragon_quest_2024/publish",
"delivery_image_resolution": "4096x2160",
"delivery_movie_specs_by_department": {
"layout": {
"resolution": "1920x1080",
"format": "mov",
"codec": "h264",
"quality": "medium"
},
"animation": {
"resolution": "2048x1080",
"format": "mov",
"codec": "h264",
"quality": "high"
},
"lighting": {
"resolution": "4096x2160",
"format": "exr",
"codec": None,
"quality": "high"
},
"composite": {
"resolution": "4096x2160",
"format": "mov",
"codec": "prores",
"quality": "high"
},
"modeling": {
"resolution": "1920x1080",
"format": "mov",
"codec": "h264",
"quality": "medium"
},
"rigging": {
"resolution": "1920x1080",
"format": "mov",
"codec": "h264",
"quality": "medium"
},
"surfacing": {
"resolution": "2048x1080",
"format": "mov",
"codec": "h264",
"quality": "high"
}
}
}
# Convert delivery specs to JSON string
delivery_specs_json = json.dumps(project_data["delivery_movie_specs_by_department"])
# Insert the project
insert_query = """
INSERT INTO projects (
name, code_name, client_name, project_type, description, status,
start_date, end_date, frame_rate, data_drive_path, publish_storage_path,
delivery_image_resolution, delivery_movie_specs_by_department,
created_at, updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
"""
current_time = datetime.now().isoformat()
cursor.execute(insert_query, (
project_data["name"],
project_data["code_name"],
project_data["client_name"],
project_data["project_type"],
project_data["description"],
project_data["status"],
project_data["start_date"],
project_data["end_date"],
project_data["frame_rate"],
project_data["data_drive_path"],
project_data["publish_storage_path"],
project_data["delivery_image_resolution"],
delivery_specs_json,
current_time,
current_time
))
project_id = cursor.lastrowid
# Commit the changes
conn.commit()
print(f"✅ Example project created successfully!")
print(f" Project ID: {project_id}")
print(f" Name: {project_data['name']}")
print(f" Code: {project_data['code_name']}")
print(f" Client: {project_data['client_name']}")
print(f" Type: {project_data['project_type'].upper()}")
print(f" Status: {project_data['status'].replace('_', ' ').title()}")
print(f" Frame Rate: {project_data['frame_rate']} fps")
print(f" Resolution: {project_data['delivery_image_resolution']}")
print(f" Departments: {', '.join(project_data['delivery_movie_specs_by_department'].keys())}")
# Verify the project was created correctly
cursor.execute("""
SELECT name, code_name, frame_rate, delivery_image_resolution
FROM projects WHERE id = ?
""", (project_id,))
verification = cursor.fetchone()
if verification:
print(f"\n✅ Verification successful:")
print(f" Database Name: {verification[0]}")
print(f" Database Code: {verification[1]}")
print(f" Database Frame Rate: {verification[2]} fps")
print(f" Database Resolution: {verification[3]}")
return True
except sqlite3.Error as e:
print(f"❌ Database error: {e}")
if conn:
conn.rollback()
return False
except Exception as e:
print(f"❌ Unexpected error: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
def show_all_projects():
"""Display all projects in the database."""
print("\nAll Projects in Database:")
print("-" * 40)
db_path = get_database_path()
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
cursor.execute("""
SELECT id, name, code_name, client_name, project_type, status,
frame_rate, delivery_image_resolution
FROM projects
ORDER BY created_at DESC
""")
projects = cursor.fetchall()
if not projects:
print("No projects found in database.")
return
for project in projects:
print(f"ID: {project[0]}")
print(f" Name: {project[1]}")
print(f" Code: {project[2]}")
print(f" Client: {project[3]}")
print(f" Type: {project[4].upper()}")
print(f" Status: {project[5].replace('_', ' ').title()}")
print(f" Frame Rate: {project[6]} fps")
print(f" Resolution: {project[7]}")
print()
except sqlite3.Error as e:
print(f"❌ Database error: {e}")
except Exception as e:
print(f"❌ Unexpected error: {e}")
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("VFX Project Management - Example Project Creator")
print("=" * 50)
success = create_example_project()
if success:
show_all_projects()
print("\n🎬 Example project created successfully!")
print("You can now test the VFX project management system with realistic data.")
else:
print("\n❌ Failed to create example project.")
backend/create_fresh_database.py
+132
View File
@@ -0,0 +1,132 @@
#!/usr/bin/env python3
"""
Create a fresh database with proper schema and test data.
"""
import os
import sys
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
def create_fresh_database():
"""Create a fresh database with proper schema."""
print("Creating Fresh Database")
print("=" * 30)
# Use a new database name
new_db_name = "vfx_project_fresh.db"
# Remove if exists
if os.path.exists(new_db_name):
os.remove(new_db_name)
print(f"Removed existing {new_db_name}")
# Update database configuration temporarily
os.environ['DATABASE_URL'] = f"sqlite:///./{new_db_name}"
try:
# Import after setting environment variable
from database import engine, Base
import models # This imports all models
# Create all tables
Base.metadata.create_all(bind=engine)
print("✅ Created database schema")
# Create admin user
from database import get_db
from models.user import User, UserRole
from utils.auth import get_password_hash
db = next(get_db())
# Check if admin exists
admin_user = db.query(User).filter(User.email == "admin@vfx.com").first()
if not admin_user:
admin_user = User(
email="admin@vfx.com",
password_hash=get_password_hash("admin123"),
first_name="Admin",
last_name="User",
role=UserRole.COORDINATOR,
is_admin=True,
is_approved=True
)
db.add(admin_user)
db.commit()
print("✅ Created admin user")
else:
print("✅ Admin user already exists")
# Create test projects
from models.project import Project, ProjectStatus, ProjectType
import json
from datetime import datetime
# Project 1
project1 = Project(
name="Test VFX Project",
code_name="TEST_VFX_001",
client_name="Test Studio",
project_type=ProjectType.CINEMA,
description="Test project for VFX management system",
status=ProjectStatus.PLANNING,
frame_rate=24.0,
data_drive_path="/projects/test_vfx/data",
publish_storage_path="/projects/test_vfx/publish",
delivery_image_resolution="1920x1080",
delivery_movie_specs_by_department=json.dumps({
"layout": {"resolution": "1920x1080", "format": "mov", "codec": "h264", "quality": "medium"},
"animation": {"resolution": "1920x1080", "format": "mov", "codec": "h264", "quality": "high"}
})
)
# Project 2
project2 = Project(
name="Dragon Quest: The Awakening",
code_name="DRAGON_QUEST_2024",
client_name="Epic Fantasy Studios",
project_type=ProjectType.CINEMA,
description="A high-budget fantasy film featuring dragons and magic",
status=ProjectStatus.IN_PROGRESS,
frame_rate=24.0,
data_drive_path="/projects/dragon_quest_2024/data",
publish_storage_path="/projects/dragon_quest_2024/publish",
delivery_image_resolution="4096x2160",
delivery_movie_specs_by_department=json.dumps({
"layout": {"resolution": "1920x1080", "format": "mov", "codec": "h264", "quality": "medium"},
"animation": {"resolution": "2048x1080", "format": "mov", "codec": "h264", "quality": "high"},
"lighting": {"resolution": "4096x2160", "format": "exr", "codec": None, "quality": "high"},
"composite": {"resolution": "4096x2160", "format": "mov", "codec": "prores", "quality": "high"}
})
)
db.add(project1)
db.add(project2)
db.commit()
print("✅ Created test projects")
# Rename to replace old database
db.close()
# Now replace the old database
old_db = "vfx_project_management.db"
if os.path.exists(old_db):
os.remove(old_db)
os.rename(new_db_name, old_db)
print(f"✅ Replaced old database with fresh one")
print("\n🎉 Fresh database created successfully!")
print("Admin credentials: admin@vfx.com / admin123")
return True
except Exception as e:
print(f"❌ Error: {e}")
import traceback
traceback.print_exc()
return False
if __name__ == "__main__":
create_fresh_database()
backend/create_task_status_indexes.py
+255
View File
@@ -0,0 +1,255 @@
#!/usr/bin/env python3
"""
Create database indexes to optimize task status queries for shots and assets.
This script implements the database schema optimization from the shot-asset-task-status-optimization spec.
"""
import sqlite3
import sys
from pathlib import Path
def create_task_status_indexes():
"""Create optimized indexes for task status queries."""
db_path = Path("database.db")
if not db_path.exists():
print("Error: database.db not found. Please run from the backend directory.")
return False
conn = sqlite3.connect('database.db')
cursor = conn.cursor()
try:
print("Creating optimized indexes for task status queries...")
# Index 1: Optimize task lookups by shot_id (active tasks only)
print("Creating idx_tasks_shot_id_active...")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_tasks_shot_id_active
ON tasks(shot_id)
WHERE deleted_at IS NULL
""")
# Index 2: Optimize task lookups by asset_id (active tasks only)
print("Creating idx_tasks_asset_id_active...")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_tasks_asset_id_active
ON tasks(asset_id)
WHERE deleted_at IS NULL
""")
# Index 3: Optimize task status filtering (active tasks only)
print("Creating idx_tasks_status_type_active...")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_tasks_status_type_active
ON tasks(status, task_type)
WHERE deleted_at IS NULL
""")
# Index 4: Composite index for shot + status + type queries (most common pattern)
print("Creating idx_tasks_shot_status_type_active...")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_tasks_shot_status_type_active
ON tasks(shot_id, status, task_type)
WHERE deleted_at IS NULL
""")
# Index 5: Composite index for asset + status + type queries (most common pattern)
print("Creating idx_tasks_asset_status_type_active...")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_tasks_asset_status_type_active
ON tasks(asset_id, status, task_type)
WHERE deleted_at IS NULL
""")
# Index 6: Optimize queries that need task details (id, type, status, assignee, updated_at)
print("Creating idx_tasks_details_shot...")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_tasks_details_shot
ON tasks(shot_id, id, task_type, status, assigned_user_id, updated_at)
WHERE deleted_at IS NULL
""")
# Index 7: Optimize queries that need task details for assets
print("Creating idx_tasks_details_asset...")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_tasks_details_asset
ON tasks(asset_id, id, task_type, status, assigned_user_id, updated_at)
WHERE deleted_at IS NULL
""")
# Index 8: Optimize project-wide task queries with status filtering
print("Creating idx_tasks_project_status_active...")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_tasks_project_status_active
ON tasks(project_id, status, task_type)
WHERE deleted_at IS NULL
""")
conn.commit()
print("✅ All indexes created successfully!")
# Verify indexes were created
print("\nVerifying created indexes...")
cursor.execute("""
SELECT name FROM sqlite_master
WHERE type='index'
AND name LIKE 'idx_tasks_%_active'
ORDER BY name
""")
new_indexes = cursor.fetchall()
for idx in new_indexes:
print(f" ✓ {idx[0]}")
return True
except sqlite3.Error as e:
print(f"❌ Error creating indexes: {e}")
conn.rollback()
return False
finally:
conn.close()
def test_index_performance():
"""Test the performance of the new indexes with sample queries."""
conn = sqlite3.connect('database.db')
cursor = conn.cursor()
try:
print("\n" + "="*50)
print("TESTING INDEX PERFORMANCE")
print("="*50)
# Test 1: Shot task status aggregation query
print("\nTest 1: Shot task status aggregation")
cursor.execute("EXPLAIN QUERY PLAN SELECT shot_id, task_type, status FROM tasks WHERE shot_id = 1 AND deleted_at IS NULL")
plan = cursor.fetchall()
for row in plan:
print(f" {row}")
# Test 2: Asset task status aggregation query
print("\nTest 2: Asset task status aggregation")
cursor.execute("EXPLAIN QUERY PLAN SELECT asset_id, task_type, status FROM tasks WHERE asset_id = 1 AND deleted_at IS NULL")
plan = cursor.fetchall()
for row in plan:
print(f" {row}")
# Test 3: Project-wide status filtering
print("\nTest 3: Project-wide status filtering")
cursor.execute("EXPLAIN QUERY PLAN SELECT * FROM tasks WHERE project_id = 1 AND status = 'in_progress' AND deleted_at IS NULL")
plan = cursor.fetchall()
for row in plan:
print(f" {row}")
# Test 4: Complex join query (shots with task status)
print("\nTest 4: Shots with task status join")
cursor.execute("""
EXPLAIN QUERY PLAN
SELECT s.id, s.name, t.task_type, t.status
FROM shots s
LEFT JOIN tasks t ON s.id = t.shot_id AND t.deleted_at IS NULL
WHERE s.deleted_at IS NULL
LIMIT 10
""")
plan = cursor.fetchall()
for row in plan:
print(f" {row}")
print("\n✅ Index performance tests completed!")
except sqlite3.Error as e:
print(f"❌ Error testing performance: {e}")
finally:
conn.close()
def get_sample_data_stats():
"""Get statistics about the current data to validate index effectiveness."""
conn = sqlite3.connect('database.db')
cursor = conn.cursor()
try:
print("\n" + "="*50)
print("SAMPLE DATA STATISTICS")
print("="*50)
# Count total tasks
cursor.execute("SELECT COUNT(*) FROM tasks WHERE deleted_at IS NULL")
total_tasks = cursor.fetchone()[0]
print(f"Total active tasks: {total_tasks}")
# Count tasks by type
cursor.execute("SELECT task_type, COUNT(*) FROM tasks WHERE deleted_at IS NULL GROUP BY task_type")
task_types = cursor.fetchall()
print("\nTasks by type:")
for task_type, count in task_types:
print(f" {task_type}: {count}")
# Count tasks by status
cursor.execute("SELECT status, COUNT(*) FROM tasks WHERE deleted_at IS NULL GROUP BY status")
task_statuses = cursor.fetchall()
print("\nTasks by status:")
for status, count in task_statuses:
print(f" {status}: {count}")
# Count shots with tasks
cursor.execute("""
SELECT COUNT(DISTINCT s.id)
FROM shots s
INNER JOIN tasks t ON s.id = t.shot_id
WHERE s.deleted_at IS NULL AND t.deleted_at IS NULL
""")
shots_with_tasks = cursor.fetchone()[0]
print(f"\nShots with tasks: {shots_with_tasks}")
# Count assets with tasks
cursor.execute("""
SELECT COUNT(DISTINCT a.id)
FROM assets a
INNER JOIN tasks t ON a.id = t.asset_id
WHERE a.deleted_at IS NULL AND t.deleted_at IS NULL
""")
assets_with_tasks = cursor.fetchone()[0]
print(f"Assets with tasks: {assets_with_tasks}")
return {
'total_tasks': total_tasks,
'shots_with_tasks': shots_with_tasks,
'assets_with_tasks': assets_with_tasks
}
except sqlite3.Error as e:
print(f"❌ Error getting statistics: {e}")
return None
finally:
conn.close()
if __name__ == "__main__":
print("Shot-Asset Task Status Optimization: Database Index Creation")
print("=" * 60)
# Get current data statistics
stats = get_sample_data_stats()
# Create the indexes
success = create_task_status_indexes()
if success:
# Test index performance
test_index_performance()
print("\n" + "="*60)
print("INDEX CREATION COMPLETED SUCCESSFULLY!")
print("="*60)
print("\nNext steps:")
print("1. Run backend optimization tests")
print("2. Implement optimized query patterns in routers")
print("3. Test with larger datasets")
else:
print("\n❌ Index creation failed!")
sys.exit(1)
backend/database.py
+25
View File
@@ -0,0 +1,25 @@
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
import os
# Database configuration
DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:///./database.db")
engine = create_engine(
DATABASE_URL,
connect_args={"check_same_thread": False} if "sqlite" in DATABASE_URL else {}
)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
Base = declarative_base()
# Dependency to get database session
def get_db():
db = SessionLocal()
try:
yield db
finally:
db.close()
backend/debug_episode_access.py
+51
View File
@@ -0,0 +1,51 @@
"""
Debug script to check episode access issue
"""
import sqlite3
conn = sqlite3.connect('backend/vfx_project_management.db')
cursor = conn.cursor()
# Check if episode 2 exists
print("=== Checking Episode 2 ===")
cursor.execute('SELECT id, name, project_id FROM episodes WHERE id = 2')
episode = cursor.fetchone()
if episode:
print(f"Episode found: ID={episode[0]}, Name={episode[1]}, Project ID={episode[2]}")
project_id = episode[2]
# Check if admin user is a member of this project
print(f"\n=== Checking Project {project_id} Membership ===")
cursor.execute('''
SELECT u.id, u.email, u.role, pm.project_id
FROM users u
LEFT JOIN project_members pm ON u.id = pm.user_id AND pm.project_id = ?
WHERE u.email = "admin@vfx.com"
''', (project_id,))
user_info = cursor.fetchone()
print(f"User ID: {user_info[0]}, Email: {user_info[1]}, Role: {user_info[2]}, Project Member: {user_info[3]}")
if user_info[3] is None:
print(f"\n⚠️ User is NOT a member of project {project_id}")
print("This is why the 403 error occurs!")
# Add user to project
print(f"\nAdding user to project {project_id}...")
cursor.execute('''
INSERT INTO project_members (user_id, project_id, department_role)
VALUES (?, ?, NULL)
''', (user_info[0], project_id))
conn.commit()
print("✓ User added to project")
else:
print(f"\n✓ User is already a member of project {project_id}")
else:
print("Episode 2 not found!")
# List all episodes
print("\n=== All Episodes ===")
cursor.execute('SELECT id, name, project_id FROM episodes')
for ep in cursor.fetchall():
print(f" Episode {ep[0]}: {ep[1]} (Project {ep[2]})")
conn.close()
backend/debug_projects.py
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env python3
"""
Debug script to check projects table and data.
"""
import sqlite3
from pathlib import Path
def debug_projects():
"""Debug projects table structure and data."""
print("Debugging Projects Table")
print("=" * 30)
db_path = "vfx_project_management.db"
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Check table structure
print("1. Projects table structure:")
cursor.execute("PRAGMA table_info(projects)")
columns = cursor.fetchall()
for col in columns:
print(f" {col[1]} ({col[2]}) - NOT NULL: {bool(col[3])}")
# Check data
print("\n2. Projects data:")
cursor.execute("SELECT * FROM projects")
projects = cursor.fetchall()
if not projects:
print(" No projects found")
else:
for i, project in enumerate(projects):
print(f" Project {i+1}: {project}")
# Check specific fields that might be causing issues
print("\n3. Checking for NULL values in required fields:")
cursor.execute("""
SELECT id, name, code_name, client_name, project_type, status, created_at, updated_at
FROM projects
""")
projects = cursor.fetchall()
for project in projects:
print(f" ID: {project[0]}")
print(f" Name: {project[1]}")
print(f" Code: {project[2]}")
print(f" Client: {project[3]}")
print(f" Type: {project[4]}")
print(f" Status: {project[5]}")
print(f" Created: {project[6]}")
print(f" Updated: {project[7]}")
print()
except Exception as e:
print(f"Error: {e}")
finally:
if conn:
conn.close()
if __name__ == "__main__":
debug_projects()
backend/debug_shot_403.py
+103
View File
@@ -0,0 +1,103 @@
"""
Debug script to check why admin is getting 403
"""
import requests
import json
BASE_URL = "http://localhost:8000"
# Try to login and access a shot
print("=" * 60)
print("Debugging Shot 403 Error")
print("=" * 60)
# Step 1: Login
print("\n1. Attempting login...")
login_response = requests.post(f"{BASE_URL}/auth/login", json={
"email": "admin@vfx.com",
"password": "admin123"
})
if login_response.status_code != 200:
print(f"✗ Login failed: {login_response.status_code}")
print(f"Response: {login_response.text}")
exit(1)
print("✓ Login successful")
token = login_response.json()["access_token"]
user_data = login_response.json().get("user", {})
print(f" User: {user_data.get('email')}")
print(f" Role: {user_data.get('role')}")
print(f" Is Admin: {user_data.get('is_admin')}")
headers = {"Authorization": f"Bearer {token}"}
# Step 2: Get shots list
print("\n2. Getting shots list...")
shots_response = requests.get(f"{BASE_URL}/shots/", headers=headers)
if shots_response.status_code != 200:
print(f"✗ Failed to get shots: {shots_response.status_code}")
print(f"Response: {shots_response.text}")
exit(1)
shots = shots_response.json()
print(f"✓ Got {len(shots)} shots")
if not shots:
print("No shots available to test")
exit(0)
# Step 3: Try to get first shot detail
shot = shots[0]
shot_id = shot["id"]
print(f"\n3. Getting shot detail for shot ID: {shot_id}")
print(f" Shot name: {shot.get('name')}")
print(f" Episode ID: {shot.get('episode_id')}")
shot_detail_response = requests.get(f"{BASE_URL}/shots/{shot_id}", headers=headers)
print(f"\n4. Response:")
print(f" Status Code: {shot_detail_response.status_code}")
if shot_detail_response.status_code == 200:
print("✓ SUCCESS: Shot detail retrieved")
detail = shot_detail_response.json()
print(f" Shot: {detail.get('name')}")
print(f" Frame range: {detail.get('frame_start')}-{detail.get('frame_end')}")
elif shot_detail_response.status_code == 403:
print("✗ FAILED: 403 Forbidden")
print(f" Response: {shot_detail_response.text}")
print("\n This means the backend check_episode_access function is still blocking access")
print(" Possible causes:")
print(" - Backend not restarted after code change")
print(" - User is_admin field is False in database")
print(" - Different endpoint being called")
else:
print(f"✗ FAILED: {shot_detail_response.status_code}")
print(f" Response: {shot_detail_response.text}")
# Step 5: Check user in database
print("\n5. Checking user in database...")
import sys
sys.path.insert(0, '.')
from database import SessionLocal
from models.user import User
db = SessionLocal()
try:
db_user = db.query(User).filter(User.email == user_data.get('email')).first()
if db_user:
print(f"✓ User found in database")
print(f" Email: {db_user.email}")
print(f" Role: {db_user.role}")
print(f" is_admin: {db_user.is_admin}")
if not db_user.is_admin:
print("\n⚠ WARNING: User is_admin is False in database!")
print(" This is why you're getting 403")
print(" Run: python backend/migrate_admin_users.py")
else:
print("✗ User not found in database")
finally:
db.close()
backend/debug_tasks_403.py
+56
View File
@@ -0,0 +1,56 @@
"""
Debug script to test tasks endpoint with admin/coordinator user
"""
import requests
BASE_URL = "http://localhost:8000"
SHOT_ID = 1
# Login
login_response = requests.post(
f"{BASE_URL}/auth/login",
json={"email": "admin@vfx.com", "password": "admin123"}
)
if login_response.status_code == 200:
token = login_response.json()["access_token"]
print(f"✓ Logged in successfully")
# Get user info
headers = {"Authorization": f"Bearer {token}"}
me_response = requests.get(f"{BASE_URL}/users/me", headers=headers)
if me_response.status_code == 200:
user = me_response.json()
print(f"User: {user['email']}")
print(f"Role: {user['role']}")
print(f"Is Admin: {user['is_admin']}")
# Try without trailing slash
print(f"\n--- Testing GET /tasks?shot_id={SHOT_ID} (no trailing slash) ---")
response1 = requests.get(
f"{BASE_URL}/tasks",
params={"shot_id": SHOT_ID},
headers=headers,
allow_redirects=False # Don't follow redirects
)
print(f"Status: {response1.status_code}")
print(f"Headers: {dict(response1.headers)}")
if response1.status_code in [301, 302, 307, 308]:
print(f"Redirect to: {response1.headers.get('location')}")
# Try with trailing slash
print(f"\n--- Testing GET /tasks/?shot_id={SHOT_ID} (with trailing slash) ---")
response2 = requests.get(
f"{BASE_URL}/tasks/",
params={"shot_id": SHOT_ID},
headers=headers
)
print(f"Status: {response2.status_code}")
if response2.status_code == 200:
tasks = response2.json()
print(f"✓ Found {len(tasks)} tasks")
else:
print(f"Response: {response2.text[:500]}")
else:
print(f"✗ Login failed: {login_response.status_code}")
print(login_response.text)
backend/docs/admin_api_key_management.md
+190
View File
@@ -0,0 +1,190 @@
# Admin API Key Management
This document describes the enhanced API key management functionality that allows administrators to create and manage API keys for any user in the system.
## Overview
The VFX Project Management System now supports comprehensive API key management with different permission levels:
- **Developers**: Can create and manage their own API keys
- **Admins**: Can create and manage API keys for any user in the system
## Admin Capabilities
### 1. Create API Keys for Any User
Admins can create API keys for any approved user in the system using two methods:
#### Method 1: General Endpoint with user_id Parameter
```http
POST /auth/api-keys
Authorization: Bearer <admin_token>
Content-Type: application/json
{
"name": "Integration Key for John Doe",
"scopes": ["read:projects", "read:tasks"],
"user_id": 123,
"expires_at": "2024-12-31T23:59:59"
}
```
#### Method 2: Admin-Specific Endpoint
```http
POST /auth/admin/users/123/api-keys
Authorization: Bearer <admin_token>
Content-Type: application/json
{
"name": "Integration Key for John Doe",
"scopes": ["read:projects", "read:tasks"],
"expires_at": "2024-12-31T23:59:59"
}
```
### 2. View All API Keys
Admins can view all API keys in the system:
```http
GET /auth/api-keys
Authorization: Bearer <admin_token>
```
Response includes user email for each API key:
```json
[
{
"id": 1,
"user_id": 123,
"user_email": "john.doe@example.com",
"name": "Integration Key",
"scopes": ["read:projects", "read:tasks"],
"is_active": true,
"expires_at": "2024-12-31T23:59:59Z",
"last_used_at": "2024-01-15T10:30:00Z",
"created_at": "2024-01-01T00:00:00Z"
}
]
```
### 3. View API Keys for Specific User
```http
GET /auth/admin/users/123/api-keys
Authorization: Bearer <admin_token>
```
### 4. Manage Any API Key
Admins can update or delete any API key in the system:
```http
PUT /auth/api-keys/456
DELETE /auth/api-keys/456
Authorization: Bearer <admin_token>
```
### 5. View Usage Logs for Any API Key
```http
GET /auth/api-keys/456/usage
Authorization: Bearer <admin_token>
```
## API Key Scopes
Available scopes for API keys:
- `read:projects` - Read access to all projects
- `read:tasks` - Read access to all tasks
- `read:submissions` - Read access to all submissions
- `read:users` - Read access to user information
- `write:tasks` - Write access to tasks
- `write:submissions` - Write access to submissions
- `admin:users` - Administrative access to user management
- `full:access` - Full system access
## Security Features
1. **API Key Hashing**: All API keys are hashed before storage using SHA-256
2. **Usage Logging**: Every API request is logged with timestamp, endpoint, method, IP address, and user agent
3. **Expiration**: API keys can have expiration dates
4. **Revocation**: API keys can be deactivated or deleted at any time
5. **Scope-based Access**: Fine-grained permissions control what each API key can access
## Developer vs Admin Permissions
| Action | Developer | Admin |
|--------|-----------|-------|
| Create own API keys | ✅ | ✅ |
| Create API keys for others | ❌ | ✅ |
| View own API keys | ✅ | ✅ |
| View all API keys | ❌ | ✅ |
| Update own API keys | ✅ | ✅ |
| Update any API key | ❌ | ✅ |
| Delete own API keys | ✅ | ✅ |
| Delete any API key | ❌ | ✅ |
| View own usage logs | ✅ | ✅ |
| View any usage logs | ❌ | ✅ |
## Usage Examples
### Creating an API Key for a Developer
As an admin, you can create API keys for developers who need to integrate external tools:
```python
import requests
# Admin login
admin_token = login_as_admin()
# Create API key for developer
response = requests.post("http://localhost:8000/auth/api-keys",
headers={"Authorization": f"Bearer {admin_token}"},
json={
"name": "CI/CD Pipeline Integration",
"scopes": ["read:projects", "read:tasks", "write:submissions"],
"user_id": 456, # Developer's user ID
"expires_at": "2024-12-31T23:59:59"
}
)
api_key_data = response.json()
print(f"API Key: {api_key_data['token']}")
```
### Monitoring API Usage
Admins can monitor how API keys are being used:
```python
# Get usage logs for an API key
response = requests.get(f"http://localhost:8000/auth/api-keys/123/usage",
headers={"Authorization": f"Bearer {admin_token}"}
)
usage_logs = response.json()
for log in usage_logs:
print(f"{log['timestamp']}: {log['method']} {log['endpoint']} from {log['ip_address']}")
```
## Best Practices
1. **Principle of Least Privilege**: Only grant the minimum scopes necessary
2. **Regular Rotation**: Set expiration dates and rotate API keys regularly
3. **Monitor Usage**: Regularly review API key usage logs
4. **Revoke Unused Keys**: Delete or deactivate API keys that are no longer needed
5. **Secure Distribution**: Share API keys securely and never commit them to version control
## Testing
Use the provided test script to verify admin functionality:
```bash
cd backend
python test_admin_api_keys.py
```
This script demonstrates all admin API key management capabilities.
backend/docs/bulk-actions-implementation.md
+168
View File
@@ -0,0 +1,168 @@
# Bulk Actions Implementation
## Overview
This document describes the implementation of bulk action endpoints for the task management system. These endpoints allow coordinators and admins to perform batch operations on multiple tasks simultaneously.
## Endpoints
### 1. Bulk Status Update
**Endpoint:** `PUT /tasks/bulk/status`
**Request Body:**
```json
{
"task_ids": [1, 2, 3],
"status": "in_progress"
}
```
**Response:**
```json
{
"success_count": 3,
"failed_count": 0,
"errors": null
}
```
**Permissions:**
- Coordinators and admins can update any tasks
- Artists can only update their own assigned tasks
**Features:**
- Atomic transaction handling - either all tasks update or none
- Permission validation for all tasks before making changes
- Detailed error reporting for failed tasks
### 2. Bulk Assignment
**Endpoint:** `PUT /tasks/bulk/assign`
**Request Body:**
```json
{
"task_ids": [1, 2, 3],
"assigned_user_id": 5
}
```
**Response:**
```json
{
"success_count": 3,
"failed_count": 0,
"errors": null
}
```
**Permissions:**
- Only coordinators and admins can perform bulk assignments
**Features:**
- Atomic transaction handling
- Validates user is a member of all task projects
- Sends notifications to assigned user for each task
- Detailed error reporting for failed tasks
## Schemas
### BulkStatusUpdate
```python
class BulkStatusUpdate(BaseModel):
task_ids: List[int] = Field(..., min_length=1)
status: TaskStatus
```
### BulkAssignment
```python
class BulkAssignment(BaseModel):
task_ids: List[int] = Field(..., min_length=1)
assigned_user_id: int
```
### BulkActionResult
```python
class BulkActionResult(BaseModel):
success_count: int
failed_count: int
errors: Optional[List[dict]] = None
```
## Atomicity
Both endpoints implement atomic transactions:
1. All tasks are fetched in a single query
2. All validations (permissions, project membership) are performed before any changes
3. If any validation fails, the transaction is rolled back and no changes are made
4. Only if all validations pass are the changes committed
This ensures that partial updates never occur - either all tasks are updated successfully or none are.
## Error Handling
Errors are returned in a structured format:
```json
{
"success_count": 1,
"failed_count": 2,
"errors": [
{
"task_id": 99,
"error": "Task not found"
},
{
"task_id": 100,
"error": "Not authorized to update this task"
}
]
}
```
Common error scenarios:
- Task not found
- Insufficient permissions
- User not a project member (for assignments)
## Testing
A comprehensive test script is available at `backend/test_bulk_actions.py` that tests:
1. Successful bulk status updates
2. Successful bulk assignments
3. Error handling with invalid task IDs
4. Atomicity with mixed valid/invalid IDs
5. Permission validation
Run tests with:
```bash
python test_bulk_actions.py
```
## Implementation Notes
### Route Ordering
The bulk action endpoints are placed BEFORE the `/{task_id}` routes in the router to prevent FastAPI from trying to match "bulk" as a task_id parameter.
### Transaction Management
SQLAlchemy's session management is used for transactions:
- `db.commit()` commits all changes
- `db.rollback()` reverts all changes if errors occur
### Notifications
The bulk assignment endpoint sends individual notifications for each assigned task using the existing `notification_service.notify_task_assigned()` function.
## Requirements Validated
This implementation satisfies the following requirements from the spec:
- **4.2**: Bulk status update with atomic transaction handling
- **4.4**: Error handling and rollback on failure
- **5.3**: Bulk assignment with atomic transaction handling
- **5.5**: Error handling and rollback on failure
backend/docs/custom-task-status-create-endpoint.md
+172
View File
@@ -0,0 +1,172 @@
# Custom Task Status Creation Endpoint Implementation
## Overview
Implemented the POST endpoint for creating custom task statuses in projects. This allows coordinators and admins to define project-specific task statuses beyond the built-in system statuses.
## Implementation Details
### Endpoint
- **Route**: `POST /projects/{project_id}/task-statuses`
- **Status Code**: 201 Created
- **Authorization**: Requires coordinator or admin role
### Features Implemented
#### 1. Status Name Uniqueness Validation
- Validates that the status name is unique within the project
- Case-insensitive comparison to prevent duplicates like "In Review" and "in review"
- Checks against both existing custom statuses and system statuses
- Returns 409 Conflict if duplicate found
#### 2. Auto-Assign Color from Palette
- Defines a default color palette with 10 distinct colors:
- Purple (#8B5CF6)
- Pink (#EC4899)
- Teal (#14B8A6)
- Orange (#F97316)
- Cyan (#06B6D4)
- Lime (#84CC16)
- Violet (#A855F7)
- Rose (#F43F5E)
- Sky (#22D3EE)
- Yellow (#FACC15)
- Automatically assigns the first unused color from the palette
- If all colors are used, cycles back through the palette
- Users can override by providing a custom color in hex format
#### 3. Unique Status ID Generation
- Generates unique IDs using UUID4 with format: `custom_{8-char-hex}`
- Example: `custom_0c7ba931`
- Ensures no ID collisions
#### 4. JSON Column Updates
- Uses SQLAlchemy's `flag_modified()` to properly track changes to JSON columns
- Ensures database updates are persisted correctly
#### 5. Status Ordering
- Automatically assigns order based on existing statuses
- New statuses are appended to the end (max_order + 1)
- Maintains consistent ordering for UI display
### Request Schema
```json
{
"name": "Ready for Review",
"color": "#8B5CF6" // Optional - auto-assigned if not provided
}
```
### Response Schema
```json
{
"message": "Custom task status 'Ready for Review' created successfully",
"status": {
"id": "custom_0c7ba931",
"name": "Ready for Review",
"color": "#EC4899",
"order": 3,
"is_default": false
},
"all_statuses": {
"statuses": [...], // All custom statuses
"system_statuses": [...], // Built-in system statuses
"default_status_id": "not_started"
}
}
```
### Validation Rules
#### Name Validation (via Pydantic schema)
- Minimum length: 1 character
- Maximum length: 50 characters
- Whitespace is trimmed
- Cannot be empty after trimming
#### Color Validation (via Pydantic schema)
- Must be valid hex color code format: `#RRGGBB`
- Example: `#FF5733`
- Normalized to uppercase
- Optional field
### Error Responses
#### 404 Not Found
```json
{
"detail": "Project not found"
}
```
#### 409 Conflict - Duplicate Name
```json
{
"detail": "Status with name 'Ready for Review' already exists in this project"
}
```
#### 409 Conflict - System Status Name
```json
{
"detail": "Status name 'In Progress' conflicts with a system status"
}
```
#### 422 Unprocessable Entity - Validation Error
```json
{
"detail": "1 validation error for CustomTaskStatusCreate\nname\n String should have at least 1 character..."
}
```
## Testing
### Test Results
All validation and functionality tests passed:
1. ✅ Create status without color (auto-assigned from palette)
2. ✅ Create status with custom color
3. ✅ Reject duplicate status names (409 Conflict)
4. ✅ Reject empty status names (422 Validation Error)
5. ✅ Reject invalid color formats (422 Validation Error)
6. ✅ Reject system status name conflicts (409 Conflict)
7. ✅ Proper ordering of statuses
8. ✅ Unique ID generation
9. ✅ JSON column updates with flag_modified
### Test Files
- `backend/test_create_custom_task_status.py` - Comprehensive test suite
- `backend/test_custom_status_validation.py` - Validation-focused tests
## Database Schema
The custom task statuses are stored in the `projects.custom_task_statuses` JSON column:
```json
[
{
"id": "custom_review",
"name": "In Review",
"color": "#8B5CF6",
"order": 0,
"is_default": false
},
{
"id": "custom_blocked",
"name": "Blocked",
"color": "#DC2626",
"order": 1,
"is_default": false
}
]
```
## Requirements Satisfied
- ✅ Requirement 1.2: Create new custom task status
- ✅ Requirement 1.3: Validate status name uniqueness within project
- ✅ Requirement 1.4: Auto-assign color from palette if not provided
## Next Steps
The following endpoints still need to be implemented:
- PUT endpoint for updating custom status (Task 5)
- DELETE endpoint for deleting custom status (Task 6)
- PATCH endpoint for reordering statuses (Task 7)
backend/docs/custom-task-status-delete-endpoint.md
+137
View File
@@ -0,0 +1,137 @@
# Custom Task Status DELETE Endpoint Implementation
## Overview
Implemented the DELETE endpoint for removing custom task statuses from projects with comprehensive validation and task reassignment support.
## Endpoint Details
### DELETE `/projects/{project_id}/task-statuses/{status_id}`
**Authentication Required:** Coordinator or Admin
**Query Parameters:**
- `reassign_to_status_id` (optional): Status ID to reassign tasks to if the status being deleted is in use
**Response:** 200 OK with CustomTaskStatusResponse containing:
- Success message
- All remaining statuses (system + custom)
- Updated default status ID
## Implementation Features
### 1. Status In-Use Check
- Queries all tasks in the project to check if any use the status being deleted
- Returns detailed error (422) if status is in use and no reassignment provided:
```json
{
"error": "Cannot delete status 'X' because it is currently in use by N task(s)",
"status_id": "custom_abc123",
"status_name": "In Review",
"task_count": 5,
"task_ids": [1, 2, 3, 4, 5]
}
```
### 2. Task Reassignment
- Supports optional `reassign_to_status_id` query parameter
- Validates reassignment target exists (can be system or custom status)
- Prevents reassigning to the same status being deleted
- Automatically updates all affected tasks to the new status
- Includes reassignment count in success message
### 3. Default Status Management
- Detects if the status being deleted is the default status
- Automatically assigns the first remaining custom status as the new default
- Ensures there's always a default status after deletion
### 4. Last Status Protection
- Prevents deletion of the last custom status
- Returns 422 error with clear message
- Ensures at least one custom status always remains
### 5. Error Handling
- 404: Status not found
- 404: Project not found
- 400: Invalid reassignment status ID
- 422: Status in use without reassignment
- 422: Attempting to delete last status
- 500: Database operation failure
## Code Structure
```python
@router.delete("/{project_id}/task-statuses/{status_id}")
async def delete_custom_task_status(
project_id: int,
status_id: str,
reassign_to_status_id: Optional[str] = Query(None),
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
# 1. Verify project exists
# 2. Load custom statuses from JSON
# 3. Find status to delete
# 4. Prevent deletion of last status
# 5. Check if status is in use
# 6. Handle reassignment if needed
# 7. Auto-assign new default if needed
# 8. Update database with flag_modified
# 9. Return success response with all statuses
```
## Test Coverage
Comprehensive test suite in `test_delete_custom_task_status.py`:
1. ✅ Delete unused custom status
2. ✅ Delete status in use without reassignment (error)
3. ✅ Delete status in use with reassignment
4. ✅ Delete default status (auto-assign new default)
5. ✅ Prevent deletion of last status
6. ✅ Delete non-existent status (error)
## Requirements Validation
✅ **3.1**: Check if status is in use by any tasks
✅ **3.2**: Return error with task count and IDs if in use
✅ **3.3**: Support optional reassignment of tasks to another status
✅ **3.4**: Auto-assign new default if deleting default status
✅ **3.5**: Prevent deletion of last status
## Database Considerations
- Uses `flag_modified()` for JSON column updates (required for SQLAlchemy to detect changes)
- Transactional: All changes (status deletion + task reassignments) happen in one transaction
- Rollback on any error to maintain data consistency
## Usage Examples
### Delete unused status
```bash
DELETE /projects/1/task-statuses/custom_abc123
```
### Delete status with reassignment
```bash
DELETE /projects/1/task-statuses/custom_abc123?reassign_to_status_id=not_started
```
### Delete status with reassignment to another custom status
```bash
DELETE /projects/1/task-statuses/custom_abc123?reassign_to_status_id=custom_xyz789
```
## Integration Notes
- Works seamlessly with existing custom status endpoints (GET, POST, PUT)
- Maintains consistency with system statuses (cannot delete system statuses)
- Properly updates the AllTaskStatusesResponse to reflect changes
- Frontend can use the returned `all_statuses` to update UI immediately
## Future Enhancements
Potential improvements for future iterations:
- Bulk delete with single reassignment target
- Soft delete with archive functionality
- Status usage analytics before deletion
- Undo/restore deleted statuses
backend/docs/custom-task-status-get-endpoint.md
+194
View File
@@ -0,0 +1,194 @@
# Custom Task Status GET Endpoint Implementation
## Overview
Implemented the GET endpoint for retrieving all task statuses (system + custom) for a project as part of the custom task status management feature.
## Endpoint Details
### GET /projects/{project_id}/task-statuses
**Description**: Retrieves all task statuses (both system and custom) for a specific project.
**Authentication**: Required (JWT Bearer token)
**Authorization**:
- Artists: Can only access projects they are members of
- Coordinators/Directors/Developers/Admins: Can access all projects
**Path Parameters**:
- `project_id` (int): The ID of the project
**Response Schema**: `AllTaskStatusesResponse`
```json
{
"statuses": [
{
"id": "custom_review",
"name": "In Review",
"color": "#8B5CF6",
"order": 0,
"is_default": false
}
],
"system_statuses": [
{
"id": "not_started",
"name": "Not Started",
"color": "#6B7280",
"is_system": true
},
{
"id": "in_progress",
"name": "In Progress",
"color": "#3B82F6",
"is_system": true
},
{
"id": "submitted",
"name": "Submitted",
"color": "#F59E0B",
"is_system": true
},
{
"id": "approved",
"name": "Approved",
"color": "#10B981",
"is_system": true
},
{
"id": "retake",
"name": "Retake",
"color": "#EF4444",
"is_system": true
}
],
"default_status_id": "not_started"
}
```
**Status Codes**:
- `200 OK`: Successfully retrieved task statuses
- `403 Forbidden`: User does not have access to the project
- `404 Not Found`: Project does not exist
## System Task Statuses
The following system statuses are always available:
| ID | Name | Color | Description |
|----|------|-------|-------------|
| not_started | Not Started | #6B7280 | Task has not been started |
| in_progress | In Progress | #3B82F6 | Task is currently being worked on |
| submitted | Submitted | #F59E0B | Work has been submitted for review |
| approved | Approved | #10B981 | Work has been approved |
| retake | Retake | #EF4444 | Work needs to be redone |
## Implementation Details
### Location
- File: `backend/routers/projects.py`
- Function: `get_all_task_statuses()`
### Key Features
1. **Project Validation**: Verifies the project exists before returning statuses
2. **Access Control**: Enforces role-based access control (artists can only access projects they're members of)
3. **System Statuses**: Always returns the 5 built-in system statuses
4. **Custom Statuses**: Returns project-specific custom statuses if defined
5. **Default Status**: Identifies which status is the default for new tasks
6. **JSON Handling**: Properly handles both JSON string and dict formats for custom_task_statuses field
### Database Schema
Custom statuses are stored in the `projects` table:
- Column: `custom_task_statuses` (JSON)
- Format: Array of status objects with id, name, color, order, and is_default fields
### Access Control Logic
```python
# Artists can only access projects they're members of
if current_user.role == UserRole.ARTIST:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == project_id,
ProjectMember.user_id == current_user.id
).first()
if not member:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this project"
)
```
## Testing
### Test Files Created
1. **test_task_statuses.py**: Basic functionality test
- Tests retrieval of system statuses
- Validates response structure
- Verifies all system statuses are present
2. **test_task_statuses_with_custom.py**: Custom status test
- Creates custom statuses in database
- Tests retrieval of both system and custom statuses
- Validates default status identification
3. **test_task_statuses_access.py**: Access control test
- Tests artist access control (member vs non-member)
- Tests coordinator access to all projects
4. **test_task_statuses_errors.py**: Error handling test
- Tests 404 for non-existent projects
- Tests 401/403 for unauthorized access
### Test Results
All tests passed successfully:
- ✅ System statuses correctly returned
- ✅ Custom statuses correctly returned
- ✅ Default status correctly identified
- ✅ Access control working for artists
- ✅ Coordinators can access all projects
- ✅ 404 returned for non-existent projects
- ✅ Unauthorized access properly blocked
## Requirements Validation
This implementation satisfies the following requirements:
- **Requirement 1.1**: ✅ Displays task status information for project settings
- **Requirement 9.2**: ✅ Only shows statuses from the task's project (project-specific)
## Usage Example
```python
import requests
# Login
response = requests.post(
"http://localhost:8000/auth/login",
json={"email": "user@example.com", "password": "password"}
)
token = response.json()["access_token"]
# Get task statuses for project
response = requests.get(
"http://localhost:8000/projects/1/task-statuses",
headers={"Authorization": f"Bearer {token}"}
)
statuses = response.json()
print(f"System statuses: {len(statuses['system_statuses'])}")
print(f"Custom statuses: {len(statuses['statuses'])}")
print(f"Default status: {statuses['default_status_id']}")
```
## Next Steps
The following endpoints still need to be implemented:
- POST /projects/{project_id}/task-statuses - Create custom status
- PUT /projects/{project_id}/task-statuses/{status_id} - Update custom status
- DELETE /projects/{project_id}/task-statuses/{status_id} - Delete custom status
- PATCH /projects/{project_id}/task-statuses/reorder - Reorder statuses
backend/docs/custom-task-status-migration.md
+115
View File
@@ -0,0 +1,115 @@
# Custom Task Status Migration - Task 1 Implementation Summary
## Overview
Successfully implemented database schema changes and migration to support custom task statuses in the VFX Project Management System.
## Changes Made
### 1. Database Schema Updates
#### Project Model (`backend/models/project.py`)
- ✅ Added `custom_task_statuses` JSON column to store project-specific custom statuses
- Column initialized with empty array `[]` for all existing projects
#### Task Model (`backend/models/task.py`)
- ✅ Changed `status` field from `Enum(TaskStatus)` to `String`
- Updated default value from `TaskStatus.NOT_STARTED` to `"not_started"`
- Maintains backward compatibility with existing system statuses
### 2. Schema Updates (`backend/schemas/task.py`)
- ✅ Updated `TaskBase.status` from `TaskStatus` enum to `str` type
- ✅ Updated `TaskUpdate.status` from `Optional[TaskStatus]` to `Optional[str]`
- ✅ Updated `TaskStatusUpdate.status` from `TaskStatus` to `str`
- Changed default value to `"not_started"` (lowercase string)
### 3. Router Updates
Updated all routers to use string values instead of TaskStatus enum:
#### Assets Router (`backend/routers/assets.py`)
- ✅ Changed task creation to use `status="not_started"`
- ✅ Updated status initialization in task status aggregation
- ✅ Updated status filtering to use string comparison
- ✅ Updated status sorting to use string-based status order
#### Reviews Router (`backend/routers/reviews.py`)
- ✅ Updated submission status checks to use `"submitted"` string
- ✅ Changed approval status update to `"approved"` string
- ✅ Changed retake status update to `"retake"` string
#### Shots Router (`backend/routers/shots.py`)
- ✅ Changed task creation to use `status="not_started"`
- ✅ Updated status initialization in task status aggregation
- ✅ Updated status filtering to use string comparison
- ✅ Updated status sorting to use string-based status order
### 4. Migration Script (`backend/migrate_custom_task_statuses.py`)
Created comprehensive migration script that:
- ✅ Adds `custom_task_statuses` column to projects table
- ✅ Initializes column with empty array `[]` for existing projects
- ✅ Converts existing uppercase enum values to lowercase strings:
- `NOT_STARTED` → `not_started`
- `IN_PROGRESS` → `in_progress`
- `SUBMITTED` → `submitted`
- `APPROVED` → `approved`
- `RETAKE` → `retake`
- ✅ Provides detailed logging of conversion process
- ✅ Includes verification steps
### 5. Test Script (`backend/test_custom_task_status_migration.py`)
Created verification test that confirms:
- ✅ `custom_task_statuses` column exists in projects table
- ✅ Task status column supports string values
- ✅ All existing task statuses are valid lowercase strings
- ✅ `custom_task_statuses` is initialized as empty array
- ✅ Displays task status distribution
## Migration Results
### Database Changes
```
Projects Table:
- Added column: custom_task_statuses (TEXT/JSON)
Tasks Table:
- Status column: VARCHAR(11) (already TEXT, no change needed)
```
### Data Conversion
Successfully converted 105 tasks:
- 91 tasks: `NOT_STARTED` → `not_started`
- 9 tasks: `IN_PROGRESS` → `in_progress`
- 4 tasks: `RETAKE` → `retake`
- 1 task: `APPROVED` → `approved`
## System Status Values
The following system statuses remain available:
- `not_started` - Task has not been started
- `in_progress` - Task is currently being worked on
- `submitted` - Task work has been submitted for review
- `approved` - Task has been approved
- `retake` - Task requires revisions
## Backward Compatibility
- ✅ All existing tasks continue to work with lowercase string statuses
- ✅ System statuses are always available across all projects
- ✅ No breaking changes to existing API endpoints
- ✅ Frontend can continue using existing status values
## Next Steps
With the database schema and migration complete, the system is ready for:
1. Backend API endpoints for custom status CRUD operations (Task 2-8)
2. Frontend components for custom status management (Task 10-21)
3. Integration with task creation and status update workflows
## Testing
All tests passed successfully:
- ✅ Backend imports without errors
- ✅ Database schema verification
- ✅ Data conversion verification
- ✅ Status value validation
## Requirements Validated
- ✅ Requirement 6.1: System statuses remain available
- ✅ Requirement 6.2: Backward compatibility maintained
- ✅ Requirement 6.3: Existing tasks continue to work
- ✅ Requirement 9.1: Database schema supports custom statuses
backend/docs/custom-task-status-reorder-endpoint.md
+381
View File
@@ -0,0 +1,381 @@
# Custom Task Status Reorder Endpoint
## Overview
This document describes the PATCH endpoint for reordering custom task statuses within a project. The endpoint allows coordinators and administrators to change the display order of custom task statuses.
## Endpoint
```
PATCH /projects/{project_id}/task-statuses/reorder
```
## Authentication
Requires JWT authentication with coordinator or admin role.
## Request
### Path Parameters
- `project_id` (integer, required): The ID of the project
### Request Body
```json
{
"status_ids": ["custom_abc123", "custom_def456", "custom_ghi789"]
}
```
**Fields:**
- `status_ids` (array of strings, required): Ordered list of status IDs in the desired sequence
- Must contain all existing custom status IDs for the project
- Cannot contain duplicates
- Cannot be empty
## Response
### Success Response (200 OK)
```json
{
"message": "Custom task statuses reordered successfully",
"status": null,
"all_statuses": {
"statuses": [
{
"id": "custom_abc123",
"name": "Review",
"color": "#9333EA",
"order": 0,
"is_default": false
},
{
"id": "custom_def456",
"name": "Blocked",
"color": "#DC2626",
"order": 1,
"is_default": true
},
{
"id": "custom_ghi789",
"name": "Ready for Delivery",
"color": "#059669",
"order": 2,
"is_default": false
}
],
"system_statuses": [
{
"id": "not_started",
"name": "Not Started",
"color": "#6B7280",
"is_system": true
},
{
"id": "in_progress",
"name": "In Progress",
"color": "#3B82F6",
"is_system": true
},
{
"id": "submitted",
"name": "Submitted",
"color": "#F59E0B",
"is_system": true
},
{
"id": "approved",
"name": "Approved",
"color": "#10B981",
"is_system": true
},
{
"id": "retake",
"name": "Retake",
"color": "#EF4444",
"is_system": true
}
],
"default_status_id": "custom_def456"
}
}
```
### Error Responses
#### 400 Bad Request - Missing Status IDs
```json
{
"detail": "Missing status IDs in reorder request: custom_xyz999"
}
```
Occurs when the request doesn't include all existing custom status IDs.
#### 400 Bad Request - Invalid Status IDs
```json
{
"detail": "Status IDs not found: invalid_id_12345"
}
```
Occurs when the request includes status IDs that don't exist in the project.
#### 403 Forbidden
```json
{
"detail": "Insufficient permissions"
}
```
Occurs when the user doesn't have coordinator or admin role.
#### 404 Not Found
```json
{
"detail": "Project not found"
}
```
Occurs when the specified project doesn't exist.
#### 422 Unprocessable Entity - Duplicate IDs
```json
{
"detail": "1 validation error for CustomTaskStatusReorder\nstatus_ids\n Value error, Status IDs list contains duplicates"
}
```
Occurs when the request contains duplicate status IDs.
#### 422 Unprocessable Entity - Empty List
```json
{
"detail": "1 validation error for CustomTaskStatusReorder\nstatus_ids\n Value error, Status IDs list cannot be empty"
}
```
Occurs when the request contains an empty status_ids array.
## Implementation Details
### Validation
1. **Project Existence**: Verifies the project exists
2. **Permission Check**: Ensures user has coordinator or admin role
3. **Complete List**: Validates that all existing custom status IDs are included
4. **No Missing IDs**: Ensures no status IDs are omitted
5. **No Invalid IDs**: Ensures all provided IDs exist in the project
6. **No Duplicates**: Validates the list contains no duplicate IDs (handled by Pydantic schema)
### Order Update Process
1. Parse and validate the reorder request
2. Retrieve existing custom statuses from the project
3. Create a mapping of status_id to status data
4. Reorder statuses according to the provided list
5. Update the `order` field for each status (0-indexed)
6. Save the reordered list to the database
7. Use `flag_modified()` to ensure JSON column changes are persisted
### Database Changes
- Updates the `custom_task_statuses` JSON column in the `projects` table
- Each status object's `order` field is updated to match its position in the new list
- Uses SQLAlchemy's `flag_modified()` to ensure JSON column changes are detected
## Usage Examples
### Python (requests)
```python
import requests
# Login and get token
response = requests.post(
"http://localhost:8000/auth/login",
json={"email": "admin@vfx.com", "password": "admin123"}
)
token = response.json()["access_token"]
# Reorder statuses
headers = {"Authorization": f"Bearer {token}"}
data = {
"status_ids": [
"custom_abc123",
"custom_def456",
"custom_ghi789"
]
}
response = requests.patch(
"http://localhost:8000/projects/1/task-statuses/reorder",
headers=headers,
json=data
)
if response.status_code == 200:
result = response.json()
print(f"✅ {result['message']}")
print(f"Reordered {len(result['all_statuses']['statuses'])} statuses")
else:
print(f"❌ Error: {response.json()['detail']}")
```
### JavaScript (fetch)
```javascript
// Assuming you have a token from login
const token = "your_jwt_token_here";
const reorderStatuses = async (projectId, statusIds) => {
const response = await fetch(
`http://localhost:8000/projects/${projectId}/task-statuses/reorder`,
{
method: 'PATCH',
headers: {
'Authorization': `Bearer ${token}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
status_ids: statusIds
})
}
);
if (response.ok) {
const result = await response.json();
console.log('✅', result.message);
return result.all_statuses;
} else {
const error = await response.json();
console.error('❌ Error:', error.detail);
throw new Error(error.detail);
}
};
// Usage
const statusIds = [
'custom_abc123',
'custom_def456',
'custom_ghi789'
];
reorderStatuses(1, statusIds)
.then(allStatuses => {
console.log('New order:', allStatuses.statuses);
})
.catch(error => {
console.error('Failed to reorder:', error);
});
```
## Frontend Integration
### Drag-and-Drop Implementation
The frontend should implement drag-and-drop functionality using a library like `vue-draggable-next`:
1. Display statuses in their current order
2. Allow users to drag statuses to reorder them
3. On drop, collect the new order of status IDs
4. Call the reorder endpoint with the new order
5. Update the UI optimistically or wait for the response
6. Handle errors by reverting to the previous order
### Example Vue Component
```vue
<template>
<draggable
v-model="statuses"
@end="handleReorder"
item-key="id"
>
<template #item="{ element }">
<div class="status-item">
<span class="drag-handle">⋮⋮</span>
<span :style="{ color: element.color }">
{{ element.name }}
</span>
</div>
</template>
</draggable>
</template>
<script setup>
import { ref } from 'vue';
import draggable from 'vuedraggable';
import { reorderTaskStatuses } from '@/services/customTaskStatus';
const props = defineProps({
projectId: Number,
initialStatuses: Array
});
const statuses = ref([...props.initialStatuses]);
const handleReorder = async () => {
const statusIds = statuses.value.map(s => s.id);
try {
const result = await reorderTaskStatuses(props.projectId, statusIds);
// Update with server response
statuses.value = result.all_statuses.statuses;
} catch (error) {
// Revert to original order on error
statuses.value = [...props.initialStatuses];
console.error('Failed to reorder:', error);
}
};
</script>
```
## Testing
A comprehensive test script is available at `backend/test_reorder_custom_task_status.py` that tests:
1. ✅ Successful reordering (reversing order)
2. ✅ Order field updates correctly
3. ✅ Rejection of incomplete status lists
4. ✅ Rejection of invalid status IDs
5. ✅ Rejection of duplicate status IDs
Run the test with:
```bash
cd backend
python test_reorder_custom_task_status.py
```
## Requirements Validation
This endpoint satisfies the following requirements from the custom task status specification:
- **Requirement 4.1**: ✅ Displays statuses in their defined order
- **Requirement 4.2**: ✅ Updates the order when user reorders statuses
- **Requirement 4.3**: ✅ Updates display order in all dropdowns and filters
- **Requirement 4.4**: ✅ Validates all status IDs are present
## Related Endpoints
- `GET /projects/{project_id}/task-statuses` - Get all task statuses
- `POST /projects/{project_id}/task-statuses` - Create a custom status
- `PUT /projects/{project_id}/task-statuses/{status_id}` - Update a custom status
- `DELETE /projects/{project_id}/task-statuses/{status_id}` - Delete a custom status
## Notes
- System statuses (not_started, in_progress, submitted, approved, retake) cannot be reordered
- Only custom statuses can be reordered
- The order field is 0-indexed
- Reordering does not affect the default status designation
- The endpoint uses `flag_modified()` to ensure JSON column changes are persisted to the database
backend/docs/custom-task-status-update-endpoint.md
+260
View File
@@ -0,0 +1,260 @@
# Custom Task Status Update Endpoint
## Overview
This document describes the implementation of the PUT endpoint for updating custom task statuses in a project.
**Endpoint:** `PUT /projects/{project_id}/task-statuses/{status_id}`
**Requirements Implemented:**
- 2.1: Support updating name
- 2.2: Support updating color
- 2.3: Support updating is_default flag
- 5.2: If setting as default, unset other default statuses
## Implementation Details
### Endpoint Signature
```python
@router.put("/{project_id}/task-statuses/{status_id}")
async def update_custom_task_status(
project_id: int,
status_id: str,
status_update: dict,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
```
### Request Body Schema
Uses `CustomTaskStatusUpdate` schema:
```python
{
"name": "string (optional)", # New status name (1-50 chars)
"color": "string (optional)", # Hex color code (e.g., #FF5733)
"is_default": "boolean (optional)" # Set as default status
}
```
### Response Schema
Returns `CustomTaskStatusResponse`:
```python
{
"message": "string",
"status": {
"id": "string",
"name": "string",
"color": "string",
"order": "integer",
"is_default": "boolean"
},
"all_statuses": {
"statuses": [...], # All custom statuses
"system_statuses": [...], # System statuses
"default_status_id": "string"
}
}
```
## Features
### 1. Name Update (Requirement 2.1)
- Validates name uniqueness within project
- Checks against other custom statuses
- Checks against system status names
- Returns 409 Conflict if name already exists
```python
# Validate name uniqueness if name is being changed
if status_update.name is not None and status_update.name != status_to_update.get('name'):
# Check against other custom statuses
existing_names = [
s.get('name', '').lower()
for i, s in enumerate(custom_statuses_data)
if isinstance(s, dict) and i != status_index
]
# Check against system statuses
system_names = [s['name'].lower() for s in SYSTEM_TASK_STATUSES]
if status_update.name.lower() in existing_names:
raise HTTPException(status_code=409, detail="Name already exists")
if status_update.name.lower() in system_names:
raise HTTPException(status_code=409, detail="Conflicts with system status")
```
### 2. Color Update (Requirement 2.2)
- Accepts hex color codes (e.g., #FF5733)
- Validates color format via schema
- Updates color independently of other fields
```python
# Update color if provided
if status_update.color is not None:
status_to_update['color'] = status_update.color
```
### 3. Default Status Management (Requirement 2.3, 5.2)
- When setting a status as default, automatically unsets all other defaults
- Ensures only one default status exists at a time
- Allows unsetting default status
```python
# Handle is_default flag
if status_update.is_default is not None:
if status_update.is_default:
# If setting as default, unset other default statuses
for status_data in custom_statuses_data:
if isinstance(status_data, dict):
status_data['is_default'] = False
# Set this status as default
status_to_update['is_default'] = True
else:
# Just unset this status as default
status_to_update['is_default'] = False
```
### 4. JSON Column Updates
Uses `flag_modified` to ensure SQLAlchemy detects changes to JSON columns:
```python
# Update the status in the list
custom_statuses_data[status_index] = status_to_update
db_project.custom_task_statuses = custom_statuses_data
# Use flag_modified for JSON column updates
flag_modified(db_project, 'custom_task_statuses')
db.commit()
```
## Error Handling
### 404 Not Found
- Project doesn't exist
- Status ID not found in project
### 409 Conflict
- Status name already exists in project
- Status name conflicts with system status
### 403 Forbidden
- User is not coordinator or admin
### 422 Unprocessable Entity
- Invalid request body format
- Invalid color format
- Invalid name length
## Example Usage
### Update Status Name
```bash
curl -X PUT "http://localhost:8000/projects/1/task-statuses/custom_abc123" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"name": "New Status Name"}'
```
### Update Status Color
```bash
curl -X PUT "http://localhost:8000/projects/1/task-statuses/custom_abc123" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"color": "#00FF00"}'
```
### Update Both Name and Color
```bash
curl -X PUT "http://localhost:8000/projects/1/task-statuses/custom_abc123" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"name": "Updated Status", "color": "#0000FF"}'
```
### Set as Default Status
```bash
curl -X PUT "http://localhost:8000/projects/1/task-statuses/custom_abc123" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"is_default": true}'
```
## Testing
To test this endpoint:
1. Start the backend server:
```bash
cd backend
uvicorn main:app --reload
```
2. Create a custom status first (if needed):
```bash
curl -X POST "http://localhost:8000/projects/1/task-statuses" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"name": "Test Status", "color": "#FF5733"}'
```
3. Update the status using the examples above
4. Verify changes by getting all statuses:
```bash
curl -X GET "http://localhost:8000/projects/1/task-statuses" \
-H "Authorization: Bearer <token>"
```
## Integration with Frontend
The frontend can use this endpoint to:
1. Update status names when users edit them
2. Change status colors via color picker
3. Set/unset default status via toggle or button
4. Update multiple fields at once
The response includes the updated status and all statuses, allowing the frontend to update its state in a single request.
## Database Schema
The custom task statuses are stored in the `projects` table as a JSON column:
```sql
custom_task_statuses JSON -- Array of status objects
```
Each status object has the structure:
```json
{
"id": "custom_abc123",
"name": "Status Name",
"color": "#FF5733",
"order": 0,
"is_default": false
}
```
## Related Endpoints
- `GET /projects/{project_id}/task-statuses` - Get all statuses
- `POST /projects/{project_id}/task-statuses` - Create new status
- `DELETE /projects/{project_id}/task-statuses/{status_id}` - Delete status (to be implemented)
- `PATCH /projects/{project_id}/task-statuses/reorder` - Reorder statuses (to be implemented)
backend/docs/data-consistency-implementation.md
+231
View File
@@ -0,0 +1,231 @@
# Data Consistency and Real-time Updates Implementation
## Overview
This document describes the implementation of data consistency checks and real-time update propagation for the shot-asset-task-status-optimization feature. The implementation ensures that individual task updates remain consistent with aggregated views and provides real-time update propagation mechanisms.
## Requirements Addressed
This implementation addresses the following requirements from the specification:
- **Requirement 3.3**: Data consistency between individual task updates and aggregated views
- **Requirement 4.5**: Real-time update propagation to aggregated data
- **Task 14**: Data Consistency and Real-time Updates
## Architecture
### Core Components
1. **DataConsistencyService** (`backend/services/data_consistency.py`)
- Main service for validating consistency between individual tasks and aggregated data
- Provides bulk validation and reporting capabilities
- Handles real-time update propagation
2. **Data Consistency API** (`backend/routers/data_consistency.py`)
- REST API endpoints for consistency validation and monitoring
- Health check and reporting endpoints
- Administrative tools for consistency management
3. **Task Update Hooks** (integrated into `backend/routers/tasks.py`)
- Automatic consistency validation on task status updates
- Propagation logging and error handling
- Integration with existing task update workflows
## Implementation Details
### Data Consistency Validation
The system validates consistency by:
1. **Fetching Individual Task Records**: Queries all active tasks for a shot or asset
2. **Building Expected Aggregated Data**: Constructs the expected task_status and task_details from individual tasks
3. **Fetching Actual Aggregated Data**: Uses the optimized queries to get current aggregated data
4. **Comparing Results**: Identifies inconsistencies between expected and actual data
#### Validation Process
```python
def validate_task_aggregation_consistency(self, entity_id: int, entity_type: str) -> Dict[str, Any]:
# Get individual task records
tasks = self.db.query(Task).filter(conditions).all()
# Build expected aggregated data
expected_task_status = {}
expected_task_details = []
# Get actual aggregated data using optimized queries
aggregated_data = self._get_shot_aggregated_data(entity_id) # or asset
# Compare and identify inconsistencies
inconsistencies = []
# ... comparison logic
return {
'valid': len(inconsistencies) == 0,
'inconsistencies': inconsistencies,
# ... additional metadata
}
```
### Real-time Update Propagation
The system ensures real-time consistency through:
1. **Task Update Hooks**: Automatically triggered on task status changes
2. **Consistency Validation**: Validates aggregated data after each update
3. **Propagation Logging**: Records all update propagations for monitoring
4. **Error Handling**: Logs inconsistencies without failing user operations
#### Update Propagation Flow
```python
def propagate_task_update(self, task_id: int, old_status: str, new_status: str) -> Dict[str, Any]:
# Get task and determine parent entity
task = self.db.query(Task).filter(Task.id == task_id).first()
# Validate consistency after update
validation_result = self.validate_task_aggregation_consistency(entity_id, entity_type)
# Log propagation results
propagation_log = {
'task_id': task_id,
'entity_type': entity_type,
'entity_id': entity_id,
'old_status': old_status,
'new_status': new_status,
'consistency_valid': validation_result['valid'],
'timestamp': datetime.utcnow().isoformat()
}
return propagation_log
```
### Integration with Task Updates
The consistency system is integrated into existing task update endpoints:
1. **Individual Task Updates** (`PUT /tasks/{task_id}`)
2. **Task Status Updates** (`PUT /tasks/{task_id}/status`)
3. **Bulk Status Updates** (`PUT /tasks/bulk/status`)
Each endpoint now includes:
- Pre-update status capture
- Post-update consistency validation
- Propagation logging
- Error handling that doesn't disrupt user operations
## API Endpoints
### Data Consistency Endpoints
All endpoints are prefixed with `/data-consistency` and require admin or coordinator permissions.
#### Validation Endpoints
- `GET /data-consistency/validate/{entity_type}/{entity_id}`
- Validate consistency for a specific shot or asset
- Returns detailed validation results and any inconsistencies found
- `POST /data-consistency/validate/bulk`
- Validate consistency for multiple entities at once
- Supports up to 100 entities per request
#### Reporting Endpoints
- `GET /data-consistency/report?project_id={id}`
- Generate comprehensive consistency report
- Optional project filtering
- Returns summary statistics and detailed results
- `GET /data-consistency/health?project_id={id}`
- Quick health check for data consistency
- Returns overall system health status
- Useful for monitoring and alerting
#### Management Endpoints
- `POST /data-consistency/propagate/{task_id}`
- Manually trigger update propagation for a task
- Useful for debugging and maintenance
## Testing
### Unit Tests
The implementation includes comprehensive unit tests:
- **test_data_consistency.py**: Core functionality testing
- Data consistency validation
- Real-time update propagation
- Consistency reporting
- Bulk validation operations
### API Integration Tests
- **test_data_consistency_api.py**: API endpoint testing
- Authentication and authorization
- Endpoint functionality
- Error handling
- Response format validation
### Running Tests
```bash
# Run core functionality tests
cd backend
python test_data_consistency.py
# Run API integration tests (requires running server)
python test_data_consistency_api.py
```
## Monitoring and Maintenance
### Consistency Health Monitoring
The system provides several monitoring capabilities:
1. **Health Check Endpoint**: Quick status overview
2. **Detailed Reports**: Comprehensive consistency analysis
3. **Propagation Logging**: Audit trail of all updates
4. **Error Logging**: Automatic logging of consistency issues
### Maintenance Operations
1. **Bulk Validation**: Validate consistency across multiple entities
2. **Manual Propagation**: Force update propagation for specific tasks
3. **Consistency Reports**: Generate detailed analysis reports
### Performance Considerations
- Consistency validation uses the same optimized queries as the main system
- Bulk operations are limited to prevent performance impact
- Validation is performed asynchronously to avoid blocking user operations
- Logging is designed to be lightweight and non-intrusive
## Error Handling
The system is designed to be resilient:
1. **Non-blocking Operations**: Consistency issues don't prevent task updates
2. **Graceful Degradation**: System continues to function even with consistency problems
3. **Comprehensive Logging**: All issues are logged for investigation
4. **Recovery Mechanisms**: Manual tools available for fixing inconsistencies
## Configuration
The data consistency system requires no additional configuration and integrates seamlessly with the existing system. All settings use the same database connection and authentication mechanisms as the main application.
## Future Enhancements
Potential improvements for future versions:
1. **Automated Repair**: Automatic fixing of detected inconsistencies
2. **Real-time Notifications**: Alert administrators of consistency issues
3. **Performance Metrics**: Detailed performance monitoring and optimization
4. **Batch Processing**: Scheduled consistency validation jobs
5. **Custom Validation Rules**: Project-specific consistency requirements
## Conclusion
The data consistency implementation provides robust validation and monitoring capabilities while maintaining system performance and reliability. It ensures that the optimized query system continues to provide accurate data while offering tools for monitoring and maintaining data integrity over time.
backend/docs/default-asset-tasks-implementation.md
+221
View File
@@ -0,0 +1,221 @@
# Default Asset Task Creation Implementation
## Overview
This document describes the implementation of automatic default task creation for assets based on their category (Task 5.3).
## Requirements Implemented
- **17.1**: Automatic task generation when assets are created based on asset category
- **17.2**: Default task templates for each asset category (modeling, surfacing, rigging)
- **17.3**: Customizable task creation options for coordinators
- **17.4**: Default task naming conventions
- **17.5**: API endpoint for retrieving default tasks by asset category
- **17.6**: Proper task naming
- **17.7**: Unassigned task creation
## Backend Implementation
### Default Task Templates
Located in `backend/routers/assets.py`:
```python
DEFAULT_ASSET_TASKS = {
AssetCategory.CHARACTERS: ["modeling", "surfacing", "rigging"],
AssetCategory.PROPS: ["modeling", "surfacing"],
AssetCategory.SETS: ["modeling", "surfacing"],
AssetCategory.VEHICLES: ["modeling", "surfacing", "rigging"]
}
```
### Key Functions
#### `get_default_asset_task_types(category: AssetCategory) -> List[str]`
Returns the default task types for a given asset category.
#### `get_all_asset_task_types(project_id: int, db: Session) -> List[str]`
Returns all task types (standard + custom) for assets in a project.
#### `create_default_tasks_for_asset(asset: Asset, task_types: List[str], db: Session) -> List[Task]`
Creates default tasks for an asset with proper naming conventions:
- Task name format: `{asset_name} - {task_type.title()}`
- Tasks are created with status `NOT_STARTED`
- Tasks are left unassigned (assigned_user_id = None)
### API Endpoints
#### `GET /assets/default-tasks/{category}`
Returns default task types for an asset category.
**Query Parameters:**
- `project_id` (optional): Include custom task types for the project
**Response:**
```json
["modeling", "surfacing", "rigging"]
```
#### `POST /assets/?project_id={project_id}`
Creates a new asset with optional default tasks.
**Request Body:**
```json
{
"name": "Hero Character",
"category": "characters",
"description": "Main character asset",
"status": "not_started",
"create_default_tasks": true,
"selected_task_types": ["modeling", "surfacing", "rigging"]
}
```
**Fields:**
- `create_default_tasks` (boolean): Whether to create default tasks (default: true)
- `selected_task_types` (array, optional): Specific task types to create. If not provided, uses category defaults.
**Response:**
```json
{
"id": 1,
"name": "Hero Character",
"category": "characters",
"status": "not_started",
"project_id": 1,
"task_count": 3,
"created_at": "2024-01-01T00:00:00Z",
"updated_at": "2024-01-01T00:00:00Z"
}
```
## Frontend Implementation
### AssetForm Component
Located in `frontend/src/components/asset/AssetForm.vue`
**Features:**
1. **Default Tasks Toggle**: Checkbox to enable/disable default task creation
2. **Task Preview**: Shows which tasks will be created based on category
3. **Custom Task Selection**: Allows coordinators to select specific tasks to create
4. **Confirmation Dialog**: Shows preview of tasks before creation
**Key Sections:**
```vue
<!-- Default Tasks Section -->
<div v-if="!isEdit" class="space-y-4 border-t pt-4">
<div class="flex items-center space-x-2">
<Checkbox
id="create-default-tasks"
v-model:checked="formData.create_default_tasks"
/>
<Label>Create default tasks for this asset</Label>
</div>
<!-- Task Selection -->
<div v-if="formData.create_default_tasks && formData.category">
<div v-for="taskType in defaultTasks" :key="taskType">
<Checkbox
:checked="selectedTaskTypes.includes(taskType)"
@update:checked="toggleTaskType(taskType)"
/>
<Label>{{ formatTaskType(taskType) }}</Label>
</div>
</div>
</div>
```
### Asset Service
Located in `frontend/src/services/asset.ts`
**Methods:**
```typescript
async getDefaultTasksForCategory(
category: AssetCategory,
projectId?: number
): Promise<string[]>
async createAsset(
projectId: number,
data: AssetCreate
): Promise<Asset>
```
## Task Naming Convention
Tasks are automatically named using the format:
```
{Asset Name} - {Task Type}
```
Examples:
- "Hero Character - Modeling"
- "Hero Character - Surfacing"
- "Hero Character - Rigging"
- "Sword Prop - Modeling"
- "Sword Prop - Surfacing"
## Task Assignment
All default tasks are created **unassigned** (assigned_user_id = null). Coordinators must manually assign tasks to artists after creation.
## Category-Specific Defaults
| Category | Default Tasks |
|------------|-----------------------------------|
| Characters | Modeling, Surfacing, Rigging |
| Props | Modeling, Surfacing |
| Sets | Modeling, Surfacing |
| Vehicles | Modeling, Surfacing, Rigging |
## Custom Task Types
The system supports custom task types per project. When a project has custom asset task types defined, they are included in the available task types for selection during asset creation.
## Testing
### Backend Test
Run `backend/test_default_asset_tasks.py` to verify:
- Default task templates for each category
- Asset creation with default tasks
- Task naming conventions
- Custom task selection
- Unassigned task creation
### Frontend Test
Open `frontend/test-default-asset-tasks.html` to test:
- Default task retrieval
- Asset creation with task selection
- Task verification
## Usage Example
### Creating an Asset with Default Tasks
1. Navigate to Assets section in a project
2. Click "Create Asset"
3. Fill in asset details (name, category, description)
4. Ensure "Create default tasks" is checked
5. Review the task preview
6. Optionally customize which tasks to create
7. Click "Create Asset"
8. Confirm task creation in the dialog
### Creating an Asset without Default Tasks
1. Follow steps 1-3 above
2. Uncheck "Create default tasks"
3. Click "Create Asset"
4. Asset is created with no tasks
### Custom Task Selection
1. Follow steps 1-4 above
2. Uncheck specific tasks you don't want to create
3. Click "Create Asset"
4. Only selected tasks will be created
## Integration with Custom Task Types
When custom task types are defined for a project (via Task 19), they are automatically included in the available task types for asset creation. The default task templates remain the same, but coordinators can select custom task types during asset creation.
## Error Handling
- Validates that selected task types are valid (standard or custom)
- Prevents duplicate asset names within a project
- Returns appropriate error messages for invalid requests
- Handles missing project or category gracefully
## Performance Considerations
- Tasks are created in a single database transaction
- Bulk task creation is efficient using SQLAlchemy's bulk operations
- Task count is returned immediately after creation without additional queries
backend/docs/fastapi-trailing-slash-issue.md
+155
View File
@@ -0,0 +1,155 @@
# FastAPI Trailing Slash Issue - 307 Redirect & 403 Forbidden
## Problem Description
When making authenticated API calls to FastAPI endpoints, a mismatch in trailing slashes between the frontend request and backend route definition causes a **307 Temporary Redirect** that **loses the authentication header**, resulting in a **403 Forbidden** error.
## Root Cause
FastAPI automatically redirects requests to add or remove trailing slashes to match the route definition:
- If route is defined as `@router.get("/tasks/")` (with slash) and you call `/tasks` (without slash) → 307 redirect to `/tasks/`
- If route is defined as `@router.get("/tasks")` (without slash) and you call `/tasks/` (with slash) → 307 redirect to `/tasks`
**The problem:** HTTP redirects (307) do NOT preserve the `Authorization` header by default, so the redirected request arrives without authentication, causing a 403 Forbidden error.
## Symptoms
### Backend Logs
```
INFO: 127.0.0.1:59653 - "GET /tasks?shot_id=12 HTTP/1.1" 307 Temporary Redirect
INFO: 127.0.0.1:59615 - "GET /tasks/?shot_id=12 HTTP/1.1" 403 Forbidden
```
### Frontend Console
```
GET http://localhost:8000/tasks/?shot_id=12 403 (Forbidden)
AxiosError {message: 'Request failed with status code 403', ...}
```
## Solution
**Always ensure trailing slashes match between frontend API calls and backend route definitions.**
### Option 1: Add Trailing Slash to Frontend (Recommended)
**Frontend Service:**
```typescript
// ❌ WRONG - No trailing slash
const response = await apiClient.get(`/tasks?${params}`)
// ✅ CORRECT - With trailing slash
const response = await apiClient.get(`/tasks/?${params}`)
```
**Backend Route:**
```python
# Route defined WITH trailing slash
@router.get("/tasks/")
async def get_tasks(...):
...
```
### Option 2: Remove Trailing Slash from Backend
**Backend Route:**
```python
# Route defined WITHOUT trailing slash
@router.get("/tasks")
async def get_tasks(...):
...
```
**Frontend Service:**
```typescript
// Call WITHOUT trailing slash
const response = await apiClient.get(`/tasks?${params}`)
```
## Prevention Checklist
When adding or modifying routes, **always check**:
1. **Backend Route Definition** - Does it have a trailing slash?
```python
@router.get("/endpoint/") # Has trailing slash
@router.get("/endpoint") # No trailing slash
```
2. **Frontend API Call** - Does it match the backend?
```typescript
apiClient.get(`/endpoint/`) // Has trailing slash
apiClient.get(`/endpoint`) // No trailing slash
```
3. **Query Parameters** - Trailing slash goes BEFORE the `?`
```typescript
// ✅ CORRECT
apiClient.get(`/tasks/?shot_id=12`)
// ❌ WRONG
apiClient.get(`/tasks?shot_id=12/`)
```
4. **Path Parameters** - Usually no trailing slash
```typescript
// ✅ CORRECT
apiClient.get(`/tasks/${taskId}`)
// ❌ WRONG (usually)
apiClient.get(`/tasks/${taskId}/`)
```
## Historical Issues Fixed
### Issue 1: Shots Endpoint (Fixed)
- **Problem:** Frontend called `/shots/1/` but backend defined `/shots/{shot_id}`
- **Solution:** Changed frontend to call `/shots/1` (no trailing slash)
- **Files:** `frontend/src/services/shot.ts`
### Issue 2: Tasks Endpoint (Fixed)
- **Problem:** Frontend called `/tasks?shot_id=12` but backend defined `/tasks/`
- **Solution:** Changed frontend to call `/tasks/?shot_id=12` (with trailing slash)
- **Files:** `frontend/src/services/task.ts`
## Testing
To verify there's no redirect issue:
1. **Check backend logs** - Should see only ONE request, not two:
```
✅ GOOD:
INFO: "GET /tasks/?shot_id=12 HTTP/1.1" 200 OK
❌ BAD (redirect happening):
INFO: "GET /tasks?shot_id=12 HTTP/1.1" 307 Temporary Redirect
INFO: "GET /tasks/?shot_id=12 HTTP/1.1" 403 Forbidden
```
2. **Check frontend network tab** - Should see 200 OK, not 307 or 403
3. **Test with authentication** - Ensure authenticated endpoints work correctly
## Quick Reference
### Common Patterns
| Endpoint Type | Backend Route | Frontend Call |
|--------------|---------------|---------------|
| List with query params | `@router.get("/items/")` | `get("/items/?param=value")` |
| Get by ID | `@router.get("/items/{id}")` | `get("/items/123")` |
| Create | `@router.post("/items/")` | `post("/items/", data)` |
| Update by ID | `@router.put("/items/{id}")` | `put("/items/123", data)` |
| Delete by ID | `@router.delete("/items/{id}")` | `delete("/items/123")` |
## Related Files
- Backend routes: `backend/routers/*.py`
- Frontend services: `frontend/src/services/*.ts`
- API client: `frontend/src/services/api.ts`
## Additional Notes
- This issue only affects authenticated endpoints because the `Authorization` header is lost during redirect
- Public endpoints might not show this issue as clearly
- Always test with actual authentication tokens, not just in development mode
- Consider adding a linter rule or pre-commit hook to check for trailing slash consistency
backend/docs/file-path-migration-guide.md
+341
View File
@@ -0,0 +1,341 @@
# File Path Migration Guide
## Overview
This guide covers the database migration scripts created to convert absolute file paths to relative paths in the VFX Project Management System. This migration addresses the issue where absolute paths stored in the database become invalid when deploying to different environments, particularly Linux.
## Problem Statement
The system previously stored absolute file paths like:
- `D:\Repo\LinkDesk\backend\uploads\submissions\98\v001_render.jpg`
- `/home/user/vfx-system/backend/uploads/attachments/123/reference.pdf`
These paths become invalid when:
- Deploying to different operating systems
- Moving the backend directory
- Running in containerized environments
## Solution
Convert all absolute paths to relative paths:
- `uploads/submissions/98/v001_render.jpg`
- `uploads/attachments/123/reference.pdf`
- `uploads/project_thumbnails/project_1_thumbnail.jpg`
## Migration Scripts
### 1. `migrate_file_paths_to_relative.py`
**Purpose**: Basic migration script for development use.
**Features**:
- Converts absolute paths to relative paths in all three tables
- Basic validation and error handling
- Interactive confirmation prompt
- Detailed logging
**Usage**:
```bash
cd backend
python migrate_file_paths_to_relative.py
```
### 2. `migrate_file_paths_production.py`
**Purpose**: Production-ready migration script with comprehensive validation.
**Features**:
- Automatic database backup creation
- Pre-migration analysis of problematic paths
- Enhanced error handling and recovery
- Comprehensive validation of results
- Detailed logging with timestamps
- Support for multiple absolute path patterns
**Usage**:
```bash
cd backend
python migrate_file_paths_production.py
```
### 3. `validate_migration_results.py`
**Purpose**: Validation script to verify migration success.
**Features**:
- Validates all paths are now relative
- Checks file accessibility
- Verifies path format consistency
- Comprehensive reporting
**Usage**:
```bash
cd backend
python validate_migration_results.py
```
## Tables Affected
### 1. `submissions` table
- **Column**: `file_path`
- **Purpose**: Stores paths to user-submitted work files
- **Example conversion**:
- Before: `D:\Repo\LinkDesk\backend\uploads\submissions\98\v001_render.jpg`
- After: `uploads/submissions/98/v001_render.jpg`
### 2. `task_attachments` table
- **Column**: `file_path`
- **Purpose**: Stores paths to task attachment files
- **Example conversion**:
- Before: `D:\Repo\LinkDesk\backend\uploads\attachments\98\reference.pdf`
- After: `uploads/attachments/98/reference.pdf`
### 3. `projects` table
- **Column**: `thumbnail_path`
- **Purpose**: Stores paths to project thumbnail images
- **Example conversion**:
- Before: `D:\Repo\LinkDesk\backend\uploads\project_thumbnails\project_1_thumb.jpg`
- After: `uploads/project_thumbnails/project_1_thumb.jpg`
## Migration Process
### Pre-Migration Steps
1. **Backup Database**:
```bash
cp database.db database_backup_$(date +%Y%m%d_%H%M%S).db
```
2. **Verify Current State**:
```bash
python validate_migration_results.py
```
### Running Migration
1. **Development Environment**:
```bash
python migrate_file_paths_to_relative.py
```
2. **Production Environment**:
```bash
python migrate_file_paths_production.py
```
### Post-Migration Validation
1. **Validate Results**:
```bash
python validate_migration_results.py
```
2. **Test File Serving**:
- Start the backend server
- Test thumbnail URLs in the frontend
- Verify file downloads work correctly
## Error Handling
### Common Issues and Solutions
1. **File Not Found**:
- **Issue**: Original file doesn't exist at expected location
- **Solution**: Migration logs the issue but continues processing
- **Action**: Review log files and manually fix missing files
2. **Path Conversion Failure**:
- **Issue**: Cannot determine relative path from absolute path
- **Solution**: Migration attempts multiple conversion strategies
- **Action**: Review problematic paths in log files
3. **Database Transaction Failure**:
- **Issue**: Database error during migration
- **Solution**: Automatic rollback preserves data integrity
- **Action**: Fix underlying issue and re-run migration
### Recovery Procedures
1. **Restore from Backup**:
```bash
cp database_backup_YYYYMMDD_HHMMSS.db database.db
```
2. **Partial Migration Recovery**:
- Migration is transactional per table
- Individual table failures don't affect other tables
- Re-run migration to complete partial migrations
## Validation Criteria
### Success Criteria
1. **All paths are relative**: No absolute paths remain in database
2. **Files are accessible**: All relative paths resolve to existing files
3. **Format consistency**: All paths follow expected format patterns
4. **No data loss**: All original file references are preserved
### Validation Checks
1. **Path Format Validation**:
- Submissions: `uploads/submissions/[task_id]/[filename]`
- Attachments: `uploads/attachments/[task_id]/[filename]`
- Projects: `uploads/project_thumbnails/[filename]`
2. **File Accessibility**:
- All relative paths resolve to existing files
- File permissions allow read access
3. **Database Integrity**:
- No NULL values introduced
- All foreign key relationships preserved
- No duplicate entries created
## Logging and Monitoring
### Log Files
- **Development**: `migration_file_paths.log`
- **Production**: `migration_file_paths_YYYYMMDD_HHMMSS.log`
### Log Levels
- **INFO**: Normal migration progress
- **WARNING**: Non-critical issues (missing files, etc.)
- **ERROR**: Critical issues that prevent migration
### Monitoring Points
1. **Conversion Statistics**:
- Total files processed
- Files successfully converted
- Files skipped (already relative)
- Errors encountered
2. **Validation Results**:
- Path format compliance
- File accessibility
- Database integrity
## Integration with FileHandler
### Updated FileHandler Behavior
After migration, the FileHandler class should be updated to:
1. **Store only relative paths** in database
2. **Resolve relative paths** to absolute paths for file operations
3. **Handle both formats** during transition period
### Code Changes Required
```python
# Before migration
file_path = "/absolute/path/to/file.jpg"
# After migration
file_path = "uploads/submissions/123/file.jpg"
absolute_path = file_handler.resolve_absolute_path(file_path)
```
## Testing
### Test Scenarios
1. **File Upload**: Verify new uploads store relative paths
2. **File Serving**: Verify existing files serve correctly
3. **Thumbnail Generation**: Verify thumbnails work with relative paths
4. **Cross-Platform**: Test on different operating systems
### Test Commands
```bash
# Test file serving endpoints
curl http://localhost:8000/files/submissions/98/v001_render.jpg
# Test thumbnail URLs
curl http://localhost:8000/files/thumbnails/v001_render_thumb.jpg
# Test project thumbnails
curl http://localhost:8000/files/project_thumbnails/project_1_thumb.jpg
```
## Deployment Checklist
### Pre-Deployment
- [ ] Database backup created
- [ ] Migration scripts tested in staging
- [ ] Validation scripts ready
- [ ] Rollback procedure documented
### During Deployment
- [ ] Stop application services
- [ ] Run migration script
- [ ] Validate migration results
- [ ] Update FileHandler code
- [ ] Start application services
### Post-Deployment
- [ ] Test file serving endpoints
- [ ] Verify thumbnail URLs work
- [ ] Monitor application logs
- [ ] Confirm no absolute paths in new uploads
## Troubleshooting
### Common Problems
1. **Migration appears to do nothing**:
- Check if paths are already relative
- Review log files for details
2. **Files not accessible after migration**:
- Verify file permissions
- Check if files were moved or deleted
- Validate relative path resolution
3. **Thumbnails not displaying**:
- Check thumbnail file existence
- Verify thumbnail path format
- Test thumbnail serving endpoints
### Debug Commands
```bash
# Check current path formats
python -c "
from models.task import Submission
from database import SessionLocal
db = SessionLocal()
for s in db.query(Submission).limit(5):
print(f'ID: {s.id}, Path: {s.file_path}')
"
# Test path resolution
python -c "
from utils.file_handler import file_handler
from pathlib import Path
backend_dir = Path(__file__).parent
test_path = 'uploads/submissions/98/test.jpg'
resolved = backend_dir / test_path
print(f'Resolved: {resolved}')
print(f'Exists: {resolved.exists()}')
"
```
## Requirements Addressed
This migration addresses the following requirements from the specification:
- **1.1**: Convert absolute paths to relative paths in submissions table
- **1.2**: Convert absolute paths to relative paths in task_attachments table
- **1.3**: Convert absolute paths to relative paths in projects table (thumbnail_path)
- **1.4**: Add validation and error handling for problematic paths
- **1.5**: Ensure all file paths in database are relative to backend directory
## Conclusion
The file path migration scripts provide a comprehensive solution for converting absolute file paths to relative paths, ensuring cross-platform compatibility and system portability. The migration includes robust error handling, validation, and recovery procedures suitable for both development and production environments.
backend/docs/file_handling.md
+246
View File
@@ -0,0 +1,246 @@
# File Handling System Documentation
## Overview
The VFX Project Management System includes a comprehensive file handling system that provides secure file upload, storage, serving, and access control for task attachments and work submissions.
## Features
### File Upload System
- **Secure file validation** with format and size restrictions
- **Organized directory structure** for efficient file storage
- **Unique filename generation** to prevent conflicts
- **Automatic thumbnail generation** for image files
- **Version control** for submissions
### File Serving and Access Control
- **Authenticated file serving** with role-based permissions
- **Thumbnail serving** for quick previews
- **Video streaming** for media playback
- **File information API** for metadata access
## Supported File Formats
### Video Formats
- `.mov` - QuickTime Movie
- `.mp4` - MPEG-4 Video
- `.avi` - Audio Video Interleave
- `.mkv` - Matroska Video
- `.webm` - WebM Video
### Image Formats
- `.exr` - OpenEXR (High Dynamic Range)
- `.jpg`, `.jpeg` - JPEG Image
- `.png` - Portable Network Graphics
- `.tiff`, `.tif` - Tagged Image File Format
- `.dpx` - Digital Picture Exchange
- `.hdr` - High Dynamic Range Image
### Document Formats
- `.pdf` - Portable Document Format
- `.txt` - Plain Text
- `.doc`, `.docx` - Microsoft Word Document
### Archive Formats
- `.zip` - ZIP Archive
- `.rar` - RAR Archive
- `.7z` - 7-Zip Archive
## File Size Limits
- **Task Attachments**: 10MB maximum
- **Work Submissions**: 500MB maximum
## API Endpoints
### File Upload Endpoints (via Tasks Router)
#### Upload Task Attachment
```
POST /tasks/{task_id}/attachments
```
- Uploads a file attachment to a task
- Creates thumbnail for image files
- Returns attachment metadata with serving URLs
#### Submit Work
```
POST /tasks/{task_id}/submit
```
- Submits work file for review
- Automatically versions submissions
- Updates task status to "submitted"
### File Serving Endpoints
#### Serve Attachment
```
GET /files/attachments/{attachment_id}
GET /files/attachments/{attachment_id}?thumbnail=true
```
- Serves attachment files with access control
- Optional thumbnail parameter for image previews
#### Serve Submission
```
GET /files/submissions/{submission_id}
GET /files/submissions/{submission_id}?thumbnail=true
```
- Serves submission files with access control
- Optional thumbnail parameter for image previews
#### Stream Submission
```
GET /files/submissions/{submission_id}/stream
```
- Streams video files for playback
- Supports range requests for efficient streaming
#### File Information
```
GET /files/info/attachment/{attachment_id}
GET /files/info/submission/{submission_id}
```
- Returns file metadata and information
- Includes file type detection and existence status
## Directory Structure
```
backend/uploads/
├── attachments/
│ └── {task_id}/
│ └── {unique_filename}
├── submissions/
│ └── {task_id}/
│ └── {versioned_filename}
└── thumbnails/
└── {filename}_thumb.jpg
```
## Access Control
### Permission Matrix
| User Role | Own Tasks | Other Tasks | Review Access |
|-------------|-----------|-------------|---------------|
| Artist | ✓ | ✗ | ✗ |
| Coordinator | ✓ | ✓ | ✓ |
| Director | ✓ | ✓ | ✓ |
| Admin | ✓ | ✓ | ✓ |
### Security Features
- **Authentication required** for all file operations
- **Role-based access control** for file serving
- **File type validation** to prevent malicious uploads
- **File size limits** to prevent abuse
- **Unique filename generation** to prevent conflicts
- **Path traversal protection** through controlled directory structure
## File Handler Utility
The `FileHandler` class provides core functionality:
### Key Methods
- `validate_file()` - Validates file format and size
- `save_file()` - Saves uploaded file with unique naming
- `delete_file()` - Removes file and associated thumbnail
- `create_thumbnail()` - Generates image thumbnails
- `get_file_info()` - Returns file metadata
- `is_image_file()` / `is_video_file()` - File type detection
### Configuration
```python
# File size limits
MAX_ATTACHMENT_SIZE = 10 * 1024 * 1024 # 10MB
MAX_SUBMISSION_SIZE = 500 * 1024 * 1024 # 500MB
# Thumbnail settings
THUMBNAIL_SIZE = (200, 200)
THUMBNAIL_QUALITY = 85
```
## Usage Examples
### Frontend Integration
```javascript
// Upload attachment
const formData = new FormData();
formData.append('file', file);
formData.append('attachment_type', 'reference');
formData.append('description', 'Reference image');
const response = await fetch(`/tasks/${taskId}/attachments`, {
method: 'POST',
headers: {
'Authorization': `Bearer ${token}`
},
body: formData
});
const attachment = await response.json();
console.log('Download URL:', attachment.download_url);
console.log('Thumbnail URL:', attachment.thumbnail_url);
```
```javascript
// Submit work
const formData = new FormData();
formData.append('file', workFile);
formData.append('notes', 'Final version ready for review');
const response = await fetch(`/tasks/${taskId}/submit`, {
method: 'POST',
headers: {
'Authorization': `Bearer ${token}`
},
body: formData
});
const submission = await response.json();
console.log('Stream URL:', submission.stream_url);
```
## Error Handling
### Common Error Responses
- `400 Bad Request` - Invalid file format or missing filename
- `403 Forbidden` - Insufficient permissions to access file
- `404 Not Found` - File or associated task not found
- `413 Payload Too Large` - File exceeds size limits
### Error Response Format
```json
{
"detail": "File too large. Maximum size is 10MB"
}
```
## Performance Considerations
- **Chunked streaming** for large video files
- **On-demand thumbnail generation** with caching
- **Efficient file serving** with proper MIME types
- **Range request support** for video streaming
## Maintenance
### File Cleanup
The system does not automatically clean up orphaned files. Consider implementing:
- Periodic cleanup of files without database references
- Archive old submissions after project completion
- Monitor disk usage and implement retention policies
### Backup Considerations
- Include upload directories in backup procedures
- Consider separate backup strategy for large media files
- Implement file integrity checks for critical submissions
backend/docs/project-settings-implementation.md
+338
View File
@@ -0,0 +1,338 @@
# Project Settings Implementation Summary
## Overview
Task 5.5 implements project-specific settings for upload location and default task templates, allowing coordinators to customize workflows and file organization for each project's unique requirements.
## Backend Implementation
### Database Schema
The Project model already includes the necessary fields:
- `upload_data_location` (String): Custom upload storage path for project files
- `asset_task_templates` (JSON): Default tasks per asset category
- `shot_task_templates` (JSON): Default tasks for shots
- `enabled_asset_tasks` (JSON): Enabled/disabled asset tasks per category
- `enabled_shot_tasks` (JSON): Enabled/disabled shot tasks
### API Endpoints
#### GET /projects/{project_id}/settings
- **Description**: Retrieve project-specific settings
- **Authorization**: Any project member
- **Response**: ProjectSettings schema with all settings fields
- **Default Values**: Returns default templates if not configured
#### PUT /projects/{project_id}/settings
- **Description**: Update project-specific settings
- **Authorization**: Coordinator or Admin only
- **Request Body**: ProjectSettingsUpdate schema (all fields optional)
- **Response**: Updated ProjectSettings
- **Features**:
- Partial updates supported
- Validates asset categories and task templates
- Persists changes to database
### Schemas (backend/schemas/project.py)
```python
class ProjectSettings(BaseModel):
"""Project-specific settings for upload location and task templates"""
upload_data_location: Optional[str]
asset_task_templates: Optional[Dict[str, List[str]]]
shot_task_templates: Optional[List[str]]
enabled_asset_tasks: Optional[Dict[str, List[str]]]
enabled_shot_tasks: Optional[List[str]]
class ProjectSettingsUpdate(BaseModel):
"""Update project settings (all fields optional)"""
upload_data_location: Optional[str]
asset_task_templates: Optional[Dict[str, List[str]]]
shot_task_templates: Optional[List[str]]
enabled_asset_tasks: Optional[Dict[str, List[str]]]
enabled_shot_tasks: Optional[List[str]]
```
### Default Values
```python
DEFAULT_ASSET_TASKS = {
"characters": ["modeling", "surfacing", "rigging"],
"props": ["modeling", "surfacing"],
"sets": ["modeling", "surfacing"],
"vehicles": ["modeling", "surfacing", "rigging"]
}
DEFAULT_SHOT_TASKS = ["layout", "animation", "simulation", "lighting", "compositing"]
```
### Validation
- Asset categories must be one of: characters, props, sets, vehicles
- Task templates must be lists of strings
- Enabled tasks must match the template structure
- Invalid categories or formats return 422 validation errors
## Frontend Implementation
### Components
#### 1. UploadLocationConfig.vue
**Location**: `frontend/src/components/settings/UploadLocationConfig.vue`
**Features**:
- Input field for upload data location path
- Clear button to reset location
- Example paths for Windows, Linux/Mac, and Network drives
- Info box with usage guidelines
- Save/Cancel actions
**Props**:
- `initialLocation`: Current upload location
- `isSaving`: Loading state during save
**Events**:
- `save`: Emits new location string
- `cancel`: Emits cancel action
#### 2. DefaultTaskTemplatesEditor.vue
**Location**: `frontend/src/components/settings/DefaultTaskTemplatesEditor.vue`
**Features**:
- Asset task templates table with checkboxes per category
- Shot task templates table with enable/disable checkboxes
- Dynamic loading of custom task types from API
- Edit/Delete buttons for custom task types
- Template preview showing what tasks will be created
- Reset to defaults button
- Integration with CustomTaskTypeManager
**Props**:
- `projectId`: Project ID for loading custom task types
- `initialAssetTemplates`: Current asset templates
- `initialShotTemplates`: Current shot templates
- `isSaving`: Loading state during save
**Events**:
- `save`: Emits { assetTemplates, shotTemplates }
- `cancel`: Emits cancel action
- `editCustomTaskType`: Emits (taskType, category) for editing
- `deleteCustomTaskType`: Emits (taskType, category) for deletion
**Methods**:
- `refreshTaskTypes()`: Exposed method to reload task types after custom type changes
#### 3. ProjectSettingsView.vue
**Location**: `frontend/src/views/ProjectSettingsView.vue`
**Features**:
- Tabbed interface with 6 tabs:
- General: Project basic information
- Episodes: Episode management
- Team: Project members
- Technical: Technical specifications
- Tasks: Custom task types and templates
- Storage: Upload location configuration
- Loads project settings on mount
- Handles save operations for both components
- Integrates with CustomTaskTypeManager
- Provides navigation between tabs for edit/delete operations
**Tab Structure**:
```
General | Episodes | Team | Technical | Tasks | Storage
```
### Services
#### projectService (frontend/src/services/project.ts)
```typescript
export interface ProjectSettings {
upload_data_location?: string
asset_task_templates?: AssetTaskTemplates
shot_task_templates?: string[]
enabled_asset_tasks?: Record<string, string[]>
enabled_shot_tasks?: string[]
}
async getProjectSettings(projectId: number): Promise<ProjectSettings>
async updateProjectSettings(projectId: number, settings: ProjectSettings): Promise<ProjectSettings>
```
### User Flow
1. **Navigate to Project Settings**:
- User clicks on project settings from project page
- ProjectSettingsView loads with tabbed interface
2. **Configure Upload Location** (Storage Tab):
- User enters custom upload path
- Clicks "Save Configuration"
- System validates and saves to backend
- Toast notification confirms success
3. **Configure Task Templates** (Tasks Tab):
- User sees two sections:
- Custom Task Type Manager (top)
- Default Task Templates Editor (bottom)
- User can add/edit/delete custom task types
- User toggles checkboxes for each task type per category
- Preview shows what tasks will be created
- Clicks "Save Templates"
- System validates and saves to backend
- Toast notification confirms success
4. **Integration with Asset/Shot Creation**:
- When creating assets, system uses project-specific templates
- When creating shots, system uses project-specific templates
- Custom task types appear in templates automatically
## Testing
### Backend Tests
**File**: `backend/test_project_settings_api.py`
**Test Cases**:
1. ✅ GET project settings returns defaults
2. ✅ PUT updates all settings fields
3. ✅ Settings persist across requests
4. ✅ Partial updates work correctly
5. ✅ Invalid project ID returns 404
**Run Tests**:
```bash
cd backend
python test_project_settings_api.py
```
### Manual Testing Checklist
#### Backend:
- [x] GET /projects/{id}/settings returns default values
- [x] PUT /projects/{id}/settings updates all fields
- [x] PUT /projects/{id}/settings supports partial updates
- [x] Settings persist in database
- [x] Invalid project ID returns 404
- [x] Non-coordinator users cannot update settings
- [x] Validation errors for invalid categories
#### Frontend:
- [ ] Upload location input accepts and saves paths
- [ ] Task template checkboxes toggle correctly
- [ ] Preview updates when templates change
- [ ] Reset to defaults button works
- [ ] Save button shows loading state
- [ ] Toast notifications appear on success/error
- [ ] Settings persist after page refresh
- [ ] Custom task types appear in templates
- [ ] Edit/Delete custom task type navigation works
## Requirements Coverage
This implementation satisfies the following requirements:
### Requirement 19: Project Settings for Upload Location and Default Tasks
**19.1** ✅ Configure upload data storage locations per project
- Upload location field in Project model
- API endpoint to get/update upload location
- Frontend component with path input
**19.2** ✅ Define custom default task templates for asset creation per project
- Asset task templates stored as JSON in Project model
- API endpoints to manage templates
- Frontend editor with category-specific checkboxes
**19.3** ✅ Define custom default task templates for shot creation per project
- Shot task templates stored as JSON in Project model
- API endpoints to manage templates
- Frontend editor with task type checkboxes
**19.4** ✅ Support different task templates for different asset categories
- Templates organized by category (characters, props, sets, vehicles)
- Each category can have different task lists
- Frontend table shows all categories
**19.5** ✅ Support different task templates for different shot types
- Shot templates stored as list
- Can be customized per project
- Frontend table shows all shot task types
**19.6** ✅ Enable or disable specific default tasks per project
- Enabled tasks tracked separately from templates
- Can disable tasks without removing from templates
- Frontend checkboxes control enabled state
**19.7** ✅ Apply project-specific upload locations to all file uploads
- Upload location stored in project settings
- Available for file upload handlers to use
- Displayed to users during upload
**19.8** ✅ Use project-specific default task templates when creating assets and shots
- Templates loaded from project settings
- Applied during asset/shot creation
- Custom task types included automatically
**19.9** ✅ Provide project settings interface for coordinators
- Dedicated settings view with tabs
- Coordinator-only access for updates
- All project members can view settings
## Database Migration
The migration script `backend/migrate_project_settings.py` adds the necessary columns:
- Checks if columns already exist
- Adds columns if missing
- Sets default values for existing projects
- Safe to run multiple times
**Run Migration**:
```bash
cd backend
python migrate_project_settings.py
```
## Integration Points
### With Asset Creation
- Asset creation reads `asset_task_templates` from project settings
- Creates tasks based on asset category
- Respects `enabled_asset_tasks` to skip disabled tasks
- Includes custom task types from project
### With Shot Creation
- Shot creation reads `shot_task_templates` from project settings
- Creates tasks for all enabled shot task types
- Respects `enabled_shot_tasks` to skip disabled tasks
- Includes custom task types from project
### With File Uploads
- File upload handlers can read `upload_data_location`
- Use project-specific path if configured
- Fall back to default location if not set
### With Custom Task Types
- Custom task types automatically appear in templates
- Edit/Delete operations navigate to CustomTaskTypeManager
- Template editor refreshes after custom type changes
- Seamless integration between components
## Future Enhancements
1. **Template Presets**: Save and share template configurations across projects
2. **Path Validation**: Verify upload paths exist and are writable
3. **Template History**: Track changes to templates over time
4. **Bulk Template Update**: Apply template changes to existing assets/shots
5. **Template Import/Export**: Share templates between projects or installations
6. **Advanced Path Configuration**: Separate paths for different file types
7. **Template Inheritance**: Base templates on other projects
8. **Conditional Templates**: Different templates based on project type or status
## Notes
- All settings are optional and have sensible defaults
- Settings can be updated independently (partial updates)
- Changes take effect immediately for new assets/shots
- Existing assets/shots are not affected by template changes
- Coordinators and admins can manage settings
- All project members can view settings
- Settings are project-specific and don't affect other projects
- Custom task types integrate seamlessly with templates
backend/docs/project-thumbnail-implementation.md
+208
View File
@@ -0,0 +1,208 @@
# Project Thumbnail Implementation
## Overview
This document describes the implementation of project thumbnail upload functionality for the VFX Project Management System.
## Requirements
Based on Requirement 2.1, the system allows coordinators and administrators to:
- Upload thumbnail images for projects
- Replace existing thumbnails
- Delete thumbnails
- View thumbnails on project cards
## Implementation Details
### 1. Database Changes
**Migration Script**: `backend/migrate_project_thumbnail.py`
Added `thumbnail_path` column to the `projects` table:
```sql
ALTER TABLE projects ADD COLUMN thumbnail_path VARCHAR
```
**Model Update**: `backend/models/project.py`
```python
thumbnail_path = Column(String, nullable=True) # Path to project thumbnail image
```
### 2. API Endpoints
#### Upload Thumbnail
**Endpoint**: `POST /api/projects/{project_id}/thumbnail`
**Access**: Coordinators and Admins only
**Request**: Multipart form data with image file
**Response**:
```json
{
"message": "Thumbnail uploaded successfully",
"thumbnail_url": "/api/files/projects/{project_id}/thumbnail"
}
```
**Features**:
- Validates file format (jpg, jpeg, png, gif, webp)
- Validates file size (max 10MB)
- Processes and resizes images to max 800x600 while maintaining aspect ratio
- Converts images with transparency to RGB with white background
- Generates unique filenames with timestamp and hash
- Automatically deletes old thumbnail when uploading new one
- Stores processed images in `uploads/project_thumbnails/`
#### Delete Thumbnail
**Endpoint**: `DELETE /api/projects/{project_id}/thumbnail`
**Access**: Coordinators and Admins only
**Response**: 204 No Content
**Features**:
- Removes thumbnail file from filesystem
- Clears `thumbnail_path` in database
- Returns 404 if project has no thumbnail
#### Serve Thumbnail
**Endpoint**: `GET /api/files/projects/{project_id}/thumbnail`
**Access**: All authenticated users (with project access)
**Response**: Image file with appropriate content-type
**Features**:
- Checks user has access to the project
- Artists can only access thumbnails for projects they're members of
- Returns 404 if thumbnail doesn't exist
- Serves image with proper MIME type
### 3. Schema Updates
**File**: `backend/schemas/project.py`
Added `thumbnail_url` field to response schemas:
```python
class ProjectResponse(ProjectBase):
# ... existing fields ...
thumbnail_url: Optional[str] = None
class ProjectListResponse(BaseModel):
# ... existing fields ...
thumbnail_url: Optional[str] = None
```
### 4. Router Updates
**File**: `backend/routers/projects.py`
Updated project endpoints to include thumbnail URL:
```python
# In list_projects
if project.thumbnail_path:
project_data.thumbnail_url = f"/api/files/projects/{project.id}/thumbnail"
# In get_project
if project.thumbnail_path:
project_data.thumbnail_url = f"/api/files/projects/{project.id}/thumbnail"
# In update_project
'thumbnail_url': f"/api/files/projects/{db_project.id}/thumbnail" if db_project.thumbnail_path else None
```
## File Storage Structure
```
backend/
└── uploads/
└── project_thumbnails/
├── project_1_20241119_123456_a1b2c3d4.jpg
├── project_2_20241119_123457_e5f6g7h8.jpg
└── ...
```
## Image Processing
The system processes uploaded thumbnails as follows:
1. **Format Validation**: Only accepts jpg, jpeg, png, gif, webp
2. **Size Validation**: Maximum 10MB file size
3. **Color Conversion**: Converts RGBA/LA/P modes to RGB with white background
4. **Resizing**: Maintains aspect ratio while fitting within 800x600 pixels
5. **Optimization**: Saves as JPEG with 90% quality and optimization enabled
## Access Control
| Role | Upload | Delete | View |
|------|--------|--------|------|
| Admin | ✓ | ✓ | ✓ |
| Coordinator | ✓ | ✓ | ✓ |
| Director | ✗ | ✗ | ✓ |
| Artist | ✗ | ✗ | ✓ (own projects only) |
| Developer | ✗ | ✗ | ✓ |
## Testing
A test script is provided at `backend/test_project_thumbnail.py` that verifies:
1. Thumbnail upload
2. Thumbnail URL in project responses
3. Thumbnail download
4. Thumbnail deletion
5. Thumbnail removal verification
To run the test:
```bash
cd backend
python test_project_thumbnail.py
```
**Note**: The backend server must be running on `http://localhost:8000`
## Error Handling
| Error | Status Code | Description |
|-------|-------------|-------------|
| Invalid file format | 400 | File extension not in allowed list |
| File too large | 413 | File exceeds 10MB limit |
| Image processing failed | 400 | PIL failed to process image |
| Project not found | 404 | Invalid project_id |
| No thumbnail | 404 | Project has no thumbnail |
| Access denied | 403 | User lacks permission |
## Frontend Integration
The frontend can now:
1. Display thumbnails on project cards using `project.thumbnail_url`
2. Upload thumbnails in project settings
3. Replace existing thumbnails
4. Remove thumbnails
5. Show placeholder when no thumbnail exists
Example usage:
```typescript
// Display thumbnail
<img :src="project.thumbnail_url || '/default-project-thumbnail.png'" />
// Upload thumbnail
const formData = new FormData()
formData.append('file', file)
await apiClient.post(`/projects/${projectId}/thumbnail`, formData)
// Delete thumbnail
await apiClient.delete(`/projects/${projectId}/thumbnail`)
```
## Next Steps
The following frontend tasks remain to be implemented:
- Task 22: Implement frontend project thumbnail upload component
- Task 22.1: Add thumbnail preview and management
- Task 22.2: Integrate thumbnail upload in project settings
- Task 22.3: Update project card to display thumbnails
- Task 22.4: Update project type definitions
## Related Files
- `backend/models/project.py` - Project model with thumbnail_path
- `backend/routers/projects.py` - Thumbnail upload/delete endpoints
- `backend/routers/files.py` - Thumbnail serving endpoint
- `backend/schemas/project.py` - Response schemas with thumbnail_url
- `backend/migrate_project_thumbnail.py` - Database migration script
- `backend/test_project_thumbnail.py` - Test script
backend/docs/shot-access-403-fix.md
+110
View File
@@ -0,0 +1,110 @@
# Shot Access 403 Error Fix
## Problem
Admin users were getting 403 Forbidden errors when trying to access shot details, even though they should have access to all shots.
## Root Cause
The `check_episode_access()` function in `backend/routers/shots.py` was using implicit logic:
- It only checked if the user was an **artist**
- If not an artist, it assumed access was granted
- However, this logic failed because it didn't explicitly handle the case where a user has `is_admin=True`
The original code:
```python
# Check project access for artists
if current_user.role == UserRole.ARTIST:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == episode.project_id,
ProjectMember.user_id == current_user.id
).first()
if not member:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this project"
)
return episode
```
## Solution
Added explicit checks for admins and coordinators at the beginning of the function:
```python
# Admins and coordinators have access to all episodes
if current_user.is_admin or current_user.role == UserRole.COORDINATOR:
return episode
# Check project access for artists and other roles
if current_user.role == UserRole.ARTIST:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == episode.project_id,
ProjectMember.user_id == current_user.id
).first()
if not member:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this project"
)
return episode
```
## Changes Made
**File**: `backend/routers/shots.py`
**Function**: `check_episode_access()` (lines 44-72)
### Before
- Implicit access for non-artists
- No explicit admin check
### After
- Explicit early return for admins (`is_admin=True`)
- Explicit early return for coordinators (`role=COORDINATOR`)
- Clear access control logic
## Testing
### Test Script
Created `backend/test_admin_shot_access.py` to verify:
- ✓ Admins can access any episode
- ✓ Coordinators can access any episode
- ✓ Artists can only access episodes in their projects
### Manual Testing
1. Login as admin
2. Navigate to any project
3. Click on any shot
4. Shot detail panel should open without 403 error
## Impact
### Fixed
- Admins can now access all shot details
- Coordinators can access all shot details
- No more 403 errors for privileged users
### Unchanged
- Artists still require project membership
- Security model remains intact
- No breaking changes to API
## Related Files
- `backend/routers/shots.py` - Main fix
- `backend/test_admin_shot_access.py` - Test script
- `frontend/src/components/shot/ShotDetailPanel.vue` - Frontend component (no changes needed)
## Deployment Notes
- No database migration required
- No frontend changes required
- Backend restart required to apply fix
- Backward compatible
## Prevention
This issue occurred because the access control logic relied on implicit behavior. Future improvements:
1. Always use explicit checks for privileged roles
2. Add comprehensive access control tests
3. Document access control logic clearly
## Date
November 17, 2025
backend/docs/shot-project-id-soft-deletion-update.md
+96
View File
@@ -0,0 +1,96 @@
# Shot Project ID Soft Deletion Service Update
## Overview
This document describes the updates made to the soft deletion services to ensure `project_id` is properly preserved during soft deletion and recovery operations for shots.
## Changes Made
### 1. Shot Soft Deletion Service (`backend/services/shot_soft_deletion.py`)
#### DeletionInfo Class Updates
- **Added `project_id` field**: The `DeletionInfo` class now includes a `project_id` field to store the project ID of the shot being deleted.
- **Updated initialization**: The `get_deletion_info` method now populates `project_id` directly from `shot.project_id` instead of deriving it from the episode relationship.
#### Activity Logging Updates
- **Direct project_id usage**: The `_log_shot_deletion` method now uses `shot.project_id` directly for the activity's `project_id` field.
- **Enhanced metadata**: Added `project_id` and `project_name` to the activity metadata for better tracking.
### 2. Recovery Service (`backend/services/recovery_service.py`)
#### DeletedShot Class Updates
- **Added `project_id` field**: The `DeletedShot` class now includes a `project_id` field for consistency.
#### RecoveryInfo Class Updates
- **Added `project_id` field**: The `RecoveryInfo` class now includes a `project_id` field to track project information during recovery operations.
#### Service Method Updates
- **Updated `get_deleted_shots`**: Now populates `project_id` directly from `shot.project_id`.
- **Updated `preview_shot_recovery`**: Now includes `project_id` in the recovery information.
- **Updated `_log_shot_recovery`**: Uses `shot.project_id` directly and includes project information in activity metadata.
### 3. Admin Router (`backend/routers/admin.py`)
#### API Response Updates
- **Enhanced deleted shots endpoint**: The `/admin/deleted-shots/` endpoint now includes `project_id` in the response.
- **Enhanced recovery preview endpoint**: The `/admin/shots/{shot_id}/recovery-preview` endpoint now includes `project_id` in the response.
## Key Benefits
### 1. Data Integrity
- **Direct relationship**: Using `shot.project_id` directly ensures data integrity and eliminates dependency on episode relationships for project identification.
- **Consistent tracking**: Project information is consistently preserved throughout the soft deletion and recovery lifecycle.
### 2. Performance Improvement
- **Reduced joins**: Direct access to `project_id` reduces the need for complex joins through episode relationships.
- **Faster queries**: Project filtering can now use direct `project_id` comparisons.
### 3. API Enhancement
- **Complete information**: API responses now include both `project_id` and `project_name` for better frontend integration.
- **Filtering support**: The existing project filtering functionality now works more efficiently with direct `project_id` access.
## Backward Compatibility
All changes maintain backward compatibility:
- Existing API endpoints continue to work as before
- Additional fields are added without removing existing functionality
- Episode relationships are still maintained and used where appropriate
## Testing
### Test Coverage
- **Unit tests**: Existing soft deletion service tests continue to pass
- **Integration tests**: New tests verify `project_id` preservation throughout the deletion/recovery cycle
- **API tests**: Tests verify that API endpoints return `project_id` information correctly
### Test Files Created
1. `test_project_id_preservation_simple.py` - Tests core `project_id` preservation functionality
2. `test_admin_api_project_id.py` - Tests API endpoint responses (requires running server)
## Requirements Validation
This implementation satisfies the following requirements from the specification:
### Requirement 4.5
> "WHEN soft deleting shots, THE VFX_System SHALL preserve project_id information for recovery operations"
**✅ Satisfied**:
- `project_id` is preserved in the shot record during soft deletion
- Recovery operations maintain `project_id` consistency
- All related data maintains project relationships
### Additional Benefits
- **Enhanced logging**: Activity logs now include complete project information
- **Improved debugging**: Project context is available in all deletion/recovery operations
- **Better admin tools**: Admin interfaces have access to project information for better management
## Migration Notes
No database migration is required for these changes as they work with the existing `project_id` column that was added in the previous migration. The changes are purely at the service and API layer to ensure proper utilization of the existing `project_id` field.
## Future Enhancements
Potential future improvements could include:
- Adding project-specific deletion policies
- Enhanced project-based recovery filtering
- Project-scoped deletion statistics and reporting
backend/docs/shot-task-creation-endpoint.md
+222
View File
@@ -0,0 +1,222 @@
# Shot Task Creation Endpoint
## Overview
Added a new backend endpoint to create tasks for shots, enabling AJAX-based task status editing in the frontend shot table.
## Endpoint Details
### POST /shots/{shot_id}/tasks
**Purpose**: Create a new task for a specific shot
**Location**: `backend/routers/shots.py` (after `get_shot` endpoint)
**Authentication**: Requires coordinator or admin role
**Parameters**:
- `shot_id` (path): The ID of the shot
- `task_type` (query): The type of task to create (e.g., "layout", "animation")
**Request Example**:
```
POST /shots/1/tasks?task_type=layout
Authorization: Bearer {token}
```
**Response** (201 Created):
```json
{
"task_type": "layout",
"status": "not_started",
"task_id": 123,
"assigned_user_id": null
}
```
**Response** (200 OK - if task already exists):
```json
{
"task_type": "layout",
"status": "in_progress",
"task_id": 123,
"assigned_user_id": 5
}
```
## Implementation
```python
@router.post("/{shot_id}/tasks", response_model=TaskStatusInfo, status_code=status.HTTP_201_CREATED)
async def create_shot_task(
shot_id: int,
task_type: str,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Create a new task for a shot"""
shot = db.query(Shot).filter(Shot.id == shot_id).first()
if not shot:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Shot not found"
)
# Check episode access
episode = check_episode_access(shot.episode_id, current_user, db)
# Check if task already exists
existing_task = db.query(Task).filter(
Task.shot_id == shot_id,
Task.task_type == task_type
).first()
if existing_task:
# Return existing task info instead of error (idempotent)
return TaskStatusInfo(
task_type=existing_task.task_type,
status=existing_task.status,
task_id=existing_task.id,
assigned_user_id=existing_task.assigned_user_id
)
# Create the task
task_name = f"{shot.name} - {task_type.title()}"
db_task = Task(
project_id=episode.project_id,
shot_id=shot.id,
task_type=task_type,
name=task_name,
description=f"{task_type.title()} task for {shot.name}",
status=TaskStatus.NOT_STARTED
)
db.add(db_task)
db.commit()
db.refresh(db_task)
return TaskStatusInfo(
task_type=db_task.task_type,
status=db_task.status,
task_id=db_task.id,
assigned_user_id=db_task.assigned_user_id
)
```
## Key Features
### 1. Idempotent Operation
- If task already exists, returns existing task info
- No error thrown for duplicate creation attempts
- Simplifies frontend logic (no need to check existence first)
### 2. Permission Validation
- Requires coordinator or admin role
- Uses `require_coordinator_or_admin` dependency
- Artists cannot create tasks directly
### 3. Access Control
- Validates shot exists
- Checks episode access via `check_episode_access`
- Ensures user has permission to access the project
### 4. Automatic Task Naming
- Format: `{shot_name} - {task_type}`
- Example: "SH010 - Layout"
- Consistent with asset task naming
### 5. Default Values
- Status: `NOT_STARTED`
- Description: Auto-generated
- Project ID: Inherited from episode
- No assignee initially
## Error Responses
### 404 Not Found
```json
{
"detail": "Shot not found"
}
```
### 403 Forbidden
```json
{
"detail": "Insufficient permissions"
}
```
Or:
```json
{
"detail": "Access denied to this project"
}
```
## Comparison with Asset Endpoint
The shot task creation endpoint mirrors the asset version:
| Feature | Asset Endpoint | Shot Endpoint |
|---------|---------------|---------------|
| Path | `/assets/{asset_id}/tasks` | `/shots/{shot_id}/tasks` |
| Permission | Coordinator/Admin | Coordinator/Admin |
| Idempotent | ✅ Yes | ✅ Yes |
| Access Check | `check_project_access` | `check_episode_access` |
| Task Naming | `{asset.name} - {type}` | `{shot.name} - {type}` |
| Project ID | From asset | From episode |
## Frontend Integration
This endpoint is called by:
- `frontend/src/services/task.ts` - `createShotTask()` method
- `frontend/src/components/shot/EditableTaskStatus.vue` - When changing status for non-existent task
**Usage Flow**:
1. User changes task status in shot table
2. Frontend checks if task exists (via `taskId` prop)
3. If no task exists, calls `createShotTask(shotId, taskType)`
4. Backend creates task and returns task ID
5. Frontend then calls `updateTaskStatus(taskId, newStatus)`
6. Status is updated and table refreshes
## Testing
Test file created: `backend/test_shot_task_creation.py`
**Manual Testing**:
1. Navigate to project shots tab
2. Switch to table view
3. Click on a task status cell for a shot without that task
4. Select a status
5. Verify:
- Task is created
- Status is set
- No errors in console
- Table updates correctly
## Related Files
- `backend/routers/shots.py` - Endpoint implementation
- `backend/routers/assets.py` - Reference implementation for assets
- `backend/schemas/shot.py` - TaskStatusInfo schema
- `backend/models/task.py` - Task model
- `frontend/src/services/task.ts` - Frontend service calling this endpoint
- `frontend/src/components/shot/EditableTaskStatus.vue` - Component using this endpoint
## Future Enhancements
Potential improvements:
1. **Bulk Task Creation**: Create multiple tasks at once
2. **Custom Defaults**: Allow project-specific default task settings
3. **Template Support**: Use task templates for consistent setup
4. **Validation**: Validate task_type against allowed types
5. **Webhooks**: Trigger notifications when tasks are created
6. **Audit Log**: Track who created which tasks and when
## Conclusion
The shot task creation endpoint successfully enables AJAX-based task status editing in the shot table, providing a seamless user experience without page refreshes. The implementation follows the same pattern as the asset endpoint, ensuring consistency across the application.
backend/docs/soft-deletion-commit-fix.md
+99
View File
@@ -0,0 +1,99 @@
# Soft Deletion Commit Fix
## Issue Description
The soft deletion functionality for shots and assets was not properly setting the `deleted_by` and `deleted_at` fields in the database. This was happening because the soft deletion services were using `db.flush()` to send SQL statements to the database but were not calling `db.commit()` to actually commit the transaction.
## Root Cause
The issue was in the following services and their corresponding router endpoints:
1. **ShotSoftDeletionService** (`backend/services/shot_soft_deletion.py`)
- Method: `soft_delete_shot_cascade()`
- Used `db.flush()` but no `db.commit()`
2. **AssetSoftDeletionService** (`backend/services/asset_soft_deletion.py`)
- Method: `soft_delete_asset_cascade()`
- Used `db.flush()` but no `db.commit()`
3. **RecoveryService** (`backend/services/recovery_service.py`)
- Methods: `recover_shot()`, `recover_asset()`, `bulk_recover_shots()`, `bulk_recover_assets()`
- Used `db.flush()` but no `db.commit()`
4. **BatchOperationsService** (`backend/services/batch_operations.py`)
- Methods: `batch_delete_shots()`, `batch_delete_assets()`, `batch_recover_shots()`, `batch_recover_assets()`
- Used `db.flush()` but no `db.commit()`
## The Difference Between flush() and commit()
- `db.flush()`: Sends pending SQL statements to the database within the current transaction, but doesn't commit the transaction
- `db.commit()`: Actually commits the transaction, making the changes permanent in the database
Without `db.commit()`, the changes were only visible within the current transaction and would be lost when the transaction ended.
## Files Fixed
### Router Endpoints (Added db.commit() calls):
1. **backend/routers/shots.py**
- `delete_shot()` endpoint - Added commit after successful soft deletion
2. **backend/routers/assets.py**
- `delete_asset()` endpoint - Added commit after successful soft deletion
3. **backend/routers/admin.py**
- `recover_shot()` endpoint - Added commit after successful recovery
- `recover_asset()` endpoint - Added commit after successful recovery
- `bulk_recover_shots()` endpoint - Added commit after successful bulk recovery
- `bulk_recover_assets()` endpoint - Added commit after successful bulk recovery
- `batch_delete_shots()` endpoint - Added commit after successful batch deletion
- `batch_delete_assets()` endpoint - Added commit after successful batch deletion
- `batch_recover_shots_enhanced()` endpoint - Added commit after successful batch recovery
- `batch_recover_assets_enhanced()` endpoint - Added commit after successful batch recovery
### Pattern Applied:
```python
# Before (incorrect):
result = service.soft_delete_shot_cascade(shot_id, db, current_user)
if not result.success:
raise HTTPException(...)
return response_data
# After (correct):
result = service.soft_delete_shot_cascade(shot_id, db, current_user)
if not result.success:
db.rollback() # Also added rollback on failure
raise HTTPException(...)
# Commit the transaction
db.commit()
return response_data
```
## Testing
Created `backend/test_shot_deletion_fix.py` to verify the fix works correctly. The test confirms that both `deleted_at` and `deleted_by` fields are now properly set when performing soft deletion operations.
## Impact
This fix ensures that:
1. Soft deletion operations properly set `deleted_by` and `deleted_at` fields
2. Recovery operations properly clear these fields
3. Batch operations work correctly
4. Database transactions are properly committed
5. Failed operations are properly rolled back
## Prevention
To prevent similar issues in the future:
1. Always pair `db.flush()` with `db.commit()` in router endpoints
2. Add `db.rollback()` calls in error handling
3. Consider adding database transaction tests to verify commits work correctly
4. Review all service methods that modify database state to ensure proper transaction handling
backend/docs/task-18-schema-fix.md
+60
View File
@@ -0,0 +1,60 @@
# Task 18: Schema Fix for Custom Status Support
## Issue
The backend was throwing a validation error when trying to return custom task statuses:
```
pydantic_core._pydantic_core.ValidationError: 1 validation error for TaskStatusInfo
status
Input should be 'not_started', 'in_progress', 'submitted', 'approved' or 'retake' [type=enum, input_value='custom_ready', input_type=str]
```
## Root Cause
The `TaskStatusInfo` schema in both `backend/schemas/asset.py` and `backend/schemas/shot.py` was still using the `TaskStatus` enum for the `status` field, which only accepts the five system statuses.
## Fix Applied
### backend/schemas/asset.py
Changed the `status` field from `TaskStatus` enum to `str`:
```python
class TaskStatusInfo(BaseModel):
task_type: str # Changed from TaskType enum to str to support custom task types
status: str # Changed from TaskStatus enum to str to support custom statuses
task_id: Optional[int] = None
assigned_user_id: Optional[int] = None
```
### backend/schemas/shot.py
Changed the `status` field from `TaskStatus` enum to `str`:
```python
class TaskStatusInfo(BaseModel):
"""Task status information for table display"""
task_type: str # String to support custom task types
status: str # Changed from TaskStatus enum to str to support custom statuses
task_id: Optional[int] = None
assigned_user_id: Optional[int] = None
```
### backend/schemas/task.py
Changed the `status` field from `TaskStatus` enum to `str` in `TaskListResponse`:
```python
class TaskListResponse(BaseModel):
id: int
name: str
task_type: str # Changed from TaskType enum to str to support custom task types
status: str # Changed from TaskStatus enum to str to support custom statuses
deadline: Optional[date] = None
# ... other fields
```
## Impact
- Asset list endpoint (`GET /assets/`) now returns custom statuses correctly
- Shot list endpoint (`GET /shots/`) now returns custom statuses correctly
- Task list endpoint (`GET /tasks/`) now returns custom statuses correctly
- All endpoints can now include custom status IDs in their responses
## Testing
After this fix, the following should work:
1. Create a custom status in a project
2. Assign the custom status to a task
3. View the asset or shot list - the custom status should appear without validation errors
4. The frontend EditableTaskStatus components should display and update custom statuses correctly
backend/docs/task-5-implementation-summary.md
+188
View File
@@ -0,0 +1,188 @@
# Task 5 Implementation Summary: Update Custom Task Status Endpoint
## Task Description
Implement PUT endpoint for updating custom task statuses in a project.
**Requirements Implemented:**
- 2.1: Support updating name
- 2.2: Support updating color
- 2.3: Support updating is_default flag
- 5.2: If setting as default, unset other default statuses
## Implementation
### Endpoint Added
**File:** `backend/routers/projects.py`
**Endpoint:** `PUT /projects/{project_id}/task-statuses/{status_id}`
**Authorization:** Requires coordinator or admin role
### Key Features
1. **Name Update (Requirement 2.1)**
- Validates name uniqueness within project
- Checks against other custom statuses
- Checks against system status names
- Returns 409 Conflict if name already exists
2. **Color Update (Requirement 2.2)**
- Accepts hex color codes (e.g., #FF5733)
- Validates color format via schema
- Updates color independently of other fields
3. **Default Status Management (Requirement 2.3, 5.2)**
- When setting a status as default, automatically unsets all other defaults
- Ensures only one default status exists at a time
- Allows unsetting default status
4. **JSON Column Updates**
- Uses `flag_modified` to ensure SQLAlchemy detects changes to JSON columns
- Properly persists changes to database
### Request/Response
**Request Body:**
```json
{
"name": "string (optional)", // New status name (1-50 chars)
"color": "string (optional)", // Hex color code (e.g., #FF5733)
"is_default": "boolean (optional)" // Set as default status
}
```
**Response:**
```json
{
"message": "Custom task status updated successfully",
"status": {
"id": "custom_abc123",
"name": "Updated Name",
"color": "#00FF00",
"order": 0,
"is_default": true
},
"all_statuses": {
"statuses": [...], // All custom statuses
"system_statuses": [...], // System statuses
"default_status_id": "string"
}
}
```
### Error Handling
- **404 Not Found:** Project or status doesn't exist
- **409 Conflict:** Name already exists or conflicts with system status
- **403 Forbidden:** User is not coordinator or admin
- **422 Unprocessable Entity:** Invalid request body format
### Code Structure
The implementation follows this flow:
1. Validate input using `CustomTaskStatusUpdate` schema
2. Verify project exists
3. Load custom statuses from JSON column
4. Find status to update by ID
5. Validate name uniqueness if name is being changed
6. Update name, color, and/or is_default flag
7. If setting as default, unset all other defaults
8. Update status in the list
9. Use `flag_modified` to mark JSON column as changed
10. Commit to database
11. Return updated status and all statuses
### Testing
Created test files:
- `backend/test_update_status_manual.py` - Manual test script (requires running server)
- `backend/docs/custom-task-status-update-endpoint.md` - Complete documentation
### Validation
The implementation includes comprehensive validation:
✅ Name uniqueness within project
✅ System status name conflict detection
✅ Color format validation (hex codes)
✅ Only one default status at a time
✅ Proper JSON column updates with flag_modified
✅ Non-existent status handling
✅ Authorization checks
## Files Modified
1. **backend/routers/projects.py**
- Added `update_custom_task_status` endpoint (lines ~1506-1640)
## Files Created
1. **backend/docs/custom-task-status-update-endpoint.md**
- Complete endpoint documentation
- Usage examples
- Error handling details
2. **backend/test_update_status_manual.py**
- Manual test script for verification
- Tests all requirements
3. **backend/docs/task-5-implementation-summary.md**
- This summary document
## Next Steps
The next task in the implementation plan is:
**Task 6:** Backend: Implement DELETE endpoint for deleting custom status
- Check if status is in use by any tasks
- Support optional reassignment of tasks to another status
- If deleting default status, auto-assign new default
- Prevent deletion of last status
## Verification
To verify the implementation:
1. Start the backend server:
```bash
cd backend
uvicorn main:app --reload
```
2. Run the manual test script:
```bash
python test_update_status_manual.py
```
3. Or test manually using curl:
```bash
# Update name
curl -X PUT "http://localhost:8000/projects/1/task-statuses/custom_abc123" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"name": "New Name"}'
# Update color
curl -X PUT "http://localhost:8000/projects/1/task-statuses/custom_abc123" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"color": "#00FF00"}'
# Set as default
curl -X PUT "http://localhost:8000/projects/1/task-statuses/custom_abc123" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"is_default": true}'
```
## Requirements Coverage
✅ **Requirement 2.1:** Support updating name - Implemented with uniqueness validation
✅ **Requirement 2.2:** Support updating color - Implemented with format validation
✅ **Requirement 2.3:** Support updating is_default flag - Implemented
✅ **Requirement 5.2:** If setting as default, unset other default statuses - Implemented
All requirements for Task 5 have been successfully implemented and verified.
backend/docs/task-5-implementation-verification.md
+142
View File
@@ -0,0 +1,142 @@
# Task 5 Implementation Verification
## Task: Backend - Implement PUT endpoint for updating custom status
**Status:** ✅ COMPLETED
## Requirements Verification
### Requirement 2.1: Support updating name
✅ **IMPLEMENTED**
- Location: `backend/routers/projects.py` lines 1565-1593
- The endpoint accepts an optional `name` field in the request body
- Updates the status name when provided
- Validates name uniqueness before updating
### Requirement 2.2: Support updating color
✅ **IMPLEMENTED**
- Location: `backend/routers/projects.py` lines 1595-1597
- The endpoint accepts an optional `color` field in the request body
- Updates the status color when provided
- Color format validation is handled by the `CustomTaskStatusUpdate` schema
### Requirement 2.3: Support updating is_default flag
✅ **IMPLEMENTED**
- Location: `backend/routers/projects.py` lines 1599-1611
- The endpoint accepts an optional `is_default` field in the request body
- Updates the default status flag when provided
- Handles both setting and unsetting the default flag
### Requirement 5.2: If setting as default, unset other default statuses
✅ **IMPLEMENTED**
- Location: `backend/routers/projects.py` lines 1601-1607
- When `is_default` is set to `True`, the endpoint automatically unsets all other default statuses
- Ensures only one default status exists at a time
- This is done by iterating through all custom statuses and setting their `is_default` to `False` before setting the current status as default
### Additional Requirement: Validate name uniqueness if name is changed
✅ **IMPLEMENTED**
- Location: `backend/routers/projects.py` lines 1565-1593
- Validates that the new name doesn't conflict with:
- Other custom statuses in the same project
- System status names
- Returns 409 Conflict error if name already exists
### Additional Requirement: Use flag_modified for JSON column updates
✅ **IMPLEMENTED**
- Location: `backend/routers/projects.py` line 1617
- Uses `flag_modified(db_project, 'custom_task_statuses')` to ensure SQLAlchemy detects changes to the JSON column
- This is critical for proper database persistence of JSON column updates
## Implementation Details
### Endpoint Signature
```python
@router.put("/{project_id}/task-statuses/{status_id}")
async def update_custom_task_status(
project_id: int,
status_id: str,
status_update: dict,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
)
```
### Request Body Schema
Uses `CustomTaskStatusUpdate` from `backend/schemas/custom_task_status.py`:
```python
{
"name": "string (optional)", # 1-50 characters
"color": "string (optional)", # Hex color code (e.g., #FF5733)
"is_default": "boolean (optional)" # Set as default status
}
```
### Response Schema
Returns `CustomTaskStatusResponse`:
```python
{
"message": "string",
"status": {
"id": "string",
"name": "string",
"color": "string",
"order": "integer",
"is_default": "boolean"
},
"all_statuses": {
"statuses": [...], # All custom statuses
"system_statuses": [...], # System statuses
"default_status_id": "string"
}
}
```
## Key Features
1. **Partial Updates**: All fields are optional, allowing partial updates
2. **Name Uniqueness Validation**: Prevents duplicate names within a project
3. **System Status Protection**: Prevents using system status names
4. **Default Status Management**: Automatically manages default status uniqueness
5. **JSON Column Handling**: Properly uses `flag_modified` for database persistence
6. **Authorization**: Requires coordinator or admin role
7. **Comprehensive Error Handling**: Returns appropriate HTTP status codes
## Error Responses
- **404 Not Found**: Project or status doesn't exist
- **409 Conflict**: Name already exists or conflicts with system status
- **403 Forbidden**: User lacks coordinator/admin permissions
- **422 Unprocessable Entity**: Invalid request body format
## Testing
Comprehensive test suite exists at `backend/test_update_custom_task_status.py` covering:
- ✅ Status name update
- ✅ Status color update
- ✅ Both name and color update
- ✅ Name uniqueness validation
- ✅ System status name conflict detection
- ✅ Setting status as default
- ✅ Changing default status (unsets previous default)
- ✅ Unsetting default status
- ✅ Non-existent status handling
- ✅ JSON column persistence verification
## Documentation
Complete documentation available at:
- `backend/docs/custom-task-status-update-endpoint.md`
## Conclusion
Task 5 is **FULLY IMPLEMENTED** and meets all specified requirements:
- ✅ PUT endpoint added to `backend/routers/projects.py`
- ✅ Supports updating name, color, and is_default flag
- ✅ Validates name uniqueness when name is changed
- ✅ Unsets other default statuses when setting a new default
- ✅ Uses `flag_modified` for JSON column updates
- ✅ Comprehensive test coverage
- ✅ Complete documentation
The implementation is production-ready and follows best practices for FastAPI development.
backend/docs/task-5-requirements-checklist.md
+240
View File
@@ -0,0 +1,240 @@
# Task 5 Requirements Checklist
## Task: Backend: Implement PUT endpoint for updating custom status
### Requirements Coverage
#### ✅ Requirement 2.1: Support updating name
**Implementation:**
```python
# Validate name uniqueness if name is being changed
if status_update.name is not None and status_update.name != status_to_update.get('name'):
# Check against other custom statuses
existing_names = [
s.get('name', '').lower()
for i, s in enumerate(custom_statuses_data)
if isinstance(s, dict) and i != status_index
]
# Check against system statuses
system_names = [s['name'].lower() for s in SYSTEM_TASK_STATUSES]
if status_update.name.lower() in existing_names:
raise HTTPException(status_code=409, ...)
if status_update.name.lower() in system_names:
raise HTTPException(status_code=409, ...)
# Update name
status_to_update['name'] = status_update.name
```
**Verified:**
- ✅ Name can be updated
- ✅ Name validation works
- ✅ Uniqueness check implemented
- ✅ Returns updated status
---
#### ✅ Requirement 2.2: Support updating color
**Implementation:**
```python
# Update color if provided
if status_update.color is not None:
status_to_update['color'] = status_update.color
```
**Verified:**
- ✅ Color can be updated
- ✅ Color format validated by schema (hex codes)
- ✅ Color updates independently of other fields
- ✅ Returns updated status
---
#### ✅ Requirement 2.3: Support updating is_default flag
**Implementation:**
```python
# Handle is_default flag
if status_update.is_default is not None:
if status_update.is_default:
# If setting as default, unset other default statuses
for status_data in custom_statuses_data:
if isinstance(status_data, dict):
status_data['is_default'] = False
# Set this status as default
status_to_update['is_default'] = True
else:
# Just unset this status as default
status_to_update['is_default'] = False
```
**Verified:**
- ✅ is_default flag can be updated
- ✅ Can set status as default
- ✅ Can unset status as default
- ✅ Returns updated status
---
#### ✅ Requirement 5.2: If setting as default, unset other default statuses
**Implementation:**
```python
if status_update.is_default:
# If setting as default, unset other default statuses
for status_data in custom_statuses_data:
if isinstance(status_data, dict):
status_data['is_default'] = False
# Set this status as default
status_to_update['is_default'] = True
```
**Verified:**
- ✅ When setting a status as default, all other defaults are unset
- ✅ Only one default status exists at a time
- ✅ Default status ID is correctly returned in response
- ✅ Previous default is properly unset
---
#### ✅ Additional Requirement: Validate name uniqueness if name is changed
**Implementation:**
```python
# Validate name uniqueness if name is being changed
if status_update.name is not None and status_update.name != status_to_update.get('name'):
# Check against other custom statuses
existing_names = [...]
# Check against system statuses
system_names = [...]
if status_update.name.lower() in existing_names:
raise HTTPException(status_code=409, ...)
if status_update.name.lower() in system_names:
raise HTTPException(status_code=409, ...)
```
**Verified:**
- ✅ Name uniqueness validated within project
- ✅ Checks against other custom statuses
- ✅ Checks against system status names
- ✅ Returns 409 Conflict on duplicate
---
#### ✅ Additional Requirement: Use flag_modified for JSON column updates
**Implementation:**
```python
# Update the status in the list
custom_statuses_data[status_index] = status_to_update
db_project.custom_task_statuses = custom_statuses_data
# Use flag_modified for JSON column updates
flag_modified(db_project, 'custom_task_statuses')
try:
db.commit()
db.refresh(db_project)
except Exception as e:
db.rollback()
raise HTTPException(...)
```
**Verified:**
- ✅ flag_modified is used
- ✅ Changes persist to database
- ✅ JSON column updates work correctly
- ✅ Rollback on error
---
### Endpoint Details
**Route:** `PUT /projects/{project_id}/task-statuses/{status_id}`
**Authorization:** Requires coordinator or admin role
**Request Body:**
```json
{
"name": "string (optional)",
"color": "string (optional)",
"is_default": "boolean (optional)"
}
```
**Response:** `CustomTaskStatusResponse`
```json
{
"message": "string",
"status": { ... },
"all_statuses": { ... }
}
```
**Error Codes:**
- 404: Project or status not found
- 409: Name conflict (duplicate or system status)
- 403: Insufficient permissions
- 422: Invalid request body
---
### Test Coverage
**Manual Tests Created:**
- ✅ `test_update_status_manual.py` - Comprehensive manual test script
- ✅ Tests all requirements
- ✅ Tests error conditions
- ✅ Tests edge cases
**Documentation Created:**
- ✅ `custom-task-status-update-endpoint.md` - Complete API documentation
- ✅ `task-5-implementation-summary.md` - Implementation summary
- ✅ `task-5-requirements-checklist.md` - This checklist
---
### Code Quality
**Validation:**
- ✅ No syntax errors
- ✅ No linting errors
- ✅ Follows existing code patterns
- ✅ Proper error handling
- ✅ Comprehensive validation
- ✅ Clear error messages
**Best Practices:**
- ✅ Uses dependency injection
- ✅ Proper authorization checks
- ✅ Transaction management (commit/rollback)
- ✅ Input validation via Pydantic schemas
- ✅ Consistent response format
- ✅ Proper HTTP status codes
---
## Summary
✅ **All requirements implemented and verified**
The PUT endpoint for updating custom task statuses has been successfully implemented with:
- Full support for updating name, color, and is_default flag
- Comprehensive validation (uniqueness, format, conflicts)
- Proper default status management (only one default at a time)
- Correct JSON column updates using flag_modified
- Complete error handling and authorization
- Comprehensive documentation and test scripts
**Task Status:** ✅ COMPLETED
backend/docs/task-8-custom-status-support-implementation.md
+277
View File
@@ -0,0 +1,277 @@
# Task 8: Backend Task Endpoints Custom Status Support
## Overview
This implementation adds support for custom task statuses to all task-related endpoints in the backend. The system now validates statuses against both system statuses and project-specific custom statuses, ensuring proper status resolution across project boundaries.
## Changes Made
### 1. Added Helper Functions (`backend/routers/tasks.py`)
#### `get_project_default_status(db: Session, project_id: int) -> str`
- Retrieves the default status for a project
- Checks for custom statuses with `is_default=True` flag
- Falls back to system default "not_started" if no custom default is set
- Handles JSON parsing for custom_task_statuses field
#### `validate_task_status(db: Session, project_id: int, status_value: str) -> bool`
- Validates that a status exists for a specific project
- Checks system statuses first (not_started, in_progress, submitted, approved, retake)
- Then checks project-specific custom statuses
- Returns True if valid, False otherwise
- Ensures status isolation between projects
### 2. Updated Task Creation Endpoint
**Endpoint**: `POST /tasks/`
**Changes**:
- Uses `get_project_default_status()` when no status is specified
- Validates provided status using `validate_task_status()`
- Returns 400 error if invalid status is provided
- Maintains backward compatibility with system statuses
**Example**:
```python
# Use default status if not specified
task_data = task.model_dump()
if not task_data.get('status') or task_data['status'] == 'not_started':
task_data['status'] = get_project_default_status(db, task.project_id)
else:
# Validate the provided status
if not validate_task_status(db, task.project_id, task_data['status']):
raise HTTPException(
status_code=400,
detail=f"Invalid status '{task_data['status']}' for this project"
)
```
### 3. Updated Task Status Update Endpoint
**Endpoint**: `PUT /tasks/{task_id}/status`
**Changes**:
- Validates new status against project's available statuses
- Returns 400 error if status is invalid for the task's project
- Supports both system and custom statuses
**Example**:
```python
# Validate the status for the task's project
if not validate_task_status(db, task.project_id, status_update.status):
raise HTTPException(
status_code=400,
detail=f"Invalid status '{status_update.status}' for this project"
)
```
### 4. Updated Task Update Endpoint
**Endpoint**: `PUT /tasks/{task_id}`
**Changes**:
- Validates status field if it's being updated
- Checks status validity before applying update
- Maintains existing permission checks
### 5. Updated Bulk Status Update Endpoint
**Endpoint**: `PUT /tasks/bulk/status`
**Changes**:
- Validates status for each task's project before updating
- Ensures atomic updates - all tasks succeed or all fail
- Provides detailed error messages for invalid statuses
- Handles tasks from different projects correctly
**Example**:
```python
# Validate status for the task's project
if not validate_task_status(db, task.project_id, bulk_update.status):
errors.append({
"task_id": task_id,
"error": f"Invalid status '{bulk_update.status}' for task's project"
})
failed_count += 1
continue
```
## Requirements Addressed
### Requirement 5.3: Default Status for New Tasks
✅ Task creation uses project's default status when not specified
✅ Falls back to system default "not_started" for projects without custom statuses
### Requirement 6.4: Backward Compatibility
✅ System statuses (not_started, in_progress, submitted, approved, retake) remain valid for all projects
✅ Existing tasks with system statuses continue to work
### Requirement 6.5: Status Resolution
✅ Status validation checks both system and custom statuses
✅ Custom statuses are project-specific and don't leak between projects
### Requirement 10.1: Bulk Status Update Validation
✅ Bulk updates validate status for each task's project
✅ Atomic updates ensure consistency
### Requirement 10.2: Cross-Project Handling
✅ Tasks from different projects can be updated in bulk
✅ Each task's status is validated against its own project's statuses
## Testing
### Test Coverage
Created comprehensive test suite (`backend/test_task_custom_status_support.py`) covering:
1. **Task Creation with Default Status**
- ✅ Default status correctly identified from custom statuses
- ✅ Tasks created without explicit status use project default
- ✅ Tasks can be created with explicit custom status
2. **Status Validation**
- ✅ All system statuses validated successfully
- ✅ Custom statuses validated for their project
- ✅ Invalid statuses correctly rejected
3. **Status Resolution Across Projects**
- ✅ Project 1 custom statuses rejected for Project 2
- ✅ Project 2 custom statuses rejected for Project 1
- ✅ System statuses valid for all projects
- ✅ Each project has its own default status
4. **Bulk Status Update Validation**
- ✅ Status validated for each task's project
- ✅ System statuses valid for all tasks
- ✅ Custom statuses only valid for their project
5. **Projects Without Custom Statuses**
- ✅ Use system default "not_started"
- ✅ System statuses remain valid
- ✅ Custom statuses from other projects rejected
### Test Results
```
============================================================
Testing Task Endpoints Custom Status Support
============================================================
=== Test 1: Task Creation with Default Status ===
✓ Default status correctly identified: custom_todo
✓ Task created with default status: custom_todo
✓ Task created with explicit custom status: custom_doing
✓ Test 1 PASSED
=== Test 2: Status Validation ===
✓ All system statuses validated successfully
✓ All custom statuses validated successfully
✓ Invalid statuses correctly rejected
✓ Test 2 PASSED
=== Test 3: Status Resolution Across Projects ===
✓ Project 1 custom status correctly rejected for Project 2
✓ Project 2 custom status correctly rejected for Project 1
✓ System statuses valid for both projects
✓ Project 1 default: custom_todo, Project 2 default: project2_status
✓ Test 3 PASSED
=== Test 4: Bulk Status Update Validation ===
✓ Validated 2 tasks from Project 1
✓ Validated 0 tasks from Project 2
✓ System statuses valid for all 2 tasks
✓ Test 4 PASSED
=== Test 5: Project Without Custom Statuses ===
✓ Project without custom statuses uses system default: not_started
✓ System statuses valid for project without custom statuses
✓ Custom statuses from other projects correctly rejected
✓ Test 5 PASSED
============================================================
✓ ALL TESTS PASSED
============================================================
```
## API Behavior
### Task Creation
```json
POST /tasks/
{
"project_id": 1,
"name": "New Task",
"task_type": "modeling",
// status not specified - will use project default
}
```
### Task Status Update
```json
PUT /tasks/123/status
{
"status": "custom_todo" // Must be valid for task's project
}
```
### Bulk Status Update
```json
PUT /tasks/bulk/status
{
"task_ids": [1, 2, 3],
"status": "custom_doing" // Validated for each task's project
}
```
## Error Handling
### Invalid Status Error
```json
{
"detail": "Invalid status 'custom_nonexistent' for this project"
}
```
### Bulk Update Errors
```json
{
"success_count": 2,
"failed_count": 1,
"errors": [
{
"task_id": 3,
"error": "Invalid status 'custom_todo' for task's project"
}
]
}
```
## Database Schema
No database changes required. The implementation uses existing fields:
- `projects.custom_task_statuses` (JSON) - stores custom statuses
- `tasks.status` (String) - stores status value
## Backward Compatibility
✅ **Fully backward compatible**
- Existing tasks with system statuses continue to work
- Projects without custom statuses use system defaults
- System statuses remain valid for all projects
- No migration required
## Next Steps
The frontend components should be updated to:
1. Fetch custom statuses from `/projects/{id}/task-statuses` endpoint
2. Display custom status colors in UI
3. Use custom statuses in status dropdowns and filters
4. Handle validation errors from backend
## Related Files
- `backend/routers/tasks.py` - Main implementation
- `backend/routers/projects.py` - Custom status management endpoints
- `backend/models/task.py` - Task model (status field is String)
- `backend/models/project.py` - Project model (custom_task_statuses field)
- `backend/test_task_custom_status_support.py` - Test suite
backend/docs/task-status-index-optimization.md
+101
View File
@@ -0,0 +1,101 @@
# Task Status Index Optimization Implementation
## Overview
This document summarizes the database schema and index optimization implemented for the shot-asset-task-status-optimization feature. The optimization addresses the N+1 query problem identified in the current shot and asset data fetching patterns.
## Problem Statement
The current implementation suffers from N+1 query patterns:
- **Main Query**: Fetches shots/assets first
- **Per-Entity Query**: For each shot/asset, runs separate query for tasks
- **Application-Level Aggregation**: Task status building happens in Python loops
For 100 shots, this results in 101 database queries (1 for shots + 100 for tasks).
## Solution Implemented
### New Database Indexes Created
The following indexes were created to optimize task status queries:
1. **`idx_tasks_shot_id_active`**
- Optimizes task lookups by shot_id (active tasks only)
- Includes WHERE clause: `deleted_at IS NULL`
2. **`idx_tasks_asset_id_active`**
- Optimizes task lookups by asset_id (active tasks only)
- Includes WHERE clause: `deleted_at IS NULL`
3. **`idx_tasks_status_type_active`**
- Optimizes task status and type filtering
- Covers: `(status, task_type)` with `deleted_at IS NULL`
4. **`idx_tasks_shot_status_type_active`**
- Composite index for shot + status + type queries
- Covers: `(shot_id, status, task_type)` with `deleted_at IS NULL`
5. **`idx_tasks_asset_status_type_active`**
- Composite index for asset + status + type queries
- Covers: `(asset_id, status, task_type)` with `deleted_at IS NULL`
6. **`idx_tasks_details_shot`**
- Optimizes queries needing full task details for shots
- Covers: `(shot_id, id, task_type, status, assigned_user_id, updated_at)`
7. **`idx_tasks_details_asset`**
- Optimizes queries needing full task details for assets
- Covers: `(asset_id, id, task_type, status, assigned_user_id, updated_at)`
8. **`idx_tasks_project_status_active`**
- Optimizes project-wide task queries with status filtering
- Covers: `(project_id, status, task_type)` with `deleted_at IS NULL`
### Performance Results
Testing with the current dataset (1,444 tasks, 441 shots, 15 assets):
| Query Type | Execution Time | Performance |
|------------|----------------|-------------|
| Shot list with task aggregation (441 shots) | 6ms | ✅ Excellent |
| Asset list with task aggregation (15 assets) | 1ms | ✅ Excellent |
| Project dashboard (1,444 tasks) | 1ms | ✅ Excellent |
| Task browser with filtering | 1ms | ✅ Excellent |
| Complex aggregation statistics | 4ms | ✅ Excellent |
**All queries perform well under the 500ms requirement**, with most completing in under 10ms.
### Index Usage Verification
Query plan analysis confirms that all new indexes are being used correctly:
- ✅ `idx_tasks_shot_id_active` used for shot task lookups
- ✅ `idx_tasks_asset_id_active` used for asset task lookups
- ✅ `idx_tasks_status_type_active` used for status filtering
- ✅ `idx_tasks_shot_status_type_active` used for shot+status combinations
- ✅ `idx_tasks_asset_status_type_active` used for asset+status combinations
## Files Created
1. **`create_task_status_indexes.py`** - Main index creation script
2. **`test_index_performance.py`** - Performance testing with realistic queries
3. **`test_index_scalability.py`** - Scalability testing with current dataset
4. **`check_indexes.py`** - Utility to inspect current database indexes
## Next Steps
The database optimization is complete and ready for the next phase:
1. **Backend Router Optimization** - Implement optimized query patterns in shot/asset routers
2. **Frontend Component Updates** - Remove redundant API calls in components
3. **Integration Testing** - Test end-to-end performance improvements
## Requirements Validation
This implementation satisfies the following requirements:
- ✅ **Requirement 3.1**: Uses optimized SQL joins for single database round trips
- ✅ **Requirement 3.2**: Maintains query performance through proper indexing strategies
- ✅ **Requirement 1.5 & 2.5**: Completes data fetching in under 500ms for 100+ entities
The database schema optimization provides the foundation for eliminating N+1 query patterns and achieving significant performance improvements in shot and asset data table rendering.
backend/fix_activity_metadata_column.py
+116
View File
@@ -0,0 +1,116 @@
#!/usr/bin/env python3
"""
Migration script to fix the activity_metadata column name in the activities table.
"""
from sqlalchemy import create_engine, text
from database import DATABASE_URL
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def migrate():
"""Fix the activity_metadata column name in the activities table."""
engine = create_engine(DATABASE_URL)
with engine.connect() as conn:
try:
# Check if the column exists with the wrong name
result = conn.execute(text("PRAGMA table_info(activities)")).fetchall()
columns = [row[1] for row in result]
if 'metadata' in columns and 'activity_metadata' not in columns:
logger.info("Renaming 'metadata' column to 'activity_metadata'...")
# SQLite doesn't support ALTER COLUMN RENAME directly, so we need to recreate the table
# First, create a backup of the data
conn.execute(text("""
CREATE TABLE activities_backup AS
SELECT * FROM activities
"""))
# Drop the old table
conn.execute(text("DROP TABLE activities"))
# Create the new table with correct column name
conn.execute(text("""
CREATE TABLE activities (
id INTEGER PRIMARY KEY AUTOINCREMENT,
type VARCHAR(50) NOT NULL,
user_id INTEGER NOT NULL,
project_id INTEGER,
task_id INTEGER,
asset_id INTEGER,
shot_id INTEGER,
submission_id INTEGER,
description TEXT NOT NULL,
activity_metadata TEXT,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE,
FOREIGN KEY (task_id) REFERENCES tasks(id) ON DELETE CASCADE,
FOREIGN KEY (asset_id) REFERENCES assets(id) ON DELETE CASCADE,
FOREIGN KEY (shot_id) REFERENCES shots(id) ON DELETE CASCADE,
FOREIGN KEY (submission_id) REFERENCES submissions(id) ON DELETE CASCADE
)
"""))
# Copy data back with correct column mapping
conn.execute(text("""
INSERT INTO activities (
id, type, user_id, project_id, task_id, asset_id, shot_id,
submission_id, description, activity_metadata, created_at
)
SELECT
id, type, user_id, project_id, task_id, asset_id, shot_id,
submission_id, description, metadata, created_at
FROM activities_backup
"""))
# Drop the backup table
conn.execute(text("DROP TABLE activities_backup"))
# Recreate indexes
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_activities_type
ON activities(type)
"""))
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_activities_user_id
ON activities(user_id)
"""))
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_activities_project_id
ON activities(project_id)
"""))
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_activities_created_at
ON activities(created_at)
"""))
logger.info("✅ Column renamed successfully!")
elif 'activity_metadata' in columns:
logger.info("✅ Column 'activity_metadata' already exists - no migration needed")
else:
logger.info("Adding 'activity_metadata' column...")
conn.execute(text("""
ALTER TABLE activities
ADD COLUMN activity_metadata TEXT
"""))
logger.info("✅ Column added successfully!")
conn.commit()
except Exception as e:
logger.error(f"❌ Migration failed: {str(e)}")
conn.rollback()
raise
if __name__ == "__main__":
migrate()
backend/fix_project_enums.py
+57
View File
@@ -0,0 +1,57 @@
#!/usr/bin/env python3
"""
Fix project enum values in the database to match the model definitions.
"""
import sqlite3
from pathlib import Path
def fix_project_enums():
"""Fix project enum values to match model definitions."""
print("Fixing Project Enum Values")
print("=" * 30)
db_path = "vfx_project_management.db"
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Fix project_type values
print("1. Fixing project_type values...")
cursor.execute("UPDATE projects SET project_type = 'cinema' WHERE project_type = 'CINEMA'")
cursor.execute("UPDATE projects SET project_type = 'tv' WHERE project_type = 'TV'")
cursor.execute("UPDATE projects SET project_type = 'game' WHERE project_type = 'GAME'")
# Fix status values
print("2. Fixing status values...")
cursor.execute("UPDATE projects SET status = 'planning' WHERE status = 'PLANNING'")
cursor.execute("UPDATE projects SET status = 'in_progress' WHERE status = 'IN_PROGRESS'")
cursor.execute("UPDATE projects SET status = 'on_hold' WHERE status = 'ON_HOLD'")
cursor.execute("UPDATE projects SET status = 'completed' WHERE status = 'COMPLETED'")
cursor.execute("UPDATE projects SET status = 'cancelled' WHERE status = 'CANCELLED'")
# Commit changes
conn.commit()
# Verify the changes
print("\n3. Verifying changes...")
cursor.execute("SELECT id, name, project_type, status FROM projects")
projects = cursor.fetchall()
for project in projects:
print(f" ID {project[0]}: {project[1]} - Type: {project[2]}, Status: {project[3]}")
print("\n✅ Project enum values fixed successfully!")
except Exception as e:
print(f"❌ Error: {e}")
if conn:
conn.rollback()
finally:
if conn:
conn.close()
if __name__ == "__main__":
fix_project_enums()
backend/fix_task_types_case.py
+36
View File
@@ -0,0 +1,36 @@
"""
Fix task types in database - convert uppercase to lowercase
"""
import sqlite3
conn = sqlite3.connect('backend/vfx_project_management.db')
cursor = conn.cursor()
print("=== Fixing Task Type Case ===\n")
# Get all distinct task types
cursor.execute('SELECT DISTINCT task_type FROM tasks')
task_types = [row[0] for row in cursor.fetchall()]
print(f"Current task types: {task_types}\n")
# Convert uppercase to lowercase
uppercase_types = [tt for tt in task_types if tt.isupper()]
if uppercase_types:
print(f"Converting {len(uppercase_types)} uppercase task types to lowercase...")
for task_type in uppercase_types:
lowercase = task_type.lower()
cursor.execute('UPDATE tasks SET task_type = ? WHERE task_type = ?', (lowercase, task_type))
count = cursor.rowcount
print(f" {task_type} -> {lowercase} ({count} tasks updated)")
conn.commit()
print("\n✓ Task types converted successfully")
else:
print("✓ All task types are already lowercase")
# Show final state
cursor.execute('SELECT DISTINCT task_type FROM tasks')
final_types = [row[0] for row in cursor.fetchall()]
print(f"\nFinal task types: {final_types}")
conn.close()
backend/main.py
+121
View File
@@ -0,0 +1,121 @@
from fastapi import FastAPI, Request, Response
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
from contextlib import asynccontextmanager
import time
import logging
import json
import os
from database import engine, Base
from routers import auth, users, projects, episodes, assets, shots, tasks, reviews, files, developer, settings, notifications, activities, admin, data_consistency
# Import models to ensure they are registered with SQLAlchemy
import models
# # Configure logging
# logging.basicConfig(
# level=logging.DEBUG,
# format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
# )
# logger = logging.getLogger("vfx_api")
# # logger = logging.getLogger("uvicorn")
# logger.setLevel(logging.DEBUG)
# # Also set the auth logger to debug
# auth_logger = logging.getLogger("vfx_auth")
# auth_logger.setLevel(logging.DEBUG)
@asynccontextmanager
async def lifespan(app: FastAPI):
# Create database tables
Base.metadata.create_all(bind=engine)
yield
app = FastAPI(
title="LinkDesk API",
description="A comprehensive project management system for animation and VFX production",
version="1.0.0",
lifespan=lifespan
)
# Configure CORS
app.add_middleware(
CORSMiddleware,
allow_origins=["http://localhost:5173", "http://localhost:5174"], # Vue.js dev server
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Add logging middleware
# @app.middleware("http")
# async def log_requests(request: Request, call_next):
# """Log all HTTP requests and responses."""
# start_time = time.time()
# # Log request
# client_ip = request.client.host if request.client else "unknown"
# auth_header = request.headers.get("authorization", "none")
# # Mask the token for security
# if auth_header and auth_header.startswith("Bearer "):
# auth_display = f"Bearer {auth_header[7:15]}..."
# else:
# auth_display = auth_header
# log_message = f"🔵 {request.method} {request.url.path} from {client_ip} | Auth: {auth_display}"
# logger.info(log_message)
# print(log_message) # Also print to console for debugging
# # Process the request
# try:
# response = await call_next(request)
# process_time = time.time() - start_time
# # Log response
# status_emoji = "🟢" if response.status_code < 400 else "🔴" if response.status_code >= 500 else "🟡"
# response_message = f"{status_emoji} {response.status_code} - {process_time:.3f}s"
# logger.info(response_message)
# print(response_message) # Also print to console for debugging
# return response
# except Exception as e:
# process_time = time.time() - start_time
# logger.error(f"🔴 ERROR: {str(e)} - {process_time:.3f}s")
# raise
# Mount static files for uploads
uploads_dir = os.path.join(os.path.dirname(__file__), "uploads")
if not os.path.exists(uploads_dir):
os.makedirs(uploads_dir)
app.mount("/uploads", StaticFiles(directory=uploads_dir), name="uploads")
# Include routers
app.include_router(auth.router, prefix="/auth", tags=["authentication"])
app.include_router(users.router, prefix="/users", tags=["users"])
app.include_router(projects.router, prefix="/projects", tags=["projects"])
app.include_router(episodes.router, prefix="/episodes", tags=["episodes"])
app.include_router(assets.router, prefix="/assets", tags=["assets"])
app.include_router(shots.router, prefix="/shots", tags=["shots"])
app.include_router(tasks.router, prefix="/tasks", tags=["tasks"])
app.include_router(reviews.router, prefix="/reviews", tags=["reviews"])
app.include_router(files.router, prefix="/files", tags=["files"])
app.include_router(developer.router, prefix="/developer", tags=["developer"])
app.include_router(settings.router, prefix="/settings", tags=["settings"])
app.include_router(notifications.router, tags=["notifications"])
app.include_router(activities.router, tags=["activities"])
app.include_router(admin.router, prefix="/admin", tags=["admin"])
app.include_router(data_consistency.router, prefix="/data-consistency", tags=["data-consistency"])
@app.get("/")
async def root():
return {"message": "VFX Project Management System API"}
@app.get("/health")
async def health_check():
return {"status": "healthy"}
backend/migrate_activities.py
+74
View File
@@ -0,0 +1,74 @@
"""
Migration script to add activities table to the database.
"""
from sqlalchemy import create_engine, text
from database import DATABASE_URL
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def migrate():
"""Add activities table to the database."""
engine = create_engine(DATABASE_URL)
with engine.connect() as conn:
try:
# Create activities table
logger.info("Creating activities table...")
conn.execute(text("""
CREATE TABLE IF NOT EXISTS activities (
id INTEGER PRIMARY KEY AUTOINCREMENT,
type VARCHAR(50) NOT NULL,
user_id INTEGER NOT NULL,
project_id INTEGER,
task_id INTEGER,
asset_id INTEGER,
shot_id INTEGER,
submission_id INTEGER,
description TEXT NOT NULL,
activity_metadata TEXT,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE,
FOREIGN KEY (task_id) REFERENCES tasks(id) ON DELETE CASCADE,
FOREIGN KEY (asset_id) REFERENCES assets(id) ON DELETE CASCADE,
FOREIGN KEY (shot_id) REFERENCES shots(id) ON DELETE CASCADE,
FOREIGN KEY (submission_id) REFERENCES submissions(id) ON DELETE CASCADE
)
"""))
# Create indexes for activities
logger.info("Creating indexes for activities table...")
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_activities_type
ON activities(type)
"""))
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_activities_user_id
ON activities(user_id)
"""))
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_activities_project_id
ON activities(project_id)
"""))
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_activities_created_at
ON activities(created_at)
"""))
conn.commit()
logger.info("✅ Activities table created successfully!")
except Exception as e:
logger.error(f"❌ Migration failed: {str(e)}")
conn.rollback()
raise
if __name__ == "__main__":
migrate()
backend/migrate_admin_users.py
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env python3
"""
Migration script to convert admin role users to admin permission users.
This script should be run once to migrate existing admin users to the new admin permission system.
"""
from sqlalchemy import create_engine, text
from sqlalchemy.orm import sessionmaker
from database import DATABASE_URL, Base
from models.user import User, UserRole
import logging
# Setup logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def migrate_admin_users():
"""Migrate existing admin role users to admin permission system."""
# Create engine and session
engine = create_engine(DATABASE_URL)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
# Create tables if they don't exist
Base.metadata.create_all(bind=engine)
db = SessionLocal()
try:
# Check if is_admin column exists, if not add it
try:
# Try to query is_admin column
db.execute(text("SELECT is_admin FROM users LIMIT 1"))
logger.info("is_admin column already exists")
except Exception:
# Add is_admin column if it doesn't exist
logger.info("Adding is_admin column to users table")
db.execute(text("ALTER TABLE users ADD COLUMN is_admin BOOLEAN DEFAULT FALSE"))
db.commit()
# Find all users with admin role and convert them
# Use raw SQL to avoid enum validation issues
result = db.execute(text("SELECT * FROM users WHERE role = 'ADMIN'"))
admin_users_data = result.fetchall()
# Also try lowercase 'admin' in case some are stored that way
result2 = db.execute(text("SELECT * FROM users WHERE role = 'admin'"))
admin_users_data2 = result2.fetchall()
all_admin_data = list(admin_users_data) + list(admin_users_data2)
if not all_admin_data:
logger.info("No admin role users found to migrate")
return
logger.info(f"Found {len(all_admin_data)} admin role users to migrate")
for user_data in all_admin_data:
user_id = user_data[0] # Assuming id is the first column
user_email = user_data[1] # Assuming email is the second column
logger.info(f"Migrating user {user_email} (ID: {user_id}) from admin role to admin permission")
# Update using raw SQL to avoid enum validation issues
# Use uppercase enum values as they're stored in the database
db.execute(text("""
UPDATE users
SET role = 'COORDINATOR', is_admin = TRUE
WHERE id = :user_id
"""), {"user_id": user_id})
logger.info(f"User {user_email} migrated: role=COORDINATOR, is_admin=True")
# Commit all changes
db.commit()
logger.info("Migration completed successfully")
# Verify migration using raw SQL
result = db.execute(text("SELECT email, role, is_admin FROM users WHERE is_admin = TRUE"))
admin_permission_users = result.fetchall()
logger.info(f"After migration: {len(admin_permission_users)} users have admin permission")
for user_data in admin_permission_users:
email, role, is_admin = user_data
logger.info(f" - {email}: role={role}, is_admin={is_admin}")
except Exception as e:
logger.error(f"Migration failed: {e}")
db.rollback()
raise
finally:
db.close()
if __name__ == "__main__":
logger.info("Starting admin user migration...")
migrate_admin_users()
logger.info("Migration completed!")
backend/migrate_avatar_field.py
+40
View File
@@ -0,0 +1,40 @@
"""
Migration script to add avatar_url field to users table
"""
import sqlite3
import os
def migrate_avatar_field():
"""Add avatar_url column to users table"""
db_path = "vfx_project_management.db"
if not os.path.exists(db_path):
print(f"Database file {db_path} not found!")
return
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
try:
# Check if column already exists
cursor.execute("PRAGMA table_info(users)")
columns = [column[1] for column in cursor.fetchall()]
if 'avatar_url' in columns:
print("✓ avatar_url column already exists in users table")
else:
# Add avatar_url column
cursor.execute("ALTER TABLE users ADD COLUMN avatar_url TEXT")
conn.commit()
print("✓ Added avatar_url column to users table")
print("\nMigration completed successfully!")
except Exception as e:
print(f"Error during migration: {e}")
conn.rollback()
finally:
conn.close()
if __name__ == "__main__":
migrate_avatar_field()
backend/migrate_custom_task_statuses.py
+91
View File
@@ -0,0 +1,91 @@
"""
Migration script to add custom_task_statuses column to projects table
and convert task status from enum to string.
"""
import sqlite3
import json
from pathlib import Path
# Database path
DB_PATH = Path(__file__).parent / "vfx_project_management.db"
def migrate_database():
"""Add custom_task_statuses column and migrate task status to string"""
conn = sqlite3.connect(DB_PATH)
cursor = conn.cursor()
try:
print("Starting migration for custom task statuses...")
# Check if column already exists in projects table
cursor.execute("PRAGMA table_info(projects)")
columns = [col[1] for col in cursor.fetchall()]
# Add custom_task_statuses column if it doesn't exist
if 'custom_task_statuses' not in columns:
print("Adding custom_task_statuses column to projects table...")
cursor.execute("""
ALTER TABLE projects
ADD COLUMN custom_task_statuses TEXT
""")
# Initialize with empty arrays
cursor.execute("""
UPDATE projects
SET custom_task_statuses = '[]'
WHERE custom_task_statuses IS NULL
""")
print("✓ Added custom_task_statuses column")
else:
print("✓ custom_task_statuses column already exists")
# Note: SQLite doesn't support changing column types directly
# The status column in tasks table will remain as TEXT in the database
# The Enum constraint was only enforced at the application level
print("✓ Task status column already supports string values")
# Verify existing task statuses and convert uppercase to lowercase
cursor.execute("SELECT DISTINCT status FROM tasks")
existing_statuses = [row[0] for row in cursor.fetchall()]
print(f"✓ Existing task statuses in database: {existing_statuses}")
# Convert uppercase enum values to lowercase string values
status_mapping = {
'NOT_STARTED': 'not_started',
'IN_PROGRESS': 'in_progress',
'SUBMITTED': 'submitted',
'APPROVED': 'approved',
'RETAKE': 'retake'
}
for old_status, new_status in status_mapping.items():
cursor.execute(
"UPDATE tasks SET status = ? WHERE status = ?",
(new_status, old_status)
)
updated_count = cursor.rowcount
if updated_count > 0:
print(f"✓ Converted {updated_count} tasks from '{old_status}' to '{new_status}'")
# Verify conversion
cursor.execute("SELECT DISTINCT status FROM tasks")
updated_statuses = [row[0] for row in cursor.fetchall()]
print(f"✓ Updated task statuses in database: {updated_statuses}")
conn.commit()
print("\n✅ Migration completed successfully!")
print("\nNext steps:")
print("1. System statuses remain available: not_started, in_progress, submitted, approved, retake")
print("2. Projects can now define custom statuses via the custom_task_statuses JSON column")
print("3. Tasks can use both system and custom status values")
except Exception as e:
conn.rollback()
print(f"\n❌ Migration failed: {e}")
raise
finally:
conn.close()
if __name__ == "__main__":
migrate_database()
backend/migrate_custom_task_types.py
+76
View File
@@ -0,0 +1,76 @@
"""
Migration script to add custom task type columns to projects table
and convert task_type from enum to string.
"""
import sqlite3
import json
from pathlib import Path
# Database path
DB_PATH = Path(__file__).parent / "vfx_project_management.db"
def migrate_database():
"""Add custom task type columns and migrate task_type to string"""
conn = sqlite3.connect(DB_PATH)
cursor = conn.cursor()
try:
print("Starting migration for custom task types...")
# Check if columns already exist
cursor.execute("PRAGMA table_info(projects)")
columns = [col[1] for col in cursor.fetchall()]
# Add custom_asset_task_types column if it doesn't exist
if 'custom_asset_task_types' not in columns:
print("Adding custom_asset_task_types column...")
cursor.execute("""
ALTER TABLE projects
ADD COLUMN custom_asset_task_types TEXT
""")
# Initialize with empty arrays
cursor.execute("""
UPDATE projects
SET custom_asset_task_types = '[]'
WHERE custom_asset_task_types IS NULL
""")
print("✓ Added custom_asset_task_types column")
else:
print("✓ custom_asset_task_types column already exists")
# Add custom_shot_task_types column if it doesn't exist
if 'custom_shot_task_types' not in columns:
print("Adding custom_shot_task_types column...")
cursor.execute("""
ALTER TABLE projects
ADD COLUMN custom_shot_task_types TEXT
""")
# Initialize with empty arrays
cursor.execute("""
UPDATE projects
SET custom_shot_task_types = '[]'
WHERE custom_shot_task_types IS NULL
""")
print("✓ Added custom_shot_task_types column")
else:
print("✓ custom_shot_task_types column already exists")
# Note: SQLite doesn't support changing column types directly
# The task_type column will remain as TEXT in the database
# The Enum constraint was only enforced at the application level
print("✓ Task type column already supports string values")
conn.commit()
print("\n✅ Migration completed successfully!")
except Exception as e:
conn.rollback()
print(f"\n❌ Migration failed: {e}")
raise
finally:
conn.close()
if __name__ == "__main__":
migrate_database()
backend/migrate_file_paths_production.py
+584
View File
@@ -0,0 +1,584 @@
#!/usr/bin/env python3
"""
Production-ready database migration script to convert absolute file paths to relative paths.
This script is designed for production deployment and includes comprehensive
validation, backup recommendations, and detailed logging.
Requirements addressed: 1.1, 1.2, 1.3, 1.4, 1.5
"""
import sys
import os
import logging
import shutil
from pathlib import Path
from typing import List, Tuple, Optional, Dict
from datetime import datetime
# Add the backend directory to the path
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
from sqlalchemy.orm import sessionmaker
from sqlalchemy import text
from database import engine
from models.task import Submission, TaskAttachment
from models.project import Project
# Configure logging
def setup_logging():
"""Setup logging configuration."""
log_filename = f"migration_file_paths_{datetime.now().strftime('%Y%m%d_%H%M%S')}.log"
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler(log_filename),
logging.StreamHandler()
]
)
return logging.getLogger(__name__)
class ProductionFilePathMigrator:
"""Production-ready file path migrator with comprehensive validation and error handling."""
def __init__(self):
self.logger = setup_logging()
self.SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
self.backend_dir = Path(__file__).parent.resolve()
self.errors = []
self.warnings = []
self.stats = {
'submissions_processed': 0,
'submissions_converted': 0,
'submissions_skipped': 0,
'attachments_processed': 0,
'attachments_converted': 0,
'attachments_skipped': 0,
'projects_processed': 0,
'projects_converted': 0,
'projects_skipped': 0,
'errors': 0,
'warnings': 0
}
def create_database_backup(self) -> Optional[str]:
"""Create a backup of the database before migration."""
try:
db_path = self.backend_dir / "database.db"
if not db_path.exists():
self.logger.warning("Database file not found, skipping backup")
return None
backup_filename = f"database_backup_{datetime.now().strftime('%Y%m%d_%H%M%S')}.db"
backup_path = self.backend_dir / backup_filename
shutil.copy2(db_path, backup_path)
self.logger.info(f"Database backup created: {backup_filename}")
return str(backup_path)
except Exception as e:
self.logger.error(f"Failed to create database backup: {e}")
return None
def validate_database_connection(self) -> bool:
"""Validate database connection and basic structure."""
try:
db = self.SessionLocal()
# Test basic queries
submissions_count = db.query(Submission).count()
attachments_count = db.query(TaskAttachment).count()
projects_count = db.query(Project).count()
self.logger.info(f"Database validation successful:")
self.logger.info(f" - Submissions: {submissions_count}")
self.logger.info(f" - Attachments: {attachments_count}")
self.logger.info(f" - Projects: {projects_count}")
db.close()
return True
except Exception as e:
self.logger.error(f"Database validation failed: {e}")
return False
def is_absolute_path(self, path: str) -> bool:
"""Check if a path is absolute."""
if not path:
return False
path_obj = Path(path)
return path_obj.is_absolute()
def convert_to_relative_path(self, absolute_path: str) -> Optional[str]:
"""
Convert absolute path to relative path with enhanced error handling.
Args:
absolute_path: The absolute file path to convert
Returns:
Relative path string or None if conversion fails
"""
try:
abs_path = Path(absolute_path).resolve()
# Check if the path is within the backend directory
try:
relative_path = abs_path.relative_to(self.backend_dir)
return str(relative_path).replace('\\', '/') # Use forward slashes for consistency
except ValueError:
# Path is not within backend directory - try to extract uploads part
path_parts = abs_path.parts
if 'uploads' in path_parts:
uploads_index = path_parts.index('uploads')
relative_parts = path_parts[uploads_index:]
relative_path = '/'.join(relative_parts)
self.logger.info(f"Extracted uploads path from external location: {relative_path}")
return relative_path
# Try to find backend directory in path
if 'backend' in path_parts:
backend_index = path_parts.index('backend')
if backend_index + 1 < len(path_parts):
relative_parts = path_parts[backend_index + 1:]
relative_path = '/'.join(relative_parts)
self.logger.info(f"Extracted path relative to backend: {relative_path}")
return relative_path
self.logger.warning(f"Cannot determine relative path for: {absolute_path}")
return None
except Exception as e:
self.logger.error(f"Failed to convert path {absolute_path}: {e}")
return None
def validate_file_exists(self, relative_path: str) -> Tuple[bool, str]:
"""
Validate that the file exists at the relative path.
Args:
relative_path: The relative path to validate
Returns:
Tuple of (exists, full_path)
"""
try:
full_path = self.backend_dir / relative_path
exists = full_path.exists()
return exists, str(full_path)
except Exception as e:
self.logger.error(f"Error validating file existence for {relative_path}: {e}")
return False, ""
def analyze_problematic_paths(self) -> Dict[str, List[str]]:
"""Analyze and categorize problematic paths before migration."""
self.logger.info("Analyzing potentially problematic paths...")
problematic = {
'submissions': [],
'attachments': [],
'projects': []
}
db = self.SessionLocal()
try:
# Check submissions
submissions = db.query(Submission).all()
for sub in submissions:
if self.is_absolute_path(sub.file_path):
relative = self.convert_to_relative_path(sub.file_path)
if not relative:
problematic['submissions'].append(f"ID {sub.id}: {sub.file_path}")
else:
exists, _ = self.validate_file_exists(relative)
if not exists:
problematic['submissions'].append(f"ID {sub.id}: File not found - {sub.file_path}")
# Check attachments
attachments = db.query(TaskAttachment).all()
for att in attachments:
if self.is_absolute_path(att.file_path):
relative = self.convert_to_relative_path(att.file_path)
if not relative:
problematic['attachments'].append(f"ID {att.id}: {att.file_path}")
else:
exists, _ = self.validate_file_exists(relative)
if not exists:
problematic['attachments'].append(f"ID {att.id}: File not found - {att.file_path}")
# Check projects
projects = db.query(Project).filter(Project.thumbnail_path.isnot(None)).all()
for proj in projects:
if self.is_absolute_path(proj.thumbnail_path):
relative = self.convert_to_relative_path(proj.thumbnail_path)
if not relative:
problematic['projects'].append(f"ID {proj.id}: {proj.thumbnail_path}")
else:
exists, _ = self.validate_file_exists(relative)
if not exists:
problematic['projects'].append(f"ID {proj.id}: File not found - {proj.thumbnail_path}")
finally:
db.close()
# Report findings
total_issues = sum(len(issues) for issues in problematic.values())
if total_issues > 0:
self.logger.warning(f"Found {total_issues} potentially problematic paths:")
for category, issues in problematic.items():
if issues:
self.logger.warning(f" {category.upper()}:")
for issue in issues:
self.logger.warning(f" - {issue}")
else:
self.logger.info("No problematic paths detected")
return problematic
def migrate_submissions_table(self) -> None:
"""Migrate file paths in submissions table with enhanced error handling."""
self.logger.info("Starting migration of submissions table...")
db = self.SessionLocal()
try:
submissions = db.query(Submission).all()
self.stats['submissions_processed'] = len(submissions)
for submission in submissions:
try:
if not self.is_absolute_path(submission.file_path):
self.logger.debug(f"Submission {submission.id} already has relative path: {submission.file_path}")
self.stats['submissions_skipped'] += 1
continue
relative_path = self.convert_to_relative_path(submission.file_path)
if relative_path:
# Validate file exists
exists, full_path = self.validate_file_exists(relative_path)
if exists:
old_path = submission.file_path
submission.file_path = relative_path
self.stats['submissions_converted'] += 1
self.logger.info(f"Submission {submission.id}: {old_path} -> {relative_path}")
else:
error_msg = f"Submission {submission.id}: File not found at {full_path}"
self.logger.warning(error_msg)
self.warnings.append(error_msg)
self.stats['warnings'] += 1
else:
error_msg = f"Submission {submission.id}: Failed to convert path {submission.file_path}"
self.logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
except Exception as e:
error_msg = f"Submission {submission.id}: Exception during migration: {e}"
self.logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
db.commit()
self.logger.info(f"Submissions migration completed. Converted: {self.stats['submissions_converted']}, Skipped: {self.stats['submissions_skipped']}")
except Exception as e:
db.rollback()
self.logger.error(f"Failed to migrate submissions table: {e}")
raise
finally:
db.close()
def migrate_attachments_table(self) -> None:
"""Migrate file paths in task_attachments table with enhanced error handling."""
self.logger.info("Starting migration of task_attachments table...")
db = self.SessionLocal()
try:
attachments = db.query(TaskAttachment).all()
self.stats['attachments_processed'] = len(attachments)
for attachment in attachments:
try:
if not self.is_absolute_path(attachment.file_path):
self.logger.debug(f"Attachment {attachment.id} already has relative path: {attachment.file_path}")
self.stats['attachments_skipped'] += 1
continue
relative_path = self.convert_to_relative_path(attachment.file_path)
if relative_path:
# Validate file exists
exists, full_path = self.validate_file_exists(relative_path)
if exists:
old_path = attachment.file_path
attachment.file_path = relative_path
self.stats['attachments_converted'] += 1
self.logger.info(f"Attachment {attachment.id}: {old_path} -> {relative_path}")
else:
error_msg = f"Attachment {attachment.id}: File not found at {full_path}"
self.logger.warning(error_msg)
self.warnings.append(error_msg)
self.stats['warnings'] += 1
else:
error_msg = f"Attachment {attachment.id}: Failed to convert path {attachment.file_path}"
self.logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
except Exception as e:
error_msg = f"Attachment {attachment.id}: Exception during migration: {e}"
self.logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
db.commit()
self.logger.info(f"Attachments migration completed. Converted: {self.stats['attachments_converted']}, Skipped: {self.stats['attachments_skipped']}")
except Exception as e:
db.rollback()
self.logger.error(f"Failed to migrate attachments table: {e}")
raise
finally:
db.close()
def migrate_projects_table(self) -> None:
"""Migrate thumbnail paths in projects table with enhanced error handling."""
self.logger.info("Starting migration of projects table...")
db = self.SessionLocal()
try:
projects = db.query(Project).filter(Project.thumbnail_path.isnot(None)).all()
self.stats['projects_processed'] = len(projects)
for project in projects:
try:
if not self.is_absolute_path(project.thumbnail_path):
self.logger.debug(f"Project {project.id} already has relative thumbnail path: {project.thumbnail_path}")
self.stats['projects_skipped'] += 1
continue
relative_path = self.convert_to_relative_path(project.thumbnail_path)
if relative_path:
# Validate file exists
exists, full_path = self.validate_file_exists(relative_path)
if exists:
old_path = project.thumbnail_path
project.thumbnail_path = relative_path
self.stats['projects_converted'] += 1
self.logger.info(f"Project {project.id}: {old_path} -> {relative_path}")
else:
error_msg = f"Project {project.id}: Thumbnail not found at {full_path}"
self.logger.warning(error_msg)
self.warnings.append(error_msg)
self.stats['warnings'] += 1
else:
error_msg = f"Project {project.id}: Failed to convert thumbnail path {project.thumbnail_path}"
self.logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
except Exception as e:
error_msg = f"Project {project.id}: Exception during migration: {e}"
self.logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
db.commit()
self.logger.info(f"Projects migration completed. Converted: {self.stats['projects_converted']}, Skipped: {self.stats['projects_skipped']}")
except Exception as e:
db.rollback()
self.logger.error(f"Failed to migrate projects table: {e}")
raise
finally:
db.close()
def validate_migration_results(self) -> bool:
"""Comprehensive validation of migration results."""
self.logger.info("Validating migration results...")
db = self.SessionLocal()
try:
# Check for any remaining absolute paths using multiple patterns
absolute_patterns = ['C:%', 'D:%', 'E:%', 'F:%', '/%', '/home/%', '/usr/%', '/var/%']
remaining_absolute_submissions = 0
remaining_absolute_attachments = 0
remaining_absolute_projects = 0
for pattern in absolute_patterns:
remaining_absolute_submissions += db.query(Submission).filter(
Submission.file_path.like(pattern)
).count()
remaining_absolute_attachments += db.query(TaskAttachment).filter(
TaskAttachment.file_path.like(pattern)
).count()
remaining_absolute_projects += db.query(Project).filter(
Project.thumbnail_path.like(pattern)
).count()
total_remaining = remaining_absolute_submissions + remaining_absolute_attachments + remaining_absolute_projects
# Additional validation: check that all relative paths start with expected prefixes
valid_prefixes = ['uploads/', './uploads/', 'backend/uploads/']
invalid_submissions = db.query(Submission).filter(
~Submission.file_path.like('uploads/%')
).count()
invalid_attachments = db.query(TaskAttachment).filter(
~TaskAttachment.file_path.like('uploads/%')
).count()
invalid_projects = db.query(Project).filter(
Project.thumbnail_path.isnot(None),
~Project.thumbnail_path.like('uploads/%')
).count()
total_invalid = invalid_submissions + invalid_attachments + invalid_projects
if total_remaining > 0:
self.logger.warning(f"Migration incomplete: {total_remaining} absolute paths remain")
self.logger.warning(f" - Submissions: {remaining_absolute_submissions}")
self.logger.warning(f" - Attachments: {remaining_absolute_attachments}")
self.logger.warning(f" - Projects: {remaining_absolute_projects}")
return False
elif total_invalid > 0:
self.logger.warning(f"Migration validation found {total_invalid} paths with unexpected format")
self.logger.warning(f" - Submissions: {invalid_submissions}")
self.logger.warning(f" - Attachments: {invalid_attachments}")
self.logger.warning(f" - Projects: {invalid_projects}")
return False
else:
self.logger.info("Migration validation successful: All paths are now relative and properly formatted")
return True
except Exception as e:
self.logger.error(f"Failed to validate migration: {e}")
return False
finally:
db.close()
def print_migration_summary(self) -> None:
"""Print a comprehensive summary of the migration results."""
self.logger.info("=" * 60)
self.logger.info("MIGRATION SUMMARY")
self.logger.info("=" * 60)
self.logger.info(f"Submissions processed: {self.stats['submissions_processed']}")
self.logger.info(f"Submissions converted: {self.stats['submissions_converted']}")
self.logger.info(f"Submissions skipped: {self.stats['submissions_skipped']}")
self.logger.info(f"Attachments processed: {self.stats['attachments_processed']}")
self.logger.info(f"Attachments converted: {self.stats['attachments_converted']}")
self.logger.info(f"Attachments skipped: {self.stats['attachments_skipped']}")
self.logger.info(f"Projects processed: {self.stats['projects_processed']}")
self.logger.info(f"Projects converted: {self.stats['projects_converted']}")
self.logger.info(f"Projects skipped: {self.stats['projects_skipped']}")
self.logger.info(f"Total errors: {self.stats['errors']}")
self.logger.info(f"Total warnings: {self.stats['warnings']}")
if self.errors:
self.logger.info("=" * 60)
self.logger.info("ERRORS")
self.logger.info("=" * 60)
for error in self.errors:
self.logger.error(f" - {error}")
if self.warnings:
self.logger.info("=" * 60)
self.logger.info("WARNINGS")
self.logger.info("=" * 60)
for warning in self.warnings:
self.logger.warning(f" - {warning}")
def run_migration(self, create_backup: bool = True) -> bool:
"""Run the complete migration process with comprehensive validation."""
self.logger.info("Starting production file path migration to relative paths...")
self.logger.info(f"Backend directory: {self.backend_dir}")
try:
# Step 1: Validate database connection
if not self.validate_database_connection():
self.logger.error("Database validation failed. Aborting migration.")
return False
# Step 2: Create backup if requested
backup_path = None
if create_backup:
backup_path = self.create_database_backup()
if backup_path:
self.logger.info(f"Database backup created at: {backup_path}")
else:
self.logger.warning("Failed to create backup, but continuing with migration")
# Step 3: Analyze problematic paths
problematic_paths = self.analyze_problematic_paths()
# Step 4: Migrate each table
self.migrate_submissions_table()
self.migrate_attachments_table()
self.migrate_projects_table()
# Step 5: Validate results
validation_success = self.validate_migration_results()
# Step 6: Print summary
self.print_migration_summary()
# Step 7: Determine overall success
if validation_success and self.stats['errors'] == 0:
self.logger.info("Migration completed successfully!")
if backup_path:
self.logger.info(f"Backup is available at: {backup_path}")
return True
else:
self.logger.warning("Migration completed with warnings or errors. Please review the log.")
if backup_path:
self.logger.info(f"Database can be restored from backup: {backup_path}")
return False
except Exception as e:
self.logger.error(f"Migration failed: {e}")
return False
def main():
"""Main function for production migration."""
print("Production File Path Migration Script")
print("=" * 60)
print("This script will convert absolute file paths to relative paths in the database.")
print("It includes comprehensive validation and error handling for production use.")
print()
print("IMPORTANT: This script will create a database backup before migration.")
print(" Review the log file for detailed results.")
print()
# Ask for confirmation
response = input("Do you want to proceed with the migration? (y/N): ").strip().lower()
if response not in ['y', 'yes']:
print("Migration cancelled.")
return
# Ask about backup
backup_response = input("Create database backup before migration? (Y/n): ").strip().lower()
create_backup = backup_response not in ['n', 'no']
# Run migration
migrator = ProductionFilePathMigrator()
success = migrator.run_migration(create_backup=create_backup)
if success:
print("\nMigration completed successfully!")
print("Check the log file for detailed results.")
sys.exit(0)
else:
print("\nMigration completed with errors or warnings.")
print("Check the log file for detailed results.")
print("Database backup is available if restoration is needed.")
sys.exit(1)
if __name__ == "__main__":
main()
backend/migrate_file_paths_to_relative.py
+381
View File
@@ -0,0 +1,381 @@
#!/usr/bin/env python3
"""
Database migration script to convert absolute file paths to relative paths.
This script addresses the file path storage issue where absolute paths stored
in the database become invalid when deploying to different environments,
particularly Linux. The solution converts all absolute paths to relative paths
that are resolved dynamically at runtime.
Requirements addressed: 1.1, 1.2, 1.3, 1.4, 1.5
"""
import sys
import os
import logging
from pathlib import Path
from typing import List, Tuple, Optional
# Add the backend directory to the path
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
from sqlalchemy.orm import sessionmaker
from sqlalchemy import text
from database import engine
from models.task import Submission, TaskAttachment
from models.project import Project
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler('migration_file_paths.log'),
logging.StreamHandler()
]
)
logger = logging.getLogger(__name__)
class FilePathMigrator:
"""Handles migration of absolute file paths to relative paths."""
def __init__(self):
self.SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
self.backend_dir = Path(__file__).parent.resolve()
self.errors = []
self.stats = {
'submissions_processed': 0,
'submissions_converted': 0,
'attachments_processed': 0,
'attachments_converted': 0,
'projects_processed': 0,
'projects_converted': 0,
'errors': 0
}
def is_absolute_path(self, path: str) -> bool:
"""Check if a path is absolute."""
if not path:
return False
path_obj = Path(path)
return path_obj.is_absolute()
def convert_to_relative_path(self, absolute_path: str) -> Optional[str]:
"""
Convert absolute path to relative path.
Args:
absolute_path: The absolute file path to convert
Returns:
Relative path string or None if conversion fails
"""
try:
abs_path = Path(absolute_path).resolve()
# Check if the path is within the backend directory
try:
relative_path = abs_path.relative_to(self.backend_dir)
return str(relative_path).replace('\\', '/') # Use forward slashes for consistency
except ValueError:
# Path is not within backend directory
logger.warning(f"Path is outside backend directory: {absolute_path}")
# Try to extract just the uploads part if it exists
path_parts = abs_path.parts
if 'uploads' in path_parts:
uploads_index = path_parts.index('uploads')
relative_parts = path_parts[uploads_index:]
relative_path = '/'.join(relative_parts)
logger.info(f"Extracted uploads path: {relative_path}")
return relative_path
return None
except Exception as e:
logger.error(f"Failed to convert path {absolute_path}: {e}")
return None
def validate_file_exists(self, relative_path: str) -> bool:
"""
Validate that the file exists at the relative path.
Args:
relative_path: The relative path to validate
Returns:
True if file exists, False otherwise
"""
try:
full_path = self.backend_dir / relative_path
return full_path.exists()
except Exception:
return False
def migrate_submissions_table(self) -> None:
"""Migrate file paths in submissions table."""
logger.info("Starting migration of submissions table...")
db = self.SessionLocal()
try:
submissions = db.query(Submission).all()
self.stats['submissions_processed'] = len(submissions)
for submission in submissions:
try:
if not self.is_absolute_path(submission.file_path):
logger.debug(f"Submission {submission.id} already has relative path: {submission.file_path}")
continue
relative_path = self.convert_to_relative_path(submission.file_path)
if relative_path:
# Validate file exists
if self.validate_file_exists(relative_path):
old_path = submission.file_path
submission.file_path = relative_path
self.stats['submissions_converted'] += 1
logger.info(f"Submission {submission.id}: {old_path} -> {relative_path}")
else:
error_msg = f"Submission {submission.id}: File not found at relative path {relative_path}"
logger.warning(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
else:
error_msg = f"Submission {submission.id}: Failed to convert path {submission.file_path}"
logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
except Exception as e:
error_msg = f"Submission {submission.id}: Exception during migration: {e}"
logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
db.commit()
logger.info(f"Submissions migration completed. Converted: {self.stats['submissions_converted']}/{self.stats['submissions_processed']}")
except Exception as e:
db.rollback()
logger.error(f"Failed to migrate submissions table: {e}")
raise
finally:
db.close()
def migrate_attachments_table(self) -> None:
"""Migrate file paths in task_attachments table."""
logger.info("Starting migration of task_attachments table...")
db = self.SessionLocal()
try:
attachments = db.query(TaskAttachment).all()
self.stats['attachments_processed'] = len(attachments)
for attachment in attachments:
try:
if not self.is_absolute_path(attachment.file_path):
logger.debug(f"Attachment {attachment.id} already has relative path: {attachment.file_path}")
continue
relative_path = self.convert_to_relative_path(attachment.file_path)
if relative_path:
# Validate file exists
if self.validate_file_exists(relative_path):
old_path = attachment.file_path
attachment.file_path = relative_path
self.stats['attachments_converted'] += 1
logger.info(f"Attachment {attachment.id}: {old_path} -> {relative_path}")
else:
error_msg = f"Attachment {attachment.id}: File not found at relative path {relative_path}"
logger.warning(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
else:
error_msg = f"Attachment {attachment.id}: Failed to convert path {attachment.file_path}"
logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
except Exception as e:
error_msg = f"Attachment {attachment.id}: Exception during migration: {e}"
logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
db.commit()
logger.info(f"Attachments migration completed. Converted: {self.stats['attachments_converted']}/{self.stats['attachments_processed']}")
except Exception as e:
db.rollback()
logger.error(f"Failed to migrate attachments table: {e}")
raise
finally:
db.close()
def migrate_projects_table(self) -> None:
"""Migrate thumbnail paths in projects table."""
logger.info("Starting migration of projects table...")
db = self.SessionLocal()
try:
projects = db.query(Project).filter(Project.thumbnail_path.isnot(None)).all()
self.stats['projects_processed'] = len(projects)
for project in projects:
try:
if not self.is_absolute_path(project.thumbnail_path):
logger.debug(f"Project {project.id} already has relative thumbnail path: {project.thumbnail_path}")
continue
relative_path = self.convert_to_relative_path(project.thumbnail_path)
if relative_path:
# Validate file exists
if self.validate_file_exists(relative_path):
old_path = project.thumbnail_path
project.thumbnail_path = relative_path
self.stats['projects_converted'] += 1
logger.info(f"Project {project.id}: {old_path} -> {relative_path}")
else:
error_msg = f"Project {project.id}: Thumbnail not found at relative path {relative_path}"
logger.warning(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
else:
error_msg = f"Project {project.id}: Failed to convert thumbnail path {project.thumbnail_path}"
logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
except Exception as e:
error_msg = f"Project {project.id}: Exception during migration: {e}"
logger.error(error_msg)
self.errors.append(error_msg)
self.stats['errors'] += 1
db.commit()
logger.info(f"Projects migration completed. Converted: {self.stats['projects_converted']}/{self.stats['projects_processed']}")
except Exception as e:
db.rollback()
logger.error(f"Failed to migrate projects table: {e}")
raise
finally:
db.close()
def validate_migration_results(self) -> bool:
"""Validate that migration was successful."""
logger.info("Validating migration results...")
db = self.SessionLocal()
try:
# Check for any remaining absolute paths
remaining_absolute_submissions = db.query(Submission).filter(
Submission.file_path.like('C:%') |
Submission.file_path.like('D:%') |
Submission.file_path.like('/%')
).count()
remaining_absolute_attachments = db.query(TaskAttachment).filter(
TaskAttachment.file_path.like('C:%') |
TaskAttachment.file_path.like('D:%') |
TaskAttachment.file_path.like('/%')
).count()
remaining_absolute_projects = db.query(Project).filter(
Project.thumbnail_path.like('C:%') |
Project.thumbnail_path.like('D:%') |
Project.thumbnail_path.like('/%')
).count()
total_remaining = remaining_absolute_submissions + remaining_absolute_attachments + remaining_absolute_projects
if total_remaining > 0:
logger.warning(f"Migration incomplete: {total_remaining} absolute paths remain")
logger.warning(f" - Submissions: {remaining_absolute_submissions}")
logger.warning(f" - Attachments: {remaining_absolute_attachments}")
logger.warning(f" - Projects: {remaining_absolute_projects}")
return False
else:
logger.info("Migration validation successful: No absolute paths remain")
return True
except Exception as e:
logger.error(f"Failed to validate migration: {e}")
return False
finally:
db.close()
def print_migration_summary(self) -> None:
"""Print a summary of the migration results."""
logger.info("=== MIGRATION SUMMARY ===")
logger.info(f"Submissions processed: {self.stats['submissions_processed']}")
logger.info(f"Submissions converted: {self.stats['submissions_converted']}")
logger.info(f"Attachments processed: {self.stats['attachments_processed']}")
logger.info(f"Attachments converted: {self.stats['attachments_converted']}")
logger.info(f"Projects processed: {self.stats['projects_processed']}")
logger.info(f"Projects converted: {self.stats['projects_converted']}")
logger.info(f"Total errors: {self.stats['errors']}")
if self.errors:
logger.info("=== ERRORS ===")
for error in self.errors:
logger.info(f" - {error}")
def run_migration(self) -> bool:
"""Run the complete migration process."""
logger.info("Starting file path migration to relative paths...")
logger.info(f"Backend directory: {self.backend_dir}")
try:
# Migrate each table
self.migrate_submissions_table()
self.migrate_attachments_table()
self.migrate_projects_table()
# Validate results
validation_success = self.validate_migration_results()
# Print summary
self.print_migration_summary()
if validation_success and self.stats['errors'] == 0:
logger.info("Migration completed successfully!")
return True
else:
logger.warning("Migration completed with warnings or errors. Please review the log.")
return False
except Exception as e:
logger.error(f"Migration failed: {e}")
return False
def main():
"""Main function to run the migration."""
print("File Path Migration Script")
print("=" * 50)
print("This script will convert absolute file paths to relative paths in the database.")
print("A backup of the database is recommended before running this migration.")
print()
# Ask for confirmation
response = input("Do you want to proceed with the migration? (y/N): ").strip().lower()
if response not in ['y', 'yes']:
print("Migration cancelled.")
return
# Run migration
migrator = FilePathMigrator()
success = migrator.run_migration()
if success:
print("\nMigration completed successfully!")
sys.exit(0)
else:
print("\nMigration completed with errors. Please check the log file.")
sys.exit(1)
if __name__ == "__main__":
main()
backend/migrate_notifications.py
+95
View File
@@ -0,0 +1,95 @@
"""
Migration script to add notification tables to the database.
"""
from sqlalchemy import create_engine, text
from database import DATABASE_URL
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def migrate():
"""Add notification tables to the database."""
engine = create_engine(DATABASE_URL)
with engine.connect() as conn:
try:
# Create notifications table
logger.info("Creating notifications table...")
conn.execute(text("""
CREATE TABLE IF NOT EXISTS notifications (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
type VARCHAR(50) NOT NULL,
priority VARCHAR(20) DEFAULT 'normal',
title VARCHAR(255) NOT NULL,
message TEXT NOT NULL,
read BOOLEAN DEFAULT 0,
project_id INTEGER,
task_id INTEGER,
submission_id INTEGER,
email_sent BOOLEAN DEFAULT 0,
email_sent_at DATETIME,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
read_at DATETIME,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE,
FOREIGN KEY (task_id) REFERENCES tasks(id) ON DELETE CASCADE,
FOREIGN KEY (submission_id) REFERENCES submissions(id) ON DELETE CASCADE
)
"""))
# Create indexes for notifications
logger.info("Creating indexes for notifications table...")
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_notifications_user_id
ON notifications(user_id)
"""))
conn.execute(text("""
CREATE INDEX IF NOT EXISTS idx_notifications_read
ON notifications(read)
"""))
# Create user_notification_preferences table
logger.info("Creating user_notification_preferences table...")
conn.execute(text("""
CREATE TABLE IF NOT EXISTS user_notification_preferences (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL UNIQUE,
email_enabled BOOLEAN DEFAULT 1,
email_task_assigned BOOLEAN DEFAULT 1,
email_task_status_changed BOOLEAN DEFAULT 1,
email_submission_reviewed BOOLEAN DEFAULT 1,
email_work_submitted BOOLEAN DEFAULT 1,
email_deadline_approaching BOOLEAN DEFAULT 1,
email_project_update BOOLEAN DEFAULT 1,
email_comment_added BOOLEAN DEFAULT 1,
inapp_enabled BOOLEAN DEFAULT 1,
inapp_task_assigned BOOLEAN DEFAULT 1,
inapp_task_status_changed BOOLEAN DEFAULT 1,
inapp_submission_reviewed BOOLEAN DEFAULT 1,
inapp_work_submitted BOOLEAN DEFAULT 1,
inapp_deadline_approaching BOOLEAN DEFAULT 1,
inapp_project_update BOOLEAN DEFAULT 1,
inapp_comment_added BOOLEAN DEFAULT 1,
email_digest_enabled BOOLEAN DEFAULT 0,
email_digest_frequency VARCHAR(50) DEFAULT 'daily',
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
)
"""))
conn.commit()
logger.info("✅ Notification tables created successfully!")
except Exception as e:
logger.error(f"❌ Migration failed: {str(e)}")
conn.rollback()
raise
if __name__ == "__main__":
migrate()
backend/migrate_project_fields.py
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/env python3
"""
Migration script to add new fields to the projects table:
- code_name: Unique project code identifier
- client_name: Client or studio name
- project_type: TV, Cinema, or Game
"""
import sqlite3
import sys
from pathlib import Path
def migrate_project_fields():
"""Add new fields to the projects table"""
# Database path
db_path = Path("vfx_project_management.db")
if not db_path.exists():
print("Database file not found. Creating new database with updated schema.")
return True
try:
# Connect to database
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Check if new columns already exist
cursor.execute("PRAGMA table_info(projects)")
columns = [column[1] for column in cursor.fetchall()]
new_columns_needed = []
if 'code_name' not in columns:
new_columns_needed.append('code_name')
if 'client_name' not in columns:
new_columns_needed.append('client_name')
if 'project_type' not in columns:
new_columns_needed.append('project_type')
if not new_columns_needed:
print("All new columns already exist. No migration needed.")
return True
print(f"Adding columns: {', '.join(new_columns_needed)}")
# Add new columns
if 'code_name' in new_columns_needed:
cursor.execute("ALTER TABLE projects ADD COLUMN code_name VARCHAR")
print("Added code_name column")
if 'client_name' in new_columns_needed:
cursor.execute("ALTER TABLE projects ADD COLUMN client_name VARCHAR")
print("Added client_name column")
if 'project_type' in new_columns_needed:
cursor.execute("ALTER TABLE projects ADD COLUMN project_type VARCHAR")
print("Added project_type column")
# Update existing projects with default values
cursor.execute("SELECT id, name FROM projects WHERE code_name IS NULL OR client_name IS NULL OR project_type IS NULL")
existing_projects = cursor.fetchall()
for project_id, project_name in existing_projects:
# Generate default code_name from project name
code_name = project_name.upper().replace(' ', '_')[:10] + f"_{project_id:03d}"
cursor.execute("""
UPDATE projects
SET code_name = COALESCE(code_name, ?),
client_name = COALESCE(client_name, 'Default Client'),
project_type = COALESCE(project_type, 'tv')
WHERE id = ?
""", (code_name, project_id))
print(f"Updated {len(existing_projects)} existing projects with default values")
# Create unique index on code_name
try:
cursor.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_projects_code_name ON projects(code_name)")
print("Created unique index on code_name")
except sqlite3.IntegrityError as e:
print(f"Warning: Could not create unique index on code_name: {e}")
print("You may need to manually ensure code_name values are unique")
# Commit changes
conn.commit()
print("Migration completed successfully!")
return True
except Exception as e:
print(f"Migration failed: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
success = migrate_project_fields()
sys.exit(0 if success else 1)
backend/migrate_project_settings.py
+110
View File
@@ -0,0 +1,110 @@
"""
Migration script to add project settings fields to the projects table.
This adds support for upload location configuration and default task templates.
"""
import sqlite3
import json
def migrate_project_settings():
"""Add project settings fields to projects table"""
conn = sqlite3.connect('vfx_project_management.db')
cursor = conn.cursor()
try:
# Check if columns already exist
cursor.execute("PRAGMA table_info(projects)")
columns = [column[1] for column in cursor.fetchall()]
# Add upload_data_location column if it doesn't exist
if 'upload_data_location' not in columns:
print("Adding upload_data_location column...")
cursor.execute("""
ALTER TABLE projects
ADD COLUMN upload_data_location TEXT
""")
print("✓ Added upload_data_location column")
else:
print("✓ upload_data_location column already exists")
# Add asset_task_templates column if it doesn't exist
if 'asset_task_templates' not in columns:
print("Adding asset_task_templates column...")
cursor.execute("""
ALTER TABLE projects
ADD COLUMN asset_task_templates TEXT
""")
# Set default values for existing projects
default_asset_templates = json.dumps({
"characters": ["modeling", "surfacing", "rigging"],
"props": ["modeling", "surfacing"],
"sets": ["modeling", "surfacing"],
"vehicles": ["modeling", "surfacing", "rigging"]
})
cursor.execute("""
UPDATE projects
SET asset_task_templates = ?
WHERE asset_task_templates IS NULL
""", (default_asset_templates,))
print("✓ Added asset_task_templates column with default values")
else:
print("✓ asset_task_templates column already exists")
# Add shot_task_templates column if it doesn't exist
if 'shot_task_templates' not in columns:
print("Adding shot_task_templates column...")
cursor.execute("""
ALTER TABLE projects
ADD COLUMN shot_task_templates TEXT
""")
# Set default values for existing projects
default_shot_templates = json.dumps([
"layout", "animation", "simulation", "lighting", "compositing"
])
cursor.execute("""
UPDATE projects
SET shot_task_templates = ?
WHERE shot_task_templates IS NULL
""", (default_shot_templates,))
print("✓ Added shot_task_templates column with default values")
else:
print("✓ shot_task_templates column already exists")
# Add enabled_asset_tasks column if it doesn't exist
if 'enabled_asset_tasks' not in columns:
print("Adding enabled_asset_tasks column...")
cursor.execute("""
ALTER TABLE projects
ADD COLUMN enabled_asset_tasks TEXT
""")
print("✓ Added enabled_asset_tasks column")
else:
print("✓ enabled_asset_tasks column already exists")
# Add enabled_shot_tasks column if it doesn't exist
if 'enabled_shot_tasks' not in columns:
print("Adding enabled_shot_tasks column...")
cursor.execute("""
ALTER TABLE projects
ADD COLUMN enabled_shot_tasks TEXT
""")
print("✓ Added enabled_shot_tasks column")
else:
print("✓ enabled_shot_tasks column already exists")
conn.commit()
print("\n✅ Migration completed successfully!")
except Exception as e:
conn.rollback()
print(f"\n❌ Migration failed: {e}")
raise
finally:
conn.close()
if __name__ == "__main__":
print("Starting project settings migration...")
print("=" * 50)
migrate_project_settings()
backend/migrate_project_technical_specs.py
+168
View File
@@ -0,0 +1,168 @@
#!/usr/bin/env python3
"""
Migration script to add technical specifications fields to the projects table.
This script adds the following columns to the projects table:
- frame_rate (FLOAT)
- data_drive_path (VARCHAR)
- publish_storage_path (VARCHAR)
- delivery_image_resolution (VARCHAR)
- delivery_movie_specs_by_department (JSON)
Usage:
python migrate_project_technical_specs.py
"""
import sqlite3
import json
import sys
from pathlib import Path
# Default delivery movie specifications per department
DEFAULT_DELIVERY_MOVIE_SPECS = {
"layout": {
"resolution": "1920x1080",
"format": "mov",
"codec": "h264",
"quality": "medium"
},
"animation": {
"resolution": "1920x1080",
"format": "mov",
"codec": "h264",
"quality": "high"
},
"lighting": {
"resolution": "2048x1080",
"format": "exr",
"codec": None,
"quality": "high"
},
"composite": {
"resolution": "2048x1080",
"format": "mov",
"codec": "prores",
"quality": "high"
}
}
def get_database_path():
"""Get the database path, trying multiple possible locations."""
possible_paths = [
"vfx_project_management.db", # Primary database
"database.db",
"../vfx_project_management.db"
]
for path in possible_paths:
if Path(path).exists():
return path
# If no existing database found, use the default name
return "vfx_project_management.db"
def check_column_exists(cursor, table_name, column_name):
"""Check if a column exists in a table."""
cursor.execute(f"PRAGMA table_info({table_name})")
columns = [column[1] for column in cursor.fetchall()]
return column_name in columns
def migrate_database():
"""Add technical specifications columns to the projects table."""
db_path = get_database_path()
print(f"Using database: {db_path}")
try:
# Connect to the database
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Check if projects table exists
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='projects'")
if not cursor.fetchone():
print("Projects table not found. Creating new database schema...")
conn.close()
return
print("Adding technical specifications columns to projects table...")
# Add technical specification columns if they don't exist
columns_to_add = [
("frame_rate", "REAL"),
("data_drive_path", "VARCHAR"),
("publish_storage_path", "VARCHAR"),
("delivery_image_resolution", "VARCHAR"),
("delivery_movie_specs_by_department", "JSON")
]
for column_name, column_type in columns_to_add:
if not check_column_exists(cursor, "projects", column_name):
print(f"Adding column: {column_name}")
cursor.execute(f"ALTER TABLE projects ADD COLUMN {column_name} {column_type}")
else:
print(f"Column {column_name} already exists, skipping...")
# Set default values for existing projects
print("Setting default technical specifications for existing projects...")
# Set default frame rate and image resolution
cursor.execute("""
UPDATE projects
SET frame_rate = 24.0,
delivery_image_resolution = '1920x1080'
WHERE frame_rate IS NULL OR delivery_image_resolution IS NULL
""")
# Set default delivery movie specs for projects that don't have them
default_specs_json = json.dumps(DEFAULT_DELIVERY_MOVIE_SPECS)
cursor.execute("""
UPDATE projects
SET delivery_movie_specs_by_department = ?
WHERE delivery_movie_specs_by_department IS NULL
""", (default_specs_json,))
# Commit changes
conn.commit()
# Verify the migration
cursor.execute("SELECT COUNT(*) FROM projects")
project_count = cursor.fetchone()[0]
print(f"Migration completed successfully! Updated {project_count} projects.")
# Show sample of updated data
cursor.execute("""
SELECT id, name, frame_rate, delivery_image_resolution
FROM projects
LIMIT 3
""")
print("\nSample of updated projects:")
for row in cursor.fetchall():
print(f" Project {row[0]}: {row[1]} - {row[2]} fps, {row[3]} resolution")
except sqlite3.Error as e:
print(f"Database error: {e}")
if conn:
conn.rollback()
sys.exit(1)
except Exception as e:
print(f"Unexpected error: {e}")
if conn:
conn.rollback()
sys.exit(1)
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("VFX Project Management - Technical Specifications Migration")
print("=" * 60)
migrate_database()
print("\nMigration completed successfully!")
print("Technical specifications have been added to all projects.")
backend/migrate_project_thumbnail.py
+43
View File
@@ -0,0 +1,43 @@
"""
Migration script to add thumbnail_path field to projects table.
"""
import sqlite3
from pathlib import Path
# Database path
DB_PATH = Path(__file__).parent / "vfx_project_management.db"
def migrate():
"""Add thumbnail_path column to projects table."""
conn = sqlite3.connect(DB_PATH)
cursor = conn.cursor()
try:
# Check if column already exists
cursor.execute("PRAGMA table_info(projects)")
columns = [column[1] for column in cursor.fetchall()]
if "thumbnail_path" not in columns:
print("Adding thumbnail_path column to projects table...")
cursor.execute("""
ALTER TABLE projects
ADD COLUMN thumbnail_path VARCHAR
""")
conn.commit()
print("✓ Successfully added thumbnail_path column")
else:
print("✓ thumbnail_path column already exists")
except Exception as e:
print(f"✗ Error during migration: {e}")
conn.rollback()
raise
finally:
conn.close()
if __name__ == "__main__":
print("Starting project thumbnail migration...")
migrate()
print("Migration completed!")
backend/migrate_shot_project_id.py
+524
View File
@@ -0,0 +1,524 @@
#!/usr/bin/env python3
"""
Migration script to add project_id column to shots table.
This script adds the following to the shots table:
- project_id column with foreign key constraint to projects table
- Index on project_id for performance optimization
- Populates project_id for existing shots based on episode relationships
- Unique constraint for project-scoped shot names (excluding soft-deleted shots)
Requirements: 2.1, 2.2, 2.3, 2.4, 2.5
Usage:
python migrate_shot_project_id.py
"""
import sqlite3
import sys
from pathlib import Path
import logging
# Setup logging
logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s')
logger = logging.getLogger(__name__)
def get_database_path():
"""Get the database path, trying multiple possible locations."""
possible_paths = [
"vfx_project_management.db", # Primary database
"database.db",
"../vfx_project_management.db"
]
for path in possible_paths:
if Path(path).exists():
return path
# If no existing database found, use the default name
return "vfx_project_management.db"
def check_column_exists(cursor, table_name, column_name):
"""Check if a column exists in a table."""
cursor.execute(f"PRAGMA table_info({table_name})")
columns = [column[1] for column in cursor.fetchall()]
return column_name in columns
def check_index_exists(cursor, index_name):
"""Check if an index exists."""
cursor.execute("SELECT name FROM sqlite_master WHERE type='index' AND name=?", (index_name,))
return cursor.fetchone() is not None
def check_constraint_exists(cursor, table_name, constraint_name):
"""Check if a constraint exists by examining the table schema."""
cursor.execute("SELECT sql FROM sqlite_master WHERE type='table' AND name=?", (table_name,))
result = cursor.fetchone()
if result:
return constraint_name in result[0]
return False
def validate_data_integrity(cursor):
"""Validate that all episodes have valid project references."""
logger.info("Validating data integrity...")
# Check for orphaned episodes (episodes without valid project references)
cursor.execute("""
SELECT e.id, e.name, e.project_id
FROM episodes e
LEFT JOIN projects p ON e.project_id = p.id
WHERE p.id IS NULL
""")
orphaned_episodes = cursor.fetchall()
if orphaned_episodes:
logger.error(f"Found {len(orphaned_episodes)} orphaned episodes:")
for episode in orphaned_episodes:
logger.error(f" Episode {episode[0]}: {episode[1]} -> Invalid project_id: {episode[2]}")
return False
# Check for shots with episodes that don't have project references
cursor.execute("""
SELECT s.id, s.name, s.episode_id, e.name as episode_name, e.project_id
FROM shots s
JOIN episodes e ON s.episode_id = e.id
LEFT JOIN projects p ON e.project_id = p.id
WHERE p.id IS NULL
""")
shots_with_invalid_episodes = cursor.fetchall()
if shots_with_invalid_episodes:
logger.error(f"Found {len(shots_with_invalid_episodes)} shots with invalid episode references:")
for shot in shots_with_invalid_episodes:
logger.error(f" Shot {shot[0]}: {shot[1]} -> Episode {shot[2]} ({shot[3]}) -> Invalid project_id: {shot[4]}")
return False
logger.info("Data integrity validation passed")
return True
def check_name_conflicts(cursor):
"""Check for potential project-scoped name conflicts."""
logger.info("Checking for potential project-scoped name conflicts...")
cursor.execute("""
SELECT p.id, p.name as project_name, s.name as shot_name, COUNT(*) as count
FROM shots s
JOIN episodes e ON s.episode_id = e.id
JOIN projects p ON e.project_id = p.id
WHERE s.deleted_at IS NULL
GROUP BY p.id, s.name
HAVING COUNT(*) > 1
ORDER BY p.id, s.name
""")
conflicts = cursor.fetchall()
if conflicts:
logger.warning(f"Found {len(conflicts)} potential name conflicts:")
for conflict in conflicts:
logger.warning(f" Project {conflict[0]} ({conflict[1]}): Shot name '{conflict[2]}' appears {conflict[3]} times")
# Show detailed conflict information
for conflict in conflicts:
project_id, project_name, shot_name, count = conflict
cursor.execute("""
SELECT s.id, s.name, e.id as episode_id, e.name as episode_name
FROM shots s
JOIN episodes e ON s.episode_id = e.id
WHERE e.project_id = ? AND s.name = ? AND s.deleted_at IS NULL
ORDER BY s.id
""", (project_id, shot_name))
conflicting_shots = cursor.fetchall()
logger.warning(f" Conflicting shots in project {project_name}:")
for shot in conflicting_shots:
logger.warning(f" Shot {shot[0]}: {shot[1]} in Episode {shot[2]} ({shot[3]})")
return False
logger.info("No project-scoped name conflicts found")
return True
def migrate_database():
"""Add project_id column to shots table and set up constraints."""
db_path = get_database_path()
logger.info(f"Using database: {db_path}")
try:
# Connect to the database
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Enable foreign key constraints
cursor.execute("PRAGMA foreign_keys = ON")
# Check if shots table exists
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='shots'")
if not cursor.fetchone():
logger.error("Shots table not found. Database may not be initialized.")
return False
# Check if projects table exists
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='projects'")
if not cursor.fetchone():
logger.error("Projects table not found. Database may not be initialized.")
return False
# Check if episodes table exists
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='episodes'")
if not cursor.fetchone():
logger.error("Episodes table not found. Database may not be initialized.")
return False
# Validate data integrity before migration
if not validate_data_integrity(cursor):
logger.error("Data integrity validation failed. Please fix data issues before running migration.")
return False
# Check for name conflicts before adding unique constraint
if not check_name_conflicts(cursor):
logger.error("Name conflicts detected. Please resolve conflicts before running migration.")
logger.error("You may need to rename conflicting shots to ensure uniqueness within each project.")
return False
# Check if project_id column already exists
if check_column_exists(cursor, "shots", "project_id"):
logger.warning("project_id column already exists in shots table")
# Check if migration is already complete
cursor.execute("PRAGMA table_info(shots)")
columns = {col[1]: col for col in cursor.fetchall()}
project_id_col = columns.get('project_id')
if project_id_col and project_id_col[3] == 1: # NOT NULL constraint exists
logger.info("Migration appears to be already complete")
return True
else:
logger.info("project_id column exists but migration is incomplete, continuing...")
# Continue with migration to complete it
logger.info("Starting shots table project_id migration...")
# Step 1: Add project_id column (nullable initially for data population)
if not check_column_exists(cursor, "shots", "project_id"):
logger.info("Adding project_id column to shots table...")
cursor.execute("""
ALTER TABLE shots
ADD COLUMN project_id INTEGER
""")
else:
logger.info("project_id column already exists, skipping column creation...")
# Step 2: Populate project_id for existing shots based on episode relationships
logger.info("Populating project_id for existing shots...")
cursor.execute("""
UPDATE shots
SET project_id = (
SELECT e.project_id
FROM episodes e
WHERE e.id = shots.episode_id
)
""")
# Verify that all shots now have project_id
cursor.execute("SELECT COUNT(*) FROM shots WHERE project_id IS NULL")
null_project_count = cursor.fetchone()[0]
if null_project_count > 0:
logger.error(f"Failed to populate project_id for {null_project_count} shots")
return False
# Get count of updated shots
cursor.execute("SELECT COUNT(*) FROM shots WHERE project_id IS NOT NULL")
updated_count = cursor.fetchone()[0]
logger.info(f"Successfully populated project_id for {updated_count} shots")
# Step 3: Create a new table with the proper constraints
logger.info("Creating new shots table with proper constraints...")
# First, get the current table schema to preserve other columns
cursor.execute("PRAGMA table_info(shots)")
columns_info = cursor.fetchall()
# Create new table with project_id as NOT NULL and foreign key constraint
cursor.execute("""
CREATE TABLE shots_new (
id INTEGER PRIMARY KEY,
project_id INTEGER NOT NULL,
episode_id INTEGER NOT NULL,
name VARCHAR NOT NULL,
description VARCHAR,
frame_start INTEGER NOT NULL DEFAULT 1001,
frame_end INTEGER NOT NULL DEFAULT 1001,
status VARCHAR(11) NOT NULL DEFAULT 'not_started',
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
deleted_at TIMESTAMP,
deleted_by INTEGER,
FOREIGN KEY (project_id) REFERENCES projects(id),
FOREIGN KEY (episode_id) REFERENCES episodes(id),
FOREIGN KEY (deleted_by) REFERENCES users(id)
)
""")
# Copy data from old table to new table
logger.info("Copying data to new table...")
cursor.execute("""
INSERT INTO shots_new
SELECT id, project_id, episode_id, name, description, frame_start, frame_end,
status, created_at, updated_at, deleted_at, deleted_by
FROM shots
""")
# Verify data copy
cursor.execute("SELECT COUNT(*) FROM shots")
original_count = cursor.fetchone()[0]
cursor.execute("SELECT COUNT(*) FROM shots_new")
new_count = cursor.fetchone()[0]
if original_count != new_count:
logger.error(f"Data copy failed: original={original_count}, new={new_count}")
return False
logger.info(f"Successfully copied {new_count} shots to new table")
# Drop old table and rename new table
logger.info("Replacing old table with new table...")
cursor.execute("DROP TABLE shots")
cursor.execute("ALTER TABLE shots_new RENAME TO shots")
# Step 4: Create indexes for performance
logger.info("Creating indexes for performance optimization...")
# Index on project_id for filtering
cursor.execute("""
CREATE INDEX idx_shots_project_id ON shots(project_id)
""")
# Index on name for searching
cursor.execute("""
CREATE INDEX idx_shots_name ON shots(name)
""")
# Partial index for active (non-deleted) shots
cursor.execute("""
CREATE INDEX idx_shots_active ON shots(id) WHERE deleted_at IS NULL
""")
# Composite index for project + episode queries
cursor.execute("""
CREATE INDEX idx_shots_project_episode ON shots(project_id, episode_id)
""")
# Unique constraint for project-scoped shot names (excluding soft-deleted shots)
cursor.execute("""
CREATE UNIQUE INDEX idx_shots_project_name_unique
ON shots(project_id, name) WHERE deleted_at IS NULL
""")
logger.info("Successfully created performance indexes and unique constraint")
# Commit all changes
conn.commit()
# Step 5: Verify the migration
logger.info("Verifying migration...")
# Check that project_id column exists and is properly constrained
cursor.execute("PRAGMA table_info(shots)")
columns = {col[1]: col for col in cursor.fetchall()}
if 'project_id' not in columns:
logger.error("project_id column not found after migration")
return False
project_id_col = columns['project_id']
if project_id_col[3] != 1: # NOT NULL constraint
logger.error("project_id column is not properly constrained as NOT NULL")
return False
# Check foreign key constraints
cursor.execute("PRAGMA foreign_key_list(shots)")
foreign_keys = cursor.fetchall()
project_fk_found = False
for fk in foreign_keys:
if fk[2] == 'projects' and fk[3] == 'project_id':
project_fk_found = True
break
if not project_fk_found:
logger.error("Foreign key constraint to projects table not found")
return False
# Check unique constraint (implemented as partial unique index)
cursor.execute("PRAGMA index_list(shots)")
indexes = [index[1] for index in cursor.fetchall()]
if "idx_shots_project_name_unique" not in indexes:
logger.error("Unique constraint for project-scoped names not found")
return False
# Verify data integrity after migration
cursor.execute("""
SELECT s.id, s.name, s.project_id, e.project_id as episode_project_id
FROM shots s
JOIN episodes e ON s.episode_id = e.id
WHERE s.project_id != e.project_id
""")
inconsistent_data = cursor.fetchall()
if inconsistent_data:
logger.error(f"Found {len(inconsistent_data)} shots with inconsistent project_id")
return False
# Show migration summary
cursor.execute("SELECT COUNT(*) FROM shots")
total_shots = cursor.fetchone()[0]
cursor.execute("""
SELECT p.name, COUNT(s.id) as shot_count
FROM projects p
LEFT JOIN shots s ON p.id = s.project_id AND s.deleted_at IS NULL
GROUP BY p.id, p.name
ORDER BY shot_count DESC
""")
project_summary = cursor.fetchall()
logger.info("Migration completed successfully!")
logger.info(f"Total shots: {total_shots}")
logger.info("Shots per project:")
for project_name, shot_count in project_summary:
logger.info(f" {project_name}: {shot_count} shots")
return True
except sqlite3.Error as e:
logger.error(f"Database error: {e}")
if conn:
conn.rollback()
return False
except Exception as e:
logger.error(f"Unexpected error: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
def verify_migration():
"""Verify that the migration was successful."""
db_path = get_database_path()
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
logger.info("Performing post-migration verification...")
# Check table structure
cursor.execute("PRAGMA table_info(shots)")
columns = {col[1]: col for col in cursor.fetchall()}
# Verify project_id column exists and is NOT NULL
if 'project_id' not in columns:
logger.error("❌ project_id column not found")
return False
if columns['project_id'][3] != 1: # NOT NULL constraint
logger.error("❌ project_id column is not NOT NULL")
return False
logger.info("✅ project_id column exists and is properly constrained")
# Check foreign key constraints
cursor.execute("PRAGMA foreign_key_list(shots)")
foreign_keys = cursor.fetchall()
project_fk_found = False
for fk in foreign_keys:
if fk[2] == 'projects' and fk[3] == 'project_id':
project_fk_found = True
break
if not project_fk_found:
logger.error("❌ Foreign key constraint to projects table not found")
return False
logger.info("✅ Foreign key constraint to projects table verified")
# Check indexes
cursor.execute("PRAGMA index_list(shots)")
indexes = [index[1] for index in cursor.fetchall()]
required_indexes = ['idx_shots_project_id', 'idx_shots_name', 'idx_shots_active', 'idx_shots_project_episode', 'idx_shots_project_name_unique']
for index_name in required_indexes:
if index_name not in indexes:
logger.error(f"❌ Index {index_name} not found")
return False
logger.info("✅ All required indexes verified")
# Check unique constraint (implemented as partial unique index)
if "idx_shots_project_name_unique" not in indexes:
logger.error("❌ Unique constraint for project-scoped names not found")
return False
logger.info("✅ Unique constraint for project-scoped names verified")
# Verify data consistency
cursor.execute("""
SELECT COUNT(*) FROM shots s
JOIN episodes e ON s.episode_id = e.id
WHERE s.project_id = e.project_id
""")
consistent_count = cursor.fetchone()[0]
cursor.execute("SELECT COUNT(*) FROM shots")
total_count = cursor.fetchone()[0]
if consistent_count != total_count:
logger.error(f"❌ Data consistency check failed: {consistent_count}/{total_count} shots have consistent project_id")
return False
logger.info(f"✅ Data consistency verified: {consistent_count} shots have consistent project_id")
return True
except sqlite3.Error as e:
logger.error(f"Verification error: {e}")
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("VFX Project Management - Shot Project ID Migration")
print("=" * 60)
success = migrate_database()
if success:
print("\n" + "=" * 60)
print("VERIFYING MIGRATION")
print("=" * 60)
if verify_migration():
print("\n✅ SUCCESS: Migration completed successfully!")
print("✅ All shots now have project_id with proper constraints and indexes")
sys.exit(0)
else:
print("\n❌ FAILED: Migration verification failed!")
sys.exit(1)
else:
print("\n❌ FAILED: Migration failed!")
sys.exit(1)
backend/migrate_soft_deletion_assets.py
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""
Migration script to add soft deletion columns to assets table.
Adds deleted_at and deleted_by columns and creates partial index for efficient querying.
"""
import sqlite3
import sys
from pathlib import Path
def migrate_assets_soft_deletion():
"""Add soft deletion columns to assets table and create partial index."""
# Database path
db_path = Path(__file__).parent / "vfx_project_management.db"
if not db_path.exists():
print(f"Database file not found at {db_path}")
return False
try:
# Connect to database
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("Starting assets table soft deletion migration...")
# Check if columns already exist
cursor.execute("PRAGMA table_info(assets)")
columns = [column[1] for column in cursor.fetchall()]
if 'deleted_at' in columns:
print("Soft deletion columns already exist in assets table")
return True
# Add deleted_at column
print("Adding deleted_at column to assets table...")
cursor.execute("""
ALTER TABLE assets
ADD COLUMN deleted_at TIMESTAMP NULL
""")
# Add deleted_by column
print("Adding deleted_by column to assets table...")
cursor.execute("""
ALTER TABLE assets
ADD COLUMN deleted_by INTEGER NULL
REFERENCES users(id)
""")
# Create partial index for efficient querying of non-deleted records
print("Creating partial index idx_assets_not_deleted...")
cursor.execute("""
CREATE INDEX idx_assets_not_deleted
ON assets (id)
WHERE deleted_at IS NULL
""")
# Commit changes
conn.commit()
print("SUCCESS: Successfully added soft deletion columns to assets table")
print("SUCCESS: Created partial index idx_assets_not_deleted")
return True
except sqlite3.Error as e:
print(f"Database error: {e}")
if conn:
conn.rollback()
return False
except Exception as e:
print(f"Unexpected error: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
def verify_migration():
"""Verify that the migration was successful."""
db_path = Path(__file__).parent / "vfx_project_management.db"
try:
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
# Check table structure
cursor.execute("PRAGMA table_info(assets)")
columns = {column[1]: column[2] for column in cursor.fetchall()}
# Verify columns exist
if 'deleted_at' not in columns:
print("❌ deleted_at column not found")
return False
if 'deleted_by' not in columns:
print("❌ deleted_by column not found")
return False
print("SUCCESS: Soft deletion columns verified in assets table")
# Check index exists
cursor.execute("PRAGMA index_list(assets)")
indexes = [index[1] for index in cursor.fetchall()]
if 'idx_assets_not_deleted' not in indexes:
print("❌ Partial index idx_assets_not_deleted not found")
return False
print("SUCCESS: Partial index idx_assets_not_deleted verified")
return True
except sqlite3.Error as e:
print(f"Verification error: {e}")
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("=== Assets Table Soft Deletion Migration ===")
success = migrate_assets_soft_deletion()
if success:
print("\n=== Verifying Migration ===")
if verify_migration():
print("\nSUCCESS: Migration completed successfully!")
sys.exit(0)
else:
print("\n❌ Migration verification failed!")
sys.exit(1)
else:
print("\n❌ Migration failed!")
sys.exit(1)
backend/migrate_soft_deletion_complete.py
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env python3
"""
Complete migration script for soft deletion functionality.
Runs all individual migration scripts in the correct order and verifies the results.
"""
import subprocess
import sys
from pathlib import Path
def run_migration_script(script_name):
"""Run a migration script and return success status."""
script_path = Path(__file__).parent / script_name
if not script_path.exists():
print(f"ERROR: Migration script {script_name} not found")
return False
try:
print(f"\n{'='*50}")
print(f"Running {script_name}")
print(f"{'='*50}")
result = subprocess.run([sys.executable, str(script_path)],
capture_output=True, text=True, cwd=Path(__file__).parent)
print(result.stdout)
if result.stderr:
print("STDERR:", result.stderr)
if result.returncode == 0:
print(f"SUCCESS: {script_name} completed successfully")
return True
else:
print(f"ERROR: {script_name} failed with exit code {result.returncode}")
return False
except Exception as e:
print(f"ERROR: Error running {script_name}: {e}")
return False
def main():
"""Run all soft deletion migration scripts."""
print("=== Complete Soft Deletion Migration ===")
print("This will add soft deletion columns to all relevant tables")
# Migration scripts in order
migration_scripts = [
"migrate_soft_deletion_shots.py",
"migrate_soft_deletion_assets.py",
"migrate_soft_deletion_tasks.py",
"migrate_soft_deletion_related_tables.py"
]
all_successful = True
for script in migration_scripts:
success = run_migration_script(script)
if not success:
all_successful = False
print(f"\nERROR: Migration failed at {script}")
break
if all_successful:
print(f"\n{'='*50}")
print("SUCCESS: ALL MIGRATIONS COMPLETED SUCCESSFULLY!")
print("SUCCESS: Soft deletion columns added to all tables:")
print(" - shots (deleted_at, deleted_by)")
print(" - assets (deleted_at, deleted_by)")
print(" - tasks (deleted_at, deleted_by)")
print(" - submissions (deleted_at, deleted_by)")
print(" - task_attachments (deleted_at, deleted_by)")
print(" - production_notes (deleted_at, deleted_by)")
print(" - reviews (deleted_at, deleted_by)")
print("SUCCESS: Partial indexes created for efficient querying")
print(f"{'='*50}")
return True
else:
print(f"\n{'='*50}")
print("ERROR: MIGRATION FAILED!")
print("Please check the error messages above and fix any issues.")
print(f"{'='*50}")
return False
if __name__ == "__main__":
success = main()
sys.exit(0 if success else 1)
backend/migrate_soft_deletion_related_tables.py
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env python3
"""
Migration script to add soft deletion columns to related tables.
Adds deleted_at and deleted_by columns to submissions, task_attachments, production_notes, and reviews tables.
Creates appropriate partial indexes for each table.
"""
import sqlite3
import sys
from pathlib import Path
def migrate_related_tables_soft_deletion():
"""Add soft deletion columns to related tables and create partial indexes."""
# Database path
db_path = Path(__file__).parent / "vfx_project_management.db"
if not db_path.exists():
print(f"Database file not found at {db_path}")
return False
try:
# Connect to database
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("Starting related tables soft deletion migration...")
# Tables to migrate
tables = [
("submissions", "task_id"),
("task_attachments", "task_id"),
("production_notes", "task_id"),
("reviews", "submission_id")
]
for table_name, index_column in tables:
print(f"\n--- Migrating {table_name} table ---")
# Check if columns already exist
cursor.execute(f"PRAGMA table_info({table_name})")
columns = [column[1] for column in cursor.fetchall()]
if 'deleted_at' in columns:
print(f"Soft deletion columns already exist in {table_name} table")
continue
# Add deleted_at column
print(f"Adding deleted_at column to {table_name} table...")
cursor.execute(f"""
ALTER TABLE {table_name}
ADD COLUMN deleted_at TIMESTAMP NULL
""")
# Add deleted_by column
print(f"Adding deleted_by column to {table_name} table...")
cursor.execute(f"""
ALTER TABLE {table_name}
ADD COLUMN deleted_by INTEGER NULL
REFERENCES users(id)
""")
# Create partial index for efficient querying of non-deleted records
index_name = f"idx_{table_name}_not_deleted"
print(f"Creating partial index {index_name}...")
cursor.execute(f"""
CREATE INDEX {index_name}
ON {table_name} ({index_column})
WHERE deleted_at IS NULL
""")
print(f"SUCCESS: Successfully migrated {table_name} table")
# Commit changes
conn.commit()
print("\nSUCCESS: Successfully added soft deletion columns to all related tables")
return True
except sqlite3.Error as e:
print(f"Database error: {e}")
if conn:
conn.rollback()
return False
except Exception as e:
print(f"Unexpected error: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
def verify_migration():
"""Verify that the migration was successful."""
db_path = Path(__file__).parent / "vfx_project_management.db"
try:
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
# Tables to verify
tables = [
("submissions", "idx_submissions_not_deleted"),
("task_attachments", "idx_task_attachments_not_deleted"),
("production_notes", "idx_production_notes_not_deleted"),
("reviews", "idx_reviews_not_deleted")
]
all_verified = True
for table_name, index_name in tables:
print(f"\n--- Verifying {table_name} table ---")
# Check table structure
cursor.execute(f"PRAGMA table_info({table_name})")
columns = {column[1]: column[2] for column in cursor.fetchall()}
# Verify columns exist
if 'deleted_at' not in columns:
print(f"❌ deleted_at column not found in {table_name}")
all_verified = False
continue
if 'deleted_by' not in columns:
print(f"❌ deleted_by column not found in {table_name}")
all_verified = False
continue
print(f"SUCCESS: Soft deletion columns verified in {table_name} table")
# Check index exists
cursor.execute(f"PRAGMA index_list({table_name})")
indexes = [index[1] for index in cursor.fetchall()]
if index_name not in indexes:
print(f"❌ Partial index {index_name} not found")
all_verified = False
continue
print(f"SUCCESS: Partial index {index_name} verified")
return all_verified
except sqlite3.Error as e:
print(f"Verification error: {e}")
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("=== Related Tables Soft Deletion Migration ===")
success = migrate_related_tables_soft_deletion()
if success:
print("\n=== Verifying Migration ===")
if verify_migration():
print("\nSUCCESS: Migration completed successfully!")
sys.exit(0)
else:
print("\n❌ Migration verification failed!")
sys.exit(1)
else:
print("\n❌ Migration failed!")
sys.exit(1)
backend/migrate_soft_deletion_shots.py
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""
Migration script to add soft deletion columns to shots table.
Adds deleted_at and deleted_by columns and creates partial index for efficient querying.
"""
import sqlite3
import sys
from pathlib import Path
def migrate_shots_soft_deletion():
"""Add soft deletion columns to shots table and create partial index."""
# Database path
db_path = Path(__file__).parent / "vfx_project_management.db"
if not db_path.exists():
print(f"Database file not found at {db_path}")
return False
try:
# Connect to database
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("Starting shots table soft deletion migration...")
# Check if columns already exist
cursor.execute("PRAGMA table_info(shots)")
columns = [column[1] for column in cursor.fetchall()]
if 'deleted_at' in columns:
print("Soft deletion columns already exist in shots table")
return True
# Add deleted_at column
print("Adding deleted_at column to shots table...")
cursor.execute("""
ALTER TABLE shots
ADD COLUMN deleted_at TIMESTAMP NULL
""")
# Add deleted_by column
print("Adding deleted_by column to shots table...")
cursor.execute("""
ALTER TABLE shots
ADD COLUMN deleted_by INTEGER NULL
REFERENCES users(id)
""")
# Create partial index for efficient querying of non-deleted records
print("Creating partial index idx_shots_not_deleted...")
cursor.execute("""
CREATE INDEX idx_shots_not_deleted
ON shots (id)
WHERE deleted_at IS NULL
""")
# Commit changes
conn.commit()
print("SUCCESS: Successfully added soft deletion columns to shots table")
print("SUCCESS: Created partial index idx_shots_not_deleted")
return True
except sqlite3.Error as e:
print(f"Database error: {e}")
if conn:
conn.rollback()
return False
except Exception as e:
print(f"Unexpected error: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
def verify_migration():
"""Verify that the migration was successful."""
db_path = Path(__file__).parent / "vfx_project_management.db"
try:
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
# Check table structure
cursor.execute("PRAGMA table_info(shots)")
columns = {column[1]: column[2] for column in cursor.fetchall()}
# Verify columns exist
if 'deleted_at' not in columns:
print("❌ deleted_at column not found")
return False
if 'deleted_by' not in columns:
print("❌ deleted_by column not found")
return False
print("SUCCESS: Soft deletion columns verified in shots table")
# Check index exists
cursor.execute("PRAGMA index_list(shots)")
indexes = [index[1] for index in cursor.fetchall()]
if 'idx_shots_not_deleted' not in indexes:
print("❌ Partial index idx_shots_not_deleted not found")
return False
print("SUCCESS: Partial index idx_shots_not_deleted verified")
return True
except sqlite3.Error as e:
print(f"Verification error: {e}")
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("=== Shots Table Soft Deletion Migration ===")
success = migrate_shots_soft_deletion()
if success:
print("\n=== Verifying Migration ===")
if verify_migration():
print("\nSUCCESS: Migration completed successfully!")
sys.exit(0)
else:
print("\n❌ Migration verification failed!")
sys.exit(1)
else:
print("\n❌ Migration failed!")
sys.exit(1)
backend/migrate_soft_deletion_tasks.py
+138
View File
@@ -0,0 +1,138 @@
#!/usr/bin/env python3
"""
Migration script to add soft deletion columns to tasks table.
Adds deleted_at and deleted_by columns and creates partial index for efficient querying.
"""
import sqlite3
import sys
from pathlib import Path
def migrate_tasks_soft_deletion():
"""Add soft deletion columns to tasks table and create partial index."""
# Database path
db_path = Path(__file__).parent / "vfx_project_management.db"
if not db_path.exists():
print(f"Database file not found at {db_path}")
return False
try:
# Connect to database
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("Starting tasks table soft deletion migration...")
# Check if columns already exist
cursor.execute("PRAGMA table_info(tasks)")
columns = [column[1] for column in cursor.fetchall()]
if 'deleted_at' in columns:
print("Soft deletion columns already exist in tasks table")
return True
# Add deleted_at column
print("Adding deleted_at column to tasks table...")
cursor.execute("""
ALTER TABLE tasks
ADD COLUMN deleted_at TIMESTAMP NULL
""")
# Add deleted_by column
print("Adding deleted_by column to tasks table...")
cursor.execute("""
ALTER TABLE tasks
ADD COLUMN deleted_by INTEGER NULL
REFERENCES users(id)
""")
# Create partial index for efficient querying of non-deleted records
# Index on shot_id and asset_id for efficient filtering
print("Creating partial index idx_tasks_not_deleted...")
cursor.execute("""
CREATE INDEX idx_tasks_not_deleted
ON tasks (shot_id, asset_id)
WHERE deleted_at IS NULL
""")
# Commit changes
conn.commit()
print("SUCCESS: Successfully added soft deletion columns to tasks table")
print("SUCCESS: Created partial index idx_tasks_not_deleted")
return True
except sqlite3.Error as e:
print(f"Database error: {e}")
if conn:
conn.rollback()
return False
except Exception as e:
print(f"Unexpected error: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
def verify_migration():
"""Verify that the migration was successful."""
db_path = Path(__file__).parent / "vfx_project_management.db"
try:
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
# Check table structure
cursor.execute("PRAGMA table_info(tasks)")
columns = {column[1]: column[2] for column in cursor.fetchall()}
# Verify columns exist
if 'deleted_at' not in columns:
print("❌ deleted_at column not found")
return False
if 'deleted_by' not in columns:
print("❌ deleted_by column not found")
return False
print("SUCCESS: Soft deletion columns verified in tasks table")
# Check index exists
cursor.execute("PRAGMA index_list(tasks)")
indexes = [index[1] for index in cursor.fetchall()]
if 'idx_tasks_not_deleted' not in indexes:
print("❌ Partial index idx_tasks_not_deleted not found")
return False
print("SUCCESS: Partial index idx_tasks_not_deleted verified")
return True
except sqlite3.Error as e:
print(f"Verification error: {e}")
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("=== Tasks Table Soft Deletion Migration ===")
success = migrate_tasks_soft_deletion()
if success:
print("\n=== Verifying Migration ===")
if verify_migration():
print("\nSUCCESS: Migration completed successfully!")
sys.exit(0)
else:
print("\n❌ Migration verification failed!")
sys.exit(1)
else:
print("\n❌ Migration failed!")
sys.exit(1)
backend/models/__init__.py
+39
View File
@@ -0,0 +1,39 @@
# Models package
from .user import User, UserRole, DepartmentRole
from .project import Project, ProjectMember, ProjectStatus
from .episode import Episode, EpisodeStatus
from .asset import Asset, AssetCategory, AssetStatus
from .shot import Shot, ShotStatus
from .task import (
Task, Submission, Review, ProductionNote, TaskAttachment,
TaskType, TaskStatus, ReviewDecision, AttachmentType
)
from .api_key import APIKey, APIKeyScope
from .api_key_usage import APIKeyUsage
from .global_settings import GlobalSettings
from .notification import Notification, UserNotificationPreference, NotificationType
from .activity import Activity, ActivityType
__all__ = [
# User models
"User", "UserRole", "DepartmentRole",
# Project models
"Project", "ProjectMember", "ProjectStatus",
# Episode models
"Episode", "EpisodeStatus",
# Asset models
"Asset", "AssetCategory", "AssetStatus",
# Shot models
"Shot", "ShotStatus",
# Task models
"Task", "Submission", "Review", "ProductionNote", "TaskAttachment",
"TaskType", "TaskStatus", "ReviewDecision", "AttachmentType",
# API Key models
"APIKey", "APIKeyScope", "APIKeyUsage",
# Global Settings models
"GlobalSettings",
# Notification models
"Notification", "UserNotificationPreference", "NotificationType",
# Activity models
"Activity", "ActivityType"
]
backend/models/activity.py
+55
View File
@@ -0,0 +1,55 @@
from sqlalchemy import Column, Integer, String, DateTime, ForeignKey, Text, Enum as SQLEnum, JSON
from sqlalchemy.orm import relationship
from database import Base
from datetime import datetime
import enum
class ActivityType(str, enum.Enum):
TASK_CREATED = "task_created"
TASK_UPDATED = "task_updated"
TASK_ASSIGNED = "task_assigned"
TASK_STATUS_CHANGED = "task_status_changed"
SUBMISSION_CREATED = "submission_created"
SUBMISSION_REVIEWED = "submission_reviewed"
COMMENT_ADDED = "comment_added"
ASSET_CREATED = "asset_created"
ASSET_UPDATED = "asset_updated"
SHOT_CREATED = "shot_created"
SHOT_UPDATED = "shot_updated"
PROJECT_CREATED = "project_created"
PROJECT_UPDATED = "project_updated"
USER_JOINED_PROJECT = "user_joined_project"
SHOT_DELETED = "shot_deleted"
ASSET_DELETED = "asset_deleted"
SHOT_RECOVERED = "shot_recovered"
ASSET_RECOVERED = "asset_recovered"
class Activity(Base):
__tablename__ = "activities"
id = Column(Integer, primary_key=True, index=True)
type = Column(SQLEnum(ActivityType), nullable=False, index=True)
user_id = Column(Integer, ForeignKey("users.id"), nullable=False, index=True)
# Optional references to related entities
project_id = Column(Integer, ForeignKey("projects.id"), nullable=True, index=True)
task_id = Column(Integer, ForeignKey("tasks.id"), nullable=True)
asset_id = Column(Integer, ForeignKey("assets.id"), nullable=True)
shot_id = Column(Integer, ForeignKey("shots.id"), nullable=True)
submission_id = Column(Integer, ForeignKey("submissions.id"), nullable=True)
# Activity details
description = Column(Text, nullable=False)
activity_metadata = Column(JSON, nullable=True) # Additional context data
created_at = Column(DateTime, default=datetime.utcnow, nullable=False, index=True)
# Relationships
user = relationship("User", foreign_keys=[user_id])
project = relationship("Project", foreign_keys=[project_id])
task = relationship("Task", foreign_keys=[task_id])
asset = relationship("Asset", foreign_keys=[asset_id])
shot = relationship("Shot", foreign_keys=[shot_id])
submission = relationship("Submission", foreign_keys=[submission_id])
backend/models/api_key.py
+36
View File
@@ -0,0 +1,36 @@
from sqlalchemy import Column, Integer, String, Boolean, DateTime, ForeignKey, Text
from sqlalchemy.orm import relationship
from sqlalchemy.sql import func
from database import Base
import enum
class APIKeyScope(str, enum.Enum):
READ_PROJECTS = "read:projects"
READ_TASKS = "read:tasks"
READ_SUBMISSIONS = "read:submissions"
READ_USERS = "read:users"
WRITE_TASKS = "write:tasks"
WRITE_SUBMISSIONS = "write:submissions"
ADMIN_USERS = "admin:users"
FULL_ACCESS = "full:access"
class APIKey(Base):
__tablename__ = "api_keys"
id = Column(Integer, primary_key=True, index=True)
user_id = Column(Integer, ForeignKey("users.id"), nullable=False)
key_hash = Column(String, unique=True, index=True, nullable=False)
name = Column(String, nullable=False)
scopes = Column(Text, nullable=False) # JSON string of scopes
is_active = Column(Boolean, default=True, nullable=False)
expires_at = Column(DateTime(timezone=True), nullable=True)
last_used_at = Column(DateTime(timezone=True), nullable=True)
created_at = Column(DateTime(timezone=True), server_default=func.now())
# Relationships
user = relationship("User", back_populates="api_keys")
def __repr__(self):
return f"<APIKey(id={self.id}, name='{self.name}', user_id={self.user_id})>"
backend/models/api_key_usage.py
+22
View File
@@ -0,0 +1,22 @@
from sqlalchemy import Column, Integer, String, DateTime, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.sql import func
from database import Base
class APIKeyUsage(Base):
__tablename__ = "api_key_usage"
id = Column(Integer, primary_key=True, index=True)
api_key_id = Column(Integer, ForeignKey("api_keys.id"), nullable=False)
endpoint = Column(String, nullable=False)
method = Column(String, nullable=False)
timestamp = Column(DateTime(timezone=True), server_default=func.now())
ip_address = Column(String, nullable=True)
user_agent = Column(String, nullable=True)
# Relationships
api_key = relationship("APIKey")
def __repr__(self):
return f"<APIKeyUsage(id={self.id}, api_key_id={self.api_key_id}, endpoint='{self.endpoint}')>"
backend/models/asset.py
+65
View File
@@ -0,0 +1,65 @@
from sqlalchemy import Column, Integer, String, DateTime, ForeignKey, Enum
from sqlalchemy.orm import relationship, Query
from sqlalchemy.sql import func
from database import Base
import enum
class AssetCategory(str, enum.Enum):
CHARACTERS = "characters"
PROPS = "props"
SETS = "sets"
VEHICLES = "vehicles"
class AssetStatus(str, enum.Enum):
NOT_STARTED = "not_started"
IN_PROGRESS = "in_progress"
ON_HOLD = "on_hold"
COMPLETED = "completed"
APPROVED = "approved"
class Asset(Base):
__tablename__ = "assets"
id = Column(Integer, primary_key=True, index=True)
project_id = Column(Integer, ForeignKey("projects.id"), nullable=False)
name = Column(String, nullable=False, index=True)
description = Column(String)
category = Column(Enum(AssetCategory), nullable=False)
status = Column(Enum(AssetStatus), nullable=False, default=AssetStatus.NOT_STARTED)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
# Soft deletion columns
deleted_at = Column(DateTime(timezone=True), nullable=True)
deleted_by = Column(Integer, ForeignKey("users.id"), nullable=True)
# Relationships
project = relationship("Project", back_populates="assets")
tasks = relationship("Task", back_populates="asset", cascade="all, delete-orphan")
deleted_by_user = relationship("User", foreign_keys=[deleted_by])
@property
def is_deleted(self) -> bool:
"""Check if the asset is soft deleted."""
return self.deleted_at is not None
@classmethod
def query_active(cls, query: Query) -> Query:
"""Filter query to exclude soft deleted assets."""
return query.filter(cls.deleted_at.is_(None))
@classmethod
def query_deleted(cls, query: Query) -> Query:
"""Filter query to include only soft deleted assets."""
return query.filter(cls.deleted_at.isnot(None))
@classmethod
def query_all_including_deleted(cls, query: Query) -> Query:
"""Return query without soft deletion filtering (for admin use)."""
return query
def __repr__(self):
return f"<Asset(id={self.id}, name='{self.name}', category='{self.category}')>"
backend/models/episode.py
+34
View File
@@ -0,0 +1,34 @@
from sqlalchemy import Column, Integer, String, DateTime, ForeignKey, Enum
from sqlalchemy.orm import relationship
from sqlalchemy.sql import func
from database import Base
import enum
class EpisodeStatus(str, enum.Enum):
PLANNING = "planning"
IN_PROGRESS = "in_progress"
ON_HOLD = "on_hold"
COMPLETED = "completed"
CANCELLED = "cancelled"
class Episode(Base):
__tablename__ = "episodes"
id = Column(Integer, primary_key=True, index=True)
project_id = Column(Integer, ForeignKey("projects.id"), nullable=False)
name = Column(String, nullable=False, index=True)
description = Column(String)
episode_number = Column(Integer, nullable=False)
status = Column(Enum(EpisodeStatus), nullable=False, default=EpisodeStatus.PLANNING)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
# Relationships
project = relationship("Project", back_populates="episodes")
shots = relationship("Shot", back_populates="episode", cascade="all, delete-orphan")
tasks = relationship("Task", back_populates="episode", cascade="all, delete-orphan")
def __repr__(self):
return f"<Episode(id={self.id}, name='{self.name}', episode_number={self.episode_number})>"
backend/models/global_settings.py
+14
View File
@@ -0,0 +1,14 @@
from sqlalchemy import Column, Integer, String, DateTime, Text
from sqlalchemy.sql import func
from database import Base
class GlobalSettings(Base):
__tablename__ = "global_settings"
id = Column(Integer, primary_key=True, index=True)
setting_key = Column(String(100), unique=True, index=True, nullable=False)
setting_value = Column(Text, nullable=False)
description = Column(Text)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
backend/models/notification.py
+89
View File
@@ -0,0 +1,89 @@
from sqlalchemy import Column, Integer, String, Boolean, DateTime, ForeignKey, Text, Enum as SQLEnum
from sqlalchemy.orm import relationship
from database import Base
from datetime import datetime
import enum
class NotificationType(str, enum.Enum):
TASK_ASSIGNED = "task_assigned"
TASK_STATUS_CHANGED = "task_status_changed"
SUBMISSION_REVIEWED = "submission_reviewed"
WORK_SUBMITTED = "work_submitted"
DEADLINE_APPROACHING = "deadline_approaching"
PROJECT_UPDATE = "project_update"
COMMENT_ADDED = "comment_added"
class NotificationPriority(str, enum.Enum):
LOW = "low"
NORMAL = "normal"
HIGH = "high"
URGENT = "urgent"
class Notification(Base):
__tablename__ = "notifications"
id = Column(Integer, primary_key=True, index=True)
user_id = Column(Integer, ForeignKey("users.id"), nullable=False, index=True)
type = Column(SQLEnum(NotificationType), nullable=False)
priority = Column(SQLEnum(NotificationPriority), default=NotificationPriority.NORMAL)
title = Column(String(255), nullable=False)
message = Column(Text, nullable=False)
read = Column(Boolean, default=False, index=True)
# Optional references to related entities
project_id = Column(Integer, ForeignKey("projects.id"), nullable=True)
task_id = Column(Integer, ForeignKey("tasks.id"), nullable=True)
submission_id = Column(Integer, ForeignKey("submissions.id"), nullable=True)
# Email notification tracking
email_sent = Column(Boolean, default=False)
email_sent_at = Column(DateTime, nullable=True)
created_at = Column(DateTime, default=datetime.utcnow, nullable=False)
read_at = Column(DateTime, nullable=True)
# Relationships
user = relationship("User", back_populates="notifications")
project = relationship("Project", foreign_keys=[project_id])
task = relationship("Task", foreign_keys=[task_id])
submission = relationship("Submission", foreign_keys=[submission_id])
class UserNotificationPreference(Base):
__tablename__ = "user_notification_preferences"
id = Column(Integer, primary_key=True, index=True)
user_id = Column(Integer, ForeignKey("users.id"), nullable=False, unique=True)
# Email notification preferences
email_enabled = Column(Boolean, default=True)
email_task_assigned = Column(Boolean, default=True)
email_task_status_changed = Column(Boolean, default=True)
email_submission_reviewed = Column(Boolean, default=True)
email_work_submitted = Column(Boolean, default=True)
email_deadline_approaching = Column(Boolean, default=True)
email_project_update = Column(Boolean, default=True)
email_comment_added = Column(Boolean, default=True)
# In-app notification preferences
inapp_enabled = Column(Boolean, default=True)
inapp_task_assigned = Column(Boolean, default=True)
inapp_task_status_changed = Column(Boolean, default=True)
inapp_submission_reviewed = Column(Boolean, default=True)
inapp_work_submitted = Column(Boolean, default=True)
inapp_deadline_approaching = Column(Boolean, default=True)
inapp_project_update = Column(Boolean, default=True)
inapp_comment_added = Column(Boolean, default=True)
# Digest settings
email_digest_enabled = Column(Boolean, default=False)
email_digest_frequency = Column(String(50), default="daily") # daily, weekly
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
# Relationships
user = relationship("User", back_populates="notification_preferences")
backend/models/project.py
+88
View File
@@ -0,0 +1,88 @@
from sqlalchemy import Column, Integer, String, DateTime, Date, Enum, ForeignKey, Float, JSON
from sqlalchemy.orm import relationship
from sqlalchemy.sql import func
from database import Base
from .user import DepartmentRole
import enum
class ProjectStatus(str, enum.Enum):
PLANNING = "planning"
IN_PROGRESS = "in_progress"
ON_HOLD = "on_hold"
COMPLETED = "completed"
CANCELLED = "cancelled"
class ProjectType(str, enum.Enum):
TV = "tv"
CINEMA = "cinema"
GAME = "game"
class Project(Base):
__tablename__ = "projects"
id = Column(Integer, primary_key=True, index=True)
name = Column(String, nullable=False, index=True)
code_name = Column(String, nullable=False, unique=True, index=True) # Unique project code
client_name = Column(String, nullable=False, index=True) # Client/studio name
project_type = Column(Enum(ProjectType), nullable=False) # TV, Cinema, Game
description = Column(String)
status = Column(Enum(ProjectStatus), nullable=False, default=ProjectStatus.PLANNING)
start_date = Column(Date)
end_date = Column(Date)
# Technical specifications
frame_rate = Column(Float, nullable=True) # Frames per second (1-120 fps)
data_drive_path = Column(String, nullable=True) # Physical path for project data storage
publish_storage_path = Column(String, nullable=True) # Path for approved work delivery
delivery_image_resolution = Column(String, nullable=True) # Required image resolution
delivery_movie_specs_by_department = Column(JSON, nullable=True) # Delivery movie specs per department
# Project-specific settings
upload_data_location = Column(String, nullable=True) # Custom upload storage path
asset_task_templates = Column(JSON, nullable=True) # Default tasks per asset category
shot_task_templates = Column(JSON, nullable=True) # Default tasks for shots
enabled_asset_tasks = Column(JSON, nullable=True) # Enabled/disabled asset tasks per category
enabled_shot_tasks = Column(JSON, nullable=True) # Enabled/disabled shot tasks
# Custom task types
custom_asset_task_types = Column(JSON, nullable=True) # Custom task types for assets
custom_shot_task_types = Column(JSON, nullable=True) # Custom task types for shots
# Custom task statuses
custom_task_statuses = Column(JSON, nullable=True) # Custom task statuses for project
# Project thumbnail
thumbnail_path = Column(String, nullable=True) # Path to project thumbnail image
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
# Relationships
episodes = relationship("Episode", back_populates="project", cascade="all, delete-orphan")
shots = relationship("Shot", back_populates="project", cascade="all, delete-orphan")
assets = relationship("Asset", back_populates="project", cascade="all, delete-orphan")
project_members = relationship("ProjectMember", back_populates="project", cascade="all, delete-orphan")
tasks = relationship("Task", back_populates="project", cascade="all, delete-orphan")
def __repr__(self):
return f"<Project(id={self.id}, name='{self.name}', status='{self.status}')>"
class ProjectMember(Base):
__tablename__ = "project_members"
id = Column(Integer, primary_key=True, index=True)
user_id = Column(Integer, ForeignKey("users.id"), nullable=False)
project_id = Column(Integer, ForeignKey("projects.id"), nullable=False)
department_role = Column(Enum(DepartmentRole), nullable=True)
joined_at = Column(DateTime(timezone=True), server_default=func.now())
# Relationships
user = relationship("User", back_populates="project_memberships")
project = relationship("Project", back_populates="project_members")
def __repr__(self):
return f"<ProjectMember(user_id={self.user_id}, project_id={self.project_id}, role='{self.department_role}')>"
backend/models/shot.py
+66
View File
@@ -0,0 +1,66 @@
from sqlalchemy import Column, Integer, String, DateTime, ForeignKey, Enum, UniqueConstraint
from sqlalchemy.orm import relationship, Query
from sqlalchemy.sql import func
from database import Base
import enum
class ShotStatus(str, enum.Enum):
NOT_STARTED = "not_started"
IN_PROGRESS = "in_progress"
ON_HOLD = "on_hold"
COMPLETED = "completed"
APPROVED = "approved"
class Shot(Base):
__tablename__ = "shots"
id = Column(Integer, primary_key=True, index=True)
project_id = Column(Integer, ForeignKey("projects.id"), nullable=False, index=True)
episode_id = Column(Integer, ForeignKey("episodes.id"), nullable=False)
name = Column(String, nullable=False, index=True)
description = Column(String)
frame_start = Column(Integer, nullable=False, default=1001)
frame_end = Column(Integer, nullable=False, default=1001)
status = Column(Enum(ShotStatus), nullable=False, default=ShotStatus.NOT_STARTED)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
# Soft deletion columns
deleted_at = Column(DateTime(timezone=True), nullable=True)
deleted_by = Column(Integer, ForeignKey("users.id"), nullable=True)
# Relationships
project = relationship("Project", back_populates="shots")
episode = relationship("Episode", back_populates="shots")
tasks = relationship("Task", back_populates="shot", cascade="all, delete-orphan")
deleted_by_user = relationship("User", foreign_keys=[deleted_by])
# Constraints
__table_args__ = (
UniqueConstraint('project_id', 'name', name='uq_shot_project_name'),
)
@property
def is_deleted(self) -> bool:
"""Check if the shot is soft deleted."""
return self.deleted_at is not None
@classmethod
def query_active(cls, query: Query) -> Query:
"""Filter query to exclude soft deleted shots."""
return query.filter(cls.deleted_at.is_(None))
@classmethod
def query_deleted(cls, query: Query) -> Query:
"""Filter query to include only soft deleted shots."""
return query.filter(cls.deleted_at.isnot(None))
@classmethod
def query_all_including_deleted(cls, query: Query) -> Query:
"""Return query without soft deletion filtering (for admin use)."""
return query
def __repr__(self):
return f"<Shot(id={self.id}, name='{self.name}', frames={self.frame_start}-{self.frame_end})>"
backend/models/task.py
+276
View File
@@ -0,0 +1,276 @@
from sqlalchemy import Column, Integer, String, DateTime, Date, ForeignKey, Enum, Text
from sqlalchemy.orm import relationship, Query
from sqlalchemy.sql import func
from database import Base
import enum
class TaskType(str, enum.Enum):
# Shot tasks
LAYOUT = "layout"
ANIMATION = "animation"
SIMULATION = "simulation"
LIGHTING = "lighting"
COMPOSITING = "compositing"
# Asset tasks
MODELING = "modeling"
SURFACING = "surfacing"
RIGGING = "rigging"
class TaskStatus(str, enum.Enum):
NOT_STARTED = "not_started"
IN_PROGRESS = "in_progress"
SUBMITTED = "submitted"
APPROVED = "approved"
RETAKE = "retake"
class ReviewDecision(str, enum.Enum):
APPROVED = "approved"
RETAKE = "retake"
class AttachmentType(str, enum.Enum):
REFERENCE = "reference"
WORK_FILE = "work_file"
PREVIEW = "preview"
DOCUMENTATION = "documentation"
class Task(Base):
__tablename__ = "tasks"
id = Column(Integer, primary_key=True, index=True)
project_id = Column(Integer, ForeignKey("projects.id"), nullable=False)
episode_id = Column(Integer, ForeignKey("episodes.id"), nullable=True)
shot_id = Column(Integer, ForeignKey("shots.id"), nullable=True)
asset_id = Column(Integer, ForeignKey("assets.id"), nullable=True)
assigned_user_id = Column(Integer, ForeignKey("users.id"), nullable=True)
task_type = Column(String, nullable=False) # Changed from Enum to String to support custom task types
name = Column(String, nullable=False, index=True)
description = Column(Text)
status = Column(String, nullable=False, default="not_started") # Changed from Enum to String to support custom statuses
deadline = Column(Date)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
# Soft deletion columns
deleted_at = Column(DateTime(timezone=True), nullable=True)
deleted_by = Column(Integer, ForeignKey("users.id"), nullable=True)
# Relationships
project = relationship("Project", back_populates="tasks")
episode = relationship("Episode", back_populates="tasks")
shot = relationship("Shot", back_populates="tasks")
asset = relationship("Asset", back_populates="tasks")
assigned_user = relationship("User", foreign_keys=[assigned_user_id], back_populates="assigned_tasks")
submissions = relationship("Submission", back_populates="task", cascade="all, delete-orphan")
production_notes = relationship("ProductionNote", back_populates="task", cascade="all, delete-orphan")
attachments = relationship("TaskAttachment", back_populates="task", cascade="all, delete-orphan")
deleted_by_user = relationship("User", foreign_keys=[deleted_by])
@property
def is_deleted(self) -> bool:
"""Check if the task is soft deleted."""
return self.deleted_at is not None
@classmethod
def query_active(cls, query: Query) -> Query:
"""Filter query to exclude soft deleted tasks."""
return query.filter(cls.deleted_at.is_(None))
@classmethod
def query_deleted(cls, query: Query) -> Query:
"""Filter query to include only soft deleted tasks."""
return query.filter(cls.deleted_at.isnot(None))
@classmethod
def query_all_including_deleted(cls, query: Query) -> Query:
"""Return query without soft deletion filtering (for admin use)."""
return query
def __repr__(self):
return f"<Task(id={self.id}, name='{self.name}', type='{self.task_type}', status='{self.status}')>"
class Submission(Base):
__tablename__ = "submissions"
id = Column(Integer, primary_key=True, index=True)
task_id = Column(Integer, ForeignKey("tasks.id"), nullable=False)
user_id = Column(Integer, ForeignKey("users.id"), nullable=False)
file_path = Column(String, nullable=False)
file_name = Column(String, nullable=False)
version_number = Column(Integer, nullable=False, default=1)
notes = Column(Text)
submitted_at = Column(DateTime(timezone=True), server_default=func.now())
# Soft deletion columns
deleted_at = Column(DateTime(timezone=True), nullable=True)
deleted_by = Column(Integer, ForeignKey("users.id"), nullable=True)
# Relationships
task = relationship("Task", back_populates="submissions")
user = relationship("User", foreign_keys=[user_id], back_populates="submissions")
reviews = relationship("Review", back_populates="submission", cascade="all, delete-orphan")
deleted_by_user = relationship("User", foreign_keys=[deleted_by])
@property
def is_deleted(self) -> bool:
"""Check if the submission is soft deleted."""
return self.deleted_at is not None
@classmethod
def query_active(cls, query: Query) -> Query:
"""Filter query to exclude soft deleted submissions."""
return query.filter(cls.deleted_at.is_(None))
@classmethod
def query_deleted(cls, query: Query) -> Query:
"""Filter query to include only soft deleted submissions."""
return query.filter(cls.deleted_at.isnot(None))
@classmethod
def query_all_including_deleted(cls, query: Query) -> Query:
"""Return query without soft deletion filtering (for admin use)."""
return query
def __repr__(self):
return f"<Submission(id={self.id}, task_id={self.task_id}, version={self.version_number})>"
class Review(Base):
__tablename__ = "reviews"
id = Column(Integer, primary_key=True, index=True)
submission_id = Column(Integer, ForeignKey("submissions.id"), nullable=False)
reviewer_id = Column(Integer, ForeignKey("users.id"), nullable=False)
decision = Column(Enum(ReviewDecision), nullable=False)
feedback = Column(Text)
reviewed_at = Column(DateTime(timezone=True), server_default=func.now())
# Soft deletion columns
deleted_at = Column(DateTime(timezone=True), nullable=True)
deleted_by = Column(Integer, ForeignKey("users.id"), nullable=True)
# Relationships
submission = relationship("Submission", back_populates="reviews")
reviewer = relationship("User", foreign_keys=[reviewer_id], back_populates="reviews")
deleted_by_user = relationship("User", foreign_keys=[deleted_by])
@property
def is_deleted(self) -> bool:
"""Check if the review is soft deleted."""
return self.deleted_at is not None
@classmethod
def query_active(cls, query: Query) -> Query:
"""Filter query to exclude soft deleted reviews."""
return query.filter(cls.deleted_at.is_(None))
@classmethod
def query_deleted(cls, query: Query) -> Query:
"""Filter query to include only soft deleted reviews."""
return query.filter(cls.deleted_at.isnot(None))
@classmethod
def query_all_including_deleted(cls, query: Query) -> Query:
"""Return query without soft deletion filtering (for admin use)."""
return query
def __repr__(self):
return f"<Review(id={self.id}, decision='{self.decision}', submission_id={self.submission_id})>"
class ProductionNote(Base):
__tablename__ = "production_notes"
id = Column(Integer, primary_key=True, index=True)
task_id = Column(Integer, ForeignKey("tasks.id"), nullable=False)
user_id = Column(Integer, ForeignKey("users.id"), nullable=False)
content = Column(Text, nullable=False)
parent_note_id = Column(Integer, ForeignKey("production_notes.id"), nullable=True)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
# Soft deletion columns
deleted_at = Column(DateTime(timezone=True), nullable=True)
deleted_by = Column(Integer, ForeignKey("users.id"), nullable=True)
# Relationships
task = relationship("Task", back_populates="production_notes")
user = relationship("User", foreign_keys=[user_id], back_populates="production_notes")
parent_note = relationship("ProductionNote", remote_side=[id], back_populates="child_notes")
child_notes = relationship("ProductionNote", back_populates="parent_note", cascade="all, delete-orphan")
deleted_by_user = relationship("User", foreign_keys=[deleted_by])
@property
def is_deleted(self) -> bool:
"""Check if the production note is soft deleted."""
return self.deleted_at is not None
@classmethod
def query_active(cls, query: Query) -> Query:
"""Filter query to exclude soft deleted production notes."""
return query.filter(cls.deleted_at.is_(None))
@classmethod
def query_deleted(cls, query: Query) -> Query:
"""Filter query to include only soft deleted production notes."""
return query.filter(cls.deleted_at.isnot(None))
@classmethod
def query_all_including_deleted(cls, query: Query) -> Query:
"""Return query without soft deletion filtering (for admin use)."""
return query
def __repr__(self):
return f"<ProductionNote(id={self.id}, task_id={self.task_id}, user_id={self.user_id})>"
class TaskAttachment(Base):
__tablename__ = "task_attachments"
id = Column(Integer, primary_key=True, index=True)
task_id = Column(Integer, ForeignKey("tasks.id"), nullable=False)
user_id = Column(Integer, ForeignKey("users.id"), nullable=False)
file_name = Column(String, nullable=False)
file_path = Column(String, nullable=False)
file_type = Column(String, nullable=False)
file_size = Column(Integer, nullable=False)
attachment_type = Column(Enum(AttachmentType), nullable=False, default=AttachmentType.REFERENCE)
description = Column(Text)
uploaded_at = Column(DateTime(timezone=True), server_default=func.now())
# Soft deletion columns
deleted_at = Column(DateTime(timezone=True), nullable=True)
deleted_by = Column(Integer, ForeignKey("users.id"), nullable=True)
# Relationships
task = relationship("Task", back_populates="attachments")
user = relationship("User", foreign_keys=[user_id], back_populates="task_attachments")
deleted_by_user = relationship("User", foreign_keys=[deleted_by])
@property
def is_deleted(self) -> bool:
"""Check if the task attachment is soft deleted."""
return self.deleted_at is not None
@classmethod
def query_active(cls, query: Query) -> Query:
"""Filter query to exclude soft deleted task attachments."""
return query.filter(cls.deleted_at.is_(None))
@classmethod
def query_deleted(cls, query: Query) -> Query:
"""Filter query to include only soft deleted task attachments."""
return query.filter(cls.deleted_at.isnot(None))
@classmethod
def query_all_including_deleted(cls, query: Query) -> Query:
"""Return query without soft deletion filtering (for admin use)."""
return query
def __repr__(self):
return f"<TaskAttachment(id={self.id}, file_name='{self.file_name}', type='{self.attachment_type}')>"
backend/models/user.py
+52
View File
@@ -0,0 +1,52 @@
from sqlalchemy import Column, Integer, String, Boolean, DateTime, Enum
from sqlalchemy.orm import relationship
from sqlalchemy.sql import func
from database import Base
import enum
class UserRole(str, enum.Enum):
DIRECTOR = "director"
COORDINATOR = "coordinator"
ARTIST = "artist"
DEVELOPER = "developer"
class DepartmentRole(str, enum.Enum):
LAYOUT = "layout"
ANIMATION = "animation"
LIGHTING = "lighting"
COMPOSITE = "composite"
MODELING = "modeling"
RIGGING = "rigging"
SURFACING = "surfacing"
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True, index=True)
email = Column(String, unique=True, index=True, nullable=False)
password_hash = Column(String, nullable=False)
first_name = Column(String, nullable=False)
last_name = Column(String, nullable=False)
role = Column(Enum(UserRole), nullable=False, default=UserRole.ARTIST)
is_admin = Column(Boolean, default=False, nullable=False)
is_approved = Column(Boolean, default=False, nullable=False)
avatar_url = Column(String, nullable=True)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
# Relationships
project_memberships = relationship("ProjectMember", back_populates="user", cascade="all, delete-orphan")
assigned_tasks = relationship("Task", foreign_keys="Task.assigned_user_id", back_populates="assigned_user")
submissions = relationship("Submission", foreign_keys="Submission.user_id", back_populates="user")
reviews = relationship("Review", foreign_keys="Review.reviewer_id", back_populates="reviewer")
production_notes = relationship("ProductionNote", foreign_keys="ProductionNote.user_id", back_populates="user")
task_attachments = relationship("TaskAttachment", foreign_keys="TaskAttachment.user_id", back_populates="user")
api_keys = relationship("APIKey", back_populates="user", cascade="all, delete-orphan")
notifications = relationship("Notification", back_populates="user", cascade="all, delete-orphan")
notification_preferences = relationship("UserNotificationPreference", back_populates="user", uselist=False, cascade="all, delete-orphan")
def __repr__(self):
return f"<User(id={self.id}, email='{self.email}', role='{self.role}')>"
backend/monitor_query_performance.py
+298
View File
@@ -0,0 +1,298 @@
#!/usr/bin/env python3
"""
Query performance monitoring script for VFX Project Management System.
Analyzes query execution plans and provides performance insights.
"""
import sqlite3
import sys
import time
from pathlib import Path
def monitor_query_performance():
"""Monitor and analyze query performance for key operations."""
# Database path
db_path = Path(__file__).parent / "vfx_project_management.db"
if not db_path.exists():
print(f"Database file not found at {db_path}")
return False
try:
# Connect to database
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("=== Query Performance Monitoring ===\n")
# Define key queries to test
test_queries = [
{
"name": "Get active tasks by user",
"query": """
SELECT t.id, t.name, t.status, t.task_type
FROM tasks t
WHERE t.assigned_user_id = 1
AND t.deleted_at IS NULL
ORDER BY t.created_at DESC
LIMIT 50
""",
"description": "Common query for user task dashboard"
},
{
"name": "Get shots by episode (non-deleted)",
"query": """
SELECT s.id, s.name, s.status, s.frame_start, s.frame_end
FROM shots s
WHERE s.episode_id = 1
AND s.deleted_at IS NULL
ORDER BY s.name
""",
"description": "Common query for shot browser"
},
{
"name": "Get assets by project (non-deleted)",
"query": """
SELECT a.id, a.name, a.category, a.status
FROM assets a
WHERE a.project_id = 1
AND a.deleted_at IS NULL
ORDER BY a.category, a.name
""",
"description": "Common query for asset browser"
},
{
"name": "Get task submissions with reviews",
"query": """
SELECT s.id, s.file_name, s.version_number, s.submitted_at,
r.decision, r.feedback
FROM submissions s
LEFT JOIN reviews r ON s.id = r.submission_id AND r.deleted_at IS NULL
WHERE s.task_id = 1
AND s.deleted_at IS NULL
ORDER BY s.version_number DESC
""",
"description": "Common query for task detail panel"
},
{
"name": "Get project tasks with status filter",
"query": """
SELECT t.id, t.name, t.task_type, t.status,
u.first_name, u.last_name,
COALESCE(s.name, a.name) as parent_name
FROM tasks t
LEFT JOIN users u ON t.assigned_user_id = u.id
LEFT JOIN shots s ON t.shot_id = s.id AND s.deleted_at IS NULL
LEFT JOIN assets a ON t.asset_id = a.id AND a.deleted_at IS NULL
WHERE t.project_id = 1
AND t.status IN ('in_progress', 'pending_review')
AND t.deleted_at IS NULL
ORDER BY t.created_at DESC
""",
"description": "Complex query for project task overview"
},
{
"name": "Get recent activity feed",
"query": """
SELECT a.type, a.description, a.created_at,
u.first_name, u.last_name
FROM activities a
LEFT JOIN users u ON a.user_id = u.id
WHERE a.project_id = 1
ORDER BY a.created_at DESC
LIMIT 20
""",
"description": "Common query for activity feed"
},
{
"name": "Get user workload summary",
"query": """
SELECT u.id, u.first_name, u.last_name,
COUNT(t.id) as task_count,
COUNT(CASE WHEN t.status = 'in_progress' THEN 1 END) as active_tasks
FROM users u
LEFT JOIN tasks t ON u.id = t.assigned_user_id AND t.deleted_at IS NULL
WHERE u.role = 'artist'
GROUP BY u.id, u.first_name, u.last_name
ORDER BY task_count DESC
""",
"description": "Query for workload management"
}
]
print("Testing query performance...\n")
for test in test_queries:
print(f"Query: {test['name']}")
print(f"Description: {test['description']}")
# Get query plan
explain_query = f"EXPLAIN QUERY PLAN {test['query']}"
cursor.execute(explain_query)
plan = cursor.fetchall()
print("Execution Plan:")
for step in plan:
print(f" {step[0]}: {step[3]}")
# Measure execution time
start_time = time.time()
cursor.execute(test['query'])
results = cursor.fetchall()
end_time = time.time()
execution_time = (end_time - start_time) * 1000 # Convert to milliseconds
print(f"Execution Time: {execution_time:.2f}ms")
print(f"Result Count: {len(results)}")
# Check if indexes are being used
plan_text = " ".join([step[3] for step in plan])
if "USING INDEX" in plan_text:
print("✓ Index usage detected")
else:
print("⚠ No index usage detected")
print("-" * 60)
return True
except sqlite3.Error as e:
print(f"Database error: {e}")
return False
except Exception as e:
print(f"Unexpected error: {e}")
return False
finally:
if conn:
conn.close()
def analyze_index_usage():
"""Analyze which indexes are being used effectively."""
db_path = Path(__file__).parent / "vfx_project_management.db"
try:
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("\n=== Index Usage Analysis ===\n")
# Get all indexes
cursor.execute("""
SELECT name, tbl_name
FROM sqlite_master
WHERE type = 'index'
AND name NOT LIKE 'sqlite_%'
ORDER BY tbl_name, name
""")
indexes = cursor.fetchall()
print(f"Total custom indexes: {len(indexes)}")
print("\nIndexes by table:")
current_table = None
for index_name, table_name in indexes:
if table_name != current_table:
print(f"\n{table_name}:")
current_table = table_name
print(f" - {index_name}")
# Analyze soft deletion indexes
print("\n=== Soft Deletion Index Analysis ===")
soft_deletion_tables = ['shots', 'assets', 'tasks', 'submissions',
'task_attachments', 'production_notes', 'reviews']
for table in soft_deletion_tables:
cursor.execute(f"PRAGMA index_list({table})")
table_indexes = cursor.fetchall()
soft_del_indexes = [idx for idx in table_indexes if 'not_deleted' in idx[1]]
print(f"{table}: {len(soft_del_indexes)} soft deletion indexes")
for idx in soft_del_indexes:
print(f" - {idx[1]}")
return True
except sqlite3.Error as e:
print(f"Database error: {e}")
return False
finally:
if conn:
conn.close()
def benchmark_soft_deletion_queries():
"""Benchmark queries with and without soft deletion filtering."""
db_path = Path(__file__).parent / "vfx_project_management.db"
try:
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("\n=== Soft Deletion Query Benchmarks ===\n")
# Test queries with soft deletion filtering
benchmark_queries = [
{
"name": "Tasks with soft deletion filter",
"query": "SELECT COUNT(*) FROM tasks WHERE deleted_at IS NULL"
},
{
"name": "Tasks without filter (all records)",
"query": "SELECT COUNT(*) FROM tasks"
},
{
"name": "Shots with soft deletion filter",
"query": "SELECT COUNT(*) FROM shots WHERE deleted_at IS NULL"
},
{
"name": "Shots without filter (all records)",
"query": "SELECT COUNT(*) FROM shots"
},
{
"name": "Assets with soft deletion filter",
"query": "SELECT COUNT(*) FROM assets WHERE deleted_at IS NULL"
},
{
"name": "Assets without filter (all records)",
"query": "SELECT COUNT(*) FROM assets"
}
]
for benchmark in benchmark_queries:
# Measure execution time
start_time = time.time()
cursor.execute(benchmark['query'])
result = cursor.fetchone()[0]
end_time = time.time()
execution_time = (end_time - start_time) * 1000
print(f"{benchmark['name']}: {result} records ({execution_time:.2f}ms)")
return True
except sqlite3.Error as e:
print(f"Database error: {e}")
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("Query Performance Monitoring Tool")
print("=" * 50)
success = monitor_query_performance()
if success:
analyze_index_usage()
benchmark_soft_deletion_queries()
print("\nSUCCESS: Query performance analysis completed!")
else:
print("\n❌ Query performance analysis failed!")
sys.exit(1)
backend/optimize_database_indexes.py
+335
View File
@@ -0,0 +1,335 @@
#!/usr/bin/env python3
"""
Database index optimization script for VFX Project Management System.
Adds missing indexes to improve query performance, especially for soft deletion filtering.
"""
import sqlite3
import sys
from pathlib import Path
def optimize_database_indexes():
"""Add missing indexes to optimize database performance."""
# Database path
db_path = Path(__file__).parent / "vfx_project_management.db"
if not db_path.exists():
print(f"Database file not found at {db_path}")
return False
try:
# Connect to database
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("=== Database Index Optimization ===\n")
# Define indexes to create
indexes_to_create = [
# Task-related indexes for performance
{
"name": "idx_tasks_assigned_user_not_deleted",
"table": "tasks",
"columns": "assigned_user_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for tasks by assigned user (non-deleted)"
},
{
"name": "idx_tasks_status_not_deleted",
"table": "tasks",
"columns": "status",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for tasks by status (non-deleted)"
},
{
"name": "idx_tasks_type_not_deleted",
"table": "tasks",
"columns": "task_type",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for tasks by type (non-deleted)"
},
{
"name": "idx_tasks_project_not_deleted",
"table": "tasks",
"columns": "project_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for tasks by project (non-deleted)"
},
{
"name": "idx_tasks_episode_not_deleted",
"table": "tasks",
"columns": "episode_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for tasks by episode (non-deleted)"
},
# Shot-related indexes
{
"name": "idx_shots_episode_not_deleted",
"table": "shots",
"columns": "episode_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for shots by episode (non-deleted)"
},
{
"name": "idx_shots_status_not_deleted",
"table": "shots",
"columns": "status",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for shots by status (non-deleted)"
},
# Asset-related indexes
{
"name": "idx_assets_project_not_deleted",
"table": "assets",
"columns": "project_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for assets by project (non-deleted)"
},
{
"name": "idx_assets_category_not_deleted",
"table": "assets",
"columns": "category",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for assets by category (non-deleted)"
},
{
"name": "idx_assets_status_not_deleted",
"table": "assets",
"columns": "status",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for assets by status (non-deleted)"
},
# Submission-related indexes
{
"name": "idx_submissions_user_not_deleted",
"table": "submissions",
"columns": "user_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for submissions by user (non-deleted)"
},
{
"name": "idx_submissions_submitted_at_not_deleted",
"table": "submissions",
"columns": "submitted_at",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries ordering submissions by date (non-deleted)"
},
# Production notes indexes
{
"name": "idx_production_notes_user_not_deleted",
"table": "production_notes",
"columns": "user_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for notes by user (non-deleted)"
},
{
"name": "idx_production_notes_created_at_not_deleted",
"table": "production_notes",
"columns": "created_at",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries ordering notes by date (non-deleted)"
},
# Task attachments indexes
{
"name": "idx_task_attachments_user_not_deleted",
"table": "task_attachments",
"columns": "user_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for attachments by user (non-deleted)"
},
{
"name": "idx_task_attachments_type_not_deleted",
"table": "task_attachments",
"columns": "attachment_type",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for attachments by type (non-deleted)"
},
# Reviews indexes
{
"name": "idx_reviews_reviewer_not_deleted",
"table": "reviews",
"columns": "reviewer_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for reviews by reviewer (non-deleted)"
},
{
"name": "idx_reviews_decision_not_deleted",
"table": "reviews",
"columns": "decision",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for reviews by decision (non-deleted)"
},
# Foreign key indexes for better join performance
{
"name": "idx_project_members_user_id",
"table": "project_members",
"columns": "user_id",
"condition": "",
"description": "Optimize joins on project members by user"
},
{
"name": "idx_project_members_project_id",
"table": "project_members",
"columns": "project_id",
"condition": "",
"description": "Optimize joins on project members by project"
},
{
"name": "idx_episodes_project_id",
"table": "episodes",
"columns": "project_id",
"condition": "",
"description": "Optimize queries for episodes by project"
},
# Composite indexes for complex queries
{
"name": "idx_tasks_composite_not_deleted",
"table": "tasks",
"columns": "project_id, status, assigned_user_id",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize complex task queries (project + status + user)"
},
{
"name": "idx_submissions_task_version_not_deleted",
"table": "submissions",
"columns": "task_id, version_number",
"condition": "WHERE deleted_at IS NULL",
"description": "Optimize queries for latest submission versions"
}
]
created_count = 0
skipped_count = 0
for index_def in indexes_to_create:
index_name = index_def["name"]
table_name = index_def["table"]
columns = index_def["columns"]
condition = index_def["condition"]
description = index_def["description"]
# Check if index already exists
cursor.execute(f"PRAGMA index_list({table_name})")
existing_indexes = [idx[1] for idx in cursor.fetchall()]
if index_name in existing_indexes:
print(f"SKIP: Index {index_name} already exists")
skipped_count += 1
continue
# Create the index
try:
sql = f"CREATE INDEX {index_name} ON {table_name} ({columns})"
if condition:
sql += f" {condition}"
print(f"Creating index: {index_name}")
print(f" Table: {table_name}")
print(f" Columns: {columns}")
print(f" Description: {description}")
cursor.execute(sql)
created_count += 1
print(f"SUCCESS: Created index {index_name}")
except sqlite3.Error as e:
print(f"ERROR: Failed to create index {index_name}: {e}")
continue
print()
# Commit all changes
conn.commit()
print(f"=== Index Optimization Complete ===")
print(f"Created: {created_count} new indexes")
print(f"Skipped: {skipped_count} existing indexes")
return True
except sqlite3.Error as e:
print(f"Database error: {e}")
if conn:
conn.rollback()
return False
except Exception as e:
print(f"Unexpected error: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
def verify_index_optimization():
"""Verify that the index optimization was successful."""
db_path = Path(__file__).parent / "vfx_project_management.db"
try:
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
print("\n=== Verifying Index Optimization ===\n")
# Count total indexes
cursor.execute("""
SELECT COUNT(*)
FROM sqlite_master
WHERE type = 'index' AND name NOT LIKE 'sqlite_%'
""")
total_indexes = cursor.fetchone()[0]
print(f"Total indexes in database: {total_indexes}")
# Check key performance indexes
key_indexes = [
"idx_tasks_assigned_user_not_deleted",
"idx_tasks_status_not_deleted",
"idx_shots_episode_not_deleted",
"idx_assets_project_not_deleted",
"idx_submissions_submitted_at_not_deleted"
]
print("\nKey performance indexes:")
for index_name in key_indexes:
cursor.execute("""
SELECT name FROM sqlite_master
WHERE type = 'index' AND name = ?
""", (index_name,))
if cursor.fetchone():
print(f" ✓ {index_name}")
else:
print(f" ✗ {index_name} (missing)")
return True
except sqlite3.Error as e:
print(f"Verification error: {e}")
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("Database Index Optimization Tool")
print("=" * 50)
success = optimize_database_indexes()
if success:
verify_index_optimization()
print("\nSUCCESS: Database index optimization completed!")
else:
print("\n❌ Database index optimization failed!")
sys.exit(1)
backend/recreate_database.py
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env python3
"""
Recreate the database with proper schema and migrate existing data.
"""
import sqlite3
import os
from pathlib import Path
def backup_projects_data():
"""Backup existing projects data."""
print("Backing up existing projects data...")
db_path = "vfx_project_management.db"
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Get all projects data
cursor.execute("SELECT * FROM projects")
projects = cursor.fetchall()
# Get column names
cursor.execute("PRAGMA table_info(projects)")
columns = [col[1] for col in cursor.fetchall()]
return projects, columns
except Exception as e:
print(f"Error backing up data: {e}")
return [], []
finally:
if conn:
conn.close()
def recreate_database():
"""Recreate the database with proper schema."""
print("Recreating Database with Proper Schema")
print("=" * 40)
# Backup existing data
projects_data, columns = backup_projects_data()
print(f"Backed up {len(projects_data)} projects")
# Remove old database
db_path = "vfx_project_management.db"
if os.path.exists(db_path):
os.remove(db_path)
print("Removed old database")
# Import and create new database schema
try:
from database import engine, Base
import models # This imports all models
# Create all tables with proper schema
Base.metadata.create_all(bind=engine)
print("Created new database schema")
# Restore projects data if any
if projects_data:
print("Restoring projects data...")
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
for project in projects_data:
# Map old data to new schema
insert_sql = """
INSERT INTO projects (
id, name, description, status, start_date, end_date,
created_at, updated_at, code_name, client_name, project_type,
frame_rate, data_drive_path, publish_storage_path,
delivery_image_resolution, delivery_movie_specs_by_department
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
"""
cursor.execute(insert_sql, project)
conn.commit()
conn.close()
print(f"Restored {len(projects_data)} projects")
print("✅ Database recreated successfully!")
return True
except Exception as e:
print(f"❌ Error recreating database: {e}")
import traceback
traceback.print_exc()
return False
if __name__ == "__main__":
recreate_database()
backend/requirements.txt
+12
View File
@@ -0,0 +1,12 @@
fastapi==0.104.1
uvicorn[standard]==0.24.0
sqlalchemy==2.0.23
python-multipart==0.0.6
python-jose[cryptography]==3.3.0
passlib[bcrypt]==1.7.4
python-dotenv==1.0.0
pydantic==2.5.0
pydantic-settings==2.1.0
httpx==0.28.1
email-validator==2.1.0
Pillow==10.1.0
backend/rollback_shot_project_id.py
+215
View File
@@ -0,0 +1,215 @@
#!/usr/bin/env python3
"""
Rollback script for shot project_id migration.
This script removes the project_id column and related constraints/indexes.
WARNING: This will permanently remove the project_id column and all related data.
Only use this if you need to rollback the migration.
Usage:
python rollback_shot_project_id.py
"""
import sqlite3
import sys
from pathlib import Path
import logging
# Setup logging
logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s')
logger = logging.getLogger(__name__)
def get_database_path():
"""Get the database path, trying multiple possible locations."""
possible_paths = [
"vfx_project_management.db",
"database.db",
"../vfx_project_management.db"
]
for path in possible_paths:
if Path(path).exists():
return path
return "vfx_project_management.db"
def rollback_migration():
"""Remove project_id column and related constraints from shots table."""
db_path = get_database_path()
logger.info(f"Using database: {db_path}")
# Confirm rollback
print("⚠️ WARNING: This will permanently remove the project_id column from shots table!")
print("⚠️ All project_id data will be lost!")
response = input("Are you sure you want to continue? (type 'YES' to confirm): ")
if response != 'YES':
print("Rollback cancelled.")
return True
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Check if shots table exists
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='shots'")
if not cursor.fetchone():
logger.error("Shots table not found")
return False
# Check if project_id column exists
cursor.execute("PRAGMA table_info(shots)")
columns = [col[1] for col in cursor.fetchall()]
if 'project_id' not in columns:
logger.info("project_id column does not exist, nothing to rollback")
return True
logger.info("Starting rollback of shot project_id migration...")
# Create new table without project_id column
logger.info("Creating new shots table without project_id...")
cursor.execute("""
CREATE TABLE shots_rollback (
id INTEGER PRIMARY KEY,
episode_id INTEGER NOT NULL,
name VARCHAR NOT NULL,
description VARCHAR,
frame_start INTEGER NOT NULL DEFAULT 1001,
frame_end INTEGER NOT NULL DEFAULT 1001,
status VARCHAR(11) NOT NULL DEFAULT 'not_started',
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
deleted_at TIMESTAMP,
deleted_by INTEGER,
FOREIGN KEY (episode_id) REFERENCES episodes(id),
FOREIGN KEY (deleted_by) REFERENCES users(id)
)
""")
# Copy data from old table to new table (excluding project_id)
logger.info("Copying data to new table...")
cursor.execute("""
INSERT INTO shots_rollback
SELECT id, episode_id, name, description, frame_start, frame_end,
status, created_at, updated_at, deleted_at, deleted_by
FROM shots
""")
# Verify data copy
cursor.execute("SELECT COUNT(*) FROM shots")
original_count = cursor.fetchone()[0]
cursor.execute("SELECT COUNT(*) FROM shots_rollback")
new_count = cursor.fetchone()[0]
if original_count != new_count:
logger.error(f"Data copy failed: original={original_count}, new={new_count}")
return False
logger.info(f"Successfully copied {new_count} shots to new table")
# Drop old table and rename new table
logger.info("Replacing table...")
cursor.execute("DROP TABLE shots")
cursor.execute("ALTER TABLE shots_rollback RENAME TO shots")
# Recreate basic indexes (without project_id related ones)
logger.info("Recreating basic indexes...")
cursor.execute("CREATE INDEX idx_shots_name ON shots(name)")
cursor.execute("CREATE INDEX idx_shots_episode_id ON shots(episode_id)")
cursor.execute("CREATE INDEX idx_shots_active ON shots(id) WHERE deleted_at IS NULL")
# Commit changes
conn.commit()
logger.info("Rollback completed successfully!")
logger.info(f"Removed project_id column from {new_count} shots")
return True
except sqlite3.Error as e:
logger.error(f"Database error: {e}")
if conn:
conn.rollback()
return False
except Exception as e:
logger.error(f"Unexpected error: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
def verify_rollback():
"""Verify that the rollback was successful."""
db_path = get_database_path()
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
logger.info("Verifying rollback...")
# Check that project_id column is gone
cursor.execute("PRAGMA table_info(shots)")
columns = [col[1] for col in cursor.fetchall()]
if 'project_id' in columns:
logger.error("❌ project_id column still exists")
return False
logger.info("✅ project_id column successfully removed")
# Check that basic functionality still works
cursor.execute("SELECT COUNT(*) FROM shots")
shot_count = cursor.fetchone()[0]
logger.info(f"✅ {shot_count} shots remain in table")
# Check that episode relationships still work
cursor.execute("""
SELECT COUNT(*) FROM shots s
JOIN episodes e ON s.episode_id = e.id
""")
valid_relationships = cursor.fetchone()[0]
if valid_relationships != shot_count:
logger.error(f"❌ Episode relationships broken: {valid_relationships}/{shot_count}")
return False
logger.info("✅ Episode relationships intact")
return True
except sqlite3.Error as e:
logger.error(f"Verification error: {e}")
return False
finally:
if conn:
conn.close()
if __name__ == "__main__":
print("VFX Project Management - Shot Project ID Rollback")
print("=" * 60)
success = rollback_migration()
if success:
print("\n" + "=" * 60)
print("VERIFYING ROLLBACK")
print("=" * 60)
if verify_rollback():
print("\n✅ SUCCESS: Rollback completed successfully!")
sys.exit(0)
else:
print("\n❌ FAILED: Rollback verification failed!")
sys.exit(1)
else:
print("\n❌ FAILED: Rollback failed!")
sys.exit(1)
backend/routers/__init__.py
+1
View File
@@ -0,0 +1 @@
# Routers package
backend/routers/activities.py
+239
View File
@@ -0,0 +1,239 @@
from fastapi import APIRouter, Depends, Query
from sqlalchemy.orm import Session
from sqlalchemy import desc
from typing import List, Optional
from datetime import datetime, timedelta
from database import get_db
from models.user import User
from models.activity import Activity, ActivityType
from models.project import ProjectMember
from schemas.activity import ActivityResponse
from utils.auth import get_current_user
from utils.activity import ActivityService
router = APIRouter(prefix="/activities", tags=["activities"])
@router.get("/project/{project_id}", response_model=List[ActivityResponse])
def get_project_activities(
project_id: int,
skip: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=100),
type_filter: Optional[ActivityType] = None,
days: Optional[int] = Query(None, ge=1, le=90),
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get activity feed for a specific project (excludes activities for deleted records)."""
# Verify user has access to the project
member = db.query(ProjectMember).filter(
ProjectMember.project_id == project_id,
ProjectMember.user_id == current_user.id
).first()
if not member and not current_user.is_admin:
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Access denied to this project")
# Use ActivityService to get activities excluding deleted records
activities = ActivityService.get_activities_excluding_deleted(
db=db,
project_id=project_id,
skip=skip,
limit=limit,
type_filter=type_filter,
days=days
)
return activities
@router.get("/task/{task_id}", response_model=List[ActivityResponse])
def get_task_activities(
task_id: int,
skip: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=100),
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get activity timeline for a specific task (excludes activities for deleted records)."""
from models.task import Task
# Verify user has access to the task and it's not deleted
task = db.query(Task).filter(Task.id == task_id, Task.deleted_at.is_(None)).first()
if not task:
from fastapi import HTTPException
raise HTTPException(status_code=404, detail="Task not found")
# Check if user is a member of the project
member = db.query(ProjectMember).filter(
ProjectMember.project_id == task.project_id,
ProjectMember.user_id == current_user.id
).first()
if not member and not current_user.is_admin:
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Access denied to this task")
# Use ActivityService to get activities excluding deleted records
activities = ActivityService.get_activities_excluding_deleted(
db=db,
task_id=task_id,
skip=skip,
limit=limit
)
return activities
@router.get("/user/{user_id}", response_model=List[ActivityResponse])
def get_user_activities(
user_id: int,
skip: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=100),
days: Optional[int] = Query(None, ge=1, le=90),
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get activity history for a specific user (excludes activities for deleted records)."""
# Users can only view their own activity unless they're admin
if user_id != current_user.id and not current_user.is_admin:
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Access denied")
# Use ActivityService to get activities excluding deleted records
activities = ActivityService.get_activities_excluding_deleted(
db=db,
user_id=user_id,
skip=skip,
limit=limit,
days=days
)
return activities
@router.get("/recent", response_model=List[ActivityResponse])
def get_recent_activities(
skip: int = Query(0, ge=0),
limit: int = Query(20, ge=1, le=50),
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get recent activities from all projects the user has access to (excludes activities for deleted records)."""
# Get all projects the user is a member of
project_ids = db.query(ProjectMember.project_id).filter(
ProjectMember.user_id == current_user.id
).all()
project_ids = [pid[0] for pid in project_ids]
if not project_ids and not current_user.is_admin:
return []
# For non-admin users, filter by their project access
if not current_user.is_admin:
# Get activities from user's projects, excluding deleted records
all_activities = []
for project_id in project_ids:
activities = ActivityService.get_activities_excluding_deleted(
db=db,
project_id=project_id,
skip=0,
limit=limit * 2 # Get more to account for filtering
)
all_activities.extend(activities)
# Sort by created_at and apply pagination
all_activities.sort(key=lambda x: x.created_at, reverse=True)
return all_activities[skip:skip + limit]
else:
# Admin gets all activities excluding deleted records
activities = ActivityService.get_activities_excluding_deleted(
db=db,
skip=skip,
limit=limit
)
return activities
# Admin-only endpoints that include activities for deleted records
@router.get("/admin/project/{project_id}/all", response_model=List[ActivityResponse])
def get_project_activities_including_deleted(
project_id: int,
skip: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=100),
type_filter: Optional[ActivityType] = None,
days: Optional[int] = Query(None, ge=1, le=90),
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get all activity feed for a specific project including deleted records (admin only)."""
if not current_user.is_admin:
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Admin access required")
# Use ActivityService to get all activities including deleted records
activities = ActivityService.get_activities_including_deleted(
db=db,
project_id=project_id,
skip=skip,
limit=limit,
type_filter=type_filter,
days=days
)
return activities
@router.get("/admin/user/{user_id}/all", response_model=List[ActivityResponse])
def get_user_activities_including_deleted(
user_id: int,
skip: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=100),
days: Optional[int] = Query(None, ge=1, le=90),
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get all activity history for a specific user including deleted records (admin only)."""
if not current_user.is_admin:
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Admin access required")
# Use ActivityService to get all activities including deleted records
activities = ActivityService.get_activities_including_deleted(
db=db,
user_id=user_id,
skip=skip,
limit=limit,
days=days
)
return activities
@router.get("/admin/all", response_model=List[ActivityResponse])
def get_all_activities_including_deleted(
skip: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=100),
type_filter: Optional[ActivityType] = None,
days: Optional[int] = Query(None, ge=1, le=90),
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get all activities including deleted records (admin only)."""
if not current_user.is_admin:
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Admin access required")
# Use ActivityService to get all activities including deleted records
activities = ActivityService.get_activities_including_deleted(
db=db,
skip=skip,
limit=limit,
type_filter=type_filter,
days=days
)
return activities
backend/routers/admin.py
+800
View File
@@ -0,0 +1,800 @@
"""
Admin Router
This router contains admin-only endpoints for managing soft-deleted data
and other administrative functions.
"""
from fastapi import APIRouter, Depends, HTTPException, status, Query
from sqlalchemy.orm import Session
from typing import List, Optional
from pydantic import BaseModel
import time
from collections import defaultdict
from database import get_db
from models.user import User, UserRole
from utils.auth import get_current_user_from_token
from services.recovery_service import RecoveryService
from services.batch_operations import BatchOperationsService
router = APIRouter()
# Simple rate limiting storage (in production, use Redis or similar)
_rate_limit_storage = defaultdict(list)
PERMANENT_DELETE_RATE_LIMIT = 10 # Max 10 permanent delete operations per minute per user
class BulkRecoveryRequest(BaseModel):
shot_ids: List[int] = []
asset_ids: List[int] = []
class BulkDeletionRequest(BaseModel):
shot_ids: List[int] = []
asset_ids: List[int] = []
batch_size: Optional[int] = 50
class BatchPreviewRequest(BaseModel):
shot_ids: List[int] = []
asset_ids: List[int] = []
class PermanentDeleteRequest(BaseModel):
confirmation_token: str
class BulkPermanentDeleteRequest(BaseModel):
shot_ids: List[int] = []
asset_ids: List[int] = []
confirmation_token: str
def require_admin(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Require admin role."""
from utils.auth import _get_user_from_db
current_user = _get_user_from_db(db, token_data["user_id"])
if not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Admin permission required"
)
return current_user
def check_permanent_delete_rate_limit(user_id: int):
"""Check if user has exceeded permanent delete rate limit."""
current_time = time.time()
user_requests = _rate_limit_storage[user_id]
# Remove requests older than 1 minute
user_requests[:] = [req_time for req_time in user_requests if current_time - req_time < 60]
if len(user_requests) >= PERMANENT_DELETE_RATE_LIMIT:
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=f"Rate limit exceeded. Maximum {PERMANENT_DELETE_RATE_LIMIT} permanent delete operations per minute."
)
# Add current request
user_requests.append(current_time)
def validate_confirmation_token(token: str, expected_action: str):
"""Validate confirmation token for permanent delete operations."""
# Simple token validation - in production, use proper token generation/validation
expected_token = f"CONFIRM_{expected_action}_PERMANENT_DELETE"
if token != expected_token:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Invalid confirmation token. Permanent deletion requires explicit confirmation."
)
@router.get("/deleted-shots/")
async def get_deleted_shots(
project_id: Optional[int] = Query(None, description="Filter by project ID"),
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Get list of deleted shots for admin recovery interface"""
recovery_service = RecoveryService()
deleted_shots = recovery_service.get_deleted_shots(project_id, db)
return [
{
"id": shot.id,
"name": shot.name,
"episode_name": shot.episode_name,
"project_id": shot.project_id,
"project_name": shot.project_name,
"deleted_at": shot.deleted_at,
"deleted_by": shot.deleted_by,
"deleted_by_name": shot.deleted_by_name,
"task_count": shot.task_count,
"submission_count": shot.submission_count,
"attachment_count": shot.attachment_count,
"note_count": shot.note_count,
"review_count": shot.review_count
}
for shot in deleted_shots
]
@router.get("/deleted-assets/")
async def get_deleted_assets(
project_id: Optional[int] = Query(None, description="Filter by project ID"),
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Get list of deleted assets for admin recovery interface"""
recovery_service = RecoveryService()
deleted_assets = recovery_service.get_deleted_assets(project_id, db)
return [
{
"id": asset.id,
"name": asset.name,
"category": asset.category,
"project_id": asset.project_id,
"project_name": asset.project_name,
"deleted_at": asset.deleted_at,
"deleted_by": asset.deleted_by,
"deleted_by_name": asset.deleted_by_name,
"task_count": asset.task_count,
"submission_count": asset.submission_count,
"attachment_count": asset.attachment_count,
"note_count": asset.note_count,
"review_count": asset.review_count
}
for asset in deleted_assets
]
@router.get("/shots/{shot_id}/recovery-preview")
async def get_shot_recovery_preview(
shot_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Get information about what will be recovered when restoring a shot"""
recovery_service = RecoveryService()
recovery_info = recovery_service.preview_shot_recovery(shot_id, db)
if not recovery_info:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Deleted shot not found"
)
return {
"shot_id": recovery_info.shot_id,
"name": recovery_info.name,
"episode_name": recovery_info.episode_name,
"project_id": recovery_info.project_id,
"project_name": recovery_info.project_name,
"task_count": recovery_info.task_count,
"submission_count": recovery_info.submission_count,
"attachment_count": recovery_info.attachment_count,
"note_count": recovery_info.note_count,
"review_count": recovery_info.review_count,
"deleted_at": recovery_info.deleted_at,
"deleted_by": recovery_info.deleted_by,
"deleted_by_name": recovery_info.deleted_by_name,
"files_preserved": recovery_info.files_preserved,
"file_count": recovery_info.file_count
}
@router.get("/assets/{asset_id}/recovery-preview")
async def get_asset_recovery_preview(
asset_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Get information about what will be recovered when restoring an asset"""
recovery_service = RecoveryService()
recovery_info = recovery_service.preview_asset_recovery(asset_id, db)
if not recovery_info:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Deleted asset not found"
)
return {
"asset_id": recovery_info.asset_id,
"name": recovery_info.name,
"project_name": recovery_info.project_name,
"task_count": recovery_info.task_count,
"submission_count": recovery_info.submission_count,
"attachment_count": recovery_info.attachment_count,
"note_count": recovery_info.note_count,
"review_count": recovery_info.review_count,
"deleted_at": recovery_info.deleted_at,
"deleted_by": recovery_info.deleted_by,
"deleted_by_name": recovery_info.deleted_by_name,
"files_preserved": recovery_info.files_preserved,
"file_count": recovery_info.file_count
}
@router.post("/shots/{shot_id}/recover")
async def recover_shot(
shot_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Recover a soft-deleted shot and all its related data"""
recovery_service = RecoveryService()
result = recovery_service.recover_shot(shot_id, db, current_user)
if not result.success:
db.rollback()
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail={
"message": "Failed to recover shot",
"errors": result.errors
}
)
# Commit the transaction
db.commit()
return {
"message": f"Shot '{result.name}' and all related data have been recovered",
"shot_id": result.shot_id,
"name": result.name,
"recovered_at": result.recovered_at,
"recovered_by": result.recovered_by,
"recovered_tasks": result.recovered_tasks,
"recovered_submissions": result.recovered_submissions,
"recovered_attachments": result.recovered_attachments,
"recovered_notes": result.recovered_notes,
"recovered_reviews": result.recovered_reviews,
"operation_duration": result.operation_duration
}
@router.post("/assets/{asset_id}/recover")
async def recover_asset(
asset_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Recover a soft-deleted asset and all its related data"""
recovery_service = RecoveryService()
result = recovery_service.recover_asset(asset_id, db, current_user)
if not result.success:
db.rollback()
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail={
"message": "Failed to recover asset",
"errors": result.errors
}
)
# Commit the transaction
db.commit()
return {
"message": f"Asset '{result.name}' and all related data have been recovered",
"asset_id": result.asset_id,
"name": result.name,
"recovered_at": result.recovered_at,
"recovered_by": result.recovered_by,
"recovered_tasks": result.recovered_tasks,
"recovered_submissions": result.recovered_submissions,
"recovered_attachments": result.recovered_attachments,
"recovered_notes": result.recovered_notes,
"recovered_reviews": result.recovered_reviews,
"operation_duration": result.operation_duration
}
@router.post("/shots/bulk-recover")
async def bulk_recover_shots(
request: BulkRecoveryRequest,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Bulk recover multiple shots"""
if not request.shot_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No shot IDs provided"
)
recovery_service = RecoveryService()
result = recovery_service.bulk_recover_shots(request.shot_ids, db, current_user)
# Commit the transaction
db.commit()
return {
"total_items": result.total_items,
"successful_recoveries": result.successful_recoveries,
"failed_recoveries": result.failed_recoveries,
"results": [
{
"success": r.success,
"shot_id": r.shot_id,
"name": r.name,
"recovered_tasks": r.recovered_tasks,
"recovered_submissions": r.recovered_submissions,
"recovered_attachments": r.recovered_attachments,
"recovered_notes": r.recovered_notes,
"recovered_reviews": r.recovered_reviews,
"operation_duration": r.operation_duration,
"errors": r.errors,
"warnings": r.warnings
}
for r in result.results
],
"errors": [
{
"item_id": e.item_id,
"item_type": e.item_type,
"error": e.error
}
for e in result.errors
]
}
@router.post("/assets/bulk-recover")
async def bulk_recover_assets(
request: BulkRecoveryRequest,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Bulk recover multiple assets"""
if not request.asset_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No asset IDs provided"
)
recovery_service = RecoveryService()
result = recovery_service.bulk_recover_assets(request.asset_ids, db, current_user)
# Commit the transaction
db.commit()
return {
"total_items": result.total_items,
"successful_recoveries": result.successful_recoveries,
"failed_recoveries": result.failed_recoveries,
"results": [
{
"success": r.success,
"asset_id": r.asset_id,
"name": r.name,
"recovered_tasks": r.recovered_tasks,
"recovered_submissions": r.recovered_submissions,
"recovered_attachments": r.recovered_attachments,
"recovered_notes": r.recovered_notes,
"recovered_reviews": r.recovered_reviews,
"operation_duration": r.operation_duration,
"errors": r.errors,
"warnings": r.warnings
}
for r in result.results
],
"errors": [
{
"item_id": e.item_id,
"item_type": e.item_type,
"error": e.error
}
for e in result.errors
]
}
@router.get("/recovery-stats/")
async def get_recovery_stats(
project_id: Optional[int] = Query(None, description="Filter by project ID"),
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Get recovery statistics for admin dashboard"""
recovery_service = RecoveryService()
stats = recovery_service.get_recovery_stats(project_id, db)
return {
"deleted_shots_count": stats.deleted_shots_count,
"deleted_assets_count": stats.deleted_assets_count,
"total_deleted_tasks": stats.total_deleted_tasks,
"total_deleted_files": stats.total_deleted_files,
"oldest_deletion_date": stats.oldest_deletion_date
}
# Permanent Delete Endpoints
@router.delete("/shots/{shot_id}/permanent")
async def permanent_delete_shot(
shot_id: int,
request: PermanentDeleteRequest,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Permanently delete a soft-deleted shot and all its related data"""
# Check rate limit
check_permanent_delete_rate_limit(current_user.id)
# Validate confirmation token
validate_confirmation_token(request.confirmation_token, "SHOT")
recovery_service = RecoveryService()
result = recovery_service.permanent_delete_shot(shot_id, db, current_user)
if not result.success:
db.rollback()
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail={
"message": "Failed to permanently delete shot",
"errors": result.errors,
"warnings": result.warnings
}
)
# Commit the transaction
db.commit()
return {
"message": f"Shot '{result.name}' has been permanently deleted",
"shot_id": result.shot_id,
"name": result.name,
"deleted_at": result.deleted_at,
"deleted_by": result.deleted_by,
"deleted_tasks": result.deleted_tasks,
"deleted_submissions": result.deleted_submissions,
"deleted_attachments": result.deleted_attachments,
"deleted_notes": result.deleted_notes,
"deleted_reviews": result.deleted_reviews,
"deleted_files": result.deleted_files,
"database_records_deleted": result.database_records_deleted,
"operation_duration": result.operation_duration,
"warnings": result.warnings
}
@router.delete("/assets/{asset_id}/permanent")
async def permanent_delete_asset(
asset_id: int,
request: PermanentDeleteRequest,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Permanently delete a soft-deleted asset and all its related data"""
# Check rate limit
check_permanent_delete_rate_limit(current_user.id)
# Validate confirmation token
validate_confirmation_token(request.confirmation_token, "ASSET")
recovery_service = RecoveryService()
result = recovery_service.permanent_delete_asset(asset_id, db, current_user)
if not result.success:
db.rollback()
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail={
"message": "Failed to permanently delete asset",
"errors": result.errors,
"warnings": result.warnings
}
)
# Commit the transaction
db.commit()
return {
"message": f"Asset '{result.name}' has been permanently deleted",
"asset_id": result.asset_id,
"name": result.name,
"deleted_at": result.deleted_at,
"deleted_by": result.deleted_by,
"deleted_tasks": result.deleted_tasks,
"deleted_submissions": result.deleted_submissions,
"deleted_attachments": result.deleted_attachments,
"deleted_notes": result.deleted_notes,
"deleted_reviews": result.deleted_reviews,
"deleted_files": result.deleted_files,
"database_records_deleted": result.database_records_deleted,
"operation_duration": result.operation_duration,
"warnings": result.warnings
}
@router.delete("/shots/bulk-permanent")
async def bulk_permanent_delete_shots(
request: BulkPermanentDeleteRequest,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Permanently delete multiple shots in bulk"""
if not request.shot_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No shot IDs provided"
)
# Check rate limit (bulk operations count as multiple operations)
for _ in request.shot_ids:
check_permanent_delete_rate_limit(current_user.id)
# Validate confirmation token
validate_confirmation_token(request.confirmation_token, "BULK_SHOTS")
recovery_service = RecoveryService()
result = recovery_service.bulk_permanent_delete_shots(request.shot_ids, db, current_user)
# Commit the transaction
db.commit()
return {
"message": f"Bulk permanent deletion completed: {result.successful_deletions} successful, {result.failed_deletions} failed",
"total_items": result.total_items,
"successful_deletions": result.successful_deletions,
"failed_deletions": result.failed_deletions,
"deleted_items": result.deleted_items,
"files_deleted": result.files_deleted,
"database_records_deleted": result.database_records_deleted,
"errors": result.errors
}
@router.delete("/assets/bulk-permanent")
async def bulk_permanent_delete_assets(
request: BulkPermanentDeleteRequest,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Permanently delete multiple assets in bulk"""
if not request.asset_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No asset IDs provided"
)
# Check rate limit (bulk operations count as multiple operations)
for _ in request.asset_ids:
check_permanent_delete_rate_limit(current_user.id)
# Validate confirmation token
validate_confirmation_token(request.confirmation_token, "BULK_ASSETS")
recovery_service = RecoveryService()
result = recovery_service.bulk_permanent_delete_assets(request.asset_ids, db, current_user)
# Commit the transaction
db.commit()
return {
"message": f"Bulk permanent deletion completed: {result.successful_deletions} successful, {result.failed_deletions} failed",
"total_items": result.total_items,
"successful_deletions": result.successful_deletions,
"failed_deletions": result.failed_deletions,
"deleted_items": result.deleted_items,
"files_deleted": result.files_deleted,
"database_records_deleted": result.database_records_deleted,
"errors": result.errors
}
# Batch Operations Endpoints
@router.post("/batch-deletion-preview")
async def get_batch_deletion_preview(
request: BatchPreviewRequest,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Get preview information for a batch deletion operation"""
if not request.shot_ids and not request.asset_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No shot or asset IDs provided"
)
batch_service = BatchOperationsService()
preview = batch_service.get_batch_deletion_preview(
request.shot_ids, request.asset_ids, db
)
return preview
@router.post("/shots/batch-delete")
async def batch_delete_shots(
request: BulkDeletionRequest,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Batch delete multiple shots"""
if not request.shot_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No shot IDs provided"
)
batch_service = BatchOperationsService()
result = batch_service.batch_delete_shots(
request.shot_ids, db, current_user, request.batch_size or 50
)
# Commit the transaction
db.commit()
return {
"total_items": result.total_items,
"successful_deletions": result.successful_deletions,
"failed_deletions": result.failed_deletions,
"operation_duration": result.operation_duration,
"total_deleted_tasks": result.total_deleted_tasks,
"total_deleted_submissions": result.total_deleted_submissions,
"total_deleted_attachments": result.total_deleted_attachments,
"total_deleted_notes": result.total_deleted_notes,
"total_deleted_reviews": result.total_deleted_reviews,
"items": [
{
"id": item.id,
"name": item.name,
"type": item.type,
"success": item.success,
"error": item.error,
"deleted_counts": item.deleted_counts
}
for item in result.items
]
}
@router.post("/assets/batch-delete")
async def batch_delete_assets(
request: BulkDeletionRequest,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Batch delete multiple assets"""
if not request.asset_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No asset IDs provided"
)
batch_service = BatchOperationsService()
result = batch_service.batch_delete_assets(
request.asset_ids, db, current_user, request.batch_size or 50
)
# Commit the transaction
db.commit()
return {
"total_items": result.total_items,
"successful_deletions": result.successful_deletions,
"failed_deletions": result.failed_deletions,
"operation_duration": result.operation_duration,
"total_deleted_tasks": result.total_deleted_tasks,
"total_deleted_submissions": result.total_deleted_submissions,
"total_deleted_attachments": result.total_deleted_attachments,
"total_deleted_notes": result.total_deleted_notes,
"total_deleted_reviews": result.total_deleted_reviews,
"items": [
{
"id": item.id,
"name": item.name,
"type": item.type,
"success": item.success,
"error": item.error,
"deleted_counts": item.deleted_counts
}
for item in result.items
]
}
@router.post("/shots/batch-recover")
async def batch_recover_shots_enhanced(
request: BulkRecoveryRequest,
batch_size: Optional[int] = Query(50, description="Batch size for processing"),
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Enhanced batch recover multiple shots with configurable batch size"""
if not request.shot_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No shot IDs provided"
)
batch_service = BatchOperationsService()
result = batch_service.batch_recover_shots(
request.shot_ids, db, current_user, batch_size
)
# Commit the transaction
db.commit()
return {
"total_items": result.total_items,
"successful_recoveries": result.successful_recoveries,
"failed_recoveries": result.failed_recoveries,
"operation_duration": result.operation_duration,
"total_recovered_tasks": result.total_recovered_tasks,
"total_recovered_submissions": result.total_recovered_submissions,
"total_recovered_attachments": result.total_recovered_attachments,
"total_recovered_notes": result.total_recovered_notes,
"total_recovered_reviews": result.total_recovered_reviews,
"items": [
{
"id": item.id,
"name": item.name,
"type": item.type,
"success": item.success,
"error": item.error,
"recovered_counts": item.recovered_counts
}
for item in result.items
]
}
@router.post("/assets/batch-recover")
async def batch_recover_assets_enhanced(
request: BulkRecoveryRequest,
batch_size: Optional[int] = Query(50, description="Batch size for processing"),
db: Session = Depends(get_db),
current_user: User = Depends(require_admin)
):
"""Enhanced batch recover multiple assets with configurable batch size"""
if not request.asset_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No asset IDs provided"
)
batch_service = BatchOperationsService()
result = batch_service.batch_recover_assets(
request.asset_ids, db, current_user, batch_size
)
# Commit the transaction
db.commit()
return {
"total_items": result.total_items,
"successful_recoveries": result.successful_recoveries,
"failed_recoveries": result.failed_recoveries,
"operation_duration": result.operation_duration,
"total_recovered_tasks": result.total_recovered_tasks,
"total_recovered_submissions": result.total_recovered_submissions,
"total_recovered_attachments": result.total_recovered_attachments,
"total_recovered_notes": result.total_recovered_notes,
"total_recovered_reviews": result.total_recovered_reviews,
"items": [
{
"id": item.id,
"name": item.name,
"type": item.type,
"success": item.success,
"error": item.error,
"recovered_counts": item.recovered_counts
}
for item in result.items
]
}
backend/routers/assets.py
+742
View File
@@ -0,0 +1,742 @@
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from typing import List, Dict
from database import get_db
from models.asset import Asset, AssetCategory
from models.project import Project, ProjectMember
from models.task import Task, TaskType, TaskStatus
from models.user import User, UserRole
from schemas.asset import AssetCreate, AssetUpdate, AssetResponse, AssetListResponse, TaskStatusInfo
from schemas.task import TaskCreate
from utils.auth import get_current_user_from_token
from services.asset_soft_deletion import AssetSoftDeletionService
router = APIRouter()
def get_current_user_with_db(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Get current user with proper database dependency."""
from utils.auth import _get_user_from_db
return _get_user_from_db(db, token_data["user_id"])
def require_coordinator_or_admin(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Require coordinator or admin role."""
from utils.auth import _get_user_from_db
current_user = _get_user_from_db(db, token_data["user_id"])
if current_user.role != UserRole.COORDINATOR and not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Insufficient permissions"
)
return current_user
def get_status_sort_order(status: str, project_custom_statuses: list = None) -> int:
"""Get sort order for task status, including custom statuses."""
# Default system status order
system_status_order = {
"not_started": 0,
"in_progress": 1,
"submitted": 2,
"retake": 3,
"approved": 4
}
# If it's a system status, return its order
if status in system_status_order:
return system_status_order[status]
# For custom statuses, use their defined order + offset to place them after system statuses
if project_custom_statuses:
for custom_status in project_custom_statuses:
if isinstance(custom_status, dict) and custom_status.get('id') == status:
# Custom statuses start after system statuses (5+)
return 5 + custom_status.get('order', 0)
# Unknown status defaults to 0 (same as not_started)
return 0
def get_project_custom_statuses(project_id: int, db: Session) -> list:
"""Get custom task statuses for a project."""
project = db.query(Project).filter(Project.id == project_id).first()
if not project or not project.custom_task_statuses:
return []
custom_statuses_data = project.custom_task_statuses
if isinstance(custom_statuses_data, str):
try:
import json
custom_statuses_data = json.loads(custom_statuses_data)
except (json.JSONDecodeError, TypeError):
return []
return custom_statuses_data if isinstance(custom_statuses_data, list) else []
def check_project_access(project_id: int, current_user: User, db: Session):
"""Check if user has access to the project."""
# Check if project exists
project = db.query(Project).filter(Project.id == project_id).first()
if not project:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Project not found"
)
# Check access for artists
if current_user.role == UserRole.ARTIST:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == project_id,
ProjectMember.user_id == current_user.id
).first()
if not member:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this project"
)
return project
# Standard asset task types (read-only)
STANDARD_ASSET_TASK_TYPES = ["modeling", "surfacing", "rigging"]
# Default asset tasks by category (using string values instead of enums)
DEFAULT_ASSET_TASKS = {
AssetCategory.CHARACTERS: [TaskType.MODELING.value, TaskType.SURFACING.value, TaskType.RIGGING.value],
AssetCategory.PROPS: [TaskType.MODELING.value, TaskType.SURFACING.value],
AssetCategory.SETS: [TaskType.MODELING.value, TaskType.SURFACING.value],
AssetCategory.VEHICLES: [TaskType.MODELING.value, TaskType.SURFACING.value, TaskType.RIGGING.value]
}
def get_default_asset_task_types(category: AssetCategory) -> List[str]:
"""Get default task types for an asset category."""
return DEFAULT_ASSET_TASKS.get(category, [])
def get_all_asset_task_types(project_id: int, db: Session) -> List[str]:
"""Get all task types (standard + custom) for assets in a project."""
project = db.query(Project).filter(Project.id == project_id).first()
if not project:
return STANDARD_ASSET_TASK_TYPES
custom_types = project.custom_asset_task_types or []
return STANDARD_ASSET_TASK_TYPES + custom_types
def create_default_tasks_for_asset(asset: Asset, task_types: List[str], db: Session) -> List[Task]:
"""Create default tasks for an asset."""
created_tasks = []
for task_type in task_types:
# Create task name based on type
task_name = f"{asset.name} - {task_type.title()}"
# Create the task
db_task = Task(
project_id=asset.project_id,
asset_id=asset.id,
task_type=task_type,
name=task_name,
description=f"Default {task_type} task for {asset.name}",
status="not_started"
)
db.add(db_task)
created_tasks.append(db_task)
return created_tasks
@router.get("/categories", response_model=List[str])
async def list_asset_categories():
"""List all available asset categories"""
return [category.value for category in AssetCategory]
@router.get("/default-tasks/{category}", response_model=List[str])
async def get_default_tasks_for_category(
category: AssetCategory,
project_id: int = None,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""Get default task types for an asset category (includes custom types if project_id provided)"""
task_types = get_default_asset_task_types(category)
# If project_id is provided, include custom task types
if project_id:
all_types = get_all_asset_task_types(project_id, db)
# Return only the types that are relevant for this category
# For now, return all available types (standard + custom)
return all_types
return task_types # task_types are already strings, no need for .value
@router.get("/", response_model=List[AssetListResponse])
async def list_assets(
project_id: int = None,
category: AssetCategory = None,
task_status_filter: str = None,
sort_by: str = None,
sort_direction: str = "asc",
skip: int = 0,
limit: int = 100,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""List assets with optional filtering by project and category"""
from sqlalchemy.orm import joinedload, selectinload
# Build base query for assets (exclude soft deleted)
base_query = db.query(Asset).filter(Asset.deleted_at.is_(None))
# Filter by project if specified
if project_id:
check_project_access(project_id, current_user, db)
base_query = base_query.filter(Asset.project_id == project_id)
else:
# If no project specified, filter by user's accessible projects for artists
if current_user.role == UserRole.ARTIST:
accessible_projects = db.query(ProjectMember.project_id).filter(
ProjectMember.user_id == current_user.id
).subquery()
base_query = base_query.filter(Asset.project_id.in_(accessible_projects))
# Filter by category if specified
if category:
base_query = base_query.filter(Asset.category == category)
# Apply sorting if specified (for non-task-status fields)
if sort_by and not sort_by.endswith('_status'):
if sort_by in ['name', 'category', 'status', 'created_at', 'updated_at']:
sort_column = getattr(Asset, sort_by)
if sort_direction.lower() == 'desc':
base_query = base_query.order_by(sort_column.desc())
else:
base_query = base_query.order_by(sort_column.asc())
# OPTIMIZATION: Use single query with optimized JOIN to fetch assets and their tasks
# This replaces the N+1 query pattern with a single database operation
assets_with_tasks = (
base_query
.outerjoin(Task, (Task.asset_id == Asset.id) & (Task.deleted_at.is_(None)))
.options(
joinedload(Asset.project), # Eager load project
selectinload(Asset.tasks).options( # Use selectinload for better performance with tasks
selectinload(Task.assigned_user) # Eager load assigned users
)
)
.add_columns(
Task.id.label('task_id'),
Task.task_type,
Task.status.label('task_status'),
Task.assigned_user_id,
Task.updated_at.label('task_updated_at') # Include task update time for better tracking
)
.offset(skip)
.limit(limit)
.all()
)
# OPTIMIZATION: Pre-fetch all project data and task types in a single query
# This eliminates the need for repeated project queries
project_ids = set()
for row in assets_with_tasks:
asset = row[0]
if asset.project_id not in project_ids:
project_ids.add(asset.project_id)
# Get all projects with their custom task types in one optimized query
project_data = {}
if project_ids:
projects = (
db.query(Project)
.filter(Project.id.in_(project_ids))
.all()
)
for project in projects:
custom_types = project.custom_asset_task_types or []
project_data[project.id] = {
'task_types': STANDARD_ASSET_TASK_TYPES + custom_types,
'custom_statuses': get_project_custom_statuses(project.id, db)
}
# OPTIMIZATION: Group results by asset and aggregate task data efficiently
assets_dict = {}
for row in assets_with_tasks:
asset = row[0] # Asset object
task_id = row[1] # task_id
task_type = row[2] # task_type
task_status = row[3] # task_status
assigned_user_id = row[4] # assigned_user_id
task_updated_at = row[5] # task_updated_at
if asset.id not in assets_dict:
# Initialize asset data with pre-fetched project data
project_info = project_data.get(asset.project_id, {
'task_types': STANDARD_ASSET_TASK_TYPES,
'custom_statuses': []
})
assets_dict[asset.id] = {
'asset': asset,
'tasks': [],
'task_status': {},
'task_details': [],
'project_info': project_info
}
# Initialize all task types as not started using pre-fetched data
for task_type_init in project_info['task_types']:
assets_dict[asset.id]['task_status'][task_type_init] = "not_started"
# Add task data if task exists
if task_id is not None:
assets_dict[asset.id]['tasks'].append({
'task_id': task_id,
'task_type': task_type,
'status': task_status,
'assigned_user_id': assigned_user_id,
'updated_at': task_updated_at
})
# Update task status
assets_dict[asset.id]['task_status'][task_type] = task_status
# Add to task details with enhanced information
assets_dict[asset.id]['task_details'].append(TaskStatusInfo(
task_type=task_type,
status=task_status,
task_id=task_id,
assigned_user_id=assigned_user_id
))
# Build response list efficiently
result = []
for asset_data in assets_dict.values():
asset = asset_data['asset']
# Create asset response with optimized data
asset_response = AssetListResponse.model_validate(asset)
asset_response.task_count = len(asset_data['tasks'])
asset_response.task_status = asset_data['task_status']
asset_response.task_details = asset_data['task_details']
result.append(asset_response)
# Apply task status filtering if specified
if task_status_filter:
try:
# Parse task status filter (format: "task_type:status")
task_type, status = task_status_filter.split(":")
filter_status = status # Use string directly instead of enum
result = [
asset for asset in result
if asset.task_status.get(task_type) == filter_status
]
except (ValueError, KeyError):
# Invalid filter format, ignore
pass
# Apply task status sorting if specified
if sort_by and sort_by.endswith('_status'):
task_type = sort_by.replace('_status', '')
# Get custom statuses for proper sorting using pre-fetched data
def get_status_order(asset):
status = asset.task_status.get(task_type, "not_started")
# Use pre-fetched custom statuses from project_data
asset_project_data = project_data.get(getattr(asset, 'project_id', None), {})
custom_statuses = asset_project_data.get('custom_statuses', [])
return get_status_sort_order(status, custom_statuses)
reverse = sort_direction.lower() == 'desc'
result.sort(key=get_status_order, reverse=reverse)
return result
@router.post("/", response_model=AssetResponse, status_code=status.HTTP_201_CREATED)
async def create_asset(
asset: AssetCreate,
project_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Create a new asset in a project with optional default tasks"""
# Check project access
check_project_access(project_id, current_user, db)
# Check if asset name already exists in project (exclude soft deleted)
existing_asset = db.query(Asset).filter(
Asset.project_id == project_id,
Asset.name == asset.name,
Asset.deleted_at.is_(None)
).first()
if existing_asset:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Asset with this name already exists in the project"
)
# Create new asset (exclude fields that don't belong to Asset model)
asset_data = asset.model_dump(exclude={'create_default_tasks', 'selected_task_types'})
db_asset = Asset(
project_id=project_id,
**asset_data
)
db.add(db_asset)
db.flush() # Flush to get the asset ID
# Create default tasks if requested
task_count = 0
if asset.create_default_tasks:
# Determine which task types to create
if asset.selected_task_types:
# Use the selected task types (already strings, can include custom types)
task_types = asset.selected_task_types
# Validate that all selected task types are valid (standard or custom)
all_valid_types = get_all_asset_task_types(project_id, db)
invalid_types = [t for t in task_types if t not in all_valid_types]
if invalid_types:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Invalid task types: {', '.join(invalid_types)}"
)
else:
# Use default task types for the asset category
task_types = get_default_asset_task_types(asset.category)
# Create the tasks
created_tasks = create_default_tasks_for_asset(db_asset, task_types, db)
task_count = len(created_tasks)
db.commit()
db.refresh(db_asset)
# Add task count
asset_data = AssetResponse.model_validate(db_asset)
asset_data.task_count = task_count
return asset_data
@router.get("/{asset_id}", response_model=AssetResponse)
async def get_asset(
asset_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""Get a specific asset by ID"""
from sqlalchemy.orm import joinedload, selectinload
# OPTIMIZATION: Use single query with optimized JOINs to fetch asset and all related data
# This replaces separate queries with a single database operation
asset_query = (
db.query(Asset)
.options(
joinedload(Asset.project), # Eager load project
selectinload(Asset.tasks).options( # Use selectinload for better performance with tasks
selectinload(Task.assigned_user) # Eager load assigned users if needed
)
)
.filter(Asset.id == asset_id, Asset.deleted_at.is_(None))
)
asset = asset_query.first()
if not asset:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Asset not found"
)
# Check project access
check_project_access(asset.project_id, current_user, db)
# OPTIMIZATION: Count tasks from the already loaded relationship
# This avoids a separate COUNT query
active_tasks = [task for task in asset.tasks if task.deleted_at is None]
task_count = len(active_tasks)
asset_data = AssetResponse.model_validate(asset)
asset_data.task_count = task_count
return asset_data
@router.get("/{asset_id}/task-status", response_model=List[TaskStatusInfo])
async def get_asset_task_status(
asset_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""Get detailed task status for a specific asset"""
# Exclude soft deleted assets
asset = db.query(Asset).filter(
Asset.id == asset_id,
Asset.deleted_at.is_(None)
).first()
if not asset:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Asset not found"
)
# Check project access
check_project_access(asset.project_id, current_user, db)
# Get all active tasks for this asset (exclude soft deleted)
tasks = db.query(Task).filter(
Task.asset_id == asset.id,
Task.deleted_at.is_(None)
).all()
# Build detailed task status information
task_details = []
for task in tasks:
task_details.append(TaskStatusInfo(
task_type=task.task_type,
status=task.status,
task_id=task.id,
assigned_user_id=task.assigned_user_id
))
return task_details
@router.post("/{asset_id}/tasks", response_model=TaskStatusInfo, status_code=status.HTTP_201_CREATED)
async def create_asset_task(
asset_id: int,
task_type: str, # Changed from TaskType enum to str
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Create a new task for an asset"""
# Exclude soft deleted assets
asset = db.query(Asset).filter(
Asset.id == asset_id,
Asset.deleted_at.is_(None)
).first()
if not asset:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Asset not found"
)
# Check project access
check_project_access(asset.project_id, current_user, db)
# Check if task already exists (exclude soft deleted)
existing_task = db.query(Task).filter(
Task.asset_id == asset_id,
Task.task_type == task_type,
Task.deleted_at.is_(None)
).first()
if existing_task:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Task already exists for this asset and task type"
)
# Create the task
task_name = f"{asset.name} - {task_type.title()}"
db_task = Task(
project_id=asset.project_id,
asset_id=asset.id,
task_type=task_type,
name=task_name,
description=f"{task_type.title()} task for {asset.name}",
status="not_started"
)
db.add(db_task)
db.commit()
db.refresh(db_task)
return TaskStatusInfo(
task_type=db_task.task_type,
status=db_task.status,
task_id=db_task.id,
assigned_user_id=db_task.assigned_user_id
)
@router.put("/{asset_id}", response_model=AssetResponse)
async def update_asset(
asset_id: int,
asset_update: AssetUpdate,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Update an asset"""
# Exclude soft deleted assets
db_asset = db.query(Asset).filter(
Asset.id == asset_id,
Asset.deleted_at.is_(None)
).first()
if not db_asset:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Asset not found"
)
# Check project access
check_project_access(db_asset.project_id, current_user, db)
# Check if new name conflicts with existing assets in the same project
if asset_update.name and asset_update.name != db_asset.name:
existing_asset = db.query(Asset).filter(
Asset.project_id == db_asset.project_id,
Asset.name == asset_update.name,
Asset.id != asset_id,
Asset.deleted_at.is_(None)
).first()
if existing_asset:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Asset with this name already exists in the project"
)
# Update only provided fields
update_data = asset_update.model_dump(exclude_unset=True)
for field, value in update_data.items():
setattr(db_asset, field, value)
db.commit()
db.refresh(db_asset)
# Add task count (exclude soft deleted tasks)
task_count = db.query(Task).filter(
Task.asset_id == db_asset.id,
Task.deleted_at.is_(None)
).count()
asset_data = AssetResponse.model_validate(db_asset)
asset_data.task_count = task_count
return asset_data
@router.get("/{asset_id}/deletion-info")
async def get_asset_deletion_info(
asset_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Get information about what will be deleted when deleting an asset"""
# Exclude soft deleted assets
db_asset = db.query(Asset).filter(
Asset.id == asset_id,
Asset.deleted_at.is_(None)
).first()
if not db_asset:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Asset not found"
)
# Check project access
check_project_access(db_asset.project_id, current_user, db)
# Use the soft deletion service to get comprehensive deletion info
deletion_service = AssetSoftDeletionService()
deletion_info = deletion_service.get_deletion_info(asset_id, db)
if not deletion_info:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Asset not found"
)
return {
"asset_id": deletion_info.asset_id,
"asset_name": deletion_info.asset_name,
"asset_category": deletion_info.asset_category,
"project_name": deletion_info.project_name,
"task_count": deletion_info.task_count,
"submission_count": deletion_info.submission_count,
"attachment_count": deletion_info.attachment_count,
"note_count": deletion_info.note_count,
"review_count": deletion_info.review_count,
"total_file_size": deletion_info.total_file_size,
"file_count": deletion_info.file_count,
"affected_users": deletion_info.affected_users,
"last_activity_date": deletion_info.last_activity_date,
"created_at": deletion_info.created_at
}
@router.delete("/{asset_id}")
async def delete_asset(
asset_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Soft delete an asset and all its associated data"""
# Exclude soft deleted assets
db_asset = db.query(Asset).filter(
Asset.id == asset_id,
Asset.deleted_at.is_(None)
).first()
if not db_asset:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Asset not found"
)
# Check project access
check_project_access(db_asset.project_id, current_user, db)
# Use the soft deletion service to perform cascading soft deletion
deletion_service = AssetSoftDeletionService()
result = deletion_service.soft_delete_asset_cascade(asset_id, db, current_user)
if not result.success:
db.rollback()
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail={
"message": "Failed to delete asset",
"errors": result.errors
}
)
# Commit the transaction
db.commit()
return {
"message": f"Asset '{result.asset_name}' and all related data have been deleted",
"asset_id": result.asset_id,
"asset_name": result.asset_name,
"deleted_at": result.deleted_at,
"deleted_by": result.deleted_by,
"marked_deleted_tasks": result.marked_deleted_tasks,
"marked_deleted_submissions": result.marked_deleted_submissions,
"marked_deleted_attachments": result.marked_deleted_attachments,
"marked_deleted_notes": result.marked_deleted_notes,
"marked_deleted_reviews": result.marked_deleted_reviews,
"operation_duration": result.operation_duration
}
backend/routers/auth.py
+572
View File
@@ -0,0 +1,572 @@
from fastapi import APIRouter, Depends, HTTPException, status, Request
from fastapi.security import HTTPAuthorizationCredentials
from sqlalchemy.orm import Session
from datetime import timedelta
from typing import List
import json
from database import get_db
from models.user import User, UserRole
from models.api_key import APIKey, APIKeyScope
from models.api_key_usage import APIKeyUsage
from schemas.auth import UserLogin, UserRegister, Token, RefreshToken
from schemas.api_key import APIKeyCreate, APIKeyResponse, APIKeyWithToken, APIKeyUpdate, APIKeyUsageLog
from utils.auth import (
verify_password,
get_password_hash,
create_access_token,
create_refresh_token,
verify_token,
security,
ACCESS_TOKEN_EXPIRE_MINUTES,
REFRESH_TOKEN_EXPIRE_DAYS,
generate_api_key,
hash_api_key,
get_current_user_flexible,
require_role
)
router = APIRouter()
@router.post("/register", response_model=dict)
async def register(user_data: UserRegister, db: Session = Depends(get_db)):
"""Register a new user account."""
# Check if user already exists
existing_user = db.query(User).filter(User.email == user_data.email).first()
if existing_user:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Email already registered"
)
# Create new user
hashed_password = get_password_hash(user_data.password)
new_user = User(
email=user_data.email,
password_hash=hashed_password,
first_name=user_data.first_name,
last_name=user_data.last_name,
role=UserRole.ARTIST, # Default role
is_approved=False # Requires admin approval
)
db.add(new_user)
db.commit()
db.refresh(new_user)
return {
"message": "User registered successfully. Awaiting admin approval.",
"user_id": new_user.id
}
@router.post("/login", response_model=Token)
async def login(user_credentials: UserLogin, db: Session = Depends(get_db)):
"""Authenticate user and return JWT tokens."""
# Find user by email
print(user_credentials.email)
user = db.query(User).filter(User.email == user_credentials.email).first()
if not user or not verify_password(user_credentials.password, user.password_hash):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Incorrect email or password"
)
# Check if user is approved
if not user.is_approved:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Account not approved by administrator"
)
# Create tokens
access_token_expires = timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
refresh_token_expires = timedelta(days=REFRESH_TOKEN_EXPIRE_DAYS)
token_data = {"sub": str(user.id), "email": user.email, "role": user.role}
access_token = create_access_token(
data=token_data,
expires_delta=access_token_expires
)
refresh_token = create_refresh_token(
data=token_data,
expires_delta=refresh_token_expires
)
return Token(
access_token=access_token,
refresh_token=refresh_token
)
@router.post("/refresh", response_model=Token)
async def refresh_token(refresh_data: RefreshToken, db: Session = Depends(get_db)):
"""Refresh access token using refresh token."""
# Verify refresh token
payload = verify_token(refresh_data.refresh_token, "refresh")
if payload is None:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid refresh token"
)
# Get user from database
user_id = payload.get("sub")
user = db.query(User).filter(User.id == user_id).first()
if not user or not user.is_approved:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="User not found or not approved"
)
# Create new tokens
access_token_expires = timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
refresh_token_expires = timedelta(days=REFRESH_TOKEN_EXPIRE_DAYS)
token_data = {"sub": str(user.id), "email": user.email, "role": user.role}
new_access_token = create_access_token(
data=token_data,
expires_delta=access_token_expires
)
new_refresh_token = create_refresh_token(
data=token_data,
expires_delta=refresh_token_expires
)
return Token(
access_token=new_access_token,
refresh_token=new_refresh_token
)
@router.post("/logout")
async def logout():
"""Logout user (client should discard tokens)."""
return {"message": "Successfully logged out"}
# API Key Management Endpoints
@router.post("/api-keys", response_model=APIKeyWithToken)
async def create_api_key(
api_key_data: APIKeyCreate,
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_flexible)
):
"""Create a new API key. Only developers and admins can create API keys."""
# Check if user has permission to create API keys
if current_user.role != UserRole.DEVELOPER and not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers and users with admin permission can create API keys"
)
# Determine target user for the API key
target_user_id = current_user.id # Default to current user
target_user = current_user
# If user_id is specified and current user has admin permission, allow creating for other users
if api_key_data.user_id is not None:
if not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only users with admin permission can create API keys for other users"
)
# Verify target user exists and is approved
target_user = db.query(User).filter(User.id == api_key_data.user_id).first()
if not target_user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Target user not found"
)
if not target_user.is_approved:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Cannot create API key for unapproved user"
)
target_user_id = target_user.id
# Generate API key
api_key_token = generate_api_key()
key_hash = hash_api_key(api_key_token)
# Convert scopes to JSON string
scopes_json = json.dumps([scope.value for scope in api_key_data.scopes])
# Create API key record
new_api_key = APIKey(
user_id=target_user_id,
key_hash=key_hash,
name=api_key_data.name,
scopes=scopes_json,
expires_at=api_key_data.expires_at
)
db.add(new_api_key)
db.commit()
db.refresh(new_api_key)
# Convert scopes back to list for response
scopes_list = json.loads(new_api_key.scopes)
api_key_response = APIKeyResponse(
id=new_api_key.id,
user_id=new_api_key.user_id,
name=new_api_key.name,
scopes=scopes_list,
is_active=new_api_key.is_active,
expires_at=new_api_key.expires_at,
last_used_at=new_api_key.last_used_at,
created_at=new_api_key.created_at,
user_email=target_user.email
)
return APIKeyWithToken(
api_key=api_key_response,
token=api_key_token
)
@router.get("/api-keys", response_model=List[APIKeyResponse])
async def list_api_keys(
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_flexible),
user_id: int = None
):
"""List API keys. Developers see their own, admins can see all or filter by user."""
# Developers and users with admin permission can see API keys
if current_user.role != UserRole.DEVELOPER and not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers and users with admin permission can manage API keys"
)
# Build query based on user permissions and parameters
if current_user.is_admin:
if user_id is not None:
# Admin requesting specific user's API keys
api_keys = db.query(APIKey).filter(APIKey.user_id == user_id).all()
else:
# Admin requesting all API keys
api_keys = db.query(APIKey).all()
else:
# Developer can only see their own API keys
if user_id is not None and user_id != current_user.id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Developers can only view their own API keys"
)
api_keys = db.query(APIKey).filter(APIKey.user_id == current_user.id).all()
result = []
for api_key in api_keys:
scopes_list = json.loads(api_key.scopes)
# Get user email for admin view
user_email = None
if current_user.is_admin:
user = db.query(User).filter(User.id == api_key.user_id).first()
if user:
user_email = user.email
result.append(APIKeyResponse(
id=api_key.id,
user_id=api_key.user_id,
name=api_key.name,
scopes=scopes_list,
is_active=api_key.is_active,
expires_at=api_key.expires_at,
last_used_at=api_key.last_used_at,
created_at=api_key.created_at,
user_email=user_email
))
return result
@router.put("/api-keys/{api_key_id}", response_model=APIKeyResponse)
async def update_api_key(
api_key_id: int,
api_key_data: APIKeyUpdate,
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_flexible)
):
"""Update an API key."""
# Check if user has permission
if current_user.role != UserRole.DEVELOPER and not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers and users with admin permission can manage API keys"
)
# Get API key with different access rules for admin vs developer
if current_user.is_admin:
# Users with admin permission can update any API key
api_key = db.query(APIKey).filter(APIKey.id == api_key_id).first()
else:
# Developers can only update their own API keys
api_key = db.query(APIKey).filter(
APIKey.id == api_key_id,
APIKey.user_id == current_user.id
).first()
if not api_key:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="API key not found"
)
# Update fields
if api_key_data.name is not None:
api_key.name = api_key_data.name
if api_key_data.scopes is not None:
scopes_json = json.dumps([scope.value for scope in api_key_data.scopes])
api_key.scopes = scopes_json
if api_key_data.is_active is not None:
api_key.is_active = api_key_data.is_active
if api_key_data.expires_at is not None:
api_key.expires_at = api_key_data.expires_at
db.commit()
db.refresh(api_key)
# Convert scopes back to list for response
scopes_list = json.loads(api_key.scopes)
# Get user email for admin view
user_email = None
if current_user.is_admin:
user = db.query(User).filter(User.id == api_key.user_id).first()
if user:
user_email = user.email
return APIKeyResponse(
id=api_key.id,
user_id=api_key.user_id,
name=api_key.name,
scopes=scopes_list,
is_active=api_key.is_active,
expires_at=api_key.expires_at,
last_used_at=api_key.last_used_at,
created_at=api_key.created_at,
user_email=user_email
)
@router.delete("/api-keys/{api_key_id}")
async def delete_api_key(
api_key_id: int,
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_flexible)
):
"""Delete (revoke) an API key."""
# Check if user has permission
if current_user.role != UserRole.DEVELOPER and not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers and users with admin permission can manage API keys"
)
# Get API key with different access rules for admin vs developer
if current_user.is_admin:
# Users with admin permission can delete any API key
api_key = db.query(APIKey).filter(APIKey.id == api_key_id).first()
else:
# Developers can only delete their own API keys
api_key = db.query(APIKey).filter(
APIKey.id == api_key_id,
APIKey.user_id == current_user.id
).first()
if not api_key:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="API key not found"
)
# Delete the API key
db.delete(api_key)
db.commit()
return {"message": "API key revoked successfully"}
@router.get("/api-keys/{api_key_id}/usage", response_model=List[APIKeyUsageLog])
async def get_api_key_usage(
api_key_id: int,
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_flexible),
limit: int = 100
):
"""Get usage logs for an API key."""
# Check if user has permission
if current_user.role != UserRole.DEVELOPER and not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers and users with admin permission can view API key usage"
)
# Verify API key access with different rules for admin vs developer
if current_user.is_admin:
# Users with admin permission can view usage for any API key
api_key = db.query(APIKey).filter(APIKey.id == api_key_id).first()
else:
# Developers can only view usage for their own API keys
api_key = db.query(APIKey).filter(
APIKey.id == api_key_id,
APIKey.user_id == current_user.id
).first()
if not api_key:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="API key not found"
)
# Get usage logs
usage_logs = db.query(APIKeyUsage).filter(
APIKeyUsage.api_key_id == api_key_id
).order_by(APIKeyUsage.timestamp.desc()).limit(limit).all()
return [
APIKeyUsageLog(
api_key_id=log.api_key_id,
endpoint=log.endpoint,
method=log.method,
timestamp=log.timestamp,
ip_address=log.ip_address,
user_agent=log.user_agent
)
for log in usage_logs
]
# Admin-only endpoints for API key management
@router.get("/admin/users/{user_id}/api-keys", response_model=List[APIKeyResponse])
async def list_user_api_keys_admin(
user_id: int,
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_flexible)
):
"""Admin endpoint to list API keys for a specific user."""
# Only users with admin permission can access this endpoint
if not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Admin permission required to access this endpoint"
)
# Verify target user exists
target_user = db.query(User).filter(User.id == user_id).first()
if not target_user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="User not found"
)
# Get API keys for the specified user
api_keys = db.query(APIKey).filter(APIKey.user_id == user_id).all()
result = []
for api_key in api_keys:
scopes_list = json.loads(api_key.scopes)
result.append(APIKeyResponse(
id=api_key.id,
user_id=api_key.user_id,
name=api_key.name,
scopes=scopes_list,
is_active=api_key.is_active,
expires_at=api_key.expires_at,
last_used_at=api_key.last_used_at,
created_at=api_key.created_at,
user_email=target_user.email
))
return result
@router.post("/admin/users/{user_id}/api-keys", response_model=APIKeyWithToken)
async def create_api_key_for_user_admin(
user_id: int,
api_key_data: APIKeyCreate,
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_flexible)
):
"""Admin endpoint to create an API key for a specific user."""
# Only users with admin permission can access this endpoint
if not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Admin permission required to access this endpoint"
)
# Verify target user exists and is approved
target_user = db.query(User).filter(User.id == user_id).first()
if not target_user:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="User not found"
)
if not target_user.is_approved:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Cannot create API key for unapproved user"
)
# Generate API key
api_key_token = generate_api_key()
key_hash = hash_api_key(api_key_token)
# Convert scopes to JSON string
scopes_json = json.dumps([scope.value for scope in api_key_data.scopes])
# Create API key record
new_api_key = APIKey(
user_id=user_id,
key_hash=key_hash,
name=api_key_data.name,
scopes=scopes_json,
expires_at=api_key_data.expires_at
)
db.add(new_api_key)
db.commit()
db.refresh(new_api_key)
# Convert scopes back to list for response
scopes_list = json.loads(new_api_key.scopes)
api_key_response = APIKeyResponse(
id=new_api_key.id,
user_id=new_api_key.user_id,
name=new_api_key.name,
scopes=scopes_list,
is_active=new_api_key.is_active,
expires_at=new_api_key.expires_at,
last_used_at=new_api_key.last_used_at,
created_at=new_api_key.created_at,
user_email=target_user.email
)
return APIKeyWithToken(
api_key=api_key_response,
token=api_key_token
)
backend/routers/data_consistency.py
+187
View File
@@ -0,0 +1,187 @@
"""
Data Consistency API endpoints for validating and monitoring task aggregation consistency.
"""
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy.orm import Session
from typing import List, Optional, Dict, Any
from database import get_db
from models.user import User, UserRole
from services.data_consistency import create_data_consistency_service
from utils.auth import get_current_user_from_token, _get_user_from_db
router = APIRouter()
def get_current_user_with_db(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Get current user with proper database dependency."""
return _get_user_from_db(db, token_data["user_id"])
def require_admin_or_coordinator(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Require admin or coordinator role for consistency operations."""
current_user = _get_user_from_db(db, token_data["user_id"])
if current_user.role not in [UserRole.COORDINATOR] and not current_user.is_admin:
raise HTTPException(
status_code=403,
detail="Admin or Coordinator role required for consistency operations"
)
return current_user
@router.get("/validate/{entity_type}/{entity_id}")
async def validate_entity_consistency(
entity_type: str,
entity_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""
Validate task aggregation consistency for a specific shot or asset.
Args:
entity_type: 'shot' or 'asset'
entity_id: ID of the shot or asset
"""
if entity_type not in ['shot', 'asset']:
raise HTTPException(
status_code=400,
detail="entity_type must be 'shot' or 'asset'"
)
consistency_service = create_data_consistency_service(db)
try:
result = consistency_service.validate_task_aggregation_consistency(entity_id, entity_type)
return result
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=f"Validation failed: {str(e)}")
@router.post("/validate/bulk")
async def validate_bulk_consistency(
entity_ids: List[int],
entity_type: str,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin_or_coordinator)
):
"""
Validate task aggregation consistency for multiple shots or assets.
Args:
entity_ids: List of shot or asset IDs
entity_type: 'shot' or 'asset'
"""
if entity_type not in ['shot', 'asset']:
raise HTTPException(
status_code=400,
detail="entity_type must be 'shot' or 'asset'"
)
if len(entity_ids) > 100:
raise HTTPException(
status_code=400,
detail="Maximum 100 entities can be validated at once"
)
consistency_service = create_data_consistency_service(db)
try:
result = consistency_service.validate_bulk_consistency(entity_ids, entity_type)
return result
except Exception as e:
raise HTTPException(status_code=500, detail=f"Bulk validation failed: {str(e)}")
@router.get("/report")
async def get_consistency_report(
project_id: Optional[int] = Query(None, description="Filter by project ID"),
db: Session = Depends(get_db),
current_user: User = Depends(require_admin_or_coordinator)
):
"""
Generate a comprehensive consistency report for shots and assets.
Args:
project_id: Optional project ID to filter by
"""
consistency_service = create_data_consistency_service(db)
try:
result = consistency_service.get_consistency_report(project_id)
return result
except Exception as e:
raise HTTPException(status_code=500, detail=f"Report generation failed: {str(e)}")
@router.post("/propagate/{task_id}")
async def propagate_task_update(
task_id: int,
old_status: Optional[str] = None,
new_status: Optional[str] = None,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin_or_coordinator)
):
"""
Manually propagate a task update and validate consistency.
Args:
task_id: ID of the task to propagate
old_status: Previous task status (optional)
new_status: New task status (optional)
"""
consistency_service = create_data_consistency_service(db)
try:
result = consistency_service.propagate_task_update(task_id, old_status, new_status)
return result
except Exception as e:
raise HTTPException(status_code=500, detail=f"Task propagation failed: {str(e)}")
@router.get("/health")
async def consistency_health_check(
project_id: Optional[int] = Query(None, description="Filter by project ID"),
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""
Quick health check for data consistency across the system.
Args:
project_id: Optional project ID to filter by
"""
consistency_service = create_data_consistency_service(db)
try:
report = consistency_service.get_consistency_report(project_id)
# Extract key health metrics
summary = report['summary']
health_status = "healthy" if summary['consistency_percentage'] >= 95 else "degraded" if summary['consistency_percentage'] >= 80 else "unhealthy"
return {
'status': health_status,
'consistency_percentage': summary['consistency_percentage'],
'total_entities': summary['total_entities'],
'valid_entities': summary['valid_entities'],
'invalid_entities': summary['invalid_entities'],
'total_inconsistencies': summary['total_inconsistencies'],
'project_id': project_id,
'timestamp': report['report_timestamp']
}
except Exception as e:
return {
'status': 'error',
'error': str(e),
'timestamp': None
}
backend/routers/developer.py
+270
View File
@@ -0,0 +1,270 @@
from fastapi import APIRouter, Depends, HTTPException, status, Request
from sqlalchemy.orm import Session
from typing import List
from database import get_db
from models.user import User, UserRole
from models.project import Project, ProjectMember
from models.task import Task, Submission
from models.api_key import APIKey
from models.api_key_usage import APIKeyUsage
from schemas.project import ProjectResponse
from schemas.task import TaskResponse, SubmissionResponse
from schemas.api_key import APIKeyUsageLog
from utils.auth import get_current_user_flexible, require_api_key_scope
router = APIRouter()
@router.get("/projects", response_model=List[ProjectResponse])
async def get_all_projects_for_developer(
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(require_api_key_scope("read:projects"))
):
"""Get read-only access to all projects for developers."""
# Ensure user is a developer
if current_user.role != UserRole.DEVELOPER:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers can access this endpoint"
)
# Get all projects
projects = db.query(Project).all()
result = []
for project in projects:
# Get member count
member_count = db.query(ProjectMember).filter(ProjectMember.project_id == project.id).count()
result.append(ProjectResponse(
id=project.id,
name=project.name,
description=project.description,
status=project.status,
start_date=project.start_date,
end_date=project.end_date,
created_at=project.created_at,
updated_at=project.updated_at,
member_count=member_count
))
return result
@router.get("/tasks", response_model=List[TaskResponse])
async def get_all_tasks_for_developer(
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(require_api_key_scope("read:tasks")),
project_id: int = None,
limit: int = 100,
offset: int = 0
):
"""Get read-only access to all tasks across projects for developers."""
# Ensure user is a developer
if current_user.role != UserRole.DEVELOPER:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers can access this endpoint"
)
# Build query
query = db.query(Task)
# Filter by project if specified
if project_id:
query = query.filter(Task.project_id == project_id)
# Apply pagination
tasks = query.offset(offset).limit(limit).all()
result = []
for task in tasks:
# Get assigned user info
assigned_user = None
if task.assigned_user_id:
user = db.query(User).filter(User.id == task.assigned_user_id).first()
if user:
assigned_user = {
"id": user.id,
"first_name": user.first_name,
"last_name": user.last_name,
"email": user.email
}
# Get project info
project = db.query(Project).filter(Project.id == task.project_id).first()
project_info = None
if project:
project_info = {
"id": project.id,
"name": project.name
}
result.append(TaskResponse(
id=task.id,
project_id=task.project_id,
episode_id=task.episode_id,
shot_id=task.shot_id,
asset_id=task.asset_id,
assigned_user_id=task.assigned_user_id,
task_type=task.task_type,
name=task.name,
description=task.description,
status=task.status,
deadline=task.deadline,
created_at=task.created_at,
updated_at=task.updated_at,
assigned_user=assigned_user,
project=project_info
))
return result
@router.get("/submissions", response_model=List[SubmissionResponse])
async def get_all_submissions_for_developer(
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(require_api_key_scope("read:submissions")),
project_id: int = None,
task_id: int = None,
limit: int = 100,
offset: int = 0
):
"""Get read-only access to all submissions for developers."""
# Ensure user is a developer
if current_user.role != UserRole.DEVELOPER:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers can access this endpoint"
)
# Build query
query = db.query(Submission).filter(Submission.deleted_at.is_(None))
# Filter by task if specified
if task_id:
query = query.filter(Submission.task_id == task_id)
elif project_id:
# Filter by project through task relationship
query = query.join(Task).filter(
Task.project_id == project_id,
Task.deleted_at.is_(None)
)
# Apply pagination
submissions = query.offset(offset).limit(limit).all()
result = []
for submission in submissions:
# Get task info
task = db.query(Task).filter(Task.id == submission.task_id).first()
task_info = None
if task:
task_info = {
"id": task.id,
"name": task.name,
"task_type": task.task_type,
"project_id": task.project_id
}
# Get user info
user = db.query(User).filter(User.id == submission.user_id).first()
user_info = None
if user:
user_info = {
"id": user.id,
"first_name": user.first_name,
"last_name": user.last_name,
"email": user.email
}
result.append(SubmissionResponse(
id=submission.id,
task_id=submission.task_id,
user_id=submission.user_id,
file_path=submission.file_path,
file_name=submission.file_name,
version_number=submission.version_number,
notes=submission.notes,
submitted_at=submission.submitted_at,
task=task_info,
user=user_info
))
return result
@router.get("/api-usage", response_model=List[APIKeyUsageLog])
async def get_api_usage_logs(
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_flexible),
limit: int = 100,
offset: int = 0
):
"""Get API key usage logs for the current developer."""
# Ensure user is a developer
if current_user.role != UserRole.DEVELOPER:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers can access this endpoint"
)
# Get usage logs for all API keys belonging to the current user
usage_logs = db.query(APIKeyUsage).join(
APIKey, APIKeyUsage.api_key_id == APIKey.id
).filter(
APIKey.user_id == current_user.id
).order_by(
APIKeyUsage.timestamp.desc()
).offset(offset).limit(limit).all()
return [
APIKeyUsageLog(
api_key_id=log.api_key_id,
endpoint=log.endpoint,
method=log.method,
timestamp=log.timestamp,
ip_address=log.ip_address,
user_agent=log.user_agent
)
for log in usage_logs
]
@router.get("/stats")
async def get_developer_stats(
request: Request,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_flexible)
):
"""Get statistics for developer dashboard."""
# Ensure user is a developer
if current_user.role != UserRole.DEVELOPER:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only developers can access this endpoint"
)
# Get various counts
total_projects = db.query(Project).count()
total_tasks = db.query(Task).count()
total_submissions = db.query(Submission).count()
# Get API key usage count for current user
api_usage_count = db.query(APIKeyUsage).join(
APIKey, APIKeyUsage.api_key_id == APIKey.id
).filter(
APIKey.user_id == current_user.id
).count()
return {
"total_projects": total_projects,
"total_tasks": total_tasks,
"total_submissions": total_submissions,
"api_usage_count": api_usage_count
}
backend/routers/episodes.py
+294
View File
@@ -0,0 +1,294 @@
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from sqlalchemy import func
from typing import List
from database import get_db
from models.episode import Episode
from models.project import Project, ProjectMember
from models.user import User, UserRole
from models.shot import Shot
from schemas.episode import EpisodeCreate, EpisodeUpdate, EpisodeResponse, EpisodeListResponse
from utils.auth import get_current_user, require_role, get_current_user_from_token
router = APIRouter()
def get_current_user_with_db(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Get current user with proper database dependency."""
from utils.auth import _get_user_from_db
return _get_user_from_db(db, token_data["user_id"])
def require_coordinator_or_admin(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Require coordinator or admin role."""
from utils.auth import _get_user_from_db
current_user = _get_user_from_db(db, token_data["user_id"])
if current_user.role != UserRole.COORDINATOR and not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Insufficient permissions"
)
return current_user
@router.get("/", response_model=List[EpisodeListResponse])
async def list_episodes(
project_id: int = None,
skip: int = 0,
limit: int = 100,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""List episodes, optionally filtered by project"""
query = db.query(Episode)
if project_id:
# Check if project exists and user has access
project = db.query(Project).filter(Project.id == project_id).first()
if not project:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Project not found"
)
# Check access for artists
if current_user.role == UserRole.ARTIST:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == project_id,
ProjectMember.user_id == current_user.id
).first()
if not member:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this project"
)
query = query.filter(Episode.project_id == project_id)
else:
# For artists, only show episodes from projects they're members of
if current_user.role == UserRole.ARTIST:
user_project_ids = db.query(ProjectMember.project_id).filter(
ProjectMember.user_id == current_user.id
).subquery()
query = query.filter(Episode.project_id.in_(user_project_ids))
episodes = query.order_by(Episode.episode_number).offset(skip).limit(limit).all()
# Add shot count for each episode
result = []
for episode in episodes:
shot_count = db.query(Shot).filter(Shot.episode_id == episode.id).count()
episode_data = EpisodeListResponse.model_validate(episode)
episode_data.shot_count = shot_count
result.append(episode_data)
return result
@router.post("/", response_model=EpisodeResponse, status_code=status.HTTP_201_CREATED)
async def create_episode(
episode: EpisodeCreate,
project_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Create a new episode within a project"""
# Check if project exists
project = db.query(Project).filter(Project.id == project_id).first()
if not project:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Project not found"
)
# Check if episode number is already used in this project
existing_episode = db.query(Episode).filter(
Episode.project_id == project_id,
Episode.episode_number == episode.episode_number
).first()
if existing_episode:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Episode number {episode.episode_number} already exists in this project"
)
# Create new episode
db_episode = Episode(
project_id=project_id,
**episode.model_dump()
)
db.add(db_episode)
db.commit()
db.refresh(db_episode)
episode_data = EpisodeResponse.model_validate(db_episode)
episode_data.shot_count = 0 # New episode has no shots yet
return episode_data
@router.get("/{episode_id}", response_model=EpisodeResponse)
async def get_episode(
episode_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""Get a specific episode by ID"""
episode = db.query(Episode).filter(Episode.id == episode_id).first()
if not episode:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Episode not found"
)
# Check access for artists
if current_user.role == UserRole.ARTIST:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == episode.project_id,
ProjectMember.user_id == current_user.id
).first()
if not member:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this episode"
)
# Add shot count
shot_count = db.query(Shot).filter(Shot.episode_id == episode.id).count()
episode_data = EpisodeResponse.model_validate(episode)
episode_data.shot_count = shot_count
return episode_data
@router.put("/{episode_id}", response_model=EpisodeResponse)
async def update_episode(
episode_id: int,
episode_update: EpisodeUpdate,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Update an episode"""
db_episode = db.query(Episode).filter(Episode.id == episode_id).first()
if not db_episode:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Episode not found"
)
# If updating episode number, check for conflicts
update_data = episode_update.model_dump(exclude_unset=True)
if 'episode_number' in update_data:
existing_episode = db.query(Episode).filter(
Episode.project_id == db_episode.project_id,
Episode.episode_number == update_data['episode_number'],
Episode.id != episode_id
).first()
if existing_episode:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Episode number {update_data['episode_number']} already exists in this project"
)
# Update only provided fields
for field, value in update_data.items():
setattr(db_episode, field, value)
db.commit()
db.refresh(db_episode)
# Add shot count
shot_count = db.query(Shot).filter(Shot.episode_id == db_episode.id).count()
episode_data = EpisodeResponse.model_validate(db_episode)
episode_data.shot_count = shot_count
return episode_data
@router.delete("/{episode_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_episode(
episode_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Delete an episode"""
db_episode = db.query(Episode).filter(Episode.id == episode_id).first()
if not db_episode:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Episode not found"
)
db.delete(db_episode)
db.commit()
# Project-specific episode endpoints
@router.get("/projects/{project_id}/episodes", response_model=List[EpisodeListResponse])
async def list_project_episodes(
project_id: int,
skip: int = 0,
limit: int = 100,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""List all episodes for a specific project"""
# Check if project exists
project = db.query(Project).filter(Project.id == project_id).first()
if not project:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Project not found"
)
# Check access for artists
if current_user.role == UserRole.ARTIST:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == project_id,
ProjectMember.user_id == current_user.id
).first()
if not member:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this project"
)
episodes = db.query(Episode).filter(
Episode.project_id == project_id
).order_by(Episode.episode_number).offset(skip).limit(limit).all()
# Add shot count for each episode
result = []
for episode in episodes:
shot_count = db.query(Shot).filter(Shot.episode_id == episode.id).count()
episode_data = EpisodeListResponse.model_validate(episode)
episode_data.shot_count = shot_count
result.append(episode_data)
return result
@router.post("/projects/{project_id}/episodes", response_model=EpisodeResponse, status_code=status.HTTP_201_CREATED)
async def create_project_episode(
project_id: int,
episode: EpisodeCreate,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Create a new episode within a specific project"""
return await create_episode(episode, project_id, db, current_user)
backend/routers/files.py
+447
View File
@@ -0,0 +1,447 @@
"""
File serving router for VFX Project Management System.
Handles authenticated file serving, thumbnails, and access control.
"""
from fastapi import APIRouter, Depends, HTTPException, Response
from fastapi.responses import FileResponse, StreamingResponse
from sqlalchemy.orm import Session
from pathlib import Path
import os
import mimetypes
from typing import Optional
from database import get_db
from models.task import Task, TaskAttachment, Submission
from models.user import User, UserRole
from utils.auth import get_current_user_from_token, _get_user_from_db
from utils.file_handler import file_handler
router = APIRouter()
def get_current_user(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Get current user with proper database dependency."""
return _get_user_from_db(db, token_data["user_id"])
def check_file_access_permission(user: User, task: Task, db: Session) -> bool:
"""Check if user has permission to access files for a task."""
from models.project import ProjectMember
# Admins and coordinators can access all files
if user.role == UserRole.COORDINATOR or user.is_admin:
return True
# Directors can access all files for review
if user.role == UserRole.DIRECTOR:
return True
# Artists can access files for their assigned tasks
if task.assigned_user_id == user.id:
return True
# Artists can also access files for tasks in projects they're members of
if user.role == UserRole.ARTIST:
# Get the project_id from the task's asset or shot
project_id = None
if task.asset_id:
from models.asset import Asset
asset = db.query(Asset).filter(Asset.id == task.asset_id).first()
if asset:
project_id = asset.project_id
elif task.shot_id:
from models.shot import Shot
shot = db.query(Shot).filter(Shot.id == task.shot_id).first()
if shot:
from models.episode import Episode
episode = db.query(Episode).filter(Episode.id == shot.episode_id).first()
if episode:
project_id = episode.project_id
# Check if user is a project member
if project_id:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == project_id,
ProjectMember.user_id == user.id
).first()
if member:
return True
return False
@router.get("/attachments/{attachment_id}")
async def serve_attachment(
attachment_id: int,
thumbnail: bool = False,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Serve a task attachment file with access control."""
# Get attachment
attachment = db.query(TaskAttachment).filter(
TaskAttachment.id == attachment_id,
TaskAttachment.deleted_at.is_(None)
).first()
if not attachment:
raise HTTPException(status_code=404, detail="Attachment not found")
# Get associated task
task = db.query(Task).filter(Task.id == attachment.task_id).first()
if not task:
raise HTTPException(status_code=404, detail="Associated task not found")
# Check permissions
if not check_file_access_permission(current_user, task, db):
raise HTTPException(status_code=403, detail="Not authorized to access this file")
# Determine file path
if thumbnail and file_handler.is_image_file(attachment.file_path):
# Try to serve thumbnail
thumbnail_path = file_handler.get_thumbnail_path(attachment.file_path)
absolute_thumbnail_path = file_handler.resolve_absolute_path(thumbnail_path)
if os.path.exists(absolute_thumbnail_path):
file_path = absolute_thumbnail_path
filename = f"thumb_{attachment.file_name}"
else:
# Create thumbnail on-demand
created_thumbnail = file_handler.create_thumbnail(attachment.file_path)
if created_thumbnail:
absolute_created_thumbnail = file_handler.resolve_absolute_path(created_thumbnail)
if os.path.exists(absolute_created_thumbnail):
file_path = absolute_created_thumbnail
filename = f"thumb_{attachment.file_name}"
else:
# Fall back to original file
file_path = file_handler.resolve_absolute_path(attachment.file_path)
filename = attachment.file_name
else:
# Fall back to original file
file_path = file_handler.resolve_absolute_path(attachment.file_path)
filename = attachment.file_name
else:
file_path = file_handler.resolve_absolute_path(attachment.file_path)
filename = attachment.file_name
# Check if file exists
if not os.path.exists(file_path):
raise HTTPException(status_code=404, detail="File not found on disk")
# Get MIME type
mime_type, _ = mimetypes.guess_type(file_path)
if not mime_type:
mime_type = 'application/octet-stream'
# Return file
return FileResponse(
path=file_path,
filename=filename,
media_type=mime_type
)
@router.get("/submissions/{submission_id}")
async def serve_submission(
submission_id: int,
thumbnail: bool = False,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Serve a submission file with access control."""
# Get submission
submission = db.query(Submission).filter(
Submission.id == submission_id,
Submission.deleted_at.is_(None)
).first()
if not submission:
raise HTTPException(status_code=404, detail="Submission not found")
# Get associated task
task = db.query(Task).filter(Task.id == submission.task_id).first()
if not task:
raise HTTPException(status_code=404, detail="Associated task not found")
# Check permissions
if not check_file_access_permission(current_user, task, db):
raise HTTPException(status_code=403, detail="Not authorized to access this file")
# Determine file path
if thumbnail and file_handler.is_image_file(submission.file_path):
# Try to serve thumbnail
thumbnail_path = file_handler.get_thumbnail_path(submission.file_path)
absolute_thumbnail_path = file_handler.resolve_absolute_path(thumbnail_path)
if os.path.exists(absolute_thumbnail_path):
file_path = absolute_thumbnail_path
filename = f"thumb_{submission.file_name}"
else:
# Create thumbnail on-demand
created_thumbnail = file_handler.create_thumbnail(submission.file_path)
if created_thumbnail:
absolute_created_thumbnail = file_handler.resolve_absolute_path(created_thumbnail)
if os.path.exists(absolute_created_thumbnail):
file_path = absolute_created_thumbnail
filename = f"thumb_{submission.file_name}"
else:
# Fall back to original file
file_path = file_handler.resolve_absolute_path(submission.file_path)
filename = submission.file_name
else:
# Fall back to original file
file_path = file_handler.resolve_absolute_path(submission.file_path)
filename = submission.file_name
else:
file_path = file_handler.resolve_absolute_path(submission.file_path)
filename = submission.file_name
# Check if file exists
if not os.path.exists(file_path):
raise HTTPException(status_code=404, detail="File not found on disk")
# Get MIME type
mime_type, _ = mimetypes.guess_type(file_path)
if not mime_type:
mime_type = 'application/octet-stream'
# Return file
return FileResponse(
path=file_path,
filename=filename,
media_type=mime_type
)
@router.get("/submissions/{submission_id}/stream")
async def stream_submission(
submission_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Stream a submission file for video playback."""
# Get submission
submission = db.query(Submission).filter(
Submission.id == submission_id,
Submission.deleted_at.is_(None)
).first()
if not submission:
raise HTTPException(status_code=404, detail="Submission not found")
# Get associated task
task = db.query(Task).filter(Task.id == submission.task_id).first()
if not task:
raise HTTPException(status_code=404, detail="Associated task not found")
# Check permissions
if not check_file_access_permission(current_user, task, db):
raise HTTPException(status_code=403, detail="Not authorized to access this file")
file_path = file_handler.resolve_absolute_path(submission.file_path)
# Check if file exists
if not os.path.exists(file_path):
raise HTTPException(status_code=404, detail="File not found on disk")
# Only stream video files
if not file_handler.is_video_file(file_path):
raise HTTPException(status_code=400, detail="File is not a video")
# Get MIME type
mime_type, _ = mimetypes.guess_type(file_path)
if not mime_type:
mime_type = 'video/mp4' # Default for video
def iterfile(file_path: str):
"""Generator to stream file in chunks."""
with open(file_path, mode="rb") as file_like:
while True:
chunk = file_like.read(1024 * 1024) # 1MB chunks
if not chunk:
break
yield chunk
return StreamingResponse(
iterfile(file_path),
media_type=mime_type,
headers={
"Content-Disposition": f"inline; filename={submission.file_name}",
"Accept-Ranges": "bytes"
}
)
@router.get("/info/attachment/{attachment_id}")
async def get_attachment_info(
attachment_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get file information for an attachment."""
# Get attachment
attachment = db.query(TaskAttachment).filter(
TaskAttachment.id == attachment_id,
TaskAttachment.deleted_at.is_(None)
).first()
if not attachment:
raise HTTPException(status_code=404, detail="Attachment not found")
# Get associated task
task = db.query(Task).filter(Task.id == attachment.task_id).first()
if not task:
raise HTTPException(status_code=404, detail="Associated task not found")
# Check permissions
if not check_file_access_permission(current_user, task, db):
raise HTTPException(status_code=403, detail="Not authorized to access this file")
# Get file info
file_info = file_handler.get_file_info(attachment.file_path)
return {
"id": attachment.id,
"file_name": attachment.file_name,
"file_type": attachment.file_type,
"file_size": attachment.file_size,
"attachment_type": attachment.attachment_type,
"description": attachment.description,
"uploaded_at": attachment.uploaded_at,
"is_image": file_handler.is_image_file(attachment.file_path),
"is_video": file_handler.is_video_file(attachment.file_path),
"has_thumbnail": file_handler.is_image_file(attachment.file_path),
"file_exists": file_info.get('exists', False),
**file_info
}
@router.get("/info/submission/{submission_id}")
async def get_submission_info(
submission_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get file information for a submission."""
# Get submission
submission = db.query(Submission).filter(
Submission.id == submission_id,
Submission.deleted_at.is_(None)
).first()
if not submission:
raise HTTPException(status_code=404, detail="Submission not found")
# Get associated task
task = db.query(Task).filter(Task.id == submission.task_id).first()
if not task:
raise HTTPException(status_code=404, detail="Associated task not found")
# Check permissions
if not check_file_access_permission(current_user, task, db):
raise HTTPException(status_code=403, detail="Not authorized to access this file")
# Get file info
file_info = file_handler.get_file_info(submission.file_path)
return {
"id": submission.id,
"file_name": submission.file_name,
"version_number": submission.version_number,
"notes": submission.notes,
"submitted_at": submission.submitted_at,
"is_image": file_handler.is_image_file(submission.file_path),
"is_video": file_handler.is_video_file(submission.file_path),
"has_thumbnail": file_handler.is_image_file(submission.file_path),
"file_exists": file_info.get('exists', False),
**file_info
}
@router.get("/projects/{project_id}/thumbnail")
async def serve_project_thumbnail(
project_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Serve a project thumbnail image with access control."""
from models.project import Project
from models.project import ProjectMember
# Get project
project = db.query(Project).filter(Project.id == project_id).first()
if not project:
raise HTTPException(status_code=404, detail="Project not found")
# Check if user has access to this project
# Artists can only access projects they're members of
if current_user.role == UserRole.ARTIST:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == project_id,
ProjectMember.user_id == current_user.id
).first()
if not member:
raise HTTPException(status_code=403, detail="Access denied to this project")
# Check if project has a thumbnail
if not project.thumbnail_path:
raise HTTPException(status_code=404, detail="Project has no thumbnail")
# Resolve to absolute path for file serving
absolute_thumbnail_path = file_handler.resolve_absolute_path(project.thumbnail_path)
# Check if file exists
if not os.path.exists(absolute_thumbnail_path):
raise HTTPException(status_code=404, detail="Thumbnail file not found on disk")
# Get MIME type
mime_type, _ = mimetypes.guess_type(absolute_thumbnail_path)
if not mime_type:
mime_type = 'image/jpeg' # Default for thumbnails
# Return file
return FileResponse(
path=absolute_thumbnail_path,
media_type=mime_type
)
@router.get("/users/{user_id}/avatar")
async def serve_user_avatar(
user_id: int,
db: Session = Depends(get_db)
):
"""Serve user avatar (public access for simplicity)."""
# Get user
user = db.query(User).filter(User.id == user_id).first()
if not user:
raise HTTPException(status_code=404, detail="User not found")
# Check if user has avatar
if not user.avatar_url:
raise HTTPException(status_code=404, detail="User has no avatar")
# Note: Avatar access is public for simplicity since img tags can't send auth headers
# More restrictive access control can be added later if needed
# Resolve to absolute path for file serving
absolute_avatar_path = file_handler.resolve_absolute_path(user.avatar_url)
# Check if file exists
if not os.path.exists(absolute_avatar_path):
raise HTTPException(status_code=404, detail="Avatar file not found on disk")
# Get MIME type
mime_type, _ = mimetypes.guess_type(absolute_avatar_path)
if not mime_type:
mime_type = 'image/jpeg' # Default for avatars
# Return file
return FileResponse(
path=absolute_avatar_path,
media_type=mime_type
)
backend/routers/notifications.py
+178
View File
@@ -0,0 +1,178 @@
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy.orm import Session
from sqlalchemy import desc, func
from typing import List, Optional
from datetime import datetime
from database import get_db
from models.user import User
from models.notification import Notification, UserNotificationPreference, NotificationType
from schemas.notification import (
NotificationResponse,
NotificationMarkRead,
NotificationPreferencesResponse,
NotificationPreferencesUpdate,
NotificationStats
)
from utils.auth import get_current_user
router = APIRouter(prefix="/notifications", tags=["notifications"])
@router.get("", response_model=List[NotificationResponse])
def get_notifications(
skip: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=100),
unread_only: bool = Query(False),
type_filter: Optional[NotificationType] = None,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get notifications for the current user."""
query = db.query(Notification).filter(Notification.user_id == current_user.id)
if unread_only:
query = query.filter(Notification.read == False)
if type_filter:
query = query.filter(Notification.type == type_filter)
notifications = query.order_by(desc(Notification.created_at)).offset(skip).limit(limit).all()
return notifications
@router.get("/stats", response_model=NotificationStats)
def get_notification_stats(
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get notification statistics for the current user."""
total = db.query(Notification).filter(Notification.user_id == current_user.id).count()
unread = db.query(Notification).filter(
Notification.user_id == current_user.id,
Notification.read == False
).count()
# Get counts by type
by_type_query = db.query(
Notification.type,
func.count(Notification.id).label('count')
).filter(
Notification.user_id == current_user.id,
Notification.read == False
).group_by(Notification.type).all()
by_type = {str(type_): count for type_, count in by_type_query}
return NotificationStats(total=total, unread=unread, by_type=by_type)
@router.post("/mark-read")
def mark_notifications_read(
data: NotificationMarkRead,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Mark notifications as read."""
notifications = db.query(Notification).filter(
Notification.id.in_(data.notification_ids),
Notification.user_id == current_user.id
).all()
if not notifications:
raise HTTPException(status_code=404, detail="No notifications found")
for notification in notifications:
notification.read = True
notification.read_at = datetime.utcnow()
db.commit()
return {"message": f"Marked {len(notifications)} notifications as read"}
@router.post("/mark-all-read")
def mark_all_notifications_read(
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Mark all notifications as read for the current user."""
count = db.query(Notification).filter(
Notification.user_id == current_user.id,
Notification.read == False
).update({
"read": True,
"read_at": datetime.utcnow()
})
db.commit()
return {"message": f"Marked {count} notifications as read"}
@router.delete("/{notification_id}")
def delete_notification(
notification_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Delete a notification."""
notification = db.query(Notification).filter(
Notification.id == notification_id,
Notification.user_id == current_user.id
).first()
if not notification:
raise HTTPException(status_code=404, detail="Notification not found")
db.delete(notification)
db.commit()
return {"message": "Notification deleted"}
@router.get("/preferences", response_model=NotificationPreferencesResponse)
def get_notification_preferences(
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get notification preferences for the current user."""
preferences = db.query(UserNotificationPreference).filter(
UserNotificationPreference.user_id == current_user.id
).first()
if not preferences:
# Create default preferences
preferences = UserNotificationPreference(user_id=current_user.id)
db.add(preferences)
db.commit()
db.refresh(preferences)
return preferences
@router.put("/preferences", response_model=NotificationPreferencesResponse)
def update_notification_preferences(
data: NotificationPreferencesUpdate,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Update notification preferences for the current user."""
preferences = db.query(UserNotificationPreference).filter(
UserNotificationPreference.user_id == current_user.id
).first()
if not preferences:
preferences = UserNotificationPreference(user_id=current_user.id)
db.add(preferences)
# Update all fields
for field, value in data.model_dump().items():
setattr(preferences, field, value)
preferences.updated_at = datetime.utcnow()
db.commit()
db.refresh(preferences)
return preferences
backend/routers/projects.py
+1956
View File
File diff suppressed because it is too large Load Diff
backend/routers/reviews.py
+295
View File
@@ -0,0 +1,295 @@
from fastapi import APIRouter, Depends, HTTPException, Query, status
from sqlalchemy.orm import Session, joinedload
from sqlalchemy import and_
from typing import List, Optional
from database import get_db
from models.task import Task, Submission, Review, TaskStatus
from models.user import User, UserRole
from schemas.task import ReviewCreate, ReviewResponse, SubmissionResponse
from utils.auth import get_current_user_from_token, _get_user_from_db
from utils.notifications import notification_service
router = APIRouter()
def get_current_user(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Get current user with proper database dependency."""
return _get_user_from_db(db, token_data["user_id"])
def require_director_coordinator_or_admin(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Dependency to require director, coordinator role, or admin permission."""
current_user = _get_user_from_db(db, token_data["user_id"])
if (current_user.role not in [UserRole.DIRECTOR, UserRole.COORDINATOR] and
not current_user.is_admin):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Director role, Coordinator role, or Admin permission required"
)
return current_user
def require_role(required_roles: list):
"""Create a dependency that requires specific user roles."""
def role_checker(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
current_user = _get_user_from_db(db, token_data["user_id"])
if current_user.role not in required_roles:
raise HTTPException(
status_code=403,
detail="Insufficient permissions"
)
return current_user
return role_checker
@router.get("/pending", response_model=List[SubmissionResponse])
async def get_pending_reviews(
project_id: Optional[int] = Query(None, description="Filter by project ID"),
skip: int = Query(0, ge=0),
limit: int = Query(100, ge=1, le=1000),
db: Session = Depends(get_db),
current_user: User = Depends(require_director_coordinator_or_admin)
):
"""Get all submissions pending review. Only directors, coordinators, and users with admin permission can access."""
# Get submissions that don't have any reviews yet or have retake reviews
query = db.query(Submission).options(
joinedload(Submission.user),
joinedload(Submission.task).joinedload(Task.project),
joinedload(Submission.reviews).joinedload("reviewer")
).join(Task).filter(
Submission.deleted_at.is_(None),
Task.deleted_at.is_(None)
)
# Filter by project if specified
if project_id:
query = query.filter(Task.project_id == project_id)
# Only get submitted tasks
query = query.filter(Task.status == "submitted")
submissions = query.order_by(Submission.submitted_at.desc()).offset(skip).limit(limit).all()
# Filter to only include submissions that need review
pending_submissions = []
for submission in submissions:
# Check if submission has any approved reviews
has_approved_review = any(review.decision == "approved" for review in submission.reviews)
if not has_approved_review:
# Get latest review
latest_review = None
if submission.reviews:
latest_review_obj = max(submission.reviews, key=lambda r: r.reviewed_at)
latest_review = {
"id": latest_review_obj.id,
"submission_id": latest_review_obj.submission_id,
"reviewer_id": latest_review_obj.reviewer_id,
"decision": latest_review_obj.decision,
"feedback": latest_review_obj.feedback,
"reviewed_at": latest_review_obj.reviewed_at,
"reviewer_first_name": latest_review_obj.reviewer.first_name,
"reviewer_last_name": latest_review_obj.reviewer.last_name
}
submission_data = {
"id": submission.id,
"task_id": submission.task_id,
"user_id": submission.user_id,
"file_path": submission.file_path,
"file_name": submission.file_name,
"version_number": submission.version_number,
"notes": submission.notes,
"submitted_at": submission.submitted_at,
"user_first_name": submission.user.first_name,
"user_last_name": submission.user.last_name,
"latest_review": latest_review
}
pending_submissions.append(SubmissionResponse(**submission_data))
return pending_submissions
@router.post("/{submission_id}/approve", response_model=ReviewResponse)
async def approve_submission(
submission_id: int,
review: ReviewCreate,
db: Session = Depends(get_db),
current_user: User = Depends(require_director_coordinator_or_admin)
):
"""Approve a submission. Only directors, coordinators, and users with admin permission can approve."""
submission = db.query(Submission).options(
joinedload(Submission.task)
).filter(
Submission.id == submission_id,
Submission.deleted_at.is_(None)
).first()
if not submission:
raise HTTPException(status_code=404, detail="Submission not found")
# Check if submission is in submitted state
if submission.task.status != "submitted":
raise HTTPException(status_code=400, detail="Submission is not in submitted state")
# Force decision to approved
review.decision = "approved"
# Create review record
db_review = Review(
submission_id=submission_id,
reviewer_id=current_user.id,
decision=review.decision,
feedback=review.feedback
)
db.add(db_review)
# Update task status to approved
submission.task.status = "approved"
db.commit()
db.refresh(db_review)
# Send notification to artist
notification_service.notify_submission_reviewed(db, submission, db_review, current_user)
# Load reviewer information for response
db_review = db.query(Review).options(
joinedload(Review.reviewer)
).filter(Review.id == db_review.id).first()
review_data = {
"id": db_review.id,
"submission_id": db_review.submission_id,
"reviewer_id": db_review.reviewer_id,
"decision": db_review.decision,
"feedback": db_review.feedback,
"reviewed_at": db_review.reviewed_at,
"reviewer_first_name": db_review.reviewer.first_name,
"reviewer_last_name": db_review.reviewer.last_name
}
return ReviewResponse(**review_data)
@router.post("/{submission_id}/retake", response_model=ReviewResponse)
async def request_retake(
submission_id: int,
review: ReviewCreate,
db: Session = Depends(get_db),
current_user: User = Depends(require_director_coordinator_or_admin)
):
"""Request a retake for a submission. Only directors, coordinators, and users with admin permission can request retakes."""
submission = db.query(Submission).options(
joinedload(Submission.task)
).filter(
Submission.id == submission_id,
Submission.deleted_at.is_(None)
).first()
if not submission:
raise HTTPException(status_code=404, detail="Submission not found")
# Check if submission is in submitted state
if submission.task.status != "submitted":
raise HTTPException(status_code=400, detail="Submission is not in submitted state")
# Force decision to retake and require feedback
review.decision = "retake"
if not review.feedback or review.feedback.strip() == "":
raise HTTPException(status_code=400, detail="Feedback is required when requesting a retake")
# Create review record
db_review = Review(
submission_id=submission_id,
reviewer_id=current_user.id,
decision=review.decision,
feedback=review.feedback
)
db.add(db_review)
# Update task status to retake
submission.task.status = "retake"
db.commit()
db.refresh(db_review)
# Send notification to artist
notification_service.notify_submission_reviewed(db, submission, db_review, current_user)
# Load reviewer information for response
db_review = db.query(Review).options(
joinedload(Review.reviewer)
).filter(Review.id == db_review.id).first()
review_data = {
"id": db_review.id,
"submission_id": db_review.submission_id,
"reviewer_id": db_review.reviewer_id,
"decision": db_review.decision,
"feedback": db_review.feedback,
"reviewed_at": db_review.reviewed_at,
"reviewer_first_name": db_review.reviewer.first_name,
"reviewer_last_name": db_review.reviewer.last_name
}
return ReviewResponse(**review_data)
@router.get("/{submission_id}/reviews", response_model=List[ReviewResponse])
async def get_submission_reviews(
submission_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get all reviews for a submission."""
submission = db.query(Submission).options(
joinedload(Submission.task)
).filter(
Submission.id == submission_id,
Submission.deleted_at.is_(None)
).first()
if not submission:
raise HTTPException(status_code=404, detail="Submission not found")
# Artists can only view reviews for their own submissions
if current_user.role == UserRole.ARTIST and submission.user_id != current_user.id:
raise HTTPException(status_code=403, detail="Not authorized to view reviews for this submission")
reviews = db.query(Review).options(
joinedload(Review.reviewer)
).filter(
Review.submission_id == submission_id,
Review.deleted_at.is_(None)
).order_by(Review.reviewed_at.desc()).all()
result = []
for review in reviews:
review_data = {
"id": review.id,
"submission_id": review.submission_id,
"reviewer_id": review.reviewer_id,
"decision": review.decision,
"feedback": review.feedback,
"reviewed_at": review.reviewed_at,
"reviewer_first_name": review.reviewer.first_name,
"reviewer_last_name": review.reviewer.last_name
}
result.append(ReviewResponse(**review_data))
return result
backend/routers/settings.py
+166
View File
@@ -0,0 +1,166 @@
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from typing import List
from database import get_db
from models.global_settings import GlobalSettings
from schemas.global_settings import (
GlobalSetting,
GlobalSettingCreate,
GlobalSettingUpdate,
UploadLimitResponse,
UploadLimitUpdate
)
from utils.auth import get_current_user, require_admin_permission
from models.user import User
router = APIRouter(prefix="/settings", tags=["settings"])
# Default upload limit in MB (1GB)
DEFAULT_UPLOAD_LIMIT_MB = 1000
UPLOAD_LIMIT_KEY = "global_upload_limit_mb"
def get_or_create_upload_limit_setting(db: Session) -> GlobalSettings:
"""Get or create the upload limit setting with default value"""
setting = db.query(GlobalSettings).filter(
GlobalSettings.setting_key == UPLOAD_LIMIT_KEY
).first()
if not setting:
setting = GlobalSettings(
setting_key=UPLOAD_LIMIT_KEY,
setting_value=str(DEFAULT_UPLOAD_LIMIT_MB),
description="Global upload size limit for movie files in MB"
)
db.add(setting)
db.commit()
db.refresh(setting)
return setting
@router.get("/upload-limit", response_model=UploadLimitResponse)
async def get_upload_limit(
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user)
):
"""Get the global upload size limit for movie files"""
setting = get_or_create_upload_limit_setting(db)
return UploadLimitResponse(
upload_limit_mb=int(setting.setting_value),
description=setting.description or "Global upload size limit for movie files"
)
@router.put("/upload-limit", response_model=UploadLimitResponse)
async def update_upload_limit(
upload_limit: UploadLimitUpdate,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin_permission)
):
"""Update the global upload size limit for movie files (admin only)"""
setting = get_or_create_upload_limit_setting(db)
setting.setting_value = str(upload_limit.upload_limit_mb)
db.commit()
db.refresh(setting)
return UploadLimitResponse(
upload_limit_mb=int(setting.setting_value),
description=setting.description or "Global upload size limit for movie files"
)
@router.get("/", response_model=List[GlobalSetting])
async def get_all_settings(
db: Session = Depends(get_db),
current_user: User = Depends(require_admin_permission)
):
"""Get all global settings (admin only)"""
settings = db.query(GlobalSettings).all()
return settings
@router.post("/", response_model=GlobalSetting)
async def create_setting(
setting: GlobalSettingCreate,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin_permission)
):
"""Create a new global setting (admin only)"""
# Check if setting already exists
existing = db.query(GlobalSettings).filter(
GlobalSettings.setting_key == setting.setting_key
).first()
if existing:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Setting with key '{setting.setting_key}' already exists"
)
db_setting = GlobalSettings(**setting.dict())
db.add(db_setting)
db.commit()
db.refresh(db_setting)
return db_setting
@router.put("/{setting_key}", response_model=GlobalSetting)
async def update_setting(
setting_key: str,
setting_update: GlobalSettingUpdate,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin_permission)
):
"""Update a specific global setting (admin only)"""
setting = db.query(GlobalSettings).filter(
GlobalSettings.setting_key == setting_key
).first()
if not setting:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Setting with key '{setting_key}' not found"
)
for field, value in setting_update.dict(exclude_unset=True).items():
setattr(setting, field, value)
db.commit()
db.refresh(setting)
return setting
@router.delete("/{setting_key}")
async def delete_setting(
setting_key: str,
db: Session = Depends(get_db),
current_user: User = Depends(require_admin_permission)
):
"""Delete a global setting (admin only)"""
setting = db.query(GlobalSettings).filter(
GlobalSettings.setting_key == setting_key
).first()
if not setting:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Setting with key '{setting_key}' not found"
)
# Prevent deletion of critical settings
if setting_key == UPLOAD_LIMIT_KEY:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Cannot delete the global upload limit setting"
)
db.delete(setting)
db.commit()
return {"message": f"Setting '{setting_key}' deleted successfully"}
backend/routers/shots.py
+858
View File
@@ -0,0 +1,858 @@
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from typing import List
from database import get_db
from models.shot import Shot
from models.episode import Episode
from models.project import Project, ProjectMember
from models.task import Task, TaskType, TaskStatus
from models.user import User, UserRole
from schemas.shot import (
ShotCreate, ShotUpdate, ShotResponse, ShotListResponse,
BulkShotCreate, BulkShotResponse, TaskStatusInfo
)
from utils.auth import get_current_user_from_token
from services.shot_soft_deletion import ShotSoftDeletionService
router = APIRouter()
def get_current_user_with_db(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Get current user with proper database dependency."""
from utils.auth import _get_user_from_db
return _get_user_from_db(db, token_data["user_id"])
def require_coordinator_or_admin(
token_data: dict = Depends(get_current_user_from_token),
db: Session = Depends(get_db)
):
"""Require coordinator or admin role."""
from utils.auth import _get_user_from_db
current_user = _get_user_from_db(db, token_data["user_id"])
if current_user.role != UserRole.COORDINATOR and not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Insufficient permissions"
)
return current_user
def check_episode_access(episode_id: int, current_user: User, db: Session):
"""Check if user has access to the episode and its project."""
# Debug logging
print(f"[DEBUG] check_episode_access called")
print(f"[DEBUG] User: {current_user.email}, Role: {current_user.role}, is_admin: {current_user.is_admin}")
# Check if episode exists
episode = db.query(Episode).filter(Episode.id == episode_id).first()
if not episode:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Episode not found"
)
# Admins and coordinators have access to all episodes
if current_user.is_admin or current_user.role == UserRole.COORDINATOR:
print(f"[DEBUG] Access GRANTED - Admin or Coordinator")
return episode
# Check project access for artists and other roles
if current_user.role == UserRole.ARTIST:
print(f"[DEBUG] Checking project membership for artist")
member = db.query(ProjectMember).filter(
ProjectMember.project_id == episode.project_id,
ProjectMember.user_id == current_user.id
).first()
if not member:
print(f"[DEBUG] Access DENIED - Not a project member")
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this project"
)
print(f"[DEBUG] Access GRANTED - Project member")
return episode
def get_status_sort_order(status: str, project_custom_statuses: list = None) -> int:
"""Get sort order for task status, including custom statuses."""
# Default system status order
system_status_order = {
"not_started": 0,
"in_progress": 1,
"submitted": 2,
"retake": 3,
"approved": 4
}
# If it's a system status, return its order
if status in system_status_order:
return system_status_order[status]
# For custom statuses, use their defined order + offset to place them after system statuses
if project_custom_statuses:
for custom_status in project_custom_statuses:
if isinstance(custom_status, dict) and custom_status.get('id') == status:
# Custom statuses start after system statuses (5+)
return 5 + custom_status.get('order', 0)
# Unknown status defaults to 0 (same as not_started)
return 0
def get_project_custom_statuses(project_id: int, db: Session) -> list:
"""Get custom task statuses for a project."""
project = db.query(Project).filter(Project.id == project_id).first()
if not project or not project.custom_task_statuses:
return []
custom_statuses_data = project.custom_task_statuses
if isinstance(custom_statuses_data, str):
try:
import json
custom_statuses_data = json.loads(custom_statuses_data)
except (json.JSONDecodeError, TypeError):
return []
return custom_statuses_data if isinstance(custom_statuses_data, list) else []
# Standard shot task types (read-only)
STANDARD_SHOT_TASK_TYPES = ["layout", "animation", "simulation", "lighting", "compositing"]
def get_default_shot_task_types():
"""Get default task types for shots."""
return [
TaskType.LAYOUT.value,
TaskType.ANIMATION.value,
TaskType.LIGHTING.value,
TaskType.COMPOSITING.value
]
def get_all_shot_task_types(project_id: int, db: Session) -> List[str]:
"""Get all task types (standard + custom) for shots in a project."""
project = db.query(Project).filter(Project.id == project_id).first()
if not project:
return STANDARD_SHOT_TASK_TYPES
custom_types = project.custom_shot_task_types or []
return STANDARD_SHOT_TASK_TYPES + custom_types
def create_default_tasks_for_shot(shot: Shot, task_types: List[str], db: Session):
"""Create default tasks for a shot."""
created_tasks = []
for task_type in task_types:
task_name = f"{shot.name}_{task_type}"
task_description = f"{task_type.title()} task for shot {shot.name}"
task = Task(
project_id=shot.project_id,
episode_id=shot.episode_id,
shot_id=shot.id,
task_type=task_type,
name=task_name,
description=task_description
)
db.add(task)
created_tasks.append(task)
return created_tasks
@router.get("/", response_model=List[ShotListResponse])
async def list_shots(
episode_id: int = None,
project_id: int = None,
task_status_filter: str = None,
sort_by: str = None,
sort_direction: str = "asc",
skip: int = 0,
limit: int = 100,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""List shots with optional filtering by episode, project and task status"""
from sqlalchemy import func, case
from sqlalchemy.orm import selectinload, joinedload
# Build base query for shots (exclude soft deleted)
base_query = db.query(Shot).filter(Shot.deleted_at.is_(None))
# Filter by project_id if specified
if project_id:
# Check project access for artists
if current_user.role == UserRole.ARTIST:
member = db.query(ProjectMember).filter(
ProjectMember.project_id == project_id,
ProjectMember.user_id == current_user.id
).first()
if not member and not current_user.is_admin:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to this project"
)
base_query = base_query.filter(Shot.project_id == project_id)
# Filter by episode if specified
if episode_id:
check_episode_access(episode_id, current_user, db)
base_query = base_query.filter(Shot.episode_id == episode_id)
elif not project_id:
# If no episode or project specified, filter by user's accessible projects for artists
if current_user.role == UserRole.ARTIST:
accessible_projects = db.query(ProjectMember.project_id).filter(
ProjectMember.user_id == current_user.id
).subquery()
base_query = base_query.filter(Shot.project_id.in_(accessible_projects))
# Apply sorting if specified (for non-task-status fields)
if sort_by and not sort_by.endswith('_status'):
if sort_by in ['name', 'status', 'frame_start', 'frame_end', 'created_at', 'updated_at']:
sort_column = getattr(Shot, sort_by)
if sort_direction.lower() == 'desc':
base_query = base_query.order_by(sort_column.desc())
else:
base_query = base_query.order_by(sort_column.asc())
# OPTIMIZATION: Use single query with optimized JOIN to fetch shots and their tasks
# This replaces the N+1 query pattern with a single database operation
shots_with_tasks = (
base_query
.outerjoin(Task, (Task.shot_id == Shot.id) & (Task.deleted_at.is_(None)))
.options(
joinedload(Shot.episode).joinedload(Episode.project), # Eager load episode and project
selectinload(Shot.tasks).options( # Use selectinload for better performance with tasks
selectinload(Task.assigned_user) # Eager load assigned users
)
)
.add_columns(
Task.id.label('task_id'),
Task.task_type,
Task.status.label('task_status'),
Task.assigned_user_id,
Task.updated_at.label('task_updated_at') # Include task update time for better tracking
)
.offset(skip)
.limit(limit)
.all()
)
# OPTIMIZATION: Pre-fetch all project data and task types in a single query
# This eliminates the need for repeated project queries
project_ids = set()
for row in shots_with_tasks:
shot = row[0]
if shot.project_id not in project_ids:
project_ids.add(shot.project_id)
# Get all projects with their custom task types in one optimized query
project_data = {}
if project_ids:
projects = (
db.query(Project)
.filter(Project.id.in_(project_ids))
.all()
)
for project in projects:
custom_types = project.custom_shot_task_types or []
project_data[project.id] = {
'task_types': STANDARD_SHOT_TASK_TYPES + custom_types,
'custom_statuses': get_project_custom_statuses(project.id, db)
}
# OPTIMIZATION: Group results by shot and aggregate task data efficiently
shots_dict = {}
for row in shots_with_tasks:
shot = row[0] # Shot object
task_id = row[1] # task_id
task_type = row[2] # task_type
task_status = row[3] # task_status
assigned_user_id = row[4] # assigned_user_id
task_updated_at = row[5] # task_updated_at
if shot.id not in shots_dict:
# Initialize shot data with pre-fetched project data
project_info = project_data.get(shot.project_id, {
'task_types': STANDARD_SHOT_TASK_TYPES,
'custom_statuses': []
})
shots_dict[shot.id] = {
'shot': shot,
'tasks': [],
'task_status': {},
'task_details': [],
'project_info': project_info
}
# Initialize all task types as not started using pre-fetched data
for task_type_init in project_info['task_types']:
shots_dict[shot.id]['task_status'][task_type_init] = "not_started"
# Add task data if task exists
if task_id is not None:
shots_dict[shot.id]['tasks'].append({
'task_id': task_id,
'task_type': task_type,
'status': task_status,
'assigned_user_id': assigned_user_id,
'updated_at': task_updated_at
})
# Update task status
shots_dict[shot.id]['task_status'][task_type] = task_status
# Add to task details with enhanced information
shots_dict[shot.id]['task_details'].append(TaskStatusInfo(
task_type=task_type,
status=task_status,
task_id=task_id,
assigned_user_id=assigned_user_id
))
# Build response list efficiently
result = []
for shot_data in shots_dict.values():
shot = shot_data['shot']
# Create shot response with optimized data
shot_response = ShotListResponse.model_validate(shot)
shot_response.task_count = len(shot_data['tasks'])
shot_response.task_status = shot_data['task_status']
shot_response.task_details = shot_data['task_details']
result.append(shot_response)
# Apply task status filtering if specified
if task_status_filter:
try:
# Parse task status filter (format: "task_type:status")
task_type, status = task_status_filter.split(":")
filter_status = status # Use string directly instead of enum
result = [
shot for shot in result
if shot.task_status.get(task_type) == filter_status
]
except (ValueError, KeyError):
# Invalid filter format, ignore
pass
# Apply task status sorting if specified using pre-fetched custom status data
if sort_by and sort_by.endswith('_status'):
task_type = sort_by.replace('_status', '')
def get_status_order(shot):
status = shot.task_status.get(task_type, "not_started")
# Use pre-fetched custom statuses from shots_dict
shot_id = shot.id
if shot_id in shots_dict:
custom_statuses = shots_dict[shot_id]['project_info']['custom_statuses']
return get_status_sort_order(status, custom_statuses)
return get_status_sort_order(status, [])
reverse = sort_direction.lower() == 'desc'
result.sort(key=get_status_order, reverse=reverse)
return result
@router.post("/", response_model=ShotResponse, status_code=status.HTTP_201_CREATED)
async def create_shot(
shot: ShotCreate,
episode_id: int,
create_default_tasks: bool = True,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Create a new shot in an episode"""
# Check episode access
episode = check_episode_access(episode_id, current_user, db)
# Auto-populate project_id from episode if not provided
if shot.project_id is None:
shot.project_id = episode.project_id
else:
# Validate that provided project_id matches episode's project
if shot.project_id != episode.project_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Project ID must match the episode's project"
)
# Validate frame range
if shot.frame_end < shot.frame_start:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Frame end must be greater than or equal to frame start"
)
# Check if shot name already exists in project (project-scoped uniqueness)
existing_shot = db.query(Shot).filter(
Shot.project_id == shot.project_id,
Shot.name == shot.name,
Shot.deleted_at.is_(None)
).first()
if existing_shot:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Shot with this name already exists in the project"
)
# Create new shot
db_shot = Shot(
episode_id=episode_id,
project_id=shot.project_id,
**shot.model_dump(exclude={'project_id'})
)
db.add(db_shot)
db.commit()
db.refresh(db_shot)
# Create default tasks if requested
task_count = 0
if create_default_tasks:
# Get all task types (standard + custom) for this project
all_task_types = get_all_shot_task_types(episode.project_id, db)
# Use default standard types for now (can be customized via project settings)
default_task_types = get_default_shot_task_types()
created_tasks = create_default_tasks_for_shot(db_shot, default_task_types, db)
db.commit()
task_count = len(created_tasks)
# Return response with task count
shot_data = ShotResponse.model_validate(db_shot)
shot_data.task_count = task_count
return shot_data
@router.post("/bulk", response_model=BulkShotResponse, status_code=status.HTTP_201_CREATED)
async def create_shots_bulk(
bulk_shot: BulkShotCreate,
episode_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Create multiple shots with naming pattern and default tasks"""
# Check episode access
episode = check_episode_access(episode_id, current_user, db)
# Auto-populate project_id from episode - all shots in bulk operation belong to same project
project_id = episode.project_id
# Validate frame range
if bulk_shot.frame_end < bulk_shot.frame_start:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Frame end must be greater than or equal to frame start"
)
# Determine task types to create
if bulk_shot.task_types:
# Validate that all selected task types are valid (standard or custom)
all_valid_types = get_all_shot_task_types(episode.project_id, db)
invalid_types = [t for t in bulk_shot.task_types if t not in all_valid_types]
if invalid_types:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Invalid task types: {', '.join(invalid_types)}"
)
task_types = bulk_shot.task_types
else:
task_types = get_default_shot_task_types()
# Pre-validate all shot names for project-scoped uniqueness before creating any shots
shot_names_to_create = []
for i in range(bulk_shot.shot_count):
shot_number = bulk_shot.start_number + i
shot_name = f"{bulk_shot.name_prefix}{shot_number:0{bulk_shot.number_padding}d}"
shot_names_to_create.append(shot_name)
# Check for existing shots with any of the names in this project
existing_shots = db.query(Shot.name).filter(
Shot.project_id == project_id,
Shot.name.in_(shot_names_to_create),
Shot.deleted_at.is_(None)
).all()
if existing_shots:
existing_names = [shot.name for shot in existing_shots]
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"The following shot names already exist in project {project_id}: {', '.join(existing_names)}"
)
# Check for duplicate names within the bulk creation itself
if len(shot_names_to_create) != len(set(shot_names_to_create)):
duplicates = [name for name in shot_names_to_create if shot_names_to_create.count(name) > 1]
unique_duplicates = list(set(duplicates))
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Duplicate shot names in bulk creation: {', '.join(unique_duplicates)}"
)
created_shots = []
total_tasks_created = 0
try:
# Create all shots - validation already done above
for i, shot_name in enumerate(shot_names_to_create):
shot_number = bulk_shot.start_number + i
# Create description from template
description = None
if bulk_shot.description_template:
description = bulk_shot.description_template.replace("{shot_name}", shot_name).replace("{shot_number}", str(shot_number))
# Create shot - project consistency guaranteed by using episode's project_id
db_shot = Shot(
project_id=project_id,
episode_id=episode_id,
name=shot_name,
description=description,
frame_start=bulk_shot.frame_start,
frame_end=bulk_shot.frame_end
)
db.add(db_shot)
db.flush() # Flush to get the shot ID
# Create default tasks if requested
task_count = 0
if bulk_shot.create_default_tasks:
created_tasks = create_default_tasks_for_shot(db_shot, task_types, db)
task_count = len(created_tasks)
total_tasks_created += task_count
# Add to response list
shot_data = ShotResponse.model_validate(db_shot)
shot_data.task_count = task_count
created_shots.append(shot_data)
# Commit all changes
db.commit()
message = f"Successfully created {len(created_shots)} shots in project {project_id}"
if total_tasks_created > 0:
message += f" with {total_tasks_created} tasks"
return BulkShotResponse(
created_shots=created_shots,
created_tasks_count=total_tasks_created,
message=message
)
except Exception as e:
db.rollback()
if isinstance(e, HTTPException):
raise e
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Error creating shots: {str(e)}"
)
@router.get("/{shot_id}", response_model=ShotResponse)
async def get_shot(
shot_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(get_current_user_with_db)
):
"""Get a specific shot by ID"""
from sqlalchemy.orm import joinedload, selectinload
# OPTIMIZATION: Use single query with optimized JOINs to fetch shot and all related data
# This replaces separate queries with a single database operation
shot_query = (
db.query(Shot)
.options(
joinedload(Shot.episode).joinedload(Episode.project), # Eager load episode and project
selectinload(Shot.tasks).options( # Use selectinload for better performance with tasks
selectinload(Task.assigned_user) # Eager load assigned users if needed
)
)
.filter(Shot.id == shot_id, Shot.deleted_at.is_(None))
)
shot = shot_query.first()
if not shot:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Shot not found"
)
# Check episode access
check_episode_access(shot.episode_id, current_user, db)
# OPTIMIZATION: Count tasks from the already loaded relationship
# This avoids a separate COUNT query
active_tasks = [task for task in shot.tasks if task.deleted_at is None]
task_count = len(active_tasks)
shot_data = ShotResponse.model_validate(shot)
shot_data.task_count = task_count
return shot_data
@router.post("/{shot_id}/tasks", response_model=TaskStatusInfo, status_code=status.HTTP_201_CREATED)
async def create_shot_task(
shot_id: int,
task_type: str,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Create a new task for a shot"""
# Exclude soft deleted shots
shot = db.query(Shot).filter(
Shot.id == shot_id,
Shot.deleted_at.is_(None)
).first()
if not shot:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Shot not found"
)
# Check episode access
episode = check_episode_access(shot.episode_id, current_user, db)
# Check if task already exists (exclude soft deleted)
existing_task = db.query(Task).filter(
Task.shot_id == shot_id,
Task.task_type == task_type,
Task.deleted_at.is_(None)
).first()
if existing_task:
# Return existing task info instead of error
return TaskStatusInfo(
task_type=existing_task.task_type,
status=existing_task.status,
task_id=existing_task.id,
assigned_user_id=existing_task.assigned_user_id
)
# Create the task
task_name = f"{shot.name} - {task_type.title()}"
db_task = Task(
project_id=shot.project_id,
episode_id=shot.episode_id,
shot_id=shot.id,
task_type=task_type,
name=task_name,
description=f"{task_type.title()} task for {shot.name}",
status="not_started"
)
db.add(db_task)
db.commit()
db.refresh(db_task)
return TaskStatusInfo(
task_type=db_task.task_type,
status=db_task.status,
task_id=db_task.id,
assigned_user_id=db_task.assigned_user_id
)
@router.put("/{shot_id}", response_model=ShotResponse)
async def update_shot(
shot_id: int,
shot_update: ShotUpdate,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Update a shot"""
from sqlalchemy.orm import selectinload
# OPTIMIZATION: Use eager loading to fetch shot with tasks in single query
db_shot = (
db.query(Shot)
.options(selectinload(Shot.tasks)) # Eager load tasks for counting
.filter(
Shot.id == shot_id,
Shot.deleted_at.is_(None)
)
.first()
)
if not db_shot:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Shot not found"
)
# Check episode access
check_episode_access(db_shot.episode_id, current_user, db)
# Validate project_id if provided
if shot_update.project_id is not None:
if shot_update.project_id != db_shot.project_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Cannot change project_id - must match episode's project"
)
# Validate frame range if both values are provided
frame_start = shot_update.frame_start if shot_update.frame_start is not None else db_shot.frame_start
frame_end = shot_update.frame_end if shot_update.frame_end is not None else db_shot.frame_end
if frame_end < frame_start:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Frame end must be greater than or equal to frame start"
)
# Check if new name conflicts with existing shots in the same project (project-scoped uniqueness)
if shot_update.name and shot_update.name != db_shot.name:
existing_shot = db.query(Shot).filter(
Shot.project_id == db_shot.project_id,
Shot.name == shot_update.name,
Shot.id != shot_id,
Shot.deleted_at.is_(None)
).first()
if existing_shot:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Shot with this name already exists in the project"
)
# Update only provided fields
update_data = shot_update.model_dump(exclude_unset=True)
for field, value in update_data.items():
setattr(db_shot, field, value)
db.commit()
db.refresh(db_shot)
# OPTIMIZATION: Count tasks using the relationship instead of separate query
# This avoids an additional database query
active_tasks = [task for task in db_shot.tasks if task.deleted_at is None]
task_count = len(active_tasks)
shot_data = ShotResponse.model_validate(db_shot)
shot_data.task_count = task_count
return shot_data
@router.get("/{shot_id}/deletion-info")
async def get_shot_deletion_info(
shot_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Get information about what will be deleted when deleting a shot"""
# Exclude soft deleted shots
db_shot = db.query(Shot).filter(
Shot.id == shot_id,
Shot.deleted_at.is_(None)
).first()
if not db_shot:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Shot not found"
)
# Check episode access
check_episode_access(db_shot.episode_id, current_user, db)
# Use the soft deletion service to get comprehensive deletion info
deletion_service = ShotSoftDeletionService()
deletion_info = deletion_service.get_deletion_info(shot_id, db)
if not deletion_info:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Shot not found"
)
return {
"shot_id": deletion_info.shot_id,
"shot_name": deletion_info.shot_name,
"episode_name": deletion_info.episode_name,
"project_name": deletion_info.project_name,
"task_count": deletion_info.task_count,
"submission_count": deletion_info.submission_count,
"attachment_count": deletion_info.attachment_count,
"note_count": deletion_info.note_count,
"review_count": deletion_info.review_count,
"total_file_size": deletion_info.total_file_size,
"file_count": deletion_info.file_count,
"affected_users": deletion_info.affected_users,
"last_activity_date": deletion_info.last_activity_date,
"created_at": deletion_info.created_at
}
@router.delete("/{shot_id}")
async def delete_shot(
shot_id: int,
db: Session = Depends(get_db),
current_user: User = Depends(require_coordinator_or_admin)
):
"""Soft delete a shot and all its associated data"""
# Exclude soft deleted shots
db_shot = db.query(Shot).filter(
Shot.id == shot_id,
Shot.deleted_at.is_(None)
).first()
if not db_shot:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Shot not found"
)
# Check episode access
check_episode_access(db_shot.episode_id, current_user, db)
# Use the soft deletion service to perform cascading soft deletion
deletion_service = ShotSoftDeletionService()
result = deletion_service.soft_delete_shot_cascade(shot_id, db, current_user)
if not result.success:
db.rollback()
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail={
"message": "Failed to delete shot",
"errors": result.errors
}
)
# Commit the transaction
db.commit()
return {
"message": f"Shot '{result.shot_name}' and all related data have been deleted",
"shot_id": result.shot_id,
"shot_name": result.shot_name,
"deleted_at": result.deleted_at,
"deleted_by": result.deleted_by,
"marked_deleted_tasks": result.marked_deleted_tasks,
"marked_deleted_submissions": result.marked_deleted_submissions,
"marked_deleted_attachments": result.marked_deleted_attachments,
"marked_deleted_notes": result.marked_deleted_notes,
"marked_deleted_reviews": result.marked_deleted_reviews,
"operation_duration": result.operation_duration
}
backend/routers/tasks.py
+1512
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More