indigo
e03cc62548
Wire SCAIL-2 driving + in-context mask conditioning into the trainer via the unified FlexibleStrategy, so the widened patchify_proj (Phase 2) can be trained. - flexible.py: new DrivingConditionConfig (driving-latent concat with a RoPE width offset ΔW) and MaskChannelsConditionConfig (semantic masks -> per-token channels), added to the condition union and get_data_sources. Driving is prepended (cond-first, target stays at the tail for loss slicing); mask channels are written onto the driving tokens via Modality.cond_channels, reusing ltx-core encode_mask_channels. The noisy target keeps a zero mask. - model_loader.load_transformer gains mask_conditioning_channels, widening the video patchify_proj with zero-init columns via a new live-module helper (widen_module_patchify_proj_for_mask_channels). ModelConfig exposes the field. - trainer unfreezes patchify_proj in LoRA mode when mask channels are active (the new input columns are new base params LoRA cannot reach). - configs/scail_animation_lora.yaml plus README / training-modes table rows. Verified on CPU (verify_phase3_trainer.py): config round-trips, prepare_training _inputs builds cond_channels [B,T,56] with the mask on the driving tokens, a tiny widened model forwards and compute_loss returns a finite [B] loss, and the widen helper is output-preserving at zero init. Real training needs Linux+GPU+checkpoint; dataset preprocessing (driving latents + semantic masks) and validation-runner wiring are left for later. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LTX-2 Trainer
This package provides tools and scripts for training and fine-tuning Lightricks' LTX-2 audio-video generation model. It supports LoRA training, full fine-tuning, and a flexible conditioning framework covering text-to-video, text-to-audio, image-to-video, video extension, audio extension, video inpainting, audio inpainting, video outpainting, IC-LoRA for video, audio, and joint audio-video references, audio-to-video, and video-to-audio.
📖 Documentation
All detailed guides and technical documentation are in the docs directory:
- ⚡ Quick Start Guide
- 🎬 Dataset Preparation
- 🛠️ Training Modes
- ⚙️ Configuration Reference
- 🚀 Training Guide
- 🧪 Inference Guide
- 🔧 Utility Scripts
- 🧩 Custom Training Strategies
- 📚 LTX-Core Documentation
- 🛡️ Troubleshooting Guide
🤖 Agent-Assisted Training
Use the train-model repository skill for an end-to-end guided run:
it probes your data and hardware, chooses the matching training mode, prepares/preprocesses the dataset, launches
training, and monitors the job while using the docs above as the source of truth.
🔧 Requirements
- LTX-2 Model Checkpoint - Local
.safetensorsfile - Gemma Text Encoder - Local Gemma model directory (required for LTX-2)
- Linux with CUDA - CUDA 13+ recommended for optimal performance
- Nvidia GPU with 80GB+ VRAM - Recommended for the standard config. For GPUs with 32GB VRAM (e.g., RTX 5090), use the low VRAM config which enables INT8 quantization and other memory optimizations
🤝 Contributing
We welcome contributions from the community! Here's how you can help:
- Share Your Work: If you've trained interesting LoRAs or achieved cool results, please share them with the community.
- Report Issues: Found a bug or have a suggestion? Open an issue on GitHub.
- Submit PRs: Help improve the codebase with bug fixes or general improvements.
- Feature Requests: Have ideas for new features? Let us know through GitHub issues.
💬 Join the Community
Have questions, want to share your results, or need real-time help?
Join our community Discord server to connect with other users and the development team!
- Get troubleshooting help
- Share your training results and workflows
- Collaborate on new ideas and features
- Stay up to date with announcements and updates
We look forward to seeing you there!
Happy training! 🎉