indigo
06c0870bbb
Two-stage distilled inference pipeline that animates a character from a driving video, wiring the Phase 1-3 SCAIL-2 conditioning into a runnable CLI. - scail_animation.py: ScailAnimationPipeline (mirrors distilled.py) plus a pure, testable build_scail_conditionings that assembles VideoConditionByDrivingLatent + VideoConditionByMaskChannels (driving appended, mask on the trailing driving tokens), and a load_masks helper. main() + scail_animation_arg_parser add --driving-video / --mask-path / --mode / --driving-strength on top of the standard two-stage distilled parser. - LTXModelConfigurator now reads config `mask_conditioning_channels`, so a SCAIL-trained checkpoint whose config declares it builds the widened patchify_proj automatically — no runtime widening wrapper needed at inference. - CLAUDE.md pipeline table row. Verified on CPU (verify_phase4_pipeline.py): the module imports, the CLI parses the SCAIL flags, build_scail_conditionings grows the sequence and places the mask channels on the driving tokens (target stays zero), driving-only leaves cond_channels None, and the configurator honors mask_conditioning_channels (patchify_proj widened from config). End-to-end runs still need a GPU and a SCAIL-trained checkpoint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LTX-2 Pipelines
High-level pipeline implementations for generating audio-video content with Lightricks' LTX-2 model. This package provides ready-to-use pipelines for text-to-video, image-to-video, video-to-video, audio-to-video, keyframe interpolation, and retake tasks.
Pipelines are built using building blocks from ltx-core (schedulers, guiders, noisers, patchifiers) and handle the complete inference flow including model loading, encoding, decoding, and file I/O.
Key Features:
- 🎬 Multiple Pipeline Types: Text-to-video, image-to-video, video-to-video, audio-to-video, keyframe interpolation, and retake
- ⚡ Optimized Performance: Support for FP8 transformers, gradient estimation, and memory optimization
- 🎯 Production Ready: Two-stage pipelines for best quality output
- 🔧 LoRA Support: Easy integration with trained LoRA adapters
- 📦 Self-Contained: Handles model loading, encoding, decoding, and file I/O
- 🚀 CLI Support: All pipelines can be run as command-line scripts
Quick Start
# From the repository root
uv sync --frozen
# Run a pipeline (example: two-stage text-to-video)
python -m ltx_pipelines.ti2vid_two_stages \
--checkpoint-path path/to/checkpoint.safetensors \
--distilled-lora path/to/distilled_lora.safetensors 0.8 \
--spatial-upsampler-path path/to/upsampler.safetensors \
--gemma-root path/to/gemma \
--prompt "A beautiful sunset over the ocean" \
--output-path output.mp4
See Installation & Usage for full setup, CLI modules, and shared flags.
📚 Documentation
| Topic | Description |
|---|---|
| Installation & Usage | Install, requirements, running pipelines from the CLI, common flags |
| Pipeline Selection Guide | Decision tree + feature comparison to pick the right pipeline |
| Available Pipelines | Full reference for all 11 pipelines |
| Conditioning Types | Image and video conditioning methods |
| Multimodal Guidance | CFG / STG / modality guidance parameters and tuning |
| Optimization Tips | FP8 quantization, torch.compile, gradient estimation |
| Multi-GPU Inference | Run a single generation across GPUs for latency (SP, TDP, distributed VAE, distributed Gemma) |
🔗 Related Projects
- LTX-Core - Core model implementation and inference components (schedulers, guiders, noisers, patchifiers)
- LTX-Trainer - Training and fine-tuning tools