indigo/LTX-2
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Automated PR - 2026-06-17

Browse Source
This commit is contained in:
github-actions[bot]
2026-06-17 14:21:07 +00:00
parent d6053703e0
commit f4b06fb977
103 changed files with 12887 additions and 3665 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/ltx-trainer/docs/quick-start.md
+50 -20
View File
@@ -7,12 +7,14 @@ Get up and running with LTX-2 training in just a few steps!
Before you begin, ensure you have:
1. **LTX-2 Model Checkpoint** - A local `.safetensors` file containing the LTX-2 model weights.
Download `ltx-2-19b-dev.safetensors` from: [HuggingFace Hub](https://huggingface.co/Lightricks/LTX-2)
Download `ltx-2.3-22b-dev.safetensors` from: [HuggingFace Hub](https://huggingface.co/Lightricks/LTX-2.3)
The trainer supports LTX-2 and LTX-2.3 checkpoints through the same configuration API; version-specific components
are detected from the checkpoint.
2. **Gemma Text Encoder** - A local directory containing the Gemma model (required for LTX-2).
Download from: [HuggingFace Hub](https://huggingface.co/google/gemma-3-12b-it-qat-q4_0-unquantized/)
3. **Linux with CUDA** - The trainer requires `triton` which is Linux-only
3. **Linux with CUDA** - The trainer requires `triton` which is Linux-only; CUDA 13+ is recommended
4. **GPU with sufficient VRAM** - 80GB recommended for the standard config. For GPUs with 32GB VRAM (e.g., RTX 5090),
use the [low VRAM config](../configs/ltx2_av_lora_low_vram.yaml) which enables INT8 quantization and other
use the [low VRAM config](../configs/t2v_lora_low_vram.yaml) which enables INT8 quantization and other
memory optimizations
## ⚡ Installation
@@ -39,7 +41,18 @@ cd packages/ltx-trainer
## 🏋 Training Workflow
### 1. Prepare Your Dataset
If you are using an agent-enabled environment with repository skills, you can ask for the
[`train-model`](../../../.claude/skills/train-model/SKILL.md) skill to run this workflow with you.
It creates a run workspace, confirms the training mode, prepares data, preprocesses latents,
launches training, and monitors the run while stopping for approval before expensive steps.
### 1. Choose a Training Mode
Start with [`t2v_lora.yaml`](../configs/t2v_lora.yaml) for a first run with videos and captions. For modes such as
IC-LoRA, inpainting, or outpainting, check [Training Modes](training-modes.md) first because your metadata needs extra
columns such as `reference_video`, `video_mask`, or `audio_mask` before preprocessing.
### 2. Prepare Your Dataset
Organize your videos and captions, then preprocess them:
@@ -57,15 +70,18 @@ uv run python scripts/process_dataset.py dataset.json \
--text-encoder-path /path/to/gemma-model
```
By default, preprocessing writes to `.precomputed/`. Use that directory as `data.preprocessed_data_root`
in your training config.
See [Dataset Preparation](dataset-preparation.md) for detailed instructions.
### 2. Configure Training
### 3. Configure Training
Create or modify a configuration YAML file. Start with one of the example configs:
- [`configs/ltx2_av_lora.yaml`](../configs/ltx2_av_lora.yaml) - Audio-video LoRA training
- [`configs/ltx2_av_lora_low_vram.yaml`](../configs/ltx2_av_lora_low_vram.yaml) - Audio-video LoRA training (optimized for 32GB VRAM)
- [`configs/ltx2_v2v_ic_lora.yaml`](../configs/ltx2_v2v_ic_lora.yaml) - IC-LoRA video-to-video
- [`configs/t2v_lora.yaml`](../configs/t2v_lora.yaml) - Text-to-video LoRA
- [`configs/t2v_lora_low_vram.yaml`](../configs/t2v_lora_low_vram.yaml) - Same as above, tuned for ~32GB VRAM (INT8 quantization and memory optimizations)
- [`configs/v2v_ic_lora.yaml`](../configs/v2v_ic_lora.yaml) - IC-LoRA video-to-video
Key settings to update:
@@ -82,33 +98,47 @@ output_dir: "outputs/my_training_run"
See [Configuration Reference](configuration-reference.md) for all available options.
### 3. Start Training
### 4. Start Training
```bash
uv run python scripts/train.py configs/ltx2_av_lora.yaml
uv run python scripts/train.py configs/t2v_lora.yaml
```
For multi-GPU training:
```bash
uv run accelerate launch scripts/train.py configs/ltx2_av_lora.yaml
uv run accelerate launch scripts/train.py configs/t2v_lora.yaml
```
See [Training Guide](training-guide.md) for distributed training and advanced options.
## 🎯 Training Modes
> [!TIP]
> **First time?** Start with [`t2v_lora.yaml`](../configs/t2v_lora.yaml) — it's the simplest mode
> and only requires videos with captions. You can explore other modes once you've confirmed your
> setup works.
The trainer supports several training modes:
| Mode | Description | Config Example |
|----------------------|--------------------------------|--------------------------------------------|
| **LoRA** | Efficient adapter training | `training_strategy.name: "text_to_video"` |
| **Audio-Video LoRA** | Joint audio-video training | `training_strategy.with_audio: true` |
| **IC-LoRA** | Video-to-video transformations | `training_strategy.name: "video_to_video"` |
| **Full Fine-tuning** | Full model training | `model.training_mode: "full"` |
| Mode | Description | Example Config |
|-----------------------|--------------------------------------------|-------------------------------------------------------------------|
| **Text-to-Video** | Generate video+audio from text prompts | [`t2v_lora.yaml`](../configs/t2v_lora.yaml) |
| **Image-to-Video** | Animate from a starting image | [`i2v_lora.yaml`](../configs/i2v_lora.yaml) |
| **Video Extension** | Extend videos temporally (forward/backward)| [`video_extend_lora.yaml`](../configs/video_extend_lora.yaml), [`video_suffix_lora.yaml`](../configs/video_suffix_lora.yaml) |
| **IC-LoRA (V2V)** | Video-to-video transformations | [`v2v_ic_lora.yaml`](../configs/v2v_ic_lora.yaml) |
| **Audio-to-Video** | Generate video conditioned on audio | [`a2v_lora.yaml`](../configs/a2v_lora.yaml) |
| **Video-to-Audio** | Generate audio/foley from video | [`v2a_lora.yaml`](../configs/v2a_lora.yaml) |
| **Video Inpainting** | Fill in masked regions of video | [`video_inpainting_lora.yaml`](../configs/video_inpainting_lora.yaml) |
| **Video Outpainting** | Extend video spatially | [`video_outpainting_lora.yaml`](../configs/video_outpainting_lora.yaml) |
| **Text-to-Audio** | Generate audio from text prompts | [`t2a_lora.yaml`](../configs/t2a_lora.yaml) |
| **Audio Extension** | Extend audio temporally | [`audio_extend_lora.yaml`](../configs/audio_extend_lora.yaml), [`audio_suffix_lora.yaml`](../configs/audio_suffix_lora.yaml) |
| **Audio Inpainting** | Fill in masked regions of audio | [`audio_inpainting_lora.yaml`](../configs/audio_inpainting_lora.yaml) |
| **IC-LoRA (A2A)** | Audio-to-audio transformations | [`a2a_ic_lora.yaml`](../configs/a2a_ic_lora.yaml) |
| **AV2AV IC-LoRA** | Audio+video IC-LoRA transformations | [`av2av_ic_lora.yaml`](../configs/av2av_ic_lora.yaml) |
| **Full Fine-tuning** | Full model training (any mode above) | Set `model.training_mode: "full"` |
See [Training Modes](training-modes.md) for detailed explanations,
or [Custom Training Strategies](custom-training-strategies.md) if you need to implement your own training recipe.
See [Training Modes](training-modes.md) for detailed explanations of each mode.
## Next Steps
@@ -118,7 +148,7 @@ Once you've completed your first training run, you can:
production-ready inference
pipelines for various use cases (T2V, I2V, IC-LoRA, etc.). See the package documentation for details.
- Learn more about [Dataset Preparation](dataset-preparation.md) for advanced preprocessing
- Explore different [Training Modes](training-modes.md) (LoRA, Audio-Video, IC-LoRA)
- Explore different [Training Modes](training-modes.md)
- Dive deeper into [Training Configuration](configuration-reference.md)
- Understand the model architecture in [LTX-Core Documentation](../../ltx-core/README.md)