Automated PR - 2026-06-17

2026-06-17 14:21:07 +00:00
parent d6053703e0
commit f4b06fb977
103 changed files with 12887 additions and 3665 deletions
@@ -10,7 +10,7 @@ sub-configurations:

 - **ModelConfig**: Base model and training mode settings
 - **LoraConfig**: LoRA training parameters
- **TrainingStrategyConfig**: Training strategy settings (text-to-video or video-to-video)
+- **TrainingStrategyConfig**: Training strategy settings (flexible conditioning framework)
 - **OptimizationConfig**: Learning rate, batch sizes, and scheduler settings
 - **AccelerationConfig**: Mixed precision and quantization settings
 - **DataConfig**: Data loading parameters
@@ -24,13 +24,29 @@ sub-configurations:

 Check out our example configurations in the `configs` directory:

- 📄 [Audio-Video LoRA Training](../configs/ltx2_av_lora.yaml) - Joint audio-video generation training
- 📄 [Audio-Video LoRA Training (Low VRAM)](../configs/ltx2_av_lora_low_vram.yaml) - Memory-optimized config for 32GB
-  GPUs (uses 8-bit optimizer, INT8 quantization, and reduced LoRA rank)
- 📄 [IC-LoRA Training](../configs/ltx2_v2v_ic_lora.yaml) - Video-to-video transformation training
+- 📄 [Text-to-Video LoRA](../configs/t2v_lora.yaml) - Text-to-video LoRA training
+- 📄 [Image-to-Video LoRA](../configs/i2v_lora.yaml) - Image-to-video LoRA training
+- 📄 [IC-LoRA Video-to-Video](../configs/v2v_ic_lora.yaml) - IC-LoRA video-to-video training
+- 📄 [Audio-to-Video LoRA](../configs/a2v_lora.yaml) - Audio-to-video LoRA training
+- 📄 [Video-to-Audio LoRA](../configs/v2a_lora.yaml) - Video-to-audio (Foley) LoRA training
+- 📄 [Video Extension LoRA](../configs/video_extend_lora.yaml) - Video extension (forward) LoRA training
+- 📄 [Video Suffix LoRA](../configs/video_suffix_lora.yaml) - Video extension (backward) LoRA training
+- 📄 [Video Inpainting LoRA](../configs/video_inpainting_lora.yaml) - Video inpainting LoRA training
+- 📄 [Video Outpainting LoRA](../configs/video_outpainting_lora.yaml) - Video outpainting (spatial crop) LoRA training
+- 📄 [Text-to-Audio LoRA](../configs/t2a_lora.yaml) - Text-to-audio LoRA training
+- 📄 [Audio Extension LoRA](../configs/audio_extend_lora.yaml) - Audio extension (forward) LoRA training
+- 📄 [Audio Suffix LoRA](../configs/audio_suffix_lora.yaml) - Audio extension (backward) LoRA training
+- 📄 [Audio Inpainting LoRA](../configs/audio_inpainting_lora.yaml) - Audio inpainting LoRA training
+- 📄 [Audio-to-Audio IC-LoRA](../configs/a2a_ic_lora.yaml) - Audio IC-LoRA transformation training
+- 📄 [AV2AV IC-LoRA](../configs/av2av_ic_lora.yaml) - Audio+video IC-LoRA transformation training
+- 📄 [T2V LoRA (Low VRAM)](../configs/t2v_lora_low_vram.yaml) - Memory-optimized config for 32GB GPUs

 ## ⚙️ Configuration Sections

+> [!NOTE]
+> The YAML snippets below show **recommended starting values**, not necessarily the code defaults.
+> Fields you omit from your config file will use the code defaults from [`config.py`](../src/ltx_trainer/config.py).
+
 ### ModelConfig

 Controls the base model and training mode settings.
@@ -149,37 +165,60 @@ target_modules:

 ### TrainingStrategyConfig

-Configures the training strategy. The trainer includes two built-in strategies described below.
-For custom use cases, see [Implementing Custom Training Strategies](custom-training-strategies.md).
+Configures the training strategy. The recommended strategy is `"flexible"`, which supports all conditioning scenarios through configuration.

-#### Text-to-Video Strategy
+#### Flexible Strategy
+
+The flexible strategy provides a unified conditioning framework. Each modality (video, audio) is configured
+independently with its own latents directory, generation flag, and list of conditions.

 ```yaml
 training_strategy:
-  name: "text_to_video"
-  first_frame_conditioning_p: 0.1     # Probability of first-frame conditioning
-  with_audio: false                   # Enable joint audio-video training
-  audio_latents_dir: "audio_latents"  # Directory for audio latents (when with_audio: true)
+  name: "flexible"
+  video:
+    is_generated: true                 # Video is denoised during training
+    latents_dir: "latents"             # Directory containing precomputed video latents
+    conditions:
+      - type: first_frame              # Use first frame as conditioning
+        probability: 0.5               # Apply this condition 50% of the time
+  audio:
+    is_generated: true                 # Audio is denoised during training
+    latents_dir: "audio_latents"       # Directory containing precomputed audio latents
+    conditions: []                     # No additional audio conditions (text-only)
 ```

-#### Video-to-Video Strategy (IC-LoRA)
+**ModalityConfig parameters:**

-```yaml
-training_strategy:
-  name: "video_to_video"
-  first_frame_conditioning_p: 0.1
-  reference_latents_dir: "reference_latents"  # Directory for reference video latents
-```
+| Parameter      | Description                                                                                                      |
+|----------------|------------------------------------------------------------------------------------------------------------------|
+| `is_generated` | `true` = modality is denoised (contributes to loss). `false` = frozen conditioning (sigma=0, no loss).           |
+| `latents_dir`  | Directory name within `preprocessed_data_root` containing precomputed latents for this modality.                 |
+| `conditions`   | List of conditioning configs applied during training (see condition types below). Text conditioning is implicit. |

-**Key parameters:**
+**Condition types:**

-| Parameter                    | Description                                                      |
-|------------------------------|------------------------------------------------------------------|
-| `name`                       | Strategy type: `"text_to_video"` or `"video_to_video"`           |
-| `first_frame_conditioning_p` | Probability of using first frame as conditioning (0.0-1.0)       |
-| `with_audio`                 | (text_to_video only) Enable joint audio-video training           |
-| `audio_latents_dir`          | (text_to_video only) Directory name for audio latents            |
-| `reference_latents_dir`      | (video_to_video only) Directory name for reference video latents |
+| Type           | Parameters                                          | Description                                                                           |
+|----------------|-----------------------------------------------------|---------------------------------------------------------------------------------------|
+| `first_frame`  | `probability`                                       | First latent frame is clean, excluded from loss. **Video only.**                      |
+| `prefix`       | `temporal_boundary`, `probability`                  | First N latent temporal units are clean. For extension forward.                       |
+| `suffix`       | `temporal_boundary`, `probability`                  | Last N latent temporal units are clean. For extension backward.                       |
+| `spatial_crop` | `spatial_region` (y1, x1, y2, x2 in px), `probability` | Rectangular region is clean, excluded from loss. For outpainting. **Video only.** |
+| `mask`         | `mask_dir`, `probability`                           | Per-sample mask directory. Masks are thresholded at `0.5`; `1` means conditioning, `0` means generate. |
+| `reference`    | `latents_dir`, `probability`                        | IC-LoRA style concatenation. Reference tokens are prepended, clean (timestep=0), no loss. |
+
+> [!NOTE]
+> The `prefix`, `suffix`, `mask`, and `reference` condition types work on both video and audio modalities —
+> place them in the `video.conditions` or `audio.conditions` list as appropriate.
+> `first_frame` and `spatial_crop` are video-only conditions.
+
+> [!NOTE]
+> Training conditions reference **directories** of precomputed data (within `preprocessed_data_root`),
+> while validation conditions reference **individual files** (images, videos, masks) that are encoded
+> on-the-fly during validation. The condition `type` names are the same, but the fields differ.
+
+> [!NOTE]
+> The legacy `text_to_video` and `video_to_video` strategies are deprecated but remain forward-compatible.
+> New configs should use `name: "flexible"`.

 ### OptimizationConfig

@@ -206,7 +245,7 @@ optimization:
 | `steps`                         | Total number of training steps                                                               |
 | `batch_size`                    | Batch size per GPU (reduce if running out of memory)                                         |
 | `gradient_accumulation_steps`   | Accumulate gradients over multiple steps                                                     |
-| `scheduler_type`                | LR scheduler: `"constant"`, `"linear"`, `"cosine"`, `"cosine_with_restarts"`, `"polynomial"` |
+| `scheduler_type`                | LR scheduler: `"constant"`, `"linear"`, `"cosine"`, `"cosine_with_restarts"`, `"polynomial"`, `"step"` |
 | `enable_gradient_checkpointing` | Trade training speed for GPU memory savings (recommended for large models)                   |

 ### AccelerationConfig
@@ -226,7 +265,7 @@ acceleration:
 | Parameter                             | Description                                                                                                                                                                              |
 |---------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | `mixed_precision_mode`                | Precision mode - `"bf16"` recommended for modern GPUs                                                                                                                                    |
-| `quantization`                        | Model quantization: `null`, `"int8-quanto"`, `"int4-quanto"`, `"fp8-quanto"`, etc.                                                                                                       |
+| `quantization`                        | Model quantization: `null`, `"int8-quanto"`, `"int4-quanto"`, `"int2-quanto"`, `"fp8-quanto"`, or `"fp8uz-quanto"`                                                                       |
 | `load_text_encoder_in_8bit`           | Load the Gemma text encoder in 8-bit to save GPU memory                                                                                                                                  |
 | `offload_optimizer_during_validation` | Move optimizer state to CPU before validation video sampling and back afterwards. Useful when validation OOMs because VAE decoder + transformer + optimizer state can't coexist on the GPU (full fine-tune, high-rank LoRA). No effect for FSDP. |

@@ -244,50 +283,84 @@ data:

 | Parameter                | Description                                                                                |
 |--------------------------|--------------------------------------------------------------------------------------------|
-| `preprocessed_data_root` | Path to your preprocessed dataset (contains `latents/`, `conditions/`, etc.)               |
+| `preprocessed_data_root` | Path to your preprocessed dataset directory produced by `process_dataset.py` (contains `latents/`, `conditions/`, etc.) |
 | `num_dataloader_workers` | Number of parallel data loading processes (0 = synchronous loading, useful when debugging) |

 ### ValidationConfig

-Validation and inference settings for monitoring training progress.
+Validation and inference settings for monitoring training progress. Validation samples use a self-describing
+format where each sample specifies its own prompt and conditions.

 ```yaml
 validation:
-  prompts: # Validation prompts
-    - "A cat playing with a ball"
-    - "A dog running in a field"
-  negative_prompt: "worst quality, inconsistent motion, blurry, jittery, distorted"
-  images: null                        # Optional image paths for image-to-video
-  reference_videos: null              # Reference video paths (IC-LoRA only)
-  video_dims: [ 576, 576, 89 ]        # Video dimensions [width, height, frames]
-  frame_rate: 25.0                    # Frame rate for generated videos
-  seed: 42                            # Random seed for reproducibility
-  inference_steps: 30                 # Number of inference steps
-  interval: 100                       # Steps between validation runs
-  guidance_scale: 4.0                 # CFG guidance strength
-  stg_scale: 1.0                      # STG guidance strength (0.0 to disable)
-  stg_blocks: [ 29 ]                  # Transformer blocks to perturb for STG
-  stg_mode: "stg_av"                  # "stg_av" or "stg_v" (video only)
-  generate_audio: true                # Whether to generate audio
-  skip_initial_validation: false      # Skip validation at step 0
-  include_reference_in_output: false  # Include reference video side-by-side (IC-LoRA)
+  samples:
+    - prompt: "A cat playing with a ball"
+      conditions:
+        - type: first_frame
+          image_or_video: "/path/to/image.png"
+    - prompt: "A dog running in a field"
+  video_dims: [576, 576, 89]                # Output dimensions: [width, height, frames]
+  negative_prompt: "worst quality, inconsistent motion, blurry, jittery, distorted"  # Negative prompt for all samples
+  frame_rate: 25.0                          # Output video frame rate (fps)
+  seed: 42                                  # Random seed for reproducibility
+  inference_steps: 30                       # Number of denoising steps
+  interval: 100                             # Run validation every N steps (null to disable)
+  guidance_scale: 4.0                       # CFG scale (higher = stronger prompt adherence)
+  stg_scale: 1.0                            # STG scale (0.0 to disable)
+  stg_blocks: [29]                          # Transformer blocks to apply STG perturbation
+  stg_mode: "stg_av"                        # STG mode: "stg_av" (audio+video) or "stg_v" (video only)
+  generate_audio: true                      # Whether to generate audio during validation
+  generate_video: true                      # Whether to generate video during validation
+  skip_initial_validation: false            # Skip validation at step 0
 ```

 **Key parameters:**

-| Parameter                     | Description                                                                                                              |
-|-------------------------------|--------------------------------------------------------------------------------------------------------------------------|
-| `prompts`                     | List of text prompts for validation video generation                                                                     |
-| `images`                      | List of image paths for image-to-video validation (must match number of prompts)                                         |
-| `reference_videos`            | List of reference video paths for IC-LoRA validation (must match number of prompts)                                      |
-| `video_dims`                  | Output dimensions `[width, height, frames]`. Width/height must be divisible by 32, frames must satisfy `frames % 8 == 1` |
-| `interval`                    | Steps between validation runs (set to `null` to disable)                                                                 |
-| `guidance_scale`              | CFG (Classifier-Free Guidance) scale. Recommended: 4.0                                                                   |
-| `stg_scale`                   | STG (Spatio-Temporal Guidance) scale. 0.0 disables STG. Recommended: 1.0                                                 |
-| `stg_blocks`                  | Transformer blocks to perturb for STG. Recommended: `[29]` (single block)                                                |
-| `stg_mode`                    | STG mode: `"stg_av"` perturbs both audio and video, `"stg_v"` perturbs video only                                        |
-| `generate_audio`              | Whether to generate audio in validation samples                                                                          |
-| `include_reference_in_output` | For IC-LoRA: concatenate reference video side-by-side with output                                                        |
+| Parameter                | Description                                                                                                              |
+|--------------------------|--------------------------------------------------------------------------------------------------------------------------|
+| `samples`                | List of `ValidationSample` objects (see below). Replaces the legacy `prompts`/`images`/`reference_videos` fields.        |
+| `video_dims`             | Output dimensions `[width, height, frames]`. Width/height must be divisible by 32, frames must satisfy `frames % 8 == 1` |
+| `interval`               | Steps between validation runs (set to `null` to disable)                                                                 |
+| `guidance_scale`         | CFG (Classifier-Free Guidance) scale. Recommended: 4.0                                                                   |
+| `stg_scale`              | STG (Spatio-Temporal Guidance) scale. 0.0 disables STG. Recommended: 1.0                                                 |
+| `stg_blocks`             | Transformer blocks to perturb for STG. Recommended: `[29]` (single block)                                                |
+| `stg_mode`               | STG mode: `"stg_av"` perturbs both audio and video, `"stg_v"` perturbs video only                                        |
+| `generate_audio`         | Whether to generate audio in validation samples                                                                          |
+| `generate_video`         | Whether to generate video in validation samples. Set to `false` for V2A (video-to-audio) validation. Default: `true`     |
+| `skip_initial_validation`| Skip validation video sampling at step 0 (beginning of training)                                                         |
+
+#### ValidationSample
+
+Each sample in the `samples` list has:
+
+| Field        | Description                                                                                     |
+|--------------|-------------------------------------------------------------------------------------------------|
+| `prompt`     | Text prompt for this validation sample.                                                         |
+| `conditions` | List of validation conditions (see types below). Empty list = text-only generation.             |
+| `video_dims` | Optional per-sample override for `(width, height, frames)`. Inherits from `ValidationConfig` if not set. |
+| `seed`       | Optional per-sample override for random seed. Inherits from `ValidationConfig` if not set.      |
+
+#### Validation Condition Types
+
+| Type             | Parameters                                                 | Description                                                             |
+|------------------|------------------------------------------------------------|-------------------------------------------------------------------------|
+| `first_frame`    | `image_or_video` (path)                                    | Use the first frame of the image/video as conditioning.                 |
+| `prefix`         | `video` or `audio` (path), optional `num_frames`/`duration`| Use a video/audio clip as temporal prefix (for extension forward).      |
+| `suffix`         | `video` or `audio` (path), optional `num_frames`/`duration`| Use a video/audio clip as temporal suffix (for extension backward).     |
+| `spatial_crop`   | `video` (path), `spatial_region` (y1, x1, y2, x2)         | Provide spatial context for outpainting. Video only.                    |
+| `mask`           | `video` or `audio` (path), `mask` (path)                   | Mask-based inpainting with a binary mask file.                          |
+| `reference`      | `video` or `audio` (path), optional video-reference `downscale_factor`, `temporal_scale_factor`, `include_in_output` | IC-LoRA style reference conditioning. |
+| `video_to_audio` | `video` (path)                                             | Freeze video, generate audio. For Foley/V2A tasks.                      |
+| `audio_to_video` | `audio` (path)                                             | Freeze audio, generate video. For audio-driven generation.              |
+
+For video `reference` validation conditions, `downscale_factor` is the spatial reference scale and
+`temporal_scale_factor` is the temporal reference scale. Set both to match the factors used when
+preprocessing video reference latents for training; validation media is encoded on the fly and cannot infer
+those factors from the training dataset.
+
+> [!NOTE]
+> The legacy fields `prompts`, `images`, and `reference_videos` are deprecated but auto-converted to `samples`
+> internally. New configs should use the `samples` format.

 ### CheckpointsConfig

@@ -298,6 +371,8 @@ checkpoints:
  interval: 250       # Steps between checkpoint saves (null = disabled)
  keep_last_n: 3      # Number of recent checkpoints to retain
  precision: bfloat16 # Precision for saved weights (bfloat16 or float32)
+  no_resume: false            # Ignore saved state, start from step 0
+  save_training_state: "minimal"  # "full", "minimal", or "off"
 ```

 **Key parameters:**
@@ -307,6 +382,8 @@ checkpoints:
 | `interval`    | Steps between intermediate checkpoint saves (set to `null` to disable)        |
 | `keep_last_n` | Number of most recent checkpoints to keep (-1 = keep all)                     |
 | `precision`   | Precision for saved checkpoint weights: `"bfloat16"` (default) or `"float32"` |
+| `no_resume` | When `true`, ignore saved training state and start from step 0. Model weights from `load_checkpoint` are still loaded. |
+| `save_training_state` | Save training state for resume: `"full"` (optimizer + scheduler + RNG), `"minimal"` (scheduler + RNG only, sufficient for LoRA), `"off"` (no resume). |

 ### HubConfig

@@ -364,6 +441,20 @@ flow_matching:
 | `timestep_sampling_mode`   | Sampling strategy: `"uniform"` or `"shifted_logit_normal"` |
 | `timestep_sampling_params` | Additional parameters for the sampling strategy            |

+### General Configuration
+
+Top-level settings for the training run.
+
+```yaml
+seed: 42                                  # Random seed for reproducibility
+output_dir: "outputs/my_training_run"     # Directory for outputs (checkpoints, validation videos, logs)
+```
+
+| Parameter    | Description                                              |
+|--------------|----------------------------------------------------------|
+| `seed`       | Random seed for reproducibility (default: `42`)          |
+| `output_dir` | Directory to save outputs (default: `"outputs"`)         |
+
 ## 🚀 Next Steps

 Once you've configured your training parameters:
@@ -1,7 +1,7 @@
 # Implementing Custom Training Strategies

-This guide explains how to implement your own training strategy for specialized use cases like audio-only training,
-video inpainting, or other custom training recipes.
+This guide explains how to implement your own training strategy for specialized recipes that cannot be expressed with
+the built-in `flexible` strategy.

 ## 📋 Overview

@@ -15,12 +15,20 @@ This architecture lets you implement new training modes without modifying the co

 ### When You Need a Custom Strategy

+> [!NOTE]
+> The built-in `flexible` strategy already supports most conditioning scenarios out of the box:
+> first-frame conditioning, video extension (prefix/suffix), spatial crop (outpainting),
+> mask-based inpainting, IC-LoRA reference conditioning, and frozen modality cross-conditioning
+> (audio-to-video, video-to-audio). Only implement a custom strategy if your use case requires
+> fundamentally different training logic that cannot be expressed through the flexible strategy's
+> configuration.
+
 Consider implementing a custom strategy when you need:

- **Different input modalities** (e.g., audio-only, audio-to-video conditioning)
- **Additional conditioning signals** (e.g., masks for inpainting, depth maps)
- **Custom loss computation** (e.g., weighted losses, auxiliary losses)
- **Different noise application patterns** (e.g., partial masking)
+- **Custom loss computation** (e.g., weighted losses, auxiliary losses, perceptual losses)
+- **Non-standard noise application** (e.g., noise schedules different from flow matching)
+- **Novel conditioning mechanisms** not covered by the flexible strategy's condition types
+- **Additional model outputs** beyond the standard video/audio predictions

 ## 🏗️ Architecture Overview

@@ -28,7 +36,7 @@ Consider implementing a custom strategy when you need:

 The trainer delegates all training-mode-specific logic to the strategy:

-1. **Initialization** — The trainer calls `get_data_sources()` to determine which preprocessed data directories to load
+1. **Initialization** — The trainer calls `config.get_data_sources()` to determine which preprocessed data directories to load
 2. **Each training step:**
    - Calls `prepare_training_inputs()` to transform the raw batch into model-ready inputs
    - Runs the transformer forward pass
@@ -52,8 +60,8 @@ The trainer handles everything else: optimization, checkpointing, validation, an
 Before writing code, answer these questions:

 1. **What additional data does your strategy need?**
-    - Example: Inpainting needs mask latents alongside video latents
-    - Example: Audio-to-video needs reference audio embeddings
+    - Example: A perceptual-loss strategy may need auxiliary feature targets
+    - Example: A novel conditioning mechanism may need an additional precomputed directory

 2. **What does conditioning look like?**
    - Which tokens should be noised vs. kept clean?
@@ -164,6 +172,20 @@ class InpaintingConfig(TrainingStrategyConfigBase):
        ge=0.0,
        le=1.0,
    )
+
+    def get_data_sources(self) -> dict[str, str]:
+        """Define which data directories to load.
+
+        Returns a mapping of directory names (under preprocessed_data_root) to
+        batch keys. The trainer loads .pt files from each directory and exposes
+        them in the batch under the specified key. The trainer also uses this
+        mapping to validate that all required directories exist.
+        """
+        return {
+            "latents": "latents",  # -> batch["latents"]
+            "conditions": "conditions",  # -> batch["conditions"]
+            self.mask_latents_dir: "masks",  # -> batch["masks"]
+        }
 ```

 **Key points:**
@@ -171,6 +193,7 @@ class InpaintingConfig(TrainingStrategyConfigBase):
 - Inherit from `TrainingStrategyConfigBase`
 - Use `Literal["your_strategy_name"]` for the `name` field - this enables automatic strategy selection
 - Use Pydantic `Field` for validation and documentation
+- Implement `get_data_sources()` on the config — it's the single source of truth for data directories (used for both dataset wiring and existence validation)

 ### Step 4: Implement the Strategy Class

@@ -187,24 +210,6 @@ class InpaintingStrategy(TrainingStrategy):
    def __init__(self, config: InpaintingConfig):
        super().__init__(config)

-    @property
-    def requires_audio(self) -> bool:
-        """Whether this strategy requires audio components."""
-        return False  # Set to True if your strategy needs audio
-
-    def get_data_sources(self) -> dict[str, str]:
-        """Define which data directories to load.
-
-        Returns a mapping of directory names to batch keys.
-        The trainer will load .pt files from each directory and
-        make them available in the batch under the specified key.
-        """
-        return {
-            "latents": "latents",  # -> batch["latents"]
-            "conditions": "conditions",  # -> batch["conditions"]
-            self.config.mask_latents_dir: "masks",  # -> batch["masks"]
-        }
-
    def prepare_training_inputs(
            self,
            batch: dict[str, Any],
@@ -275,7 +280,6 @@ class InpaintingStrategy(TrainingStrategy):
            batch_size=batch_size,
            fps=24.0,  # Or get from latents_data
            device=device,
-            dtype=dtype,
        )

        # Create video Modality
@@ -328,7 +332,7 @@ You need to register your strategy in two places:
 from ltx_trainer.training_strategies.inpainting import InpaintingConfig, InpaintingStrategy

 # Add to the TrainingStrategyConfig type alias
-TrainingStrategyConfig = TextToVideoConfig | VideoToVideoConfig | InpaintingConfig
+TrainingStrategyConfig = TextToVideoConfig | VideoToVideoConfig | FlexibleStrategyConfig | InpaintingConfig

 # Add to __all__
 __all__ = [
@@ -356,7 +360,8 @@ from ltx_trainer.training_strategies.inpainting import InpaintingConfig
 TrainingStrategyConfig = Annotated[
    Annotated[TextToVideoConfig, Tag("text_to_video")]
    | Annotated[VideoToVideoConfig, Tag("video_to_video")]
-    | Annotated[InpaintingConfig, Tag("inpainting")],  # Add your config
+    | Annotated[FlexibleStrategyConfig, Tag("flexible")]
+    | Annotated[InpaintingConfig, Tag("inpainting")],
    Discriminator(_get_strategy_discriminator),
 ]
 ```
@@ -366,7 +371,7 @@ TrainingStrategyConfig = Annotated[
 Create an example config in `configs/`:

 ```yaml
-# configs/ltx2_inpainting_lora.yaml
+# configs/custom_inpainting_lora.yaml

 model:
  model_path: "/path/to/ltx2.safetensors"
@@ -408,8 +413,8 @@ The base `TrainingStrategy` class provides these helper methods:
 | `_audio_patchifier.patchify(latents)`        | Convert `[B, C, T, F]` → `[B, T, C*F]`          |
 | `_get_video_positions(...)`                  | Generate position embeddings for video          |
 | `_get_audio_positions(...)`                  | Generate position embeddings for audio          |
-| `_create_per_token_timesteps(mask, sigma)`   | Create timesteps with 0 for conditioning tokens |
-| `_create_first_frame_conditioning_mask(...)` | Create mask for first-frame conditioning        |
+| `_create_per_token_timesteps(conditioning_mask, sampled_sigma)` | Create timesteps with 0 for conditioning tokens |
+| `_create_first_frame_conditioning_mask(...)`                    | Create mask for first-frame conditioning        |

 ## 📊 Understanding ModelInputs

@@ -418,16 +423,14 @@ The `ModelInputs` dataclass contains everything needed for the forward pass and
 ```python
@dataclass
 class ModelInputs:
-    video: Modality  # Video modality data
-    audio: Modality | None  # Audio modality (None if video-only)
+    video: Modality | None  # Video modality data
+    audio: Modality | None  # Audio modality data

-    video_targets: Tensor  # Target values for loss (velocity)
-    audio_targets: Tensor | None
+    video_targets: Tensor | None  # Target values for video loss (velocity)
+    audio_targets: Tensor | None  # Target values for audio loss (velocity)

-    video_loss_mask: Tensor  # Boolean: True = compute loss for this token
-    audio_loss_mask: Tensor | None
-
-    ref_seq_len: int | None = None  # For IC-LoRA: reference sequence length
+    video_loss_mask: Tensor | None  # Boolean loss mask for video tokens
+    audio_loss_mask: Tensor | None  # Boolean loss mask for audio tokens
 ```

 ## 📊 Understanding Modality
@@ -437,18 +440,20 @@ The `Modality` dataclass (from ltx-core) represents a single modality's data:
 ```python
@dataclass(frozen=True)
 class Modality:
-    enabled: bool  # Whether this modality is active
-    latent: Tensor  # [B, seq_len, C] - the latent tokens
-    timesteps: Tensor  # [B, seq_len] - per-token timesteps (sigmas)
-    positions: Tensor  # [B, dims, seq_len, 2] - position bounds
-    context: Tensor  # [B, ctx_len, C] - text embeddings
-    context_mask: Tensor  # [B, ctx_len] - attention mask for context
+    latent: Tensor     # [B, T, D] — patchified latent tokens
+    sigma: Tensor      # [B,] — per-batch noise level (for cross-attn conditioning)
+    timesteps: Tensor  # [B, T] — per-token timestep embeddings
+    positions: Tensor  # [B, 3, T, 2] for video, [B, 1, T, 2] for audio — positional bounds
+    context: Tensor    # text conditioning embeddings
+    enabled: bool = True
+    context_mask: Tensor | None = None     # attention mask for text context
+    attention_mask: Tensor | None = None   # optional 2D self-attention mask [B, T, T]
 ```

 > [!NOTE]
 > **Per-token timesteps:** Each token in the sequence has its own timestep. Conditioning tokens—those that should remain
 > un-noised—must have `timestep=0`. This is how the model distinguishes clean reference tokens from tokens to denoise. Use
-`_create_per_token_timesteps(conditioning_mask, sigma)` to set this up correctly.
+> `_create_per_token_timesteps(conditioning_mask, sampled_sigma)` to set this up correctly.

 > [!NOTE]
 > `Modality` is immutable (frozen dataclass). Use `dataclasses.replace()` to create modified copies.
@@ -461,7 +466,7 @@ class Modality:
   from ltx_trainer.config import LtxTrainerConfig
   import yaml

-   with open('configs/ltx2_inpainting_lora.yaml') as f:
+   with open('configs/custom_inpainting_lora.yaml') as f:
       config = LtxTrainerConfig(**yaml.safe_load(f))
   print(f'Strategy: {config.training_strategy.name}')
   "
@@ -475,13 +480,13 @@ class Modality:

   config = InpaintingConfig()
   strategy = get_training_strategy(config)
-   print(f'Data sources: {strategy.get_data_sources()}')
+   print(f'Data sources: {config.get_data_sources()}')
   "
   ```

 3. **Run a short training test:**
   ```bash
-   uv run python scripts/train.py configs/ltx2_inpainting_lora.yaml
+   uv run python scripts/train.py configs/custom_inpainting_lora.yaml
   ```

 ## 💡 Tips and Best Practices
@@ -503,7 +508,8 @@ class Modality:

 Study these implementations for guidance:

-| Strategy                                                                           | Complexity | Key Features                                   |
-|------------------------------------------------------------------------------------|------------|------------------------------------------------|
-| [`TextToVideoStrategy`](../src/ltx_trainer/training_strategies/text_to_video.py)   | Simple     | First-frame conditioning, optional audio       |
-| [`VideoToVideoStrategy`](../src/ltx_trainer/training_strategies/video_to_video.py) | Medium     | Reference video concatenation, split loss mask |
+| Strategy | Complexity | Key Features |
+|----------|------------|--------------|
+| [`FlexibleStrategy`](../src/ltx_trainer/training_strategies/flexible.py) | Medium | Unified conditioning framework — supports all built-in modes |
+| [`TextToVideoStrategy`](../src/ltx_trainer/training_strategies/text_to_video.py) | Simple | First-frame conditioning, optional audio (deprecated) |
+| [`VideoToVideoStrategy`](../src/ltx_trainer/training_strategies/video_to_video.py) | Medium | Reference video concatenation, split loss mask (deprecated) |
@@ -33,41 +33,33 @@ uv run python scripts/split_scenes.py --help

 If your dataset doesn't include captions, you can automatically generate them using multimodal models that understand both video and audio.

+The default `qwen_omni` backend talks to a local vLLM server, which you launch once in a separate terminal:
+
 ```bash
-uv run python scripts/caption_videos.py scenes_output_dir/ \
-    --output scenes_output_dir/dataset.json
+# Terminal 1: start the captioner server (stays running)
+uv run python scripts/serve_captioner.py
 ```

-If you're running into VRAM issues, try enabling 8-bit quantization to reduce memory usage:
-
 ```bash
+# Terminal 2: caption your videos
 uv run python scripts/caption_videos.py scenes_output_dir/ \
-    --output scenes_output_dir/dataset.json \
-    --use-8bit
+    --output scenes_output_dir/dataset.json
 ```

 This will create a `dataset.json` file containing video paths and their captions.

 **Captioning options:**

-
-| Option             | Description                                                |
-| ------------------ | ---------------------------------------------------------- |
-| `--captioner-type` | `qwen_omni` (default, local) or `gemini_flash` (API)       |
-| `--use-8bit`       | Enable 8-bit quantization for lower VRAM usage             |
-| `--no-audio`       | Disable audio processing (video-only captions)             |
-| `--override`       | Re-caption files that already have captions                |
-| `--api-key`        | API key for Gemini Flash (or set `GOOGLE_API_KEY` env var) |
-
+| Option             | Description                                                      |
+| ------------------ | --------------------------------------------------------------- |
+| `--captioner-type` | `qwen_omni` (default, local vLLM server) or `gemini_flash` (API) |
+| `--vllm-url`       | Base URL of the vLLM server (default `http://127.0.0.1:8001/v1`) |
+| `--override`       | Re-caption files that already have captions                     |
+| `--api-key`        | Gemini API key (else `GEMINI_API_KEY`/`GOOGLE_API_KEY`; with no key, uses gcloud/Vertex AI auth) |

 **Caption format:**

-The captioner produces structured captions with sections for:
-
- **Visual content**: People, objects, actions, settings, colors, movements
- **Speech transcription**: Word-for-word transcription of spoken content
- **Sounds**: Music, ambient sounds, sound effects
- **On-screen text**: Any visible text overlays
+Each caption is a single, detailed paragraph describing both the visual content and the audio (speech, music, ambient sounds) of the clip. See the [Utility Scripts Reference](utility-scripts.md#automatic-video-captioning) for backend setup and the full list of options.

 > [!NOTE]
 > The automatically generated captions may contain inaccuracies or hallucinated content.
@@ -80,7 +72,7 @@ This step preprocesses your video dataset by:
 1. Resizing and cropping videos to fit specified resolution buckets
 2. Computing and caching video latent representations
 3. Computing and caching text embeddings for captions
-4. (Optional) Computing and caching audio latents
+4. Extracting and caching audio latents from videos (automatic, use `--skip-audio` to disable)

 > [!WARNING]
 > Very large videos (especially high spatial resolution and/or many frames) can cause GPU out-of-memory (OOM)
@@ -97,17 +89,9 @@ uv run python scripts/process_dataset.py dataset.json \
    --text-encoder-path /path/to/gemma-model
 ```

-### With Audio Processing
-
-For audio-video training, add the `--with-audio` flag:
-
-```bash
-uv run python scripts/process_dataset.py dataset.json \
-    --resolution-buckets "960x544x49" \
-    --model-path /path/to/ltx-2-model.safetensors \
-    --text-encoder-path /path/to/gemma-model \
-    --with-audio
-```
+Audio latents are automatically extracted from video files — no extra flag is needed. Use `--skip-audio`
+to disable this. For standalone audio files (`.wav`), use the `audio` column in your dataset instead
+(see [Convention-Based Column Detection](#convention-based-column-detection) below).

 ### 🚀 Multi-GPU Preprocessing

@@ -126,7 +110,7 @@ Outputs are written atomically (via a per-process temporary file, then renamed),
 corrupt files. By default a rerun **resumes** — items whose output `.pt` already exists are skipped.

 > [!IMPORTANT]
-> Pass `**--overwrite`** when rerunning with changed parameters (different model checkpoint, resolution buckets,
+> Pass **`--overwrite`** when rerunning with changed parameters (different model checkpoint, resolution buckets,
 > text encoder, `--lora-trigger`, etc.). Without it the script keeps the stale outputs from the previous run.
 >
 > ```bash
@@ -152,7 +136,7 @@ The trainer supports videos, single images, or a mix of both in the same dataset
 > `--resolution-buckets "960x544x1;960x544x49"`. Images are automatically assigned to the `F=1` bucket and
 > videos to an `F>1` bucket.
 > - You **must** set `optimization.batch_size: 1` in your training config (see the warning under
-> [Resolution Buckets](#-resolution-buckets)), since samples with different shapes cannot be collated into a
+> [Resolution Buckets](#resolution-buckets)), since samples with different shapes cannot be collated into a
 > single batch. Use `gradient_accumulation_steps` if you need a larger effective batch.
 > - Per-step cost differs substantially between a single-frame sample and a many-frame sample, which can lead to
 > uneven gradient magnitudes across steps. Consider weighting the two subsets or tuning the learning rate if
@@ -160,7 +144,24 @@ The trainer supports videos, single images, or a mix of both in the same dataset
 > - If you prefer a fully officially-supported path, train two separate LoRAs (one on stills, one on video) and
 > stack them at inference.

-The dataset must be a CSV, JSON, or JSONL metadata file with columns for captions and video paths:
+The dataset must be a CSV, JSON, or JSONL metadata file with columns for captions and media paths.
+
+#### Convention-Based Column Detection
+
+The preprocessing script automatically detects and processes columns based on their names. The following columns are recognized:
+
+| Column | Output Dir | Description |
+|--------|-----------|-------------|
+| `video` (or legacy `media_path`) | `latents/` | Target video to encode |
+| `audio` | `audio_latents/` | Explicit audio file (overrides auto-extraction from video) |
+| `caption` | `conditions/` | Text caption for the sample |
+| `reference_video` (or legacy `ref_media_path`) | `reference_latents/` | IC-LoRA reference video |
+| `reference_audio` | `reference_audio_latents/` | IC-LoRA reference audio |
+| `video_mask` | `video_masks/` | Binary mask for video inpainting |
+| `audio_mask` | `audio_masks/` | Binary mask for audio inpainting |
+
+> [!NOTE]
+> **Legacy column names:** `media_path` and `ref_media_path` are accepted as aliases for `video` and `reference_video` respectively. Existing datasets using these names will continue to work without modification.

 **JSON format example:**

@@ -168,11 +169,11 @@ The dataset must be a CSV, JSON, or JSONL metadata file with columns for caption
 [
  {
    "caption": "A cat playing with a ball of yarn",
-    "media_path": "videos/cat_playing.mp4"
+    "video": "videos/cat_playing.mp4"
  },
  {
    "caption": "A dog running in the park",
-    "media_path": "videos/dog_running.mp4"
+    "video": "videos/dog_running.mp4"
  }
 ]
 ```
@@ -180,18 +181,42 @@ The dataset must be a CSV, JSON, or JSONL metadata file with columns for caption
 **JSONL format example:**

 ```jsonl
-{"caption": "A cat playing with a ball of yarn", "media_path": "videos/cat_playing.mp4"}
-{"caption": "A dog running in the park", "media_path": "videos/dog_running.mp4"}
+{"caption": "A cat playing with a ball of yarn", "video": "videos/cat_playing.mp4"}
+{"caption": "A dog running in the park", "video": "videos/dog_running.mp4"}
 ```

 **CSV format example:**

 ```csv
-caption,media_path
+caption,video
 "A cat playing with a ball of yarn","videos/cat_playing.mp4"
 "A dog running in the park","videos/dog_running.mp4"
 ```

+**Additional dataset format examples:**
+
+Audio-only dataset:
+```json
+{"audio": "song.wav", "caption": "piano melody"}
+```
+
+V2V IC-LoRA with reference video:
+```json
+{"video": "clip.mp4", "reference_video": "depth.mp4", "caption": "depth to video"}
+```
+
+A2A IC-LoRA with reference audio:
+```json
+{"video": "clip.mp4", "reference_audio": "ref.wav", "caption": "match this style"}
+```
+This form auto-extracts the target audio from `clip.mp4`. For pure audio datasets, use `audio` plus
+`reference_audio` columns and preprocess with `--audio-durations`.
+
+Video inpainting with mask:
+```json
+{"video": "clip.mp4", "video_mask": "mask.mp4", "caption": "fill the sky"}
+```
+
 ### 📐 Resolution Buckets

 Videos are organized into "buckets" of specific dimensions (width × height × frames).
@@ -268,12 +293,31 @@ The preprocessed data is saved in a `.precomputed` directory:
 ```
 dataset/
 └── .precomputed/
-    ├── latents/            # Cached video latents
-    ├── conditions/         # Cached text embeddings
-    ├── audio_latents/      # (only if --with-audio) Cached audio latents
-    └── reference_latents/  # (only for IC-LoRA) Cached reference video latents
+    ├── latents/                  # Video latents
+    ├── conditions/               # Text embeddings
+    ├── audio_latents/            # Audio latents (auto-extracted or explicit)
+    ├── reference_latents/        # Reference video latents (IC-LoRA)
+    ├── reference_audio_latents/  # Reference audio latents (audio IC-LoRA)
+    ├── video_masks/              # Video masks (inpainting)
+    └── audio_masks/              # Audio masks (audio inpainting)
 ```

+Set `data.preprocessed_data_root` in your training config to this `.precomputed` directory — the parent directory that
+contains `latents/`, `conditions/`, and any mode-specific audio/reference/mask directories.
+
+## 🔊 Audio-Only Dataset Preprocessing
+
+For datasets containing only audio files (no `video` column), use `--audio-durations` to specify duration buckets:
+
+```bash
+uv run python scripts/process_dataset.py dataset.json \
+    --audio-durations "2.0;4.0;8.0" \
+    --model-path /path/to/ltx-2-model.safetensors \
+    --text-encoder-path /path/to/gemma-model
+```
+
+The `--audio-durations` flag provides duration buckets (in seconds) for audio-only datasets. Since there is no video column to derive timing from, explicit duration buckets are required.
+
 ## 🪄 IC-LoRA Reference Video Preprocessing

 For IC-LoRA training, you need to preprocess datasets that include reference videos.
@@ -281,14 +325,16 @@ Reference videos provide the conditioning input while target videos represent th

 ### Dataset Format with Reference Videos

+The `reference_video` column is automatically detected by convention — no extra CLI flags are needed.
+
 **JSON format:**

 ```json
 [
  {
    "caption": "A cat playing with a ball of yarn",
-    "media_path": "videos/cat_playing.mp4",
-    "reference_path": "references/cat_playing_depth.mp4"
+    "video": "videos/cat_playing.mp4",
+    "reference_video": "references/cat_playing_depth.mp4"
  }
 ]
 ```
@@ -296,32 +342,39 @@ Reference videos provide the conditioning input while target videos represent th
 **JSONL format:**

 ```jsonl
-{"caption": "A cat playing with a ball of yarn", "media_path": "videos/cat_playing.mp4", "reference_path": "references/cat_playing_depth.mp4"}
-{"caption": "A dog running in the park", "media_path": "videos/dog_running.mp4", "reference_path": "references/dog_running_depth.mp4"}
+{"caption": "A cat playing with a ball of yarn", "video": "videos/cat_playing.mp4", "reference_video": "references/cat_playing_depth.mp4"}
+{"caption": "A dog running in the park", "video": "videos/dog_running.mp4", "reference_video": "references/dog_running_depth.mp4"}
 ```

 ### Preprocessing with Reference Videos

-To preprocess a dataset with reference videos, add the `--reference-column` argument specifying the name of the field
-in your dataset JSON/JSONL/CSV that contains the reference video paths:
+Convention-based detection means you just need the `reference_video` column in your dataset, and `process_dataset.py` will automatically detect and process it. No `--reference-column` flag is needed:

 ```bash
 uv run python scripts/process_dataset.py dataset.json \
    --resolution-buckets "960x544x49" \
    --model-path /path/to/ltx-2-model.safetensors \
    --text-encoder-path /path/to/gemma-model \
-    --reference-column "reference_path"
+    --reference-downscale-factor 2 \
+    --reference-temporal-scale-factor 1
 ```

 This will create an additional `reference_latents/` directory containing the preprocessed reference video latents.
+Use `--reference-downscale-factor` for spatial subsampling and `--reference-temporal-scale-factor` for temporal
+subsampling. Validation reference conditions should use matching `downscale_factor` and `temporal_scale_factor` values.
+
+> [!NOTE]
+> **Legacy column names:** If your dataset uses `ref_media_path`, it is accepted as an alias for `reference_video`.

 ### Generating Reference Videos

 **Dataset Requirements for IC-LoRA:**

 - Your dataset must contain paired videos where each target video has a corresponding reference video
- Reference and target videos must have *identical* resolution and length
- Both reference and target videos should be preprocessed together using the same resolution buckets
+- Reference and target videos should cover the same content. Reference videos can optionally be lower spatial
+  resolution or temporally subsampled (see Scaled Reference Conditioning in [Training Modes](training-modes.md)).
+- Both reference and target videos should be preprocessed together using the same target resolution buckets, plus any
+  reference scale factors you choose.

 We provide an example script, `[scripts/compute_reference.py](../scripts/compute_reference.py)`, to generate reference
 videos for a given dataset. The default implementation generates Canny edge reference videos.
@@ -333,12 +386,63 @@ uv run python scripts/compute_reference.py scenes_output_dir/ \

 The script accepts a JSON file as the dataset configuration and updates it in-place by adding the filenames of the generated reference videos.

+> [!NOTE]
+> `compute_reference.py` writes generated references to the `reference_video` column, which `process_dataset.py`
+> detects automatically. The legacy `ref_media_path` column is also accepted.
+
 If you want to generate a different type of condition (depth maps, pose skeletons, etc.), modify or replace the `compute_reference()` function within this script.

 ### Example Dataset

 For reference, see our **[Canny Control Dataset](https://huggingface.co/datasets/Lightricks/Canny-Control-Dataset)** which demonstrates proper IC-LoRA dataset structure with paired videos and Canny edge maps.

+## 🎭 Mask Preprocessing for Inpainting
+
+For inpainting training with the `mask` condition type, provide `video_mask` or `audio_mask` columns in your dataset
+metadata. These columns point to mask media files (for example a mask image/video for video inpainting, or a waveform or
+`.pt` tensor for audio inpainting). `process_dataset.py` downsamples and thresholds them into per-sample `.pt` tensors
+under `video_masks/` or `audio_masks/`.
+
+### Processed Video Mask Format
+
+If you create masks manually instead of using `process_dataset.py`, save them as `.pt` files with the key `"mask"`
+containing a tensor of shape `[F, H, W]` where:
+
+- `F` = number of latent frames (temporal dimension)
+- `H` = latent height (pixel height / 32)
+- `W` = latent width (pixel width / 32)
+- Values are thresholded at `0.5`: values `> 0.5` are conditioning tokens (clean, excluded from loss),
+  and values `<= 0.5` are generated tokens (noised, contributes to loss).
+
+### Audio Mask Format
+
+Audio masks follow the same thresholding pattern as video masks but with shape `[T]` (temporal dimension only), where `T` is the number of audio latent frames. They are stored in `audio_masks/`.
+
+### Directory Structure
+
+Place masks in a directory within your preprocessed data root:
+
+```
+preprocessed_data_root/
+├── latents/          # Video latents
+├── conditions/       # Text embeddings
+├── video_masks/      # Video masks (one .pt per sample, matching latent filenames)
+└── audio_masks/      # Audio masks (one .pt per sample, matching latent filenames)
+```
+
+Then reference the mask directory in your training config:
+
+```yaml
+training_strategy:
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+    conditions:
+      - type: mask
+        mask_dir: "video_masks"
+```
+
 ## 🎯 LoRA Trigger Words

 When training a LoRA, you can specify a trigger token that will be prepended to all captions:
@@ -359,9 +463,9 @@ This acts as a trigger word that activates the LoRA during inference when you in

 ## 🔍 Decoding Videos for Verification

-If you add the `--decode` flag, the script will VAE-decode the precomputed latents and save the resulting videos
-in `.precomputed/decoded_videos`. When audio preprocessing is enabled (`--with-audio`), audio latents will also be
-decoded and saved to `.precomputed/decoded_audio`. This allows you to visually and audibly inspect the processed data.
+If you add the `--decode` flag, the script will VAE-decode the precomputed video latents and save the resulting videos
+in `.precomputed/decoded_videos`. Reference video latents are decoded to `.precomputed/decoded_reference_videos` when
+present. To inspect audio latents, run `scripts/decode_latents.py` with `--with-audio`.

 ```bash
 uv run python scripts/process_dataset.py dataset.json \
@@ -382,6 +486,4 @@ Once your dataset is preprocessed, you can proceed to:
 - Start training with the [Training Guide](training-guide.md)

 > [!TIP]
-> If your training recipe requires additional preprocessed data (e.g., masks, conditioning signals), see
-> [Implementing Custom Training Strategies](custom-training-strategies.md) for guidance on extending the
-> preprocessing pipeline.
+> The `flexible` strategy supports masks for inpainting (`mask` condition type) and spatial crop regions for outpainting (`spatial_crop` condition type) out of the box. For other custom preprocessing needs, see [Custom Training Strategies](custom-training-strategies.md).
@@ -7,12 +7,14 @@ Get up and running with LTX-2 training in just a few steps!
 Before you begin, ensure you have:

 1. **LTX-2 Model Checkpoint** - A local `.safetensors` file containing the LTX-2 model weights.
-   Download `ltx-2-19b-dev.safetensors` from: [HuggingFace Hub](https://huggingface.co/Lightricks/LTX-2)
+   Download `ltx-2.3-22b-dev.safetensors` from: [HuggingFace Hub](https://huggingface.co/Lightricks/LTX-2.3)
+   The trainer supports LTX-2 and LTX-2.3 checkpoints through the same configuration API; version-specific components
+   are detected from the checkpoint.
 2. **Gemma Text Encoder** - A local directory containing the Gemma model (required for LTX-2).
   Download from: [HuggingFace Hub](https://huggingface.co/google/gemma-3-12b-it-qat-q4_0-unquantized/)
-3. **Linux with CUDA** - The trainer requires `triton` which is Linux-only
+3. **Linux with CUDA** - The trainer requires `triton` which is Linux-only; CUDA 13+ is recommended
 4. **GPU with sufficient VRAM** - 80GB recommended for the standard config. For GPUs with 32GB VRAM (e.g., RTX 5090),
-   use the [low VRAM config](../configs/ltx2_av_lora_low_vram.yaml) which enables INT8 quantization and other
+   use the [low VRAM config](../configs/t2v_lora_low_vram.yaml) which enables INT8 quantization and other
   memory optimizations

 ## ⚡ Installation
@@ -39,7 +41,18 @@ cd packages/ltx-trainer

 ## 🏋 Training Workflow

-### 1. Prepare Your Dataset
+If you are using an agent-enabled environment with repository skills, you can ask for the
+[`train-model`](../../../.claude/skills/train-model/SKILL.md) skill to run this workflow with you.
+It creates a run workspace, confirms the training mode, prepares data, preprocesses latents,
+launches training, and monitors the run while stopping for approval before expensive steps.
+
+### 1. Choose a Training Mode
+
+Start with [`t2v_lora.yaml`](../configs/t2v_lora.yaml) for a first run with videos and captions. For modes such as
+IC-LoRA, inpainting, or outpainting, check [Training Modes](training-modes.md) first because your metadata needs extra
+columns such as `reference_video`, `video_mask`, or `audio_mask` before preprocessing.
+
+### 2. Prepare Your Dataset

 Organize your videos and captions, then preprocess them:

@@ -57,15 +70,18 @@ uv run python scripts/process_dataset.py dataset.json \
    --text-encoder-path /path/to/gemma-model
 ```

+By default, preprocessing writes to `.precomputed/`. Use that directory as `data.preprocessed_data_root`
+in your training config.
+
 See [Dataset Preparation](dataset-preparation.md) for detailed instructions.

-### 2. Configure Training
+### 3. Configure Training

 Create or modify a configuration YAML file. Start with one of the example configs:

- [`configs/ltx2_av_lora.yaml`](../configs/ltx2_av_lora.yaml) - Audio-video LoRA training
- [`configs/ltx2_av_lora_low_vram.yaml`](../configs/ltx2_av_lora_low_vram.yaml) - Audio-video LoRA training (optimized for 32GB VRAM)
- [`configs/ltx2_v2v_ic_lora.yaml`](../configs/ltx2_v2v_ic_lora.yaml) - IC-LoRA video-to-video
+- [`configs/t2v_lora.yaml`](../configs/t2v_lora.yaml) - Text-to-video LoRA
+- [`configs/t2v_lora_low_vram.yaml`](../configs/t2v_lora_low_vram.yaml) - Same as above, tuned for ~32GB VRAM (INT8 quantization and memory optimizations)
+- [`configs/v2v_ic_lora.yaml`](../configs/v2v_ic_lora.yaml) - IC-LoRA video-to-video

 Key settings to update:

@@ -82,33 +98,47 @@ output_dir: "outputs/my_training_run"

 See [Configuration Reference](configuration-reference.md) for all available options.

-### 3. Start Training
+### 4. Start Training

 ```bash
-uv run python scripts/train.py configs/ltx2_av_lora.yaml
+uv run python scripts/train.py configs/t2v_lora.yaml
 ```

 For multi-GPU training:

 ```bash
-uv run accelerate launch scripts/train.py configs/ltx2_av_lora.yaml
+uv run accelerate launch scripts/train.py configs/t2v_lora.yaml
 ```

 See [Training Guide](training-guide.md) for distributed training and advanced options.

 ## 🎯 Training Modes

+> [!TIP]
+> **First time?** Start with [`t2v_lora.yaml`](../configs/t2v_lora.yaml) — it's the simplest mode
+> and only requires videos with captions. You can explore other modes once you've confirmed your
+> setup works.
+
 The trainer supports several training modes:

-| Mode                 | Description                    | Config Example                             |
-|----------------------|--------------------------------|--------------------------------------------|
-| **LoRA**             | Efficient adapter training     | `training_strategy.name: "text_to_video"`  |
-| **Audio-Video LoRA** | Joint audio-video training     | `training_strategy.with_audio: true`       |
-| **IC-LoRA**          | Video-to-video transformations | `training_strategy.name: "video_to_video"` |
-| **Full Fine-tuning** | Full model training            | `model.training_mode: "full"`              |
+| Mode                  | Description                                | Example Config                                                    |
+|-----------------------|--------------------------------------------|-------------------------------------------------------------------|
+| **Text-to-Video**     | Generate video+audio from text prompts     | [`t2v_lora.yaml`](../configs/t2v_lora.yaml)                      |
+| **Image-to-Video**    | Animate from a starting image              | [`i2v_lora.yaml`](../configs/i2v_lora.yaml)                      |
+| **Video Extension**   | Extend videos temporally (forward/backward)| [`video_extend_lora.yaml`](../configs/video_extend_lora.yaml), [`video_suffix_lora.yaml`](../configs/video_suffix_lora.yaml) |
+| **IC-LoRA (V2V)**     | Video-to-video transformations             | [`v2v_ic_lora.yaml`](../configs/v2v_ic_lora.yaml)                |
+| **Audio-to-Video**    | Generate video conditioned on audio        | [`a2v_lora.yaml`](../configs/a2v_lora.yaml)                      |
+| **Video-to-Audio**    | Generate audio/foley from video            | [`v2a_lora.yaml`](../configs/v2a_lora.yaml)                      |
+| **Video Inpainting**  | Fill in masked regions of video            | [`video_inpainting_lora.yaml`](../configs/video_inpainting_lora.yaml) |
+| **Video Outpainting** | Extend video spatially                     | [`video_outpainting_lora.yaml`](../configs/video_outpainting_lora.yaml) |
+| **Text-to-Audio**     | Generate audio from text prompts           | [`t2a_lora.yaml`](../configs/t2a_lora.yaml)                      |
+| **Audio Extension**   | Extend audio temporally                    | [`audio_extend_lora.yaml`](../configs/audio_extend_lora.yaml), [`audio_suffix_lora.yaml`](../configs/audio_suffix_lora.yaml) |
+| **Audio Inpainting**  | Fill in masked regions of audio            | [`audio_inpainting_lora.yaml`](../configs/audio_inpainting_lora.yaml) |
+| **IC-LoRA (A2A)**     | Audio-to-audio transformations             | [`a2a_ic_lora.yaml`](../configs/a2a_ic_lora.yaml)                |
+| **AV2AV IC-LoRA**     | Audio+video IC-LoRA transformations        | [`av2av_ic_lora.yaml`](../configs/av2av_ic_lora.yaml)            |
+| **Full Fine-tuning**  | Full model training (any mode above)       | Set `model.training_mode: "full"`                                 |

-See [Training Modes](training-modes.md) for detailed explanations,
-or [Custom Training Strategies](custom-training-strategies.md) if you need to implement your own training recipe.
+See [Training Modes](training-modes.md) for detailed explanations of each mode.

 ## Next Steps

@@ -118,7 +148,7 @@ Once you've completed your first training run, you can:
  production-ready inference
  pipelines for various use cases (T2V, I2V, IC-LoRA, etc.). See the package documentation for details.
 - Learn more about [Dataset Preparation](dataset-preparation.md) for advanced preprocessing
- Explore different [Training Modes](training-modes.md) (LoRA, Audio-Video, IC-LoRA)
+- Explore different [Training Modes](training-modes.md)
 - Dive deeper into [Training Configuration](configuration-reference.md)
 - Understand the model architecture in [LTX-Core Documentation](../../ltx-core/README.md)

@@ -8,7 +8,7 @@ model uploads.
 After preprocessing your dataset and preparing a configuration file, you can start training using the trainer script:

 ```bash
-uv run python scripts/train.py configs/ltx2_av_lora.yaml
+uv run python scripts/train.py configs/t2v_lora.yaml
 ```

 The trainer will:
@@ -19,19 +19,31 @@ The trainer will:
 4. **Generate validation videos** (if configured)
 5. **Save the trained weights** in your output directory

+### Agent-Assisted Training
+
+If your environment supports repository skills, the
+[`train-model`](../../../.claude/skills/train-model/SKILL.md) skill provides an end-to-end
+orchestrator for this package. It asks what you want the model to learn, maps that intent to
+one of the documented [training modes](training-modes.md), probes your filesystem and GPU,
+prepares/preprocesses the dataset, writes a run-specific config, launches training, and
+monitors the job. It uses the trainer docs as its source of truth and stops for approval before
+captioning, preprocessing, or starting expensive training work.
+
 ### Output Files

 **For LoRA training:**

- `lora_weights.safetensors` - Main LoRA weights file
+- `checkpoints/lora_weights_step_00000.safetensors` - LoRA checkpoint weights, with the current step in the filename
 - `training_config.yaml` - Copy of training configuration
- `validation_samples/` - Generated validation videos (if enabled)
+- `samples/` - Generated validation samples (if enabled)
+- `checkpoints/training_state_step_00000.pt` - Optional resume state, depending on `checkpoints.save_training_state`

 **For full model fine-tuning:**

- `model_weights.safetensors` - Full model weights
+- `checkpoints/model_weights_step_00000.safetensors` - Full model checkpoint weights, with the current step in the filename
 - `training_config.yaml` - Copy of training configuration
- `validation_samples/` - Generated validation videos (if enabled)
+- `samples/` - Generated validation samples (if enabled)
+- `checkpoints/training_state_step_00000.pt` - Optional resume state, depending on `checkpoints.save_training_state`

 ## 🖥️ Distributed / Multi-GPU Training

@@ -62,22 +74,22 @@ Launch with a specific config using `--config_file`:
 # DDP (2 GPUs shown as example)
 CUDA_VISIBLE_DEVICES=0,1 \
 uv run accelerate launch --config_file configs/accelerate/ddp.yaml \
-  scripts/train.py configs/ltx2_av_lora.yaml
+  scripts/train.py configs/t2v_lora.yaml

 # DDP + torch.compile
 CUDA_VISIBLE_DEVICES=0,1 \
 uv run accelerate launch --config_file configs/accelerate/ddp_compile.yaml \
-  scripts/train.py configs/ltx2_av_lora.yaml
+  scripts/train.py configs/t2v_lora.yaml

 # FSDP (4 GPUs shown as example)
 CUDA_VISIBLE_DEVICES=0,1,2,3 \
 uv run accelerate launch --config_file configs/accelerate/fsdp.yaml \
-  scripts/train.py configs/ltx2_av_lora.yaml
+  scripts/train.py configs/t2v_lora.yaml

 # FSDP + torch.compile
 CUDA_VISIBLE_DEVICES=0,1,2,3 \
 uv run accelerate launch --config_file configs/accelerate/fsdp_compile.yaml \
-  scripts/train.py configs/ltx2_av_lora.yaml
+  scripts/train.py configs/t2v_lora.yaml
 ```

 **Notes:**
@@ -93,13 +105,13 @@ If you prefer to use your default Accelerate profile:

 ```bash
 # Use settings from your default accelerate config
-uv run accelerate launch scripts/train.py configs/ltx2_av_lora.yaml
+uv run accelerate launch scripts/train.py configs/t2v_lora.yaml

 # Override number of processes on the fly (e.g., 2 GPUs)
-uv run accelerate launch --num_processes 2 scripts/train.py configs/ltx2_av_lora.yaml
+uv run accelerate launch --num_processes 2 scripts/train.py configs/t2v_lora.yaml

 # Select specific GPUs
-CUDA_VISIBLE_DEVICES=0,1 uv run accelerate launch scripts/train.py configs/ltx2_av_lora.yaml
+CUDA_VISIBLE_DEVICES=0,1 uv run accelerate launch scripts/train.py configs/t2v_lora.yaml
 ```

 > [!TIP]
@@ -1,165 +1,167 @@
 # Training Modes Guide

-The trainer supports several training modes, each suited for different use cases and requirements.
+The trainer uses the **flexible** training strategy (`name: "flexible"`) — a unified conditioning framework that
+supports all training modes through configuration. Every scenario is expressed by setting `is_generated` on each
+modality and adding optional conditions, rather than choosing a separate strategy class.

-## 🎯 Standard LoRA Training (Video-Only)
+## Key Concepts

-Standard LoRA (Low-Rank Adaptation) training fine-tunes the model by adding small, trainable adapter layers while
-keeping the base model frozen. This approach:
+Before diving into individual modes, here are the core ideas behind the flexible strategy:

- **Requires significantly less memory and compute** than full fine-tuning
- **Produces small, portable weight files** (typically a few hundred MB)
- **Is ideal for learning specific styles, effects, or concepts**
- **Can be easily combined with other LoRAs** during inference
+- **`is_generated: true`** — the modality is denoised during training and contributes to the loss. This is the
+  modality the model learns to generate.
+- **`is_generated: false`** — the modality is frozen (sigma=0, no noise, no loss). It passes through the transformer
+  clean and acts as cross-modal conditioning for the generated modality.
+- **At least one modality must have `is_generated: true`.**
+- **Conditions** are per-modality and can be composed (e.g., `reference` + `first_frame` together on the video
+  modality).
+- Audio does **not** support `first_frame` or `spatial_crop` conditions — only `prefix`, `suffix`, `mask`,
+  and `reference`.

-Configure standard LoRA training with:
+> [!TIP]
+> If you are using an agent-enabled environment with repository skills and are unsure which mode to choose,
+> ask for the [`train-model`](../../../.claude/skills/train-model/SKILL.md) skill. It maps your intent to one of
+> these configs and walks through dataset preparation, preprocessing, launch, and monitoring.
+
+## 📊 Quick Reference
+
+| Mode                  | Video     | Audio     | Conditions          | Config |
+|-----------------------|-----------|-----------|---------------------|--------|
+| **T2V**               | Generated | Generated | —                   | [`t2v_lora`](../configs/t2v_lora.yaml) |
+| **I2V**               | Generated | Generated | `first_frame`       | [`i2v_lora`](../configs/i2v_lora.yaml) |
+| **Video Extension**   | Generated | Generated | `prefix`/`suffix`   | [`video_extend_lora`](../configs/video_extend_lora.yaml) |
+| **V2V IC-LoRA**       | Generated | —         | `reference`         | [`v2v_ic_lora`](../configs/v2v_ic_lora.yaml) |
+| **A2V**               | Generated | Frozen    | —                   | [`a2v_lora`](../configs/a2v_lora.yaml) |
+| **V2A (Foley)**       | Frozen    | Generated | —                   | [`v2a_lora`](../configs/v2a_lora.yaml) |
+| **Video Inpainting**  | Generated | —         | `mask`              | [`video_inpainting_lora`](../configs/video_inpainting_lora.yaml) |
+| **Video Outpainting** | Generated | —         | `spatial_crop`      | [`video_outpainting_lora`](../configs/video_outpainting_lora.yaml) |
+| **T2A**               | —         | Generated | —                   | [`t2a_lora`](../configs/t2a_lora.yaml) |
+| **Audio Extension**   | —         | Generated | `prefix`/`suffix`   | [`audio_extend_lora`](../configs/audio_extend_lora.yaml) |
+| **Audio Inpainting**  | —         | Generated | `mask`              | [`audio_inpainting_lora`](../configs/audio_inpainting_lora.yaml) |
+| **A2A IC-LoRA**       | —         | Generated | `reference`         | [`a2a_ic_lora`](../configs/a2a_ic_lora.yaml) |
+| **AV2AV IC-LoRA**     | Generated | Generated | `reference` (both)  | [`av2av_ic_lora`](../configs/av2av_ic_lora.yaml) |
+
+---
+
+## 🎯 Text-to-Video (T2V)
+
+Generate video and audio from text prompts. Both modalities are denoised with no additional conditions.

 ```yaml
-model:
-  training_mode: "lora"
-
 training_strategy:
-  name: "text_to_video"
-  first_frame_conditioning_p: 0.1
-  with_audio: false  # Video-only training
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
 ```

-## 🔊 Audio-Video LoRA Training
+**Example config:** 📄 [t2v_lora.yaml](../configs/t2v_lora.yaml)

-LTX-2 supports joint audio-video generation. You can train LoRA adapters that affect both video and audio output:
+---

- **Synchronized audio-video generation** - Audio matches the visual content
- **Same efficient LoRA approach** - Just enable audio training
- **Requires audio latents** - Dataset must include preprocessed audio
+## 🖼️ Image-to-Video (I2V)

-Configure audio-video training with:
+Generate video conditioned on a starting image. The first frame is provided as a clean conditioning signal — no noise,
+timestep=0, excluded from loss. The `probability` parameter controls how often first-frame conditioning is applied;
+remaining samples train in pure T2V mode.

 ```yaml
-model:
-  training_mode: "lora"
-
 training_strategy:
-  name: "text_to_video"
-  first_frame_conditioning_p: 0.1
-  with_audio: true  # Enable audio training
-  audio_latents_dir: "audio_latents"  # Directory containing audio latents
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+    conditions:
+      - type: first_frame
+        probability: 0.5
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
 ```

-**Example configuration file:**
+**Example config:** 📄 [i2v_lora.yaml](../configs/i2v_lora.yaml)

- 📄 [Audio-Video LoRA Training](../configs/ltx2_av_lora.yaml)
+---

-**Dataset structure for audio-video training:**
+## ⏩ Video Extension

+Extend a video forward (or backward) in time. Prefix or suffix conditioning provides a span of existing latent frames
+as clean conditioning. The `temporal_boundary` sets the number of **latent frames** used as context (each latent frame
+= 8 pixel frames due to temporal compression).
+
+```yaml
+training_strategy:
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+    conditions:
+      - type: prefix            # or "suffix" for backward extension
+        temporal_boundary: 8    # 8 latent frames = 64 pixel frames
+        probability: 1.0
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
 ```
-preprocessed_data_root/
-├── latents/           # Video latents
-├── conditions/        # Text embeddings
-└── audio_latents/     # Audio latents (required when with_audio: true)
-```
-
-> [!IMPORTANT]
-> When training audio-video LoRAs, ensure your `target_modules` configuration captures video, audio, and
-> cross-modal attention branches. Use patterns like `"to_k"` instead of `"attn1.to_k"` to match:
-> - Video modules: `attn1.to_k`, `attn2.to_k`
-> - Audio modules: `audio_attn1.to_k`, `audio_attn2.to_k`
-> - Cross-modal modules: `audio_to_video_attn.to_k`, `video_to_audio_attn.to_k`
->
-> The cross-modal attention modules (`audio_to_video_attn` and `video_to_audio_attn`) enable bidirectional
-> information flow between audio and video, which is critical for synchronized audiovisual generation.
-> See [Understanding Target Modules](configuration-reference.md#understanding-target-modules) for detailed guidance.

 > [!NOTE]
-> You can generate audio during validation even if you're not training the audio branch.
-> Set `validation.generate_audio: true` independently of `training_strategy.with_audio`.
+> The `prefix` and `suffix` conditions also work on the audio modality for audio extension.
+> Set `temporal_boundary` on the audio modality's conditions list to condition on a prefix or suffix
+> of the audio latents.

-## 🔥 Full Model Fine-tuning
+**Example configs:** 📄 [video_extend_lora.yaml](../configs/video_extend_lora.yaml) (forward), 📄 [video_suffix_lora.yaml](../configs/video_suffix_lora.yaml) (backward)

-Full model fine-tuning updates all parameters of the base model, providing maximum flexibility but
-requiring substantial computational resources and larger training datasets:
+---

- **Offers the highest potential quality and capability improvements**
- **Requires multiple GPUs** and distributed training techniques (e.g., FSDP)
- **Produces large checkpoint files** (several GB)
- **Best for major model adaptations** or when LoRA limitations are reached
+## 🔄 IC-LoRA / Video-to-Video (V2V)

-Configure full fine-tuning with:
+In-Context LoRA learns transformations from paired videos. Pre-encoded reference latents are concatenated to the target
+sequence — reference tokens participate in bidirectional self-attention but receive no noise and are excluded from loss.
+This enables control adapters (depth, pose), style transfer, deblurring, colorization, and more.

 ```yaml
-model:
-  training_mode: "full"
-
 training_strategy:
-  name: "text_to_video"
-  first_frame_conditioning_p: 0.1
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+    conditions:
+      - type: reference
+        latents_dir: "reference_latents"
+        probability: 1.0
+      - type: first_frame       # optional — composable with reference
+        probability: 0.2
 ```

-> [!IMPORTANT]
-> Full fine-tuning of LTX-2 requires multiple high-end GPUs (e.g., 4-8× H100 80GB) and distributed
-> training with FSDP. See [Training Guide](training-guide.md) for multi-GPU setup instructions.
+> [!NOTE]
+> IC-LoRA is video-only by default (no audio modality block). Conditions can be composed — the example above also
+> applies first-frame conditioning with 20% probability alongside the reference.
+> Use [AV2AV IC-LoRA](#av2av-ic-lora) when both video and audio references should be trained jointly.

-## 🔄 In-Context LoRA (IC-LoRA) Training
+**Example config:** 📄 [v2v_ic_lora.yaml](../configs/v2v_ic_lora.yaml)

-IC-LoRA is a specialized training mode for video-to-video transformations.
-Unlike standard training modes that learn from individual videos, IC-LoRA learns transformations from pairs of videos.
-IC-LoRA enables a wide range of advanced video-to-video applications, such as:
+### Dataset Requirements

- **Control adapters** (e.g., Depth, Pose): Learn to map from a control signal (like a depth map or pose skeleton) to a
-  target video
- **Video deblurring**: Transform blurry input videos into sharp, high-quality outputs
- **Style transfer**: Apply the style of a reference video to a target video sequence
- **Colorization**: Convert grayscale reference videos into colorized outputs
- **Restoration and enhancement**: Denoise, upscale, or restore old or degraded videos
+- **Paired videos** — each target video has a corresponding reference video
+- **Same frame count** between reference and target
+- Reference videos can optionally be at **lower spatial resolution** (see [Scaled Reference](#scaled-reference-conditioning) below)
+- Both must be **preprocessed** before training

-By providing paired reference and target videos, IC-LoRA can learn complex transformations that go beyond caption-based
-conditioning.
-
-IC-LoRA training fundamentally differs from standard LoRA and full fine-tuning:
-
- **Reference videos** provide clean, unnoised conditioning input showing the "before" state
- **Target videos** are noised during training and represent the desired "after" state
- **The model learns transformations** from reference videos to target videos
- **Loss is applied only to the target portion**, not the reference
- **Training and inference time increase significantly** due to the doubled sequence length
-
-To enable IC-LoRA training, configure your YAML file with:
-
-```yaml
-model:
-  training_mode: "lora"  # Required: IC-LoRA uses LoRA mode
-
-training_strategy:
-  name: "video_to_video"
-  first_frame_conditioning_p: 0.1
-  reference_latents_dir: "reference_latents"  # Directory for reference video latents
-```
-
-**Example configuration file:**
-
- 📄 [IC-LoRA Training](../configs/ltx2_v2v_ic_lora.yaml) - Video-to-video transformation training
-
-### Dataset Requirements for IC-LoRA
-
- Your dataset must contain **paired videos** where each target video has a corresponding reference video
- Reference and target videos must have the **same frame count** (length)
- Reference videos can optionally be at **lower spatial resolution** than target videos (
-  see [Scaled Reference Conditioning](#scaled-reference-conditioning) below)
- Both reference and target videos should be **preprocessed** before training
-
-**Dataset structure for IC-LoRA training:**
+**Dataset structure:**

 ```
 preprocessed_data_root/
-├── latents/            # Target video latents (what the model learns to generate)
-├── conditions/         # Text embeddings for each video
+├── latents/            # Target video latents
+├── conditions/         # Text embeddings
 └── reference_latents/  # Reference video latents (conditioning input)
 ```

 ### Generating Reference Videos

-We provide an example script to generate reference videos (e.g., Canny edge maps) for a given dataset.
-The script takes a JSON file as input (e.g., output of `caption_videos.py`) and updates it with the generated reference
-video paths.
+Use the `compute_reference.py` script to generate reference videos (e.g., Canny edge maps) for a dataset:

 ```bash
 uv run python scripts/compute_reference.py scenes_output_dir/ \
@@ -169,84 +171,392 @@ uv run python scripts/compute_reference.py scenes_output_dir/ \
 To compute a different condition (depth maps, pose skeletons, etc.), modify the `compute_reference()` function in the
 script.

-### Configuration Requirements for IC-LoRA
-
- You **must** provide `reference_videos` in your validation configuration when using IC-LoRA training
- The number of reference videos must match the number of validation prompts
-
-Example validation configuration for IC-LoRA:
-
-```yaml
-validation:
-  prompts:
-    - "First prompt describing the desired output"
-    - "Second prompt describing the desired output"
-  reference_videos:
-    - "/path/to/reference1.mp4"
-    - "/path/to/reference2.mp4"
-  reference_downscale_factor: 1  # Set to match preprocessing (e.g., 2 for half resolution)
-  include_reference_in_output: true  # Show reference side-by-side with output
-```
+> [!NOTE]
+> `compute_reference.py` writes generated references to the `reference_video` column, which
+> `process_dataset.py` detects automatically. The legacy `ref_media_path` column is also accepted.

 ### Scaled Reference Conditioning

-For more efficient training and inference, you can use **downscaled reference videos** while keeping target videos at
-full resolution. This reduces the number of conditioning tokens, leading to:
+For more efficient training and inference, use **downscaled reference videos** while keeping targets at full
+resolution. During training, the strategy infers the spatial and temporal scale factors from the preprocessed
+reference and target latents and adjusts positional encodings accordingly. This reduces conditioning tokens, leading to:

- **Faster training** due to shorter sequence lengths
- **Faster inference** with reduced memory usage
+- **Faster training** — shorter sequence lengths
+- **Faster inference** — reduced memory usage
 - **Same aspect ratio** maintained between reference and target

-#### How It Works
-
-When the reference video has resolution `H/n × W/n` and the target video has resolution `H × W`, the trainer
-automatically detects this scale factor `n` and adjusts the positional encodings so that the reference positions
-map to the correct locations in the target coordinate space.
-
-#### Preprocessing Datasets with Scaled References
-
-Use the `--reference-downscale-factor` option when running `process_dataset.py`:
+Preprocess with the `--reference-downscale-factor` option:

 ```bash
-# Process dataset with scaled reference videos (half resolution)
 uv run python scripts/process_dataset.py dataset.json \
    --resolution-buckets 768x768x25 \
    --model-path /path/to/ltx2.safetensors \
    --text-encoder-path /path/to/gemma \
-    --reference-column "reference_path" \
    --reference-downscale-factor 2
 ```

-This will:
+> [!NOTE]
+> The `reference_video` column is auto-detected by convention — no `--reference-column` flag needed.

- Process target videos at 768×768 resolution
- Process reference videos at 384×384 resolution (768 / 2)
- The trainer will automatically infer the scale factor from the dimension ratio
-
-**Important**: Set `reference_downscale_factor: 2` in your validation configuration to match the preprocessing:
+Validation encodes reference media on the fly, so set `downscale_factor` and `temporal_scale_factor`
+on each `reference` validation condition to match the preprocessing factors:

 ```yaml
 validation:
-  reference_downscale_factor: 2  # Must match the preprocessing factor
-  reference_videos:
-    - "/path/to/reference1.mp4"
-    - "/path/to/reference2.mp4"
+  samples:
+    - prompt: "..."
+      conditions:
+        - type: reference
+          video: "/path/to/reference.mp4"
+          downscale_factor: 2
+          temporal_scale_factor: 1
+          include_in_output: true
 ```

 > [!NOTE]
 > The scale factor must be a positive integer, and all dimensions must be divisible by 32.
-> Common scale factors are 1 (no scaling), 2 (half resolution), or 4 (quarter resolution).
+> Common values are 1 (no scaling), 2 (half resolution), or 4 (quarter resolution).

-## 📊 Training Mode Comparison
+---

-| Aspect               | LoRA                           | Audio-Video LoRA               | Full Fine-tuning | IC-LoRA                        |
-|----------------------|--------------------------------|--------------------------------|------------------|--------------------------------|
-| **Memory Usage**     | Low                            | Low-Medium                     | High             | Medium                         |
-| **Training Speed**   | Fast                           | Fast                           | Slow             | Medium                         |
-| **Output Size**      | 100MB-few GB (depends on rank) | 100MB-few GB (depends on rank) | Tens of GB       | 100MB-few GB (depends on rank) |
-| **Flexibility**      | Medium                         | Medium                         | High             | Specialized                    |
-| **Audio Support**    | Optional                       | Yes                            | Optional         | No                             |
-| **Reference Videos** | No                             | No                             | No               | Yes (required)                 |
+## 🔊 Audio-to-Video (A2V)
+
+Generate video conditioned on frozen audio. Audio passes through the transformer clean (sigma=0) and influences video
+via the built-in cross-modal attention. Only video is denoised.
+
+```yaml
+training_strategy:
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+  audio:
+    is_generated: false
+    latents_dir: "audio_latents"
+```
+
+**Example config:** 📄 [a2v_lora.yaml](../configs/a2v_lora.yaml)
+
+---
+
+## 🎵 Video-to-Audio / Foley (V2A)
+
+Generate audio (Foley) conditioned on frozen video. Video passes through the transformer clean (sigma=0) and
+conditions audio via cross-modal attention. Only audio is denoised.
+
+```yaml
+training_strategy:
+  name: "flexible"
+  video:
+    is_generated: false
+    latents_dir: "latents"
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
+```
+
+**Example config:** 📄 [v2a_lora.yaml](../configs/v2a_lora.yaml)
+
+---
+
+## 🎭 Video Inpainting
+
+Fill in masked regions of a video. Per-sample masks loaded from disk define which tokens are conditioning and which
+must be generated. Masks are thresholded at `0.5` to match validation/inference: tokens with `mask > 0.5` receive clean
+latents and timestep=0 and are excluded from loss; tokens with `mask <= 0.5` are denoised normally and contribute to
+loss.
+
+```yaml
+training_strategy:
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+    conditions:
+      - type: mask
+        mask_dir: "video_masks"
+        probability: 1.0
+```
+
+**Dataset structure:**
+
+```
+preprocessed_data_root/
+├── latents/        # Video latents
+├── conditions/     # Text embeddings
+└── video_masks/    # Per-sample binary masks (1 → conditioning, 0 → generate)
+```
+
+In dataset metadata, provide mask media via the `video_mask` column; preprocessing converts it into `video_masks/`.
+
+**Example config:** 📄 [video_inpainting_lora.yaml](../configs/video_inpainting_lora.yaml)
+
+---
+
+## 🌅 Video Outpainting
+
+Extend a video spatially beyond its original boundaries. A rectangular pixel region is provided as clean conditioning
+(no noise, timestep=0, excluded from loss) — the model learns to generate the surrounding content. The `spatial_region`
+is specified in pixel coordinates `[y1, x1, y2, x2]` and automatically converted to latent space.
+
+```yaml
+training_strategy:
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+    conditions:
+      - type: spatial_crop
+        spatial_region: [0, 0, 288, 576]  # y1, x1, y2, x2 in pixels
+        probability: 1.0
+```
+
+> [!NOTE]
+> `spatial_crop` is a video-only condition — it is not supported on the audio modality.
+
+**Example config:** 📄 [video_outpainting_lora.yaml](../configs/video_outpainting_lora.yaml)
+
+---
+
+## 🔈 Text-to-Audio (T2A)
+
+Generate audio from text prompts with no video modality. Only the audio branch of the transformer is denoised. Since
+no video modality is configured, this mode uses **audio-only LoRA targets** — explicitly targeting `audio_attn1`,
+`audio_attn2`, and `audio_ff` modules.
+
+```yaml
+training_strategy:
+  name: "flexible"
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
+```
+
+> [!NOTE]
+> With no `video` block in the strategy, the trainer only loads audio latents and text embeddings. LoRA adapters
+> should explicitly target audio modules (e.g., `audio_attn1.to_k`) rather than short patterns like `to_k` which
+> would also match video modules. See [LoRA Target Modules Guidance](#lora-target-modules-guidance) below.
+
+**Example config:** 📄 [t2a_lora.yaml](../configs/t2a_lora.yaml)
+
+---
+
+## 🔊 Audio Extension
+
+Extend audio forward (prefix) or backward (suffix) in time — the audio equivalent of Video Extension. A span of
+existing audio latent frames is provided as clean conditioning, and the model generates the continuation. The
+`temporal_boundary` sets the number of latent frames used as context. This mode uses **audio-only LoRA targets**.
+
+```yaml
+training_strategy:
+  name: "flexible"
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
+    conditions:
+      - type: prefix            # or "suffix" for backward extension
+        temporal_boundary: 8
+        probability: 1.0
+```
+
+**Example configs:** 📄 [audio_extend_lora.yaml](../configs/audio_extend_lora.yaml), 📄 [audio_suffix_lora.yaml](../configs/audio_suffix_lora.yaml)
+
+---
+
+## 🎭 Audio Inpainting
+
+Fill in masked regions of audio. Per-sample masks loaded from disk define which audio tokens are conditioning and
+which must be generated — the audio equivalent of Video Inpainting. Masks are thresholded at `0.5` with the same
+binary semantics as video inpainting. This mode uses **audio-only LoRA targets**.
+
+```yaml
+training_strategy:
+  name: "flexible"
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
+    conditions:
+      - type: mask
+        mask_dir: "audio_masks"
+        probability: 1.0
+```
+
+**Dataset structure:**
+
+```
+preprocessed_data_root/
+├── conditions/      # Text embeddings
+├── audio_latents/   # Audio latents
+└── audio_masks/     # Per-sample binary masks (1 → conditioning, 0 → generate)
+```
+
+In dataset metadata, provide mask media via the `audio_mask` column; preprocessing converts it into `audio_masks/`.
+
+**Example config:** 📄 [audio_inpainting_lora.yaml](../configs/audio_inpainting_lora.yaml)
+
+---
+
+## 🔄 IC-LoRA / Audio-to-Audio (A2A)
+
+In-Context LoRA for audio-to-audio transformations. Pre-encoded reference audio latents are concatenated to the target
+sequence — reference tokens participate in bidirectional self-attention but receive no noise and are excluded from loss.
+This enables audio style transfer, voice conversion, sound effect transformation, and more. This mode uses
+**audio-only LoRA targets**.
+
+```yaml
+training_strategy:
+  name: "flexible"
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
+    conditions:
+      - type: reference
+        latents_dir: "reference_audio_latents"
+        probability: 1.0
+```
+
+**Dataset structure:**
+
+```
+preprocessed_data_root/
+├── conditions/              # Text embeddings
+├── audio_latents/           # Target audio latents
+└── reference_audio_latents/ # Reference audio latents (conditioning input)
+```
+
+**Example config:** 📄 [a2a_ic_lora.yaml](../configs/a2a_ic_lora.yaml)
+
+---
+
+## 🔄 AV2AV IC-LoRA
+
+Joint audio-video In-Context LoRA — both modalities have reference conditioning. Pre-encoded reference latents are
+concatenated to each modality's target sequence independently. This enables joint audiovisual transformations such as
+synchronized style transfer across both video and audio.
+
+```yaml
+training_strategy:
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+    conditions:
+      - type: reference
+        latents_dir: "reference_latents"
+        probability: 1.0
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
+    conditions:
+      - type: reference
+        latents_dir: "reference_audio_latents"
+        probability: 1.0
+```
+
+> [!NOTE]
+> Unlike audio-only IC-LoRA (A2A), AV2AV uses short LoRA target patterns like `"to_k"` to match all branches
+> (video, audio, and cross-modal attention), since both modalities are trained.
+
+**Dataset structure:**
+
+```
+preprocessed_data_root/
+├── latents/                 # Target video latents
+├── audio_latents/           # Target audio latents
+├── conditions/              # Text embeddings
+├── reference_latents/       # Reference video latents (conditioning input)
+└── reference_audio_latents/ # Reference audio latents (conditioning input)
+```
+
+**Example config:** 📄 [av2av_ic_lora.yaml](../configs/av2av_ic_lora.yaml)
+
+---
+
+## 🔥 Full Model Fine-tuning
+
+All modes above default to `training_mode: "lora"`. For full fine-tuning, set `training_mode: "full"` — this updates
+all model parameters rather than adding LoRA adapters.
+
+```yaml
+model:
+  training_mode: "full"
+
+training_strategy:
+  name: "flexible"
+  video:
+    is_generated: true
+    latents_dir: "latents"
+  audio:
+    is_generated: true
+    latents_dir: "audio_latents"
+```
+
+> [!IMPORTANT]
+> Full fine-tuning requires multiple high-end GPUs (e.g., 4-8× H100 80GB) and distributed training with FSDP.
+> See [Training Guide](training-guide.md) for multi-GPU setup instructions.
+
+---
+
+## 🎛️ LoRA Target Modules Guidance
+
+The `target_modules` configuration determines which transformer modules receive LoRA adapters. The right choice depends
+on whether your training involves cross-modal (audio ↔ video) interaction.
+
+**For T2V, I2V, A2V, V2A, or any mode involving both modalities** — use short patterns to match all branches
+(video, audio, and cross-modal attention):
+
+```yaml
+target_modules:
+  - "to_k"
+  - "to_q"
+  - "to_v"
+  - "to_out.0"
+```
+
+> [!IMPORTANT]
+> Short patterns like `"to_k"` match video modules (`attn1.to_k`, `attn2.to_k`), audio modules
+> (`audio_attn1.to_k`, `audio_attn2.to_k`), and cross-modal modules (`audio_to_video_attn.to_k`,
+> `video_to_audio_attn.to_k`). The cross-modal attention modules enable bidirectional information flow between
+> audio and video, which is critical for synchronized audiovisual generation.
+> See [Understanding Target Modules](configuration-reference.md#understanding-target-modules) for detailed guidance.
+
+**For video-only IC-LoRA** — explicitly target video modules (including FFN layers for better transformation quality):
+
+```yaml
+target_modules:
+  - "attn1.to_k"
+  - "attn1.to_q"
+  - "attn1.to_v"
+  - "attn1.to_out.0"
+  - "attn2.to_k"
+  - "attn2.to_q"
+  - "attn2.to_v"
+  - "attn2.to_out.0"
+  - "ff.net.0.proj"
+  - "ff.net.2"
+```
+
+**For audio-only modes (T2A, Audio Extension, Audio Inpainting, A2A IC-LoRA)** — explicitly target audio modules:
+
+```yaml
+target_modules:
+  - "audio_attn1.to_k"
+  - "audio_attn1.to_q"
+  - "audio_attn1.to_v"
+  - "audio_attn1.to_out.0"
+  - "audio_attn2.to_k"
+  - "audio_attn2.to_q"
+  - "audio_attn2.to_v"
+  - "audio_attn2.to_out.0"
+  - "audio_ff.net.0.proj"
+  - "audio_ff.net.2"
+```
+
+> [!NOTE]
+> Audio-only modes have no `video` block in the strategy, so there is no need to train video or cross-modal
+> attention modules. Targeting only `audio_*` modules keeps the LoRA small and focused.
+
+---

 ## 🎬 Using Trained Models for Inference

@@ -255,12 +565,25 @@ LoRAs:

 | Training Mode           | Recommended Pipeline                                  |
 |-------------------------|-------------------------------------------------------|
-| LoRA / Audio-Video LoRA | `TI2VidOneStagePipeline` or `TI2VidTwoStagesPipeline` |
-| IC-LoRA                 | `ICLoraPipeline`                                      |
+| T2V / I2V / A2V / Extension / Inpainting / Outpainting | `TI2VidOneStagePipeline` or `TI2VidTwoStagesPipeline` |
+| IC-LoRA (V2V / A2A / AV2AV) | `ICLoraPipeline`                                 |
+| V2A (Foley) / T2A / Audio Extension / Audio Inpainting | `TI2VidOneStagePipeline` or `TI2VidTwoStagesPipeline` |

 All pipelines support loading custom LoRAs via the `loras` parameter. See the [`ltx-pipelines`](../../ltx-pipelines/)
-package
-documentation for detailed usage instructions.
+package documentation for detailed usage instructions.
+
+> [!NOTE]
+> You can generate audio during validation even if you're not training the audio branch.
+> Set `validation.generate_audio: true` independently of whether audio has `is_generated: true`.
+
+---
+
+## 🔄 Migration from Legacy Strategies
+
+Legacy `text_to_video` and `video_to_video` strategy configs are forward-compatible and will continue to work (with a
+deprecation warning). We recommend migrating to `flexible` for access to all conditioning modes.
+
+---

 ## 🚀 Next Steps

@@ -272,6 +595,6 @@ Once you've chosen your training mode:

 > [!TIP]
 > Need a training mode that's not covered here?
-> See [Implementing Custom Training Strategies](custom-training-strategies.md)
-> to learn how to create your own strategy for specialized use cases like video inpainting, audio-only training, or
-> custom conditioning.
+> First check whether it can be expressed by composing existing `flexible` conditions. Use
+> [Implementing Custom Training Strategies](custom-training-strategies.md) only for custom losses,
+> noising rules, model outputs, or preprocessing that cannot be represented by configuration.
@@ -8,7 +8,7 @@ Memory management is crucial for successful training with LTX-2.

 > [!TIP]
 > For GPUs with 32GB VRAM, use the pre-configured low VRAM config:
-> [`configs/ltx2_av_lora_low_vram.yaml`](../configs/ltx2_av_lora_low_vram.yaml)
+> [`configs/t2v_lora_low_vram.yaml`](../configs/t2v_lora_low_vram.yaml)
 > which combines 8-bit optimizer, INT8 quantization, and reduced LoRA rank.

 ### Memory Optimization Techniques
@@ -111,7 +111,7 @@ Ensure you've installed the dependencies and are using `uv run` to execute scrip
 # From the repository root
 uv sync
 cd packages/ltx-trainer
-uv run python scripts/train.py configs/ltx2_av_lora.yaml
+uv run python scripts/train.py configs/t2v_lora.yaml
 ```

 > [!TIP]
@@ -168,29 +168,29 @@ LTX-2 requires the number of frames to satisfy `frames % 8 == 1`:

   ```bash
   uv run accelerate launch --config_file configs/accelerate/ddp_compile.yaml \
-     scripts/train.py configs/ltx2_av_lora.yaml
+     scripts/train.py configs/t2v_lora.yaml
   ```

 ### Issue: Poor Quality Validation Outputs

 **Solutions:**

-1. **Use Image-to-Video Validation:**
-   For more reliable validation, use image-to-video (first-frame conditioning) rather than pure text-to-video:
+1. **Use conditioned validation:** For more reliable validation, use image-to-video (first-frame conditioning) rather than pure text-to-video:

   ```yaml
   validation:
-     prompts:
-       - "a professional portrait video of a person"
-     images:
-       - "/path/to/first_frame.png"  # One image per prompt
+     samples:
+       - prompt: "a professional portrait video of a person"
+         conditions:
+           - type: first_frame
+             image_or_video: "/path/to/first_frame.png"
   ```

 2. **Increase inference steps:**

   ```yaml
   validation:
-     inference_steps: 50  # Default is 30
+     inference_steps: 30
   ```

 3. **Adjust guidance settings:**
@@ -35,75 +35,53 @@ uv run python scripts/split_scenes.py video.mp4 scenes/ --max-scenes 50

 ### Automatic Video Captioning

-The `scripts/caption_videos.py` script generates captions for videos (with audio) using multimodal models.
+The `scripts/caption_videos.py` script generates a single, detailed combined audio-visual
+caption per video as a continuous paragraph of prose. Two backends are available:
+
+- **`qwen_omni` (default)** — Qwen3-Omni-30B-A3B-Thinking served via a local
+  [vLLM](https://docs.vllm.ai/) HTTP server (~1-3 s/video on H100). Highest quality, runs
+  fully offline once the model is downloaded.
+- **`gemini_flash`** — Google Gemini (cloud, `gemini-3.5-flash`). No GPU required. Auth is
+  automatic: set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`) for the Developer API, or just have
+  Google Cloud credentials available (`gcloud auth` / an attached service account) and it
+  uses Vertex AI with no extra setup.
+
+**Step 1 — launch the captioner server** (`qwen_omni` only, one-time).
+
+`scripts/serve_captioner.py` runs vLLM in an isolated environment via `uvx`, so vLLM's heavy
+CUDA dependencies never touch the trainer's venv. It defaults to dynamic FP8 quantization
+(~31 GiB weights, fits on 40 GB GPUs, same speed as BF16 on H100):

 ```bash
-# Generate captions for all videos in a directory (uses Qwen2.5-Omni by default)
-uv run python scripts/caption_videos.py videos_dir/ --output dataset.json
+# Terminal 1 - stays running
+uv run python packages/ltx-trainer/scripts/serve_captioner.py

-# Use 8-bit quantization to reduce VRAM usage
-uv run python scripts/caption_videos.py videos_dir/ --output dataset.json --use-8bit
-
-# Use Gemini Flash API instead (requires API key)
-uv run python scripts/caption_videos.py videos_dir/ --output dataset.json \
-    --captioner-type gemini_flash --api-key YOUR_API_KEY
-
-# Use Gemini Flash with parallel workers for faster throughput
-uv run python scripts/caption_videos.py videos_dir/ --output dataset.json \
-    --captioner-type gemini_flash --num-workers 5
-
-# Caption without audio processing (video-only)
-uv run python scripts/caption_videos.py videos_dir/ --output dataset.json --no-audio
-
-# Force re-caption all files
-uv run python scripts/caption_videos.py videos_dir/ --output dataset.json --override
+# Useful variants:
+#   --print-cmd           show the vLLM command without running it
+#   --quantization bf16   use BF16 instead (needs ~66 GiB free VRAM)
+#   --hf-home /mnt/disk   override where the ~65 GB model is downloaded
 ```

-**Key features:**
-
- **Audio-visual captioning**: Processes both video and audio content, including speech transcription
- **Multiple backends**:
-  - `qwen_omni` (default): Local Qwen2.5-Omni model - processes video + audio locally
-  - `gemini_flash`: Google Gemini Flash API - cloud-based, requires API key
- **Parallel captioning** (Gemini Flash only): Use `--num-workers` to run multiple API calls concurrently for faster throughput on large datasets
- **Structured output**: Captions include visual description, speech transcription, sounds, and on-screen text
- **Memory optimization**: 8-bit quantization option for limited VRAM
- **Incremental processing**: Skips already-captioned files by default; progress is saved every 5 videos
- **Multiple output formats**: JSON, JSONL, CSV, or TXT
-
-**Caption format:**
-
-The captioner produces structured captions with four sections:
- `[VISUAL]`: Detailed description of visual content
- `[SPEECH]`: Word-for-word transcription of spoken content
- `[SOUNDS]`: Description of music, ambient sounds, sound effects
- `[TEXT]`: Any on-screen text visible in the video
-
-**Parallel captioning with Gemini Flash:**
-
-When using `--captioner-type gemini_flash`, you can speed up large dataset captioning by running multiple API calls at the same time using `--num-workers` (accepts 1–10, default is 1):
+**Step 2 — caption your videos.**

 ```bash
-export GEMINI_API_KEY="your-key-here"
+# Terminal 2 - default backend talks to the server above
+uv run python packages/ltx-trainer/scripts/caption_videos.py videos_dir/ --output dataset.json

-# Caption a large dataset with 5 workers running concurrently
-uv run python scripts/caption_videos.py videos_dir/ \
-    --output dataset.json \
-    --captioner-type gemini_flash \
-    --num-workers 5
+# Remote server:           --vllm-url http://other-host:8001/v1
+# Gemini (gemini-3.5-flash): --captioner-type gemini_flash   (uses GEMINI_API_KEY, else gcloud/Vertex)
+# Gemini, parallel calls:  --captioner-type gemini_flash --num-workers 5
+# Re-caption everything:   --override
 ```

-> [!NOTE]
-> `--num-workers` is only supported with `gemini_flash`. Using it with `qwen_omni` or any other local model will raise an error, because local GPU models are not thread-safe.
+Captioning is incremental (already-captioned files are skipped, progress saves every 5 videos)
+and writes JSON, JSONL, CSV, or TXT based on the output extension.

-> [!TIP]
-> Keep `--num-workers` between 3–5 for most use cases. Very high values (8–10) may hit Gemini API rate limits depending on your quota tier.
+Qwen3-Omni-Thinking can optionally emit a `<think>...</think>` chain-of-thought before the
+caption (`--enable-thinking`). It is off by default, which is recommended for bulk captioning
+(thinking is slower as it generates the reasoning trace first).

-**Environment variables (for Gemini Flash):**
-
-Set one of these to use Gemini Flash without passing `--api-key`:
- `GOOGLE_API_KEY`
- `GEMINI_API_KEY`
+For Gemini, keep `--num-workers` at 3-5 (higher values may hit API rate limits).

 ### Dataset Preprocessing

@@ -116,13 +94,6 @@ uv run python scripts/process_dataset.py dataset.json \
    --model-path /path/to/ltx-2-model.safetensors \
    --text-encoder-path /path/to/gemma-model

-# With audio processing
-uv run python scripts/process_dataset.py dataset.json \
-    --resolution-buckets "960x544x49" \
-    --model-path /path/to/ltx-2-model.safetensors \
-    --text-encoder-path /path/to/gemma-model \
-    --with-audio
-
 # With video decoding for verification
 uv run python scripts/process_dataset.py dataset.json \
    --resolution-buckets "960x544x49" \
@@ -186,6 +157,10 @@ uv run python scripts/compute_reference.py videos_dir/ --output dataset.json
 > You can edit this script to generate other types of reference videos for IC-LoRA training,
 > such as depth maps, segmentation masks, or any custom video transformation.

+> [!NOTE]
+> `compute_reference.py` writes generated references to the `reference_video` column, which
+> `process_dataset.py` detects automatically.
+
 ## 🔍 Debugging and Verification Scripts

 ### Latents Decoding
@@ -224,73 +199,17 @@ uv run python scripts/decode_latents.py /path/to/latents/dir \
 - **Debug training data**: Visualize what the model actually sees during training
 - **Quality assessment**: Ensure latent encoding preserves important visual details

+### Inference with Trained Models

-### Inference Script
+For inference with trained LoRAs, use the [`ltx-pipelines`](../../ltx-pipelines/) package which provides
+production-ready pipelines:

-The `scripts/inference.py` script runs inference with a trained model.
+- **Text/Image-to-Video**: `TI2VidOneStagePipeline`, `TI2VidTwoStagesPipeline`
+- **Distilled (fast) inference**: `DistilledPipeline`
+- **IC-LoRA video-to-video**: `ICLoraPipeline`
+- **Keyframe interpolation**: `KeyframeInterpolationPipeline`

-> [!TIP]
-> For production inference, consider using the [`ltx-pipelines`](../../ltx-pipelines/) package which provides optimized,
-> feature-rich pipelines for various use cases:
-> - **Text/Image-to-Video**: `TI2VidOneStagePipeline`, `TI2VidTwoStagesPipeline`
-> - **Distilled (fast) inference**: `DistilledPipeline`
-> - **IC-LoRA video-to-video**: `ICLoraPipeline`
-> - **Keyframe interpolation**: `KeyframeInterpolationPipeline`
->
-> All pipelines support loading custom LoRAs trained with this trainer.
-
-```bash
-# Text-to-video inference (with audio by default)
-# By default, uses CFG scale 4.0 and STG scale 1.0 with block 29
-uv run python scripts/inference.py \
-    --checkpoint /path/to/model.safetensors \
-    --text-encoder-path /path/to/gemma \
-    --prompt "A cat playing with a ball" \
-    --output output.mp4
-
-# Video-only (skip audio generation)
-uv run python scripts/inference.py \
-    --checkpoint /path/to/model.safetensors \
-    --text-encoder-path /path/to/gemma \
-    --prompt "A cat playing with a ball" \
-    --skip-audio \
-    --output output.mp4
-
-# Image-to-video with conditioning image
-uv run python scripts/inference.py \
-    --checkpoint /path/to/model.safetensors \
-    --text-encoder-path /path/to/gemma \
-    --prompt "A cat walking" \
-    --condition-image first_frame.png \
-    --output output.mp4
-
-# Custom guidance settings
-uv run python scripts/inference.py \
-    --checkpoint /path/to/model.safetensors \
-    --text-encoder-path /path/to/gemma \
-    --prompt "A cat playing with a ball" \
-    --guidance-scale 4.0 \
-    --stg-scale 1.0 \
-    --stg-blocks 29 \
-    --output output.mp4
-
-# Disable STG (CFG only)
-uv run python scripts/inference.py \
-    --checkpoint /path/to/model.safetensors \
-    --text-encoder-path /path/to/gemma \
-    --prompt "A cat playing with a ball" \
-    --stg-scale 0.0 \
-    --output output.mp4
-```
-
-**Guidance parameters:**
-
-| Parameter | Default | Description |
-|-----------|---------|-------------|
-| `--guidance-scale` | 4.0 | CFG (Classifier-Free Guidance) scale |
-| `--stg-scale` | 1.0 | STG (Spatio-Temporal Guidance) scale. 0.0 disables STG |
-| `--stg-blocks` | 29 | Transformer block(s) to perturb for STG |
-| `--stg-mode` | stg_av | `stg_av` perturbs both audio and video, `stg_v` video only |
+All pipelines support loading custom LoRAs trained with this trainer.

 ## 🚀 Training Scripts

@@ -300,13 +219,13 @@ Use `scripts/train.py` for both single GPU and multi-GPU runs:

 ```bash
 # Single-GPU training
-uv run python scripts/train.py configs/ltx2_av_lora.yaml
+uv run python scripts/train.py configs/t2v_lora.yaml

 # Multi-GPU (uses your accelerate config)
-uv run accelerate launch scripts/train.py configs/ltx2_av_lora.yaml
+uv run accelerate launch scripts/train.py configs/t2v_lora.yaml

 # Override number of processes
-uv run accelerate launch --num_processes 4 scripts/train.py configs/ltx2_av_lora.yaml
+uv run accelerate launch --num_processes 4 scripts/train.py configs/t2v_lora.yaml
 ```

 For detailed usage, see the [Training Guide](training-guide.md).
@@ -316,5 +235,5 @@ For detailed usage, see the [Training Guide](training-guide.md).
 - **Start with `--help`**: Always check available options for each script
 - **Test on small datasets**: Verify workflows with a few files before processing large datasets
 - **Use decode verification**: Always decode a few samples to verify preprocessing quality
- **Monitor VRAM usage**: Use `--use-8bit` or quantization flags when running into memory issues
+- **Monitor VRAM usage**: Reach for quantization or lower-memory settings (e.g. FP8 for the captioner server) when running into memory issues
 - **Keep backups**: Make copies of important dataset files before running conversion scripts