# SCAIL-2 on Modal — free testing environment

Test the SCAIL-2 LTX-2 training/inference **code path** on Modal without a local
GPU. The free path uses tiny random-init models + synthetic data (no checkpoint,
no dataset), so it validates that the SCAIL wiring runs on real Linux/GPU — not
model quality.

> ⚠️ The real 19B LTX-2 model is **not** free to train/run. The `verify`/`smoke`
> targets are near-free; `train` on a real checkpoint uses a paid GPU.

## 1. One-time setup

1. Sign up at [modal.com](https://modal.com) (the free Starter plan includes a
   monthly credit allowance — enough for many `verify`/`smoke` runs).
2. Install and authenticate:
   ```bash
   pip install modal
   modal setup          # opens a browser to link your account / token
   ```

## 2. Free / near-free checks

From the repo root:

```bash
# CPU: SCAIL-2 Phase 1-4 plumbing (driving concat, mask channels, zero-init widen,
# a FlexibleStrategy training step + loss, inference conditioning assembly).
modal run modal/app.py::verify

# Same checks on a T4 GPU (validates the CUDA path). ~cents.
modal run modal/app.py::smoke
```

The first run builds the image (installs the `ltx-core`/`ltx-pipelines`/`ltx-trainer`
workspace via `uv sync`). `ltx-kernels` (CUDA-compiled) is intentionally skipped —
attention falls back to PyTorch SDPA, so no CUDA toolchain is required.

Expected tail:
```
[OK] Phase 1 driving concat: seq 48 -> 96
[OK] Phase 2 zero-init widen: in_features=72, output preserved
[OK] Phase 3 training step: cond_channels (1, 96, 56), loss ...
[OK] Phase 4 build_scail_conditionings: [driving, mask_channels]
All SCAIL-2 checks passed on cuda   # (or cpu)
```

## 3. Scaling up to real training (paid)

`train` runs `packages/ltx-trainer/scripts/train.py` against a real checkpoint.
You must supply the weights + data via the persistent `scail-data` Volume.

1. Create/populate the Volume (checkpoint, Gemma encoder, preprocessed latents):
   ```bash
   modal volume create scail-data          # if not auto-created
   modal volume put scail-data /local/ltx-2-model.safetensors /model/ltx-2.safetensors
   modal volume put scail-data /local/gemma /model/gemma
   modal volume put scail-data /local/preprocessed /data/preprocessed
   ```
2. Copy `configs/scail_animation_lora.yaml`, and point its paths at the mounted
   Volume (everything lands under `/data` in the container):
   ```yaml
   model:
     model_path: "/data/model/ltx-2.safetensors"
     text_encoder_path: "/data/model/gemma"
     mask_conditioning_channels: 56       # widens patchify_proj at load
   data:
     preprocessed_data_root: "/data/preprocessed"
   ```
   (Put your edited config on the Volume too, or bake it into the repo.)
3. Launch (pick a GPU big enough for the model — the 19B needs an A100):
   ```bash
   # edit gpu="A10G" -> "A100" in modal/app.py::train for the full model
   modal run modal/app.py::train --config-rel /data/scail_animation_lora.yaml
   ```

### Dataset preprocessing (not yet automated for SCAIL)

The SCAIL training config expects, under `preprocessed_data_root/`:

```
latents/           # target video latents
conditions/        # text embeddings
driving_latents/   # driving video latents (same F/H/W as target)
char_masks/        # per-sample "mask" = [K+1, F_pix, H, W]  (ch0 env switch, 1..K binding slots)
```

`latents/`, `conditions/`, and `driving_latents/` come from the existing
`packages/ltx-trainer/scripts/process_dataset.py` (run it once per video set).
`char_masks/` still needs a segmentation step (e.g. SAM) to produce the semantic
masks — that preprocessing is not implemented yet (see `docs/tasks.md` 3.7).

## Cost notes

| Target  | GPU   | Rough cost | Use |
|---------|-------|-----------|-----|
| `verify`| none  | ~free      | validate code path on CPU |
| `smoke` | T4    | cents      | validate CUDA path |
| `train` | A10G/A100 | paid   | real fine-tuning (needs checkpoint + data) |

Free credits are best spent on `verify`/`smoke` to catch integration issues before
committing a paid GPU to a real run. Watch usage in the Modal dashboard.