scripts/process_char_masks.py turns per-sample character label-map videos/images
(integer pixel labels: 0 = environment, 1..K = characters -> binding slots) into
the pixel-space semantic-mask tensors the SCAIL training/inference path consumes:
{"mask": [K+1, F_pix, H_pix, W_pix]} (ch0 = environment switch, ch1..K = slots).
- Aligns to the target video's latent grid read from the saved latent metadata
(F_pix=(F-1)*8+1, H*32, W*32), so char_masks/ lines up file-for-file with
latents/ / driving_latents/ for PrecomputedDataset.
- Nearest-neighbour resize so integer labels are never blended; labels > K are
dropped with a warning; ch0 filled uniformly with --environment-switch.
- Reuses process_videos.py helpers (naming, atomic save, VAE factors) and matches
its typer CLI conventions.
Verified on CPU: a synthetic 2-character label map (plus an out-of-range id)
produces mask (7,17,128,128) with ch0 uniform, slots placed correctly, id>K
dropped, and feeds encode_mask_channels to the 8*(K+1)=56 channels. README +
docs/tasks.md 3.7 updated (upstream label-map generation via SAM/tracking is
dataset-specific and still out of scope).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A free-tier-friendly Modal app to validate the SCAIL-2 code path on real
Linux/GPU without a local GPU or the 19B checkpoint.
- modal/checks.py: device-aware (CPU/GPU auto) consolidation of the Phase 1-4
plumbing checks using tiny random-init models + synthetic data (no checkpoint,
no dataset): driving concat, zero-init patchify_proj widening (output-preserving),
a FlexibleStrategy driving+mask training step + loss, and inference conditioning
assembly. Passes on CPU locally.
- modal/app.py: builds the workspace via `uv sync` (skips CUDA-only ltx-kernels;
attention falls back to SDPA). Functions: verify (CPU, ~free), smoke (T4, cents),
train (A10G/A100, paid — runs the real trainer against a checkpoint + data on the
scail-data Volume).
- modal/README.md: free-tier setup (modal setup), run commands, cost table, the
scale-up path, and the expected preprocessed dataset layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
indigo