Add SCAIL-2 animation inference pipeline (Phase 4, ltx-pipelines)
Two-stage distilled inference pipeline that animates a character from a driving video, wiring the Phase 1-3 SCAIL-2 conditioning into a runnable CLI. - scail_animation.py: ScailAnimationPipeline (mirrors distilled.py) plus a pure, testable build_scail_conditionings that assembles VideoConditionByDrivingLatent + VideoConditionByMaskChannels (driving appended, mask on the trailing driving tokens), and a load_masks helper. main() + scail_animation_arg_parser add --driving-video / --mask-path / --mode / --driving-strength on top of the standard two-stage distilled parser. - LTXModelConfigurator now reads config `mask_conditioning_channels`, so a SCAIL-trained checkpoint whose config declares it builds the widened patchify_proj automatically — no runtime widening wrapper needed at inference. - CLAUDE.md pipeline table row. Verified on CPU (verify_phase4_pipeline.py): the module imports, the CLI parses the SCAIL flags, build_scail_conditionings grows the sequence and places the mask channels on the driving tokens (target stays zero), driving-only leaves cond_channels None, and the configurator honors mask_conditioning_channels (patchify_proj widened from config). End-to-end runs still need a GPU and a SCAIL-trained checkpoint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+6
-2
@@ -52,8 +52,12 @@
|
||||
- `configs/scail_animation_lora.yaml` + docs。CPU 單元驗證通過(`verify_phase3_trainer.py`)。
|
||||
- **本機無 GPU/Linux/checkpoint → 未跑實機訓練**;dataset 前處理(driving latents + 語意 mask)與 validation runner 接線未做。
|
||||
|
||||
### Phase 4 — Pipeline + CLI 包裝 ⬜ 未開始
|
||||
- 仿 `lipdub.py` 寫 `scail_animation.py` pipeline + arg parser。
|
||||
### Phase 4 — Pipeline + CLI 包裝 ✅ 已完成(程式碼路徑;實跑需 GPU)
|
||||
- `LTXModelConfigurator` 讀 `mask_conditioning_channels` → SCAIL checkpoint 自描述、載入自動加寬(不需 runtime wrapper)。
|
||||
- `scail_animation.py`:`ScailAnimationPipeline`(仿 `distilled.py` 兩階段)+ 可測純函式 `build_scail_conditionings`(driving + mask 條件組裝)+ `load_masks` + `main()`。
|
||||
- `utils/args.py` `scail_animation_arg_parser`(`--driving-video/--mask-path/--mode/--driving-strength`)。
|
||||
- CPU 驗證通過(`verify_phase4_pipeline.py`);`ltx-pipelines/CLAUDE.md` 表格 row。
|
||||
- **實機端到端需 GPU + SCAIL-trained checkpoint**(config 含 `mask_conditioning_channels` + 訓練好的 patchify_proj + LoRA)。
|
||||
|
||||
## Phase 1 簡化取捨(記錄,Phase 2 需回頭處理)
|
||||
- (a) driving 時間座標直接複製 target 的(token-wise),故 driving 需與 target 同 F/H/W。
|
||||
|
||||
Reference in New Issue
Block a user