Skip to content

[tracking] C1: Long-video character replacement workflow #351

Open
0 of 9 issues completed
Open

Description

@FayeSpica

Goal and scope

Deliver C1, an editable long-video character replacement workflow, through What a Day, STORM, and Goodnight. Parent roadmap: #263. Reuse #212 for workbench implementation and shared project operations.

Reference implementation and examples: vllm-project/vllm-omni#9194
Goodnight source: https://www.youtube.com/watch?v=FoO7Pmx0bE4

Serving baseline

Use vLLM-Omni's MiniMax-H3 service with Swap LoRA for all three cases. FastH3 + Swap LoRA is the recommended starting configuration, following #9194.

Record the deployed model, vLLM-Omni revision, effective sampling settings, Swap LoRA version/strength and adapter activation evidence. Verify the selected deployment supports the combination before generation.

Use the cases to mature H3 character replacement, including reference handling, continuation, reverse-generation inputs, reliability and output quality. Open and link upstream bug/feature issues for serving gaps; track OpenDirector integration gaps in this repository and verify fixes against the affected cases.

User flow and key rules

Plan source intervals, character mappings, replacement scope, references, generation direction and segment dependencies before generation. Support single- and multi-character references, continuation, reverse generation, recovery and local revision. Preserve accepted execution snapshots and reuse completed work.

Implementation impact

Coordinate Agent planning/recovery with #261, editing and assembly with #262 and #212, and reference/guide/continuation/direction/LoRA contracts with #264. Verify selected model support before execution. Shared business commands serve UI and Agent.

Acceptance criteria

  • Validate vLLM-Omni H3 + Swap LoRA with effective adapter activation and case-level quality evidence; track discovered gaps and verify their fixes.
  • Complete T1–T6 capability tasks and T7–T9 case deliveries linked below.
  • Each case delivers a playable film with audio, editable project, source/output comparison and known quality issues.
  • Review identity, motion, clothing, background, continuity and audiovisual timing; complete a local revision and reopen the project.
  • Record exact source intervals, permissions, references, model/runtime versions, parameters, time and resource use. Use the source intervals below; agree quality, budget and delivery dates per case.
  • Provide Chinese and English workflow guidance and verify UI/Agent collaboration.

Tasks

Task Deliverable Acceptance cases
T1 · Single-character references #352 Person/reference mapping, appearance-scope controls and prompt guidance with reviewed outputs. STORM #359
T2 · Multi-character references #353 One-to-one mappings, per-segment references and entrance/occlusion handling. What a Day #358 (five people); Goodnight #360 (two people)
T3 · Replacement planning #354 Reviewable, persisted segment plan with source ranges, mappings, directions, dependencies and guide/keep ranges. All three cases
T4 · Continuation and recovery #355 Aligned reference/guide windows, durable task records, interruption recovery and dependency-aware reruns. STORM #359 for continuation/recovery; What a Day #358 for dependencies/reuse
T5 · Reverse generation #356 Later-anchor selection, direction-aware generation, forward-playback restoration and forward/reverse comparison. Sequential-entrance segment in What a Day #358
T6 · Assembly and delivery #357 Adopted results, original audio, editable timeline, playable export and reopen/local-revision support. All three cases

Each capability delivers applicable skill guidance, necessary shared-function fixes and case evidence. One case run may validate several capabilities.

Case deliveries

Task Focus Original-video interval Retained delivery at 24 FPS
T7 · What a Day #358 Five-character replacement 01:19–01:46 27 s / 648 frames
T8 · STORM #359 Single-character long-video continuation 04:18–04:43 25 s / 600 frames
T9 · Goodnight #360 Two-character replacement 00:17–00:40.54 566 frames / 23.5833 s

The specified excerpts define minimum delivery coverage. For Goodnight, the saved source interval is 23.54 s and the existing 24-FPS delivery is 566 frames; verify exact source timestamps and document frame-normalization rounding.

Built-in skill

Review the built-in /character-replacement skill as part of this task. Where the task exposes gaps, update its instructions, planning rules, command guidance or examples, and validate the updated skill through the corresponding workflow. Record the skill changes and execution evidence in the implementation PR; if no change is needed, record the verification result.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:agentAgent runtime, assistant chat and toolsarea:assetsAssets, library and reference inputsarea:harnessHarness and skill integration (Default, OpenCode, Codex, Claude Code, Jiuwen, DSH)area:providerModel providers and generation protocolsarea:timelineTimeline, subtitles and export

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions