Add vLLM-Omni image-to-video sample

[wirjo]

Summary

Add an end-to-end vLLM-Omni generative media sample that:

• Deploys black-forest-labs/FLUX.2-klein-4B to a SageMaker real-time endpoint.
• Deploys Wan-AI/Wan2.1-VACE-1.3B-diffusers to a SageMaker asynchronous endpoint.
• Generates a PNG, converts it to a compact video-sized JPEG reference, and retrieves the generated MP4 from Amazon S3.
• Provides command-line scripts and a Streamlit application that share the same request helpers.
• Pins the AWS vLLM-Omni DLC tag omni-sagemaker-cuda-v1.6.
• Supports deployment resume after local credential expiry and writes asynchronous failures to a separate Amazon S3 prefix.

Why this sample

The repository already contains a FLUX.2-dev example and a Wan text-to-video example. This sample connects an image model to an image-conditioned video model and uses the vLLM-Omni DLC for both endpoints. It also shows why the image endpoint uses real-time inference while the longer-running video endpoint uses SageMaker Asynchronous Inference.

The vLLM-Omni Videos API accepts multipart form data and caps an individual form part at 1 MB. A 1024 x 1024 FLUX PNG can exceed that size after base64 encoding, so the sample resizes the reference to the requested video dimensions and encodes it as JPEG before constructing the multipart request.

Files

• deploy.py creates both endpoints, records their resource names, and can resume an interrupted deployment.
• generate.py runs the complete image-to-video workflow.
• app.py provides a Streamlit interface.
• cleanup.py removes the SageMaker resources.
• vllm_omni_media.py contains shared request, polling, state, failure, and cleanup helpers.
• tests/ covers payloads, reference-image preprocessing, endpoint configuration, asynchronous request submission, failure handling, resource reuse, and local state.

Validation

Validated end to end in us-east-1 on 2026-08-30 with AWS vLLM-Omni DLC omni-sagemaker-cuda-v1.6:

• FLUX.2-klein endpoint: ml.g6.xlarge, InService after 9 minutes 30 seconds.
• Wan VACE endpoint: ml.g6e.xlarge, InService after 8 minutes 40 seconds.
• Final default CLI run:
  • FLUX image: 4.7 seconds, 1024 x 1024 PNG, approximately 3.0 MiB.
  • Wan VACE service-side model latency: 8.9 seconds.
  • Wan VACE CLI time including Amazon S3 polling: 11.8 seconds.
  • Video: H.264 MP4, 480 x 320, 17 frames, 8 fps, 2.125 seconds, 342,296 bytes.
• Reviewed the first, middle, and final video frames. The 30-step default preserved the observatory, coastline, and composition while adding motion.
• Submitted a malformed image reference. SageMaker wrote the 400 response to the configured failure prefix, and the helper surfaced the exact error instead of waiting for the output timeout.
• Verified the Streamlit application locally and confirmed that video generation stays disabled until an image is available. The user also completed a manual application check.
• Ran cleanup.py, then confirmed that the sample endpoints, endpoint configurations, and models were removed. Temporary request and failure prefixes were empty.

Local quality gates:

• 18 unit and SageMaker API contract tests
• Ruff
• Bandit
• Python bytecode compilation
• Git diff checks
• Documentation link checks

These are single-run validation measurements for the documented sample settings, not performance benchmarks.

Cost and cleanup

The sample creates two GPU endpoints. Run python cleanup.py after testing. The cleanup command removes the SageMaker endpoints, endpoint configurations, and models while retaining generated Amazon S3 artifacts for review.