Repository navigation
Description
Goal
Put the user experience first. For each provider targeting the full creation workflow, ask whether a user can enter one API key once and use it for every core scenario in OpenDirector: assistant chat and script writing, image generation, video generation, and narration with a preset voice. The user should be able to complete the workflow in one project without switching providers or configuring another key.
Track which workflows are available and which gaps prevent that experience. Track mainland China and international endpoints separately when credentials, model availability, or protocols differ. Specialist providers can be tracked for a single modality.
Integration matrix
Updated on 2026-11-16 against upstream main 526ece2a and recent PRs. Integrated status follows merged implementation and current presets; live acceptance is recorded separately below.
Video priority uses the Artificial Analysis v2.1 Overall With Audio snapshot checked on 2026-11-09. Sort by image-to-video (I2V) point rank, with text-to-video (T2V) shown separately. P1 covers I2V ranks 1–10 and P2 ranks 11–25 in this matrix. Unranked rows lack a verified match between a tracked video integration and an AA entry. Ranks with overlapping confidence intervals may change. Entries marked as unconfirmed are conditional priorities pending verification of the exact model and endpoint.
AA image-to-video · AA text-to-video
| Provider | Endpoint scope | Text | Image | Video | TTS / voices | Tracking |
|---|---|---|---|---|---|---|
| Volcengine Ark Agent Plan | mainland China | — | ✅ Integrated | ✅ Integrated | ✅ Integrated · audio path | — |
| MiniMax | mainland China | ✅ Integrated · chat preset | ✅ Integrated · H3-Image / Highspeed | ✅ Integrated · H3 V2 | ✅ Integrated · synchronous TTS / system voices | H3 #78 · Images #280 · Speech #283 |
| Kling | official endpoint; account scope to confirm | — | — | ✅ Integrated · Kling 4.0 / Flash | — | #142 · PR #239 |
| international | — | ⏳ Tracked · Imagen | ⏳ Tracked · Veo | — | #144 | |
| OpenAI | international | ❌ No dedicated preset | ✅ Integrated · gpt-image generation / editing | ❌ No preset | ❌ No dedicated preset | Images #140 · PR #155 · Quality #220 |
| Black Forest Labs | international | — | ⏳ Tracked · FLUX | — | — | #141 |
| Zhipu BigModel | mainland China | ✅ Integrated · GLM | ✅ Integrated · GLM-Image | ✅ Integrated · CogVideoX-3 | ✅ Integrated · GLM-TTS / cloning | RFC #234 · PRs #245 #251 #254 #276 · Account reuse #289 (open) |
| MOSS | mainland China | — | ✅ Integrated preset · verify scope | — | ✅ Integrated · speech / voice library | Image #138 · Voices #205 · PRs #159 #218 |
| DeepSeek | hosted API | ✅ Integrated · chat preset | — | — | — | — |
Pending consideration
| Provider | Endpoint scope | Text | Image | Video | TTS / voices | Tracking |
|---|---|---|---|---|---|---|
| fal | international | — | ❌ No preset | ❌ No preset | ❌ No preset | Check model-specific inputs and queued result handling. |
| Runway | international | — | — | ⏳ Tracked | — | #143 |
| Replicate | international | — | ❌ No preset | ❌ No preset | ❌ No preset | Check prediction lifecycle and model-specific input schemas. |
| SiliconFlow | mainland China | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Test compatible chat and speech routes. Verify image and asynchronous video contracts separately. |
| SiliconFlow | international | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Verify the international catalog and credentials independently. |
| Kimi / Moonshot | mainland China | ⏳ PR #335 open · chat | — | — | — | #335 · Open Platform and Coding Plan scope under review |
| Kimi / Moonshot | international | ❌ No preset · chat | — | — | — | Verify streaming and tool calls through the generic chat configuration. |
| OpenRouter | international | ❌ No preset · chat | ❌ No preset · image generation | ❌ No preset · async video | ❌ No preset · TTS | Verify generic chat compatibility and the media request, polling, and result formats. |
| MiniMax | international | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Separate endpoint and account validation |
| Alibaba Cloud Model Studio | mainland China · Beijing | ✅ Integrated · Qwen chat | ❌ No preset | ❌ No preset | ❌ No preset | Chat PR #288 · Other modalities remain open |
| Alibaba Cloud Model Studio | international | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Verify model availability and regional endpoints. |
| xAI Grok API | international | ❌ No preset · Grok chat | ❌ No preset · Grok Imagine | ❌ No preset · Grok Imagine | ❌ No preset · standalone speech API | Verify chat compatibility and native media workflows. |
| Google Gemini Developer API | international | ❌ No preset · Gemini chat | ❌ No preset · Gemini image generation | ❌ No preset · Veo | ❌ No preset · Gemini TTS | Verify the Gemini Developer API separately from the Vertex AI endpoint tracked in #144. |
| Z.ai | international | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Separate endpoint and account validation |
In the capability columns, ✅ indicates a merged integration available through the documented configuration path; it does not by itself establish live acceptance. ⏳ indicates an open implementation issue or PR. ❌ indicates no dedicated dropdown option. The generic OpenAI-compatible option does not count as a provider preset.
A dash indicates a modality outside the current tracking scope for that row. Self-hosted vLLM-Omni is tracked separately in #224, with deployment #259, capability discovery #266 and unified local presets #325 still open.
Work to track
- Merge mainland BigModel chat, image, video and speech integrations through [RFC] providers: add Zhipu BigModel for text, image, video, and speech generation #234 (PRs [feat] providers: add Zhipu BigModel GLM chat support #245, Add BigModel GLM-Image generation for canvas and Agent #251, Add BigModel CogVideoX-3 video generation #254, [feat] providers: add Zhipu BigModel GLM-TTS and voice management (#238) #276).
- Complete BigModel saved-account reuse ([feat] providers: reuse a BigModel account across model categories #289) and verify all four modalities in one project with one key entered once.
- Merge mainland MiniMax image generation/editing ([feat] image: add MiniMax-H3-Image generation and editing #280) and synchronous TTS with system voices ([feat] providers: add MiniMax speech with system voices #283).
- Verify MiniMax live speech and the one-account, four-modality workflow. Track cloning and voice management separately.
- Merge Beijing Model Studio assistant chat ([feat] providers: add Beijing Qwen chat to Model Studio presets #288).
- Track Model Studio image, video and speech support separately from the delivered Beijing chat preset.
- Review Moonshot Kimi (feat: add a Moonshot Kimi chat preset (#240) #335) and Atlas Cloud chat ([feat] providers: add Atlas Cloud chat provider preset #293).
- Complete shared model-category filtering ([bugfix] providers: filter models that cannot serve the selected category (#133) #294) and capability discovery (feat(providers): discover capabilities and validate generation inputs #266).
- Confirm international MiniMax and Z.ai endpoint, credential, and model coverage before marking any capability verified.
- Track specialist providers through the existing issues in the matrix.
Recent integration evidence
- BigModel: merged PRs report real streaming/tool use ([feat] providers: add Zhipu BigModel GLM chat support #245), adopted and reopened images (Add BigModel GLM-Image generation for canvas and Agent #251), three video modes (Add BigModel CogVideoX-3 video generation #254), and preset speech plus clone/create/select/synthesize/delete in both languages ([feat] providers: add Zhipu BigModel GLM-TTS and voice management (#238) #276). Clone similarity still requires listening acceptance. These per-modality runs do not establish the complete one-key workflow; account reuse is pending [feat] providers: reuse a BigModel account across model categories #289.
- MiniMax: [feat] image: add MiniMax-H3-Image generation and editing #280 includes real canvas generation/editing evidence. [feat] providers: add MiniMax speech with system voices #283 used mocked provider HTTP and real local media bytes; live speech, voice catalog access and the full one-account workflow remain unverified.
- Model Studio Beijing: [feat] providers: add Beijing Qwen chat to Model Studio presets #288 reports a real
qwen3.8-maxstreamed tool-call/result/reply flow, with the text asset and conversation preserved after restart. Other regions and modalities are not covered by that run. - OpenAI Images [feat] provider: OpenAI Images (gpt-image) provider and a label taxonomy (#139, #140, #145) #155 is merged, but its protocol tests use mocked HTTP. A dedicated image adapter does not imply a dedicated provider dropdown preset.
- Open PRs remain pending delivery, including [feat] providers: reuse a BigModel account across model categories #289, [feat] providers: add Atlas Cloud chat provider preset #293, [bugfix] providers: filter models that cannot serve the selected category (#133) #294 and feat: add a Moonshot Kimi chat preset (#240) #335.
Evidence required to mark a capability verified
Record the endpoint region, model ID, implementation PR, and tested workflow. Verify generated media can be opened and adopted in the project. For full-workflow providers, run all four modalities with one account and key. Include screenshots in each implementation PR showing the workflow and result in OpenDirector.
Activity
wuli666 commentedon Nov 9, 2026
Hi @FayeSpica, I'd like to take the mainland MiniMax gap: official image generation (
image-01) and synchronous TTS with preset voices (speech-2.8).I'll send two PRs, image first, and verify all four modalities with one minimax.cn key. Voice cloning and management stay out of scope. Happy to open sub-issues if you prefer.
PCcoding666 commentedon Nov 10, 2026
Hi @FayeSpica, I’d like to take the mainland China Alibaba Cloud Model Studio (百炼) chat entry. Could I start with a Qwen assistant preset for the Beijing region, using
qwen3.8-maxand an editable workspace endpoint?I tested the existing assistant locally with simulated Qwen responses. It called an OpenDirector tool to write a text asset, received the tool result, and continued its reply. For the PR, I’d verify that same flow with a real Beijing-region account, including a streamed reply and the saved asset, and include screenshots of the Chinese and English setup.
Does that scope work for a first PR? I’m happy to track it in a dedicated sub-issue if that’s easier.
FayeSpica commentedon Nov 10, 2026
Thanks, @wuli666! assigned.
Image first sounds good. Feel free to open the PR and reference #240. If anything needs discussion, please open a sub-issue under #240.
FayeSpica commentedon Nov 10, 2026
Thanks for testing the tool call, @PCcoding666! Go ahead with the Beijing Qwen preset and reference #240 in the PR.
If you’re interested, feel free to open a separate sub-issue under #240 to track full Alibaba Cloud Model Studio support.
FayeSpica commentedon Nov 10, 2026
Hi @wuli666, any updates on the image API integration? MiniMax-H3-Image is out now, so could we prioritize adding support for it?
wuli666 commentedon Nov 11, 2026
Hi @FayeSpica, I saw #280 already landed MiniMax-H3-Image, thanks! I'll take the remaining part: synchronous TTS with preset voices (
speech-2.8).[feat] providers: add MiniMax speech with system voices
Zhangbo-cn commentedon Nov 12, 2026
Refs #264 (Q4 P0: "update #240's region/account/model/modality evidence"). Two findings, both verified against clean upstream main
efc9dbe9.The Zhipu row is out of date
All four Zhipu cells read
⏳ Tracked, but two have landed:efc9dbe94d15b4e4), presetzhipu,glm-5.3 / glm-5.3-flash / glm-5.2efc9dbe9), presetbigmodel,glm-image39288dfd), presetzhipuwidened to["agent", "audio"],glm-ttsOne API key still has to be entered twice
The two merged PRs each added their own preset, and neither knows about the other:
bigmodel["image"]glm-imagezhipu["agent"]glm-5.3, ...https://open.bigmodel.cn/api/paas/v4), samelogoOrganization.I first assumed this showed two identical entries in the picker. It does not -
providerOptionsfilters bykind, so each category shows exactly one. Worth correcting since that would have been the more visible bug.The real symptom is credential reuse. With a saved
glm-5.3chat connection, addingglm-image:matchingConnectionsrequiresmodel.kind === kind, so a saved chat connection is never offered when adding an image model. The user re-enters the same key. That is the case the goal section of this issue describes - "enter one API key once and use it for every core scenario".Note for whoever picks this up
#289 (draft) addresses exactly this with
isBigModelConnection/is_bigmodel_connection, matching on endpoint identity instead ofkind. That looks like the right layer to me. It keeps the two presets and relaxes the matching, rather than merging the presets - worth being explicit about which approach you want, since merging the presets would be a competing change to the same lines.#276 (GLM-TTS) has now merged, widening
zhiputokinds: ["agent", "audio"], so the current split iszhipu= chat + TTS andbigmodel= image. #289 has gonedirtyas a result; the conflict is two lines in the preset plus one adjacent test, and I posted the resolution there.Happy to take the matrix update, or the preset consolidation if you prefer that over #289's approach.
Zhangbo-cn commentedon Nov 12, 2026
Hi @FayeSpica, I'd like to take the Kimi / Moonshot mainland chat row. #276 (Zhipu GLM-TTS) has merged, so I have capacity.
Scope I'm proposing: a
kimichat preset for the mainland endpoint only -kinds: ["agent"],https://api.moonshot.cn/v1,protocol: "openai", bearer auth. The international row (platform.kimi.ai) is a separate account and endpoint, so I'd leave it out of this PR rather than claim coverage I haven't tested.I verified the endpoint against a real mainland account before asking, since the matrix notes for this row ask for streaming and tool calls specifically:
GET /v1/modelsreturns two models:kimi-k3andkimi-k3.1-pro.write_text_assetfunction,kimi-k3returnedfinish_reason: tool_callsand well-formed arguments ({"content":"...","name":"scene1"}).So this is OpenAI-compatible and should fit the existing provider path without a new adapter. For the PR I'd verify the full assistant flow end to end - streamed reply, a real tool call writing a text asset, and the saved asset after reopening the project - with bilingual setup screenshots.
One question on convention before I write it.
deepseekships nomodelsByKind, so the user types the model id, whilezhipupins["glm-5.3", ...]. For Kimi I'd prefer pinning["kimi-k3", "kimi-k3.1-pro"]since those are what the account actually returns, but I'll follow whichever you consider the convention for chat presets.Happy to track it in a sub-issue under #240 if you'd rather.
[feat] providers: add MiniMax speech with system voices (#283)
FayeSpica commentedon Nov 15, 2026
@Zhangbo-cn assigned. Please go ahead.
feat: add a Moonshot Kimi chat preset (ThinkFlowLab#240)