Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@ MOLTO_LOG_LEVEL_STDERR=INFO
# External API keys
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
COHERE_API_KEY=
VOYAGE_API_KEY=
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,21 @@ asyncio.run(main())

These settings are independent. Use the same protocol for the method and `endpoint` to get native SDK passthrough. The returned `MoltoStream.response_headers` retains successful upstream HTTP headers for callers such as proxy servers.

## Embeddings

`Molto.embeddings()` accepts an OpenAI embeddings body and returns the parsed OpenAI SDK response. With `OPENAI_API_KEY` configured:

```python
async with Molto() as molto:
result = await molto.embeddings(
{"model": "text-embedding-3-small", "input": ["A document", "A question"]}
)
for item in result.data:
print(item.index, len(item.embedding))
```

The [embedding guide and support matrix](docs/provider-embeddings.md) covers OpenAI, Cohere, VoyageAI, Infinity, and vLLM, including native task options and evidence limitations. OpenAI and Cohere have successful live recordings; VoyageAI, Infinity, and vLLM require further live verification. Embeddings use one ordinary asynchronous request, without streaming or automatic batching.

## Error handling

Molto exposes recognized errors in the downstream SDK namespace, with a normalized Molto cause for portable classification. HTTP errors preserve their upstream status and body.
Expand Down
5 changes: 4 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ Molto has two design goals:
- Support any implemented downstream protocol through any implemented upstream protocol with one provider-neutral translation model.
- Bypass translation when both sides use the same protocol, preserving native requests, events, and responses with the [shared error policy](errors.md).

Streaming and non-streaming requests are supported on native and translated routes. Streaming is the translated execution primitive; translated non-streaming responses are collected from the downstream-native event stream produced by the same event projector.
Conversational streaming and non-streaming requests are supported on native and translated routes. Streaming is the conversational translated execution primitive; translated non-streaming responses are collected from the downstream-native event stream produced by the same event projector.

`Molto.embeddings()` provides a separate OpenAI-shaped dense embedding API, with native OpenAI-compatible transport and Cohere/VoyageAI translation. It does not use conversational messages or streams. See the [five-provider support matrix, examples, and evidence limits](provider-embeddings.md).

## Cross-protocol contract

Expand All @@ -32,6 +34,7 @@ The public method selects the protocol returned to the application. The `endpoin
| OpenAI Responses | `Molto.responses()` | `responses` | OpenAI Responses SDK events | OpenAI SDK `Response` | [OpenAI Responses](protocols/openai-responses.md) |
| OpenAI Chat Completions | `Molto.chat_completions()` | `chat/completions` | OpenAI `ChatCompletionChunk` values | OpenAI SDK `ChatCompletion` | [OpenAI Chat Completions](protocols/openai-chat-completions.md) |
| Anthropic Messages | `Molto.messages()` | `messages` | Anthropic Messages SDK events | Anthropic SDK `Message` | [Anthropic Messages](protocols/anthropic-messages.md) |
| OpenAI Embeddings | `Molto.embeddings()` | `embeddings` | Not supported | OpenAI SDK `CreateEmbeddingResponse` | [Embeddings](provider-embeddings.md) |

Use Responses when an application benefits from typed output items and a rich streaming lifecycle. Use Chat Completions for applications built around the widely supported `messages` and choice-delta interface. Use Messages for applications that need the Anthropic SDK contract and its content-block model. The selected upstream endpoint does not have to match that choice.

Expand Down
8 changes: 8 additions & 0 deletions docs/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,14 @@ Molto deliberately raises the downstream SDK's `AuthenticationError` instead of

The upstream client constructs the intended request using its effective base URL and endpoint, preserving SDK path-prefix and trailing-slash behavior. The adapter wraps it in the synthetic response.

## Embeddings

Embedding clients reuse OpenAI SDK HTTP transport for all three wire protocols. Errors therefore use OpenAI SDK classes, including native Cohere and VoyageAI responses. The provider status, body, response bytes, and URL remain intact. Codes `model_not_found` and `context_length_exceeded` use their corresponding normalized categories; SDK authentication and not-found classes use `AuthenticationError` and `ResourceNotFoundError`. An unknown provider error is not classified by guessed message text. Other SDK errors, including rate limits and transport failures, pass through unchanged.

Missing embedding keys fail locally before the SDK can fall back to an unrelated OpenAI environment key, using the same synthetic-401 convention above. Self-hosted routes require an explicit base URL and use the non-secret `EMPTY` key when none is supplied. Unsupported task options, input forms, or output encodings raise `UnsupportedInput` or public request validation errors before HTTP; malformed output or inconsistent vector correspondence raises output-side validation errors.

Embedding replay covers real OpenAI invalid-dimensions/missing-model errors, Cohere missing-model errors, and VoyageAI authentication failures. Engine validation errors and unavailable native-provider success cases use explicitly synthetic schema fixtures; they are not live verification. See the [embedding evidence matrix](provider-embeddings.md).

## Implementation and verification

See [error translation conventions](../python/CONVENTIONS.md#error-translation) for implementation and [error cassettes](../tests/cassettes/README.md#error-cassettes) for replay requirements. Cassette-backed coverage includes streaming and non-streaming requests across native and translated routes for every error scenario. Non-streaming tests use non-streaming recordings and compare provider SDK exceptions and their normalized cause chains; translated calls still stream upstream internally. Missing-credential tests cover both result modes and assert that no HTTP request is sent.
Loading
Loading