Skip to content

fix(guardian): fall back to prompt-only JSON when response_format is unavailable (#781) - #782

Merged
zaebee merged 2 commits into
mainfrom
fix/781-opencode-structured-output-fallback
Nov 5, 2026
Merged

zaebee merged 2 commits into
mainfrom
fix/781-opencode-structured-output-fallback

Conversation

@zaebee

@zaebee zaebee commented Nov 5, 2026

Copy link
Copy Markdown
Owner

Closes #781.

What changed

  • OpenAICompatibleProvider builds the response_format payload in one helper (_response_format), so the base pass and the new fallback describe the same schema.
  • OpenCodeProvider.generate_structured now handles the gateway's 400 invalid_request_error whose message says "This response_format type is unavailable now": it re-sends the same prompt without response_format and returns the answer. The prompt already asks for the same JSON and the caller parses defensively, so this is the prompt-only mode OpenRouter already relies on when a model ignores the parameter — a real answer instead of a failed review.
  • The refusal is remembered on the provider instance, so a chunked review pays exactly one such 400 and the remaining chunks go prompt-only (one instance per review).
  • _raise_for_http_error raises the new OpenCodeStructuredOutputUnavailableError for that body only; inference_failed keeps its retry (fix(guardian): retry the gateway's transient inference_failed 400 (#779) #780), 429 keeps OpenCodeQuotaError, every other 4xx/5xx still fails on the first attempt with its body.

Tests

  • tests/unit/test_guardian_opencode.py: fallback success (second body has no response_format); the refusal is remembered (three POSTs for two calls, the third body prompt-only); a failed fallback raises with its body; plus the existing transient/429/truncation/reasoning cases.
  • Focused run: pytest tests/unit/test_guardian_opencode.py tests/unit/test_guardian_openrouter.py tests/unit/test_guardian_providers.py tests/unit/test_guardian_runner.py — 130 passed; ruff check, ruff format --check, pyrefly check --baseline green.

Why it matters

This is the failure that made the chunked review of #777 degrade to "⚠ finder call failed" after #780: every chunk asked for structured output and the upstream refused it as unavailable. With the fallback, the request is answerable; whether a chunk still fails on the 8000-token reasoning budget is a separate, measurable question (#779).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a fallback mechanism in the OpenCode provider to handle cases where the gateway rejects structured output (response_format) as unavailable. When this occurs, the provider retries the request using a prompt-only payload and remembers the refusal for subsequent calls in the same run. Additionally, unit tests were added to verify this fallback behavior. Feedback on the changes suggests optimizing the payload modification by using payload.pop() instead of a dictionary comprehension to remove the response_format key.

Comment thread src/cgis/guardian/providers/opencode.py Resolved
@sonarqubecloud

sonarqubecloud Bot commented Nov 5, 2026

Copy link
Copy Markdown

@zaebee
zaebee merged commit 5ed805b into main Nov 5, 2026
4 checks passed
@zaebee
zaebee deleted the fix/781-opencode-structured-output-fallback branch November 5, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

guardian(opencode): fall back to prompt-only JSON when the gateway rejects response_format

1 participant