Skip to content

[BUG] Groq 400 json_validate_failed causes silent fallback failure in code-generation #150

Description

@koydas

🎯 Goal

Fix the code-generation job so that a Groq json_validate_failed error is detected, logged, and correctly falls through to the Anthropic provider instead of crashing the run.

📍 Context

  • Repo: autonomous-dev-loop
  • Component: scripts/lib/groq_client.mjs, scripts/lib/error_taxonomy.mjs, scripts/lib/llm_client.mjs
  • Environment: GitHub Actions — code-generation job

❗ Observed problem

When Groq's JSON mode cannot constrain a large structured output (large file_content strings in the changes array), it returns HTTP 400 with code: "json_validate_failed". The pipeline crashes with:

{"level":"error","msg":"Fatal error","error":"All providers failed: groq: Groq API HTTP error 400: {\"error\":{\"code\":\"json_validate_failed\",...}}"}

Root cause: classifyError('400') in error_taxonomy.mjs returns 'UNKNOWN' (400 is not in the PERMANENT or TRANSIENT lists). In groq_client.mjs, retryable is false for 400 so no retry happens. In llm_client.mjs, UNKNOWN does not trigger a break, so the loop should continue to Anthropic — but the final error message only reports the Groq failure, indicating the Anthropic fallback also fails silently without surfacing its own error clearly.

🔁 Steps to reproduce

  1. Trigger the code-generation workflow on an issue requiring multi-file changes (≥ 3 large files).
  2. Observe the Groq provider returning HTTP 400 with json_validate_failed.
  3. The job exits with code 1 — "All providers failed".

Example: run 26795257405, job 78990103447 (run ID issue-134)

✅ Expected behavior

When Groq returns json_validate_failed, the system:

  1. Immediately stops retrying Groq (already the case — retryable: false).
  2. Logs the Groq failure with its error code for traceability.
  3. Falls through to Anthropic and succeeds (or reports the Anthropic failure clearly if it also fails).

🧪 Acceptance criteria

  • When Groq returns HTTP 400 with code: "json_validate_failed", groq_client.mjs attaches err.code = 'json_validate_failed' to the thrown error so downstream handlers can inspect it.
  • llm_client.mjs logs { provider: 'groq', error_type: 'json_validate_failed', status: 400 } before moving to the next provider.
  • When Groq fails with json_validate_failed and Anthropic is available, the callLLM function returns the Anthropic response successfully (no crash).
  • When Groq fails with json_validate_failed and Anthropic also fails, the error message includes both failures (e.g. "All providers failed: groq: ..., anthropic: ...") so the actual Anthropic error is visible.
  • Unit test in scripts/tests/llm_client.test.mjs: mocking Groq to throw a json_validate_failed 400 error verifies that Anthropic is called as fallback.
  • Unit test in scripts/tests/groq_client.test.mjs: a mocked 400 response with json_validate_failed body results in an error with err.code === 'json_validate_failed' and err.retryable === false.

⚠️ Constraints

  • Do not change the retry behavior for other 400 errors (e.g. malformed payloads unrelated to JSON mode).
  • The PERMANENT break in llm_client.mjs stops all providers — do not reclassify json_validate_failed as PERMANENT; the fallback to Anthropic must remain active.
  • Backward compatible: existing error handling for 401, 403, 429, 5xx must be unaffected.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingready-for-devIssue validated and ready for automated implementation

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions