Skip to content

[FEATURE] Early validation of prerequisites and external payloads #129

Description

@koydas

🎯 Goal

Fail fast with explicit, actionable errors when prerequisites are missing or external payloads are malformed — before any costly LLM call is made.

📍 Context

  • Repo: autonomous-dev-loop
  • Domain: Reliable fail-fast behavior
  • Component: scripts/lib/config.mjs, scripts/lib/prompts.mjs, scripts/auto_fix_pr.mjs

🚀 Description

Startup validation is partially in place but incomplete:

  • requireEnv() in config.mjs validates required secrets — good.
  • Prompt files (prompts/auto-fix-system.md, prompts/generation-user.md, etc.) are loaded on demand inside prompts.mjs with no existence check before the LLM call is attempted.
  • The GitHub event payload (GITHUB_EVENT_PATH) is parsed but its structure is not validated — a missing pull_request.number field is caught, but other required fields are not checked upfront.
  • LLM response payloads are parsed with a try/catch but error messages do not indicate which field failed or why.

The fix is a single validateStartup() call at the top of each entrypoint script, plus a validatePayload(schema, data) helper used at external boundaries.

🧩 Scope

In:

  • scripts/lib/config.mjs: add validateStartup() that checks all required secrets AND verifies that required prompt files exist on disk before returning.
  • scripts/lib/prompts.mjs: on loadPrompt(name), throw immediately with the file path if the file does not exist (currently lets readFileSync throw a generic ENOENT).
  • scripts/auto_fix_pr.mjs: validate the GitHub event payload structure after parsing:
    • Required: pull_request.number (already checked), pull_request.head.ref, repository (already via requireEnv).
  • LLM response validation: when parseJsonResponse fails, include the field path that was missing/invalid in the error message.
  • All validation errors must be classified as PERMANENT (once error taxonomy from issue [FEATURE] Unified TRANSIENT/PERMANENT/UNKNOWN error taxonomy #109 is in place).

Out:

  • Full redesign of global config format.
  • Runtime schema library (no ajv or similar — plain JS checks only).

🧪 Acceptance criteria

  • Functional

    • If a required secret is missing, the process exits with a clear message naming the variable before any network call is made.
    • If a prompt file does not exist, the process exits with the file path in the error message before the LLM call.
    • If the GitHub event payload is missing pull_request.number, the process exits immediately with a message identifying the missing field.
  • Edge cases

    • A prompt file that exists but is empty (0 bytes) is treated as invalid and throws with the file path.
    • A GitHub event payload that is valid JSON but not an object (e.g., null, array) is caught and throws with a descriptive message.
    • A truncated LLM response (valid JSON but missing content[0].text) produces an error message that names the missing path, not a generic "unexpected format".
  • Tests

    • Unit: validateStartup() throws with the variable name when ANTHROPIC_API_KEY is unset.
    • Unit: loadPrompt('nonexistent') throws with the resolved file path.
    • Unit: loadPrompt('empty-file') throws indicating the file is empty.
    • Unit: parseJsonResponse on a response missing content[0].text produces an error message containing content[0].text.
    • Unit: GitHub event payload null is rejected with a descriptive error, not a TypeError.

⚙️ Constraints

  • Do not hide root causes behind generic messages — every error must name the specific missing/invalid field or file.
  • No new runtime dependencies — plain fs.existsSync / fs.statSync for file checks.
  • Validation must complete synchronously before any await (so failures are caught before the first network call)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-devIssue validated and ready for automated implementation

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions