fix(stargate): back off discovery and report registration failures

[barrygreengus]

Why

Persistent Kubernetes API errors were retried immediately. Pylon registration and discovery logged certificate failures but discarded other connection, authentication, configuration, and stream errors, obscuring recovery failures.

What changed

• Apply kube's default watcher backoff.
• Retain causal diagnostics for non-TLS gRPC failures with safe endpoint fields and suppress identical repeated errors until recovery.
• Report unexpected stream completion and delay reconnects after registration-stream loss.
• Keep secret-file parser excerpts out of registration-token error chains.

Customer Release Notes

Stargate discovery backs off during Kubernetes API failures. Pylon reports actionable registration failures without repeating identical errors continuously or exposing token-file contents.

Plan Summary

Not applicable.

Usage

Registration warnings include the failed operation, endpoint identity, and cause. Certificate-specific diagnostics retain their existing categories and corrective guidance.

Testing

The rebased stack tip passed all 1,611 Cargo workspace tests, with 3 existing ignored performance tests. Workspace Clippy passed with warnings denied. The Kubernetes router Bazel target passed. A fake Kubernetes service and paused clock verify watcher backoff. Diagnostics tests cover causes, repetition, recovery, and secret-file redaction. No live cluster was used.

Notes

Layer 4 of the Stargate maintenance stack. No new metrics or configuration controls.

Issues

Relates to #1817

References

Kube watcher backoff

Related Pull Requests

Depends on #1822. Next layer: #1826.

Dependencies

No added or upgraded dependency versions. Enabled the existing Tokio test-util feature for deterministic watcher tests and refreshed Bazel metadata. No license or NOTICE changes.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🛡️ CodeQL Analysis

🚨 Found 5 issue(s)

Severity Breakdown:

• 🔴 Errors: 0
• 🟡 Warnings: 0
• 🔵 Notes: 0

📋 Top Issues

🔗 View full details in Security tab

_{🕐 Last updated: 2026-09-11 22:27:59 UTC | Commit: b4c0176}