Skip to content

feat(rules): honour operator rules on the Slack submit path - #20

Open
johnnyhuy wants to merge 1 commit into
mainfrom
agent/issue-19-slack-operator-rules
Open

johnnyhuy wants to merge 1 commit into
mainfrom
agent/issue-19-slack-operator-rules

Conversation

@johnnyhuy

Copy link
Copy Markdown
Contributor

What changed

Operator rules now fire on the Slack path as well as the API path. RAGEngine.submit_slack checks the rule engine before retrieval, and a matched question is submitted to Pi Durable carrying the pinned answer in operator_answer. The Pi worker delivers that answer through the existing job outbox without calling the model, so Slack delivery stays durable, retried, and idempotent instead of gaining a second posting path in the bot. Reusing a request id with a different pinned answer is now a conflicting reuse, and the runtime rejects a non-string operator_answer.

Part of #6
Closes #19

How it was verified

  • mise run install - runtime and frontend installed cleanly. The backend needed the worktree venv recreated with the mise-pinned Python 3.13, because the default Python 3.14 cannot install onnxruntime. No lockfile change was kept.
  • mise run test-backend - 59 passed, including the 2 new tests in backend/tests/test_rules.py (TestSlackShortCircuit). The new matched-path test was watched failing before the fix with AssertionError: retrieval must not run.
  • mise run test-runtime - 7 passed, including the new test pinned operator answer is delivered without calling the model. It was also watched failing before the fix.
  • mise run test-frontend - passed (the repo task currently reports no frontend tests yet).
  • Backend startup - the backend came up with Slack auth stubbed offline using the repo test pattern from backend/tests/test_conversations.py. The log showed "Starting Wingman backend..." and "Database initialized", and GET /health returned 200. A plain start with a fake token fails at Slack auth.test on main as well, so nothing here changes that.

Proof

Capture method (a) and (c) succeeded offline. Nothing connected to a live Slack workspace, and no Socket Mode was run.

Backend, matched Slack question with retrieval rigged to fail if it runs:

submit_slack('How do I reset VPN access?') ->
{
  "id": "job-1",
  "status": "pending"
}
payload submitted to Pi Durable:
{
  "request_id": "event-1",
  "conversation_id": "team:thread:C1:1.2",
  "question": "How do I reset VPN access?",
  "context": "",
  "sources": [{"source": "operator_rule", "rule_id": "vpn"}],
  "operator_answer": "Use the IT portal to reset VPN.",
  "slack": {"channel": "C1", "thread_ts": "1.2"}
}
OK: retrieval skipped, pinned answer carried, Slack target preserved

Runtime, same payload shape through the job outbox with a model callback that throws if called:

{
  "modelCalls": 0,
  "delivered": {
    "target": {"channel": "C1", "thread_ts": "1.2"},
    "text": "Use the IT portal to reset VPN."
  },
  "status": "done",
  "result": {
    "answer": "Use the IT portal to reset VPN.",
    "sources": [{"source": "operator_rule", "rule_id": "vpn"}],
    "confidence": "high"
  }
}
OK: pinned answer delivered through outbox, model never called

Reviewer notes

  • Nothing user-facing in the frontend changed in this slice, so there is no screenshot. Frontend capture (b) was not applicable.
  • Full verification against a live Slack workspace would need a real workspace token and Socket Mode. That was out of bounds for this run. The Slack delivery itself is the existing Pi outbox path, exercised here with a fake deliver callback, and a live check should confirm a pinned answer lands in a thread once.
  • Parent Add deterministic operator rules for static and mutable answers #6 stays open. Mutable or fetched-source rules and dashboard rule editing are still outstanding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Honour operator rules on the Slack submit path

1 participant