diff --git a/docs/8-reference/permissions.md b/docs/8-reference/permissions.md index ecfccafd9..64cd99760 100644 --- a/docs/8-reference/permissions.md +++ b/docs/8-reference/permissions.md @@ -243,6 +243,30 @@ approval; they do not grant organization API permissions. See | cloudsensor.get.mtd | View cloud sensor metadata only | | cloudsensor.set.mtd | Modify cloud sensor metadata only | +### Investigation Engine + +See [Investigation Engine permissions](../investigation-engine/permissions.md) for what each permission allows and which combinations a task needs. + +No preset role holds any `investigation_plan.*` or `investigation_policy.*` permission, not even Owner; grant them explicitly. + +| Permission | Description | +| --- | --- | +| investigation.get | Read Investigation Engine runs, actions, statistics, templates, operators and the effective policy (together with `insight.evt.get`) | +| investigation.set | Start runs, waves and tests, cancel and compare runs, and decide actions (together with `insight.evt.get` and, to decide an action, that action class's own permission) | +| investigation.del | Delete permission of the `investigation` family, shared with Cases (held by the Owner role) | +| investigation.get.mtd | Metadata read permission of the `investigation` family, shared with Cases | +| investigation.set.mtd | Metadata write permission of the `investigation` family, shared with Cases | +| investigation\_plan.get | Read Investigation Engine plan templates | +| investigation\_plan.set | Create and modify plan templates. | +| investigation\_plan.del | Delete plan templates | +| investigation\_plan.get.mtd | View plan template metadata only | +| investigation\_plan.set.mtd | Modify plan template metadata only | +| investigation\_policy.get | Read Investigation Engine policies | +| investigation\_policy.set | Create and modify policies, including the identity runs execute as. | +| investigation\_policy.del | Delete policies | +| investigation\_policy.get.mtd | View policy metadata only | +| investigation\_policy.set.mtd | Modify policy metadata only | + ### Playbooks | Permission | Description | diff --git a/docs/8-reference/response-actions.md b/docs/8-reference/response-actions.md index 5e61d9641..29181dfac 100644 --- a/docs/8-reference/response-actions.md +++ b/docs/8-reference/response-actions.md @@ -509,6 +509,59 @@ respond: For detailed configuration options, see [D&R-Driven AI Sessions](../9-ai-sessions/dr-sessions.md). +### start investigation + +Starts an [Investigation Engine](../investigation-engine/index.md) run for the detection that the rule just reported. The engine investigates the alert on its own, with the identity set in the organization's investigation policy, and works toward a verdict. The result is a verdict, its proof and a case, and any containment the organization's [autonomy policy](../investigation-engine/policies.md) allows. + +```yaml +- action: report + name: suspicious-process +- action: start investigation + template: suspicious-process + urgency: high + coalesce_key: "sid:{{ .routing.sid }}" + inputs: + hash: "{{ .event.HASH }}" +``` + +Place the action **after** the `report` action: the run is tied to the detection that report created (its id, category and priority, plus the rule's tags). Without a preceding `report` the run only knows the rule name and the sensor, and has no detection id. + +#### Optional Parameters + +| Parameter | Description | +|-----------|-------------| +| `template` | Name of the investigation template to run. A literal (not templated), up to 128 letters, digits, `_`, `.` and `-`, starting with a letter or digit. The template must exist in the organization and have `status: active`. When omitted, the engine picks the **active** template whose `applies_to` best matches the detection. | +| `urgency` | One of `low`, `medium`, `high`, `critical`. A literal. When omitted, the engine derives it from the detection priority: 8 or more is `critical`, 6 or more `high`, 3 or more `medium`, anything lower `low`. | +| `coalesce_key` | [Template string](../4-data-queries/template-transforms.md), at most 256 bytes once rendered. Alerts with the same key join the run already open for it instead of starting another. Defaults to `sid:` when the event has a sensor id. | +| `inputs` | A map of values handed to the run as its inputs, using the same [transform syntax](../4-data-queries/template-transforms.md) as the `data` block of `start ai agent`: each value is an event path or a `{{ }}` template. At most 64 KiB once rendered. | +| `suppression` | The usual [suppression](#suppression) descriptor, supported by every action. | + +Unknown parameters are rejected when the rule is saved, as are a `template` or `urgency` that is not a valid literal. + +!!! warning "Use fields that every matching event has" + A path to a field the event lacks is dropped from `inputs`, and a template that names a missing field renders ``. Neither is an error. In `coalesce_key` that has a side effect: if the key uses a field that some events lack, every such alert renders the same key and **joins one shared run**. Build the key from fields every matching event carries, such as `{{ .routing.sid }}`. + +A template only sees the inputs this action lists, plus the detection id. A template written to take the `detection_id` and derive everything else with facts works unchanged from a rule. See [Run inputs](../investigation-engine/templates.md#run-inputs). + +#### Requirements + +- The Investigation Engine must be enabled for the datacenter. Where it is not, saving a rule that uses this action fails with `start investigation is not enabled in this environment`. +- The template must be **active**. A draft is refused for a run with no person behind it, and a template is never picked by `applies_to` while it is a draft. +- The organization's `investigation_policy` record `default` should set `identity.default` to an organization API key (see [Run identity](../investigation-engine/policies.md#run-identity)). A rule has no user token, so without an identity for the template the start is refused with a message that names `identity.default`. A named template can instead carry its own `identity.templates..default`. +- If no active template applies to the detection, the start is refused and the message says what was tried. + +#### Behavior + +- Starting a run never delays the rule: the request is sent in the background with a short timeout (5 seconds per attempt, 8 seconds in all). The detection is reported whether or not a run starts. +- A run is not started, and the failure is logged and counted but not shown to the rule, when the engine is unavailable, the organization is at its concurrent run limit, the organization exceeds the start rate (2 starts per second sustained, with a burst of 20, per host process), or the template or identity is refused as above. +- If a `coalesce_key` or an `inputs` template errors, or the rendered value is over its size limit, the action fails and no run starts. A field the event does not have is not an error: as with `data`, a path to it is dropped from `inputs`, and a template naming it renders ``. +- A given detection id starts at most one run: the detection id is the run's idempotency key (kept for the engine's record retention, and scoped to the organization and the D&R starter). +- The detection may not be stored yet when the run starts, because the report and the start run independently. The engine retries the lookup of that one detection for up to about 30 seconds (with backoff), then treats it as `unknown`, never absent. See [Automatic Starts](../investigation-engine/automation.md#dr-rules-start-investigation). +- The run is recorded as started by the rule: its trigger has `source: dr-rule`, and `rule` is the rule record name prefixed with `general.`. Use that form in a template's `applies_to` (`rule: ['general.my-rule']`). A D&R rule never sends a severity, so an `applies_to` with a `severity` key does not match. +- Resource ACL checks work like `start ai agent`: an event restricted by `acl:` tags is refused unless the rule's author scope covers it. +- A replay never starts a run: it records an `investigation` row (with the template name when the rule names one) instead. +- Not available in Email Security (mail) rules: saving one is refused with `not available to a mail rule`. + --- ## See Also diff --git a/docs/investigation-engine/api-reference.md b/docs/investigation-engine/api-reference.md new file mode 100644 index 000000000..6e2f65dd5 --- /dev/null +++ b/docs/investigation-engine/api-reference.md @@ -0,0 +1,322 @@ +# API Reference + +!!! warning "Not a public API yet" + These routes are the interface that the [console](console.md), the [CLI](cli.md) + and the [MCP tools](mcp.md) use. They are an experimental, versioned surface: + they are **not part of the public OpenAPI specification** and carry no stability + promise yet. Routes, fields and limits can change before general availability. + Prefer the CLI or the MCP tools unless you need to call the routes directly. + +The routes are served by the LimaCharlie API gateway under + +```text +https://api.limacharlie.io/v1/investigate/{oid}/... +``` + +`{oid}` is the organization's UUID. Requests are authenticated with a LimaCharlie +JWT, like the rest of the API (see [Authentication Resource +Locator](../8-reference/authentication-resource-locator.md) and +[API keys](../7-administration/access/api-keys.md)). The gateway checks the +[permissions](permissions.md), forwards the request to the engine in the +organization's datacenter, and relays the engine's answer. + +!!! info "Pre-release" + The routes exist only on gateways where the Investigation Engine is enabled. On any + other gateway they return the same "not found" as an unknown path. On a gateway + where it is enabled, an organization whose datacenter has no engine, or an engine + that cannot be reached, gets the gateway's `503`, `502` or `504` (see + [Errors](#errors)), not a `404`. + +## Conventions + +- Bodies are `application/json`, uncompressed, at most 8 MiB, and must be a JSON + object. A query string is at most 8 KiB. +- Run, action and test ids are 1 to 128 characters of letters, digits, `.`, `_`, `:` + and `-`. +- Times in a run are RFC 3339 in UTC with millisecond precision. Other timestamps can + have more precision: parse them as dates, do not sort them as text. +- Ordinary **reads** are retried by the gateway. **Writes are sent exactly once**: a + write that times out may or may not have happened, so look before you repeat it, or + use an `idempotency_key`. The two long-polls (`GET /runs/{id}/events` and + `GET /test-plan/{test_id}`) are also sent exactly once; just poll again. +- Pages are `limit` (1 to 200, default 50) and an opaque `cursor`. A cursor names a + position, not an offset, so runs started between pages never repeat or hide one. + Send the cursor back with the same filters. Run events use `limit` up to 500 (default + 200), and the evidence graph pages with `offset` (`limit` up to 1000). +- An API key must have a key id; a key without one is refused on every route, reads + included, with `403`. + +### Errors + +A failed call returns an HTTP status and `{"error": ""}`. When the **engine** +refused the call, the body also has `"error_code": ""` from the table below; a +failure of the gateway itself has only `error`. A refused template or wave also +carries `errors` and `warnings` arrays (each with `code`, `path`, `message` and often +`suggestion`). + +Failures that only the gateway can produce, with no `error_code`: + +| Status | When | +|---|---| +| `401` | A required permission is missing (the message names it). Note: the gateway answers `401`, not the engine's `403`, for a missing permission. | +| `400` | A malformed id, query string, body or long-poll parameter. | +| `403` | The API key has no key id. | +| `413`, `414` | The body is over 8 MiB, or the query string over 8 KiB. | +| `415` | The body is not uncompressed `application/json`. | + +| `error_code` | Status | Meaning | +|---|---|---| +| `bad_request` | 400 | Malformed or invalid arguments. | +| `forbidden` | 403 | Not allowed by the engine: for example adding inputs to a run you did not start, or approving or denying as a user who is not on an action class's approver list. | +| `not_found` | 404 | The run, action or test does not exist (or the retention removed it). | +| `conflict` | 409 | The state does not allow it: the run has finished, or the action is not in the right state. | +| `too_many_runs` | 429 | The organization's concurrent run cap is reached. Retry later. | +| `unavailable` | 503 | Storage or a dependency is unavailable. Retry. | +| `not_implemented` | 501 | The feature is not enabled. | + +The gateway adds `503 the investigation engine is not available for this +organization`, `504 investigation engine timed out` and `502 investigation engine +unavailable`. + +Start, wave, test and compare routes are rate limited (per hour: 300 starts, 600 +waves, 300 tests, 600 comparisons). + +## Routes + +Permission shorthand: **read** is `investigation.get` + `insight.evt.get`; **write** is +`investigation.set` + `insight.evt.get`. See [Permissions](permissions.md). + +### Runs + +| Method and route | Permission | Purpose | +|---|---|---| +| `POST /runs` | write | Start a run. `201` for a new run, `200` when it joined an existing one. | +| `GET /runs` | read | One page of the organization's runs, newest first. | +| `GET /runs/{id}` | read | State, phase, verdict, proof, effective policy, usage, critical path. | +| `GET /runs/{id}/steps` | read | The step log with timing, cost, cancellations, the plan graph edges, the belief curve and the critical path. Query `values=true` includes values. | +| `GET /runs/{id}/graph` | read | The evidence graph, paged (`limit` up to 1000, `offset`). | +| `GET /runs/{id}/events` | read | Run events by long-poll. | +| `GET /runs/{id}/case` | read | The case document the run wrote. `404` until it has one. | +| `GET /runs/{id}/trace` | read | An anonymized trace of the run. | +| `GET /runs/{id}/labels` | read | Feedback labels recorded against the run. Query `fact`, `kind`, `all`. | +| `POST /runs/{id}/inputs` | write | Add evidence or a new alert to a running run (`inputs`, `detection_id`). Only the identity that started the run may. | +| `POST /runs/{id}/cancel` | write | Cancel. In-flight steps are cancelled. Body `revert_leases` reverts active leases. | +| `POST /runs/{id}/compare` | write | Re-evaluate the run's recorded facts under another template or a tighter policy. Calls no operator. The run must have finished (`409` otherwise). | +| `GET /stats` | read | Aggregates over the most recent runs of a window (`since`, `until`, `template`). | + +**Listing runs.** Filters are combined with AND and evaluated by the engine: `state` +(`queued`, `running`, `decided`, `completed`, `cancelled`, `failed`, `budget_exhausted`), +`verdict` (`true_positive`, `false_positive`, `inconclusive`, or `undecided`), +`template` (`inline` for an inline plan), `trigger` (`detection`, `entity`, `case`, +`question` or `message`), `entity`, `started_by`, `idempotency_key`, `since` and +`until`. A bad value is `400` with a message naming the parameter. + +**Events.** `GET /runs/{id}/events?after=N&wait=S&limit=L` returns as soon as an event +with `seq > after` exists, or after `wait` seconds (0 to 10) with none. The answer is +`{"events": [{"seq", "at", "type", "data"}], "next": N, "done": false}`: pass `next` as +the next `after`, and stop when `done` is true. Event data carries structure only, never +customer values. A client that needs only the final answer can instead set +`wait_seconds` on the start request. + +### Starting a run + +```json +{ + "trigger": {"detection_id": "...", "case_id": "...", "entity_id": "...", "question": "..."}, + "plan": {"template": "suspicious-process"}, + "urgency": "high", + "mode": "full", + "overrides": {}, + "strict": false, + "case": "create", + "coalesce_key": "sid:abc", + "idempotency_key": "...", + "notify": ["output:slack-soc"], + "wait_seconds": 0 +} +``` + +| Field | Meaning | +|---|---| +| `trigger` | At least one of `detection_id`, `case_id`, `entity_id`, `question`, or a non-empty `inputs` with an explicit `plan.template` (for example a message UUID or another operator input, with no alert). `category`, `priority`, `severity` and `tags` describe what fired, and select the policy records and the template. | +| `plan` | `{"template": name}` to run a stored template, or `{"inline": ""}` to run a template you supply (always a draft). Leave it out to let the engine choose by `applies_to`. | +| `inputs` | Named values for the template's facts. | +| `urgency` | `low`, `medium`, `high` or `critical`. Derived from the trigger if you omit it. | +| `mode` | `triage` (stop at the verdict), `full` (default), or `scope` (skip triage). | +| `overrides`, `strict` | Tighten the policy for this run. See [Policies](policies.md#per-run-overrides). | +| `case` | `create`, `attach:` or `none`. | +| `coalesce_key` | A start with the same key joins the open run (same organization, same starter and same run identity). Without a key nothing joins. | +| `idempotency_key` | A repeated start with the same key returns the same run. Scoped to your identity. | +| `identity` | `caller`, or `hive://secret/`; accepted only if the policy allows it for the template. | +| `notify` | `output:` destinations for this run's events. | +| `wait_seconds` | Block up to this long for the verdict, for a synchronous caller such as an agent turn. | +| `serial` | `true` starts the serial baseline for speed comparison: one step at a time, no speculation, forced to `case: none` and `suggest` autonomy. | + +The response is `{"run": {...}, "joined": false}`. A run object, abbreviated: + +```json +{ + "run_id": "...", + "state": "decided", + "phase": ["scoping", "containment"], + "template": "suspicious-process", + "template_version": 2, + "template_by": "request", + "verdict": { + "label": "true_positive", + "p": 0.97, + "proof": {"rule": "cmd_malicious AND rare_egress", "facts": ["f12", "f31"]}, + "unknowns": [{"fact": "f40", "reason": "outside_free_window"}] + }, + "actions": [{"id": "a1", "class": "contain.network", "target": "...", "state": "active", "lease_until": "2026-11-10T21:00:00.000Z"}], + "policy": {"applied": ["default"], "clamped": []}, + "usage": {"steps": 61, "cancelled": 14, "search_events": 18200000, "llm_usd": 0.08, "wall_ms": 41000}, + "case_id": "...", + "case_final": false, + "started_by": {"identity": "...", "actor": "..."} +} +``` + +Wait for `case_final` (not just `case_id`) when you need the finished case: `case_id` +appears with the first write, `case_final` once the write made after the run ended has +landed. It stays false when no final write exists, such as with `case: none`. + +### Actions and autonomy + +| Method and route | Permission | Purpose | +|---|---|---| +| `GET /actions` | read | The approvals inbox: one page of actions across all runs, awaiting approval first. Query `state` (one or a comma-separated list), `class`, `run_id`, `limit`, `cursor`. | +| `POST /runs/{id}/actions/{aid}/{verb}` | write + the action class's permission | `verb` is `approve`, `deny`, `revert`, `extend` or `confirm`. Body: `note` (deny, revert), `seconds` (extend). | +| `GET /autonomy` | read | `{"paused": false}` or `{"paused": true, "by": "...", "at": "..."}`. | +| `POST /autonomy/pause`, `POST /autonomy/resume` | `org.conf.set` only | Stop or allow every automatic action in the organization. Answers with the same shape as the read. | + +Action states, in the order the inbox lists them: `awaiting_approval`, `proposed`, +`uncertain`, `executing`, `reverting`, `active`, `confirmed`, `reverted`, `failed`, +`denied`, `expired`. Within a state the most recently changed action is first. + +### Templates, policy and authoring + +| Method and route | Permission | Purpose | +|---|---|---| +| `GET /templates` | read | The organization's templates: `name`, `version`, `description`, `applies_to`, `status`. | +| `POST /plans/validate` | read | Body `{"yaml": "..."}`. Type-checks a template against the organization's operator catalog. | +| `GET /operators` | read | The operator catalog with metadata, typed `args` and `output`, and availability under your policy. | +| `GET /packs` | read | The fact packs. Query `name` for one. | +| `GET /authoring-kit` | read | Everything needed to write a template. Query `sections`. | +| `POST /test-plan` | write | Dry run of a template on past alerts. | +| `GET /test-plan/{test_id}` | read | Poll a test. Query `after`, `wait` (0 to 10). | +| `GET /policy/effective` | read | The effective policy, record by record. Query `category`, `priority`, `severity`, `target_tags`, `class` and `template` (the template whose run identity to explain). | + +Templates and policies themselves are written through the +[Config Hive](../7-administration/config-hive/index.md) routes for the +`investigation_plan` and `investigation_policy` types. + +#### Validate + +`POST /plans/validate` answers with `valid`, `errors[]` and `warnings[]` (each +`{code, path, message, suggestion?}`), `worst_case_steps`, `cost_class`, +`free_window_compliant`, `steps_by_class`, and an `estimate`: + +```json +"estimate": {"steps": 14, "search_steps": 3, "llm_steps": 1, "billed_steps": 0, "wall_seconds": 95, "exceeds": ["max_steps_per_run"]} +``` + +#### Authoring + +`GET /authoring-kit?sections=guide,schema,codes` returns the parts you ask for. Sections +are `schema`, `guide`, `operators`, `examples`, `policy`, `codes`, `packs` and `library`. +Without `sections`, everything except `library` is returned. The answer reports its own +size. + +| Section | Content | +|---|---| +| `schema` | The JSON Schema of the template YAML, generated from the engine's types. | +| `guide` | The authoring guide. | +| `operators` | The typed operator catalog and what your policy allows. | +| `examples` | Templates of the default library, as models (at most 8). | +| `policy` | The policy as it shapes a plan: budgets, denied operators, allowed extension requests, each action class's mode. | +| `codes` | Every validation code with its meaning and fix. | +| `packs` | The fact packs. | +| `library` | The whole [default library](templates.md#the-default-library): per entry `name`, `description`, `version`, `status`, `applies_to` and the `yaml` to store. Asked for by name only. | + +#### Test a template on past alerts + +`POST /test-plan` takes a body like: + +```json +{ + "template": "my-plan", + "samples": [ + {"detection_id": "..."}, + {"case_id": "..."}, + {"detect_cat": "mailsec-ai-triage", "last": 10}, + {"inputs": {"msg_uuid": "..."}, "expected": "false_positive"} + ], + "overrides": {"budgets": {"max_wall_seconds": 120}}, + "mode": "triage", + "wait_seconds": 5 +} +``` + +Give **exactly one** of `template` (a stored template, draft or active) and `inline` (a +template as YAML). It answers `202` with a `test_id` while the test runs, and `200` when +it is over, or when the template is invalid (`state: "invalid"` with the validation +report, and nothing started). Poll `GET /test-plan/{test_id}?after=&wait=10`. +Each sample result has `verdict`, `proof`, `unknowns`, `steps`, `cost`, `wall_ms`, +`would_propose`, and, for an alert in a closed classified case, `expected` and +`agreement` (`agree`, `disagree`, `undecided` or `none`). The summary has the counts, an +`agreement_rate`, the most frequent unknown reasons and the cost. See [Test on past +alerts](templates.md#test-on-past-alerts) for the semantics and limits. + +A test never writes a run, case, label, action or audit event; it is not in any run +list, and the result is kept for 24 hours. + +### Waves + +`POST /waves` runs a batch of operator calls now and returns compact answers. There is +no template, no verdict and no case. It runs with **your** token and counts against the +organization's concurrent run cap, step cap and wall time. + +```json +{ + "steps": [ + {"id": "alert", "use": "detection_context", "args": {"detection_id": ""}, + "select": ["sid", "atom", "hash", "command_line", "same_rule_count"]}, + {"id": "host", "use": "host_context", "args": {"sid": "$alert__trigger.sid"}, "select": ["hostname", "owner_name"]}, + {"id": "spread", "use": "hash_spread", "args": {"hash": "$alert__trigger.hash"}, "select": ["hosts_30d", "seen"]} + ], + "stop_when": ["spread__prevalence.hosts_30d >= 500 and alert__same_rule.count >= 50"], + "deadline_ms": 20000 +} +``` + +| Field | Meaning | +|---|---| +| `steps` | Each step has an `id` and exactly one of `op` (an operator) or `use` (a fact pack), plus `args`. A string argument that is a whole reference such as `"$alert__trigger.sid"` reads another step's value and makes the step wait for it. | +| `select` | Field paths of the value to return (`matches[0].entity_id`, `matches[*].entity_id`). For a pack step, the names of exports. | +| `limit` | Per step, 1 to 1000, default 10: the most list elements returned. | +| `stop_when` | Up to 8 conditions with the same rules as proofs. The first one that is true cancels the steps still running. An unknown step never makes one true. An incomplete step can make a condition true when the claim only gets stronger with more data (`machine_count > 3`), but never a claim of absence or of an exact count. Conditions are evaluated on the full results, before the answer is compacted: a step returned as `complete: false` because its list was cut may already have stopped the wave. | +| `deadline_ms` | Default 30000, capped by the policy's wall-time budget. | +| `inputs` | Named values steps and conditions may read as `$name`. | +| `full` | `true` returns raw values instead of compact ones. | +| `validate_only` | Validate and return the expanded steps without running anything. | + +The answer has one result per step (`id`, `op` or `pack`, `status`, `reason`, +`complete`, `ms`, `value`), plus `stopped_by`, `usage` and `compact`. A step's `status` +is `ok`, `unknown` (with a `reason` such as `timeout`, `forbidden`, `dependency_unknown`, +`missing_input` or `policy_denied`), `cancelled` (only because `stop_when` fired) or, +for a pack, `partial`. + +- **A cut list is not a complete answer.** When a list was cut at `limit`, the step is + `complete: false` with reason `truncated`: it is a lower bound, so do not count it or + claim something is absent from it. +- **Unknown is never absence.** An unknown step established nothing. +- An invalid wave is refused before anything runs, with `400` and the coded `errors`. + Fix them mechanically and send it again. +- Steps of an earlier wave are not available to a later one: copy the values you need. + +## See also + +- [MCP Tools](mcp.md): the same operations as tools for an AI assistant. +- [CLI](cli.md): the `limacharlie investigate` commands. +- [Concepts](concepts.md) and [Plan Templates](templates.md). diff --git a/docs/investigation-engine/automation.md b/docs/investigation-engine/automation.md new file mode 100644 index 000000000..d27bbe339 --- /dev/null +++ b/docs/investigation-engine/automation.md @@ -0,0 +1,127 @@ +# Automatic Starts + +Runs usually start from a person, an API call or an AI agent. Two entry points start +them **without a person**: a D&R rule, and a new case. A run without a person has no +token of its own, so it executes with the identity named in the organization's +[policy](policies.md#run-identity), and it can only use **active** templates. + +## Before you enable either + +1. **A policy identity.** Set `identity.default` in the `default` investigation policy + to a Hive secret holding an organization API key. This is the simplest setup. A + template can instead carry its own `identity.templates..default`, so a + least-privilege key serves just that template. Without an identity for the template, + the automatic start is refused with a message that names `identity.default` and + the per-template setting. A case trigger with `template: auto` also needs the + org-wide `identity.default`: it reads the case's tags and first detection before + a template is chosen. +2. **At least one active template** whose `applies_to` matches what you expect, or a + rule that names its template. A draft is never used. +3. **Autonomy you are comfortable with.** Start with every class in `suggest`. See + [Policies and Autonomy](policies.md). + +## D&R rules: `start investigation` + +The [`start investigation`](../8-reference/response-actions.md#start-investigation) +response action starts a run for the detection the rule just reported. + +```yaml +detect: + event: NEW_PROCESS + op: ends with + path: event/FILE_PATH + value: powershell.exe +respond: + - action: report + name: suspicious-powershell + - action: start investigation + template: suspicious-process + urgency: high + coalesce_key: "sid:{{ .routing.sid }}" +``` + +- Put the action **after** `report`. The run is tied to the detection `report` just + created, and the detection id is the run's idempotency key: one detection starts one + run (idempotency is kept for the engine's record retention and is scoped to the + organization and the D&R starter). +- The run's trigger carries the detection's category and priority and the rule's tags, + with `source: dr-rule` and `rule: general.`. Those are what + `applies_to` matches on. A rule never sends a severity. +- With no `template`, the engine picks the **active** template whose `applies_to` best + matches. With no match, no run starts and the detection is still reported. +- The action sends only what you list under `inputs`, plus the detection id. A template + that takes `detection_id` and derives everything else works unchanged. See + [Run inputs](templates.md#run-inputs). +- Starting a run never delays the rule: the request is sent in the background and a + failure never blocks the detection. +- **The detection may not be stored yet when the run starts.** `report` and `start + investigation` run independently, so the run can begin within milliseconds of the + detection, before it can be read. For a run started by a D&R rule or a case, the + engine therefore treats "not found" on the trigger detection as "not indexed yet": + it retries that one lookup with backoff (from 500 ms up to 5 s between tries) for up + to about 30 seconds in all (the engine setting `INVESTIGATE_TRIGGER_DETECTION_WAIT`; + `0` turns the wait off). Other facts that do not depend on the detection keep running + meanwhile, and a cancel or the run's own time budget ends the wait. If the detection + never appears, the fact is `unknown` with the reason `not_found`, never absent, and + the step record shows how long it waited. A lookup of any other detection is never + retried, and a person's start does not wait. + +The full parameter list, limits and behavior are in the [response actions +reference](../8-reference/response-actions.md#start-investigation). + +!!! tip "Pick `coalesce_key` on purpose" + Alerts with the same `coalesce_key` join the run that is already open instead of + starting another. The default, `sid:`, makes a burst of alerts on one + host one investigation. A key built from a field some events lack renders the same + text for all of them and joins everything into one run. + +## The case trigger + +A new case that somebody else opens can start a run **in that case**. The engine reads +case events from the case audit feed, so there is no polling and nothing to set up per +organization. It is configured in the `default` investigation policy: + +```yaml +# investigation_policy/default +triggers: + case_created: + severities: [high, critical] # required: which severities start a run + tags: [phishing] # optional: the case must carry at least one of these + template: auto # a template name, or auto (chosen by applies_to) +``` + +| Field | Meaning | +|---|---| +| `severities` | One or more of `critical`, `high`, `medium`, `low`, `info`. Required, unless `enabled: false`. | +| `tags` | Optional. A case starts a run only if it carries any one of these tags. | +| `template` | A template name, or `auto` (the default) to choose the active template whose `applies_to` matches the case (`severity`, `tag`, `detect_cat`, `source`). Only the `default` record may set it. | +| `enabled` | Set `false` to switch the trigger off. | + +Behavior: + +- **The run attaches to that case.** It never creates a second case. It writes its + entities, telemetry references, findings and a proposed classification into the case. + The case is the human record; the run is the machine record. +- **One case starts at most one run**, however many times the event is delivered + (within the engine's idempotency retention). +- When the case already has a detection, the same short wait for that detection to be + stored applies as for a D&R rule (see above). +- **Cases the engine wrote never start a run**, so it cannot trigger itself. +- The run's inputs are the case id and, if the case already has a detection, the + detection id of its first one. +- **Other policy records can only narrow it.** A matching record can intersect the + severities and tags or set `enabled: false`. It cannot enable the trigger, widen it, + or name the template. +- Only an **active** template can start from a case. If nothing applies, or there is no + policy identity, the event is acknowledged and logged with the reason, and no run + starts. +- A case is created before the rules and people that tag it act. The tags are read a few + seconds after creation, so a tag filter sees the tags the case had then. If you need + later tags, filter on severity instead. + +## What you see + +A run started this way appears in the Runs list like any other, with **Started by** +shown as **Automation**. Its case has the run linked, and its actions follow the policy. +If the policy asks for notifications, [events](policies.md#notifications) are written to +the organization's audit log. diff --git a/docs/investigation-engine/build-templates-with-ai.md b/docs/investigation-engine/build-templates-with-ai.md new file mode 100644 index 000000000..c6a7a24f0 --- /dev/null +++ b/docs/investigation-engine/build-templates-with-ai.md @@ -0,0 +1,325 @@ +# Build and Tune Templates with an AI Agent + +An AI agent is good at the mechanical part of writing a plan template: reading the +operator catalog, drafting YAML, fixing validation errors and reading test results. +It is a poor judge of whether a plan is *right* for your environment. This guide +splits the work accordingly: **the agent drafts, validates, tests and saves a draft; +a person reads the evidence and promotes it.** + +The workflow: + +1. Read the authoring kit. +2. Look at real alerts of the kind you are covering. +3. Draft a small template. +4. Validate until there are no errors, then read the warnings. +5. Test on past alerts. +6. Refine and test again. +7. Save as a draft. +8. A person reviews and promotes it. + +The agent cannot promote a template. The save tool always writes `status: draft` and +refuses to overwrite an active template, and the Hive refuses an `active` write that +comes from an API key (unless the organization opted out; see +[Permissions](permissions.md#active-templates-can-only-be-written-by-a-person)). + +## Set up the agent + +The agent needs the engine's tools and a credential that can use them. + +**With the MCP server.** Use the `investigation_engine` profile, which holds the +ten investigation tools, or `investigation_engine_readonly` if the agent should only +read, validate and look up (no start, wave, test or save). See [MCP Tools](mcp.md) +for the tool list and how to connect. + +**With the CLI.** The `limacharlie investigate` command group offers the same +operations. See [CLI](cli.md). + +Give the agent's API key only what the workflow needs: + +| To do this | The key needs | +|---|---| +| Read the kit, operators, packs and templates, validate a template | `investigation.get`, `insight.evt.get` | +| Test a template on past alerts | `investigation.set`, `investigation.get`, `insight.evt.get` (starting the test needs the first, polling its result the second) | +| Start a run or a wave | `investigation.set`, `insight.evt.get` | +| Save a draft | `investigation_plan.get`, `investigation_plan.set`, and `investigation.get`, `insight.evt.get` | + +The two `investigation_plan.*` permissions are not part of the Owner role. Grant them +to the key explicitly. A test reads the data the key can read, so a key that cannot +read a detection or its case reports those samples as unreadable, not as failures. A +run the key starts executes as the policy identity unless it picks `caller` (see [Run +identity](policies.md#run-identity)). + +!!! warning "Do not give the agent key any `investigation_policy.*` permission" + The opt-out tag `allow-api-key-promotion` is set on a policy record with + `investigation_policy.set.mtd`. A key that holds any broad `investigation_policy.*` + grant could set it, turn off the promotion gate and then promote its own drafts. The + agent only needs the `investigation_plan.*` permissions above. + +!!! tip "Keep the key away from containment" + The agent saves and tests templates. It does not need `sensor.task` or + `mailsec.act`, which approving actions requires. Withholding them does not stop + a run: a run the key starts on an **active** template acts as the policy identity, + not as the key. Do not give the key `investigation.set` unless it should be able to + start runs. + +### Or let the console start the agent + +The console can start this workflow for you. **Draft with AI** (on the Templates tab, +and on a detection) starts an AI session with the *Investigation plan author* agent from +the AI agent catalog. Describe the alerts in plain words, or start from a detection as +the example alert. The agent follows the steps below and saves a draft that appears in +the Templates list. See [Console](console.md#draft-a-template-with-ai). + +## 1. Read the kit, in sections + +The authoring kit is the one call that gives an agent everything it needs. It is +large (about 33,000 tokens as a whole), so fetch it in parts: + +| Call | Sections | Size | Why | +|---|---|---|---| +| First | `guide`, `schema`, `codes` | about 13k tokens | How to write a plan, the template JSON Schema, every validation code. | +| Second | `operators` | about 14k tokens | What each operator takes and returns, and which ones your policy denies (`available: false`). Never guess a field name. | +| As needed | `examples` or `library` | varies | Real templates to copy a shape from. | +| As needed | `policy` | small | The budgets a plan must fit and each action class's autonomy mode. Worth fetching when you plan `respond` steps. | +| For one-off questions | `packs` | small | The fact packs. | + +Ask the agent to read the guide first. The guide the kit returns is the same text +the engine's tests check its examples against. + +## 2. Look at real alerts + +Before writing, have the agent list the detections of the category you want to +cover, their cases, and how analysts classified them. A template written from a +handful of real alerts is far better than one written from the category name alone. +Note which fields the detections actually carry. + +## 3. Draft a small template + +Start with three to six facts and one or two proofs on each side. Start from the +`detection_id` input and derive everything else with facts, as described in +[Run inputs](templates.md#run-inputs). + +This first draft triages a `rare-process` detection from what else fired on the +same host in the last week and an LLM judgement of that activity: + +```yaml +status: draft +version: 1 +description: Triage a rare-process detection from what else fired on the host in the last week and a judgement of that activity. +applies_to: + detect_cat: ['rare-process'] +prior: 0.25 +inputs: + detection_id: '{{ .detection.detect_id }}' +facts: + trigger: + op: detection.get + args: {detect_id: $detection_id, window: [-29d, now]} + host: + op: entity.resolve + args: {type: sensor_id, value: $trigger.sid} + related: + op: detection.list + args: {sids: [$trigger.sid], window: [-7d, now]} + judge: + op: llm.judge + args: {question: host_activity_malicious, data: {detections_last_week: '$related.count', host: '$host.display_name'}} +proofs: + false_positive: + - 'related.count <= 1 and judge.label == "no"' + true_positive: + - all: ['judge.label == "yes"', 'related.count >= 5'] +evidence: + - {id: busy_host, when: 'related.count >= 5', weight: 1.0} +respond: + - {class: contain.network, target: '$trigger.sid', when: 'verdict == true_positive'} +report: + case: none +``` + +Things to notice: + +- `trigger`, then `host` and `related` run **in parallel** once `trigger` answers. + `judge` waits for both. +- The `llm.judge` fact is the most expensive step. It gets only the two values it + needs, not whole records. +- Both proofs need `judge`. If it is unknown, neither completes, and the run can + only end on evidence or `inconclusive`. That is the safe behavior. +- `respond` only proposes isolating the sensor from the network, and only once the verdict + is a decided true positive. Because this is a draft, the proposal stays a suggestion. +- `report.case: none` keeps test noise out of your cases while you tune. + +## 4. Validate + +Validation is cheap and stores nothing. Call it until `errors` is empty. Every issue +has a code, a path and often a suggestion. Here is the report for a copy of the next +version of this template with two mistakes, a misspelled field and a window that +starts 45 days ago: + +```text +errors: + - code: window_outside_free_range + path: facts.related.args.window + message: window starts earlier than 30 days ago: searches must stay inside the free time range + - code: unknown_field + path: proofs.false_positive[1] + message: 'prev.hosts_30: the fact has no field "hosts_30" (did you mean "hosts_30d"?)' + suggestion: hosts_30d +``` + +Both are mechanical to fix: use the suggestion, and start the window within 30 days. +Then read the **warnings**: they are what makes a template useless without making it +invalid. Typical ones are `unused_fact`, `operator_denied`, `proof_never_completes` +and `respond_class_off`. The report also estimates the worst case of one run, so +you can see early whether it fits your budgets. + +## 5. Test on past alerts + +Test with 10 to 25 samples that include closed cases. A good mix is a few +detections whose cases were classified true positive, a few false positive, and a +few with no case at all. Use `{detect_cat, last: N}` to take the newest +detections of the category, and `{case_id}` for specific closed cases. + +The test returns a result per sample: the verdict, the proof, what stayed unknown +and why, the cost, the time, and, for detections in a classified closed case, the +expected label and whether the template **agreed**. Then it returns a summary. + +Read it with the table in [Test on past alerts](templates.md#test-on-past-alerts). +Do not optimize for agreement alone: + +- A template that is `undecided` on a sample where the data is missing is behaving + correctly. Inconclusive is a good answer. +- Agreement that comes only from evidence, never from a proof, is slow and weak. +- A single `disagree` is worth reading closely. Which fact misled the proof? + +## 6. Refine and test again + +Say the first test shows the binary is common across the fleet for most +false-positive samples, but the template cannot say so. Add the prevalence fact, a +second false-positive alternative and an evidence rule that favors a false positive +for a common binary: + +```yaml +status: draft +version: 2 +description: Triage a rare-process detection from what else fired on the host, how common its binary is, and a judgement of that activity. +applies_to: + detect_cat: ['rare-process'] +prior: 0.25 +inputs: + detection_id: '{{ .detection.detect_id }}' +facts: + trigger: + op: detection.get + args: {detect_id: $detection_id, window: [-29d, now]} + host: + op: entity.resolve + args: {type: sensor_id, value: $trigger.sid} + related: + op: detection.list + args: {sids: [$trigger.sid], window: [-7d, now]} + prev: + op: ioc.prevalence + args: {file_hash: $trigger.hash} + judge: + op: llm.judge + args: {question: host_activity_malicious, data: {detections_last_week: '$related.count', host: '$host.display_name'}} +proofs: + false_positive: + - 'related.count <= 1 and judge.label == "no"' + - 'prev.hosts_30d >= 50 and judge.label == "no"' + true_positive: + - all: ['judge.label == "yes"', 'related.count >= 5'] +evidence: + - {id: busy_host, when: 'related.count >= 5', weight: 1.0} + - {id: common_binary, when: 'prev.hosts_30d >= 50', weight: -1.0} +respond: + - {class: contain.network, target: '$trigger.sid', when: 'verdict == true_positive'} +report: + case: none +``` + +Bump `version` for each meaningful change so you can tell test runs apart. Validate, +then test the **same samples** again. Comparing the same samples between versions is +what shows whether a change helped. + +Keep these in mind while refining: + +- Count a claim of absence ("no other detection", "zero hosts") as unknown when the + window ends in the last few minutes. `ioc.prevalence` is a lagging index: a zero + there is incomplete, so never let it complete a proof of safety. +- Do not write a proof that turns missing data into a verdict. +- Keep evidence weights modest. A weight so large that it overrides every proof + defeats the proofs. +- Drop a fact no proof or evidence rule reads (`unused_fact`). +- Judge with `llm.judge` only what an indexed read cannot answer exactly. + +## 7. Save as a draft + +When the template validates cleanly and the test results are acceptable, the agent +saves it as a draft. The save tool does four things: + +- always writes `status: draft` and adds the tag `ai-draft`; +- stores a short test summary (up to 2,000 characters) as the record's comment, so + the reviewer sees what was tested; +- refuses to save a template that does not validate; +- refuses to overwrite an **active** template. An existing draft is replaced only + when the agent passes `overwrite`. Save an improved version of an active template + under a new name. + +!!! note "The agent cannot activate" + A template that says `status: active` is refused by the save tool. Leave + `status` out. + +Ask the agent to end with a summary a reviewer can read in a minute: what the +template covers, how many samples it ran on, how it agreed with analysts, what stayed +unknown and why, what it would propose, and what it costs per run. + +## 8. A person promotes + +A person opens the draft in the [console](console.md#templates), reads the template +and the test summary, tests it once more if needed, and sets `status: active`. At that +point: + +- a D&R rule or the case trigger can start it; +- `applies_to` can pick it automatically; +- it acts within the [autonomy policy](policies.md), which is a separate decision. + +Promote in two steps for anything that proposes containment. First activate the +template with the relevant action classes at `suggest` in the policy, and watch what +it proposes. Raise a class to `approve`, and only later to `auto`, when you trust it. + +## A prompt to start from + +```text +You are writing an Investigation Engine plan template for detections of category +"rare-process" in this organization. + +1. Fetch the authoring kit with sections guide, schema and codes, then operators. +2. List recent detections of this category and the cases they ended up in. Tell me + how many were classified true positive and false positive. +3. Draft a SMALL template: 3 to 6 facts, take detection_id as the only input, and + derive everything else with facts. +4. Validate it until there are no errors, then fix the warnings that matter. +5. Test it on 15 to 25 samples that include closed cases of both classifications. + Report agreement, the unknown reasons, and cost. +6. Refine at most three times, testing the same samples each time. +7. Save the best version as a draft. You cannot activate it. +8. Finish with a summary for the person who will promote it. + +Rules: unknown is never false; never write a proof that depends on a fact that +can be incomplete to prove an absence; keep every window within 30 days; never +guess a field name, read it from the operator catalog. +``` + +## Common mistakes + +| Mistake | What happens | +|---|---| +| Copying the `inputs:` block of an example and reading `$hash`, `$sid` | The trigger does not provide it. Every fact that reads it is `unknown(missing_input)`. Derive it from `detection.get`. | +| Proving absence on incomplete data | The proof never completes. Use a positive condition, or accept `inconclusive`. | +| A signal with only a `malicious` or only a `benign` condition, then a proof on `signals.all_explained` | The signal can never be explained, so the proof never completes. | +| Guessing a field name | Validation fails with `unknown_field` and a suggestion. | +| A window older than 30 days | Validation fails with `window_outside_free_range`. | +| Proposing a class the policy has `off` | The action is never proposed (`respond_class_off`). | +| Writing a template for a one-off question | Use a [wave](concepts.md#waves-and-fact-packs). Write a template only when the same alert comes back. | diff --git a/docs/investigation-engine/cli.md b/docs/investigation-engine/cli.md new file mode 100644 index 000000000..16294e955 --- /dev/null +++ b/docs/investigation-engine/cli.md @@ -0,0 +1,198 @@ +# Command Line Interface + +The `limacharlie investigate` command group is the CLI for the Investigation Engine: +build and test templates, start and follow runs, work the approvals inbox, and manage +the pause switch. + +!!! info "Pre-release" + The `investigate` group ships with a CLI release that follows the Investigation + Engine pre-release. Check that your CLI has it, and see the exact flags of your + version, with: + + ```bash + limacharlie investigate --help + limacharlie investigate templates test --help + ``` + + Every command and subgroup also answers `--ai-help` with task-oriented guidance. + +Commands take the global options (`--oid`, `--output json|yaml|toon|csv|table|jsonl`, +`--filter `, `--fields `). Use the organization's UUID for `--oid`. +The permissions each command needs are the ones of the corresponding +[API route](api-reference.md#routes). + +A command that completes but says "no" (an invalid template or wave; a save the CLI +itself refuses because the template does not validate, a record of that name exists, or +it would replace an active template; a promotion that did not happen because the +template is already active or does not validate) prints the engine's coded answer and **exits with code 4**. A call that fails +keeps the CLI's standard exit codes: **2** when the API answers 401 or 403 +(authentication or permission), **3** for a resource that does not exist, **4** for +invalid input, **5** for a rate limit, and **1** for anything else: a timeout, a test +that failed, an `install-defaults` that could not write a template, and every other +error status. That includes the Hive's own refusals, which the CLI receives as an HTTP +`400` carrying an `error_code`: a promotion or save the Hive rejects (for instance an +`active` write from an API key, `error_code` `INVALID`) exits **1**, not 4. A missing +Hive permission is also a `400` from the Hive (`error_code` `UNAUTHORIZED`) and exits +**1**; the CLI then writes `"status": 403` in the error document, which is a label for +the permission failure, not the HTTP status it received. Only a real HTTP 401 or 403 +exits 2. Read the `error_code`, not the exit code, to tell these apart. With a structured +`--output` (`json`, `yaml`, ...) the failure is also written to stderr as a document +with the message, the HTTP `status`, the `error_code` when there is one, and any coded +`errors`; a 404 without an `error_code` means the Investigation Engine is not enabled +on that API host. + +```bash +export OID= +``` + +## Templates + +```bash +# What the organization has +limacharlie investigate templates list --oid $OID +limacharlie investigate templates list --status active --oid $OID +limacharlie investigate templates get suspicious-process --yaml --oid $OID + +# What the default library offers (it is not installed until you install it) +limacharlie investigate kit --section library --oid $OID + +# Install the default library (written as drafts unless you say otherwise) +limacharlie investigate templates install-defaults --oid $OID +limacharlie investigate templates install-defaults --name suspicious-process --oid $OID +``` + +| Command | What it does | +|---|---| +| `templates list [--status draft\|active]` | The organization's templates: name, version, description, `applies_to`, status. | +| `templates get NAME [--yaml]` | A stored template record. `--yaml` prints only the source. | +| `templates validate -f FILE` | Checks a template (`-` for stdin) and runs nothing. Exit 4 when it has errors. | +| `templates test [NAME] -f FILE ...` | Dry run on past alerts. See below. | +| `templates save NAME -f FILE` | Validates, then saves a **draft** record. Exit 4 when the CLI refuses it; exit 1 when the Hive does (see above). | +| `templates promote NAME --confirm` | Makes a reviewed draft **active**. A person's decision. | +| `templates delete NAME --confirm` | Deletes the record. | +| `templates install-defaults` | Writes the [default library](templates.md#the-default-library) into the organization. | + +**`templates install-defaults`** never overwrites silently: a template that already +exists is skipped unless you pass `--overwrite`. `--name` (repeatable) installs only +the templates you name. `--status` is the status the templates are written with, +whatever the library says; the CLI default is `draft`, so review and promote what you +want to run on its own. + +**`templates save`** writes a draft. `--comment` stores what the template was tested on +and how it did as the record's comment, `--tag` (repeatable) adds tags, and +`--overwrite` replaces an existing draft. An AI agent leaves `--status` at its default, +`draft`. Writing an active template is a person's decision, and by default the Hive +refuses it from an API key (see [Permissions](permissions.md#active-templates-can-only-be-written-by-a-person)). + +**`templates promote`** requires `--confirm`. It writes `status: active`, which the Hive +refuses by default when the CLI is authenticated with an **API key** (unless an +administrator opted out; see [Permissions](permissions.md#active-templates-can-only-be-written-by-a-person)). Run it with a user login +(`limacharlie auth login`), or have a person click **Make active** in the console. +The same applies to `templates save --status active` and +`templates install-defaults --status active`. + +### Testing a template on past alerts + +```bash +# The newest 15 detections of a category +limacharlie investigate templates test -f plan.yaml \ + --detect-cat rare-process --last 15 --oid $OID + +# Specific detections and a closed case (its classification is the expected answer) +limacharlie investigate templates test -f plan.yaml \ + --detection-id --case-id --oid $OID + +# A stored template of your own, with a free-form sample +limacharlie investigate templates test my-template \ + --sample '{"inputs": {"msg_uuid": "..."}, "expected": "false_positive"}' --oid $OID +``` + +| Flag | Meaning | +|---|---| +| `NAME` or `-f FILE` | A stored template, or a template file (`-` for stdin). | +| `--detection-id`, `--case-id` | A sample (repeatable). | +| `--detect-cat` with `--last N` | The newest N alerts of the category (1 to 25, default 10). | +| `--sample JSON`, `--samples-file FILE` | Free samples. At most 25 alerts in all. | +| `--mode triage\|full` | `triage` (the default, the cheap pass) or `full`. | +| `--strict` | Refuse the test, instead of reducing the value, when an override in `--overrides-json` would loosen the policy. Without overrides it changes nothing. It does not fail the test because of unknown facts; read the unknowns in the result. | +| `--overrides-json JSON` | Tighten-only overrides, such as `{"budgets": {"max_wall_seconds": 120}}`. | +| `--wait SECONDS`, `--timeout SECONDS`, `--no-wait` | How long the start call and the whole command wait. `--no-wait` prints the `test_id` and returns. | +| `--test-id ID` | Keep waiting on a test started earlier. | +| `--include-steps` | Keep every step of every sample (large). By default only the steps that did not succeed. | + +The result is the one described in [Test on past alerts](templates.md#test-on-past-alerts): +per sample the verdict, proof, unknowns, cost and agreement with the case, and a summary. + +## Authoring helpers + +```bash +limacharlie investigate kit --section guide --section schema --section codes --oid $OID +limacharlie investigate operators --match mailsec --brief --oid $OID +limacharlie investigate packs --brief --oid $OID +``` + +| Command | What it does | +|---|---| +| `kit [--section S]...` | The authoring kit. Sections: `schema`, `guide`, `operators`, `examples`, `policy`, `codes`, `packs`, `library`. Without `--section`, everything but `library` (about 33,000 tokens). | +| `operators [--match TEXT] [--name NAME]... [--brief]` | The typed operator catalog. | +| `packs [--name NAME] [--brief]` | The fact packs. `--brief` is the compact index. | + +## Waves + +```bash +limacharlie investigate wave -f wave.yaml --oid $OID +limacharlie investigate wave -f wave.yaml --validate-only --oid $OID +``` + +`wave.yaml` (or JSON) has the fields of the [wave request](api-reference.md#waves): +`steps`, `stop_when`, `deadline_ms`, `inputs`. The flags `--stop-when` (repeatable, up to +8), `--deadline-ms` and `--input KEY=VALUE` add to the file, `--full` returns raw +values, and `--validate-only` validates and runs nothing. + +## Runs + +```bash +limacharlie investigate runs start --template suspicious-process \ + --detection-id --wait 30 --oid $OID +limacharlie investigate runs list --state running --oid $OID +limacharlie investigate runs events --follow --oid $OID +limacharlie investigate runs case --oid $OID +``` + +| Command | What it does | +|---|---| +| `runs start` | Starts a run on `--detection-id`, `--case-id`, `--entity-id` or `--question`, with exactly one of `--template NAME`, a template file `-f`, or `--auto` (sends no template: the engine picks the active template whose `applies_to` matches the trigger, and refuses with a reason when none does). `--category`, `--severity`, `--priority` and `--tag` (repeatable) describe the alert: they feed `--auto` and the run-scoped policy records, and `--severity` and `--priority` set the default urgency. `--urgency low\|medium\|high\|critical` overrides it. `--input KEY=VALUE` and `--inputs-json` give run inputs; `--mode triage\|full\|scope` (default `full`); `--case-mode create\|none\|attach:`; `--idempotency-key`; `--wait SECONDS` (up to 60) to block for the verdict; `--follow` to stream events until the run ends. | +| `runs get RUN_ID` | State, phase, verdict, actions and usage. | +| `runs list` | The organization's runs, newest first. Filters `--state`, `--verdict`, `--template`, `--trigger`, `--entity`, `--started-by`, `--idempotency-key`, `--since`, `--until`; `--limit` and `--cursor` page; `--all` follows every page. | +| `runs events RUN_ID` | The run's events after `--after`. `--follow` keeps polling until the run is over, one event per line. It exits 0 when the run finished and 1 when `--timeout` is reached. | +| `runs steps RUN_ID [--values]` | The step log, plan graph, usage and belief curve. `--values` includes customer values; by default the output is structure only. | +| `runs case RUN_ID` | The case document the run wrote. | +| `runs cancel RUN_ID [--revert-leases]` | Cancels a run, and optionally reverts the leases of its active actions. | + +A run started with a draft template runs with autonomy capped at `suggest`. + +## Approvals and the pause switch + +```bash +limacharlie investigate actions list --state awaiting_approval --oid $OID +limacharlie investigate actions approve --oid $OID +limacharlie investigate actions deny --note "Expected maintenance" --oid $OID + +limacharlie investigate policy get --category rare-process --priority 5 --oid $OID +limacharlie investigate policy status --oid $OID +limacharlie investigate policy pause --oid $OID +limacharlie investigate policy resume --oid $OID +``` + +| Command | What it does | Needs | +|---|---|---| +| `actions list [--state S] [--class C] [--run-id ID]` | The approvals inbox, what needs a person first. `--limit`, `--cursor`, `--all` page. | read | +| `actions approve RUN_ID ACTION_ID` | Approves an action. Refused for a class in `suggest` mode, and for a person who is not on the class's `approvers` list when it has one. | write, plus the action class's permission | +| `actions deny RUN_ID ACTION_ID [--note TEXT]` | Denies an action (same `approvers` check). | same | +| `policy get [--category] [--priority] [--severity] [--target-tag]... [--class] [--template]` | The effective policy, record by record, for those attributes. `--template` shows the run identity that template would use. | read | +| `policy status` | Whether automatic actions are paused. | read | +| `policy pause`, `policy resume` | Stops or allows every automatic action in the organization. | `org.conf.set` | + +Policies and templates themselves are Hive records. Create and edit them with the +standard `limacharlie hive` commands for the `investigation_plan` and +`investigation_policy` types, or with infrastructure-as-code. diff --git a/docs/investigation-engine/concepts.md b/docs/investigation-engine/concepts.md new file mode 100644 index 000000000..f829d2937 --- /dev/null +++ b/docs/investigation-engine/concepts.md @@ -0,0 +1,240 @@ +# Concepts + +This page defines the words the rest of the Investigation Engine documentation +uses. + +## The pieces + +| Object | Stored as | What it is | +|---|---|---| +| **Plan template** | Hive record `investigation_plan/` | A reusable plan for a class of alerts: the facts to gather, the proofs that settle a verdict, how weaker evidence moves the probability, and the actions to propose. See [Plan Templates](templates.md). | +| **Policy** | Hive record `investigation_policy/` | What the engine may do: budgets, allowed operators, autonomy per action class, approvers, the identity runs execute as. See [Policies and Autonomy](policies.md). | +| **Run** | Engine store | One investigation: its inputs, the effective policy, state, evidence graph, steps, actions and verdict. | +| **Step** | Part of a run | One operator call (one way of obtaining a fact), with its timing, cost and outcome. | +| **Fact** | Part of a run | A typed result of a step, with its provenance. A fact is `ok`, `unknown` (with a reason) or `incomplete`. | +| **Action** | Part of a run | A containment, eradication or collection step with a target, a class, a state and, for reversible ones, a lease. | +| **Operator** | Built into the engine | One way to obtain a fact or perform an action: an entity lookup, a search, a sensor read, a threat-intel lookup, an LLM judgement. The org's operator list shows which ones your policy allows. | + +## Facts, signals and proof + +A template lists **facts**. Each fact is one operator call whose arguments can +refer to run inputs and to other facts. A fact that reads another fact waits for +it. Every other fact starts immediately, in parallel. + +```yaml +facts: + trigger: # the detection record itself + op: detection.get + args: {detect_id: $detection_id, window: [-29d, now]} + host: # waits for `trigger`, reads its `sid` + op: entity.resolve + args: {type: sensor_id, value: $trigger.sid} + related: # also waits for `trigger`; runs in parallel with `host` + op: detection.list + args: {sids: [$trigger.sid], window: [-7d, now]} +``` + +**Proofs** are conditions over facts that settle the verdict. A proof is a list of +alternatives (any one completing is enough); each alternative is a condition or +an `all` / `any` group of conditions: + +```yaml +proofs: + false_positive: + - 'related.count <= 1 and judge.label == "no"' + true_positive: + - all: ['judge.label == "yes"', 'related.count >= 5'] +``` + +**Evidence** is the softer path. When no proof completes, each `evidence` rule +whose condition is true adds its `weight` (a log-odds value; negative favors a +false positive) to the template's `prior`. The result is the probability of a +true positive, `P(TP)`. + +**Signals** apply to detections that carry several reasons for firing, such as an +email detection. Each signal that fired is given a question, a condition under +which it is *malicious* (holds) and one under which it is *benign* (explained). +`signals.all_explained` is a common backbone of a false-positive proof. + +### How a verdict is reached + +| Outcome | When | +|---|---| +| `true_positive` | A true-positive proof completes, or the evidence alone brings `P(TP)` to the policy's `escalate_at` (0.90 by default). | +| `false_positive` | A false-positive proof completes, or the evidence alone brings `P(TP)` to `dismiss_at` (0.03) or below and nothing is left to run. **Both paths wait for the required checks** (`mandatory_checks` of the run's urgency): while one is unknown or unfinished, the run cannot be dismissed, and if it stays unknown after everything has run, the run ends `inconclusive`. | +| `inconclusive` | Nothing more can be learned and neither side was reached. The case lists what stayed unknown and asks one open question. | + +When a proof completes, the engine **cancels every step that can no longer matter** +and records why. This is where most of the time saved comes from. + +## Unknown, absent and incomplete + +A fact has one of three conditions: + +| Condition | Meaning | +|---|---| +| **ok** | The operator answered and the answer is complete. | +| **unknown** | There is no answer. The fact has a `reason` such as `timeout`, `forbidden`, `not_found`, `policy_denied`, `missing_input` or `outside_free_window`. | +| **incomplete** | There is an answer, but it may be missing data: a truncated list, a search over the last few minutes (`index_lag`), a window clamped to the free time range. | + +The engine uses three-valued (Kleene) logic, and this is the rule behind every +proof: + +- A comparison on an unknown fact is **unknown, never false**. `not unknown` is + unknown. `unknown and false` is false, and `unknown or true` is true. +- A proof completes only when its expression is **true**. A fact that is unknown + can never complete a proof. +- An **incomplete** fact supports only answers that can only get stronger with + more data: `count > 0`, `found`, `x in list`. A claim of absence (`found == false`, + `count == 0`) on an incomplete fact is unknown. + +So "no click was seen" proves nothing until the data it rests on is complete. +**Inconclusive is a good answer when data is missing.** The case says what could not +be established and why. See [Writing templates](templates.md#proofs-evidence-and-signals) for +how to write conditions that respect this. + +## Runs + +### States and phases + +A run is `queued`, `running`, `decided`, `completed`, `cancelled`, `failed` or +`budget_exhausted`. While it runs it moves through phases that overlap: + +1. **Triage** looks for the shortest proof of true or false positive. +2. **Scoping** gathers the facts the template lists under `scope` (see + [Plan Templates](templates.md#scope)). In the `balanced` and `aggressive` + speculation profiles it starts once `P(TP)` passes the policy's `scope_at` bar, usually + before the verdict is final. In the `conservative` profile it waits for a decided + true positive. The template names every fact to gather: the engine does not discover + new entities or generate searches of its own. +3. **Containment** proposes actions as the facts they depend on arrive and executes + them according to policy once their confidence bar is met. That can happen before the + verdict is final: see [Respond](templates.md#respond-and-report). + +`decided` means the verdict is final while scoping or actions may still be running. +A run is `completed` when its steps are done and every action is settled. + +### Modes and urgency + +A run starts in one of three modes: `triage` (stop at the verdict), `full` (triage, +scoping and containment) or `scope` (skip triage, for a confirmed incident). +`urgency` (`low`, `medium`, `high`, `critical`) selects how aggressively the engine +spends parallel work: a *speculation profile* of `conservative`, `balanced` or +`aggressive`. A typical policy, like the example in [Policies and Autonomy](policies.md#a-default-policy), +maps `low` to conservative, `medium` to balanced and `high` and `critical` to aggressive. If you do not set an urgency, the engine +derives it from the detection priority and the severity (priority 8 or more, or severity `critical`, is `critical`; 6 or more, or `high`, is `high`; 3 or more, or `medium`, is `medium`; anything lower is `low`) and falls back to `medium` when the start carries neither. Tags do not change the urgency; the trigger's tags can match a run-scoped [policy record](policies.md#two-match-scopes) that tightens the run. + +### Coalescing and idempotency + +A start that carries a `coalesce_key` **joins the open run that has the same key** +instead of starting a second run, and its inputs are added to that run. Nothing joins +runs merely because they concern the same entity: without a `coalesce_key` every start +is its own run. The key only matches within the same organization, the same starter +and the same run identity, so a start by a person and a start by a D&R rule never join +each other, and a request that executes as a different identity never joins a run +executing as this one. All D&R rules share one starter per organization, so alerts from different rules +with the same key do join. + +An `idempotency_key` makes a repeated start with the same starter return the same run. +For runs started by a D&R rule, the detection id is the idempotency key, so one +detection starts one run, within the retention of the engine's records. + +### Identity + +Every read a run makes and every action it takes is authorized as the run's +identity, so a run can never read or do what its identity cannot. + +- A run started by a person or an API key uses the identity the policy names + (`identity.default`), unless the request picks `caller` and the policy allows it. With + no policy identity, it uses that person's (or API key's) token. **A policy secret can + hold more permissions than the person who starts the run.** +- A run started by a D&R rule or a new case has no person. It needs an identity + named in the org's policy: an **organization API key** stored as a Hive secret, + set as `identity.default` or, for one template, as that template's own + `identity.templates..default`. Without one, the start is refused with a + message that names the missing setting. +- A run that must work for hours (a lease, a delayed approval) should use a secret + identity. A run on a person's token cannot schedule work that outlives that token, + so its leases are shortened to fit. + +See [Policies and Autonomy](policies.md#run-identity). + +### The free time range + +Every search a run makes must start no earlier than **now minus 30 days**. A +template that asks for older data is rejected when it is validated. A window that +still reaches past the boundary when it runs is clamped to it, and the result is +marked incomplete or unknown (`outside_free_window`), never "nothing found". +A policy can lower the lookback below 30 days, never raise it. +Operators that bill per use, such as artifact collection, are off in the default +policy. + +### What a run costs + +- **Searches** stay inside the free time range, so they are free of search charges. +- **Language-model judgements** (`llm.judge`) go through your organization's AI agent + configuration, which decides the model and credentials. The usage is reported as AI + usage and billed with your other AI usage, under the feature + `investigation_engine`, with the run id as the reference. The policy's + `max_llm_usd_per_run` budget **stops the run from launching further judgements** once + the cost already recorded reaches it. It is not a hard cap on spend: the engine does + not reserve the cost of a judgement before it runs, so a judgement already in flight + (or several launched together) can take the run past the budget. The dollar figures + come from the engine's configured prices, and a judgement billed with no known price + adds nothing to the dollar total, so it never counts against the budget. The spend a + run reports is therefore a lower bound, and a judgement cut short by a deadline or a + cancel can still be billed. +- **Operators that bill per use**, such as artifact collection and some extensions, are + off in the default policy. An organization enables them by policy. + +### The case + +Each run writes a [case](../5-integrations/extensions/limacharlie/cases.md) by +default. An engine-written case contains the triggering detections, a plain-language +summary with the verdict and proof (each claim cites a fact), entities and +telemetry with a verdict and a reason each, the unknowns that mattered, the actions +proposed, approved, executed and reverted with who decided, severity, status, tags +and a classification, and a link back to the run. + +Because every engine case has the same structure, what people do to it is +**feedback**: closing it unchanged weakly confirms the verdict, changing its +classification or an entity verdict corrects it, and approving or denying an action +labels that action. The engine records these against the run, the template version +and the facts involved. + +## Waves and fact packs + +A template is for an alert that comes back. A one-off question ("what did this host +do after that download?", "who else received this message?") does not need one. + +A **wave** is a batch of operator calls, or fact packs, that the engine runs now, +in parallel, and returns compact answers for. It has no plan and no verdict, but +the engine still batches, caches, enforces the free time range and applies policy. +A wave can reference earlier steps of the same wave, can `select` just the fields +you need, and can stop early with `stop_when` conditions that follow the same +unknown-is-never-false rule as proofs. + +A **fact pack** is a named, tested bundle of facts with parameters and described +exports. The engine has these built in: `detection_context`, `domain_reputation`, +`hash_spread`, `host_context`, `message_outcome`, `org_roster`, `process_tree`, +`sender_reputation` and `user_activity`. A pack can be used in a wave or as a fact +(`use:` instead of `op:`) in a template, where it expands into ordinary facts named +`__`. + +Waves are what an AI agent uses to answer a question in three or four calls instead +of thirty. See [MCP Tools](mcp.md) and [API Reference](api-reference.md#waves). + +## Draft and active templates + +A template carries `status: draft` (the default) or `status: active`. + +| | Draft | Active | +|---|---|---| +| Can a person start a run on it? | Yes | Yes | +| Autonomy | Capped at `suggest` for every action class | As the policy says | +| Can a D&R rule or the case trigger start it? | No | Yes | +| Picked automatically by `applies_to`? | No | Yes | + +**A person promotes a draft** by setting `status: active` in the Hive record. None of +the AI-agent tools can do it, and by default the Hive refuses it from an API key (see +[Permissions](permissions.md#active-templates-can-only-be-written-by-a-person)). diff --git a/docs/investigation-engine/console.md b/docs/investigation-engine/console.md new file mode 100644 index 000000000..7197e2991 --- /dev/null +++ b/docs/investigation-engine/console.md @@ -0,0 +1,295 @@ +# Console + +The Investigation Engine lives in the web console under **Investigations +(experimental)** in the sidebar, at `/orgs//investigate`. The section is shown +to users who hold both `investigation.get` and `insight.evt.get`, and only in +organizations where the feature is enabled. Controls that need more permissions are +disabled with a tooltip naming what you need. See [Permissions](permissions.md). + +Inside the section, the tabs are **Runs**, **Approvals**, **Templates**, **Policy** +and **Performance**. A **New investigation** button is always at the top. + +## Start an investigation + +### From an alert, a case, an entity or a message + +An **Investigate** button appears where you are looking at something to investigate: + +| Where | What it starts from | +|---|---| +| A detection (detection viewer) | The detection. Also offers **Draft with AI** (see [below](#draft-a-template-with-ai)). | +| A case (case header, until the case is closed) | The case. | +| A Cloud Security entity page | The entity. | +| An Email Security message (message drawer) | The message. | +| The **New investigation** button, or **Start your first investigation** on an empty Runs list | The launcher, empty. | + +On a detection or a case, **Investigate** starts a run immediately and opens it. No +template is sent, so the engine **picks the template by `applies_to`** against the +detection's category (or the case's severity and category). The caret beside the button +opens a menu: **Start with a template** lists the organization's templates so you can +pick one explicitly, and **More options…** opens the full launcher prefilled. + +If no template matches, you see **No template fits this alert. Choose one with the +arrow next to the button.** Install or write a template with an `applies_to` that covers +the alert, or choose one by hand. + +A run started on a case does not write a second case. + +### The launcher + +**New investigation** opens the launcher, **Start an investigation**. + +#### What to investigate + +- **Trigger**: Detection, Email message, Entity, Case or Question, and its + identifier or text. +- **Plan template**: the organization's templates. If your link or the trigger implies + a default that does not exist, you are asked to choose one. +- **Run as**: shown when the policy offers more than one identity. The default is the + identity the policy names for the chosen template (**policy default**); the alternatives + are the identities the policy allows, which can be **Your own token** and other + policy secrets. "Everything the run reads or does happens with these permissions." The + choices are looked up for the template you pick, so they change with it. + +#### How to run it + +- **Urgency** (Low, Medium, High, Critical; empty is Automatic) and **Mode** (**Triage + only**, **Full (triage, scoping, containment)**, **Scope only**). +- **Speculation**: Conservative, Balanced or Aggressive; empty uses the policy + default. +- **Create a case**: on by default. "Write the verdict and its proof to a case." +- **Also run a serial baseline**: starts a second run of the same alert, one step at a + time, so the speedup is measured instead of estimated. The baseline never writes a + case and never acts. +- **Strict**: refuse the run, instead of reducing the value, when an override would + loosen the policy. + +**Policy context** (category, priority, severity) feeds the policy lookup. **Tighten the +policy for this run** lets you lower budgets, raise decision thresholds, lower an +action class's autonomy mode or raise its minimum confidence. Overrides can only make +the policy stricter. A value looser than the policy is moved to the policy value, and +the launcher says so: **Some overrides were reduced to the policy value**. + +The panel on the right is the **Effective policy** for the inputs you entered. + +## Runs + +The **Runs** tab lists the organization's runs, newest first, 50 at a time. While the +first page holds a run that is still in progress it refreshes every few seconds; a page +of finished runs does not refresh by itself, so reload to see runs started since. +Columns: State, Verdict, Trigger, Started by, Template, Time to +verdict, Speedup, Started and Actions (how many await a decision). Filters are +evaluated by the engine, so they cover all runs: search (detection, case, entity, +message or question), State, Verdict, Template, Time range, Started by, and **Mine**. + +### A run + +Opening a run shows its header (template and version, how the template was picked, +who started it, the identity it **Runs as**), a verdict chip, the live **P(true +positive)**, the run state and phase, and a strip of statistics: time to verdict, +speedup versus serial, steps (and how many were cancelled), cache hits, search events, +and LLM cost. A finished run is replayed from its log, and a live one updates itself. + +The six tabs: + +| Tab | What it shows | +|---|---| +| **Plan** | A dependency graph of the facts the engine gathered, by wave, ending in the proof. Click a node for its status, latency, request and result. Legend: in the completed proof, critical path, cancelled, speculative, unknown. Beside it, the **belief** chart (P(true positive) over time with the escalate, scope and dismiss thresholds) and **Why**: the plain-language reason for the verdict, the evidence that raised or lowered the probability, and the facts that are unknown. | +| **Timeline** | Bars per kind of step (indexed read, scan, live task, LLM judgement, action) showing what ran in parallel, what was cancelled and the critical path, with the time a serial run would have taken. | +| **Evidence** | Every fact the run gathered, filterable to *unknown* or *in proof*. A fact is **Known**, **Nothing found** (a real, complete negative), **Partial** (possibly incomplete) or **Unknown** (could not be determined, which is not a finding of absence). | +| **Actions** | The actions the run proposed or took, with the **Automatic actions** switch and, for a live run, **Cancel run** (with an option to also revert active leases). | +| **Case** | The case document the run wrote: summary, proof with cited facts, conclusion, signals, detections, entities, telemetry, artifacts, unknowns and actions, and a link to open the case. | +| **Compare** | What-if evaluation of the facts the run recorded, under another template or a tighter policy. It runs no new queries and changes nothing. Available once the run has finished. | + +## Approvals + +The **Approvals** tab is the inbox of actions across **all** runs, so you see what is +waiting for a person without opening each run. By default it shows actions in +**Awaiting approval**. Filter by state (including Proposed, Outcome uncertain, Active, +Executing, Reverting, Confirmed, Reverted, Failed, Denied and Expired) and by action +class. + +Each row shows the run it came from, with its template, state, verdict, urgency and +case, and a link to open the run's evidence. The action card shows the class (such as +**Isolate from network**), the target, its state and mode, the confidence, the +deadline for a decision, the lease time left, who decided, and its history. + +| State | Buttons | +|---|---| +| Proposed, Awaiting approval | **Approve**, **Deny** | +| Active | **Revert**, **Extend lease** (+1 h or +2 h), **Confirm** | +| Outcome uncertain | None that work today (see below) | + +Every button asks for confirmation and takes an optional note recorded in the action's +history. **Confirm** keeps an action in place and stops its lease from reverting it. + +- A proposal in **Suggest** mode is never executed, and **Approve** on it is refused + with "this action class is in suggest mode", even if you hold every permission. A + proposal keeps the mode it was made with, so changing the policy afterwards does not + unlock it. To execute an action, set the class to `approve` or `auto` in the policy and + start a **new** run, with an active template: a run on a draft template is capped at + Suggest whatever the policy says. +- An **Outcome uncertain** action means the engine could not confirm whether the action + took effect. It settles itself: a late answer from the platform makes it active or + failed, and when none comes, a reversible action is undone to be safe, while an + irreversible one is escalated to a person. The console may still offer **Revert** and + **Confirm** on it, but the engine only accepts those on an active (or, for Revert, + confirmed) action, so on an uncertain one they return an error. Check the target + yourself instead. +- A class that is not available yet (see [Action classes](policies.md#action-classes)) + can be proposed but fails when approved. + +A button is disabled unless you hold `investigation.set` and `insight.evt.get` **and** +the permission of the action's class (see [Permissions](permissions.md#permission-of-the-action-class)). +**Approve** and **Deny** also require being on the class's approver list when the +policy sets one; the other decisions do not check it. + +## Templates + +The **Templates** tab lists the organization's `investigation_plan` records with +their status (Draft or Active), version, the categories each applies to, and +description. A badge marks templates that appeared while you had the page open, for +example one an AI agent just saved. + +### Install default templates + +**Install default templates** opens a dialog that lists the +[default library](templates.md#the-default-library). Each row shows the template, its +version, its status as shipped and one of **Not installed**, **Installed** or **Update +available**. + +- **Install as** is **Draft (default)** or **Active**. Draft is selected whenever you + open the dialog, **whatever status the library recommends**: the column shows + the library's recommendation ("Recommended: Draft" for the current library) for + information only. Each row has its own **Active** switch to + override the choice for that template. +- **Install selected** and **Install all** write a copy of each template into your + organization, one at a time, with your own token. A failure on one template does not + stop the others. +- Anything that would be written as **Active** shows a warning (active templates start + from detection rules and cases on their own), and installing **all** as Active asks + you to confirm. +- A template that is already installed is never replaced silently. If a newer library + version exists, selecting the update asks you to confirm: **An update replaces your + version**, including any changes you made, and writes it with the status chosen + for it. If that replaces a live (Active) template with a Draft, the dialog says so and + asks again, since it stops that template from running on its own. +- The result lists each template as **Installed**, **Updated**, **Already exists** (a + record with that name, possibly disabled, was left alone), **Already up to date** (the + record changed after the list was read and is no longer older than the library's), + **Changed meanwhile** (someone made it active or draft after you confirmed; open the + dialog again) or **Failed**. +- You need `investigation_plan.get` and `investigation_plan.set`; without them the + dialog explains what is missing. + +### Create, edit and copy + +**New template** and **Edit** open a YAML editor. Names use letters, digits, dots, +dashes and underscores and must start with a letter or digit. **Start from** offers a +blank skeleton or a library example, and **Copy** on a template creates `-copy` +as a draft at version 1. + +- **Validate** asks the engine to check the text and shows errors and warnings with + their paths and suggestions, plus the worst-case estimate (steps, searches, AI steps, + seconds, budgets exceeded). +- **Save** writes the Hive record. Saving never changes the status: "New templates and + copies are drafts; use Make active on the template's page." +- Editing an active template warns that the change applies to automatic investigations + right away. Move it back to draft first to test safely. +- Saving needs `investigation_plan.get` and `investigation_plan.set`. If the record + changed since you opened it, you are told to reload and apply your edit again. + +### Test on past alerts + +On a template's page (or in the editor, to test unsaved text), **Test on past alerts** +runs the template's real read-only steps on alerts that already happened. It creates +no run, no case and no action; only the test result is kept, for 24 hours. You build a list of samples, at most 25 alerts in +total: + +- **A detection**, by detection id; +- **A case**, by case id: its classification is the expected answer; +- **Last N of a category**: the newest N detections of that category, N from 1 to 25. + +The result shows a progress bar, then summary cards: **Alerts tested**, **Verdicts**, +**Agreement with closed cases**, **Unknowns**, **Cost** and **Time per alert**. Below, +**Results per alert** gives each alert's verdict, probability and basis, the proof, the +unknowns, the cost, the actions the template **would propose**, and a chip: **Agrees +with the case**, **Disagrees with the case**, **Undecided** or **No closed case**. + +If steps were refused for lack of permission, a banner lists them: the test ran with +the permissions of whoever started it, so grant the permission of the product each of +those facts reads (for example `mailsec.get` for email steps) and test again. Until +then those facts stay unknown and the verdicts mean little. Results are kept for 24 +hours. You need `investigation.set`, `investigation.get` and `insight.evt.get`: the +console starts the test with the first and follows it with the second. + +### Make active, or back to draft + +On the template's page: + +- **Make active** asks for confirmation and shows the last test from this browser + (alerts run, verdict counts and the agreement with closed cases), or + "This template has not been tested from this browser." A test is not required, but + look at it. **Make active** is disabled until the engine reports the template valid. + Active templates "are picked by detections, rules and cases on their own, and can + propose actions within your org policy." +- **Back to draft** stops detections and cases from picking the template. Runs already + in progress are not affected. +- **Delete** removes the template from the organization and cannot be undone. + +Promotion is made by a person in the console. An API key cannot write an active +template unless the organization opted out. See [Permissions](permissions.md#active-templates-can-only-be-written-by-a-person). + +### Draft a template with AI + +**Draft with AI** (on the Templates tab, and on a detection) opens **Draft a template +with AI**. Describe the alerts you want investigated in plain words, or start from a +detection as the example alert. An AI agent looks at your data, writes a plan, tests it +on past alerts and saves it as a **draft**. The new draft appears in the list, and you +decide whether it goes live. The workflow it follows is described in +[Build and Tune Templates with an AI Agent](build-templates-with-ai.md). + +The button needs `investigation.get`, `investigation.set`, `insight.evt.get`, +`investigation_plan.get`, `investigation_plan.set` and `ai_agent.exec`, and it uses the +**Investigation plan author** AI agent from the AI agent catalog. If that agent is not +installed in your organization, the dialog offers **Install the plan author**, or tells +you to ask an administrator. The agent works with its own API key, so it can only read +what that key allows; missing permissions show up as forbidden steps in its tests. + +## Policy + +The **Policy** tab does not edit policies. Policy records are `investigation_policy` +Hive records, which you edit with the Hive API, the CLI or infrastructure-as-code. See +[Policies and Autonomy](policies.md). The tab shows: + +- **Effective policy**: what the organization's policy allows, record by record. Leave + the lookup empty for the organization default, or enter a + category, severity, priority and a template to see what applies to that alert (the + template selects its run identity). It lists the + policy records applied, the budgets, decision thresholds, speculation by urgency, and + the actions and autonomy table (mode, minimum confidence, lease, approvers and the + record each value comes from). +- **Automatic actions**: the pause switch. + +### Pause automatic actions + +The switch shows **Running: actions in auto mode can execute** or **Paused: nothing +executes without a person**, and who paused it and when. **Pause automatic actions** +stops every action that would execute on its own, in every run of the organization. +"Actions that wait for approval are not affected." **Resume automatic actions** lets +actions in `auto` mode execute again. Everyone with console access can see the state; +changing it needs `org.conf.set`. The same switch is at the top of every run's +**Actions** tab. + +## Performance + +The **Performance** tab aggregates runs over the last 24 hours, 7 days, 30 days or a +custom range, optionally for one template. It shows the **speed-up versus serial**, +where the gain comes from (steps running together versus stopping early), time to +verdict, cancelled work, cache hit rate, batched calls, speculation efficiency, search +events (billed and free), LLM spend, analyst feedback (labels and corrections), the +operator latency table and the actions by class and state. + +The speedup is only shown for runs that have a serial baseline of the same +investigation: start it with **Also run a serial baseline**. The page reads at most the +2,000 most recent runs of the range and says when it truncated. diff --git a/docs/investigation-engine/index.md b/docs/investigation-engine/index.md new file mode 100644 index 000000000..1742374c5 --- /dev/null +++ b/docs/investigation-engine/index.md @@ -0,0 +1,70 @@ +# Investigation Engine + +!!! info "Pre-release" + The Investigation Engine is documented here ahead of general availability. + Names, defaults and screens can still change before release, and the + service is only reachable in datacenters where it has been enabled for your + organization. + +The Investigation Engine investigates an alert for you. You hand it a detection, +a case, an entity or a question. It gathers the facts its plan template lists, every +fact whose inputs are ready **in parallel**, stops as soon as the verdict is +settled, writes the result to a [case](../5-integrations/extensions/limacharlie/cases.md), +and proposes or takes containment actions **only as far as your organization's +policy allows**. Containment can start before the verdict is final, once a template's +conditions and the policy's confidence bars are met. Today two action classes can +execute: email quarantine and sensor network isolation (see +[Respond](templates.md#respond-and-report)). + +It is deterministic at its core. A language model can judge an ambiguous fact in +a small, bounded call, but it does not drive each step. The engine, not the model, +decides what runs next, what can be skipped and when to stop. + +!!! tip "In one sentence" + An analyst writes down once how to investigate a kind of alert (a *plan + template*); the engine then runs that plan on every matching alert, in + parallel, with a verdict, the proof behind it and a case, and containment + stays under the control of an org-wide autonomy policy. + +## What you get + +| Capability | What it gives you | +|---|---| +| **Runs** | One investigation per alert, case, entity or question: a verdict (`true_positive`, `false_positive` or `inconclusive`), the proof or evidence behind it, an evidence graph, a step timeline and the actions taken. See [Concepts](concepts.md). | +| **Plan templates** | Reusable, versioned plans for a class of alerts, stored as Hive records in your organization. You install the [default library](templates.md#the-default-library), edit it and write your own. | +| **Test on past alerts** | Run a draft template, read-only, on real past detections and cases before it ever acts. See [Test on past alerts](templates.md#test-on-past-alerts). | +| **Policies and autonomy** | Per action class: `off`, `suggest`, `approve` or `auto`, with confidence bars, leases, caps, approvers and a global pause switch. See [Policies and Autonomy](policies.md). | +| **Approvals inbox** | Every action that waits for a person, across all runs, in one list. | +| **Automatic starts** | A D&R rule can start a run with the [`start investigation`](automation.md#dr-rules-start-investigation) response action, and new cases can start one with the [case trigger](automation.md#the-case-trigger). | +| **Cases** | The engine writes the case for each run: summary, proof, entities, telemetry, unknowns, actions. People's edits to those cases are recorded as feedback. | +| **Waves** | For a one-off question, send a batch of questions that run in parallel and read compact answers, with no plan and no verdict. See [Waves and fact packs](concepts.md#waves-and-fact-packs). | +| **Console, API, MCP, CLI** | The same service from every surface: [console](console.md), [API](api-reference.md), [MCP tools](mcp.md) and the [CLI](cli.md). | + +## Rules that never bend + +- **Unknown is never false.** A step that failed, timed out, was not allowed or + whose data is incomplete is *unknown*. It never reads as "nothing found". See + [Unknown, absent and incomplete](concepts.md#unknown-absent-and-incomplete). +- **Every search stays inside the free time range.** No search window may start + earlier than 30 days before now. +- **A run can never do more than its identity can.** Every read and every action + uses the permissions of the identity the run executes as. +- **A draft template never acts on its own.** Drafts run only when a person starts + them, with every action capped at `suggest`. A person promotes a template to + `active`. +- **Nothing is installed for you.** An organization's templates are exactly its + `investigation_plan` Hive records. The default library is content you choose to + install. + +## Where to go next + +| I want to... | Read | +|---|---| +| Understand the vocabulary: facts, proofs, unknowns, runs | [Concepts](concepts.md) | +| Use the console | [Console](console.md) | +| Understand templates, install the defaults, test on past alerts | [Plan Templates](templates.md) | +| Let an AI agent write and test a template | [Build and Tune Templates with an AI Agent](build-templates-with-ai.md) | +| Decide what the engine may do on its own | [Policies and Autonomy](policies.md) | +| Start runs from D&R rules or new cases | [Automatic Starts](automation.md) | +| Give people and keys the right access | [Permissions](permissions.md) | +| Call it from code | [API Reference](api-reference.md), [MCP Tools](mcp.md), [CLI](cli.md) | diff --git a/docs/investigation-engine/mcp.md b/docs/investigation-engine/mcp.md new file mode 100644 index 000000000..63ca62a0f --- /dev/null +++ b/docs/investigation-engine/mcp.md @@ -0,0 +1,118 @@ +# MCP Tools + +The [LimaCharlie MCP server](https://github.com/refractionPOINT/lc-mcp-server) exposes +the Investigation Engine to AI assistants. The tools are built for an agent that +**writes, tests and saves plan templates**, and that **answers one-off questions** by +driving waves of parallel checks. Saving is always a draft: a person promotes it. + +!!! info "Pre-release" + The `investigate_*` tools exist only when the server is started with + `INVESTIGATION_ENGINE_ENABLED=true` (or `1`). Without it they are not registered: + they are absent from the tool list on every endpoint, including `all`, and the two + profiles below are not defined. The variable is read once at startup. + +## Connect + +Connect the server as described in [Connecting AI Assistants](../6-developer-guide/mcp-server.md), +with one of two profiles: + +| Profile | Tools | +|---|---| +| `investigation_engine` | All ten tools below. | +| `investigation_engine_readonly` | `investigate_operators`, `investigate_authoring_kit`, `investigate_packs`, `investigate_validate_plan`, `investigate_list_templates`, `investigate_get`. No start, wave, test or save. | + +For a local client, set `MCP_PROFILE=investigation_engine` in the server's environment. +The table below lists the permissions each tool needs **at the gateway**. Give the +API key those, plus only the permissions of the data the checks you ask for read. A +wave or a test executes with the key's own permissions, so for example the `org_roster` +pack also needs `sensor.list`, and live sensor reads need `sensor.task.read` (when enabled, or `sensor.task`) plus `output.set`; +without them the facts come back `unknown`. A **run** (`investigate_start`) is different: +it executes as the policy's identity (`identity.default`) unless the request picks +`caller` and the policy allows it, so on an active template it can act with that secret's +permissions even if the key has none of them (see [Run identity](policies.md#run-identity)). Profiles select tools; each call still +checks the key's permissions at the gateway. + +## Tools + +| Tool | What it does | Gateway permissions | +|---|---|---| +| `investigate_authoring_kit` | One call for everything needed to write a template: the template JSON Schema, the authoring guide, the organization's typed operator catalog, the default template library, the effective policy and every validation code. `sections` (`schema`, `guide`, `operators`, `examples`, `policy`, `codes`, `packs`, `library`) fetches part of it. The whole kit is large: start with `guide,schema,codes`. | `investigation.get`, `insight.evt.get` | +| `investigate_operators` | The operator catalog, with `filter`, `names` and `brief`. | same | +| `investigate_packs` | The fact packs as a compact index: parameters with types, exports with type, reference form and a short meaning. `name` returns one pack in full. Enough to write waves without the operator catalog. | same | +| `investigate_validate_plan` | Type-checks a template (`yaml`): structure, operators, references, expressions, free time range and budgets. | `investigation.get`, `insight.evt.get` | +| `investigate_test_plan` | Dry run on real past alerts: a stored `template` or `yaml`, plus `samples` (a detection id, a case id, the newest alerts of a category, or free inputs with an expected label). Waits and polls (`wait_seconds` up to 180, default 60) and returns the agreement rate, disagreements, unknowns and costs. `test_id` continues a test that is still running. `include_steps` adds every step. No run, case, label or action is written; the test result itself is kept for 24 hours. | `investigation.set`, `investigation.get`, `insight.evt.get` (starting needs `investigation.set`; the polling and `test_id` continuation read with `investigation.get`) | +| `investigate_save_plan` | Saves a template to the `investigation_plan` Hive, **always as a draft**. | `investigation_plan.get`, `investigation_plan.set`, `investigation.get`, `insight.evt.get` | +| `investigate_list_templates` | The organization's templates: name, version, description, `applies_to`, status. `status` filters draft or active. | `investigation.get`, `insight.evt.get` | +| `investigate_start` | Starts a run on a `template` for a `detection_id`, `case_id`, `entity_id` or `question`. Optional `inputs`, `urgency`, `mode` (`triage`, `full`, `scope`), `case` (`create`, `none`, `attach:`), `idempotency_key`, and `wait_seconds` (up to 60) to wait for the verdict. A draft template runs with autonomy capped at `suggest`. | `investigation.set`, `insight.evt.get` | +| `investigate_get` | Reads a run. `view` is `run` (default), `steps`, `events` (long-poll: `after`, `wait` up to 10) or `case`. | `investigation.get`, `insight.evt.get` | +| `investigate_wave` | Runs a batch of questions (operators or fact packs) in parallel and returns compact typed answers, stopping as soon as a `stop_when` condition is true. Steps carry `select` and `limit`; `full` returns raw values; `validate_only` checks a wave without running it; `deadline_ms` defaults to 30000. | `investigation.set`, `insight.evt.get` | + +All of them work against the organization the session is connected to. + +## Templates and the default library + +An organization's templates are exactly its `investigation_plan` Hive records. The +engine ships a **default library** of ready-made templates. It is content, not part of +any organization: read it with `investigate_authoring_kit` and `sections: ["library"]`, +and install one by writing it as an `investigation_plan/` record with the Hive +tools. Library entries are drafts (`status: draft`): keep that status and have a person +promote the template once it is tested, or install from the console. The Hive refuses by +default an `active` write from an API key (an administrator can opt out; see +[Permissions](permissions.md#active-templates-can-only-be-written-by-a-person)). Fact packs (`use:` in a fact) are built into the engine, like operators. + +## `investigate_save_plan` and what it refuses + +A draft never runs on its own. The tool is built so an agent cannot get around that: + +- It always writes `status: draft`, adds the tag `ai-draft`, and stores the optional + `test_summary` (up to 2,000 characters) as the record's comment so the reviewer sees + what was tested. +- It accepts a template with no `status` or with `status: draft`, and refuses + `status: active` or any other status. It also refuses a `name:` key inside the YAML + (the name is the tool's `name` argument, 1 to 64 lowercase letters, digits, `_` or + `-`, starting with a letter or digit) and YAML aliases (`*name` and merge keys). +- It refuses a template that does not validate, returning the full report. +- It never overwrites an **active** template, whether the engine lists it as active or + the Hive record says so. Save your version under a new name. +- It replaces an existing draft only with `overwrite: true`, and only if the record has + not changed since it was read. +- It fails closed: when the engine or the Hive cannot say what exists under the name, + nothing is written. + +A successful save answers with the name, the etag, the warnings, and a reminder that a +person reviews the draft in the console and promotes it. + +## The authoring loop + +See [Build and Tune Templates with an AI Agent](build-templates-with-ai.md) for the +full workflow. In tool terms: + +1. `investigate_authoring_kit` with `guide,schema,codes`, then `operators`. +2. Look at the organization's real data: detections and cases with the read tools, and + `investigate_wave` to see what an operator returns. +3. Draft the YAML. +4. `investigate_validate_plan` until `errors` is empty. +5. `investigate_test_plan` on real past alerts, including known true and false + positives; read the disagreements and unknowns, refine, test again. +6. `investigate_save_plan` with a `test_summary`. + +## Investigating a novel question without a template + +For a one-off question, drive waves directly: + +1. `investigate_packs` once: the packs, their arguments and exports. For an + indicator-driven question with no alert, start with the `org_roster` pack: the + organization's machines (offline ones included) and the people who own them. +2. `investigate_wave` with one step per pack (`use`) or operator (`op`), `select` for + the fields you will read, and `limit` to cap lists. +3. `stop_when`: the conditions that would settle the question. The engine cancels the + rest as soon as one is true and reports `stopped_by`. An unknown never makes a + condition true. An incomplete step can: a positive claim that only gets stronger with + more data (`machine_count > 3`) holds on a cut list, but a claim of absence or of an + exact count does not. Conditions are evaluated on the full results, before the answer + is compacted, so a step returned as incomplete may already have stopped the wave. +4. Read the compact answer (`status`, `complete`, `value`, `usage`), plan the next wave + from it, and stop when the question is answered. + +Read `unknown` as "not established", never as "no". A step with a cut list is a lower +bound, not a count. See [Waves](api-reference.md#waves). diff --git a/docs/investigation-engine/permissions.md b/docs/investigation-engine/permissions.md new file mode 100644 index 000000000..49453c7a7 --- /dev/null +++ b/docs/investigation-engine/permissions.md @@ -0,0 +1,138 @@ +# Permissions + +The Investigation Engine is checked at the API gateway. The engine itself trusts +the gateway and never sees a permission list, so the tables on this page are what +the gateway (and the Config Hive) require. Two things are checked beyond them: the +engine applies the policy's per-class approver list to Approve and Deny, and every +read or action a run makes is authorized as the run's identity (see +[What a run can do](#what-a-run-can-do)). + +!!! warning "Plan and policy permissions are not part of any role" + `investigation_plan.*` and `investigation_policy.*` are **not** in the Owner + role or in any preset role. They are granted explicitly, to a person or to an + API key. An organization Owner who has not been granted them is refused by the + Hive when saving a template or a policy. + +## Using runs + +The gateway requires **all** permissions of a row. + +| To do this | Required permissions | +|---|---| +| Read runs, steps, the evidence graph, events, cases, labels, statistics, templates, operators, fact packs, the effective policy, the autonomy state, the approvals inbox, the authoring kit; validate a template; read test results | `investigation.get`, `insight.evt.get` | +| Start a run or a wave; add inputs to a run; cancel or compare a run; **test a template on past alerts** | `investigation.set`, `insight.evt.get` | +| Approve, deny, revert, extend or confirm an action | `investigation.set`, `insight.evt.get`, **and the permission of the action's class** (below) | +| Pause or resume automatic actions for the organization | `org.conf.set` only | + +`insight.evt.get` is required on every route except pausing and resuming. It means a +role that can use Cases (`investigation.get`) but cannot read telemetry cannot read +the results an investigation gathered. In the roles the API assigns, only **Owner** +holds `investigation.get` and `investigation.set`. The console's own role presets also +list them for Administrator and Operator; Viewer holds neither. Operator and Viewer +carry only `insight.evt.get.simple`, which is not enough, so of those presets only +Owner and Administrator can use the engine with no further grant. Grant the +permissions in the table above explicitly to anyone else who should use the engine. + +### Permission of the action class + +Deciding an action needs `investigation.set` plus the permission of what the action +does, because approving is how a person authorizes the engine to do it: + +| Action class | Additional permission | +|---|---| +| `contain.email` | `mailsec.act` | +| `contain.network`, `contain.process`, `contain.identity` | `sensor.task` | +| `eradicate.process`, `eradicate.file` | `sensor.task` | +| `collect.memory`, `collect.file`, `collect.artifact` | `sensor.task` | +| `tag` | `sensor.tag` | + +A class with no mapping is refused (fail closed). Only `contain.email` and +`contain.network` can be carried out today (see [Action classes](policies.md#action-classes)); +the others are listed because the gateway and the policy already recognize them. + +The policy's per-class `approvers` list is checked by the engine for **Approve and +Deny only**: the person must be on that list. Revert, extend and confirm need the +gateway permissions above but do not check the list. + +### Who can see a run + +A run belongs to the organization. Anyone with `investigation.get` and +`insight.evt.get` sees all of the organization's runs, and anyone with +`investigation.set` and `insight.evt.get` can cancel or compare one. Only the +identity that **started** a run can add inputs to it. The steps that the added inputs +release run as the run's own execution identity (the policy's identity when it names +one, the starter's token otherwise), not as a new token of the person adding them. + +## Templates and policies + +Templates and policies are Hive records. The permissions follow the usual +[Config Hive](../7-administration/config-hive/index.md) convention. + +| To do this | Required permission | +|---|---| +| Read a template's record (its YAML) from the Hive | `investigation_plan.get`. Listing the templates the engine serves needs only the run read permissions. | +| Create or edit templates, promote a draft | `investigation_plan.set` (and `.get`) | +| Delete templates | `investigation_plan.del` | +| Read or change template metadata only | `investigation_plan.get.mtd`, `investigation_plan.set.mtd` | +| Read, change or delete policies and their metadata | `investigation_policy.get`, `.set`, `.del`, `.get.mtd`, `.set.mtd` | + +Running a validation or a test needs only the run permissions above. A test creates +no run, case or action; its result is retained for 24 hours. + +### Active templates can only be written by a person + +A write that results in `status: active` is **refused when it comes from an API +key**, including an API key owned by a user. Only a person signed in as a user, in the +console or with a user login in the CLI (`limacharlie auth login`), can create an active +template, promote a draft or edit an active one. Demoting a +template to `draft` always works. + +This is deliberate: an AI agent or script holding an API key can write drafts but can +never put a template in a position to start from a rule or act on its own. An +organization that manages active templates with API keys, for example from a +Git-synced pipeline, can opt out by adding the tag `allow-api-key-promotion` to +the enabled `investigation_policy/default` record. Setting that tag needs +`investigation_policy.set.mtd`, so **never give an AI agent's key any +`investigation_policy.*` permission**: it could turn the gate off and promote its own +drafts. If the policy record cannot be read, the gate stays +closed. + +### Naming a secret as the run identity + +The `identity` section of the default policy can name an organization API key stored +as a Hive secret, as `hive://secret/`. Adding a **new** secret reference +requires that the person saving the policy can read that secret (`secret.get`, or +`secret.get.` within the secret's access scope), and the secret must already +exist. Only an organization API key can be a run identity: a user API key is refused. + +## What a run can do + +A run executes with the permissions of its identity: the token of the person or key +that started it, or the API key the policy names. Everything a run reads and every +action it takes is checked against those permissions. A run can never read or do what +its identity cannot. The gateway permissions in the tables above only get a call to the +engine; each operator then needs the permission of its own data source. For example, +the `org.roster` operator (the `org_roster` fact pack) lists sensors and needs +`sensor.list`; without it the fact is `unknown` (`forbidden`), not empty. + +A **sensor read** (`sensor.read.*`) tasks a sensor and needs `sensor.task.read` (or the broader `sensor.task`) plus +`output.set` on the identity. `sensor.task.read` is an explicit grant, never part of a +default or preset role. It permits only allowlisted live metadata reads when +`INVESTIGATION_ENGINE_ENABLED` is `true` or `1`; gate off grants no tasking. File +contents, scripts, mutations and unknown commands are refused server-side. For an +observe-only policy identity, grant `sensor.task.read` instead of `sensor.task` and +keep actions at `suggest`. Action permissions in the table above are unchanged. Without them the fact is `unknown`, never "absent". + +## Console access + +The console needs `investigation.get` **and** `insight.evt.get` to show the +Investigations section at all. Controls the gateway would refuse are disabled with a +tooltip naming the permissions you need. See [Console](console.md). + +## Keys: two limits to know + +- An API key must have a key id to use the Investigation Engine. A key without one is + refused on every route, reads included, with `this API key has no key id, so it + cannot own investigation runs`. +- An API key restricted to an IP range (`allowed_ip_range`) currently fails, because + the engine presents the forwarded token from a different address. diff --git a/docs/investigation-engine/policies.md b/docs/investigation-engine/policies.md new file mode 100644 index 000000000..b1e62309d --- /dev/null +++ b/docs/investigation-engine/policies.md @@ -0,0 +1,317 @@ +# Policies and Autonomy + +A policy decides **what the engine may do**: how much it may spend, which operators +it may use, which actions it may propose or take on its own, who approves, and which +identity runs execute as. Templates say what to investigate; policies say how far the +engine may go. + +Policies are `investigation_policy` [Hive](../7-administration/config-hive/index.md) +records. They are managed with the Hive API, the CLI or infrastructure-as-code, and +need the `investigation_policy.*` permissions, which are granted explicitly (see +[Permissions](permissions.md)). The console shows the **effective** policy and the +pause switch, but does not edit records. See [Console](console.md#policy). + +!!! tip "Start in `suggest`" + With every action class in `suggest`, the engine investigates, writes cases and + proposes actions, but never executes anything. Run that way for a while, read what + it proposes, and raise classes one at a time. + +## How policy records combine + +One record named **`default`** applies to everything. Other records carry a `match` +and apply only where they match. The effective policy for a run or an action is built +from the widest to the narrowest: + +1. The built-in safe defaults (below) for every field you did not set. +2. The `default` record. +3. Every matching override record. When records disagree, **the strictest value wins** + for each field. +4. The per-run `overrides` of the request. They can only make the policy stricter. +5. The permissions of the run's identity. A run can never read or do anything its + identity cannot. + +*Strictest* means a lower autonomy mode, a higher confidence bar, a shorter lease, a +smaller cap, a smaller budget, or fewer allowed operators. Autonomy modes rank +`off < suggest < approve < auto`. + +The `default` record therefore describes what the organization allows for ordinary +endpoints, and overrides tighten it for servers, crown jewels, executives or particular +detection categories. + +### Two match scopes + +A `match` has two scopes because two decisions are made at different times: + +| Scope | Decides | Evaluated | Against | +|---|---|---|---| +| `run` | Budgets, speculation, allowed operators, LLM settings | Once, when the run starts | The trigger: detection category, priority, rule tags, case severity | +| `target` | Autonomy of an action | For each action | The action's target: sensor selector, sensor tags, entity attributes such as `human`, `external` or department | + +### Built-in defaults + +An unset field never means "unlimited" or "allowed". The safe baseline is: an +unconfigured action class is **only suggested**; `min_confidence` is 0.85; a lease is +2 hours; the approval timeout is 15 minutes; automatic containment is capped at 3 +targets per run and 10 per hour; and eradication is capped at `approve` unless a record +explicitly sets `auto`. Per-run overrides apply on top of that, so an override can +never enable an action class the organization left off. + +If a policy record cannot be parsed, it never loosens anything: the previous good set +stays, and with none the engine falls back to suggest-only. + +## A default policy + +This example lets network isolation run by itself (`auto`). For a first policy, copy the +cautious one below it instead. + +```yaml +# investigation_policy/default +case: create +budgets: + max_concurrent_runs: 20 + max_wall_seconds: 900 + max_steps_per_run: 400 + max_llm_usd_per_run: 0.50 + search_slot_share: 0.5 # the most of the org's concurrent LCQL slots the engine may hold + lookback_days: 30 # 1 to 30 +operators: + allow: [plan.*, entity.*, org.roster, endpoint.*, ioc.*, detection.*, event.*, search.*, sensor.read.*, ti.*, mailsec.*, case.*, lookup.*, llm.judge] + deny: [sensor.collect.*] +speculation: # urgency -> profile + low: conservative + medium: balanced + high: aggressive + critical: aggressive +decision: + escalate_at: 0.90 # P(true positive) that decides a true positive on evidence alone + dismiss_at: 0.03 # P(true positive) at or below which a false positive is dismissed + scope_at: 0.40 # P(true positive) at which scoping starts + ask_approval_at: 0.60 # P(true positive) at which approvals are requested +autonomy: + contain.network: {mode: auto, min_confidence: 0.85, lease: 2h, max_targets_per_run: 3, max_targets_per_hour: 10} + contain.email: {mode: approve, approvers: ['group:soc-leads'], approval_timeout: 15m, lease: 24h} +``` + +Only `contain.network` and `contain.email` can be carried out today (see [Action +classes](#action-classes)), so those are the only classes this example sets. + +A cautious first policy has every class in `suggest`: + +```yaml +# investigation_policy/default +case: create +autonomy: + contain.network: {mode: suggest} + contain.email: {mode: suggest} +``` + +Override records tighten the default where they match: + +```yaml +# investigation_policy/servers +match: + target: '"server" in tags or "dc" in tags' +autonomy: + contain.network: {mode: approve} +``` + +```yaml +# investigation_policy/low-priority-detections +match: + run: 'priority < 3' +speculation: {low: conservative, medium: conservative} +budgets: {max_wall_seconds: 300} +``` + +### Record sections + +| Section | What it sets | +|---|---| +| `match` | `run` and/or `target` expressions. Absent on the `default` record. | +| `budgets` | `max_concurrent_runs`, `max_wall_seconds`, `max_steps_per_run`, `max_llm_usd_per_run`, `search_slot_share` (above 0 up to 1) and `lookback_days` (1 to 30, never more). | +| `operators` | `allow` and `deny` patterns over operator names (`entity.*`). Allow lists intersect across records. `extension_requests` is a list of exact `/` pairs that the `extension.request` operator may call. It is **denied by default**, only the `default` record grants it, and every other record only narrows it. | +| `speculation` | Urgency (`low`, `medium`, `high`, `critical`) to profile (`conservative`, `balanced`, `aggressive`). | +| `decision` | `escalate_at`, `dismiss_at`, `scope_at`, `ask_approval_at` (each from 0 to 1) and `mandatory_checks` per urgency. | +| `autonomy` | Per action class (or pattern such as `eradicate.*`): `mode`, `min_confidence`, `lease`, `max_targets_per_run`, `max_targets_per_hour`, `approvers`, `approval_timeout`. | +| `identity` | Which credentials runs execute with. **Default record only.** See below. | +| `case` | `create` or `none`: whether runs write a case by default. **Default record only.** | +| `notify` | Events to destinations. **Default record only.** See below. | +| `triggers` | Runs without a person, such as `case_created`. See [Automatic Starts](automation.md#the-case-trigger). | +| `llm` | Settings for `llm.judge`; the model and credentials come from the organization's `ai_agent` record. | + +A record is **enabled by default** when saved, unlike most Hive types. A record with +`usr_mtd.enabled: false` is invisible to the engine. + +The billable and costly operators, such as artifact collection (`sensor.collect.*`), +extensions that bill per use and detonation, are off in the default policy. An +organization enables them by policy. + +## Autonomy modes + +Autonomy is set **per action class**. + +| Mode | Behavior | +|---|---| +| `off` | The action class is never proposed. | +| `suggest` | Proposed with its reasoning, never executed, **not even if a person approves it**: approving a suggest-mode proposal is refused with "this action class is in suggest mode". Also a dry run when rolling out a new policy. | +| `approve` | Executed after a person with `investigation.set` and the underlying permission approves. The request goes out as soon as P(true positive) passes `ask_approval_at`, so the human's response time overlaps the investigation, and it can be before the verdict is final. | +| `auto` | Executed once `min_confidence` is met, within the caps. This does not wait for the final verdict: the bar is the confidence, and the template's `when`. | + +### Action classes + +| Action class | Available | What it does | Reversible | +|---|---|---|---| +| `contain.network` | **Yes** | Sensor network isolation, plus a lease expiry. A sensor that was already isolated is left isolated at undo. | Yes | +| `contain.email` | **Yes** | Email Security message quarantine, including the other recipient copies of the message's group. Each copy is checked before and after. | Where the provider supports release | +| `contain.process` | Not yet | Suspend and resume a process | Yes | +| `contain.identity` | Not yet | The organization's identity response extensions | Yes, by re-enabling | +| `tag` | Not yet | Sensor tags | Yes | +| `eradicate.process`, `eradicate.file` | Not yet | Kill a process, delete a file | No | +| `collect.memory`, `collect.file`, `collect.artifact` | Not yet | Evidence collection | No effect on the host | + +The policy and templates accept every class, but **only the two available classes can act**. A proposal of +any other class can be shown, and approved or executed automatically, but the attempt +fails with "action class ... is not wired" and the action ends `failed` with nothing done +on the target. Leave the others at `suggest` or `off`, and do not write templates that +propose them. + +### Rules no policy can change + +- **Reversible actions run as leases.** A lease undoes itself at expiry unless the + verdict or a person confirms it. +- **Irreversible actions never run before the evidence-preserving collection** that + the template lists with `before:`, when collection is allowed. (No irreversible class + is available yet.) +- **The pause switch stops all automatic actions** in the organization at once. + Proposals and approvals continue. See [Console](console.md#pause-automatic-actions). +- **Every action transition is recorded** in the run's event log with the run, the + proof and the approver. The ones a person acts on reach the organization's audit + log when the policy asks for notifications. +- **Nothing waits forever.** Every open action state has a deadline: a proposal + waiting for confidence or a prerequisite, an approval (the class's + `approval_timeout`, default 15 minutes, after which it expires and is treated as + denied), an execution, and a lease. +- **Unknown outcomes are not failures.** If a call to execute an action times out, the + action becomes *uncertain*. A late success makes it active, a late failure makes it + failed, and if nothing answers a reversible action is undone to be safe. An + irreversible action in this state is escalated to a person. + +### Approvals + +An action in `approve` mode appears in the [approvals inbox](console.md#approvals) and, +if the policy asks for it, as a notification. Approving needs `investigation.set`, +`insight.evt.get` and the permission of the action's class, and membership of the +`approvers` list when the class has one. **Deny** is checked against the `approvers` +list the same way. Revert, extend (the lease) and confirm (keep it) are the other +decisions: they need the same permissions but do not check the `approvers` list. +Each decision, with the person and the optional note, is recorded in the action's +history. + +## Run identity + +A run executes as one of two identities: + +- **The caller**: the token of the person or API key that started it. This is the + default when the policy names nothing. +- **A secret**: the API key stored in an organization Hive secret, exchanged for a + short-lived token. Only an **organization API key** works; a user API key is refused. + +```yaml +# investigation_policy/default +identity: + default: hive://secret/investigation-engine-key # or "caller" + allow: [caller] # what a request may name besides the default + templates: # per-template overrides + mailsec-suspicious-message: {default: hive://secret/mail-key, allow: [caller]} +``` + +- A request that names an identity is accepted only if it is the template's default or + in its `allow` list; otherwise it is refused with the allowed identities in the + message. +- **A run with no person** (a D&R rule or the case trigger) **needs a policy + identity.** The simplest setup is `identity.default`. A named template can instead + carry its own `identity.templates..default` and need no org-wide default; a + template-level entry inherits any field it leaves unset. Without an identity for the + template, the start is refused with a message that names both settings. The case trigger, when it + leaves the template to `auto`, reads the case with the org-wide identity before a + template is chosen, so keep `identity.default` set for that. +- An inline plan never inherits the organization-wide identity. +- A secret that is rotated, disabled or deleted stops being used within ten minutes, + or at once when the Hive's change broadcast announces it. +- When a run uses the caller's token, the engine never schedules work that outlives it. + A reversible action needs about 17 minutes of token left to execute, settle and be + undone; its lease is shortened to fit, and an action that cannot hold a lease of at + least a minute is refused. A run that must act for hours should use a secret identity. +- The key and its token never appear in a run, a log, an event, an error or a response. + +!!! warning "A policy secret can be wider than the person who starts a run" + When the policy names a secret as `identity.default`, a run started by a person + with `investigation.set` and `insight.evt.get` executes with the **secret's** + permissions, not the person's, unless they pick `caller` and the policy allows it. + The person's own permissions do not limit what that run reads or does. Give the + identity key only what the runs need, and keep `identity.allow` and the per-template + `identity` entries as narrow as the policy allows. The console shows who a run + **Runs as**. + +Give the identity key the least it needs: it reads telemetry for every run started by a +rule, and takes the actions the policy allows. Saving a policy that names a **new** +secret requires read access to that secret. See [Permissions](permissions.md). + +## Notifications + +The engine writes events to the organization's **audit log**, and your Outputs on the +audit stream deliver them like any other audit entry. Nothing is written unless the +policy's `notify` (or the request's own `notify` list) asks for it. + +```yaml +notify: + action.needs_approval: ['output:slack-soc', 'output:pagerduty'] + run.decided: ['output:slack-soc'] +``` + +| Event | Audit event type | When | +|---|---|---| +| `run.decided` | `ie_run_decided` | The run reached a verdict (once per run). | +| `action.needs_approval` | `ie_action_needs_approval` | An action entered *awaiting approval*. | +| `action.executed` | `ie_action_executed` | An action executed and became active or confirmed. | +| `action.reverted` | `ie_action_reverted` | An action was reverted by a person or by its lease ending. | +| `run.completed` | `ie_run_completed` | The run ended, with its final state. | + +`approval_needed` and `decided` are accepted as aliases of the first two events. +Destinations are written `output:` and travel in the record's metadata +(`mtd.destinations`). To make `output:slack-soc` a real, named Output, route on them +with a D&R rule on the audit target that matches `mtd.destinations` and the event type +and uses the `output` action. Without such a rule, every Output on the audit stream +receives the events. + +Notifications carry identifiers, states and what a person needs to act, including the +**action target** (a person cannot approve a containment of "something"). They never +carry evidence values. Delivery is best effort, and a lost notification never loses an +approval, since the action stays in the approvals inbox. + +## Per-run overrides + +A request can narrow the policy for one run: + +```json +"overrides": { + "speculation": "conservative", + "budgets": {"max_wall_seconds": 120}, + "autonomy": {"contain.network": {"mode": "suggest"}} +} +``` + +Overrides can only make the policy stricter: lower the mode, raise confidence, shorten +leases and timeouts, lower caps and budgets, raise the decision thresholds `escalate_at`, +`scope_at` and `ask_approval_at` and **lower** `dismiss_at` (a lower bar to dismiss is +the riskier direction, so a higher `dismiss_at` is the looser value), add operator +denies. A looser value is **clamped** to the policy value, and the run lists the clamped +fields in `policy.clamped`. With `"strict": true` the request fails with a `400` +instead. + +## See what applies + +The effective-policy view answers "what would the engine do for this alert or this +host?" record by record. In the console it is the **Policy** tab; over the API it is +`GET /v1/investigate/{oid}/policy/effective` with a category, priority, severity and +target tags. See [API Reference](api-reference.md). diff --git a/docs/investigation-engine/templates.md b/docs/investigation-engine/templates.md new file mode 100644 index 000000000..b29d52f34 --- /dev/null +++ b/docs/investigation-engine/templates.md @@ -0,0 +1,401 @@ +# Plan Templates + +A plan template tells the Investigation Engine how to investigate one kind of +alert. This page covers where templates live, the default library, the structure +of a template, how to validate and test one, and how a template becomes active. +For a guided workflow with an AI agent, see +[Build and Tune Templates with an AI Agent](build-templates-with-ai.md). + +## Where templates live + +Templates are [Config Hive](../7-administration/config-hive/index.md) records of +the `investigation_plan` type, named by the record key. **An organization's +templates are exactly its `investigation_plan` records.** The engine adds none and +hides none. A template you did not write, edit or install does not exist for your +organization. + +Reading and writing these records needs the `investigation_plan.get` and +`investigation_plan.set` permissions, which are **granted explicitly** and are not +part of the Owner role. See [Permissions](permissions.md). + +## The default library + +LimaCharlie provides a **default library** of templates for common detection +classes. It is content you choose to install. Nothing in the library reaches your +organization until a person installs it, and an installed copy is yours to edit. + +| Template | Version | Picked for | What it does | +|---|---|---|---| +| `endpoint-detection-triage` | 3 | any detection category (`detect_cat: ['*']`) | Generic fallback for a detection on a sensor: the triggering process and its parent, its network activity, the binary's prevalence, the host and its user, related detections and what is true on the host now. Proposes `contain.network` on a confirmed true positive. | +| `suspicious-process` | 3 | `encoded-powershell`, `lolbin-*` | Judges the command line of a trusted system binary, then checks the process tree, connections, user and whether the process is still active or persisted. Proposes `contain.network` on a confirmed true positive. | +| `mailsec-suspicious-message` | 5 | `mailsec-ai-triage` | Triage of a message the Email Security AI judge rated suspicious: explains each signal that fired, checks the sender's real history and corroborating mail. Proposes `contain.email` on a confirmed true positive. | +| `phishing-campaign` | 4 | `mailsec-ai-triage` | Investigates a malicious message several people received: where its link goes, who else received it, who clicked, whether a payload ran. Proposes `contain.email` and `contain.network` on a confirmed true positive. | + +The library entries are written for production use, but **installing them writes +drafts unless you choose otherwise**: a draft never starts from a D&R rule or the case +trigger, and its actions stay capped at `suggest`. Test an installed copy, then +promote it (see [Draft, active and promotion](#draft-active-and-promotion)). + +!!! warning "Installing the generic fallback" + `endpoint-detection-triage` matches **every** detection category. Once it is + installed and active, a D&R rule that says `start investigation` without a + `template` can choose it for any detection no more specific template covers. + Install it deliberately. + +!!! note "Versions and updates" + Each library entry has a `version`. The console compares the library entry's + version with the installed record's and offers an explicit update. An update + **replaces your edits**; there is no merge. If you changed a template, + copy it under another name before updating. The CLI never replaces an active + template with a draft (move it back to draft first), and the console asks you to + confirm before it does. + +### Install the library + +- **Console:** open **Investigations**, then **Templates**, and choose **Install default + templates**. The dialog starts at **Draft** for every template, whatever the library + recommends; making a template active is an explicit choice, confirmed separately. + See [Console](console.md#install-default-templates). +- **CLI:** `limacharlie investigate templates install-defaults`, which writes drafts + unless you pass `--status`. See [CLI](cli.md#templates). +- **API or AI agent:** fetch the library with the authoring kit section + `library` (each entry has `name`, `description`, `version`, `status`, `applies_to` + and the `yaml` to store), then write the record `investigation_plan/` with + that YAML. The library entries are drafts (`status: draft`); keep that status when + you store one, and have a person promote it once it is tested. Writing `status: active` + is refused by default from an API key. Or install from the console. See + [Permissions](permissions.md#active-templates-can-only-be-written-by-a-person), + [API Reference](api-reference.md#authoring) and [MCP Tools](mcp.md). + +## Anatomy of a template + +A template is YAML. Unknown keys are errors, so a typo is never silently ignored. + +```yaml +status: draft # draft (default) or active +version: 1 +description: One line shown in lists. +applies_to: # which detections it is automatically picked for + detect_cat: ['mailsec-ai-triage'] +prior: 0.3 # P(true positive) before any evidence +inputs: # names a run may be given (see "Run inputs") + detection_id: '{{ .detection.detect_id }}' +facts: # what to gather: one operator call each + trigger: + op: detection.get + args: {detect_id: $detection_id, window: [-29d, now]} +signals: {} # optional: per-signal questions for multi-reason detections +proofs: # conditions that settle the verdict + false_positive: [] + true_positive: [] +evidence: [] # weighted conditions used when no proof completes +scope: # facts to start scoping from once a true positive is likely + from: [] +respond: [] # actions to propose, each with an optional `when` +report: # how the case is written + case: create +``` + +This skeleton shows every section. A real template needs at least one fact, and at +least one proof, signal or evidence rule, or it can never reach a verdict. + +Top-level keys are `status`, `version`, `description`, `applies_to`, `inputs`, +`prior`, `facts`, `signals`, `signals_from`, `proofs`, `evidence`, `scope`, +`respond` and `report`. The authoring kit returns a JSON Schema listing every key. + +### Facts and references + +A fact is `{op: , args: {...}}`, or `{use: , args: {...}}` for a +[fact pack](concepts.md#waves-and-fact-packs). Arguments are literals or references: + +| Form | Meaning | +|---|---| +| `$name` | A run input, or another fact. A fact that reads another waits for it. | +| `$name.path.to.field` | A field of that fact. References are checked against the operator's declared output, so a misspelled field is a validation error with a suggestion. | +| `$list[0].field` | An element of a list. | +| `$?name` | An optional input. If the run was not given it, the value is absent and the step still runs. | +| `$alert_time`, `$now` | Allowed in a `window`. | + +A `window` is `[start, end]`. A bound is `-24h`, `+30m` (relative to the alert), +`now`, `now-7d`, an RFC 3339 time, or `{at: '$message.received', offset: -2h}` (a time +read from a fact). Durations accept `s`, `m`, `h` and `d`. + +Facts can also carry `deadline: 30s`, `predict: true`, `checks: [tag]`, `case: true` +(the case always needs it) and a `report:` block that describes how the fact appears +in the case. + +Prefer indexed reads (entities, detections, email messages) when they answer the +question. Searches and the `llm.judge` operator cost more time and money. + +### Expressions + +Proofs, evidence rules, signals and `respond.when` use expressions: a path, or a +comparison of paths and literals, combined with `and`, `or`, `not` and parentheses. +Comparison operators are `==`, `!=`, `<`, `<=`, `>`, `>=` and `in`. + +```text +related.count >= 5 +message.sender_domain in org.trusted_domains +"server" in host.attrs.tags or host.attrs.role == "dc" +judge.label == "yes" +p >= 0.7 verdict == true_positive +signals.all_explained signals.any_holds +``` + +Only numbers are ordered. A string is never equal to a number. `p` is the current +probability of a true positive and `verdict` is the current label. + +### Run inputs + +The `inputs:` block declares input names and documents what a D&R rule would map +to each one. **The engine never evaluates those expressions.** A run gets only what +its trigger provides: + +| Started by | The run has | +|---|---| +| Console on a **detection** | `detection_id` | +| Console on a **case** | `case_id` | +| Console on an **email message** | `msg_uuid` | +| Console on an **entity** | `entity_id` | +| Console launcher, a question | `question` | +| D&R rule, `start investigation` | `detection_id` of the detection that fired, plus exactly the inputs the action lists | +| New case (case trigger) | `case_id`, and `detection_id` of the case's first detection when it has one | +| Test sample `{detection_id}` | `detection_id` | +| Test sample `{case_id}` | `case_id` and `detection_id` (the case's first detection) | +| Test sample `{detect_cat, last}` | `detection_id`, one run per detection | +| Test sample `{inputs: {...}}` | exactly the inputs you give | +| Every one of them | `alert_time`: when the run (or test) started. In a test this is the time of the test, not of the past alert; read the alert's own time from `detection.get`. | + +**The recommended pattern: take `detection_id` and derive everything else with +facts.** A template written this way runs unchanged from the console, from a D&R +rule, from the case trigger and from every kind of test sample: + +```yaml +inputs: + detection_id: '{{ .detection.detect_id }}' +facts: + trigger: + op: detection.get # the detection's own record + args: {detect_id: $detection_id, window: [-29d, now]} + host: + op: entity.resolve + args: {type: sensor_id, value: $trigger.sid} # a field of the detection, not an input +``` + +The classic first mistake is copying an `inputs:` block such as +`'{{ .event.HASH }}'` and then reading `$hash`. The trigger does not provide it, the +input does not exist and every fact that reads it is `unknown(missing_input)`. The +validator warns about this with `input_not_provided`. To prefer an explicit input and +fall back to a derived value, read the input with `$?name` and use the `plan.pick` +operator: + +```yaml +hash: + op: plan.pick + args: {candidates: ['$?hash', '$trigger.hash'], optional: true} +``` + +### Proofs, evidence and signals + +```yaml +proofs: + true_positive: # any ONE alternative completing proves a true positive + - all: [cmd.label == yes, {any: ['live_net.public_remote_count >= 1', live_autoruns.found]}] + false_positive: + - all: [cmd.label == no, 'prev.hosts_30d >= 50'] +evidence: + - {id: common_binary, when: 'prev.hosts_30d >= 50', weight: -0.8} +``` + +- Each list is a set of alternatives (OR). An alternative is an expression string, + `{all: [...]}` or `{any: [...]}`. They nest. +- The engine stops as soon as a proof completes and cancels the steps only the other + alternatives needed. If a true-positive and a false-positive proof both complete, + the verdict is flagged as a conflict, so avoid overlapping conditions. +- Evidence weights are log-odds. Keep them modest (0.5 to 2). Give each rule a stable + `id`. +- A proof that depends on an unknown fact never completes. See + [Unknown, absent and incomplete](concepts.md#unknown-absent-and-incomplete). + +Signals (for detections that carry several reasons for firing) are described with a +`question`, a `malicious` condition and a `benign` condition. **Give both**: a signal +with no `benign` can never be explained, so `signals.all_explained` is never true +while it fires (`signal_missing_explanation`). `signals.all_explained` is true only +when every signal that fired is explained, and it is not true when none fired. + +### Scope + +`scope.from` names the facts the scoping phase starts from, for instance the host and +the user. Scoping does not search for new entities: the engine runs the facts you +listed (and what they depend on) once they become relevant, and a +conservative-profile run waits for a decided true positive before it does. The keys +`follow` and `after` are accepted and not used yet. + +### Respond and report + +`respond` proposes actions: `{class, target, when, before, attrs}`. The engine +evaluates each step every time new facts arrive, and proposes it as soon as its +`when` is true and its `target` can be resolved. **This can be before the final +verdict.** A step with no `when` is proposed as soon as the target is known. Write +`when: 'verdict == true_positive'` to hold a step until the verdict is a decided true +positive (as the default library does); `p >= 0.7` proposes it while the evidence is +still building. What happens to a proposal then depends on its class's mode: +`approve` asks a person once `P(TP)` passes `ask_approval_at`, and `auto` executes as +soon as the class's `min_confidence` is met, both without waiting for the verdict. + +Only two classes can be carried out today: + +| Class | What it does | +|---|---| +| `contain.network` | Isolate a sensor from the network (a reversible lease). Needs `sensor.task`. | +| `contain.email` | Quarantine a message, and the other recipient copies of its group, through Email Security (reversible where the provider supports release). Needs `mailsec.act`. | + +The other classes the policy and the template format recognize (`contain.process`, +`contain.identity`, `tag`, `eradicate.process`, `eradicate.file`, `collect.memory`, +`collect.file`, `collect.artifact`) are **not available yet**: a template can propose +them, but approving or auto-executing one fails with "action class ... is not +wired", and the action ends `failed`. Do not use them in a template you intend to +run. + +**A template only proposes.** The organization's [autonomy policy](policies.md#autonomy-modes) +decides per class whether an action is only suggested, needs approval or runs by +itself. A class whose mode is `off` is never proposed. `before: ` on a step +makes **this step a prerequisite** of the steps of the named class on the same +target: those actions wait until this step's action has been carried out. For instance, +`collect.file` with `before: eradicate.file` means the collection comes first and the +eradication waits for it. If this step's class is only suggested or off, or no such +step was proposed, the later action goes ahead with a caveat saying the collection +was not performed; if the collection was allowed but was denied, failed or expired, +the later action needs a person's approval. (The classes in the example are not +available yet, see above.) + +`report` shapes the case: `case: create|none`, `severity` by verdict, `kind`, +`attack` (technique tags), `summary` sentences by verdict, and `questions` for the +analyst. A fact's `report:` block (kind `telemetry`, `entity`, `artifact` or +`detection`) says how it appears in the case, with a `note` in which `{fact.path}` +is replaced by values. + +## How templates are chosen + +A run names its template, or the engine picks one. A run **with no person** (a D&R +rule or the case trigger) that names no template gets the **active** template whose +`applies_to` matches the trigger. A person who starts a run from the console on a +detection or a case gets the same automatic choice. + +| `applies_to` key | Matches | +|---|---| +| `detect_cat` | The detection's category. | +| `rule` | The D&R rule, written `general.`. A bare record name never matches. | +| `severity` | The case severity. D&R never sends a severity, so this key only helps the case trigger. | +| `tag` | Any of the trigger's tags. For a D&R rule these are the rule record's tags. | +| `source` | `api`, `dr-rule` or `case`. | + +Values are exact (case-insensitive) or `*` globs (`lolbin-*`). A template matches +when **every** key it lists matches. With several matches, the one that lists more +keys wins, then the one with more exact (non-glob) values, then the lower name. If +none matches, the start is refused with a message that names what was tried. A +template with no `applies_to` is never chosen automatically; it can still be named. + +## Validate + +Validation type-checks a template against the operator catalog of your organization: +operators, argument types, every reference and expression path, and the free time +range. Every issue has a stable `code`, a `path`, a message and often a `suggestion` +that is the replacement to type. + +```text +errors: + - code: window_outside_free_range + path: facts.related.args.window + message: window starts earlier than 30 days ago: searches must stay inside the free time range + - code: unknown_field + path: proofs.false_positive[1] + message: 'prev.hosts_30: the fact has no field "hosts_30" (did you mean "hosts_30d"?)' + suggestion: hosts_30d +``` + +**Errors** make a template invalid. **Warnings** do not, and are what to fix next: +they are the mistakes that pass validation but make a plan useless. The most useful +ones are listed below; the authoring kit returns the complete list as `codes`. + +| Code | Meaning | +|---|---| +| `unknown_field` | A reference reads a field the operator does not return. The suggestion is the field you probably meant. | +| `window_outside_free_range` | A search window starts earlier than 30 days ago. | +| `input_not_provided` | A fact reads an input no trigger provides. Derive it from `detection.get` instead. | +| `unused_fact` | A fact nothing reads. It only costs a step. | +| `operator_denied` | Your policy does not allow the operator, so the fact is always unknown. | +| `proof_never_completes` | A proof reads a fact that is always unknown. | +| `contradictory_proofs` | A proof that can never be true, or one condition proving both sides. | +| `signal_missing_explanation` | A signal lacks its `benign` or `malicious` condition. | +| `respond_class_off` | The policy has this action class `off`; it is never proposed. | +| `applies_to_never_matches` | An `applies_to` key a D&R rule cannot match. | +| `budget_exceeded` | The worst case does not fit your policy's budgets. | + +The validation report also carries an `estimate` of the worst case of one run: the +number of steps, search steps, LLM steps, billed steps, the longest chain of step +deadlines in seconds, and which policy budgets it would exceed. + +## Test on past alerts + +Testing runs a template for real, **read-only**, on past alerts, and shows how it +would have done. Nothing is written: no run, no case, no feedback label, no action, +no audit event. The only thing kept is the test result itself, for 24 hours. + +A test takes a stored template (draft or active) or a template you paste, and up to +**25 samples**, each one of: + +| Sample | Meaning | +|---|---| +| `{detection_id}` | One past detection. | +| `{case_id}` | A case; the run gets its first detection, and the case's classification is the expected label. | +| `{detect_cat, last: N}` | The newest N stored detections of that category in the free time range (N up to 25). | +| `{inputs: {...}, expected: ...}` | Free run inputs, with an optional expected label. | + +Pick a mix: some detections whose cases were classified true positive, some false +positive, some with no case. + +A test runs **as you**, with your token and your permissions, whatever identity the +policy configures for real runs. It is forced to autonomy `suggest`, no case, and +never calls `extension.request` (those facts stay unknown with the reason +`policy_denied`). It counts against your organization's concurrent run cap. + +For each sample you get: + +- the verdict (`label`, `p`, and its basis: a proof or evidence), the proof that + completed, and what stayed unknown with its reason; +- the steps with status and timing, the cost, and the time; +- the actions the template **would** propose, by target reference only; +- when the detection belongs to a closed case that an analyst classified true + positive or false positive: the **expected** label and whether the template + **agreed**, **disagreed** or stayed **undecided**. + +The summary gives the counts, an agreement rate, the most frequent unknown reasons +and the cost and time. + +| You see | It usually means | Do | +|---|---|---| +| Many `undecided`, reason `not_found` or `forbidden` | The sample's data is not readable by you, or the template reads the wrong field. | Check the sample, then the operator arguments. | +| Many `undecided`, reason `index_lag` or `truncated` | A claim of absence on incomplete data. | Use a positive condition, or end the window before the last few minutes. | +| `disagree` | The proof is too eager, or the evidence weights are wrong. | Read the `proof` and `evidence` of that sample and tighten the alternative. | +| `agree` but always by evidence, never by proof | Slow and weak. | Add a proof alternative on the deciding fact. | +| High `wall_ms` or step count | Too many scans or LLM calls. | Drop facts that only feed unused alternatives, and reorder. | + +A sample without an `expected` label may still have a case the engine could not read: +the engine looks at the newest 100 closed cases of the organization, and the result +says when it could not read them all (`expected_complete: false`). + +## Draft, active and promotion + +New templates, and everything written by an AI agent, are **drafts**. A draft: + +- runs only through a test or a run a person starts; +- is capped at `suggest` for every action class, and says so in the run's + `policy.clamped`; +- cannot be started by a D&R rule or the case trigger, and is never picked by + `applies_to`. + +A person **promotes** a template by setting `status: active` in the Hive record +(in the console, with **Make active** on the template's page). Review what was tested first. Promotion does +not change the autonomy policy: an active template still only does what +[the policy](policies.md) allows. diff --git a/mkdocs.yml b/mkdocs.yml index a1389b8a1..4c5017659 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -708,6 +708,19 @@ nav: - API Reference: cloud-security/api-reference.md - Automation & IaC: cloud-security/automation.md + - Investigation Engine: + - Overview: investigation-engine/index.md + - Concepts: investigation-engine/concepts.md + - Console: investigation-engine/console.md + - Plan Templates: investigation-engine/templates.md + - Build and Tune Templates with an AI Agent: investigation-engine/build-templates-with-ai.md + - Policies and Autonomy: investigation-engine/policies.md + - Automatic Starts: investigation-engine/automation.md + - Permissions: investigation-engine/permissions.md + - API Reference: investigation-engine/api-reference.md + - MCP Tools: investigation-engine/mcp.md + - Command Line Interface: investigation-engine/cli.md + - Apps: - Overview: apps/index.md - Creating & Managing Apps: apps/creating-and-managing-apps.md