|
| 1 | +--- |
| 2 | +name: dpl-mcp |
| 3 | +description: Inspect a running DPL workflow through the DPL MCP server (Framework/Core/scripts/dpl-mcp-server) — devices and their state, live metrics, logs, signposts, per-mapping memory (smaps), resuming a workflow started with -s. Use when asked to look at, debug or measure a workflow while it runs, locally or on Hyperloop. |
| 4 | +--- |
| 5 | + |
| 6 | +# DPL MCP server |
| 7 | + |
| 8 | +`Framework/Core/scripts/dpl-mcp-server/dpl_mcp_server.py` is an MCP server |
| 9 | +(`FastMCP("DPL Status")`) that talks to the `/status` WebSocket of a running |
| 10 | +DPL **driver**. Everything it reports comes from the driver, which already |
| 11 | +holds the devices' metrics, logs and pids, so nothing has to be enabled in the |
| 12 | +workflow: every driver listens, `-b` or not. |
| 13 | + |
| 14 | +The driver side is `Framework/Core/src/StatusWebSocketHandler.{h,cxx}`; the |
| 15 | +header documents the wire protocol. A tool and its driver command are added |
| 16 | +together: a command the driver does not know is silently ignored, which the |
| 17 | +tool reports as a timeout. |
| 18 | + |
| 19 | +## Setup |
| 20 | + |
| 21 | +Dependencies are `mcp` and `websockets` (`pyproject.toml`, Python ≥ 3.11). |
| 22 | +Use a virtualenv next to the script and register the script itself; the server |
| 23 | +takes no command-line arguments, workflows are attached at run time with |
| 24 | +`connect`: |
| 25 | + |
| 26 | +```sh |
| 27 | +cd Framework/Core/scripts/dpl-mcp-server |
| 28 | +python3 -m venv .venv && .venv/bin/pip install mcp websockets |
| 29 | +claude mcp add dpl -- "$PWD/.venv/bin/python3" "$PWD/dpl_mcp_server.py" |
| 30 | +``` |
| 31 | + |
| 32 | +Several workflows can be connected at once; each tool takes the `workflow` |
| 33 | +name that `connect` returned. |
| 34 | + |
| 35 | +## Attaching to a workflow |
| 36 | + |
| 37 | +- **Local**: `connect(pid=<driver pid>)` or `connect(port=<port>)`. |
| 38 | + The driver binds `8080 + pid % 30000` (or `DPL_REMOTE_GUI_PORT`), but moves |
| 39 | + to the next free port if that one is taken, so when `pid=` fails take the |
| 40 | + port from the driver log: `Driver listening on port: <n>`. The driver pid is |
| 41 | + the one without `--id` on its command line. |
| 42 | +- **Hyperloop**: `connect_hyperloop(url=https://alimonitor.cern.ch/train-workdir/remote-gui/remote_proxy.html?<token>/<port>)` |
| 43 | + for a test that exposes the remote GUI. NOTE: this still targets the old |
| 44 | + `ws://localhost:8888` proxy with `HYPERLOOP_TOKEN`; it has to be moved to |
| 45 | + the security-proxy before it works again. |
| 46 | + |
| 47 | +## Tools |
| 48 | + |
| 49 | +| Tool | What it does | |
| 50 | +|------|--------------| |
| 51 | +| `connect`, `connect_hyperloop`, `disconnect`, `list_workflows` | attach / detach workflows | |
| 52 | +| `list_devices(workflow)` | name, pid, active, streaming state, device state of every device | |
| 53 | +| `list_metrics(workflow, device)` | numeric metrics a device publishes (string/enum ones are left out) | |
| 54 | +| `subscribe` / `unsubscribe(workflow, device, metrics)` | the driver pushes a subscribed metric each cycle in which it changed | |
| 55 | +| `get_updates(workflow, max_updates)` | drain the buffered metric updates, time-ordered | |
| 56 | +| `subscribe_logs` / `unsubscribe_logs(workflow, device)` | stream a device's log lines | |
| 57 | +| `get_logs(workflow, max_lines)` | drain the buffered log lines | |
| 58 | +| `enable_signpost` / `disable_signpost(workflow, device, streams)` | switch signpost streams on a device (`device=""` for the driver), e.g. `ch.cern.aliceo2.device`, `ch.cern.aliceo2.completion`, `ch.cern.aliceo2.monitoring_service`, `ch.cern.aliceo2.data_processor_context`, `ch.cern.aliceo2.stream_context`; the output arrives in the logs | |
| 59 | +| `start_devices(workflow)` | SIGCONT every device of a workflow started with `-s` (stop before device start) | |
| 60 | +| `smaps(workflow, device, top)` | memory per mapping, read by the driver from `/proc/<pid>/smaps` (Linux only) | |
| 61 | + |
| 62 | +Updates and logs are buffered by the server between calls and drained by |
| 63 | +`get_updates` / `get_logs`, so poll them rather than expecting a push. |
| 64 | + |
| 65 | +## Memory per mapping (`smaps`) |
| 66 | + |
| 67 | +The driver reads `/proc/<pid>/smaps` of the device and returns, per path, kB of |
| 68 | +rss, pss, shared/private clean/dirty and swap (unnamed mappings as `[anon]`). |
| 69 | +The tool groups them into O2Physics, O2, ROOT, arrow and other libraries, |
| 70 | +heap, anon and shm, and lists the `top` paths by PSS. |
| 71 | + |
| 72 | +- Compare **private-dirty**: it is what the device itself costs. PSS splits |
| 73 | + shared pages among every process mapping them, so it moves with how many |
| 74 | + other jobs on the node use the same libraries. |
| 75 | +- It works where an outside reader cannot: Hyperloop tests run as `alitrain`, |
| 76 | + and only a process of that user (the driver) may read the devices' smaps. |
| 77 | +- Take it while the workflow is processing, to see the working set rather than |
| 78 | + what is left after end of stream. |
| 79 | + |
| 80 | +## Typical uses |
| 81 | + |
| 82 | +- Why is a workflow stuck: `list_devices` for the states, then |
| 83 | + `subscribe_logs` + `get_logs` on the device whose streaming or device state lags the others |
| 84 | + and the `completion` signpost stream to see which inputs a |
| 85 | + device is waiting for. |
| 86 | +- Throughput or backpressure: `list_metrics`, then `subscribe` to the rate, |
| 87 | + size or relayer metrics (e.g. `inputs/relayed/*`, `total_rate_in_mb_s`) and |
| 88 | + read `get_updates` over time. |
| 89 | +- Where the memory of a device goes: `smaps`. |
| 90 | +- Debugging from the start: launch with `-s`, attach, enable signposts / |
| 91 | + subscribe to logs, then `start_devices`. |
0 commit comments