Repository navigation
feat(validate): warn on tool parameters without descriptions - #156
Conversation
Tool parameters reach the LLM as the tool's input schema. Descriptions written as `Annotated[T, "..."]` are already copied into that schema by StructuredTool.from_function, but nothing checked that they exist, so the model is often left to guess what value a parameter expects. `agentctl validate` now statically scans each declared tool (ast, no import) and reports parameters with a missing or blank description as warnings. Warnings never fail validation and are not part of SystemSpecValidator, so serve/chat startup is unaffected: every shipped example tool would otherwise fail today. Also correct docs/mcp-and-tools.mdx, which said the docstring becomes the tool description the LLM sees; the engine always sends the YAML description. First step of extra-org#114. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
|
Thanks for the contribution. This looks like a useful, focused first step toward #114. I reviewed commit One non-blocking follow-up in For example, a tool with a described However, its It would be good to exclude these internal parameters from the description check and add a regression test, especially before promoting these warnings to errors. This can be handled in a follow-up PR and does not block this one. |
The parameter-description check warned about parameters LangChain fills in at runtime and leaves out of the model-facing tool schema, e.g. `config: RunnableConfig` produced "parameter "config" is missing a description" although `tool_call_schema` exposes only the other params. Skip them in the static scan, mirroring langchain_core's filtering: `callbacks`/`run_manager` by name, a bare `RunnableConfig` (an optional one is still shown to the model), `ToolRuntime`, and `Annotated[...]` with an `Injected*` marker (InjectedToolArg, InjectedToolCallId, LangGraph InjectedState/InjectedStore). Matching is by name, since the scan never imports tool code. Follow-up to #156 review; step toward #114. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
First, non-breaking step toward #114.
Changes
agent_engine/core/tool_metadata.py(new): statically scans each declared tool inplugins/tools/<id>.py(parsed withast, never imported, same approach asplugin_stubs) and reports parameters whose description is missing or blank.*args/**kwargsare skipped; missing or unparseable files are left to the existing checks.agentctl validate:ValidationResultgainswarnings. They are printed but never fail validation:SystemSpecValidatoris unchanged, soserve/chatstartup is unaffected.plugin_stubs.find_function: made public for reuse (rename only).Annotatedexample inmcp-and-tools.mdx/MCP_AND_TOOLS.md, a note incli.mdx, and a correction:mcp-and-tools.mdxsaid the docstring becomes the tool description the LLM sees, but the engine always sends the YAMLdescription.