Repository navigation
fix: describe custom actions in HighLevelActionSet.to_tool_description - #422
Open
VitorCoelho wants to merge 1 commit into
Open
VitorCoelho wants to merge 1 commit into
VitorCoelho wants to merge 1 commit into
Conversation
to_tool_description() looked up each action function with globals()[tool_name], i.e. in the highlevel module namespace, which only contains the built-in actions. As a result: - an action set with custom_actions raised KeyError for the first custom action, so tool-use agents could not use custom actions at all; - a custom action reusing a built-in name (e.g. its own click()) was described with the built-in signature instead of its own. Keep the functions registered in the action set and read the signature from them. Add a test covering both cases.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
HighLevelActionSet.to_tool_description()now reads each action's signature from the function registered in the action set, instead ofglobals()[tool_name]inhighlevel.py.Why
globals()of thehighlevelmodule only contains the built-in actions, so:custom_actionsraisedKeyErroron the first custom action, so tool-use agents could not describe custom actions at all;click(bid, note)) was silently described with the built-inclick(bid, button, modifiers)signature.The functions are kept in a private
_action_functionsdict filled in__init__, next toaction_set.to_python_code()and the rest of the public API are unchanged.How it was tested
test_tool_description_custom_actionsintests/core/test_actions_highlevel.py(no browser needed): checks the schema of a custom action and of a custom action namedclick, for both theopenaiandanthropicformats. It fails onmainwithKeyErrorand passes with the fix.pytest -n 5 -m 'not pricy' tests/core(headless Chromium, Playwright 1.56 instead of the pinned 1.44): everything passes excepttest_task.py::test_mock_image_goal_task(needs internet access, which my environment does not have) andtest_observation.py::test_axtree_to_text; both fail the same way onmainhere, so they are unrelated to this change.black24.2.0 (repo config) is clean.