Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/retrievers/userguide/prompt.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
- Use a clear display name (page title or section title).
- If multiple excerpts support the same fact, cite them together (space-separated).
3. How-to focus: Give clear, actionable steps when the user asks how to perform a task. Name UI elements accurately (buttons, panels, tabs) as they appear in the context.
- When several Reactome tools can do the same job, **lead with the one that needs the least setup** — a web tool on reactome.org before a desktop application, a plugin, or an R package. Mention the others afterwards as alternatives.
- Concretely: gene set analysis is **ReactomeGSA** (web, at reactome.org/gsa, nothing to install). ReactomeFIViz also performs GSEA, but it is a Cytoscape plugin and requires installing Cytoscape first, so it is the alternative rather than the answer.
- Do not pick a tool because its page happens to contain the most step-by-step text. The most detailed instructions are often for the most involved tool, which is rarely what someone asking "can you run this for me" wants.
4. Tone and style:
- Write in a clear, friendly, and conversational tone.
- Use accessible language; avoid unnecessary jargon.
Expand Down
22 changes: 22 additions & 0 deletions tests/agent/test_safety_scope.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,25 @@ def test_capability_is_answered_downstream_not_refused_here() -> None:
"""Whether the assistant can run an analysis is a different question from
whether it is allowed to be asked."""
assert "decided later" in safety_check_message


def test_the_user_guide_prefers_the_tool_that_needs_no_install() -> None:
"""Asked "can you run gsea for me", beta answered with ReactomeFIViz --
the Cytoscape plugin -- rather than ReactomeGSA, the web tool.

Not a retrieval failure: the top four passages were the ReactomeGSA page.
The FIViz page arrived fifth (the rephrase expands "gsea" to "Gene Set
Enrichment Analysis", and that page has a "Perform GSEA Analysis" menu
item), and the model led with it because the prompt asks for actionable
steps and FIViz has the most step-by-step text.

The most detailed instructions are usually for the most involved tool,
which is rarely what someone asking "can you run this for me" wants.
"""
from retrievers.userguide.prompt import userguide_system_prompt

assert "least setup" in userguide_system_prompt
assert "ReactomeGSA" in userguide_system_prompt
assert "Cytoscape" in userguide_system_prompt
# The trap that caused it, named so a later edit does not reintroduce it.
assert "most step-by-step text" in userguide_system_prompt
Loading