fix: scout against every record root, not just kb/communities - #736
Merged
Conversation
Checked the `scout-communities` skill against the tree before running it. The
documented CLI matches the script exactly -- presets, flags, defaults -- but the
dedup index did not match the corpus.
`scout_communities.py` reports a paper as NEW when no curated record cites it,
and it built that index from `kb/communities/` alone. `data/isolates/` holds
records with the same root class, and **5 references are cited only from there**:
PMID:34111782, PMID:38150661,
doi:10.1007/s10163-014-0276-4, doi:10.1016/j.mineng.2013.03.022,
doi:10.1016/j.scitotenv.2021.148151
Each would have been reported as a new community to go and research, when it is
already curated. The consequence is wasted work rather than a wrong record,
which is why it sat unnoticed: nothing fails, a curator just researches
something that exists.
Same shape this repository keeps hitting -- `data/isolates` outside every
validation glob (#350), `kb/taxa` outside every CI trigger (#471), a hardcoded
root list that cannot notice a new member (#689). Same fix: read
`default_record_roots()` rather than naming a directory, so a root added later
is covered without anyone remembering this file. The index goes from 325 records
to 329.
`scripts/` is run by path rather than imported, so `src/` is put on `sys.path`
for the shared helper.
## The guard corrected the test I wrote for it
The first version globbed `kb/communities` to compute which references were
unique to the other roots, and #689's guard flagged it -- correctly, and with
the right advice: prefer fixing it to recording it. Adding the module to
`_COMMUNITY_ONLY` would have been recording a falsehood, since it is not
community-only.
Asking the question PER ROOT instead names no directory and is stronger: for
each root, every reference unique to it must be in the index. That holds for a
root added later with this file untouched, and it refuses to run vacuously if no
root has a unique reference.
Mutation-checked by reverting the scout to the single root, which reds both the
count and the per-root check.
Skill doc updated to match, and two limitations recorded that were not written
down: a false NEW costs research rather than correctness, and `--since` is a
floor rather than a window -- left at its 2024 default a pass in 2026
re-surfaces everything earlier passes already saw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checked the
scout-communitiesskill before running it. The documented CLI matches the script exactly — presets, flags, defaults — but the dedup index did not match the corpus.The gap
scout_communities.pyreports a paper asNEWwhen no curated record cites it, and it built that index fromkb/communities/alone.data/isolates/holds records with the same root class, and 5 references are cited only from there:Each would have been reported as a new community to go and research — when it is already curated.
The consequence is wasted work, not a wrong record, which is exactly why it sat unnoticed. Nothing fails. A curator simply spends a deep-research pass on a community that already exists.
Same shape this repo keeps hitting:
data/isolatesoutside every validation glob (#350),kb/taxaoutside every CI trigger (#471), a hardcoded root list that cannot notice a new member (#689). Same fix — readdefault_record_roots()instead of naming a directory. The index goes from 325 records to 329.The guard corrected the test I wrote for it
My first test globbed
kb/communitiesto compute which references were unique to the other roots, and #689's guard flagged it — correctly, and with the right advice: "Prefer fixing it to recording it." Adding the module to_COMMUNITY_ONLYwould have recorded a falsehood; it is not community-only.Asking the question per root names no directory and is stronger: for each root, every reference unique to it must be in the index. That holds for a root added later with this file untouched, and it refuses to run vacuously if no root has a unique reference.
kb/communitiesonlySkill doc
Updated to match,
version: 1.1.0, plus two limitations that were not written down:NEWcosts research, not correctness — the reason this was invisible;--sinceis a floor, not a window. Left at its 2024 default, a pass in 2026 re-surfaces everything earlier passes underresearch/scouting/already saw.Known unrelated failure:
test_no_snippet_stops_mid_word, from an untracked local cache on a record this PR does not touch.