Repository navigation
Refuse negative retries and delays; daily canary tests the browser path - #2
Merged
Merged
Conversation
- --retries, --proxy-block-retries, --delay and --retry-delay (all three engines) and --retries/--retry-delay (Scraper API client) reject negative values with an argparse error naming the flag. --retries 0 stays legal: it counts extra attempts. - Comments, docstrings and config explain each safeguard in place rather than pointing at an internal document; a suite check keeps it so. - .env.example no longer states a fixed credential lifetime for the Scraping Browser API. - The file-mode check runs under umask 022, 002 and 077. - Daily canary: one profile through a local browser; the Scraper API job fails naming TWOCAPTCHA_KEY instead of skipping green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jehrr
marked this pull request as ready for review
October 15, 2026 22:20
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
From the 2026-10-14 audit of
main@1c4e844:.env.examplerepeated a fixed credential lifetime for Scraping Browser endpoints. That figure has been retracted.--retries -1(and negative delays) were accepted without error in all three engines and inscraper_api_client.py.Behaviour after this PR
--retries,--proxy-block-retries,--delayand--retry-delay(engines) and--retries/--retry-delay(Scraper API client) reject negative values. The run exits 2 withargument --retries: must be 0 or more, got -1.--retries 0is still legal and still fetches once, because the flag counts extra attempts (max(1, retries + 1)).check_no_private_document_citations, fails if such a citation reappears. Its tokens are built from pieces, so the check does not match itself.check_outputs_get_the_umask_mode_not_0600now runs under umask 022, 002 and 077, and also checks that an existing 0640 file keeps its mode.@nasa) through local Chromium (--transport browser). It asserts exit 0,transport == "browser", one row, and a subscriber floor.::error::namingTWOCAPTCHA_KEYwhen that secret is absent. When the secret is present, it asserts exit 0 and non-empty Spotlight rows withview_count.[Unreleased]describe all of this. The behaviour change leads, in a blockquote.Evidence
697 checks passed, 0 failed, 14 group(s) skipped(it was 667 before).<engine>_scraper.py --url nasa --retries -1printederror: argument --retries: must be 0 or more, got -1.ci_checks.py --all: sample check ok; secret check ok (39 files scanned).PYTHONDONTWRITEBYTECODE=1, with__pycache__removed, and restored afterwards:page_flow.pyturned FAILno shipped file cites a private document.0o666 & ~umaskto0o644 & ~umaskturned FAILa new .json is 0o664 under umask 0o2.--retriesback totype=intturned FAILselenium_scraper --retries is validated by non_negative_int.scraper_api_client --retriesback totype=intturned FAIL the wiring check andscraper_api_client refuses --retries -1.n < -5turned FAILnon_negative_int rejects '-1'.Not changed
.env.example'sSNAPCHAT_URL=nasa: this repo has no builder flags for it to override, and an explicit--urlwins.gotocomment wording is cosmetic and was left alone.Canary
Until a
TWOCAPTCHA_KEYrepository secret is added, thescraper-apijob is expected to be red. That is the intended result. The ungated job, now including the browser step, is expected to stay green.🤖 Generated with Claude Code