archive: do not leak URL credentials in user visible messages - #706
Open
mahaase wants to merge 3 commits into
Open
archive: do not leak URL credentials in user visible messages#706mahaase wants to merge 3 commits into
mahaase wants to merge 3 commits into
Conversation
The HTTP basic authentication credentials are part of the archive URL. All
three places that turn that URL back into a string for display used the raw
netloc, which still carries the "user:password@" part:
* getArchiveName() feeds _namedErrorString(), so the credentials were
printed on *every* error message -- including the perfectly ordinary
"artifact not found" that occurs for each package on a cache miss. No
verbosity flag needed.
* _remoteName() is the "details" of the DOWNLOAD/UPLOAD/MAP-SRC/CACHE-BID/
CACHE-FPR/MAP-FPRNT status lines, shown with -v.
* getArchiveUri() is printed by "bob archive".
These messages routinely end up in build logs and CI consoles.
WebDav._getURL() already stripped the credentials before putting the URL on
the wire, so this only ever affected the display strings. Factor that logic
out into stripUserInfo() and use it in the three spots above as well.
While at it, cut at the last '@' instead of the first. urlparse() delimits
the user info at the last '@', so for a password containing an unencoded '@'
the previous split('@')[1] produced a bogus host -- and revealed part of the
password on top.
The optional 'name' archive setting was no workaround: _remoteName() and
getArchiveUri() do not consult it.
The http backend takes the basic authentication credentials as part of the URL. State that they are not echoed back in the status lines, so it is clear that putting them there does not put them into every build log.
testNoCredentialsInMessages checks the three user visible strings derived from the archive URL: the password and the '@' delimiter must be gone while the host must survive. Reverting archive.py alone makes all three subtests fail. TestStripUserInfo covers the helper itself: URL without credentials (returned unchanged), plain removal, a password containing an unencoded '@', an IPv6 literal host, and preservation of the urlsplit() result type.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #706 +/- ##
==========================================
+ Coverage 89.23% 89.25% +0.01%
==========================================
Files 50 50
Lines 16450 16459 +9
==========================================
+ Hits 14679 14690 +11
+ Misses 1771 1769 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The HTTP basic authentication credentials of the
httparchive backend arepart of the URL, as documented. All three places that turn that URL back into
a string for display use the raw
netloc, which still carries theuser:password@part:getArchiveName()(archive.py:877)_namedErrorString()(archive.py:401)_remoteName()(archive.py:902)detailsof theDOWNLOAD/UPLOAD/MAP-SRC/CACHE-BID/CACHE-FPR/MAP-FPRNTstatus lines-vgetArchiveUri()(archive.py:966)bob archive(cmds/archive.py:32)The first one is the bad one.
_namedErrorString()is used for every errormessage, including the perfectly ordinary "artifact not found" that happens for
each package on a cache miss. So a plain
bob buildon a cold cache prints thepassword once per package:
These messages routinely end up in build logs, CI consoles and pasted-into-an-issue
snippets.
The optional
namearchive setting is not a workaround:_remoteName()andgetArchiveUri()never consult it.Note this is a display-only issue.
WebDav._getURL()already strips thecredentials before the URL goes on the wire; they are sent in the
Authorizationheader (webdav.py:47-55).Fix
Factor the stripping that
WebDav._getURL()already does into astripUserInfo()helper and use it in the three places above as well.It takes and returns a parsed URL and preserves the result type, so both the
urlsplit()result used inwebdav.pyand theurlparse()result used inarchive.pywork.Drive-by: cut at the last
@, not the firstThe existing implementation used
netloc.split('@')[1].urlparse()delimitsthe user info at the last
@, so the two disagree when the passwordcontains an unencoded
@:Percent encoding is documented as required, so this was latent rather than a
live bug, but there is no reason to get it wrong.
Tests
TestHttpBasicAuthArchive.testNoCredentialsInMessages— asserts all threemethods drop the password and the
@, and still contain the host.TestStripUserInfo— no credentials (identity), removal,@in thepassword, IPv6 literal,
urlsplit()type preservation.Reverting
archive.pyalone makes the first test fail on all three subtests;test_archive+test_webdavare green with the fix (71 tests).Docs
One sentence in the basic auth section of
configuration.rststating that thecredentials are stripped from displayed URLs.