Skip to content

feat(fetch-content): record a profiler marker for each fetched file - #1083

Open
jcristau wants to merge 1 commit into
taskcluster:mainfrom
jcristau:fetch-content-markers
Open

jcristau wants to merge 1 commit into
taskcluster:mainfrom
jcristau:fetch-content-markers

Conversation

@jcristau

Copy link
Copy Markdown
Contributor

When MOZ_RESOURCE_MONITOR_FORWARD is set, append a marker covering the download and extraction of each fetch to it, using the forwarding protocol of mozsystemmonitor, so that a resource monitor started by run-task's startup hook shows them in its profile.

When MOZ_RESOURCE_MONITOR_FORWARD is set, append a marker covering the
download and extraction of each fetch to it, using the forwarding
protocol of mozsystemmonitor, so that a resource monitor started by
run-task's startup hook shows them in its profile.

Fetches run in parallel threads, and other processes may forward to the
same file, so open it the way mozsystemmonitor does: on Windows,
O_APPEND isn't atomic, so use a handle that may only append instead.
@jcristau
jcristau force-pushed the fetch-content-markers branch from ead6459 to 90850af Compare November 4, 2026 15:29
@jcristau
jcristau marked this pull request as ready for review November 4, 2026 15:29
@jcristau
jcristau requested a review from a team as a code owner November 4, 2026 15:29
@jcristau
jcristau requested a review from bhearsum November 4, 2026 15:29

@hneiva hneiva left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just one question, not really a block.

"""Open path for writes that are each appended atomically, even when
several processes append to it concurrently."""
if sys.platform != "win32":
return os.open(path, os.O_WRONLY | os.O_APPEND)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aka: brace yourselves... here comes some wonky windows stuff

if bool(gpg_sig_url) != bool(gpg_env_key):
print("--gpg-sig-url and --gpg-key-env must both be defined")
return 1
with fetch_marker(args.url):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be useful to break down this into multiple markers?
It's downloading the gpg_sig_url, then the url, then repack. Would we ever want more granularity for this marker?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants