Skip to content

fix: handle a tracing provider without otlp_http on non-leader units - #2918

Open
Indian-boult wants to merge 3 commits into
canonical:mainfrom
Indian-boult:fix/tracing-follower-no-otlp-http
Open

Indian-boult wants to merge 3 commits into
canonical:mainfrom
Indian-boult:fix/tracing-follower-no-otlp-http

Conversation

@Indian-boult

@Indian-boult Indian-boult commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

When the tracing provider publishes receivers but none for otlp_http, TracingEndpointRequirer.get_endpoint() checks which protocols were requested by reading the charm's own app databag. Only the leader can read it, so on other units the hook failed with RelationDataAccessError.

Tracing now reads the receivers with get_all_endpoints() and picks the otlp_http one itself, so the destination stays unset until the provider offers it, as it already did on the leader. Until then it logs at debug level that the provider doesn't offer otlp_http, as get_endpoint() did. ProtocolNotRequestedError was only raised by get_endpoint(), so it's no longer caught. The vendored library is unchanged.

The conformance tests for a provided endpoint and for an unsupported protocol now run as a non-leader as well as the leader.

When the tracing provider publishes receivers but none for `otlp_http`,
`TracingEndpointRequirer.get_endpoint()` checks which protocols were
requested by reading the charm's own application databag. Only the
leader can read it, so on other units the hook failed with
`RelationDataAccessError`, for example on start or on
`charm-tracing-relation-changed`.

Read the provider's receivers with `get_all_endpoints()` and pick the
`otlp_http` one directly. The destination stays unset until the
provider offers it, as it already did on the leader.

`ProtocolNotRequestedError` was only raised by `get_endpoint()`, so it
is no longer caught.

@tonyandrewmeyer tonyandrewmeyer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, thanks! I checked that the follower case does fail on main without the fix (RelationDataAccessError reading our own app databag).

Comment thread tracing/ops_tracing/_api.py Resolved
get_endpoint() logged this case, and it is the only clue as to why no
traces are sent. Also run the provided-endpoint conformance test as a
non-leader, since followers now look up the receiver too.
@tonyandrewmeyer

Copy link
Copy Markdown
Collaborator

@james-garner-canonical, please review the PR.

Please do not ping people like this.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants