Repository navigation
fix(workloadapi): keep X.509 context watch alive after rejected updates - #487
Merged
maxlambrecht merged 1 commit intoOct 29, 2026
Merged
Conversation
DefaultX509Source threw a WatcherException from the X.509 context watcher's onError. StreamObservers calls onError from inside StreamObserver.onNext when GrpcConversionUtils.toX509Context rejects an update (for example a corrupted certificate or key, or a SPIFFE ID mismatch), so the exception escaped onNext and gRPC cancelled the call. CANCELLED is non-retryable, so the watch ended permanently: the source kept serving its current SVID, never applied a later rotation, and TLS built on it started failing once that SVID expired. Apply the same handling the JWT sources got: only errors before the first update are initialization failures. The watcher records the error and init throws it, so newSource also fails when the first update is rejected asynchronously instead of returning an uninitialized source. After initialization, errors are logged and the current SVID and bundles are kept, so the stream stays open and the next valid update replaces them. Fixes spiffe#486 Signed-off-by: NopeCharon <115794254+NopeCharon@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
DefaultX509Source's X.509 context watcher no longer throws fromonError. This is the same change #482 made forDefaultJwtSourceandCachedJwtSource:initthrows it, sonewSourcefails withX509SourceException(causeError in X509Context watcher, as before).Fixes #486
Why
StreamObserverscallswatcher.onErrorfrom insideStreamObserver.onNextwhenGrpcConversionUtils.toX509Contextrejects an update. TheWatcherExceptionthrown there escapedonNext, gRPC cancelled the call withCANCELLED: Failed to read message., and since that code is non-retryable the watch ended permanently. The source kept serving its current SVID and never applied a later rotation, so TLS built on it failed once that SVID expired.Before this change, a first update that failed asynchronously also counted down the init latch without throwing, so
newSourcereturned a source with no SVID. Calls on that source then failed withX.509 SVID source not initialized. NownewSourcethrows.How tested
DefaultX509SourceTest.testX509ContextWatcherError_afterInit_keepsX509ContextAndAppliesLaterUpdates: afteronError, the source keeps its SVID and bundle, and the next update is applied.DefaultX509SourceTest.testWatchX509Context_corruptedUpdate_laterUpdatesAreApplied: an in-process fake Workload API streams a valid response, a corrupted one, then another valid one. The source ends up with the last SVID (hintrotated). Without the fix it stays oninitial.newSource_errorFetchingX509Contextstill passes unchanged../gradlew buildon JDK 17.